Approximating Instance-Dependent Noise via Instance-Confidence Embedding
Authors: Yivan Zhang
Masashi Sugiyama
Venue: ICLR 2022 Submitted (2022)
Keywords: instance-dependent noise
label noise
classification
robustness
weakly supervised learning

Abstract: Label noise in multiclass classification is a major obstacle to the deployment of learning systems. However, unlike the widely used class-conditional noise (CCN) assumption that the noisy label is independent of the input feature given the true label, label noise in real-world datasets can be aleatory and heavily dependent on individual instances. In this work, we investigate the instance-dependent noise (IDN) model and propose an efficient approximation of IDN to capture the instance-specific label corruption. Concretely, noting the fact that most columns of the IDN transition matrix have only limited influence on the class-posterior estimation, we propose a variational approximation that uses a single-scalar confidence parameter. To cope with the situation where the mapping from the instance to its confidence value could vary significantly for two adjacent instances, we suggest using instance embedding that assigns a trainable parameter to each instance. The resulting instance-confidence embedding (ICE) method not only performs well under label noise but also can effectively detect ambiguous or mislabeled instances. We validate its utility on various image and text classification tasks.

Under review as a conference paper at ICLR 2022
APPROXIMATING INSTANCE-DEPENDENY NOISE VIA INSTANCE- INSTANCE-CONFIDENCE EMBEDDING
Anonymous authors Paper under double-blind review
ABSTRACT
Label noise in multiclass classification is major obstacle to the deployment of learning systems. However, unlike the widely used class-conditional noise (CCN) assumption that the noisy label independent of the input feature given the true label, label noise in real-world datasets can be aleatory and heavily dependent on individual instances. In this work, we investigate the mnstance-dependee (IDN) model and propose an efficient approximation of IDN to capture the instance- specific label corruption. Concretely, noting the fact that most columns of the IDN transition matrix have only limited influence on the class-posterior estimation, we propose variational approximation that uses single-scalar confidence parameter. To cope with the situation where the mapping from the instance to its confidence value could vary significantly for two adjacent instances, we suggest using instance embedding that assigns trainable parameter each instance. The resulting instance-cer confice-e embedding (ICE) method not only performs well under label noise but also can effectively detect ambiguous mislabeled instances. We validate its utility on various image and text classification tasks.
noise
INTRODUCTION
In modern machine learning, large-scale data has become indispensable (Russakovsky al., 2015; Wang et al., 2019a). A prevalent approach to collecting large-scale labeled datasets is to use imperfect sources such as crowdsoureing and web crawling (Fergus al., 2005; Schroff al., 2010; Wang et al., 2019a), which is usually less expensive and time-consuming than manual annotation by domain experts. However, such methods inevitably introduce label noise that may lead to overfitting and hurt the generalization of deep models (Arpit al., 2017; Zhang al., 2017).
In such situations, it is often beneficial to (i) remove mislabeled data or abstain from using confusing instances (Hara al., 2019; Thulasidasan al., 2019); (ii) increase robustness and reduce harmful influences of noisy labels (Malach & Shalev-Shwarde 2017; Mirzasoleiman et al., 2020; Liu et al., 2020); or (iii) explicitly model the transition from the unobservable true label to the noisy observation (Goldberger & Ben-Reuven, 2017; Patrini et al., 2017; Xia et al., 2020). In this work, we focus on explicit modeling of the label corruption process, which is model-agnostic and data-efficient.
Most existing studies in this direction employ the dlass-conditional noise (CCN) assumption, i.e., the noisy label is independent of the input feature given the true label (Angluin & Laird, 1988; Natarajan et al., 2013; Patrini et al., 2017). However, this assumption could be too strong to fit some real- weal-world data well (Xiao et al., 2015; Chen et al., 2021; Liu, 2021). More importantly, CCN only captures the general label flipping patterns between classes for all instances. In applications such as data cleansing and human- -n---ii---ttooogt interaction, listance-specife noise information itself could be of central interest. This urges us to consider not only the class-conditional noise pattern but also the speciic noise modeling.
To handle this problem, in this work, we study the instance-depender noise (IDN) model, where the noisy label also depends on the input. Several methods have been reported in the literature, but they either only focus on binary classification under strong assumptions (Menon et al., 2018; Cheng et al., 2020), are based on Nomain--peciffi knowledge (Xia et al., 2020), or need extra supervision (Berthon et al., 2021). In contrast, we propose simple domain- agnostic approximation method for the multiclass IDN model, referred to as instance-come contince- embedding (ICE). Concretely, to avoid estimating the noise transition matrix for each instance, we propose variational approximation that

---

Under review as a conference paper at ICLR 2022
Y
Y
Y
(a) CCN
(b)
(c) ICE
Figure I: Graphical representations of noise models, including the class-conditional noise (CCN) model, the instance-dependent noise (IDN) model, and the proposed instance-confidet embedding (ICE) approximation of IDN. Here, X is the input feature, the true label, Y is the noisy label, and C [0, is scalar confidence parameter.
uses a scalar confidence parameter (Section 3.2). Then, we suggest using instance embedding that assigns a trainable parameter to each instance because the mapping from the instance to its confidence value could be non-smooth and is usually not required to generalize to unseen examples (Section 3.3). Lastly, show the effectiveness of the proposed method and its ability to detect ambiguous mislabeled instances through experiments on various image and text classification tasks (Section 5).
PROBLEM: INSTANCE-DEEPEDEEN NOISE
this section, we give brief overview of learning with instance-dependem noise (IDN).
2.1 NOTATION
Consider K-class classification problem, where X X is the input feature and Y {l,.... ......,,y K} is the unobservable categorical true label. We assume that the clean class-poss class-posterior p(Y comes from parametric family of distributions:
(Y == Categorical(Y p = f(X:
where p E A is the probability parameter for Y in the (K' dimeessional probability simplex A K-1 and f A K-1 a differentiable function parameterized by é that maps the feature X to its corresponding probability parameter p. Then, let Y E {1...., K} be the noisy label. The goal is to predict Y from X based on finite i.i.d. sample of (X, (Xpirss
2.2 DEPENDENCE
Next, we introduce the dependence structure between X, Y, and Y, which characterize different noise models. The graphical representations of noise models are illustrated in Fig. 1.
In IDN, we assume that the joint distribution of X, Y, and y can be factorized as follows:
p(X,V.V (V, x))x (X)))(X)...
That is, the noisy label Y depends on both the instance X and the true label Y. Then, the noisy class-posterior p (X) can be obtained by marginalizing p(Y, Y (I) over
P XX) == Categorical(? -K X))xx))))))))
where q AK- denotes the probability parameter for Y.
Note that (V X) plays central role in IDN. Since both Y and Y are categorical random variables, for a certain instance x, p(Y|Y, = x) can seen as a x stochastic matrix T(x), whose elements are (x) == p(v = j|Y = i, X = x) for i,j E {1,..., K). Conventionally, T(x) is called noise transition matrix (Patrini et al., 2017). Then, p(Ý|Y,X) can be regarded matrix- matrix-valued function T X - [o, 1] that maps each instance x to its corresponding IDN transition matrix (zx). Without any restriction, need K x K parameters for each instance
2

---

Under review as a conference paper at ICLR 2022
2.3 APPROACH
Owing to its complexity, IDN has only been studied to limited extent but is of great interest recently, straightforward method is to jointly estimate the matrix-valued function T(x) as well as the clean class- posstroottrior P&(H ((Y|)) using neural networks (Goldberger & Ben- Ren-Renven, 2017). However, the estimation error of T(x) could be high, which deteriorates the classification performance. Another direction is to restrict the problem under certain conditions, so that can provide theoretical guarantees (Menon et al., 2018; Cheng et al., 2020). However, existing work mainly focused on binary classification.
A promising approach is to approximate IDN using a simpler dependence structure, such as a mixture of noises with different semantic meanings (Xiao al., 2015) weighted combination of noises that depend on parts of the instance (Xia et al., 2020). In this work, we also suggest that it might
be unnecessary to obtain K x K matrix for each instance Note that pa Pa ([]²) can be seen linear combination of columns of (z) weighted by Pp [=);; ; the maximum value of pa P ({x)) is close to 1, i.e., the label of the instance is almost deterministic, the estimation of K - 1 columns of T()) has only limited influence on the estimated noisy class- pass-posterior p(|(.)). This suggests the ) in real-world applications. In
possibility of using relatively simple model approximate pY||)) this work, we consider a single-purameter approximation for each instance, which is introduced in Section 3.2 and illustrated in Fig. 2.
Another issue is that existing methods still introduce some level of smoothness w.r.t. æ into T(x) (Goldberger & Ben- nen-Reuven, 2017: Xiao al.. 2015; Xia al., 2020). real-world problems, however, we can only access a finite sample of (X, Y)ppiirs that are possibly annotated by non- experts or web crawlers (Fergus et al., 2005). Thus, the label noise could be aleatory and T(x) could vary significantly for two adjacent instances. Also, the classifier P [X) is desired but the generalization of T(x) to unseen examples is usually dispensable. This inspires instance embedding instead of neural network approximation, which is discussed Section 3.3 and demonstrated in Fig. 3.
use
3 PROPOSED METHOD
In this section, we present our proposed method, instonce-confidence embedding (ICE).
3.1 VARIATIONAL LOWER BOUND
Note that Txx) serves as linear mapping from p to q (Eq. (3)). Due the difficulty of estimating the matrix-valued function T(x), we use a simpler function ge.s (Y parameterized by as
variational approximation to (Y The choice of the approximation family is discussed in Section 3.2.
Then, let us consider the expected likelikel the learning objective, which can be rewritten as
E {x
(logg(((10 (KI(ps(())) ga.a (Ý|X)) L(0, X),
(4)
where DKL denotes the Kullback-Leible (KL) divergence, and the second term is
((0,;;X)
E {~p(F]x)
[log qa (VI)))
(5)
Since the KL-divergence is always non-negative, this term gives variational lower bound of the expected log- likeliodd...d. Then, we have the following learning objective to maximize:
1(0.6) == I
[C0,0) X)] X~p(XX)
E
[og qo (7Ix).
x.P~p(X,))
In practice, the expectation can be approximated using the empirical distribution based on a finite i..dd. sample of (X, Y paiis..
3

---

Under review as a conference paper at ICLR 2022
Clean
Linear interpolation
Power transfomation
An illustration of the transformation leftmost) to the noisy class- pass-posterior (Y(((z) (the rightmost). The outer black triangle depicts the probability simplex A projected to the 2-dimensional space for 3-class classification. We can see that when the label is almost deterministic ( is close a vertex), the estimation of K 1 columns of the transition matrix T Txx) (the two deviated vertices of the dotted triangle) has only limited influence on the estimated noisy class- paass-osttrioor (({ x))
Figure 2:
from the clean class cluss-postrrior po (Y (z) (the
(A is still close to This inspires to go step further and use single-parameter approximations
V) q @ (x) (Eqs. (8) and (9)).
3.2 VARIATIONAL APPROXIMATION
Next, we discuss the choice of the variational approximation family of qo.o (7IX).
To approximate the effect of multiplying
IDN transition matrix (xx) that requires K K parameters for each instance x, in this work, we use a simpler transformation from to q. which not necessarily linear. Compared with estimating full matrix for each instance without any restriction (Goldberger& & Ben-Reuven, 2017), obtaining only an approximation may cause higher approximation error, but on the other hand, it may reduce estimation error and thus improve the classification performance. The high estimation eтror of complex models might be more harmful, which is empirically validated in Section It is also case when using CCN as an approximation of IDN to balance this trade-off. The difference is that CCN obtains complete transition matrix common to all instances, but ICE aims to obtain an approximated trend for each instance, which gives useful instance-specife noise information.
Then, we suggest to use a single-scalar parameter E [0 for each instance to control this approximation, which is useful for sorting and comparing training examples. This design might also be useful in data cleansing, learning with rejection, or active learning (Hara et al., 2019; Thulasidasan et al., 2019; Charoenphakdee al., 2021). This parameter is referred to the confidence and is obtained via a function : X + [0, 1] parameterized by 0, i.e., C = g(X; 0). Then, 0 can be regarded the collection of C for all instances. The confidence C' plays central role in method, where C = 0 means that the instance is ambiguous or mislabeled and thus the classifier should not give confident prediction.
Finally, we need to design a transformation from p to parameterized by the confidence C. We denote this function by h A K-1 K-1, In summary, 40.0 (YIx) takes the following form:
4 X)= = Categorical( hit==h(f(xx h(f((:)):
Next, we analyze what characteristics h needs have. First, we suggest that h needs necessarily to be a linear transformation because the transformation is instance- and any function that maps p q as close as possible for a certain instance would suffice. Second, we require that arg max(p) = arg max(q), i.e., h(p: C') should be argmmux-presrrvnng function so that the top-1 index the probability vector does not change. This is because h should only affect the confidence of the prediction, not the final decision. Otherwise, if h is too flexible and is able to map confident prediction to different confident prediction, then the output of ff could be arbitrary, and consequently, information of the true label can be learned from the noisy label supervision.
Based on this motivation and the aforementioned semantics of C, we require that h(p; 1) =
and h (p; 0) u, where u A K-1 is the uniform probability vector (u, 1). Then, when the confidence C' is high, the classifier gives a prediction closer to the original confident prediction p: and when the confidence C is low, the classifier tends give random guess u.
4

---

Under review as a conference paper at ICLR 2022
Data
Croes erorpy
Weiht deca
Medited corfideace
Irstance embediigg
201
Colline
Figure 3: An example of the learned class-posteriors using (a) the usual cross entropy without any modification; (b) weight decay; (c) modified confidence of the prediction (Eq. (8) with neural network approximation for g X' - [0, 1]; and (d) instance-confidence embedding. The points with black edges are mislabeled instances. We can observe that noisy labels affect the decision boundary and the model complexity. Comparing the last two panels, modifying the confidence of the prediction works better with instance embedding than neural network approximation. ICE can reduce the influence of ambiguous or mislabeled instances to improve the class class-posterior posterior estimation.
Here, we propose two functions for h that satisfy the above conditions:
qi = Cpi+t + (1 C)u,,
(linear interpolation)
(8)
D
qi E
X P.
(power transformation)
(9)
for = 1,..., K. The visualization of these two transformations for =3 is given in Fig. 2.
3.3 INSTANCE EMBEDDING
The last piece of our method is the choice of X [0, 1]. the function that maps the instance to its confidence value C. is also possible to use neural network to approximate this function. However, because non-smooth w.r.t. its input x, 9 may not be well approximated by a neural network with similar complexity to the classifier f. which is illustrated an example in Fig. 3. Further, 9 may be rarely needed after training so its generalization ability is not required in many cases.
usually only have limited number of training examples and g could be
Based on these facts, we propose to use instance embedding, i.e., assign trainable parameter to each instance x. In other words, the only feature for an instance we use is its index in the training dataset. In this way, g expressive and flexible but cannot be used for predicting the confidence of unseen instances. Accordingly, for training dataset of size N, we need N parameters for one-dimensional instance embedding.
This seems to be high additional computational cost when the dataset size is large, but it is often acceptable, because (i) in modern deep learning, it is common to use wer-parameterized models (Nakkiran al., 2019), and the number of instances is usually not large as the number of parameters of the classifier o (e.g., CIFAR-10 (Krizhevsky, 2009): 5 x 10*, ResNet- 18 (He et al., 2016): 1 x 107), ); and (ii) the gradient of the instance embedding is sparse and only small subset of parameters needs to be updated at each iteration.
The idea of associating entity with scalar or vector embedding using simple lookup table with a fixed dictionary size has been widely used in natural language processing (Mikolov et al., 2013; Pennington et al., 2014; Peters et al., 2018; Devlin al., 2019) due the discrete nature of tokens, and can be seen recently in contrastive learning (Wu et al., 2018; He et al., 2020) for vision tasks. Instance embedding enables the function take any possible value on all observed instances but cannot generalize to any unseen token or image.
Combining the components introduced above, the learning objective Eq. (6) is in the following form:
q. transformation, Eqs. (8) (9)
L(O, Φ) cross-entropy (h(
s(x;0)
(X;0)
) Y).
p. neural network classifier. Ey, (1) C, mestance-cofideens embedding
We also provide code snippets in Appendix A for practitioners and gradient analysis in Appendix for theoretical implications.
5

---

Under review as a conference paper at ICLR 2022
4 RELATED WORK
In this section, we review related problem settings and methods.
consitonniiional noise (CCN), Compared with the IDN model, the instance-independeee and class-conditional noise (CCN) model has an additional assumption: p(Y| xX) = p(Y i...e the noisy label only depends on the true label Y. CCN has been well studied in both binary (Angluin & Laird, 1988; Long & Servedio, 2010; Natarajan et al., 2013; Van Rooyen et al., 2015; Liu & Tao, 2015) and multiclass (Patrini et al., 2017; Xia et al., 2019; Yao et al., 2020; Zhang et al., 202 Zhu et al., 2021) classification. Also, robust loss functions (Ghosh et al., 2017; Zhang & Sabuncu, 2018; Wang et al., 2019b; Charoenphakdee et al., 2019; Ma et al., 2020; Feng et al., 2020; Lyu & Tsang, 2020; Liu & Guo, 2020) have been mainly developed under the CCN setting. In practice, CCN methods can serve as practical approximations of IDN but the assumption could be too strong to fit some real-world data well (Xiao et al., 2015).
i.e.,
Label smoothing. Note that Eq. (8) is similar to the label smoothing (LS) technique (Szegedy et al., 2016; Pereyra et al., 2017: Lukasik et al., 2020), where the empirical distribution is linearly interpolated with a uniform distribution with fixed mixing parameter. It is also related to the soft/hard bootstrapping loss (Reed et al., 2015), where the observed label is mixed with the predicted probabilityppreiitee label. In contrast, in our method, is the prediction p that is "smoothed", not the label. We elucidate their relations and differences in Appendix C.
Temperature scaling. we use softmax as the final layer of the neural network for p the proposed method is closely related to the temperature scaling (TS) technique (Guo et al., 2017). Concretely, if p, x (xp(fi(X; Φ)) for = 1..... ......K, K, then Eq. (9) becomes
qi x (X:(()((X;))),
(10)
which shows that C is the reciprocal of the temperature. The difference is that the parameter C is instance-dependent in our formulation, rather than being fixed for all instances. Also, TS (Guo et al., 2017) and its extensions (Kull al., 2019; Rahimi et al., 2020) have been mainly used as post-hoc confidence calibration methods, while our method is used during training.
broader sense, the proposed method belongs to a category of methods that treat training examples differently in order to reduce the harmful effects of mislabeled instances. Besides the class- posterross that our method uses, these methods exploit the training dynamics, loss characteristics, gradient information, Examples include data cleansing (Liu et al., 2008; Northcutt et al., 2019; Hara et al., 2019) that first removes harmful instances and then (re-)trains the model on the remaining subset; dynamic training sample selection (Malach & Shalev-Shwartz, 2017; Jiang et al., 2018; Han et al., 2018; Wang et al., 2018; Yu et al., 2019; Mirzasoleiman et al., 2020; Wu et al., 2020; Chen et al., 2021; Cheng et al., 2021; Zhang et al., 2021a) that selects training examples dynamically during training; training techniques (Menon al., 2020; Liu al., 2020) that are designed increase robustness and avoid memorization of noisy labels; learning with rejection or selective classification YEniva & Wiener, 2010; Thulasidasan et al., 2019; Mozannar & Sontag, 2020; Charoenphakdee et al., 2021) that abstains from using confusing instances while improving the classification performance on accepted instances; and semi-supervised learning (Nguyen et al., 2020; Li et al., 2020) that exploits unlabeled data. We discuss the relationship with some of them in more detail in Appendix D.
Sample selection.
information of data itself from various perspectives.
In the same spirit, our proposed method also attempts to detect harmful instances and reduce their influences automatically so to improve the robustness of the class-posterior estimation. However, unlike explicit sample selection methods, the resulting algorithm is lightweight and has a low computational cost. Also, because the proposed method only affects the class class-pos,,rior, it is usually compatible with other training methods. Thus, the proposed method can be used alone or integrated into existing training pipeline further improve the performance.

---

Under review as a conference paper at ICLR 2022
Table 1: Accuracy (%) on the MNIST, FMNIST, KMNIST, SVHN, CIFAR-IO, and CIFAR-I00 datasets with instance-dependent noise. "Mean (standard deviation)" for 10 trials are reported. Outperforming methods are highlighted boldface using one-tailed t-tests with significance level
of 0.05.
MNIST
FMNIST
KMNIST
SVHN
CIFAR-10
CIFAR-I00
CCE
95.26(0.50)
85.41(0.56)
82.64(0.59)
76.93(1.86)
74.72(1.70)
51.90(0.38)
Bootstrapping
97.40(0.22)
87.22(0.40)
85.69(0.79)
79.60(1.93)
78.70(0.74)
52.60(0.50)
Adaptation
94.94(0.49)
81.91(0.12)
81.23
69.97(3.41)
74.71(0.81)
39.18(1.51)
Forward
95.47(0.45)
85.78(0.55)
83.71(0.86)
75.17(4.21)
75.08(0.87)
52.12(0.50)
Dual-T
97.05(0.40)
86.54(0.49)
84.91(0.86)
79.49(1.84)
81.10(0.47)
52.77(0.48)
DAC
95.78(0.29)
86.19(0.50)
83.21(1.05)
79.39(2.33)
74.95(0.85)
52.17(0.33)
GCE
97.69(0.18)
87.10(0.54)
86.50(1.04)
79.64(1.89)
(1.65(0.51)
55.75(0.30)
PLC
97. 97.87(0.17)
87.69(0.45)
87.30(0.49)
86 23(0.37)
55 55.91 82.51
ICE-LIN
97.39 (5.25)
87.07(0.29)
87.08(0.50)
17.16(2.19)
82.07(0.33)
56 36.04(0.34)
ICE-POW
97 (7.94
87 (0.30)
87. (7.73(0.62)
(1.82(0.99)
82 ( 29)
55.07 (5.07(0.48)
MNIS
FNNIST
CFAR
Figure 4: Ridgeline plots of the confidence C' during training. The red/blue curves represent the confidence of instances with flipped/original labels, respectively.
EXPERIMENTS
In this section, we experimentally verify if the proposed instance instance-confidence embedding (ICE) method is able to differentiate mislabeled instances from correct ones and consequently improve the classification performance. We also demonstrate that there already exist ambiguous or mislabeled training examples in the original datasets which can be detected by the proposed method.
5.1 IMAGE CLASSIFICATION
Datasets. We evaluated our method on six image classification datasets, namely MNIST (Le- Cun et al., 1998), -ashion-MNIST (FMNIST) (Xiao et al., 2017), Kuzushij-MNNIST (KMNIST) (Clanuwat et al., 2018), SVHN (Netzer et al., 2011), I), CIFAR-10, and CIFAR-100 (Krizhevsky, 2009) datasets. We used a method similar to the one used in Zhang et al. (2021a) generate instance- dependent label noises whose overall noise rate is around 52%. See Appendix E for details and more experimental results.
Methods. We compared the following ten methods: (1) (CCE) categorical cross-entropy loss; (2) (Bootstrapping) (hard) bootstrapping loss (Reed et al., 2015) that regularizes the output with the predicted label; (3) (Adaptation) noise adaptation layer (Goldberger & Ben-Reuvent 2017) that estimates a full K x transition matrix for each instance; (4) (Forward) forward correction (Patrini al., 2017) that estimates transition matrix for all instances; (5) (Dual-T) dual-T estimator (Yao et al., 2020) that uses the normalized confusion matrix to correct the transition matrix; (6) (DAC) deep abstaining classifier (Thulasidasan al., 2019) that uses abstention for robust learning; (7) (GCE) generalized cross-entropy loss (Zhang & Sabuncu, 2018) as a robust loss; (8) (PLC) progressive label correction (Zhang et al., 2021a); a)) (9) (ICE-LIN) interpolation (Eq. (8)): and (10) (ICE-POW) the one with the power transformation (Eq. (9)). For fair comparison, we implemented aforementioned methods using the same network architecture and hyperparameters.
caaaucc--onfidenee embedding with the linear
Models. For MNIST, FMNIST, and KMNIST, we used sequential convolutional neural network (CNN) and an Adam optimizer (Kingma & Ba, 2015). For SVHN, CIFAR-10 and CIFAR-100, used a residual network model ResNet-18 (He et al., 2016) and a stochastic gradient descent (SGD) optimizer with momentum (Sutskever et al., 2013). Hyperparameters are provided in Appendix E.
7

---

Under review as a conference paper at ICLR 2022
PM
3
1
4
1
7
ASS
FILL
2
1
9
6
9
9
a
4
1
2
3
4
(
MAM JMN
MM
a
4
S
4
1
7
(a) MNIST
(b) CIFAR-10
Figure 5: The 32 most low-confidence fraining examples in the MNIST and CIFAR- 10 datasets, ordered left-right, top-down by increasing confidence. The index in the dataset, original label, and predicted label are annotated above each image for verification.
3
3
3
3
3
3
3
5
4
4
4
4
4
4
4
(a) MNIST
(b) CIFAR-10
Figure 6: The class-wise spectra from high-confidence (left) to low-confidence (right) training examples in the MNIST (digit 3 and 4) and CIFAR-10 (cat and dog) datasets.
Improving classification performance. To verify if the proposed method is able to improve the classification performance under label noise, we constructed semi-synthetic noisy datasets so that the true labels are known. We regarded the original labels as clean labels, although as will be shown in the next experiment, label errors already exist in the original datasets to some extent. We ran 10 trials and reported the means and standard deviations of the test accuracy in Table
We can observe that the proposed method generally outperforms the baseline methods. It is worth noting that when there are a large number of classes (e.g., CIFAR-100), estimating a full transition matrix for each instance (Adaptation) may deteriorate drastically because it requires an additional K x K output and the estimation error could be high. On the contrary, the complexity of our single-paramett approximation does not increase the number of classes increases. Additionally, in the ridgeline plots in Fig. 4, we can observe the separation of instances with flipped/original labels using the learned confidence C, which may explain the performance improvement.
Detecting ambiguous/mislabeled instances. Next, we demonstrate that the proposed method can be used for detecting ambiguous or possibly mislabeled instances. We trained the model on the original datasets with the proposed method. A benefit of using a single-parameter approximation that naturally derives an order of the training examples. We sorted the training examples via the confidence and showed the 32 most low-confidence ones in Fig. 5. We also extracted high- /low-confidence training examples for each classes as shown Fig. 6. Results of other datasets are provided in Appendix E.
We can observe that, surprisingly, in these supposedly clean datasets, a number of instance might be mislabeled. In MNIST and SVHN, we found clearly mislabeled images. There are ambiguous images such as 2-7 and 4-9 pairs in MNIST and shir//Tshirr/pulloveerce photos in Fashion-MNIST. In CIFAR-10, it is interesting that images in the animal category are more likely have a low confidence. We conjecture that the spurious correlation between the object and the background color plays important role. We also found multi-modality issues, e.g., kiwi, owl, and chicken are all labeled as bird but are not visually prototypical birds. This phenomenon suggests the possibility of using the proposed method for diagnosing label issues in large-scale datasets.
8

---

Under review as a conference paper at ICLR 2022
Table 2: Performance (%) on the GLUE benchmark for natural language understanding. We reported
Matthews correlation coefficient on CoLA, F1 score/accuracy on MRPC and QQP, and accuracy otherwise. (NLI-(m/mm) denotes MultiNLI matchedd/mismatched,, respectively.
CoLA
SST2
MRPC
QQP
MNLI-(m/mm)
QNLI
RTE
WNLI
CCE
54.76
92.55
88.04/82.35 89.54/85.05
87.80/90.96
83.83/84.31
90.77
66.43
50.70
ICE
57.83
92.20
87.85/90.92
83.81/84.36
91.14
63.90
56.34
Table 3: Six selected low-confidence training examples in the CoLA dataset.
Index
Label
Guess
Text
390
acceptable
unacceptable
He often sees Mary.
7756
acceptable
unacceptable
That monkey is ate the banana.
8332
acceptable
unacceptable
I wanted Jimmy for to come with me. Paula hit the sticks. acceptable
2801
unacceptable
acceptable
2479
unacceptable
Kelly buttered the bread with butter. Henry wanted possibly marry Fanny.
6795
unacceptable
acceptable
Further, in the class-wise spectra of training examples (Fig. 6), we can observe that the confidence values may capture the level of ambiguity of instances. This suggests the possibility of using the proposed method in other tasks such as data cleansing, learning with rejection, and active learning.
5.2 TEXT CLASSIFICATION
We discovered that noisy label issues also exist in text datasets. We conducted similar experiments on the GLUE benchmark (Wang et al., 2019a), which is collection of datasets for natural language understanding. We trained a BERT-base model pretrained using a masked language modeling (MLM) objective (Devlin et al., 2019) with default Adam W optimizer (Loshchilov & Hutter, 2017). The performance in terms of the suggested evaluation metric was reported in Table 2.
We observe that, except on the RTE dataset, the performance was improved approximately the same compared with the default CCE method, which shows the benefits of using instance-speciffcally adjusted confidences. Although, if the dataset is relatively clean, the improvement might be marginal.
We also found mislabeled or ambiguous instances in these datasets. A typical example is the Corpus of Linguistic Acceptability (CoLA) (Warstadt al., 2019) dataset, which consists of English grammatical acceptability judgments. Six selected low-confidence training examples are given in Table We found that several ungrammatical sentences were mislabeled acceptable, and some syntactically acceptable sentences were labeled as unacceptable by annotators possibly because they have semantic errors. In this way, we may use the proposed method to probe if the model prediction consistent with our intent. More results are provided in Appendix E.
6 CONCLUSION
We have introduced novel variational approximation of the instance-dependent noise (IDN) model, referred to as instance-confidence embedding (ICE). Compared with existing methods based on the consissoniiionall noise (CCN) assumption, the proposed method is able to capture ristance-speciffi noise information and consequently improve the classification performance. The use of the one- dimensional instance embedding naturally derives order of training examples which can be used for detecting ambiguous or mislabeled instances. For future directions, it interesting to explore its combination with other training techniques and its extensions in data cleansing, learning with rejection, or active learning.
9

---

Under review as a conference paper at ICLR 2022
REFERENCES
Dana Angluin and Philip Laird. Learning from noisy examples. Machine Learning, 2(4))343-37,, 1988. 1, 4
Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In Proceedings of the 34th International Conference on Machine Learning, 2017.
Antonin Berthon, Bo Han, Gang Niu, Tongliang Liu, and Masashi Sugiyama. Confidence scores make instance-dependent label-noise learning possible. In Proceedings of the 38th International Conference on Machine Learning, 2021. 1, E.3
Nontawat Charoenphakdee, Jongyeong Lee, and Masashi Sugiyama. On symmetric losses for learning from corrupted labels. In Proceedings of the 36th International Conference on Machine Learning, pp. 961-970, 2019. 4
Nontawat Charoenphakdee. Zhenghang Cui, Yivan Zhang, and Masashi Sugiyama. Classification with rejection based on cost-sensitive classification. In International Conference on Machine Learning, pp. 1507-1517. PMLR, 2021. 3.2, 4
Pengfei Chen, Junjie Yc, Guangyong Chen, Jingwei Zhao, and Pheng-Ann Heng. Beyond class-
conditional assumption: A primary attempt combat instance-dependent label noise. In Proceed- ings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021. 1,4
Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. Learning with instance- dependent label noise: A sample sieve approach. International Conference on Learning Representations, 2021. 4, D. D, E.3
Jiacheng Cheng, Tongliang Liu, Kotagiri Ramamohanarao, and Dacheng Tao. Learning with bounded instance- and label-dependent label noise. In Proceedings of the 37th International Conference on Machine Learning, pp. 1789-1799, 2020. 2.3
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. arXiv preprint arXiv: 11X1::/8/2.0//18 2018. 5.1,
Jacob Devlin, Ming-Wer Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep
bidirectional transformers for language understanding. In Proceedings the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, pp. 4171-4186, 2019. 3.3, 5.2, E.2
Ran EJ- Yaniv and Yair Wiener. On the foundations of noise-free selective classification. Journal of Machine Learning Research, 11(5), 2010. 4
Lei Feng, Senlin Shu, Zhuoyi Lin, Fengmao Lx, Li, and Bo An. Can cross entropy loss be robust to label noise? In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, pp. 2206-2212, 2020.
Robert Fergus, Li Fei-Fei, Pictro Perona, and Andrew Zisserman. Learning object categories from Google's 's image search. In Tenth IEEE International Conference Computer Vision, volume 2, pp. 1816-1823, 2005. 1, 2.3
Aritra Ghosh, Himanshu Kumar, and PS Sastry. Robust loss functions under label noise for deep neural networks. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp-
1919-1925, 2017. 4
Jacob Goldberger and Ehud Ben-Reuven. Training deep neural-networks using noise adaptation layer. In International Conference on Learning Representations, 2017. 1, 2.3, 3.2, (3)
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, pp. 1321-1330, 2017.
4,
10

---

Under review as a conference paper at ICLR 2022
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in Neural Information Processing Systems, pp. 8527-8537, 2018. 4
Satoshi Hara, Atsushi Nitanda, and Takanori Machara. Data cleansing for models trained with SGD, In Advances in Neural Information Processing Systems, pp. 4215-4224, 2019. 1, 3.2, 4
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016. 3.3, 5.1, E.3
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2020. 3.3
Lu Jiang, Zhengyuan Zhou, Thomas Leung. Li-Jia Li, and Li Fei-Fei. MentorNet: Learning data- driven curriculum for very deep neural networks on corrupted labels. In Proceedings of the 35th International Conference on Machine Learning, pp. 2304-2313, 2018.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference Learning Representations, 2015. 5.1,5
Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009. 3.3, 5.1,
Meelis Kull, Miquel Perello Nieto, Markus Kängsepp, Telmo Silva Filho, Hao Song, and Peter Flach.
Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. Advances in neural information processing systems, 32:12316 11122316122666 2019. 4
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition, 1998. 5.1,
Junnan Li, Richard Socher, and Steven C.H. Hoi. DivideMix: Learning with noisy labels semi- supervised learning. In International Conference on Learning Representations, 2020.
Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou, Isolation forest. In 2008 eighth IEEE international conference on data mining, pp. 413-422. IEBE, 2008. 4
Sheng Liu, Jonathan Niles- Weed, Narges Razavian, and Carlos Bernandez-Grande Early-learning reg- ularization prevents memorization of noisy labels. In Advances Neural Information Processing Systems, pp.
2020. 1,4
Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEE Transactions on pattern analysis and machine intelligence, 38(3)4477461, 2015. 4
Yang Liu. Understanding instance-level label noise: Disparate impacts and treatments. In Interna- tional Conference on Machine Learning, 2021.
Yang Liu and Hongyi Guo. Peer loss functions: Learning from noisy labels without knowing noise rates. In Proceedings of the 37th International Conference on Machine Learning, pp. 6226-6236 2020. 4
Philip M Long and Rocco Servedio. Random classification noise defeats all convex potential boosters. Machine learning, 78(3):287-304, 2010. 4
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2017. 5.2, 6
Michal Lukasik, Srinadh Bhojanapalli, Aditya Menon, and Sanjiv Kumar. Does label smoothing mitigate label noise? In International Conference on Machine Learning, pp. 6448-6458, 2020. 4,
Yueming Lyu and Ivor W. Tsang. Curriculum loss: Robust learning and generalization against label corruption. In International Conference Learning Representations, 2020. 4
11

---

Under review as a conference paper at ICLR 2022
Xingjun Ma, Hanxun Huang, Yisen Wang, Simone Romano, Sarah Erfani, and James Bailey. Normal- ized loss functions for deep learning with noisy labels. In Proceedings of the 37th International Conference Machine Learning, pp.
2020. 4
Eran Malach and Shai Shalev-Shwartz. Decoupling "when to update" from "how to update". In Advances in Neural Information Processing Systems, pp. 960-970, 2017. 1,
Aditya Krishna Menon, Brendan Van Rooyen, and Nagarajan Natarajan. Learning from binary labels with instance-dependent noise. Machine Learning, (-- 100-100):5611155550 2018. 2.3
Aditya Krishna Menon, Ankit Singh Rawat, Sashank J Reddi, and Sanjiv Kumar. Can gradient clipping mitigate label noise? In International Conference on Learning Representations, 2020.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representa- tions in vector space. In International Conference on Learning Representations, Workshop Track Proceedings, 2013. 3.3
Baharan Mirzasoleiman, Kaidi Cao, and Jure Leskovee. Coresets for robust training of deep neural networks against noisy labels. In Advances in Neural Information Processing Systems, 2020. 1, 4
Hussein Mozannar and David Sontag. Consistent estimators for learning to defer to an expert. In International Conference on Machine Learning, pp. 7076-7087, 2020.
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt. In International Conference on Learning Representations, 2019. 3.3
Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K. Ravikumar, and Ambuj Tewari. Learning with noisy labels. In Advances in neural information processing systems, pp. 1196-1204, 2013, 1, 4
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning, 2011. 5.1,4
Due Tam Nguyen, Chaithanya Kumar Mummadi, Thi Phuong Nhung Ngo, Thi Hoai Phuong Nguyen, Laura Beggel, and Thomas Brox. SELF: Learning to filter noisy labels with self eafemensem.,ngg In International Conference Learning Representations, 2020. 4
Curtis G Northcutt, Lu Jiang, and Isaac L Chuang. Confident learning: Estimating uncertainty in dataset labels. arXiv preprint arXiv: /9/ 1.00068, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. PyTorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, pp. 8026-8037, 2019. A. E.2
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu Making deep neural networks robust to label noise: loss correction approach. In Proceedingy of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1944-1952, 2017. 1, 2,2 4, (4)
Jeffrey Pennington, Richard Socher, and Christopher D Manning. GloVe: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1532-1543, 2014. 3.3
Gabriel Pereyra, George Tucker, Jan Chorowski, Łukasz Kaiser, and Geoffrey Hinton. Regularizing neural networks by penalizing confident output distributions. In 5th International Conference on Learning Representations, Workshop Track Proceedings, 2017. 4, C
Matthew E Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HILT, 2018. 3.3
Amir Rahimi, Amirreza Shaban, Ching-An An Cheng, Richard Hartley, and Byron Boots. Intra order- preserving functions for calibration of multi-class neural networks. In Advances in Neural Infor- mation Processing Systems, 2020. 4
12

---

Under review as a conference paper at ICLR 2022
Scott E Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. Training deep neural networks on noisy labels with bootstrapping. In International Conference Learning Representations, Workshop Track Proceedings. 2015. 4, (2). C
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):21 2015.
Florian Schroff, Antonio Criminisi, and Andrew Zisserman. Harvesting image databases from the web. IEEE transactions on pattern analysis and machine intelligence, 33(4):754-766) 2010.
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initialization and momentum in deep learning. In International conference on machine learning, pp. 1 1139-1147 2013. 5.1
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 221188226, 2016. C
Sunil Thulasidasan, Tanmoy Bhattacharya, Jeff Bilmes, Gopinath Chennupati, and Jamal Mohd- Yusof. Combating label noise in deep learning using abstention. In International Conference on Machine Learning, pp. 6234 6243, 2019. 1, 3.2, 4, (6)
Brendan Van Rooyen, Aditya Menon, and Robert C Williamson. Learning with symmetric label noise: The importance of being unhinged. In Advances in Neural Information Processing Systems, pp. 10-18, 2015. 4
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations, 2019a. 5.2
Yisen Wang, Weiyang Liu, Xingjun Ma, James Bailey, Hongyuan Zha, Le Song, and Shu-Tao Xia. Iterative learning with open-set noisy labels. In Proceedings of the IEKE Conference on Computer Vision and Pattern Recognition, pp. 8688-8696, 2018. 4
Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan I Luo, Jinfeng Yi, and James Bailey. Symmetric cross entropy for robust learning with noisy labels. Proceedings of the IEFE International Conference Computer Vision, pp. 322-330, 2019b.
Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625-641, 2019. 5.2
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowiez, Joe Davison, Sam Shleifer, Patrick vOn Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-arr natural language processing. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing: System Demonstrations, 2020. E.2
Pengxiang Wu, Songzhu Zheng, Mayank Goswami, Dimitris Metaxas, and Chao Chen. A topological filter for learning with label noise. In Advances in Neural Information Processing Systems, 2020.
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin, Unsupervised feature learning via non- parametric instance discrimination. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3733-3742, 2018. 3.3
Xiaobo Xia, Tongliang Liu, Nannan Wang, Bo Han, Chen Gong, Gang Niu, and Masashi Sugiyama. Are anchor points really indispensable in label-noise learning? In Advances in Neural Information Processing Systems, pp. 6838-6849, 2019. 4, E.3
Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. Part-dependent label noise: Towards restance-dependenn label noise. In Advances in Neural Information Processing Systems, 2020. 1, 2.3, E.3
13

---

Under review as a conference paper at ICLR 2022
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion- A novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv:1708.07747: 2017. 5.1,2
Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2691-2699, 2015. 1, 2.3, 4, E.3
Yilun Xu, Peng Cao, Yuqing Kong, and Yizhou Wang. L. I_DMI: A novel information-theerrtii loss function for training deep nets robust label noise. In Advances in Neural Information Processing Systems, 2019. E.3
Yu Yao, Tongliang Liu, Bo Han, Mingming Gong, Jiankang Deng, Gang Niu, and Masashi Sugiyama. Dual T: Reducing estimation error for transition matrix label-noise learning. In Advances in Neural Information Processing Systems, pp. 7260-7271, 2020. 4, (5), E.3
Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? In Proceedings of the 36th International Conference on Machine Learning, pp. 7164-1177 2019.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understand- ing deep learning requires rethinking generalization. In International Conference on Learning Representations, 2017. 1
Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference Learning Representations, 2018. C
Yikai Zhang, Songzhu Zheng, Pengxiang Wu, Mayank Goswami, and Chao Chen. Learning with feature-dependent label noise: A progressive approach. In International Conference on Learning Representations, 2021a. 4, 5.1, (8), D, D, E.3
Yivan Zhang, Gang Niu, and Masashi Sugiyama. Learning noise transition matrix from only noisy labels via total variation regularization. In Proceedings of the 38th International Conference Machine Learning, 2021b.
Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In Advances in neural information processing systems, pp. 8778-8788, 2018. (7)
Zhaowei Zhu, Yiwen Song, and Yang Liu, Clusterability as an alternative to anchor points when learning with noisy labels. Proceedings of the 38th International Conference on Machine Learning, 2021. 4
14

---

Under review as a conference paper at ICLR 2022
A IMPLEMENTATION
In this section, we provide some implementation details of the proposed method for practitioners. Here, we only extract and highlight the core part to explain our method.
First, thanks to the development of PyTorch (Paszke et al., 2019), the instance embedding part can be actually implemented easily using torch. nn Embedding This is also partially why choose "embedding" as part of the name of the proposed method. The weight is initialized to be O for all instances and the embedding followed by sigmoid layer the confidence value is within [0, 1].
def embedding (size):
"**
size: the number of training examples #*"
embed = nn (si 1, sparse=True)
embed. wei wwtght.th.... .aaegggttttta...a... data. fill_(0,)) (0.) initialization embed nn Snnuenttt (embed, an.Sigmoid()) return embed
Second, assume that a neural network model 1 is defined and the index i of each training example is also provided, the learning objective can be calculated as follows:
def loss (i, y) ""n
1: index [batch_size)
x: input (batch_size, ...] y: label sbaech_size, num_ classess
t model (x) Section 2.1 Eq. (1)
с embed (i) Section 3.3 instance embedding
s transformation (t, c) Section 3.2 Eqs. (8) or (9)
1 cross cosss_entropyttoo (a, y) $ Section 3.1 Eq. (6)
return
Here, the trans o romooiiiiiooo function corresponds to the approximation family Eqs. (8) and (9). Please note the slight difference that and are the logits (output without softmax), not the probabilities for the purposes of easy implementation and numerical stability. We can choose from the following functions:
def inter__ineerolattion (t, C):
Section 3.2 Eq. (8)
return log softmax (t, dim=1) + (1 c) 1 / t. shape [1])
def power_transe www..trrsnooootionnnnoooo (t, c): Section 3.2 Eq. (9) and Section 4 (temperature scaling)
return c
Finally, The parameters of both embed and model can be optimized simultaneously using any gradient-based method such torch. opt apptim.. SGD and torch. optim. Adam.
15

---

Under review as a conference paper at ICLR 2022
B GRADIENT ANALYSIS
04
04
02
02
Figure 7: Contours of the log- log-likelihood w.r.t. p; and C using the linear interpolation (Eq. (8)) for K in (2,3, 5, 103.
AAAA
01
-- 0.
C-
C--07
C-099
ICE
AAAA
ICE.POW
Figure 8: Contours of the log-likelihood on the simplex when K using the linear interpolation (Eq. (8), top) and the power transformation (Eq. (9), bottom) for C in {0.1, 0.3, 0.5, 0.7, 0.9}.
In this section, we provide a basic gradient analysis and visualization for our proposed method.
The gradients of the log- likelihood using Eqs. (8) and (9) are
T P:
log (g.) = Cp: + +11 -1-C)
(linear interpolation)
(11)
Oc
i log (q:)
Σ p
log pi
(power transformation)
(12)
ac
pj
respectively. Their sign boundaries are p. and pt = H(g.p) respectively, where H(.,))
1
denotes the cross-entropy. We can find that for the linear interpolation (Eq. (8)), when Py
K"
ØL
åL
OC < and when Py
0. Similarly, for the power transformation (Eq. (9)), when OL
K ac
θL
Pý e -H(g.p)
0 and when Py e- H(g.p)
ac 0.
DC
The contours of the likelihood for different parameters are plotted in Figs. and 8. Note that the class- posterior p obtained from neural network, so can be influenced by other instances, especially adjacent instances. On the other hand, the confidence C' is obtained via instance embedding, so can take any value independently. If the predicted class-posterior Py for an instance x low (e.g.. because this instance is mislabeled and the majority of adjacent instances are predicted to belong to other classes), then the classifier tends to decrease its confidence value so as not to overfit this possibly mislabeled instance. The gradient magnitude is the largest when C' is high and Py is low (confident wrong prediction). the smallest when both C and pý are high (confident correct prediction). and in the middle when C' is low (uncertain prediction like random guess). In this way, neural network model with an option changing the confidence of prediction for individual training examples mitigate overfitting possibly mislabeled instances.
equip the
16

---

Under review as a conference paper at ICLR 2022
C LINEAR INTERPOLATION
CCE:
DKu(y/i))
Dri(Cy (1 C'ulpp)
SB
Piu(Cy (1 C'pppi
HB
Dai(Cy (1 C'
Dxu(y|CCp + C)u)
Figure 9:
Illustration of related methods, including the categorical cross-entropy (CCE), label smooth- ing (LS), soft/hard bootstrapping loss (SB/HB), and the proposed instance-confidencce embedding (ICE) with the linear transformation (Eq. (8)).
Linear interpolation between properties of an instance and other value is widely used technique for regularization machine learning, such as the bootstrapping loss Reed et al. (2015) and the label smoothing technique (Szegedy al., 2016; Pereyra al., 2017; Lukasik al., 2020). In this section, we briefly summarize related techniques and compare their differences.
Concretely, let p be the predicted probability vector for Y (Bq, (1)), y be the one-hot vector for the observed label, ÿ arg max p is the one-hot vector for the predicted label, u be the uniform
probability vector (u; K
A K-1 are all in the probability simplex. Let C' E [0, 1] be scalar linear interpolation parameter.
for i (1....,KK) Here, p. y.
Then, as also shown in Fig. 9, the learning objectives are equivalent to the following Kl.-divergences:
( I| p),
(categorical cross-entropy)
(13)
DRL(CY (1 C)u || p),
(label smoothing)
(14)
DKL + (1 C)p p),
(soft bootstrapping loss)
(15)
DRI(Oy + (1 [ p),
(hard bootstrapping loss) (16)
DRL(Y Cp + C)u).
(instance-confidencce embedding)
(17)
We see that the label smoothing and the bootstrapping loss methods smooth the target, but the proposed ICE method smooths the prediction. Note that it is impossible to let an instance- dependent parameter in other methods, because when C 0, the supervision signal Y can be completely lost.
Another technique using linear interpolation is mixup (Zhang al., 2018), which also interpolates the input features X between two instances. Therefore its characteristics could be more different than the methods mentioned above.
17

---

Under review as a conference paper at ICLR 2022
D SAMPLE SELECTION
Our method can be regarded as a "soft" sample selection method that de-emphasizes the supervision from possibly mislabeled training examples to extent. In the literature, several "hard" sample selection methods have been reported recently, e.g., Cheng et al. (2021); Zhang et al. (2021a). In this section, we discuss their relationship and the pros and cons of these methods.
Soft vs. hard, Hard sample selection methods usually use a predefined thresholding rule based on the regularized loss [(Cheng et al., 2021), Eq. (4)] or the predicted probability [(Zhang al., 2021a), Section 2.2] to select subset of possibly correct training examples at each iteration. The threshold is either determined dynamically (Cheng et al., 2021) or decreased via a manually designed schedule (Zhang et al., 2021a). Optionally, possibly incorrect labels can be flipped to the current model predictions (Zhang et al., 2021a).
We can find that such "hard" sample selection methods have two options for training example: it is either completely reliable or not reliable at all. However, since mislabeled instances are more likely to be near the decision boundary, such hard decision rules may result in unstable training. To cope with this issue, they often requires the confidence regularization, warm-up training, or a manually designed threshold decay schedule, which introduces more hyperparameters such as the regularization weight, the number of warm-up epochs, and the threshold schedule.
In contrast, in our method, the confidence C' the "soft" threshold - is continuous in [0, and is updated smoothly using the current prediction and gradient information so the training may be more stable. We do not need to introduce other hyperparameters. Although we do need specify an optimizer for it, we found that a simple SGD optimizer works well enough in experiments.
Another benefit of using continuous weights [0, 1] instead of discrete selections {0, 1} is that we can naturally derive a total order of training examples, which can be used for other tasks.
Gradient-based optimization vs. optimal solution. Next, let us focus on the soft sample selection. Note that the confidence C might have an optimal solution and can be obtained directly. However, it only optimal given the current predicted probability p. If the model is still under training and the estimation of p is not good enough yet, it is not always beneficial to give definite prediction of C, which may cancel out the advantage of the soft sample selection.
More specifically, to give prediction of C, we have the following choices. Their characteristics also listed below.
1. using a neural network
generalizes to unseen instances
depends on the trajectory of training. gradient- based optimization (heavy)
time: smooth, space: smooth
2. using instance embedding
does not generalize to unseen instances
depends the trajectory of training, gradient-based optimization (light)
time: smooth, space: non-smooth
3. using predefined rule based on the probability
generalizes to unseen instances (via the classifier)
does not depend on the trajectory of training, no optimization
time: smooth (soft)/non-smov) smooth (hard), space: non-smooth
Here, "smoothness" means if the values change gradually during training (time) and if adjacent instances tend to have similar values (space). We can use gradient-based optimization for instance embedding it is smooth in time (for stable training) and non-smooth in space (for instance- dependent noise). Mathematically, the proposed method takes accounts of not only gradient direction but also gradient magnitude. Intuitively, recording previous confidences C allows the model to have "memory" so C is not solely decided by the current prediction. For the above reasons, we prefer soft sample selection and gradient-basse based optimization.
18

---

Under review as a conference paper at ICLR 2022
E EXPERIMENTS
E. IMAGE CLASSIFICATION
Data. We used the MNIST, Fashion-MNISTT, Kuzushij--MNIST, SVHN CIFAR-10, and CIFAR- 100 datasets. The MNIST, Fashion-MNIST Kuzushiji-MNIST datasets contain 28 x 28 grayscale images in 10 classes. The size of the training set is 60000 and the size of the test set is 10000. The SVHN dataset contains 32 x 32 colour images in 10 classes. The size of the training set is 73257 and the size of the test set is 26032. The CIFAR-10 and CIFAR-100 datasets contain 32 x 32 colour images in 10 classes and in 100 classes, respectively. The size of the training set is 50000 and the size of the test set is 10000. We used 20% of the training sets for validation. We added synthetic label noise into the training and validation sets, We used a method similar to Zhang et al. (2021a) (hybrid noise with 20% Type-I+ + 40% Uniform) to generate synthetic instance- deeendddn noise. Concretely, we first approximated the clean class class-posters posterior using the original clean datasets, and corrupted the label from the most confident prediction to the second most confident one. The overall noise rate was around 52%, depending on the datasets. The test sets were not modified.
Models. For MNIST, Mashion-MNIST, and Kuzushiji-MNIST, we used sequential convolutional neural network with the following structure: Conv2d(channel=32) x2, Conv2d(channel--)) x2, MaxPoo 2d(size=2), Linea r(dim=128), Dropout
Li nea Tei r(dim=10). The kemel size of convolutional layers is 3, and rectified linear unit (ReLU) is applied after the convolutional layers and linear layers except the last one. For SVHN, CIFAR-10 and CIFAR-100 100, we used a ResNet-19 18 model (He et al., 2016). To ensure that C E [0, 1], we simply apply the sigmoid function that maps IR to [0, 1] to the embedding.
Optimization. For MNIST, Fashion-MNIST, and KuzushijiMNNIST, used an Adam optimizer (Kingma & Ba, 2015) with batch size of 512 and learning rate of x 10-3 The model was trained for 2000 iterations (17.07 epochs) and the learning rate decayed exponentially to 1 x 10 For CIFAR-I0 and CIFAR-100 100, we used a stochastic gradient descent (SGD) optimizer with batch size of 512, momentum of 0.9, and weight decay of 1 x 10- The learning rate increased from 0 to 0.1 linearly for 400 iterations and decreased to 0 linearly for 3600 iterations (4000 iterations/40. 40..9 epochs in total). For SVHN, the setting was the same as CIFAR-10 except the model was trained for 1000 iterations.
Results. The ridgeline plots of the confidence C' during training are given in Fig. 10. The densities seem bimodal in the early stage for MNIST, FMNIST, and KMNIST because we did not log the result at the end of each epoch but based on the number of batch iterations. Therefore, the confidence of some instances may have been updated more times than it of the others. In the final stage, the confidence also converges and instances with flippe//rriginal labels are almost separated by the learned confidence. Low-confidence training examples are given in Figs. 11 and 13, which are partially presented in Fig. 5. The class-wise spectra are given in Fig. 12, which are partially presented in Fig. 6.
E.2 TEXT CLASSIFICATION
We implemented the BERT-base model (Devlin et al., 2019) using PyTorch (Paszke et al., 2019) and HuggingFace's 's t ans foreess (Wolf et al., 2020) libraries. We used pretrained model and an AdamW optimizer (Loshchilov & Hutter, 2017). The batch size was 32 and the weight decay
MNIST (LeCun et al., 1998) http:://yamm wwt.://ccw.n....ceee//eeeeam
Fashion-MNIST (Xiao et
al.,
2017)
fashion-mnist
(MIT license)
Kuzushiji-MNIST (Clanuwat et al., http:
(CC BY-SA 4.0
license)
SVHN (Netzer et http:t/pe/..td...tttaaadaeaaeees
CIFAR-10, CIFAR-100 (Krizhevsky, https::/ww..es.torottooooe
html
bert b-----aaseeeee htt.aahhhttt.///ggggggg.c....ooe
19

---

Under review as a conference paper at ICLR 2022
MNIST
INNIST
SVHN
CIRAR-10 EPAR-100
IOEELN
Figure 10: Ridgeline plots of the confidence C' during training. The density is estimated via Gaussian kernel density estimation (KDE). The red/blue curves represent the confidence of instances with flipped/original labels, respectively.
was 0.01, otherwise used the default hyperparameters. The model trained 4 NVIDIA Tesla P100 GPUs in parallel with the mixed precision training option (fp16) enabled. For the CoLA, MRPC, RTE, and WNLI datasets, the model was trained for epochs and otherwise 3 epochs. Low-confidence training examples are given in Tables 6 to 13.
E.3 MORE EXPERIMENTAL RESULTS
We also conducted the experiment on common setting where the labels are randomly flipped, i.e., instance-indeeendently The noise rate was 50%. The test accuracy is listed in Table 4.
Table 4: Accuracy (%) on the MNIST, FMNIST, KMNIST, SVHN, CIFAR-1O, and CIFAR- 100 datasets where 50% of labels are randomly flipped. "Mean (standard deviation)" for 10 trials are reported. Outperforming methods are highlighted in boldface using one-tailed t-tests with significance level of 0.05.
MNIST
FMNIST
KMNIST
SVHN
CIFAR-10
CIFAR-100
CCE
94.91(0.43)
85.05(0.52)
80.40(1.25)
71.50(1.68)
68.34(0.82)
47.09(0.65)
Bootstrapping
97.30(0.28)
87.24(0.36)
84.21(1.01)
76.62(0.97)
75.97(0.45)
49.56(0.12)
Adaptation
96.27(0.41)
86.20(0.87)
81.38(2.07)
68.58(6.45)
63.95(5.94)
31.70(1.28)
Forward
95.09(0.56)
(5.51
80.76(1.29)
74.43(6.42)
68.28(0.62)
47.92(0.31)
Dual-T
97.68(0.25)
88.07(0.35)
86.98(0.80)
72.36(0.88)
78.53(0.32)
50.80(0.40)
DAC
96.60(0.47)
86.87(0.48)
82.77(0.74)
80.97(4.83)
71.55(0.34)
(0.04
GCE
98.31(0.13)
88.76(0.26)
88.39(0.60)
75.03(0.98)
80.38(0.67)
55 (0.64(0.40)
ICE-LIN
98 0...
89 89.41(0.18)
89.61(0.41)
(7.51(0.75)
82.08(0.39)
55 (0.477
ICE-POW
98. 98.60(0.09)
89 89.29(0.20)
89.21(0.53)
79.91(0.96)
82.14(0.44)
(0..3((0
Further, we tested the proposed method on the Clothing I M dataset (Xiao et al., 2015), which is real-world noisy label dataset. We followed the convention and trained ResNet-50 model (He et al., 2016) only on the 1M noisy training set for 10 epochs. On this dataset, ICE-POW achieved the accuracy of 72.67%, which comparable or superior some existing works. For reference, the reported performances of other methods including CCE, Forward, T-Revision (Xia et al., 2019). dual-T (Yao al., 2020), PTD (Xia et al., 2020), DMI (Xu al., 2019), CORES (Cheng al., 2021), ILFC (Berthon et al., 2021), and PLC (Zhang et al., 2021a), are cited in Table 5.
Table 5: Test accuracy on Clothing M.M. M.
CCE
Forward
T-Revision
Dual-T
PTD
DMI
C
ILFC
PLC 74.02
68.94
70.83
70.97
71.49
71.67
72.46
73.24
73.35
Note that some methods are superior to ours in terms accuracy on Clothing This may be due to different evaluation settings and training techniques. Further investment is left for future work.
20

---

3
1
Z
L
20M
Jam
Tedt
J00n
Labr:: 2013
Ealen 178M
3010
2010
2011
2
9
b
9
SIMT 2H1 0000
SONI AEDO BINDT
FUW LETH
TMN (EDI
2018
BONN 2014
20121
2
4
Leit
BUALL 2X1L
4
7
4
1
aa) MNIST
(b) FMNIST
SON TIOP
SITM LENEIC
NOM
TORE 1est:
Taln JUMN 99
TIR MITS 5
TWOT TIMA
5
107
9
16
aNd MONMLE
LAR
201m
DAI
177
8
9
280
8
7
899
25
8
10
0
9
5
95
18
(c) KMNIST
(d) SVHN
HINU
2011
CANNO 170M
LAVE 13445
Newo: 2013
400 Bant
EMM0Y: 200h
111
FORE THITAA
2010T 2017
20101 1961
Ealee: 4136
(e) CIFAR-10
(f) CIFAR-100
Figure 11: The 32 most low-confidence training examples in the MNIST, FMNIST, KMNIST, SVHN, CIFAR- 10 and CIFAR-100 datasets, ordered left-right, top-down by increasing confidence.
21

---

1
2
2
2
2
2
2
2
3
3
3
3
3
3
5
4
4
4
4
4
4
4
5
5
5
5
5
5
5
J
6
4
6
6
6
6
6
7
7
7
7
7
7
7
4
8
8
8
8
8
8
8
Q
9
9
9
9
9
9
9
4
(a) MNIST
(b) FMNIST
22

---

00
50
31
00
3
2
3
2
1
8
61
6
1
16
7
8
2
2
2
2
32
2
25
9
3
15
3
30
3
5
45
4
4
4
4
[48
4
4
15
15
57
5
65
5
655
J2
1
6
6
6
6
6
66
6
26
$
7
7
275
7
3
1
E
8
8
8
8 9
80
8
8
9
49
9
95
9
2
(c) KMNIST
(d) SVHN
23

---

(c) CIFAR-10
(f) CIFAR 100 (first 10 classes)
Figure 12: The class-wise spectra from high-confidence (left) to low-confidence (right) training examples in the MNIST, FMNIST, KMNIST, SVHN, CIFAR-I0 and CIFAR-I00 datasets.
24

---

1neex:
label: label:
Label:
Label:
Label:
down
Label:
DUPSSS lacket
gunes: sult
SUINES hoodie
Lacket
twnater
quaas:
thirt
guesst
chilton
gueoss Jaciet
undex: 572829
Indox: 582544 down
Index: 579161 down
Index: 589719 Tabel:
Index: 586829
Indox: 588544 dow
Indox: 589991 Label: down
Index: 568672 Tabel::
label:
down cost
Index:
label: knitwear
Label:
label:
Labeli murator
BUBSSI draan ozar
quessi shan
quaes: T-shirt
Quees: Tacket
quecst
tanator
QUARSI greas
quasst windbreaksr
Indec: $92252
Indect 179129
Index: 556243 Jacket
Indexc 598611
Indrct 582275
Indec: 563993 jarket
Indec: 566865
Index: 575311
label: dmenn one
twner
down cont
cast
Label: Jacket
label: down cost
Figure 13: The 32 most low-confidence training examples in the Clothing M dataset, ordered left-right, top-down by increasing confidence.
25

---

Table 6: The Corpus of Linguistic Acceptability (CoLA)
Index
Text
aabel:acceptable quess:unacceptabl
390
sentence: often sees Mary.
1abel:acceptable unnoopttttee
5766
sentence:Heidi believes any description herself.
label:unacceprable guess:acceptable
2801
sentence:Paula hit the sticks.
label:unaccepttable guess raceptabeee
1522
sentence:That the sun is out was obvious.
label:accceptable guess::nacoeptalle
8332
sentence:I wanted Jimmy for to come with me.
label :aceptabee guess: unncceppabee
300
sentence: They failed to tell me which problem the sooner solve, the quicker the folks up at corporate headquarters.
label:acceptable quess:unacceptable
7813
sentence:I went to the shop for to get bread.
label:accepptbble g
5904
sentence:It hailed.
Label:unacoeprable :
4159
sentence:Fifteen years represent a long period of his life.
label:unacceptable guess:acceptable
2479
sentence:Kelly buttered the bread with butter.
label:accceptable guess:acoeptable sentence:They parted the best of friends.
3846
Label:unacceprblle (uess:acceptable
7371
sentence:The hiker will reach the top of the mountain for an hour.
label:unacceptable g
430
sentence:It's probable probabb in general that he understands what/s s going on.
Label:unacoeptable :cceptablee
6795
sentence:leenry wanted to possibly marry Fanny.
:abep:acceee guess:acceptable
1115
sentence::e attributed to short circuit the fire which.
label:unacceptable quess: snacccppacee
4155
sentence:Two drops sanitize anything in your house.
aacep:aceee guess: acceptaaceee
1367
sentence :W elected president the boy's guardian's an's employer.
label: guess: unacceptable
7756
sentence: That monkey is ate the banana
label:unacceptable guess:acceptable
4445
Sentence:Georgee has went to America.
l guess:unacceptabll
4015
sentence He seems intelligent to study medicine.
26

---

Table 7: The Stanford Sentiment Treebank (SST2)
Index
Text
label:posite quess:positive
50155
sentence:a thirteen-year-old 's book report
Label:negative quess:negative
58416
sentence:blues
: quess:positive
sentence:" synthetic is the best description of this well
59724
beautifully produced film that sacrifices its promise for high-powered star pedigree
label:positive quess:negative
24696
:lamer instincts
label:positive quessipositive
34494
sentencerhad released the outtakes theatrically and used the film as a bonus feature on the dvd
label:negative guess:negative
54555
sentence:pretentioos fascinating ludicrous provocative and vainglorious
label:positive quess:negative
29155
sentence can fergiven for frequently pandering to fans of the gross-out comedy
Label:negative guess:positive
44610
sentence:below
label:negative guess:positive
66148
sentence:'s cliche to call the film refreshing
label:positive guess:negative
11869
sentence:go unnoticed and underappreciated
label :negativee quess:
sentence: the film is an earnest try at beachcombing verismo, but it would be even more indistinct than it is were it not for the striking, quietly vulnerable personality of ms. ambrose
55848
label:negative quess:negative
57359
sentence: ferrera
label:positive guess:negative
42232
sentence: fonginee any shoddy product as long there 's little girl-on-girt action
label :positive guess:negative
15783
sentence:have finally aged past his prime
label:negative
57186
sentence:hollywoodd war-movie stuff
label:positive guess:negative
52071
sentence:the gags
labil guess:negative
1896
sentence:missing from the girls big-screen blowout
label:positive quess:negative
64779
sentence:growing strain
label:positive guess:negativg sentence:you to bite your tongue keep from laughing at the ridiculous dialog or the oh-so convenient plot twists
3940
27

---

Table 8: Microsoft Research Paraphrase Corpus (MRPC)
Index
Text
labeiiequivaient guess:not equivalent
sentencel: :e need a certifiable pay as you go budget by mid-July or schools wont open in September strayhorn said
799
sentence2 Texas lawmakers must close $ 185.9 million budget gap by the middle of July or the schools wont open in September Comptroller Carole Keeton Strayhorn said Thursday.
label:not equivalent quess:equivalent
sentencel :Tt 's also strategic win for Overture given that Knight Ridder had the option of signing on Google 's services
469
sentence2:It also a strategic win for Overture given that Knight Ridder had been using Google 's advertising services
label:equivalent guess:not equivalent
sentencel: The broader Standard & Poor 's 500 Index < SPX > edged down 9 points or 0. 98 percent to 921 sentence2 The Standard & Poor 's 500 Index shed 5.20 0. 6 percent. to 924.42 as of 9 33 a.m. in New York
1037
label:equivalent guess:not equivalent
sentencel: Sens. John Kerry and Bob Graham declined invitations to speak sentence2: : no-shows Florida
1178
Sens. John Kerry of Massachusetts and Bob Graham of
label:equivaient guess:not eguivalent
sentencel The Dow Jones industrial average closed down 18.06 or 0.2 per cent at 9266.51
1753
sentence2 The blue-chip Dow Jones industrial average < .JII > slipped 44.32 points or 0.48 percent 9, 240.25
Table 9: Quora Question Pairs (QQP)
Index
Text
label:duplical quess: duessddupiccae
216515
questionl:Why does Quora censor opinions and answers?
question2:Does Quora censor questions and answers, and should they?
label:not duplicate quess:duplicate
343656
question1:Coule India's surgical strike in POK be an elaborate hoax or play? question2:Did India really conduct a surgical strike on Pakistan?
label:not duplicate guess:not duplicate questionl:Why is financial literacy generally not taught in American high schools?
266594
question2:Why isn't financial literacy taught in today's public schools?
label:not duplicate quess:not duplicate
251996
questionl:What is life like in communist countries?
question2:What would life in legitimate Communist country be like?
label:not duplicate quess:duplicate questionl:What is the best option for Indian politics and politicians?
7963
question2:Whis the options of Indian politics and politicians?
28

---

Table 10: MultiNLI (MNLI)
Index
Text
Labelientailment guess: neutral
premise:The, ruins the huge abbey of Jumiyges are perhaps the most the white- granite shells of two churches, the Romanesque Notre-Dame and the smaller Gothic Saint-Pierre. typothesissNotze-Damee is larger church than Gothic
218290
label:neutral guess:entailmen
premise:Unlest you feel really safe in French metropolitan traffic, keep your cyeling you rent bike at many railway stations for the villages and country roads.
39431
hypothesis:You should not cycle in the French metropolitan area.
Label:contradictiin quess:neutral premise:1 don't think sc. hypothesis:I have real idea.
27574
label:neut:a guess:neutral
premise se:A set of stone doors in the wall slid to the side to reveal a screen on which various torture scenes began to appear. hypothesis: The doors hid a television screer.
258544
contraicontiiiit guess guess:neutral
premi :one seems comfortable with the notion removing Clinton for sex- related misdeeds.
320518
hypothesis: don't want Clinton touching sex related ordeals
Table 11: Question NLI (QNLI)
Index
Text
label not entai ment guess :not entai 1ment
question: WWat caused Latin America's s right-wing authorities to support coup o' etats?
1659
sentence:This was further fueled by Cuban and United States intervention which led to political polarization.
label entailment guess:not entailment
question:Wan antenna type is portion of the half wave dipole? sentence The monopole antenna is essentially one half of the half-wave dipole, single 1/4-wavelength element with the other side connected to ground or equivalent ground plane (or counterpoise).
5876
label:entailment guess:entailment
question: are Toxicara canis infections spread?
5829
Contence:Toxocaa canis (dog roundworm) eggs in dog feces can cause toxocariasis.
labeltentailment guess:entailment
question:Wyy Why did Madrid cede the territory to the US sentence:Florida had become a burden to Spain, which could not afford to send settlers or garrisons.
77419
label entailment guess:not entailment question: What has distinction between the categories of voiced, voiceless, aspirated and unaspirated?
9576
sentence:Some of the Dravidian languages, such Telugu, Tamil, Malayalam, and Kannada, have a distinction between voiced and voiceless, aspirated and unaspirated only in loanwords from Indo-Aryan languages.
29

---

Table 12: Recognizing Textual Entailment (RTE)
Index
Text
label:not entailment quess:entai iment Iment
sentencel:Bogotaa 4 May 88 The dissemination of a document questioning Colombia's oil policy, is reportedly the aim of the publicity stunt carried out by the pro-Castro Army Of National Liberation, which kidnapped several honerary consuls, newsmen, and political leaders. sentence2:Several honorary consuls were kidnapped 4 May 88.
2429
label:not entailment quess:enta Iment
sentencel: : official religion is Theravada Buddhism, which is also practiced in neighboring Laos, Thailand, Burma and Sri Lanka. sentence2: The official religion of Thailand is Theravada Buddhism.
2463
label:entailmmnt guess:not entailment
sentencel: The Catering JLC formulates pay and conditions proposals of workers in the industry which, if approved by the Labour Court, legally binds employers to pay certain wage rates and provide condition ons of employment. However, the QSFA now contends that the JIC has no right to make such a legally binding
1361 provision, as Section 15 of the Constitution states that the sole and exclusive power to make laws is vested in the Oireachtas, and no other authority has power to make laws for the State. It also argued that the existence of the minimum wage and 25 other pieces of legislation protecting employees' rights means that there is no need for JLCs. The chairman of the OSFA, John Grace, warned that the situation would lead to job losses and closures. sentence2: John Grace works for QSFA.
Table 13: Winograd NLI (WNLII)
Index
Text
label: aabeleentailment guess:not entailment sentencel :usan knew that Ann' son had been in accident, so she told her about it.
266
: told her about it. label rentai Iment guess: posss:nott not entai Iment sentencel:Jos paid the detective after delivered the final report on the case.
478
sentence2: The detective delivered the final report on the case.
label:entailment guess:not entai Iment
sentencel:Dan had to stop Bill from toying with the injured bird. He is very compassionate.
294
sentence2:Dan is very compassionate.
label:entailment quess:not entailment
sentencel :aan took the rear seat while Bill claimed the front because his "Dibs !" was slow.
586
sentence2: :an took the rear seat while Bill claimed the front because Dan's s Dibs!" R was slow.
label:entailment guess:not entailment
sentencel :ark was close to Mr. Singer's heels. He heard him calling for the
243 captain, promising him, in the jargon everyone talked that night, that not one thing should be damaged and all his crew had best stay in the cabin until the work was over. sentence2:He heard Mr. Singer calling for the captain.
the ship except only the ammunition, but the captain
30


Reviews:
[{"review_id": "ffqcgQLgOB0", "reviewer": "Reviewer_cgg6", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall this work has enough novel and significant contributions, but some issues with the experiments weaken the work. The single-parameter noise model and the use of ICE to learn with IDN are novel to my best knowledge, though aspects of these ideas exist in other literature. The paper does a good job describing the proposed method, but the experiment section lacks some important details.\n\n---\nStrengths:\n\n- The single-parameter noise model proposed in this work is somewhat novel, and the idea is relatively simple (which is a good thing). This simplicity allows easier estimation (smaller estimation error), though it may have larger approximation error, compared with methods that model general noise transition function $T(x)$. But it indeed has a good balance of both approximation error and estimation error, in the middle of CCN (where $T(x)$ is a constant matrix) and general $T(x)$. This noise model is well-motivated.\n\n- The use of ICE to overcome the issue where $T(x)$ could vary significantly for two adjacent instances is interesting and seems beneficial.\n\n---\nWeaknesses:\n\n1. Lack of theoretical results. Variational approximation in this paper is only used to justify that the proposed single-parameter noise model leads to a valid lower bound. In fact, one can use any noise model and still get a valid lower bound using the same justification. The paper does not provide any theoretical analysis of the gap. Also, the connection with CCN is not clear. It looks like the proposed noise model does not include CCN as a special case.\n\n2. In the experiments, it is not clear how that clean labels and noise are generated. The authors only mentioned that they followed a similar way used in Zhang et al. (2021a). However, the exact details are missing in the paper and the appendix. I am kindly asking the authors to provide this information. Some specific questions:\n\n- In Zhang et al. Section 4, they first trained a model to approximate the posterior class probability $\\eta(x)$ using the whole data set. Then they sampled $y_x \\sim \\eta(x)$ for each instance $x$. Instead of using the original labels, they used the sampled labels as clean labels. Was this paper following the same approach?\n\n- In Zhang et al. Section 4, several types of noise generation are considered. Which type did this paper use? If for different data sets different types were used, please list them accordingly.\n\n- In Table 1 of this paper, how was the overall noise rate controlled? If this paper followed Zhang et al. by multiplying the noise function by some constant factor, please specify the constants. Please also list the controlled noise rate for each data set.\n\n3. This paper did not compare with the method in Zhang et al. (2021a) without any justification, even though the paper used Zhang et al.'s setting for noise generation. Please add the method in Zhang et al. to Table 1, or provide explanation why you chose not to compare with Zhang et al.\n\n4. In Section 3.2, one important assumption is that the noisy posterior class probability ($q$) and the clean posterior class probability ($p$) share the same argmax. This is quite a strong assumption, because effectively, with this assumption, one can train directly on the noisy examples using standard methods and predict correctly (in the Bayes optimal sense). In fact, such assumption is not necessary for good predictions. For example, in CCN setting, if the noise transition matrix is invertible, one can train a good classifier using noisy examples and predict well w.r.t. the clean distribution. In Zhang et al., no such assumption was made.  A related question: how is this assumption enforced in the noise generation for the experiments?\n", "summary_of_the_paper": "This paper studies instance-dependent noise (IDN) problems. It proposes a single-parameter (confidence) model for the noise generating process. Moreover, *instance-confidence embedding* (ICE) method is employed in the training process. The experimental results on various image and text classification tasks confirm the effectiveness of the proposed method.", "summary_of_the_review": "Overall this work has enough novel and significant contributions (single-parameter noise model and ICE). But it has a strong assumption (argmax preserving) and lacks theoretical justifications. More importantly, the experiment section lacks detailed descriptions and did not compare with Zhang et al. (2021a). These shortcomings make me skeptical about the effectiveness of the proposed method. Therefore, I vote for a weak reject.\n\n---\n\nAfter rebuttal:\n\nI raised my score to 6. See my reply below for more details.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "XkEcSrWV2eH", "reviewer": "Reviewer_ZuLD", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength: \nThe suggested method captures instance-specific noise information and thereby improves classification performance when compared to previous methods based on the class-conditional noise (CCN) assumption.\n\n\nConcerns:\n1.\tFor an instance X with label Y=i, how do you define $T_{kj}(x)$, where $k \\neq i$.\n2.\tAt the bottom of page 2, matrix-valued function $T$ is introduced. Is it different from the transition matrix $T(x)$?\n3.\tWhy do you need $K\\times K$ parameters to model the transition relation for each instance X while the $p$ and $q$ you defined are both $\\in \\Delta^{K-1}$?\n4.\tThe previous works employed some assumptions or are based on domain-specific knowledge but they fully model the IDN. In your method, the X is replaced by one postulated variable C learned by embedding an index parameter. Is this assumption strong?\n5.\tThe authors claim that most columns of the IDN transition matrix have only limited influence on the class-posterior estimation and the transition function should be argmax-preserving function. Does it mean this method can only deal with label noise with a low noise rate?\n6.\t$h(\\boldsymbol{p} ; 0)=\\boldsymbol{u}$, where $\\boldsymbol{u} \\in \\Delta^{K-1}$. Does it mean $u$ is uniformly sampled front the rest $K-1$ classes expect the true class? Why is that?\n7.\tIn Eq. (8), what is $u_i$? Is it the $i-th$ element of $u$, then what is $u$?\n8.\tEq. (9) needs to be elaborated. \n9.\tThe choices of $h$ and $g$ seem come from nowhere and need more justification. Why can this method work?\n10.\tImportant baselines are missing. All the IDN methods the authors mentioned in the introduction part were not considered as baselines. \n11.\tSome grammatical issues should be fixed.", "summary_of_the_paper": "This paper presented an instance-confidence embedding model (ICE), which is a variational approximation of the instance-dependent noise (IDN). Given the observation that most columns of the IDN transition matrix have only limited influence on the class-posterior estimation, this paper uses a single-scalar confidence parameter to model the noise and uses a simpler transformation to transfer clean class posterior to noisy class posterior. The suggested method captures instance-specific noise information and thereby improves classification performance when compared to previous methods based on the class-conditional noise (CCN) assumption.", "summary_of_the_review": "Overall, the proposed method is too heuristic without enough justification. The experiments are not sufficient.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8E5Y9-_JXgj", "reviewer": "Reviewer_4pht", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "$\\textbf{Strengths}$\n* **Novelty** This work provides a novel and interesting approach to using only a single-scalar confidence parameter to approximate the IDN transition matrix.\n* **Ambiguous/Noisy-label detection** The 1-D instance embedding is potentially beneficial for detecting ambiguous or wrongly labeled instances. \n* **Extensive empirical validation** The authors test the effectiveness of ICE in multiple image classification datasets and text classification tasks.\n\n$\\textbf{Weaknesses}$\n* **A single-scalar V.S. $K\\times K$ noise transition matrix** Admittedly, the $K\\times K$ noise transition matrix is usually of a sparse pattern. Will a single-scalar miss crucial information? Or in other words, could authors explain why this approximation captures necessary/useful noisy-label information at the instance level?\n\n* **The intuition of $g$** It would be better if the authors could provide me with more intuitions of $g$ in the role of confidence score.\n\n* **IDN-based baselines**  Learning with instance-dependent noisy labels has become a popular topic in recent years. And several approaches have been proposed to mitigate the impact of IDN. However, the majority of baselines adopted in Table 1 are designed for the CDN model. To further validate the effectiveness of ICE, it would be better if the authors could compare with some more IDN-based methods, for example, [1]-[5].\n\n* **Real-world noisy benchmarks** Evaluating the rationality of (synthetic) instance-dependent label noise is non-trivial. Real-world noisy labels are commonly assumed to follow the IDN pattern. It would be better if the authors could test the performances on real-world noisy benchmarks, especially some small-scale benchmarks such as Animal-10N [6], Controlled Noisy Web Labels [7], or CIFAR-10N, CIFAR-100N [8].\n\n\n$\\textbf{Reference}$\n\n[1] Part-dependent Label Noise: Towards Instance-dependent Label Noise. (NeurIPS, 2020)\n\n[2] Learning with Feature-Dependent Label Noise: A Progressive Approach. (ICLR, 2021)\n\n[3] Learning with Instance-Dependent Label Noise: A Sample Sieve Approach. (ICLR, 2021)\n\n[4] A Second-Order Approach to Learning With Instance-Dependent Label Noise. (CVPR, 2021)\n\n[5] Confidence Scores Make Instance-dependent Label-noise Learning Possible. (ICML, 2021)\n\n[6] https://dm.kaist.ac.kr/datasets/animal-10n/.\n\n[7] https://ai.googleblog.com/2020/08/understanding-deep-learning-on.html.\n\n[8] http://noisylabels.com/.", "summary_of_the_paper": "This paper introduces ICE, an instance-confidence embedding approach to learn with a challenging noisy label setting: instance-dependent noise (IDN). Considering the sparse property of IDN, the authors propose to use instance embedding by equipping a trainable parameter to each data instance. Experiments results on various image datasets and text data show the effectiveness of the proposed method with the presence of label noise.", "summary_of_the_review": "This is an overall novel and interesting paper. While I still have some concerns regarding the single-scale confidence parameter and the intuition of the instance-confidence embedding function $g$. Besides, I am not fully convinced of the effectiveness of ICE by referring to the baseline selections. I am willing to increase my score if some of my above concerns are well addressed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3UYyDOzvZL5", "reviewer": "Reviewer_ARcm", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n1. This paper is well written and easy to follow. The motivation is strong and interesting. It is of great importance to approximate IDN with a simple method since IDN is generally hard to model. This paper proposes a simple way to approximate IDN by using single-scalar confident parameter.\n\nConcerns:\n1. I am quite confused by the part of embedding. The embedding is built on the sample index instead of the sample (feature) itself. I think it does not follow the definition of IDN. $C$ in Equation (8) (9) should be dependent on the feature rather than the feature index. \n\n2. Many methods have been proposed in the literature to deal with IDN (A1, A2, A3, A4, A5). However, in Table 1, the authors do not compare any of them. Besides, the performance gain of ICE is minor compared to GCE on CIFAR10 and CIFAR100. \n\n3. ICE is similar to label smoothing in the formulation. Authors are encouraged to perform a comparison in the experiments. \n\n4. Why only perform 40 epochs on CIFAR10 on CIFAR100. It does not follow the convention. Does the network begin to overfit label noise as training proceeds?\n\nA1: Learning with bounded instance- and label-dependent label noise. ICML 2020\n\nA2: Part-dependent label noise: Towards instance-dependent label noise. NeurlPS 2020\n\nA3: Confidence scores make instance-dependent label-noise learning possible. ICML2021\n\nA4: Learning with feature-dependent label noise: A progressive approach. ICLR2021\n\nA5: Learning with instance-dependent label noise: A sample sieve approach. ICLR2021", "summary_of_the_paper": "This paper proposes ICE (Instance-confident embedding) method for learning with IDN (instance-dependent label noise). ICE approximates IDN by using a single-scalar confident parameter for each sample based on the assumption that $P(Y|x)$ is close to a one-hot vector. ICE is incorporated into the loss function for efficient learning process under label noise. Authors conduct experiments on both image and text classification to verify the effectiveness  of ICE and perform gradient analysis for further understanding the method.\n", "summary_of_the_review": "This paper has a strong motivation. However, it seems that the proposed approach (ICE) does not strongly connect to IDN and the performance gain is minor compared to existing approaches. Authors should provide more explanations of why embedding on the index can approximate IDN and perform more comparisons. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new method for the problem of learning under instance-dependent noise (IDN). The idea is to construct a variational approximation to the ideal training objective, which involves learning a single scalar C(x) per instance. In turn, each such scalar is treated as an additional parameter to be learned by the network.

Reviewers generally found the basic idea of the proposal to be interesting and novel, with the response clarifying some initial questions on the design of the network to learn C(x). The paper is also well-written, and presents experiments on image and text classification benchmarks. Some concerns were however raised:

(1) _Limited theoretical justification_. There is limited formal analysis of when the proposed method can work well.

(2) _Lack of comparison to IDN baselines_. The original submission did not include any IDN baselines as comparison. The revision included results of the method of (Zhang et al., '21a), which is on-par or better than the proposed method; it seems that this baseline really ought to have been included in the original submission, but it is appreciated that these have been added. A related concern was the marginal gains over the GCE method on the CIFAR datasets.

(3) _Sufficiency of learning a single parameter_. The paper learns a single scalar per sample. Several reviewers were unsure on the sufficiency of this parameter to capture the underlying noise distribution.

For (1), the authors acknowledge theoretical analysis as an important future direction. This is perfectly reasonable, but does then require weighting more any issues with the the conceptual and empirical contributions of the paper.

For (2), the response clarified that most of these operate either in the binary setting, or require auxiliary information. This is a valid motivation for the present work; it would however be more compelling to include results in a binary setting, to better understand the strengths and weaknesses compared to existing proposals. The response also clarified the present method does not claim to improve upon state-of-the-art performance, but rather proposes a simple method which has additional applications (as shown in Appendix E). This is a reasonable claim; however, to my taste, there is insufficient discussion of the PLC method (Zhang et al., '21a), and what new conceptual information the present work offers.

For (3), the response argued that the present results already demonstrate the efficacy of using a single parameter, and that using multiple parameters can be studied in future work. One reviewer was not convinced of the efficacy being shown in some of the results in Appendix E. It could strengthen the work if there is an empirical analysis of when the single parameter assumption starts to break down; e.g., perhaps under increasing levels of CCN noise?

Overall, the paper has interesting ideas and some nice analyses. At the same time, there was clear scope for improvement in the original submission. This was partially addressed in the revision, but given that several domain experts retain reservations (particularly in regards to comparisons against prior IDN works), it is encouraged that the authors incorporate the above comments for a future version of the paper.
Decision: Reject