Year: 2021

Abstract:
Label noise in multiclass classification is a major obstacle to the deployment of learning systems. However, unlike the widely used class-conditional noise (CCN) assumption that the noisy label is independent of the input feature given the true label, label noise in real-world datasets can be aleatory and heavily dependent on individual instances. In this work, we investigate the instance-dependent noise (IDN) model and propose an efficient approximation of IDN to capture the instancespecific label corruption. Concretely, noting the fact that most columns of the IDN transition matrix have only limited influence on the class-posterior estimation, we propose a variational approximation that uses a single-scalar confidence parameter. To cope with the situation where the mapping from the instance to its confidence value could vary significantly for two adjacent instances, we suggest using instance embedding that assigns a trainable parameter to each instance. The resulting instance-confidence embedding (ICE) method not only performs well under label noise but also can effectively detect ambiguous or mislabeled instances. We validate its utility on various image and text classification tasks.

heading: 1 INTRODUCTION
text: In modern machine learning, large-scale data has become indispensable (Russakovsky et al., 2015; Wang et al., 2019a). A prevalent approach to collecting large-scale labeled datasets is to use imperfect sources such as crowdsourcing and web crawling (Fergus et al., 2005; Schroff et al., 2010; Wang et al., 2019a), which is usually less expensive and time-consuming than manual annotation by domain experts. However, such methods inevitably introduce label noise that may lead to overfitting and hurt the generalization of deep models (Arpit et al., 2017; Zhang et al., 2017).
In such situations, it is often beneficial to (i) remove mislabeled data or abstain from using confusing instances (Hara et al., 2019; Thulasidasan et al., 2019); (ii) increase robustness and reduce harmful influences of noisy labels (Malach & Shalev-Shwartz, 2017; Mirzasoleiman et al., 2020; Liu et al., 2020); or (iii) explicitly model the transition from the unobservable true label to the noisy observation (Goldberger & Ben-Reuven, 2017; Patrini et al., 2017; Xia et al., 2020). In this work, we focus on explicit modeling of the label corruption process, which is model-agnostic and data-efficient.
Most existing studies in this direction employ the class-conditional noise (CCN) assumption, i.e., the noisy label is independent of the input feature given the true label (Angluin & Laird, 1988; Natarajan et al., 2013; Patrini et al., 2017). However, this assumption could be too strong to fit some real-world data well (Xiao et al., 2015; Chen et al., 2021; Liu, 2021). More importantly, CCN only captures the general label flipping patterns between classes for all instances. In applications such as data cleansing and human-in-the-loop interaction, instance-specific noise information itself could be of central interest. This urges us to consider not only the class-conditional noise pattern but also the instance-specific noise modeling.
To handle this problem, in this work, we study the instance-dependent noise (IDN) model, where the noisy label also depends on the input. Several methods have been reported in the literature, but they either only focus on binary classification under strong assumptions (Menon et al., 2018; Cheng et al., 2020), are based on domain-specific knowledge (Xia et al., 2020), or need extra supervision (Berthon et al., 2021). In contrast, we propose a simple domain-agnostic approximation method for the multiclass IDN model, referred to as instance-confidence embedding (ICE). Concretely, to avoid estimating the noise transition matrix for each instance, we propose a variational approximation that
uses a scalar confidence parameter (Section 3.2). Then, we suggest using instance embedding that assigns a trainable parameter to each instance because the mapping from the instance to its confidence value could be non-smooth and is usually not required to generalize to unseen examples (Section 3.3). Lastly, we show the effectiveness of the proposed method and its ability to detect ambiguous or mislabeled instances through experiments on various image and text classification tasks (Section 5).

heading: 2 PROBLEM: INSTANCE-DEPENDENT NOISE
text: In this section, we give a brief overview of learning with instance-dependent noise (IDN).

heading: 2.1 NOTATION
text: Consider a K-class classification problem, where X ∈ X is the input feature and Y ∈ {1, . . . ,K} is the unobservable categorical true label. We assume that the clean class-posterior p(Y |X) comes from a parametric family of distributions:
pφ(Y |X) := Categorical(Y |p = f(X;φ)), (1) where p ∈ ∆K−1 is the probability parameter for Y in the (K − 1)-dimensional probability simplex ∆K−1, and f : X → ∆K−1 is a differentiable function parameterized by φ that maps the feature X to its corresponding probability parameter p. Then, let Ỹ ∈ {1, . . . ,K} be the noisy label. The goal is to predict Y from X based on a finite i.i.d. sample of (X, Ỹ )-pairs.

heading: 2.2 DEPENDENCE
text: Next, we introduce the dependence structure between X , Y , and Ỹ , which characterize different noise models. The graphical representations of noise models are illustrated in Fig. 1.
In IDN, we assume that the joint distribution of X , Y , and Ỹ can be factorized as follows:
p(X,Y, Ỹ ) = p(Ỹ |Y,X)pφ(Y |X)p(X). (2) That is, the noisy label Ỹ depends on both the instance X and the true label Y . Then, the noisy class-posterior p(Ỹ |X) can be obtained by marginalizing p(Y, Ỹ |X) over Y :
pφ(Ỹ |X) := Categorical(Ỹ |q = ∑K Y=1 p(Ỹ |Y,X)pφ(Y |X)), (3)
where q ∈ ∆K−1 denotes the probability parameter for Ỹ . Note that p(Ỹ |Y,X) plays a central role in IDN. Since both Y and Ỹ are categorical random variables, for a certain instance x, p(Ỹ |Y,X = x) can be seen as a K ×K stochastic matrix T (x), whose elements are Tij(x) := p(Ỹ = j|Y = i,X = x) for i, j ∈ {1, . . . ,K}. Conventionally, T (x) is called a noise transition matrix (Patrini et al., 2017). Then, p(Ỹ |Y,X) can be regarded as a matrix-valued function T : X → [0, 1]K×K that maps each instance x to its corresponding IDN transition matrix T (x). Without any restriction, we need K ×K parameters for each instance x.

heading: 2.3 APPROACH
text: Owing to its complexity, IDN has only been studied to a limited extent but is of great interest recently. A straightforward method is to jointly estimate the matrix-valued function T (x) as well as the clean class-posterior pφ(Y |X) using neural networks (Goldberger & Ben-Reuven, 2017). However, the estimation error of T (x) could be high, which deteriorates the classification performance. Another direction is to restrict the problem under certain conditions, so that we can provide theoretical guarantees (Menon et al., 2018; Cheng et al., 2020). However, existing work mainly focused on binary classification.
A promising approach is to approximate IDN using a simpler dependence structure, such as a mixture of noises with different semantic meanings (Xiao et al., 2015) or a weighted combination of noises that depend on parts of the instance (Xia et al., 2020). In this work, we also suggest that it might be unnecessary to obtain a K ×K matrix for each instance x: Note that pφ(Ỹ |x) can be seen as a linear combination of columns of T (x) weighted by pφ(Y |x); If the maximum value of pφ(Y |x) is close to 1, i.e., the label of the instance x is almost deterministic, the estimation of K − 1 columns of T (x) has only limited influence on the estimated noisy class-posterior p̂(Ỹ |x). This suggests the possibility of using a relatively simple model to approximate p(Ỹ |X) in real-world applications. In this work, we consider a single-parameter approximation for each instance, which is introduced in Section 3.2 and illustrated in Fig. 2.
Another issue is that existing methods still introduce some level of smoothness w.r.t. x into T (x) (Goldberger & Ben-Reuven, 2017; Xiao et al., 2015; Xia et al., 2020). In real-world problems, however, we can only access a finite sample of (X, Ỹ )-pairs that are possibly annotated by nonexperts or web crawlers (Fergus et al., 2005). Thus, the label noise could be aleatory and T (x) could vary significantly for two adjacent instances. Also, the classifier pφ(Y |X) is desired but the generalization of T (x) to unseen examples is usually dispensable. This inspires us to use instance embedding instead of neural network approximation, which is discussed in Section 3.3 and demonstrated in Fig. 3.

heading: 3 PROPOSED METHOD
text: In this section, we present our proposed method, instance-confidence embedding (ICE).

heading: 3.1 VARIATIONAL LOWER BOUND
text: Note that T (x) serves as a linear mapping from p to q (Eq. (3)). Due to the difficulty of estimating the matrix-valued function T (x), we use a simpler function qθ,φ(Ỹ |X) parameterized by θ as a variational approximation to pφ(Ỹ |X). The choice of the approximation family is discussed in Section 3.2.
Then, let us consider the expected log-likelihood as the learning objective, which can be rewritten as
E Ỹ∼p(Ỹ |X) [log p(Ỹ |X)] = DKL(pφ(Ỹ |X) ‖ qθ,φ(Ỹ |X)) + L(θ, φ;X), (4)
where DKL denotes the Kullback-Leibler (KL) divergence, and the second term is
L(θ, φ;X) := E Ỹ∼p(Ỹ |X) [log qθ,φ(Ỹ |X)]. (5)
Since the KL-divergence is always non-negative, this term gives a variational lower bound of the expected log-likelihood. Then, we have the following learning objective to maximize:
L(θ, φ) := E X∼p(X) [L(θ, φ;X)] = E X,Ỹ∼p(X,Ỹ ) [log qθ,φ(Ỹ |X)]. (6)
In practice, the expectation can be approximated using the empirical distribution based on a finite i.i.d. sample of (X, Ỹ )-pairs.
Clean Linear interpolation Power transformation Noisy

heading: 3.2 VARIATIONAL APPROXIMATION
text: Next, we discuss the choice of the variational approximation family of qθ,φ(Ỹ |X). To approximate the effect of multiplying an IDN transition matrix T (x) that requires K × K parameters for each instance x, in this work, we use a simpler transformation from p to q, which is not necessarily linear. Compared with estimating a full matrix for each instance without any restriction (Goldberger & Ben-Reuven, 2017), obtaining only an approximation may cause higher approximation error, but on the other hand, it may reduce estimation error and thus improve the classification performance. The high estimation error of complex models might be more harmful, which is empirically validated in Section 5. It is also the case when using CCN as an approximation of IDN to balance this trade-off. The difference is that CCN obtains a complete transition matrix common to all instances, but ICE aims to obtain an approximated trend for each instance, which gives useful instance-specific noise information.
Then, we suggest to use a single-scalar parameter C ∈ [0, 1] for each instance to control this approximation, which is useful for sorting and comparing training examples. This design might also be useful in data cleansing, learning with rejection, or active learning (Hara et al., 2019; Thulasidasan et al., 2019; Charoenphakdee et al., 2021). This parameter is referred to as the confidence and is obtained via a function g : X → [0, 1] parameterized by θ, i.e., C = g(X; θ). Then, θ can be regarded as the collection of C for all instances. The confidence C plays a central role in our method, where C = 0 means that the instance is ambiguous or mislabeled and thus the classifier should not give a confident prediction.
Finally, we need to design a transformation from p to q parameterized by the confidence C. We denote this function by h : ∆K−1 → ∆K−1. In summary, qφ,θ(Ỹ |X) takes the following form:
qφ,θ(Ỹ |X) := Categorical(Ỹ |q = h(f(X;φ); g(X; θ))). (7)
Next, we analyze what characteristics h needs to have. First, we suggest that h needs not necessarily to be a linear transformation because the transformation is instance-dependent and any function that maps p to q as close as possible for a certain instance x would suffice. Second, we require that arg max(p) = arg max(q), i.e., q = h(p;C) should be an argmax-preserving function so that the top-1 index of the probability vector does not change. This is because h should only affect the confidence of the prediction, not the final decision. Otherwise, if h is too flexible and is able to map a confident prediction to a different confident prediction, then the output of f could be arbitrary, and consequently, no information of the true label can be learned from the noisy label supervision.
Based on this motivation and the aforementioned semantics of C, we require that h(p; 1) = p and h(p; 0) = u, where u ∈ ∆K−1 is the uniform probability vector (ui = 1K ). Then, when the confidence C is high, the classifier gives a prediction closer to the original confident prediction p; and when the confidence C is low, the classifier tends to give a random guess u.
Here, we propose two functions for h that satisfy the above conditions: qi = Cpi + (1− C)ui, (linear interpolation) (8)
qi = pCi∑K j=1 p C j ∝ pCi , (power transformation) (9)
for i = 1, . . . ,K. The visualization of these two transformations for K = 3 is given in Fig. 2.

heading: 3.3 INSTANCE EMBEDDING
text: The last piece of our method is the choice of g : X → [0, 1], the function that maps the instance x to its confidence value C. It is also possible to use a neural network to approximate this function. However, because we usually only have a limited number of training examples and g could be non-smooth w.r.t. its input x, g may not be well approximated by a neural network with similar complexity to the classifier f , which is illustrated in an example in Fig. 3. Further, g may be rarely needed after training so its generalization ability is not required in many cases.
Based on these facts, we propose to use instance embedding, i.e., to assign a trainable parameter to each instance x. In other words, the only feature for an instance we use is its index in the training dataset. In this way, g is expressive and flexible but cannot be used for predicting the confidence of unseen instances. Accordingly, for a training dataset of size N , we need N parameters for a one-dimensional instance embedding.
This seems to be a high additional computational cost when the dataset size is large, but it is often acceptable, because (i) in modern deep learning, it is common to use over-parameterized models (Nakkiran et al., 2019), and the number of instances is usually not as large as the number of parameters of the classifier f (e.g., CIFAR-10 (Krizhevsky, 2009): 5× 104, ResNet-18 (He et al., 2016): ∼ 1× 107); and (ii) the gradient of the instance embedding is sparse and only a small subset of parameters needs to be updated at each iteration.
The idea of associating an entity with a scalar or vector embedding using a simple lookup table with a fixed dictionary size has been widely used in natural language processing (Mikolov et al., 2013; Pennington et al., 2014; Peters et al., 2018; Devlin et al., 2019) due to the discrete nature of tokens, and can be seen recently in contrastive learning (Wu et al., 2018; He et al., 2020) for vision tasks. Instance embedding enables the function to take any possible value on all observed instances but cannot generalize to any unseen token or image.
Combining the components introduced above, the learning objective Eq. (6) is in the following form:
L(θ, φ) = cross-entropy( q, fixed transformation, Eqs. (8) and (9)︷ ︸︸ ︷ h( f(X;φ)︸ ︷︷ ︸
p, neural network classifier, Eq. (1) ; g(X; θ)︸ ︷︷ ︸ C, instance-confidence embedding ), Ỹ ).
We also provide code snippets in Appendix A for practitioners and gradient analysis in Appendix B for theoretical implications.

heading: 4 RELATED WORK
text: In this section, we review related problem settings and methods.
Class-conditional noise (CCN). Compared with the IDN model, the instance-independent and class-conditional noise (CCN) model has an additional assumption: p(Ỹ |Y,X) = p(Ỹ |Y ), i.e., the noisy label Ỹ only depends on the true label Y . CCN has been well studied in both binary (Angluin & Laird, 1988; Long & Servedio, 2010; Natarajan et al., 2013; Van Rooyen et al., 2015; Liu & Tao, 2015) and multiclass (Patrini et al., 2017; Xia et al., 2019; Yao et al., 2020; Zhang et al., 2021b; Zhu et al., 2021) classification. Also, robust loss functions (Ghosh et al., 2017; Zhang & Sabuncu, 2018; Wang et al., 2019b; Charoenphakdee et al., 2019; Ma et al., 2020; Feng et al., 2020; Lyu & Tsang, 2020; Liu & Guo, 2020) have been mainly developed under the CCN setting. In practice, CCN methods can serve as practical approximations of IDN but the assumption could be too strong to fit some real-world data well (Xiao et al., 2015).
Label smoothing. Note that Eq. (8) is similar to the label smoothing (LS) technique (Szegedy et al., 2016; Pereyra et al., 2017; Lukasik et al., 2020), where the empirical distribution is linearly interpolated with a uniform distribution with a fixed mixing parameter. It is also related to the soft/hard bootstrapping loss (Reed et al., 2015), where the observed label is mixed with the predicted probability/predicted label. In contrast, in our method, it is the prediction p that is “smoothed”, not the label. We elucidate their relations and differences in Appendix C.
Temperature scaling. If we use softmax as the final layer of the neural network for pφ(Y |X), the proposed method is closely related to the temperature scaling (TS) technique (Guo et al., 2017). Concretely, if pi ∝ exp{fi(X;φ)} for i = 1, . . . ,K, then Eq. (9) becomes
qi ∝ exp{Cfi(X;φ)}, (10) which shows that C is the reciprocal of the temperature. The difference is that the parameter C is instance-dependent in our formulation, rather than being fixed for all instances. Also, TS (Guo et al., 2017) and its extensions (Kull et al., 2019; Rahimi et al., 2020) have been mainly used as post-hoc confidence calibration methods, while our method is used during training.
Sample selection. In a broader sense, the proposed method belongs to a category of methods that treat training examples differently in order to reduce the harmful effects of mislabeled instances. Besides the class-posteriors that our method uses, these methods exploit the training dynamics, loss characteristics, gradient information, or information of data itself from various perspectives. Examples include data cleansing (Liu et al., 2008; Northcutt et al., 2019; Hara et al., 2019) that first removes harmful instances and then (re-)trains the model on the remaining subset; dynamic training sample selection (Malach & Shalev-Shwartz, 2017; Jiang et al., 2018; Han et al., 2018; Wang et al., 2018; Yu et al., 2019; Mirzasoleiman et al., 2020; Wu et al., 2020; Chen et al., 2021; Cheng et al., 2021; Zhang et al., 2021a) that selects training examples dynamically during training; training techniques (Menon et al., 2020; Liu et al., 2020) that are designed to increase robustness and avoid memorization of noisy labels; learning with rejection or selective classification (El-Yaniv & Wiener, 2010; Thulasidasan et al., 2019; Mozannar & Sontag, 2020; Charoenphakdee et al., 2021) that abstains from using confusing instances while improving the classification performance on accepted instances; and semi-supervised learning (Nguyen et al., 2020; Li et al., 2020) that exploits unlabeled data. We discuss the relationship with some of them in more detail in Appendix D.
In the same spirit, our proposed method also attempts to detect harmful instances and reduce their influences automatically so as to improve the robustness of the class-posterior estimation. However, unlike explicit sample selection methods, the resulting algorithm is lightweight and has a low computational cost. Also, because the proposed method only affects the class-posterior, it is usually compatible with other training methods. Thus, the proposed method can be used alone or integrated into an existing training pipeline to further improve the performance.

heading: 5 EXPERIMENTS
text: In this section, we experimentally verify if the proposed instance-confidence embedding (ICE) method is able to differentiate mislabeled instances from correct ones and consequently improve the classification performance. We also demonstrate that there already exist ambiguous or mislabeled training examples in the original datasets which can be detected by the proposed method.

heading: 5.1 IMAGE CLASSIFICATION
text: Datasets. We evaluated our method on six image classification datasets, namely MNIST (LeCun et al., 1998), Fashion-MNIST (FMNIST) (Xiao et al., 2017), Kuzushiji-MNIST (KMNIST) (Clanuwat et al., 2018), SVHN (Netzer et al., 2011), CIFAR-10, and CIFAR-100 (Krizhevsky, 2009) datasets. We used a method similar to the one used in Zhang et al. (2021a) to generate instancedependent label noises whose overall noise rate is around 52%. See Appendix E for details and more experimental results.
Methods. We compared the following ten methods: (1) (CCE) categorical cross-entropy loss; (2) (Bootstrapping) (hard) bootstrapping loss (Reed et al., 2015) that regularizes the output with the predicted label; (3) (Adaptation) noise adaptation layer (Goldberger & Ben-Reuven, 2017) that estimates a full K ×K transition matrix for each instance; (4) (Forward) forward correction (Patrini et al., 2017) that estimates a transition matrix for all instances; (5) (Dual-T) dual-T estimator (Yao et al., 2020) that uses the normalized confusion matrix to correct the transition matrix; (6) (DAC) deep abstaining classifier (Thulasidasan et al., 2019) that uses abstention for robust learning; (7) (GCE) generalized cross-entropy loss (Zhang & Sabuncu, 2018) as a robust loss; (8) (PLC) progressive label correction (Zhang et al., 2021a); (9) (ICE-LIN) instance-confidence embedding with the linear interpolation (Eq. (8)); and (10) (ICE-POW) the one with the power transformation (Eq. (9)). For a fair comparison, we implemented aforementioned methods using the same network architecture and hyperparameters.
Models. For MNIST, FMNIST, and KMNIST, we used a sequential convolutional neural network (CNN) and an Adam optimizer (Kingma & Ba, 2015). For SVHN, CIFAR-10 and CIFAR-100, we used a residual network model ResNet-18 (He et al., 2016) and a stochastic gradient descent (SGD) optimizer with momentum (Sutskever et al., 2013). Hyperparameters are provided in Appendix E.
Improving classification performance. To verify if the proposed method is able to improve the classification performance under label noise, we constructed semi-synthetic noisy datasets so that the true labels are known. We regarded the original labels as clean labels, although as will be shown in the next experiment, label errors already exist in the original datasets to some extent. We ran 10 trials and reported the means and standard deviations of the test accuracy in Table 1.
We can observe that the proposed method generally outperforms the baseline methods. It is worth noting that when there are a large number of classes (e.g., CIFAR-100), estimating a full transition matrix for each instance (Adaptation) may deteriorate drastically because it requires an additional K × K output and the estimation error could be high. On the contrary, the complexity of our single-parameter approximation does not increase as the number of classes increases. Additionally, in the ridgeline plots in Fig. 4, we can observe the separation of instances with flipped/original labels using the learned confidence C, which may explain the performance improvement.
Detecting ambiguous/mislabeled instances. Next, we demonstrate that the proposed method can be used for detecting ambiguous or possibly mislabeled instances. We trained the model on the original datasets with the proposed method. A benefit of using a single-parameter approximation is that it naturally derives an order of the training examples. We sorted the training examples via the confidence and showed the 32 most low-confidence ones in Fig. 5. We also extracted high/low-confidence training examples for each classes as shown in Fig. 6. Results of other datasets are provided in Appendix E.
We can observe that, surprisingly, in these supposedly clean datasets, a number of instance might be mislabeled. In MNIST and SVHN, we found clearly mislabeled images. There are ambiguous images such as 2-7 and 4-9 pairs in MNIST and shirt/T-shirt/pullover/coat photos in Fashion-MNIST. In CIFAR-10, it is interesting that images in the animal category are more likely to have a low confidence. We conjecture that the spurious correlation between the object and the background color plays an important role. We also found multi-modality issues, e.g., kiwi, owl, and chicken are all labeled as bird but are not visually prototypical birds. This phenomenon suggests the possibility of using the proposed method for diagnosing label issues in large-scale datasets.
Further, in the class-wise spectra of training examples (Fig. 6), we can observe that the confidence values may capture the level of ambiguity of instances. This suggests the possibility of using the proposed method in other tasks such as data cleansing, learning with rejection, and active learning.

heading: 5.2 TEXT CLASSIFICATION
text: We discovered that noisy label issues also exist in text datasets. We conducted similar experiments on the GLUE benchmark (Wang et al., 2019a), which is a collection of datasets for natural language understanding. We trained a BERT-base model pretrained using a masked language modeling (MLM) objective (Devlin et al., 2019) with a default AdamW optimizer (Loshchilov & Hutter, 2017). The performance in terms of the suggested evaluation metric was reported in Table 2.
We can observe that, except on the RTE dataset, the performance was improved or approximately the same compared with the default CCE method, which shows the benefits of using instance-specifically adjusted confidences. Although, if the dataset is relatively clean, the improvement might be marginal.
We also found mislabeled or ambiguous instances in these datasets. A typical example is the Corpus of Linguistic Acceptability (CoLA) (Warstadt et al., 2019) dataset, which consists of English grammatical acceptability judgments. Six selected low-confidence training examples are given in Table 3. We found that several ungrammatical sentences were mislabeled as acceptable, and some syntactically acceptable sentences were labeled as unacceptable by annotators possibly because they have semantic errors. In this way, we may use the proposed method to probe if the model prediction is consistent with our intent. More results are provided in Appendix E.

heading: 6 CONCLUSION
text: We have introduced a novel variational approximation of the instance-dependent noise (IDN) model, referred to as instance-confidence embedding (ICE). Compared with existing methods based on the class-conditional noise (CCN) assumption, the proposed method is able to capture instance-specific noise information and consequently improve the classification performance. The use of the onedimensional instance embedding naturally derives an order of training examples which can be used for detecting ambiguous or mislabeled instances. For future directions, it is interesting to explore its combination with other training techniques and its extensions in data cleansing, learning with rejection, or active learning.

heading: B GRADIENT ANALYSIS
text: In this section, we provide a basic gradient analysis and visualization for our proposed method.
The gradients of the log-likelihood using Eqs. (8) and (9) are
∂
∂C log(qi) = pi − 1K Cpi + (1− C) 1K , (linear interpolation) (11)
∂
∂C log(qi) =
K∑
j=1
pCj log pi pj , (power transformation) (12)
respectively. Their sign boundaries are pi = 1
K and pi = e−H(q,p), respectively, where H(·, ·)
denotes the cross-entropy. We can find that for the linear interpolation (Eq. (8)), when pỹ < 1
K ,
∂L ∂C < 0 and when pỹ > 1 K , ∂L ∂C > 0. Similarly, for the power transformation (Eq. (9)), when
pỹ < e −H(q,p),
∂L ∂C < 0 and when pỹ > e−H(q,p), ∂L ∂C > 0.
The contours of the likelihood for different parameters are plotted in Figs. 7 and 8. Note that the classposterior p is obtained from a neural network, so it can be influenced by other instances, especially adjacent instances. On the other hand, the confidence C is obtained via instance embedding, so it can take any value independently. If the predicted class-posterior pỹ for an instance x is low (e.g., because this instance is mislabeled and the majority of adjacent instances are predicted to belong to other classes), then the classifier tends to decrease its confidence value so as not to overfit this possibly mislabeled instance. The gradient magnitude is the largest when C is high and pỹ is low (confident wrong prediction), the smallest when both C and pỹ are high (confident correct prediction), and in the middle when C is low (uncertain prediction like a random guess). In this way, we can equip the neural network model with an option of changing the confidence of prediction for individual training examples to mitigate overfitting possibly mislabeled instances.
C LINEAR INTERPOLATION
Linear interpolation between some properties of an instance and other value is a widely used technique for regularization in machine learning, such as the bootstrapping loss Reed et al. (2015) and the label smoothing technique (Szegedy et al., 2016; Pereyra et al., 2017; Lukasik et al., 2020). In this section, we briefly summarize related techniques and compare their differences.
Concretely, let p be the predicted probability vector for Y (Eq. (1)), y be the one-hot vector for the observed label, ŷ = arg maxp is the one-hot vector for the predicted label, u be the uniform probability vector (ui = 1
K for i ∈ {1, . . . ,K}). Here, p,y, ŷ,u ∈ ∆K−1 are all in the probability
simplex. Let C ∈ [0, 1] be a scalar linear interpolation parameter. Then, as also shown in Fig. 9, the learning objectives are equivalent to the following KL-divergences:
DKL(y ‖ p), (categorical cross-entropy) (13) DKL(Cy + (1− C)u ‖ p), (label smoothing) (14) DKL(Cy + (1− C)p ‖ p), (soft bootstrapping loss) (15) DKL(Cy + (1− C)ŷ ‖ p), (hard bootstrapping loss) (16) DKL(y ‖ Cp + (1− C)u). (instance-confidence embedding) (17)
We can see that the label smoothing and the bootstrapping loss methods smooth the target, but the proposed ICE method smooths the prediction. Note that it is impossible to let C be an instancedependent parameter in other methods, because when C = 0, the supervision signal Y can be completely lost.
Another technique using linear interpolation is mixup (Zhang et al., 2018), which also interpolates the input features X between two instances. Therefore its characteristics could be more different than the methods mentioned above.

heading: D SAMPLE SELECTION
text: Our method can be regarded as a “soft” sample selection method that de-emphasizes the supervision from possibly mislabeled training examples to some extent. In the literature, several “hard” sample selection methods have been reported recently, e.g., Cheng et al. (2021); Zhang et al. (2021a). In this section, we discuss their relationship and the pros and cons of these methods.
Soft vs. hard. Hard sample selection methods usually use a predefined thresholding rule based on the regularized loss [(Cheng et al., 2021), Eq. (4)] or the predicted probability [(Zhang et al., 2021a), Section 2.2] to select a subset of possibly correct training examples at each iteration. The threshold is either determined dynamically (Cheng et al., 2021) or decreased via a manually designed schedule (Zhang et al., 2021a). Optionally, possibly incorrect labels can be flipped to the current model predictions (Zhang et al., 2021a).
We can find that such “hard” sample selection methods have two options for a training example: it is either completely reliable or not reliable at all. However, since mislabeled instances are more likely to be near the decision boundary, such hard decision rules may result in unstable training. To cope with this issue, they often requires the confidence regularization, warm-up training, or a manually designed threshold decay schedule, which introduces more hyperparameters such as the regularization weight, the number of warm-up epochs, and the threshold schedule.
In contrast, in our method, the confidence C – the “soft” threshold — is continuous in [0, 1] and is updated smoothly using the current prediction and gradient information so the training may be more stable. We do not need to introduce other hyperparameters. Although we do need to specify an optimizer for it, we found that a simple SGD optimizer works well enough in experiments.
Another benefit of using continuous weights [0, 1] instead of discrete selections {0, 1} is that we can naturally derive a total order of training examples, which can be used for other tasks.
Gradient-based optimization vs. optimal solution. Next, let us focus on the soft sample selection. Note that the confidence C might have an optimal solution and can be obtained directly. However, it is only optimal given the current predicted probability p. If the model is still under training and the estimation of p is not good enough yet, it is not always beneficial to give a definite prediction of C, which may cancel out the advantage of the soft sample selection.
More specifically, to give a prediction of C, we have the following choices. Their characteristics are also listed below.
1. using a neural network • generalizes to unseen instances • depends on the trajectory of training, gradient-based optimization (heavy) • time: smooth, space: smooth
2. using instance embedding • does not generalize to unseen instances • depends on the trajectory of training, gradient-based optimization (light) • time: smooth, space: non-smooth
3. using a predefined rule based on the probability • generalizes to unseen instances (via the classifier) • does not depend on the trajectory of training, no optimization • time: smooth (soft)/non-smooth (hard), space: non-smooth
Here, “smoothness” means if the values change gradually during training (time) and if adjacent instances tend to have similar values (space). We can use gradient-based optimization for instance embedding so it is smooth in time (for stable training) and non-smooth in space (for instancedependent noise). Mathematically, the proposed method takes accounts of not only gradient direction but also gradient magnitude. Intuitively, recording previous confidences C allows the model to have “memory” so C is not solely decided by the current prediction. For the above reasons, we prefer soft sample selection and gradient-based optimization.

heading: E EXPERIMENTS
text: E.1 IMAGE CLASSIFICATION
Data. We used the MNIST,1 Fashion-MNIST,2 Kuzushiji-MNIST,3 SVHN,4 CIFAR-10, and CIFAR-1005 datasets. The MNIST, Fashion-MNIST, Kuzushiji-MNIST datasets contain 28 × 28 grayscale images in 10 classes. The size of the training set is 60000 and the size of the test set is 10000. The SVHN dataset contains 32 × 32 colour images in 10 classes. The size of the training set is 73257 and the size of the test set is 26032. The CIFAR-10 and CIFAR-100 datasets contain 32× 32 colour images in 10 classes and in 100 classes, respectively. The size of the training set is 50000 and the size of the test set is 10000. We used 20% of the training sets for validation. We added synthetic label noise into the training and validation sets. We used a method similar to Zhang et al. (2021a) (hybrid noise with 20% Type-I + 40% Uniform) to generate synthetic instance-dependent noise. Concretely, we first approximated the clean class-posterior using the original clean datasets, and corrupted the label from the most confident prediction to the second most confident one. The overall noise rate was around 52%, depending on the datasets. The test sets were not modified.
Models. For MNIST, Fashion-MNIST, and Kuzushiji-MNIST, we used a sequential convolutional neural network with the following structure: Conv2d(channel=32) ×2, Conv2d(channel=64) ×2, MaxPool2d(size=2), Linear(dim=128), Dropout(p=0.5), Linear(dim=10). The kernel size of convolutional layers is 3, and rectified linear unit (ReLU) is applied after the convolutional layers and linear layers except the last one. For SVHN, CIFAR-10 and CIFAR-100, we used a ResNet-18 model (He et al., 2016). To ensure that C ∈ [0, 1], we simply apply the sigmoid function that maps R to [0, 1] to the embedding.
Optimization. For MNIST, Fashion-MNIST, and Kuzushiji-MNIST, we used an Adam optimizer (Kingma & Ba, 2015) with batch size of 512 and learning rate of 1× 10−3. The model was trained for 2000 iterations (17.07 epochs) and the learning rate decayed exponentially to 1× 10−4. For CIFAR-10 and CIFAR-100, we used a stochastic gradient descent (SGD) optimizer with batch size of 512, momentum of 0.9, and weight decay of 1× 10−4. The learning rate increased from 0 to 0.1 linearly for 400 iterations and decreased to 0 linearly for 3600 iterations (4000 iterations/40.96 epochs in total). For SVHN, the setting was the same as CIFAR-10 except the model was trained for 1000 iterations.
Results. The ridgeline plots of the confidence C during training are given in Fig. 10. The densities seem bimodal in the early stage for MNIST, FMNIST, and KMNIST because we did not log the result at the end of each epoch but based on the number of batch iterations. Therefore, the confidence of some instances may have been updated more times than it of the others. In the final stage, the confidence also converges and instances with flipped/original labels are almost separated by the learned confidence. Low-confidence training examples are given in Figs. 11 and 13, which are partially presented in Fig. 5. The class-wise spectra are given in Fig. 12, which are partially presented in Fig. 6.
E.2 TEXT CLASSIFICATION
We implemented the BERT-base model (Devlin et al., 2019) using PyTorch (Paszke et al., 2019) and HuggingFace’s transformers (Wolf et al., 2020) libraries. We used a pretrained model6 and an AdamW optimizer (Loshchilov & Hutter, 2017). The batch size was 32 and the weight decay
1 MNIST (LeCun et al., 1998) http://yann.lecun.com/exdb/mnist/ 2 Fashion-MNIST (Xiao et al., 2017) https://github.com/zalandoresearch/ fashion-mnist (MIT license) 3 Kuzushiji-MNIST (Clanuwat et al., 2018) http://codh.rois.ac.jp/kmnist/ (CC BY-SA 4.0 license) 4 SVHN (Netzer et al., 2011) http://ufldl.stanford.edu/housenumbers 5 CIFAR-10, CIFAR-100 (Krizhevsky, 2009) https://www.cs.toronto.edu/~kriz/cifar. html 6 bert-base-cased: https://huggingface.co/bert-base-cased
was 0.01, otherwise we used the default hyperparameters. The model was trained on 4 NVIDIA Tesla P100 GPUs in parallel with the mixed precision training option (fp16) enabled. For the CoLA, MRPC, RTE, and WNLI datasets, the model was trained for 5 epochs and otherwise 3 epochs. Low-confidence training examples are given in Tables 6 to 13.
E.3 MORE EXPERIMENTAL RESULTS
We also conducted the experiment on a common setting where the labels are randomly flipped, i.e., instance-independently. The noise rate was 50%. The test accuracy is listed in Table 4.
Further, we tested the proposed method on the Clothing1M dataset (Xiao et al., 2015), which is a real-world noisy label dataset. We followed the convention and trained a ResNet-50 model (He et al., 2016) only on the 1M noisy training set for 10 epochs. On this dataset, ICE-POW achieved the accuracy of 72.67%, which is comparable or superior to some existing works. For reference, the reported performances of other methods including CCE, Forward, T-Revision (Xia et al., 2019), dual-T (Yao et al., 2020), PTD (Xia et al., 2020), DMI (Xu et al., 2019), CORES2 (Cheng et al., 2021), ILFC (Berthon et al., 2021), and PLC (Zhang et al., 2021a), are cited in Table 5.
Note that some methods are superior to ours in terms of accuracy on Clothing1M. This may be due to different evaluation settings and training techniques. Further investment is left for future work.
0
1
2
3
4
5
6
7
8
9
(a) MNIST
t-shirt
trouser
pullover
dress
coat
sandal
shirt
sneaker
bag
boot
(b) FMNIST
o
ki
su
tsu
na
ha
ma
ya
re
wo
(c) KMNIST
0
1
2
3
4
5
6
7
8
9
(d) SVHN

Review summaries:
This paper studies instance-dependent noise (IDN) problems. It proposes a single-parameter (confidence) model for the noise generating process. Moreover, *instance-confidence embedding* (ICE) method is employed in the training process. The experimental results on various image and text classification tasks confirm the effectiveness of the proposed method.

This paper presented an instance-confidence embedding model (ICE), which is a variational approximation of the instance-dependent noise (IDN). Given the observation that most columns of the IDN transition matrix have only limited influence on the class-posterior estimation, this paper uses a single-scalar confidence parameter to model the noise and uses a simpler transformation to transfer clean class posterior to noisy class posterior. The suggested method captures instance-specific noise information and thereby improves classification performance when compared to previous methods based on the class-conditional noise (CCN) assumption.

This paper introduces ICE, an instance-confidence embedding approach to learn with a challenging noisy label setting: instance-dependent noise (IDN). Considering the sparse property of IDN, the authors propose to use instance embedding by equipping a trainable parameter to each data instance. Experiments results on various image datasets and text data show the effectiveness of the proposed method with the presence of label noise.

This paper proposes ICE (Instance-confident embedding) method for learning with IDN (instance-dependent label noise). ICE approximates IDN by using a single-scalar confident parameter for each sample based on the assumption that $P(Y|x)$ is close to a one-hot vector. ICE is incorporated into the loss function for efficient learning process under label noise. Authors conduct experiments on both image and text classification to verify the effectiveness  of ICE and perform gradient analysis for further understanding the method.