Learning Imbalanced Data with Beneficial Label Noise
Authors: Guangzheng Hu
Feng Liu
Mingming Gong
Guanghui Wang
Liuhua Peng
Venue: ICML 2025 poster (2025)
Keywords: Imbalanced learning
beneficial label noise
classification accuracy
decision boundary
Primary area: general_machine_learning->supervised_learning

Abstract: Data imbalance is a common factor hindering classifier performance. Data-level approaches for imbalanced learning, such as resampling, often lead to information loss or generative errors. Building on theoretical studies of imbalance ratio in binary classification, it is found that adding suitable label noise can adjust biased decision boundaries and improve classifier performance. This paper proposes the Label-Noise-based Re-balancing (LNR) approach to solve imbalanced learning by employing a novel design of an asymmetric label noise model. In contrast to other data-level methods, LNR alleviates the issues of informative loss and generative errors and can be integrated seamlessly with any classifier or algorithm-level method. We validated the superiority of LNR on synthetic and real-world datasets. Our work opens a new avenue for imbalanced learning, highlighting the potential of beneficial label noise.

Learning Imbalanced Data with Beneficial Label Noise
Guangzheng Hu'
Feng Liu
Mingming Gong' Guanghui Wang Liuhua Peng
Abstract
In practice, the minority class, often representing rare anomalies, is the focus, making accuracy inadequate metric, especially in binary classification. Metrics like F1 score, G-mean, and Area Under the Curve (AUC) for bi- nary classification (He & Garcia, 2009; Krawczyk, 2016), or and Many/Medium/Few-shonn for step- wise or long-tailed multi-class data, better reflect minority class performance (Cao al., 2019; Wang et al., 2021a). This paper theoretically shows that optimizing for accu- racy leads to decision boundary in binary classification misaligned with metrics like the Fl score.
Data imbalance is common factor hindering classifier performance. Data-level approaches for imbalanced learning, such as resampling, of- ten lead to information loss or generative errors. Building on theoretical studies of imbalance ra- tio in binary classification, is found that adding suitable label noise adjust biased decision boundaries and improve classifier performance. This paper proposes the Label-Noise based Re- balancing (LNR) approach to solve imbalanced learning by employing a novel design of an asym- metric label noise model. In contrast to other data-level methods, LNR alleviates the issues of informative loss and generative errors and can be integrated seamlessly with any classifier algorithm-level method. We validated the su- periority of LNR on synthetic and real-world datasets. Our work opens new avenue for imbal- anced learning, highlighting the potential of ben- eficial label noise. Code is available at https: //git giggtuub.... com/ guangzhengh/ LLNR. LLR. git
Imbalanced learning approaches can be roughly classified into data-level and algorithm-level methods. Data-level methods, such as resampling, mitigate imbalance by adding or removing samples (Kaur et al., 2019; Napierala Ste- fanowski, 2016; Wang et al., 2020; 2021a) but may in- troduce information loss, overfitting, or generative errors. Algorithm-leve methods (Chawla et al., 2004; Cao et al., 2019; Wang et al., 2021b; b; Li et al., 2022b) adjust learning algorithms to account for imbalance but are often tailored to specific models or problem settings, limiting their general applicability.
This paper theoretically studies how the imbalance ratio (IR) biases decision boundaries in binary classification with dif- ferent metrics, such as accuracy and Fl score. Presentation or introduction of label noise, which flips sample labels to another class, can significantly alter classifier decision boundaries. Inspired by studies on harmless label noise (Ahfock & McLachlan, 2021; Cannings et al., 2020), which does not degrade classifier performance under certain condi- tions, propose novel Sabel-Noise-Re-balaneee approach that introduces artificial label noise to reallocate imbalanced class labels and thus de-biases the decision boundary to improve classifier performance. We further ex- tend LNR to multi-class classification with class imbalance. As a data-level method, LNR is model-agnostic and can be seamlessly integrated with any classifier algorithm-level approach. This flexibility enables improved performance across both binary and multi-class settings without requir- ing significant modifications to existing algorithms, therehy offering a robust solution to imbalanced learning.
1. Introduction
Imbalanced data is pervasive in many real-world problems, including fraud transactions (Phua et al., 2004), disease diagnoses (Fotouhi et al., 2019), and behavioral analysis (Azaria et al., 2014). These datasets are often characterized by a high disparity in class distribution, where some classes have abundant samples while others contain very few. Such imbalance challenges traditional classifiers, which naturally require an equal sample size across classes or a sufficient number of minority class samples for effective performance (He& & Garcia, 2009; Krawezyk, 2016).
(LNR)
'School Mathematics and Statistics, University Mel- bourne, Victoria, Australia "School of Computing and Infor- mation Systems, University of Melbourne, Victoria, Australia "School of Statistics and Data Science, LPMC, KLMDASR, and LEBPS, Nankai University. Correspondence to: Liuhua Peng <liuhua.peng und.e....duu.u.>.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Figure illustrates the effect of adding beneficial label noise to imbalanced MNIST data with minority classes "5", "7".

---

Learning Imbalanced Data with Beneficial Label Noise
1-SNE Realion Imbelenced MMST
Samples labels flipped to minority classes "5"(top), "7"(middle) and (9'(botmmm)
3
3
5
8
5
5
3
3
FRISS
7
3
2
7
7
2
7
7
3
2
9
7
8
9
8
9
4
(a) T-SNE visualizations.
(b) Top- 10 flipped labels into minority classes 5,7.9 by LNR.
Adding label noises to the imbalanced MNIST data. (a) shows the t-SNE visualization of the minority class (in squares) and the majority class (in dots). Noisy samples introduced by LNR (in crosses) are displayed in (b).
Figure 1:
and "9". The flipped labels correspond to the top 10 samples most likely to be relabeled as each minority class. In Figure (a), these samples are near class boundaries, exhibiting fea- tures similar minority class samples, as shown Figure (b). Applying LNR to extremely imbalanced MNIST dataset (30 samples per minority class, IR = 200) improved classifier accuracy from 89.93% to 94.75%.
der imbalance. For classifier optimization, margin-based approaches like LDAM-DRW (Cao et al., 2019) and 7-norm (Kang et al., 2019) employ loss engineering to create larger decision margins for tail classes, while logit adjustment methods such GCL (Li et al., 2022c) address softmax saturation by expanding the tail classes' embedding space through increased cloud sizes.
Data-level methods often leverage generative models or au- toencoders such as A-encoder (Schwartz et al., 2018), DGC (Wang et al., 2020), and RSG (Wang et al., 2021a) to synthe- size few-shot samples. These approaches typically depend high-quality pre-trained models, which introduce additional challenges, especially scarce tail-class data, limiting the ability to generate diverse or meaningful sam- ples. Instead of involving a generative model, Mixup (Zhang et al., 2017) interpolates features and labels via fixed mix- ing ratio, empirically demonstrating its effectiveness for data augmentation. Building on this, Remix (Chou et al., 2020) introduces separate mixing ratios for features and labels to rebalance class distributions, though it retains ran- dom sampling. More recently, SelMix (Ramasubramanian et al., 2024) advanced this direction by selectively sampling pairs for mixing based on the gain on non-decomposable metrics (e.g., recall, G-mean), thereby enabling targeted improvements in specific metrics. However, SelMix's 's gain matrix relies a balanced augmented auxiliary dataset, With imbalanced or small auxiliary data, the metric opti- mization fails to meet theoretical constraints. This limitation is notably critical in practice, where validation data is often scarce and inherently skewed-a gap our method addresses with a carefully designed noise model, which requires nei- ther feature editing nor a balanced auxiliary dataset.
2. Related Works and Contributions
Imbalance learning. imbalanced binary classification, resampling is one-shot data-level approach that balances sample sizes through under-sampling over-sampling. To mitigate information loss from under-sampling and over- fitting from over-sampling, various approaches have been proposed. Selective under-sampling methods, such as Tink (Tomek, 1976), ENN (Wilson & Martinez, 2000), NCL. (Li et al., 2022b), OSS (Kubat & Matwin, 2000), and ClusterCentroids (Lin al., 2017), remove sam- ples based distribution characteristics. Meanwhile, syn- thetic over-sampligg techniques, including SMOTE (Chawla et al., 2002), Borderline-MMOTE (Han et al., 2005), and ADASYN (He et al., 2008), generate new minority samples to better represent class distributions.
In multi-class classification, imbalance scenarios such as long-taile or step-wise sample size distributions become more complex due multiple minority classes and varying imbalance ratios. Algorithm-level methods refine loss func- tions or training paradigms to improve tail-class accuracy, primarily by decoupling feature learning from classifier training to separately enhance feature representation and classifier fine-tuning. Contrastive learning- methods, including DRO-LT (Samuel & Chechik, 2021). TSC (Li al., 2022d), BCL (Zhu al., 2022), and SBCL (Hou et al., 2023), leverage contrastive losses during feature learning to boost feature discriminability and model robustness un-
Multi-expert ensemble methods (e.g., RIDE (Wang al., 2021b), TLC (Li al., 2022a), and SADE (Zhang al., 2022)) allocate specialized "experts" to model head- and
2

---

Learning Imbalanced Data with Beneficial Label Noise
tail-class features separately, achieving notable gains. While these approaches fall outside the scope of our work, they highlight the ensemble learning for tackling class imbalance.
velopment of our Label-Noise- Re-balancing (LNR) approach. Assume that two imbal classes have prior probabilities मा = Pr(Y
1) and To = Pr(Y = 0)
- TT, respectively where Y = 1 refers to the minority (pos- itive) class while Y = 0 represents the majority (negative) class. In the imbalanced scenario, the sample class priors satisfy क < mo and IR = =TO/= For class c E (0. 1) and the sample space X c Rd, we denote the class-conditions distribution as X Y = c Pa and the marginal distribu- tion of X as Px
Label noise. mistance-dependent noise (Frénay & Ver- leysen, 2013), where the label flip-rate (or mislabel-rate) depends on its true class and instance features, is the most general case of label noise. It is commonly assumed that samples with similar features are more likely mislabeled (Goldberger & Ben-Reuven, 2022; Song al., 2022), par- ticularly in the class-overlapping region. While label noise is typically barmful, with most research focusing on mitigat- ing its impact (Goldberger& & Ben-Reuven, 2022; Liu & Tao, 2015; Patrini et al., 2017; Xiao et al., 2015; Zhu et al., 2003), recent studies (Ahfock& & McLachlan, 2021; Cannings et al., 2020) suggest that under certain conditions, label noise can be harmless even beneficial. However, both works pri- marily focus on balanced data with symmetric label noise and do not explore how to leverage label noise to improve classifier performance for imbalanced data.
Pi (xr) + mo (2)).
3.1. Biased optimal Bayesian decision boundary by IR
Define 77(x) = Pr(Y = 1 X = =x). The Bayesian deci- sion boundary of the optimal classifier that maximizes the accuracy is defined as S = fx E X': n7(x) = 0.5}.
Lemma 3.1. The optimal Bayesian decision boundary is S S= fa E X: Pi (x)) = To/ }.
If the data balanced with sg = Ii, Lemma 3.1 implies that the Bayesian decision boundary corresponds to the intersection points of Po(x)) and P (z)). However, for im- balanced data with TΙ < To, the decision boundary intrudes deeply into the minority class region, increasing the risk of erroneously classifying the minority class and leading to a diminished true positive rate approaching 0. In pursuit of op- timizing overall accuracy, the predictive ability of minority class samples, which crucial in practice, is compromised. In extremely imbalanced cases, the accuracy remains high even if all samples are classified into the majority class.
Motivation and contributions. Previous works (Ahfock & McLachlan, 2021; Cannings et al., 2020) on harmless label noise inspire the methodology reshape decision bound- aries through label noise under general models, extending from harmless to beneficial categories when addressing real- world imbalanced learning problems. It is worth mentioning that the label noise employed in this paper is asymmetric such that only the samples from the majority class have a certain probability of being flipped to the minority class, aligning with imbalanced learning. This contrasts with the symmetric label noise considered in Ahfock & McLachlan (2021); Cannings et al. (2020), where the minority samples may also be flipped to the majority class.
N(po, Σ) and P1 ~ N(u, E). S = fo E X : w I + 3 = In (TO/TI) with w = Σ-1( - Ho) and B = =0=-00
Example 1. If Po
po - 2 TUTE-I p1. For univariate normal distributions with common variance o², S fo E X x = } + Ho) + 11-Mo To Tı, the Bayesian decision boundary lies on the mid- point of μr and po- However. for imbalanced data with Ti < To, the decision boundary shifts towards the minority class mean μr as In(Mo/ T1) > 0. The term o² //111111111 Ho) determines the relevance of the imbalance ratio To/TI to the deviation of the decision boundary from the midpoint, which inversely proportional to fr Mu, the class mean difference indicating the degree of class overlap.
The main contributions of this paper are as follows: a) Theoretical justification of how imbalance ratio biases the decision boundary and quantification of its deviation un- der different evaluation metrics in binary classification. b) Definition of beneficial label noise model that corrects the biased classifier's decision boundary by harmlessly re- allocating imbalanced data labels. e) Development of a novel methodology (LNR) to introduce carefully selected beneficial label noise to imbalanced data, improving the per- formance of classifiers on predicting minority class samples in both binary and multi-class classification. Experiment results on synthetic and real-world data validated the superi- ority of our proposed method and its versatility in integrating with existing algorithm-level methods.
o² In( }- If
Lemma 3.1 is on the Bayesian decision boundary that max- imizes the population-level accuracy. Similar phenomena exist in many classical classification algorithms, such as the k-nearest neighbors (KNN). We refer to Section B in the Appendix for more detailed discussions.
3. Impact of IR on Decision Boundary
3.2. Impact of IR on F1 score
In this section, we analyze the impact of the imbalance ra- tio (IR) on decision boundaries across various evaluation metrics in binary classification, which motivates the de-
In many scientific applications, such as fraud detection, the predictive performance of minority class samples is often prioritized over overall accuracy. Precision and re-
3

---

Learning Imbalanced Data with Beneficial Label Noise
3-
(a) [11 - Ho| = 2.5, To/ m1
(b) 1111
1, To/ R1
(c) Jp - Hol = 1, To/
9
Figure 2: Comparison of decision boundaries of S (vertical black dashed line) and S (vertical red dashed line). The red solid line represents N2 P1 (I), and the black solid line represents ToPu(r). The black dashed curve represents the accuracy with respect to v, and the red dashed curve is FI (1)).
call, defined as Precision = TP/(TP + FP) and Recall = TP/(TP- FN), are two metrics that prioritize the posi- tive samples. Here, TP, FP, and FN represent true positives, false positives, and false negatives. Precision and recall are inversely proportional and constrained by each other. The Fl score combines precision and recall into single metric, and it is widely used in imbalanced learning:
The proof Lemma 3.2 implies 77(2)) = 0.5 x F1 for Zg E S iindictting that the probability of zg belongs to the positive class is half of the maximum population F1 score the decision boundary. Consequently, the optimal Fl score decision boundary coincides with the optimal Bayesian de- cision boundary when F1(zo) = =1 for zo E SPi, In such case, Pr(ý = Y, = 0) = Pr(ý = 0, Y = 1) = 0, indicating full separability between the two classes
2 Precision Recall
2TP
F1
However, when class overlap exists such that F1(xo) 1. observe P. (2r)) (200)Po(zz) < Pi (x)/ Pa(x)) for To E SPI and S. This shows that the optimal Fl score deci- sion boundary deviates from the optimal Bayesian decision boundary. The discrepancy between S and st"l is linked to Fi((z) 2-J1(xa) which becomes severe when the maximum Fl score decreases. In addition, this term is weighted by the imbalance ratio To/ further amplifying the deviation.
Precision + Recall
2TP + FP FN
For a given classifier, TP, FP, and EN are discrete statistics based on its decision boundary. Let N be the total sample size of two classes. By the law of large numbers, as N Pr(Y
Co, TP/N - Pr(Y = 1,Y = 1), FP/N Y = 0), and FN/N + Pr(Y 0, = 1) in probability, where Y is the predicted label of X with true label Y.
We consider the classifier in the form of y = I(wTT(X
Lemma 3.2 indicates that for S the maximum F1
mo) > 0) for some w E Rd and To E X, here II()) is the indicator function. It follows that Pr(y
score satisfies g->1(#0) Po(#e) mo" which decreases as the я. / decreases. Consequently, the optimal F1 score decreases as the imbalance ratio becomes more severe, lim- iting the best performance of classifiers in terms of the FI score. In addition, optimizing accuracy under an imbalance ratio would further sacrifice the F1 score.
F1()))) (wo)
1
1) Pr(Y
1) Pr(ý =
Y 1) Ti (20).
where Pi P1()) = Pr(wT =x)) > 0 Y 1). In
addition, Pr(Y = 1, = =) =00=mooole Po(o)) with Po(xo)
Pr(wT T (X- 2o)= > 0 Y = -0) and Pr(Y = O.Y = 1)
mi - (zn)).
Example 2. Figure 2 exhibits the relationship between the deviation of S and s with the imbalance ratio To/ and the class overlup Iu pol when Po N N o²) and P1 ~ (H,, In Figure 2(a), with balanced data and small class overlap, the deviation between the optimal ac- curacy and the maximum score negligible, and the decision boundaries align closely. Figure 2(b), the devia- tion increases with larger class overlap. In Figure 2(c), the deviation grows with an increasing imbalance ratio. with lower Fl score at the Buyesian decision boundary.
To study the impact of the imbalance ratio on the Fl score and compare its optimal decision boundary the Bayesian decision boundary, define population Fl score
2m PT (aro) F1(x)) (1o) + Po +
Lemma 3.2. The decision boundary that maxinizes F1 (xx) for Do E is
(x()) F1(Io) To
Although some prior studies (Saerens et al., 2002; Xu et al., 2021) share similar findings on how the imbalance ra-
To E X
Po(zo) 2 F1(zu)

---

Learning Imbalanced Data with Beneficial Label Noise
tio impacts the decision boundaries, our theoretical analysis on misaligned optimal decision boundaries that maximize accuracy and Fl-score provides deeper insight that moti- vates us in proving the counteracted impact of asymmetric instance-dependent label noise model on decision bound- aries under class imbalance.
invasion. Therefore, we propose the flipped rates satisfy:
p(a) x n(x),
7(x) = O.
(2)
Using this label noise model, we can generate the noisy data, as outlined in Algorithm 1. As an example shown in Figure 3, the asymmetric label noise model pushes the biased deci- sion boundary (left vertical dashed line) on imbalanced data distribution (solid curves) toward the majority region at the corrected decision boundary (right vertical dashed line) on the data distribution with label noises (dashed curves).
4. Label-Noise-based Re-balancing Approach
Motivated by the decision boundary distortion due to class imbalance, we propose a novel I
(LNR) approach. By introducing beneficial asymmetric label noise, LNR improves classification performance in both binary and multi-class imbalanced learning.
By introducing artificial label noises with well-behaved flip-rate function p(z). the deviation of the decision bound- ary caused by asymmetric label noise can correct the biased decision boundary on imbalanced data. The flipped majority class samples enrich the minority class while simultancously reducing the number of the majority class. The Bayesian decision boundary that maximizes accuracy based the re- labeled sample (X, Y*) is = {x" E X:: (xxx)) 0.5}. Lemma 4.1. The optimal Bayesian decision boundary based on (X, (,,Y*) is
4.1. Methodology
The proposed LNR method consists of two steps: (1) intro- duce artificial label noises to the original data, and (2) train a classifier on the noisy data. We first focus on binary clas- sification to motivate and justify the introduction of label noise, which plays erucial role in correctly adjusting the decision boundary on the relabeled data.
= [1 [1-11[pp 22(x*)) 5 - ===f0 ===='ex Ex:
Pi(x*) Po(x*)
Suppose the original data (X, Y) ~ P. Define p(z) = Pr(Y = 11 X = x, Y = 0) as the probability of flipping a sample from the true label y 0 to the noisy label Y 1, similarly, define >xx) = Pr(Y
X:
conditional on that the flip-rate on the decision boundary satisfies p(x²) < 0.5
0 X =
r, Y 1)) With the flipped labels, we obtain the noisy data pairs (X) ~ P Consequently, 17" (z) = Pr(Y = 1|
As 2p(x2((=) < 1, Lemma 4.1 indicates that the Bayesian decision boundary on noisy data shifts back to the majority class region. While this may sacrifice some overall accuracy, it improves the classification accuracy of minority class samples, thereby enhancing the F1 score. Furthermore, the decision boundary that maximizes accuracy on the noisy data coincides with the optimal Fl decision boundary on the original data when F1(z') 2-F1(x²) = 1 - -2p(x").
X = x) = n<)]
((=)] + [1 - (())))(z).
The label noises discussed in Ahfock & McLachlan (2021); Canmings et al. (2020) are symmetric and proportional to classification difficulty 7(r)), where both positive and nega- tive samples are equally likely to be flipped on the decision boundary. The classifier remains consistent such sym- metric label noises, counterbalancing the distortion and leav- ing the decision boundary unchanged. For example, with p(z) = 7(x), p(z) + >xx) = 1, we have 17" (z) = 17(z), making the noise model harmless.
Similar as in Section 3.2, consider the classifier y I(w² - =τ) > 0) for some w E Rd and zg E X. trained on the noisy data. Define the corresponding popula-
However, in imbalanced data, the overwhelming number of majority class samples invading the minority class re- gion often results in significant misclassification of minority class samples. To achieve higher Fl score by adjusting the decision boundary and reducing the imbalance ratio near the decision boundary, we consider an asymmetric la- bel noise model that only flips majority class samples to minority class samples. This approach shifts the biased de- cision boundary toward the majority class region rather than neutralizing the shifts with symmetric noise. Since deeply invaded majority class samples cause greater harm, we aim for a noise model directly proportional to the extent of this
tion Fl score as F1 F1* (xz)) "[P[(=)) + ( (m7) where
2m (wr))
Pi = Pr(wT - 23) >0 1) and Pj = Pr(w" " (X - 2x) > 0| 0 0)) कं = Pr(Y = 1) and क = Pr(Y = = =1 γं. The decision boundary that maximizes Flaireess"""aa F16()() based on relabeled sample (X, is
SPT. = x E E: Pi(e)) = Pi(2)) 2- F1 Tī
FI"(z)) T )
Under the relabeled data, it is expected that PJ (2/6) 3/6)) //j (=)) and /i/ will increase compared to Pi (xxx) (x)))/ (o((")) and Ti/T Therefore, compared to the original decision bound- ary sol that maximizes F1(2o), we expect an increase in F =1*(())) the maximum Fl score under the noisy data.
5

---

Learning Imbalanced Data with Beneficial Label Noise
Algorithm Flip labels in binary classification with noise model p(x)
Input: Feature X and labels Y, Classifier Cy
Parameters: Threshold Eftip Indya= < index(Y
0). f) 2 zeroVectorssize = N)
for i E 1 + len do
[1] - Cy(X [i])
end
+ mean(f) [IndMa]), o at (IndMa)
for each iMA in IndMa do
Zlima] f M
[iMA]= max((ann(ZZlimma) (rtp),
U - (ernoulli(p/ima))
Figure 3: Asymmetric label noise shifts the biased decision boundary towards the majority class region, where the dashed lines are sample distributional densities after label flipping. The majority class samples deep into the minority region have the higher flip-rates p(x).
if U = then
Y [iMA] +
end
end
Return: Y
In summary, with an asymmetric instance-dependent noise model, the artificial label noise correct the biased deci- sion boundaries caused by class imbalance, improving the classification accuracy of minority class samples. The FI score is expected to increase under the noisy data, demon- strating the effectiveness of the asymmetric label noise model imbalanced classification.
is a tunable parameter for selecting appropriate samples with high similarities to minority class features. Notably, tanh (2 [iMA] frip) > O when 2 [[iMA] > Ugtip. which zeros the flip rates of majority class samples below tytip
Step 4. Flip majority class samples into minority class according to their flip rate p[iMa).
Empirically, typi can be selected by cross-validation. See Section 5 and Appendix for more discussions.
4.2. Constructing noise model in binary classification
Estimating the flip-rate p(r)) which is proportional to n)(x) in binary classification, is crucial. We propose utilizing the posterior probabilities ff(z) obtained from an arbitrary classifier trained imbalanced data approximate r7(x),
4.3. Extend multi-class classification
Unlike binary classification tasks that contain only one mi- nority class, multi-class data may include multiple minority class samples. To make LNR adopt the various imbalance ra- tios between classes, we weigh the flip rate of being flipped into different classes by the class prior weight vector Oc. where is the class vector. These weights are calculated based on the sample sizes Nc. and max (0c - Oc[v). 0) guar- antees label flipping only proceeded from majority class to minority class and preventing label flipping between classes with the same class prior. The label- fiippiifiiiing procedure for multi-class scenarios is detailed in Algorithm 2.
thereby constructing the label noise model p(æ:). The noise model is constructed using Algorithm 1:
Step 1. Train a classifier Cy on the original imbalanced data and obtain the posterior probabilities n/ = (X(()) Although ) may be biased or underestimating 77(x), the majority class samples with features similar minority class samples still tend to exhibit relatively higher posterior probabilities of being classified into minority classes. As result, these samples are expected to have relatively higher probabilities of being relabeled as the minority class.
Due to that more complex neural networks are typically used in multi-class image classification tasks, training a separate flipping rate estimator is costly, unlike in binary classification scenarios. As our method allows for using any classifier as the flipping rate estimator, we directly uti- lize the model itself to calculate the posterior probabili- ties with so f tmax(ff(u"" X, Y)) during training, where se(w"; X, Y) is the output of the last fully-connected layer Step 3. Rescale Z(iMA] to [o. 1] by calculating pliMA] = (iiMal - tytp), 0), where the threshold tytsp
Step 2. Employ z-score standardization on Í[iMA] to get ZliMA), where ÍMA is the index of each majority class sample. This magnifies the differentiation in the posterior probabilities of majority class samples. The deviation level, measured by Z[iMA]. ranks majority class samples in pro- portion to their similarity to the minority class samples.
of model w" at epoch t. Similar to the Deferred Re- Weighting (DRW) approach (Cao et al., 2019), we postpone
6

---

Learning Imbalanced Data with Beneficial Label Noise
Algorithm 2 Constructing label noises during training at round t
Input: Feature X and labels Y, and the model w² at current epoch
Parameters: Flipping threshold tyipps Deferred epochs TD
Q < woffmaa
x.,Y), Y)) where :c(w":;;)) -) the last fully-connected layer output of model w
le < 1 minMax(NN)) where Ne counts the samples of each class
1 (ea + (Q)
for (z, (X, do
Calculate 7-score for sample x: 2, <- Q/-1-
Get the flipping strength for class c E C: Oany + ma 0c(v),0)
Calculate the class-wize flip-rate for sample x: Fr t max(tanh( 2. - t(kp), x f== U < Bernoulli(F=)) # generated by independent Bernoulli distribution with Fz.
U contains and > TD then
y² + [elindexxf FFa [u == 1]))] where C is the class vector.
end
Update the model wt with (x,y²)
end
the introduction of label noise until the model has converged (after TD epochs) in the fine-tuning stage leveraging possi- ble influence on the feature extraction process.
Classifiers. As data-level methods be coupled with many classifiers, we run the experiments with three com- monly used classifiers on KEEL datasets: k-nearest neigh- bor (KNN) (Cover & Hart, 1967), CART (Quinlan, 1986), and multi-layer perceptron (MLP) (Rumelhart & Williams, 1986). For multi-class datasets, the Resnet-32 used for all methods. The classifier parameters and implement details are in Appendix C.2.
is important to note that the posterior probabilities pro- duced by multi-class model form a C' x 1 vector where C' the number of classes. majority class sample has C' posterior probabilities corresponding being classified as other classes, resulting in C- 1 distinct flipping rates. In the case of majority class samples exhibiting similar features to multiple minority classes, resulting in multiple high flip rates, we flip such majority samples into the minority class having the highest flip rate. Random flipping is employed in the event of tie on flip rates.
Baseline and prior methods. For binary classifica- tion, the baseline trains the classifier with the original imbalanced data. We compare our method, LNR, with well-known resampling method, including over-sampling methods: SMOTE (Chawla et al., 2002), ADASYN (He et al., 2008). and Borderline (Han et al., 2005); and under- sampling methods: Candom-Under- pame--Sampiing (RUS), OSS (Kubat & Matwin, 2000), and CC (ClusterCentroids (Lin et al., 2017). LNR adapts the final layer of the MLP classifier to include a soft-max function as the flip-rate estimator Cy, trained on the original data for KEEL datasets.
5. Experimental Evaluations
Datasets. Imbalanced Binary classification: We evalu- ated LNR on 32 datasets from the KEEL repository (Derrac et al., 2015), featuring a wide range of imbalance ratios from 1.82 to 49.6. Summary statistics of these datasets can found in Table in the Appendix. The criteria for selecting these 32 datasets are detailed in Appendix C.5. Imbalanced Multi-class classification: We conducted experiments two multi-class image datasets: CIFAR-10 and CIFAR-100. As the datasets are initially balanced, we artificially cre- ate step-wise and datasets with the imbalance ratio of Nmaz/ /Nmmn = 100, where Naax and Nmin are the sample size of largest class and smallest class. Step-wise imbalance was created by removing 99% of samples from half of the classes, while long-tailed imbalance was generated by exponentially re- ducing sample sizes per class to satisfy the imbalance ratio. Appendix C.5 provides further details. Additional experi- mental results on synthetic data are in Appendix C.5.
For image classification tasks, the baseline is the algorithm- level approach 1DAM-DRW (Cao al., 2019), and we com- pared our method with the data-level method RSG (Wang et al., 202 a)) integrating with the baseline. We also show the improvement of our approach combined with a more recent algorithm-level approach GCL (Li et al., 2022c). On long- tailed CIFAR-10/100, we also compared our method with mixup-based methods ReMix (Chou et al., 2020) and the latest SOTA SelMix (Ramasubramanian al., 2024) on top of the algorithm-leve SOTA MiSLAS (Zhong et al., 2021). As SelMix relies balanced auxiliary dataset Dval- denote its original implementation as 10k (DDaall = 10000). We also compare with a smaller size of [Dvall = 1000, de- noted as (Ik), and an imbalanced (imb) validation dataset.
imbalanced versions of these
7

---

Learning Imbalanced Data with Beneficial Label Noise
Hyperparameters for all methods were selected via grid search within the ranges specified in Appendix C.2.
Table 2:
Comparision on Long-tailed CIFAR-10/100
Datasets in the format of (mean % =std).
Evaluation metrics. To ensure a comprehensive evaluation, we report multiple widely used metrics. For step-wise imbal- anced multi-class datasets, we report the average minority class accuracy ACOMI and the overall accuracy ACCoveralls as we expect to obtain higher recall on minority classes without sacrificing the overall accuracy, For long-tailed datasets, we compare their average accuracy of Many-shot, Medium-shot, and Few-shot groups of classes. For the binary classification on KEEL, we used F1 score, G-mean, and AUC. The F1 score and G-mean focus on the perfor- mance of classifying minority class samples, while AUC is more conservative, as is sensitive to the error rate of the majority class samples. Appendix C.7 shows the metries definition and detailed evaluation settings.
Long-tuiled Cifar-10
Many-shot
Medium-shot
Few-shot
Overall
LDAM
82.62:0.06
76.12=0.1
75.01=0.1
78, 08.39:0000
LDAM+RSG
81,56=0.115
77.03=0.11
77.30=0.1
78.93=0.02
LDAM+LNR
81,17+0.08
76.42+0.01
79.83+0.11
79.34+0.01
88.60.0.04
19.57
70.08100.2
80.55.10.02
GCL
88,20+0004
79.50+00.7
77.60+0.22
82.41 22.41+0.033
MISLAS
91.0010.14
80.1710.22
75.72.0.199
82.10.10.12
MiSLAS+ReMix
90.04:0.20
79.8 79.82:0.116
79.78:0.20
82.92:0.110
SelLLA (Ok)
86.81 10.22
10.5010.17
83,52.10.21
83.29.10.07
M Ik)
81.61 40.14
79.89
87.60:00.0
8..72:40.22
MiSLAS+SS(Mix(mb)
82.21 10.13
81,44.1.111
81,910.21
81,8 10.09
MISLAS+LNR
84,62.0,22
80,90.10,22
86.07
83.56.10.08
Long-tailed Cifar-100
Many-shot
Medium-shot
Few-shot
Overall
LDAM
62.21:2.055
43,28:00008
20.83:0.033
42,98200003
LDAM+RSG
60.46.10.05
43.88
22.57.1.099
13.08.0.U07
LDAM+LNR
61,04
4..06.10.02
24.11:0.04
43.58 1.002
GCL
67.16.20.03
46.63.00.06
13.5720.06
43.90110.03
GCL+LNR
50.17
51.38
25.02:0.09
0.05
MiSLAS
62.05--0.00
48.42400.1
26.07+0.12
46.85 +0009
Results on step-wise imbalanced data. Table 1 reports the comparison of our method with the SOTA data-level method RSG and the SOTA algorithm-level method GCL on CIFAR-10/100 datasets. Without sacrificing the over- all accuracy, our method outperforms the previous data- level SOTA (RSG) by 8.04% and 4.2% and the previous algorithm-level SOTA (GCL) by 15.44% and 21% in terms of the accuracy of minority classes on stepwise CIFAR-10 and CIFAR- 100 respectively. Results also demonstrate the significant improvement achieved by combining LNR with algorithm-leved methods of LDAM-DRW and GCL.
MiSLAS+ReMix
59 59.06+00211
49,22:40.09
27,93+00110
46.59:00115
60,93:1.112
02.06:00.11
25.10:0.13
47.43-10.10
AiSLAS4Se/Mix(I))
61,27+0.08
50.82++0.18 18
21.34+001
46.04 +0.11
MiSLAS+ScIMiximb)
56 56.6620.12
51.17:0.066
25.31
45. +0.23
MISLAS&LNR
56. 26+00+2
51,46+00.22
08.34+0.21
48.52 +0012
Table 3: Expected Calibration Error (ECE).
ECE
MISLAS
+ReMix
+SelMix
+SclMix(im))
+LNR
CIFAR-10
3.70
19.62
2.75
6.36
5.16
CIFAR-10 100
5.43
18.28
1.32
3.01
9,55
jecting only 44 and 65 instances of label noise into the original data, respectively, demonstrating the effectiveness of our noise model.
Table 1: Comparision on Step-wise 1IFAR-10/100 Datasets in the format of (mean std).
Step-wise Cifar-10
Step-wise Cifar-100
Model Calibration. MISLAS (Zhong et al., 2021) em- pirically observed that applying mixup-based methods like
AccMi
AcCuorall
Accsn
AcCowrrll
LDAM
66.41:102
77.47=0.06
19.80:0.00
45.23=0.03
LDAM-RSG
67.02=40.07
77.74=0.08
21.67:1004
45.51=0.02
ReMix during the fine-tuning stage severely degrades model calibration (measured by ECH). As shown in Table 3, ReMix significantly increases ECE. In contrast, although LNR also slightly affects calibration accuracy, these minor degrada- tions are acceptable given its substantial improvements in classification performance.
LDAM+LNR
75.06:10.09
7..12:0.03
25.84:10.06
45.63=0.02
GCL
56.78:1008
74.80=0.04
5.48 =0.03
43.87=0.07
GCLALNR
7.02
80.8++0.02
0.03
46.20 0.03
Results on Long-tail Cifar-10/100. Due to the scarcity of tail classes, there is often heightened interest in their classi- fication accuracy, commonly termed few-shot accuracy. As shown in Table 2. our method achieves state-of-the-ant per- formance on both CIFARIO-1 L0-LT and CIFAR 100-LT under an extreme imbalance ratio of IR- = 100. By integrating with algorithm-level methods such as LDAM, GCL, and MiS- LAS, LNR significantly improves tail-class performance without compromising overall accuracy. Crucially, LNR eliminates the dependency on balanced validation data-a key advantage over SelMix, which requires 10k balanced auxiliary samples (equivalent to 20x the tail-class train- ing data) yet suffers significant performance degradation with smaller or imbalanced validation sets. Notably, LNR achieves SOTA performance on 1AAAR--00100--11 by in-
SelMix benefits from calibration using balanced auxiliary dataset, achieving lower ECE. However, when the auxil- iary dataset becomes imbalanced, SelMix's calibration im- provement weakens, resulting in higher ECE than LNR on CIFAR-10. Crucially, LNR requires no auxiliary dataset. By flipping only a small number of labels while preserving original data features (no feature modification). it maxi- mizes overall classifier performance, especially for minority classes, while maintaining ECE within acceptable bounds. This highlights L.R's practical superiority in balancing accuracy and reliability under real-world data constraints.
Model fairness. To demonstrate model faimess improve- ments brought by LNR, we compare the confusion matrices
8

---

Learning Imbalanced Data with Beneficial Label Noise
KNN F1-score
CART
MLP Fi-score
Fi-score
Baseline
LNR
SMOTE
ADASYN
Borderline
05S
ClusterCentrrad
RUS
G-mean
AUC
G-mean
AUC
G-mean
AUC
Figure 4: Relative ranking of methods from 1 (the best) to 0 (the lowest) on 32 KEEL datasets.
of GCL and GCL+LNR on long-tailed CIFARIO in Table LNR introduced 94 label noises, including 41 samples relabeled as tail class "8: ship", 44 samples relabeled tail class "9: truck" from head class "0: airplane", and 6 samples relabeled as tail class "9: truck" from head class "1: automobile". Despite the extreme imbalance ratio between head classes and the tail class "7: horse", no samples were flipped into class "7: horse", as LNR flips are based on feature similarity, making these flips controlled and self- limiting. The bold diagonal numbers demonstrate that LNR achieves effective trade-off between the classification performance of the head class (yellow) and the tail class (green). While there is a slight reduction in head classes (16 fewer true positives), LNR leads to a substantial im- provement in the performance of tail classes (204 more true positives). Simultaneously, LNR significantly reduces tail-to-head misclassifications (blue) with minor increased head-to-tail misclassifications (red), preserving the model fairness across different classes.
minority classes without sacrificing the overall accuracy.
Results on KEEL results. The relative rankings shown in Figure 4 demonstrate that our method achieved the highest average performance across 32 datasets for all three clas- sifiers in both F1 score and G-mean. In addition to these rankings, we calculated the Pearson correlation coefficients between the relative performance differences of our method and other methods, and the data distribution characteristics. The numeric results are detailed in Table 11 in Appendix C.7. Our method demonstrated superior F1 score and G-mean performance on data with higher class overlap, attributed to reduced class overlap through label flipping at decision boundaries. In highly imbalanced data settings, oversam- pling methods require synthesizing oversized samples to balance the data, which exacerbates generative errors and increases the error rate for majority class samples, thereby hampering AUC performance. On the contrary, LNR does not involve adding removing samples, significantly hancing F1 score and G-mean scores without compromising AUC performance.
1
2
0
-26
+2
-4
+1
-2
0
-1
+13
+16
1
-4
0
-1
+1
0
0
0
0
+4
6. Conclusion
5
+1
-4
-2
-7
+3
+3
-2
+3
+18
-4
-13
-5
+2
+5
In this paper, we theoretically analyze the impact mecha- nisms of imbalanced data classifiers. By investigating the influence of label noise on the data, propose a novel method, LNR, of artificially introducing label noise to en- hance the performance of imbalanced learning. LNR intro- duces a novel asymmetric label-noise model that strategi- cally flips labels to adjust decision boundaries in imbalanced learning while avoiding information loss or generative er- inherent in traditional resampling and augmentation methods. LNR not only demonstrates comprehensive su- periority on synthetic and KEEL datasets but also achieves state-of state-of-the-art performance on step-wise and toiged imbalanced image classification benchmarks. In addition, LNR can integrate seamlessly with any classifiers and other algorithm-lewe methods for imbalanced learning.
0
0
-2
-3
+3
-
+5
+1
+1
5
2
0
-1
-3
0
-1
+1
+2
0
+2
6
1
+2
-2
+6
-1
-5
-4
+2
-1
+1
-5
0
+3
-20
-1
-8
+22
+1
+5
-11
-6
+2
-2
-1
-2
+80
+20
42
488
-3
-5
+1
-2
-3
-2
+3
+102
Table 4: Confusion matrix comparison of GCL and LNR.
The signed values denote the changes made by LNR.
By selectively flipping the labels of samples near the deci- sion boundary that share similar features with minority class samples, LNR minimizes the data editing and achieves a more efficient correction of the biased decision boundary, which offers us significant improvements in the accuracy of
9

---

Learning Imbalanced Data with Beneficial Label Noise
Impact Statement
Derrac, J., Garcia, S., Sanchez, L.., and Herrera, F. Keel data- mining software tool: Data set repository, integration of algorithms and experimental analysis framework. J. Mult. Valued Log. Soft Comput, 17:255-288 2015.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which feel must be specifically highlighted here.
Fotouhi, S., Asadi, S., and Kattan, M. A comprehensive data level analysis for cancer diagnosis on imbalanced data, Journal of biomedical informatics, 90:103089, 2019.
Acknowledgements
iL supported by the Australian Research Council (ARC) (Grant No. DE240101089, I.P240100101, DP230101540)
Frénay, B, and Verleysen, M. Classification the presence of label noise: a survey. IEEE transactions on neural networks and learning systems, 25(5):845-869, 2013.
and the NSF&CSIRO Responsible AI program (Grant No. 2303037). MG was supported by ARC DE210101624, ARC DP240102088, and WIS-MBZUAL 142571. GW was supported by the National Natural Science Foundation of China (Grant No. 12471255), the Natural Science Foun-
Goldberger, J. and Ben-Reuven, E. Training deep neural- networks using a noise adaptation layer. International conference on learning representations, 2022.
dation of Shanghai (Grant No. 237R1419440 1419400); and the Fundamental Research Funds for the Central Universities (Grant No. 63253110). LP is supported by ARC (Grant No. LP240100101).
Han, H., Wang, W.-Y., and Mao, B.-H. Borderline-smote: a over-sampling method in imbalanced data sets learn ing. International conference intelligent computing, 2005.
He, and Gareia, E. Learning from imbalanced data. IEEE Transactions on knowledge and data engineering, 21(9): 1263- 1284, 2009.
References
Ahfock, D. and McLachlann, G. Harmless label noise and informative soft-labels in supervised classification. Com- putational Statistics & Data Analysis, 161:107253, 2021.
He, H., Bai, Y., Garcia, E., and Li, S. Adasyn: Adaptive syn- thetic sampling approach for imbalanced learning. 2008 IEEE international joint conference on neural networks, 2008.
Azaria, A., Richardson, A., Kraus, S., and Subrahmanian, V. S. Behavioral analysis of insider threat: A survey and bootstrapped prediction in imbalanced data. IEEE Transactions on Computational Social Systems, 1(2):135-
Hou, C., Zhang, J., Wang. H., and Zhou, T. Subclass- balancing contrastive learning for long-tailed recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 5395 00005-547,44 2023.
155, 2014.
Cannings, T. I., Fan, Y., and Samworth, R. J. Classification with imperfect training labels. Biometrika, 107(2):311- 330, 2020.
Kang. B., Xie, S., Rohrbach, M., Yan. Z., Gordo, A.. Feng, J., and Kalantidis, Y. Decoupling representation and classifier for long-tailed recognition. arXiv preprint arXiv: 1910.0921191 2019.
Cao, K., Wei, C., Gaidon, A., Arechiga, N., and Ma, T. Learning imbalanced datasets with label-distribution- aware margin loss. volume 32, 2019.
Kaur, H., Pannu, H. S., and Malhi, A. K. A systematic re- view imbalanced data challenges in machine learning: Applications and solutions. ACM Computing Surveys (CSUR), 52(4):1-36, 2019.
Chawla, N., Bowyer, K., Hall, L.,, and Kegelmeyer, W. P. Smote: synthetic minority over-sampling technique. Jour- nal of artificial intelligence research, 16:321-357, 2002.
Krawczyk, B. Learning from imbalanced data: open chal- lenges and future directions. Progress in Artificial Intelli- gence, 5(4):221 2016.
Chawla, N. V., Japkowicz, N., and Kotez, A. Special issue on learning from imbalanced data sets. ACM SIGKDD explorations newsletter, 6(1):1-6, 2004.
Kubat, M. and Matwin, S. Addressing the curse of im- balanced training sets: One-sided selection. Fourteenth International Conference on Machine Learning, 2000.
Chou, H.PP. Chang, S.-C., Pan, J.-Y., Wei, W., and Juan, D.-. Remix: rebalanced mixup. In European conference on computer vision, pp. 95-110. Springer, 2020.
Li, B., Han, Z., Li, H., Fu, H., and Zhang, C. Trust- worthy long-tailed classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6970-6979, 2022a.
Cover, T. and Hart, P. Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1):21-27, 1967.
10

---

Learning Imbalanced Data with Beneficial Label Noise
Li, J., Tan, Z., Wan, J., Lei, Z.,, and Guo, G. Nested col- laborative learning for long-tailed visual recognition. pp. 6949-6958, 2022b.
Samuel, D. and Chechik, G. Distributional robustness loss for long-tail learning. In Proceedings of the IEEF/CVF international conference computer vision, pp. 9495- 9504, 2021.
Li, M., Cheung, Y.m., and Lu, Y. Long-tailed visual recog- nition via gaussian clouded logit adjustment. In Proceed- ings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6929-6938, June 2022c.
Schwartz, E., Karlinsky, L., Shtok, J., Harary, S., Marder, M., Kumar, A., Feris, R., Giryes, R., and Bronstein, A. Delta-encoder: an effective sample synthesis method for few-shot object recognition. Advances in neural informa- tion processing systems, 31, 2018.
Li, T., Cao, P., Yuan, Y., Fan, L., Yang, Y., Feris, R. S., Indyk, P., and Katabi, D. Targeted supervised contrastive learning for long-tailed tailed recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6918-6928, 2022d.
Song, H., Kim, M., Park, D., Shin, Y., and Lee, J.-G. Learn- ing from noisy labels with deep neural networks: A sur- vey. IEEE Transactions on Neural Networks and Learn- ing Systems, 34(11))8135-88
2022.
Lin, W.-.., Tsai, C.-F, Hu, Y-H.,, and Jhang, J.-S. Clustering- based undersampling in dass-imbalanced data. Information Sciences, 409-410:17- 17, 2017. ISSN 0020- 0255.
Tomek, Two modifications of cnn. IEEE Transactions on Systems, Man, and Cybernetics, SMC-6(11);769-772 1976.
Wang, J., Lukasiewicz. T., Hu, X., Cai, J., and Xu, Z. Rsg: A simple but effective module for learning imbalanced datasets. pp. 3784-3793, June 2021a.
Liu, T. and Tao, D. Classification with noisy labels by importance reweighting. IEEE Transactions on pattern analysis and machine intelligence, 38(3):4474)) 2015.
Wang. X., Lyu, Y., and Jing. L. Deep generative model for robust imbalance classification. June 2020.
Napierala, K. and Stefanowski, J. Types of minority class examples and their influence on learning classifiers from imbalanced data. Journal of Intelligent Information Sys- tems, 46:563-597, 2016.
Wang, X., Lian, L., Miao, Z., Liu, Z., and Yu, S. Long taiged recognition by routing diverse distribution-awe aware experts. In International Conference on Learning Representations, 2021b.
Patrini, G., Rozza, A., Krishna Menon, A., Nock, R., and Qu, L. Making deep neural networks robust to label noise: A loss correction approach. Proceedings of the IEEE conference computer vision and pattern recognition, 2017.
Wilson, R. and Martinez, T. Reduction techniques for mnstance-based learning algorithms. Machine learning. 38:257-286. 2000.
Xiao, T., Xia, T., Yang. Y., Huang, C., and Wang, X. Learn ing from massive noisy labeled data for image classifica- tion. Proceedings of the IEEE conference computer vision and pultern recognition, 2015.
Phua, C., Alahakoon, D., and Lee, V. Minority report in fraud detection: classification of skewed data. ACM SIGKDD Explorations Newsletter, 6(1):50-59, 2004. ISSN 1931-0145.
Xu, H., Liu, X., Li, Y., Jain, A., and Tang, J. To be robust to be fair: Towards faimess in adversarial training. In International conference on machine learning, pp. 11492- 11501. PMLR, 2021.
Quinlan, R. Induction of decision trees. Machine learning, 1:81-106, 1986.
Ramasubramanian, S., Rangwani, H., Takemori, S., Samanta, K., Umeda, Y., and Radhakrishnan, V. B. Selec- tive mixup tuning for optimizing non- non-decomposable metrics. In The Twelfth International Conference on Learning Representations, 2024.
Zhang, H. Cisse, M., Dauphin, Y. N., and Lopez-Paz, D. mixup: Beyond empirical risk minimization. arXiv preprint
2017.
Zhang, Y., Hooi, B., Hong, L., and Feng. J. Self-supervised aggregation of diverse experts for test-agnostic long- tailed recognition. Advances in neural information pro- cessing systems, 35:34077-34090 2022.
Rumelhart, David, H. G. and Williams, R. Learning rep- resentations by propparppggginng errors. Nature, 323: 533-536, 1986.
Sacrens, M., Latinne, P., and Decaestecker, C. Adjusting the outputs of a classifier to new a priori probabilities: a simple procedure. Neural computation, 14(1):21-41, 2002.
Zhong, Z., Cui, J., Liu. S., and Jia, J. Improving calibra- tion for long-tailed recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16498, 2021.

---

Learning Imbalanced Data with Beneficial Label Noise
Zhu, J., Wang. Z., Chen, J., Chen, Y.-P. P., and Jiang, Y.- G. Balanced contrastive learning for long-tailed visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6908- 6917, 2022.
Zhu, X., Wu, X., and Chen, Q Eliminating class noise in large datasets. Proceedings of the 20th International Conference on Machine Learning, 2003.
12

---

Learning Imbalanced Data with Beneficial Label Noise
A. Proofs
A.1. Proof of Lemma
Proof. By Bayes theorem,
(T) 7Tx) = Pr(Y = x= (1)) + (o())
Thus, 7²2)= = 0.5 is equivalent to Pi(x) Nu Po(z)
For Gaussian data with Po Nup. Σ) and Pi N(1,, Σ), Po( mi
No is equivalent to
(z (1)) ) 3 (x m1)
exp
exp 1-2(2 (z - 140)²² - 1-1101110 {x - [o)
and this can be simplified to
wT x + ln
70
with w 5-1 no) and 3= 101
5- 140
5-
1 //4 --
A.2. Proof of Lemma 2
Proof. The population version of the F1 score is defined as
2T P1 (20)
F1(xo) TI + Po (z)) + π
(3)
where Pi = P mo) > 0 Y = 1). and Po(xo) Pr(w" (X xo) > 0 =0)
Here, we denote the numerator as A = 2T P1 (ao) and the denominator as B = (1Pi(z0) + toPo(ao) + nι. The xx (1x)) can be found by letting a/1(zz) 0, that is,
(F1 A'B B'A
Oxp
where A' 8A Ozo
and B'
8B Uzq
As O, we have
A'B B'A.
(4)
Assume exchangeability of integration and differentiation, the Leibniz integral rule implies 8P, (2o)/02o -Pi (x) and 8Pa(zn)/8nn = -Po By substituting
= 2 (w) B' = - Pi(xa)w To Po(oo))
into (4), we obtain that
(1 Pi(o) + ToPo(2o) = (TPP)(Z0) (- ToPo)
which is equivalent to
2T Pi (xo)
F1(0o)
(5)
m1 PI (xr) + PoPo)
Rearranging the equation results in
(1(x)) FI(
Po(
FI π
which indicates that the decision boundary that maximizes the population score of the form
5 = { E X
Pi(2)) F1(xo)
=1
Po(xo)
2 - Fl m
13

---

Learning Imbalanced Data with Beneficial Label Noise
A.3. Proof of Lemma 4.1
Proof. First,
(x))
Pr(Y = = = =
+Pr(Y* x 0) = X =
7(x) + p(=]]1 [//]
= (())[1-((z)) + pxx).
As x" satisfies 17" (2*) 0.5 with the constrain of p(²") 0.5, we have 7(x"){1 p(x²)] + +(xx) 0.5, which is equivalent to
m (x*
Ty (xt)(t)) + To Poloo - p(x*)) -(x=)==+ 2
follows that
P1(x)) Po(x*) = [1 (1-=pp(x")
2p(xx)) To
Thus, the optimal Bayesian decision boundary that maximizes the accuracy on the noisy dataset (X, Y*) is defined as:
5* x x: P1(x)) = 1-p(p=)) 40
Po(x*)
For one-dimensional normal distributed data with P1 N(4, 0.) and Po N σ?), we further take the natural logarithm of both sides to specialize further x" with the terms of (μ1, Mo: G, T1, mo):
In
T
21)²
(x" -
In(1 2p(2*)).
TTO
20²
20
After rearranging, have the optimal Bayesian decision boundary that maximizes the accuracy noisy dataset (X, Y*)
2 [1o + m1)
1
In
To
a²
In(1 2p(x*))
m
(1
140
pn - 140
For the multivariate normal distribution with equal covariance E, the decision boundary has the form:
2( -a) -E) - #1
A ET - pmo + 2ln
+ +2lnn11 2p(x*))
B. Impact of imbalance ratio on decision boundary of KNN classifier
In this section, investigate the impact of the imbalance ratio on the k-nearest neighbors (KNN) algorithm, which is widely used classification method due its non-parametric nature.
The KNN classifies new sample based the majority voting of its k nearest neighbors in the training dataset. Denote Xtrain and Xiest as the training and testing datasets, respectively. For each instance Itest E Xtests let X (xoeetttt k) (e("),...,,,(( ( C Xtrain with corresponding labels ()(1) (A)) } be its k-nearest neighbors with ascending Euclidean distances. Then, the KNN classifies Ftest to the positive class if + Σ I(y =) 2 0.5.
We introduce the following setup to study the impact of the imbalance ratio on the KNN classifier. Denote the open Euclidean ball centered at x E Xiest with the radius (x) = || - 2((i|| as (x)x) = fa E X || [x) s rx (2)), where (8) E Xerasn the k-th nearest neighbor of x and || is the Euclidean distance. Then, the decision boundary the KNN is defined as (Tenn E X: Pr(x E Bg (. 1) Pr(x E Ba (x Y 0)).
14

---

Learning Imbalanced Data with Beneficial Label Noise
Lemma B.1. The decision boundary of the KNN
Sknn { Iknn X
[Baleenn (x)x)dx (Baltrnmn Po(x)dx
Proof. The proof of this lemma follows immediately from the facts that Pr(x Bi (x(Kn))). 1) JB. (sann) Pi (2) and Pr(x E Bx Y = 0) To S. /B(ann))) Po(x)dx.
When the size of the training dataset Vltrain 7 o0, B& (Fknn) degenerates to the point Vkonn and (yllrem (a)dd (atenn) to Po In this case, the decision boundary of the KNN coincides with the Bayesian decision boundary, so the imbalance
( Po(ardd vanishes
ratio has the same impact on the decision boundary of the KNN as for the Bayesian decision boundary. In the case of finite training size, the KNN decision boundary may deviate from the Bayesian decision boundary as demonstrated the following example.
MPPr(X x|V 1)
ToPr(X x|V
Ty AX k)
Ty (Xien k) gkmn
x
B
A
Figure 5: KNN biases the decision boundary from the optimal Bayesian decision boundary towards the minority class region when data is imbalanced. Area A and area B are calculated by assuming r=(=)) = and substituting x² and (x(x*)) into Ai (z, k) and Ao (2x k) respectively.
Example 3. For one-dimensional data, By( is an interval on the r-axis of length (rx(Doan)) The theoretical decision boundary produced by the KNN classifier has the following form
= ={ {Rrm Ikon E X: A (
gkan =
TT
(6)
Au(Kknn,)) TT
where An (Kkmn, (etralzean)
Je-re(Wknn) Pi (x))da, and An( r) patro (mrnn))
Po(x)dd represent the areus under the
class-conditional probability density curves.
Let P1 ~ N(, 1), Po N(2.5, 1) with का = 0.2 and mq = 0.8 as considered Figure the Bayesian decision boundary 0.6955 (the red cross). With r& rx(x*) ) 1, we obtain area A: I A₁ (0.6955, k) 0.1147 and area B: (0.6600 k) 0.1674, thus A < B indicating that the decision boundary of KNN biases from the Bayesian decision boundary towards the minority class.
is = X
As the KNN decision boundary Skan does not have an explicit expression due to Tw rx(x²) being dependent on both x² and k. we numerically demonstrate the relation between ra(x") and the deviation of zknn and x² in Figure From Figure 6, it evident that the decision boundary Ikan of KNN deviates from the optimal decision boundary x" as rx(x²) increases on imbalanced data, while consistently aligns with on balanced data for this example.
15

---

Learning Imbalanced Data with Beneficial Label Noise
[-015
020
(a) KNN on imbalanced data
(b) KNN on balanced data
Figure 6: The thin curves in different colors represent ni A(x, ris) and To (a, with different values T r. Their intersections (blue cross) are the decision boundaries produced by KNN with the corresponding r value, and the optimal Bayesian decision boundary x²' is shown with the vertical dashed black line.
The Bayesian decision boundary tends be biased towards the minority class region comparison to optimal Fl score decision boundary, thereby contributing decrease in Fl score performance. Additionally, the decision boundary fitted by the KNN classifier exhibits a continued encroachment upon the minority class region, resulting compromised accuracy and F1 score performance.
C. Experiment details and full results
C.1. Compute resources
Hardware information. Type of CPU: One Intel(R) Core(TM) 17-117000 CPU @ 3.80GHz with 16 cores. Type of GPU: One NVIDIA GeForce RTX 4080 SUPER with memory of 16GB.
Computational workload. The experiments 8 synthetic datasets repeated 30 rounds using 3 different classifiers with 8 methods. The total amount of training is x 30 3 x 8 5760 for experiments synthetic data. The experiment conducted on CIFAR-10/100 involves 5 methods in 2 different imbalanced settings. The results are collected from 10 rounds of training with 100 epochs. The total epochs of training x 2 x 10 x 100 = 10000. The experiments on 32 KEEI. datasets are repeated in 100 rounds. The total amount training x 100 x 8 19200.
Implementation code for LNR.
Code: httpp:: com www.m/aaaggggggg//n..gg.g.
C.2. Method parameters
The hyperparameters classifiers are listed in Table Re-sampling methods are compared and implemented using the "imbalanced-leamm" package in Python. Classifers KNN, CART, and MLP are implemented with the Python "scikit-learn" package and parameters not listed set to their default values. The Resnet-32 is implemented with Pytorch.
The parameters are selected within the tuning range in Table 5 by repeating 3 times of 4-fold cross-validation reshuffled data with Fl score. The KNN-based resampling methods, including SMOTE, ADASYN, Borderline-MMOT,, and CC
'https://gt http://tttt............aeoooomoe htt.kkkttttaa---orrrriiiiii- Phtpps: //gi thub. com beikit- 1eal-eearn/sssse s
anccddd 1earn.iie
16

---

Learning Imbalanced Data with Beneficial Label Noise
Table 5: Hyperparameters of methods and classifiers
Classifiers
parameter
value
KNN
A
[5,10]
Hidden layer shape
5x10x5
Maximum training epochs Optimizer
800 synthetie data, 2000 KEEL data
MLP
Adam
Learning rate
0.001
Methods
parameter
tuning range
steps
threshold füp training epoch e
[0,4]
10
LNR
[100,4000]
KNN-based resampling
k
1imin(20./minnrrtt
OSS
Rose
[2,30]
10
All resampling methods
re-balance ratio y
S1 1]
10
(ClusterCentroids), require the user to preset the value & to synthetic rank the samples.
We introduce deferred label noise after training epoch TD for multi-class image classification. We set Tp = 80 and fyiip = 3.5 for both imbalance settings on CIFAR-10/100 datasets. For all experiments, the Resnet-32 is trained up to 100 epochs using SDG (stochastic gradient descend) optimizer with a momentum of 0.9 and a learning rate 0.1 for the first 60 epochs, 0.01 for the epochs 60-80, and 0.0001 after the 80 epochs.
For experiments based MiSLAS (Zhong al., 2021), compare ReMix, SelMix, and LNR in the fine-tune stage of MiSLAS (stage-2) with its default implementation and hyperparameters. The detailed implementation be accessed through the GitHub URL in Appendix C.1
C.3. Sensitivity analysis
KNN
CART
MLP
0.97
0.84
0.92
0.66
0.64
0.70
0.96
0.82
0.64
0.90
0.62
0.80
0.69
0.95
0.62
0.88
0.60
0.78
0.68
0.94
500
1000
1500
2000
500
1000
1500
2000
500
1000
1500
2000
Training Epoch Noise
Training Epoch Model
Training Epoch
0.70
0.96
0.68
0.700
0.64
0.66
0.675
0.95
0.90
0.64
0.62
0.80
0.650
0.62
0.88
0.60
0.94
2
0
Threshold
Threshold
Threshold
Figure 7: Sensitivity Analysis. The horizontal dashed lines represent the baseline performance in terms of F1 (blue). G-mean (green), and AUC (red). The right y-axis corresponds to the AUC values. The solid line denotes the performance of LNR. Top:
Minimal impact of noise model training epochs on performance gains, with LNR consistently outperforming the bascline. When the number of training epochs is too low (not converged) or too high (overfitting), performance slightly declines. Bottom: When the label flip threshold is too low, we observe poorer performance on MLP compared to the baseline, indicating that a low leads to excessive noise being introduced. When is too high, only few samples are flipped until the performance gain approaches zero. For KNN and CART, the AUC gradually decreases increases but sensitive MLP.
17

---

Learning Imbalanced Data with Beneficial Label Noise
Sensitivity analysis for each hyperparameter conducted using simulated data while keeping other parameters constant. In the simulations synthetic data, the minority class prior set 0.1, with sample size of 500, class mean distance of 2, and a dimensionality of 5. The results consistently showed that the performance gains from our LNR approach decrease if the threshold is either too low or too high. For KNN and CART, the classifiers always outperformed the baseline regardless of stip- However, for MLP, if stip is too low, excessive label noise can cause performance to fall below baseline levels, while if is too high, performance can degrade to baseline levels.
The impact of the epoch parameter on performance less sensitive. We observed that if the noise model trained for few epochs, the benefits of flipped labels are reduced. However, once the model converges, further training does not significantly enhance performance, the model possesses enough capability to filter out detrimental label noise. As LNR is data-driven, the parameters can be optimized using standard cross-validation techniques like other data-level methods.
The impact of data characteristics on performance. We conduct the experimental analysis on synthetic datasets with different data characteristics with respect to sample size and minority class prior to provide a better understanding of how data characteristics affect the performance LNR, shown in As simple conclusion, our method consistently outperforms other methods under various data characteristics, especially for larger sample sizes and higher imbalance ratios.
C.4. Class Fairness of LNR on long-tailed data
For multi-classification problems, LNR significantly improves the performance in terms overall accuracy and few-shot accuracy while maintaining class fairness. LNR flip samples are based on the similarity to the minority class, making these flips regulated and self-limiting. This prevents the model from having unintended bias towards specifie classes. To clearly demonstrate the harmfulness of LNR towards class fairness, compared the average confusion matrices of GCL and GCL-LNR..
Table Conn fusion matrix of GCL-LNR. The differences (higher than 10) compared the confusion matrix of GCL are reported in the brackets. The horizontal axis represents the predicted value, whereas the vertical axis represents the true value.
GCL-LNR
1
2
3
4
5
6
7
8
9
(Differ)
908
20
19
10
24
8
4
2
2
4
(-26)
(+13)
(+16)
978
0
1
0
1
0
0
13
(-4)
860
33
3
28
20
21
23
0
4
(-4)
782
72
23
4
50
25
22
13
2
8
(+18)
(-13)
829
12
2
54
39
16
18
25
2
2
(+3)
736
15
2
42
140
26
13
24
1
2
(-1)
820
11
6
58
68
13
14
1
1
(-4)
50
778
22
3
33
48
50
6
2
8
(-20)
(+22)
112
758
33
52
22
11
2
4
4
2
(-91)
(+80)
(+20)
30
148
792
2
0
10
(-42)
(-48)
(+102)
The bold numbers on the diagonal demonstrate that LNR achieves a more effective trade-off between the classification performance of the head class and the tail class, using only a small number of head class samples (reduced 16 true positives in head classes (,,,))) to achieve significant improvement in tail class performance (increasing 204 true positives in
18

---

Learning Imbalanced Data with Beneficial Label Noise
tail classes [7,8.9]). To provide a straightforward understanding of how LNR achieves the better decision boundary on the long-tailed CIF,R-10, we present the statistics of flipped labels below, where "-x" means' "flipped to":
{0: [->7: 2, -8: 41, ~9: 44], 1:[->9; 6]. 2: []. 4. []. 5: []. 6: D. 7: D. 8: 019: []}
The long-tailed CIFAR-10 before and after flipping labels by LNR:
Before: 5000, I: 2997, 2: 1796, 3: 1077, 4: 645, 5: 387, 6: 232, 7: 139, 8: 83, 9: 50
After: 0: 4913, 2991, 2: 1796, 1077, 4: 645, 5: 387, 6: 232, 7: 141, 8: 124, 9: 100
These beneficial label noises effectively correct the classifier's decision boundaries with minimal data editing.
C.5. Experiment datasets
Synthetic data: The simulations are conducted on the d-dimensional Gaussian distribution N(J,, E), with the mean of [4, and the covariance matrix Σ being the d x d identity matrix Iz- We generate the simulated data with the setting: N E), where l41 = 2 x 2 x 2 = different distributions generated with different statistical parameters combinations of |pl Hol E (1, d E (5, 25), and T E {..). The maximum training epoch of the ML. classifier is set to 800 as MLP converges faster on synthetic datasets. Due to the limited performance on high dimensional synthetic data with the constrained structure of MLP classifier, we instead use the KNN classifier with 50 as the flip-rate estimator Cy for LNR on d = 25 synthetic datasets.
let Pr(Y
1) = क [0,1] and X Y
0)2 -P0 E R. There are
KEEL dataset:
The KEEL repository (Derrac et al., 2015) provides us with 99 binary imbalanced datasets. We select 32 datasets with the following criteria to avoid meaningless and inefficient comparisons on the datasets:
a) Baseline already achieves an Fl-score over 90%, or all methods have an Fl-score lower than 10%.
b) Datasets with insufficient minority class sample size in the test dataset (less than 10).
The imbalance ratios To/T; in the selected 32 datasets widely ranged from 1.82 to 49.6, providing a comprehensive comparison. We simply removed all categorical attributes to simplify data preprocessing and standardized the remaining numerical dimensions.
C.6. KEEL Dataset Characteristics
As we discussed in Section 3.2, the degree of decision boundary deviation is affected by both the imbalanced ratio and the class overlap. The indicator of the imbalance ratio is less insightful as the degree of class overlap may vary largely on datasets of the same imbalance ratio. To study the methods' strengths and weaknesses, we summarise the characteristics of the dataset based on the categories proposed by (Napierala & Stefanowski, 2016), which suggests assigning each minority of the following groups based on the number of minority class neighbors within the d-nearest-neighhee Safe: 4+. Border: 2 3, Rare: 1, Outlier: O. The proportion of each group describes the distribution of minorities in the imbalanced dataset, especially Border and Rare reflecting the extent of class overlap, as a complementary metric to the imbalanced ratio.
class sample
C.7. Evaluation settings and results
Settings. We employed various seeds to generate 30 sets of synthetic training and testing datasets, with the sample size Ntrain = 500 and Ntest = 2000 for all 8 distributions. Due to the limited sample size for the KEEL datasets, we use different seeds to partition the data into 100 sets of training and testing datasets at a ratio of 7:3, ensuring at least 10 minority class samples each testing set. With 30 rounds on synthetic data and 100 rounds on KEEL datasets, we obtain statistically significant results, and the mean and the standard deviation are reported in Appendix C.7.
The performance on binary classification is evaluated in three different metrics: FI score, G-mean (Geometric Mean), and AUC (Area under the Curve). Different to the definition of F1, which mainly focuses on the true positives, G-mean also considers the true negatives with the following definition:
Metrics.
19

---

Learning Imbalanced Data with Beneficial Label Noise
TP
TN TN + FP
G-mean
FN+
Compared to accuracy, AUC evaluates the classification performance of classifier by calculating the area under the ROC (Receiver Operating Characteristic) curve. The ROC curve takes into account both recall (TPR) and FPR (False Positive Rate), defined as:
AUC AUC= S (EAI(FFRR)())e))
where FPR = FP/(FP TN). A classifier that correctly classifies positive samples as positive while maintaining low enror rate for negative samples will have higher AUC. Unlike F1 and G-mean, which are more sensitive to the prediction performance of minority class samples, AUC is more conservative metric for comprchensively reflecting the performance of a classifier. Methods that significantly increase the number of minority class samples to improve F1 score or G-mean often come at the cost of higher errors in negative samples (FP). Therefore, we desire a re-balancing method that should consider these three metrics comprehensively.
For multi-class image classification, we use different metrics evaluate the performance on step-wise and long-tailed datasets. On the step-wise CIFAR-10/100, due to half of classes removed 99% of samples forms 5 and 50 minority classes, we evaluated model by the average accuracy of minority classes denoted as Recallиi and the average precision of majority classes denoted as PRecIMA, as well as the overall accuracy. For long-tailed CIFAR-10, we report the average accuracy across Many-shot (N₂ > 1000), Many-shot (100 2 N₂ 2 200), and Few-shol (Ne < 2000), where No = [5000, 2997, 1796, 1077, 645, 387, 232, 139, 83, 50]. For long-tailed CIFAR-100, we define the Many-shot (Ne 200), Many-shot (200 2 No 2 20), and Few-shot (Ne 20).
Results on synthetic datasets. The synthetic datasets vary degrees of class overlap, imbalance ratio, and dimensional- ity. Table 8 presents the of क 0.1, d 25, un 2. Our approach consistently outperformed other methods across all datasets for both Fl score and G-mean, achieving the best AUC performance for most datasets. Through the relative performance ranking shown in Figure 9, our approach achieves the highest ranks across all metric classifiers, demonstrating its overall superiority over resampling methods on all metrics. The synthetic Gaussian distributional dataset results with classifiers are reported in Table 12, where each row represents the (average std) result 30 rounds on the distribution with different statistical settings. The win-times on synthetic datasets are reported in Table 9. The numerical results are reported in Table 10.
Results on KEEL datasets. The comparison results on the KEEL datasets are reported in Tables 13-21 in the format of (average% + std) of 100 rounds for each dataset, demonstrating the superiority of method KEEL datasets. The win-times of each methods for each classifier each metric summarized below and the highest value of each column are in bold:
In order to provide a deeper understanding of the strengths and weaknesses of our method, we calculate the relative performance difference as an extension to the relative rankings. The relative performance differences between our method and re-sampling methods are calculated by: 1) First we min-max normalize the result of all methods on each dataset and obtain the relative performances. 2) Minus the relative performance of our method to calculate the differences. We then calculate the Pearson correlation coefficients between the relative performance differences and the minority class sample distribution characteristics (safe, border, rare, outlier) is shown in Table 11. Coefficients higher than 0.25 lower than -0.25 are highlighted with bold fonts.
20

---

Learning Imbalanced Data with Beneficial Label Noise
0.55
0.78
Baseline INR (ouns
0.76
0.50
CART
CART
Over-samping
CART
Undessampiing
0.45
0.72
6000
Sample Size
Sa mple Sizze
Sample Size
0.60
0.88
0.55
0.86
K0NN
0.50
K70
0.50
0.45
0.40
0.45
0.82
Sample Size
Sample Size
Sample Size
(a) Sample Size
F1
G-mean
AUC
0.7
0.75
0.6
CART
CART
CART
0.70
0.4
0.4
0.3
0.0
0.2
0.4
0.1
0.3
0.0
0.3
Minority Class
Minority
Minority Class Prior
KONN
KNN
0.6
KNN
0.85
Baseline INR (oun))
0.4
0.4
Over sampling
Under-sampling
0.75
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.1
0.2
0.3
0.4
0.5
Minority Class
Minority
Minority Class Prior
(b) Minority Class Prior
Figure 8: The impact of data characteristics performance. For clarity, the average performance of over-sampling and under-sampling methods is reported. a) Our method demonstrates performance gains comparable to those of resampling methods when the sample size is 200. However, our method consistently outperforms the others as the sample size increases. b) As the data approaches more balanced state, all methods exhibit similar performance; notably, our method's AUC shows improvement even with more balanced data. The performance gains our method in terms of and G-mean become more evident as the prior of the minority class decreases.
21

---

Learning Imbalanced Data with Beneficial Label Noise
Table 7: KEEL(Derrac et al., 2015) Data statistics and characteristics: is the number dimensions, Ni and No denote the number of minority and majority class samples.
statistics
index
Dataset Name
d
(N, No.NN.
IR: To/
(S,B,R,O)
abalone- -ts---11-vss_---111
7
(2338, 58, 2280)
39.31
(0.02. 0.22, 0.33, 0.43)
abalone- 100_100--11-2213
7
(1622, 32, 1590)
49.69
(0.0, 0.0, 0.28, 0.72)
3
abalone9-11
7
(731, 42, 689)
16.4
(0.02, 0.4, 0.17, 0.4)
4
ecolil
7
(336, 77, 259)
3.36
(0.53, 0.32, 0.08, 0.06)
ecoli2
7
(336, 52, 284)
5.46
(0.77, 0.15, 0.0, 0.08)
ecoli3
7
(336, 35, 301)
8.6
(0.29, 0.49, 0.09, 0.14)
7
flare-F
9
(1066, 43, 1023)
23.79
(0.05, 0.26, 0.4, 0.3)
glass-0-1-2-3 vs 4-5-6
9
(214, 51, 163)
3.2
(0.67, 0.22, 0.06, 8.06)
9
glasso
9
(214, 70, 144)
2.06
(0.56, 0.36, 0.03, 0.06)
10
glassl
9
(214, 76, 138)
1.82
(0.46, 0.3, 0.16, 0.08)
11
haberman
3
(306, 81, 225)
2.78
(0.05, 0.47, 0.32, 0.16)
12
3
(2935, 2854)
35.23
(0.62, 0.22, 0.09, 0.07)
13
tr-ws-k-zenoomee_vs.de
3
(2901,
26.63
(0.65, 0.23, 0.06, 0.07)
leddddigi--022445567788....
7
(443, 37, 406)
10.97
(0.19, 0.65, 0.14, 0.03)
15
newthyroid2
5
(215,35, 180) (5472,
5.14
(0.69, 0.31, 0.0, 0.0)
16
page-blocks0
10
4913)
8.79
(0.7, 0.07, 0.06)
pima
8
(768, 268, 500)
1.87
(0.3, 0.45, 0.17, 0.09)
18
vehiclel
18
(846, 217, 629)
2.9
(0.24, 0.57, 0.15, 0.05)
19
vehicle3
18
(846, 212, 634)
2.99
(0.16, 0.51, 0.26, 0.06)
20
yeaxst-0-2---6vs.3-11
8
(1004, 99, 905)
9.14
(0.35, 0.3, 0.15, 0.19)
21
yeasst-02-557---.ss.333
8
(1004, 19,905
9.14
(0.68, 0.17, 0.05, 0.1)
22
yeast-0-3-5-9.vs v--788
8
(506, 456)
9.12
(0.16, 0.28, 0.22, 0.34)
23
yeast-0-5-6-7-9 vs.
8
(528, 51,
9.35
(0.08, 0.43, 0.18, 0.31)
24
yeast-1-2-8-9.vs.7
8
(947, 917)
30.57
(0.0, 0.27, 0.23, 0.5)
25
yeast-1-4-5-8 vs
8
(693, 663)
22.1
(0.0, 0.07, 0.43, 0.5)
26
yes.t-l-vs.7
7
(459, 429)
14.3
(0.07, 0.33, 0.27, 0.33)
27
yeast-22vs_4
8
(514,51, 463)
9.08
(0.55, 0.22, 0.06, 0.18)
28
yeastl
8
(1484, 429, 1055)
2.46
(0.21, 0.45, 0.22, 0.11)
29
yeast3
8
(1484, 163, 1321)
8.1
(0.56, 0.26, 0.08, 0.1)
30
yeast4
8
(1484, 51, 1433)
28.1
(0.06, 0.35, 0.2, 0.39)
31
yeast5
8
(1484, 44, 1440)
32.73
(0.34, 0.5, 0.11, 0.05)
32
yeast6
8
(1484, 35, 1449)
41.4
(0.37, 0.23, 0.11, 0.29)
22

---

Learning Imbalanced Data with Beneficial Label Noise
Table Results on synthetic data with n1 0.1, d 25, jAD
(average +-std).
Over-sampling
Under-sampling
Metric
Classifier
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
Fl
KNN
15.72-0005
40.88+0.04
31.63+00.03
31.15+0.03
33.4 +0.03
24.94+0.07
27.1 +0.07
35.74+0.04
CART
43.72-00005
49.52-+0033
40.56+00.05
39.48+0.055
40.51+0.05
42.59+0.06
36.3 +0.04
38.94+0.05
MI.P
44.32-0000
0.082++0.03
42.08+00.0
42.35+0.04
14.01+0.00
43.39+0.04
28.84+0.02
30,44+0.04
G-mean
KNN
23.89
0..2
0..86:10.22
37.5+10.02
38.28 +.03
29.07:1.06
33.54:10.06
41.48:00.03
CART
43.88=0.05
50.5 =0.03
41.08=00005
39.86=0.05
40.79+0.05
43.07=00.05
43.73=0003
44.29+00.05
MLP
44.55=0.004
48.87:10.03
42.95==0.05
42.62+00.04
44.18=0.04
43.62:+0.04
35.61=0.02
36.26+#0.04
AUC
KNN
71.06=0.03
81.53+0.03
73.98+0.02
73.52+00.3
74.5 +0.02
73.33+0003
74.41=0003
80.29+0003
CART
69.04-0.03
75.58+0003
68.67-+0033
0..83
67.96+0..44
69.72+00.3
75.77=0.03
75.45+00.04
ML.P
82.52-0.03
84.6 0000
79.94+0.04
79.77+0.04
80.25+00.04
81,13+0.03
73.5 +0.03
74.01+0.05
KNN F1-score
CART
MLP F1-score
F1-score
Daseline
1.0
1.0
1.0
SMOTE
Borderline
0S5
0.5
ClusterCentroids
G-mean
AUC
G-mean
AUC
G-mean
AUC
Figure 9: Relative ranking of methods from 1 (the best) to (the lowest) on 8 synthetic datasets.
Table 9: Win-times comparison on 8 Synthetic datasets 32 KEEL datasets.
KNN
CART
MLP
methods
F1
G-mean
AUC
F1
G-mean
AUC
F1
G-mean
AUC
baseline
0/ 2
0/
0/
0/ 4
0/ 4
0/
0/
0/
2/13
LNR
8/16
6/16
7/9
8/21
8/18
8/13
8/19
8/16
6/5 5
SMOTE
0/ 5
0/
3
0/
0/
0/
0/
0/7
ADASYN
0/3 3
0/
0/
0/
O/ 2
0/2
0/3
0/3
0/2
Borderline
0/ 4
0/4
0/
0/
0/ 1
0/
0/ 4
0/4
0/2 2
OSS
0/ 2
0/2
0/ 0
0/
0/ 1
0/2
0/
0/2
0/ 1
CC
0/ 0
0/
0/19
0/0
0/
0/2
0/ 0
0/3
0/ 2
RUS
0/ 0
2/0
1/
0/ O
0/
0/11
0/
0/
0/
23

---

Learning Imbalanced Data with Beneficial Label Noise
Table 10: Relative performance ranking on 8 synthetic and 32 KEEL datasets
Over-sampling
Under-sampling
Classifiers
Metrics
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
Synthetic Gaussian datasets
Fl
0.08
1.
0.57
0.47
0.55
0,42
0.44
0.74
KNN
G-mean
0.07
0.98
0.61
0.52
0,57
0.35
0.53
0.81
AUC
0.17
0.13
0.28
0.7
FI
0.15
1.
0.16
0.11
0.13
0.23
0.33
CART
G-mean
0.09
0.99
0.11
0.05
0.06
0.18
0.5
0.49
AUC
0.06
1.
0.08
0.03
0.03
0.15
0.51
0.49
F1
0.34
1.
0.53
0.41
0.44
0.5
0.51
0.47
MLP
G-mean
0.3
0.99
0.49
0.37
0.39
0.44
0.61
0.57
AUC
0.71
0.99
0.29
0.15
0.2
0.58
0.44
0.38
KEEL datasets
FI
0.35
0.86
0.69
0.59
0.75
0.53
0.32
0.37
KNN
G-mean
0.33
0.84
0.65
0.53
0.7
0.5
0.4
0.43
AUC
0.42
0.7
0.34
0.2
0.22
0.79
0.66
Fl
0.58
0.93
0.56
0.51
0.54
0.56
0.19
0.39
CART
G-mean
0.5
0.89
0.46
0.37
0.43
0.47
0.29
0.5
AUC
0.31
0.35
0.48
0.43
0.72
Fl
0.36
0.91
0.72
0.65
0.75
0.48
0.34
0.35
MLP
G-mean
0.35
0.9
0.69
0.61
0.73
0.44
0.41
0.44
AUC
0.81
0.8
0.64
0.46
0.46
0.45
0.41
0.39
24

---

Learning Imbalanced Data with Beneficial Label Noise
Table 11: Pearson Correlation between data distribution characteristics and relative performance differences
Over-sampling
Under-sampling
Metries
Characteristies
baseline
SMOTE
ADASYN
Borderline
OSS
CC
RUS
classifier
Dimension
0.3
0.16
0.01
0.08
0.1
-0.13
-0.12
IR
-0.03
-0.08
-0.14
-0.13
0.16
0.17
0.34
Safe
-0.77
-0.18
0.06
-0.03
-0,71
-0.06
0.04
Fl-score
Border
0.26
0.16
0.04
0.17
0.26
0.18
-0.03
Rare
0.68
0.08
-0.13
0.02
0.68
0.01
-0.12
Outlier
0.48
-0.04
-0.11
0.39
-0.05
0.04
Dimension
0.32
0.14
-0.01
0.06
0.12
-0.1
-0.1
IR
-0.02
-0.07
-0.1
-0.08
0.09
0.04
0.13
Safe
-0.76
-0.23
-0.02
-0.12
-0.63
0.09
0.22
G-mean
Border
0.3
0.16
0.04
0.12
0.31
0.25
0.08
Rare
0.69
0.15
-0.06
0.13
0.63
-0.16
-0.32
Outlier
0.02
-0.01
0.27
-0.19
Dimension
0.1
0.31
0.24
0.31
0.21
0.27
0.47
IR
0.49
0.21
0,05
0.1
0.3
-0,23
-0.18
Safe
-0.56
-0.53
-0,33
-0.43
-0.57
-0.28
-0.3
AUC
Border
-0.06
0.23
0.17
0.22
0.1
0.31
0.22
Rare
0.2
0.6
Outlier
0.2
0.27
0.38
-0.03
0.08
with CART classifier
Dimension
0.4
0.29
0.26
0.27
0.31
0,01
-0.15
-0.15
-0.16
-0.15
-0.13
0.18
0.48
Safe
-0.25
-0.59
-0.57
-0.42
-0.14
-0.26
F1-score
Border
0.23
0.27
0.18
0.21
0.05
-0.04
-0.05
Rare
0.32
0.63
0.59
0.42
0.16
0.07
0.15
Outlier
-0.03
0.24
0.3
0.17
0.06
0.17
0.32
Dimension
0.41
0.27
0.24
0.26
0.33
0.03
-0.07
IR
-0.32
-0.06
-0.08
-0.12
-0.05
Safe
-0.27
-0.66
-0.6
-0.49
-0.21
0.28
0.24
G-mean
Border
0.32
0.31
0.22
0.23
0.14
0.24
0.31
Rare
0.29
0.63
0.53
0.46
0,21
-0.36
-0.33
Outlier
-0.05
0.31
0.37
0.24
0.06
-0.38
-0.4
Dimension
0,36
0.32
0.14
-0.24
-0.04
-0.05
-0.07
0.02
-0.4
-0.56
Safe
-0.03
-0,58
-0,41
-0.36
-0,2
0.17
0.29
AUC
Border
0.16
0.23
0.1
0.14
0.12
0.22
0.26
Rare
0.04
0.52
0.36
0.32
0.27
-0.1
-0.19
Outlier
-0.13
0.31
0.29
0.2
0.01
-0.37
-0.52
MI.P classifier
Dimension
0.28
0.04
0.11
0.12
0.31
0.01
0.02
IR
-0.05
0.08
-0.02
0.03
0.03
0.14
0.3
Safe
-0.75
-0.05
0.11
0.02
-0.46
0.05
0.04
F1-score
Border
0.38
-0.24
-0.29
-0.2
-0.16
-0.09
-0.09
Rare
0.63
0.13
-0.03
0.06
0.48
-0.03
-0.16
Outlier
0.2
0.09
0.1
0.5
0.02
Dimension
0.31
0.15
0.14
0.15
0.29
0.03
-0.01
IR
-0.04
-0.02
40.04
0.03
0.02
0.01
0.15
Safe
-0.74
-0.11
0.06
-0.03
-0.48
0.18
0.18
G-mean
Border
-0.18
-0.26
-0.21
-0.01
Rare
-0.02
0.09
0.45
-0.16
-0.32
Outlier
0.21
0.13
0.16
0.49
-0.14
-0.06
Dimension
-0.25
-0.18
-0.15
0.01
-0.11
0.03
0.1
IR
-0.07
0.42
0.35
0.28
-0.08
0.12
0.39
Safe
0.05
-0.5
-0.17
-0.27
0.13
-0.56
AUC
Border
-0.21
-0.28
-0.38
-0.36
0.2
0.22
Rare
0.05
0.47
0.25
0.35
-0.2
0.36
0.35
Outlier
0.08
0.67
0.39
0.47
-0.22
0.21
0.44
25

---

Learning Imbalanced Data with Beneficial Label Noise
Table 12: Comparison on synthetic datasets with KNN, CART and MLP classifiers.
Settings
Over-samplin
Under-sampling
Metric
baseline
ENR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
with KNN classifier
0.1.1,5
8.54::0.03
2..8
22.05:+0.02
21 21.5::0.02
23.24::0.03
12.85:10.04
12.85:0.044
24.94:10.02
0,1,1,25
20+0001
21.8+0.03
19.47++0.01
19:5+0.01
19.86+0.02
5.19-0.03
5.19+003
20.73+00.22
0.1,2,5
43.38=00066
55.2::0.04
44.21=20.05
42 12.68:00.04
48.07:0.04
47.76:20.05
47.76::0055
44.86:20.04
0.1.2.25
15.72=0.00
40.88.1.04
31.63:10.03
31.1520.03
33.420:03
24.9110.07
24.9410.07
35.74.1.04
F1-score
0,3,1,5
39.7:00003
54.19:0.02
47.45::0.02
47 47.23:20.01
46.67:0.02
45.99:00.02
45.99=0.02
50.38:00.2
0,3.125
27.61=0.03
48.88.0.02
45.12::0:02
45.110001
45.08.10.01
36/19.10.03
36.4910.03
44.05
0,3.2,5
71.14=00.03
73.64:0..02
70.6:20.02
67.82:
68.3:10.02
72.66:10.02
72.66::0.02
72.58.1.02
0,322,25
54.75-0.04
66.1+9002
60.11+0..02
58.13+0..2
58.3+00.2
60.91+0.04
60.91-0.04
64.6++0.2
0.1.1,5
12.12=0.04
31.26:20.03
24.57:00.02
24.07=00.0
24.45=0.033
14,83:0.044
3.086:00022
31.4=:0.03
0,1.1,25
3.9810.02
23.87.10.03
25.43:10.02
25.69110.02
24.29
6.97
8.22.10.04
2,04
0.1.2,5
46.52=0.05
56.04:0.04
46.8=00.03
45.28=0000
49.29::0.44
49.04:10.05
51.12:0003
50.49+
0.1.2:25
23.89-0.05
03.28:10.03
37.86.10.02
37.510002
38.28.10.03
29.07.1.006
33.54100.06
41.40
G-mean
0.3.1,5
40.5+0.02
0..2
48.2=00.22
48.42:0.01
4..5
0..09
54.36=00.02
51.63-00.2
033,1225
29. 0.00
51.57+0.003
49.19:0003
49 02:00.222
48. 08.73+00.02
36.73++0.05
31 08-0000
45, 03.344+0.3
033.2.5
71.32=0.03
74.15::0002
71.07:00.02
68.76:0002
69 69.08:0.02
72.83:10.22
12.62:00011
73.3=00.2
0,3.2,25
56.22=00.03
67.83.10.02
63.71
62 63:20.00
62.42.00.22
61.16:10.04
59.37.0003
65.98.00.3
0,1,1,5
60.2+0.02
69.2+-0.22
60.18+00.02
19.94-0.02
60.98+0.02
60.36+0.003
68.02-00.33
66.35+0.03
0.1,1,25
$4.93=0.02
61.36:20.03
55.74+0.02
55 05:0020
56.33-00.03
55,75=#0022
57 03:33
09.14
0,1,2,5
81.19-0.02
87.1+00.3
80.69+0.02
79.74-0.02
80.93+00.22
82.34+0.02
89.67-0.01
16+000
AUC
0.1.2.25
71.06=0:03
01.53+0.03
73.98=00.02
73.52=0003
74,5=00.02
73,33+00.05
74.41=0003
00.29=0.03
0,3.1,5
65.27=0.01
0.054::0.02
63.95:00.2
62.81=00.02
63.11
65.92:10.01
67.96:0.02
6..05
0.1.1.25
57.36-0.01
6..5410.02
56.55.0003
55.1210.002
55.8710.02
57.710001
58.910.01
59,2221.002
0.3.2,5
87.83-002
89.63+00011
86.49+00.02
84.21+0.01
84.06+00.02
88.35++0.02
89.38-0001
89. 89.34+00001
033,2,25
78.82:0002
85.49:0.01
81,36:00.02
79.58:20.03
79. 79.64:00.3
79.79::0:03
80:22:0002
00.02
with CART classifier
0.1,1,5
19.13=0.04
2..94:0.03
19.22:40.02
19.5:-0.33
18:83:0.03
19.22-40.03
22.21 23.07
0.1.125
16.3110.03
24.0710.03
17.6200.3
17.02:10.03
16.7.00.33
17.3:0.02
21 21.66:10002
21.29
0.1,2,5
44.77=0.05
52.49:0.004
44.49:10.05
42.96-10.06
44.43:40.05
43.96:10.05
41.99:00.66
39.6:+0.05
0,1,2,25
43.72-0.05
49.52+0003
41.56+0.055
39 49-00-0
40.51+0005
42.59++0.66
36.3+0004
38 38.94+0.05
F1-score
033.1,3
41.07=0002
52.0::0.02
42.24:0.02
42.64=00002
42. 42.16:20.02
43.98:10.02
45.33::0.02
46. 14:0022
0.1.1.25
40.11=0.02
50.0110.00
40.53100.33
40.841:0.02
40.48:10.03
41.32:1.02
45.381.0.02
44.6110.02
0.3.2,5
65.66-00.02
71.68:10.02
66.03-10.02
65.0.10.2
65.93.10.02
66.88.10.02
6..32::0.2
67.12.10.33
0.3.2.25
64.18=0.02
69.710002
64.0210.02
63.391.0.02
63.38:0.03
64.7510.02
64.6310.02
65.3710.02
0.1.1,5
19.22-0.04
30.48+0.03
19.76+00.02
19:91+0.03
19.05+0.:33
19.58+00.03
28.41-0.03
28. 28.39+0.02
0.1.1,25
1642=0003
27.33++004
18.09:0003
17.48:0003
16.98=0.03
17.81:0.22
0.068:0002
26.68:00003
0,1,2,5
14.91=0.05
53.83:10.03
44.97:10.05
43.49:10.06
44.68:20.05
44.1110.05
46.82:0005
45.28
0.1.2.25
43.88-0.05
50.5:10.03
41.08:10.05
39.56100.05
10.09
43.07:00.55
43.73110.03
40.29
G-mean
0,3,1,5
41.11=0.02
$3.42+0.02
42.38:+0.02
42 -2.88-0.02
42.26+00.22
44.29+40.22
46 46.29=00.2
4..29+00.03
03.1,25
40.15=0.02
5..55=0.00
40.73+00.0
41.01=00.0
40.63-00.03
41.57:0003
46.55:0002
40.01
033.2.5
65.6920.02
72.29:0.02
66.13:t0.02
65.13:20002
66.02:0002
67.09:10022
66.86:00.02
67.73:20.33
03322,5
64.21=0.02
70.24:0000
64.15:+0.02
63.51 63.51:00.02
63.51=0003
64.92:+0.2
65 36:30.0002
08.800002
0.1.1.5
55.11=0.02
6..86.10.02
55.06:10.02
55:22110.02
54.8910.02
55.11:0.02
59.510003
58.7410..3
0,1.1,25
53.4400.02
60.01:10.03
53.9110.02
53 0.0002
53.41
53.66:10.02
$8.240.03
57.3-10.03
0.1.2,
69.66-0.03
78.42+0.03
71.22++0.03
70.35+0.04
70.26+00:03
70.93+0..04
17.12+0..3
76.46+00.3
0.1,2,25
69.04-0.03
75.58:00.03
68.67:20.03
67 08:00003
67, 0.00
69.72::0.05
76.77:00.03
75. 75.45$0..04
AUC
0.3.1.5
$7.62-0.01
64.11::0001
57.81110002
57.7620.01
57.93110.2
58.6410.02
58.3611000
58.8710.02
0,3.1.25
57.010.02
62.16:10.02
56.18:10.02
56 56.58
56.31
56.79.10.02
58.06:10.02
57.63-10.02
0.3.2.5
75.54=0.02
80.9810.02
76.02:002
75.25100.0
75.8910.02
76.77.1001
76.610.01
77.32:10:2
0,3.2,25
0.04
7..31:1001
74.51:00.02
74.03::0.02
7..02:0.002
75.09:10.02
75.32::0.2
75.8110.02
with MLP classifier
0.1.1,5
7.9
3..36:20003
23:0:00.33
22.91:0:3
23.04:20:3
14.79:10.09
27.05=0.02
05.66120.02
0,1.1,25
16.1100044
24.66.10.02
18.99.10.03
19.52:10.03
18.2910.00
16.97.10004
21.31
20.96.1.02
0.1.2,5
54.57=0.04
0.04
47.51++0.04
45.43=00.05
50.95:0.05
55.24:10.04
48 06:0044
0.
0.1,2,25
44. 44.32-0.05
47.82++0.03
42.68:+0:05
42.35 42.35=00.04
44.01
45.39-:0.04
28. 04-00.2
30 30.44++0.04
F1-score
0.3.1,5
45.19=0.03
54.57:0.02
51.84:20:03
52.37=0.02
51.93:1..02
51.13:1003
03.6622002
03.04
0331122
38.83-0.03
49.7+0002
43.2=00.02
43 64-40.02
42.93-+0.02
41.73++0.22
44 64:00002
44, 02.002
0.3.2.5
74.34=0.02
74.87::.02
74.62:0.02
72.14::0.2
12:00:0002
74.76:10.02
74.58:0.02
74.48:00001
0.3.2,25
64.5100.33
68.61:10.02
65. 15:10220
64.711:0.02
64.67100.3
64.07.10.05
62.69110.02
62.62210033
0.1.1,5
10.57-0.09
31.85+0.03
25.22-+0.03
25. 30-00-0003
23.94+0003
16.82-+0.08
32.26-0.02
31,53+0022
0.1.1.25
17:03=0:04
27.33++0.3
19.5=00.03
20.15:40.03
18.57=0.3
17.21=20044
27.93=:0.02
2..22=0002
0.1.2.5
55.74=0.04
58.02:10.04
49.29:10.04
47. 17.57110.05
51.8620.00
55.77:0.004
52.72:0003
50. 50.13100.3
0.1.2,25
44.55-0000
48.0710.03
42.95:10.05
42.62.10.04
48.10:10.04
43.62.10.04
35.61
36.2610.044
G-mean
0.3.1.5
16.37=0.03
0..3
52.45:10.03
53.42:10.02
52.7620.02
51.31:10.03
51.6410.02
14.75
0.1.1.25
18.97=0..33
51.0110.022
43.44:10.02
44.01
43.15.10.22
41.88.10.03
45.6311..02
45.6910.03
0.3.2,5
74.48-0.02
75.08+-0.02
74.83++0.02
72.85+00.22
72.76+00.2
74.82+0.02
74.88-0.02
74.8+00.1
033,2,2
64.56=0003
69.14:0.02
65.23:+0.02
64.82:0.02
64.1:00033
64.22:40.03
63.53:0002
63.01
0.1.1.5
714110033
71.3110.03
62.85:10.04
63.041.0.04
64.6110.04
69.73.10.04
69.3210.03
67.45
0.1.1,25
61.51=0.04
6..07:0.03
58.48:10.04
58.51:0.03
57.73:0004
59.13-10.04
60.73::0.03
59.01
0.1,2,5
90.28-0.01
89.9+0.02
85.39-+0.13
84.09900.04
86.77+0.03
89.65++0.01
98-72-0.02
87.74+00.2
AIC
0.1.2.25
82.52=0.03
84.6::0.02
79.94:0.04
79.77::0.04
80.25=00.44
81.13:10.03
73.5=:0.3
00.01
0,3,1,5
73.19-0.02
73.24+0.02
70.83+00002
70.01+4.02
70. 15+000002 2
73.0++0.22
12.13+00.2
72.25+0.02
0.3.1.25
59.87=0.03
65.94:0001
60.88:00.02
61.04120.02
60.76:0.02
60.62:10.03
60.28:00.2
60.61+0.05
0.1.2.5
91.210.01
91.22:1.011
90.68.10.11
89.6410.02
89.3410.01
9100110001
90.910.01
90.7910.01
0.3.2.25
81.84-0003
85.69:1.002
81.81-10.02
81.03.10.03
$1.41
0.8.:0.03
80. 80.56-00.22
80.42.10.03
26

---

Learning Imbalanced Data with Beneficial Label Noise
Table 13: F1 score with KNN classifier on KEEL datasets
Over-sampling
Under-sampling
Dataset
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
9.75+10009
39.12:10.08
33.1810.006
32.05:10.06
34.72:1.007
15.63-10.09
25.67:10.05
19.22:10.0K
0.17+00.2
15.19--0.07
13.13+0.06
13.17+0.06
11.02+0.08
0.35+0.02
8.75+00.3
7.34++0.02
26.11=0.11
43.89:10.11
36.06:10.07
35.0+0.07
42.4910.1
31.88:10.13
33.7910.07
29.54=10.05
0..5
78.29-10.06
79.57.10.05
17.64
78.6...055
78.47-10.06
78.22:10.05
18.13:10.05
88.72=0.05
88.57=00.05
82.39+00005
77.62+0.06
80.75+0.06
86.17=0.06
84.21+0.06
76.32=00.08
60.99+0.1
69.53++0.08
62.69+0.08
61.12+0.07
63.1
66.67=0.09
58.27+00.8
56.72+10.08
23.2210.12
32.73.10.08
29.610.07
29.7210.07
29.7410.07
22.68.1.112
16.58.0.04
23.3110.05
08.36+00008
80.06+00007
85.53+0005
85.84+0.05
85.25+0.06
83.59+0.08
82.86+0.06
81.98+0.06
72.1
72.17:00.05
73.4710.05
72.37+0.05
72.85+0.05
71.19=0.06
66.9810.05
70.93
10
65.72+40.07
68.51400.06
68.52-00.06
69.65
69.43+0.06
66.27
65.31
67.79+0.06
35.23+0.07
47.75++0.07
43.48+0.06
43.54
43.12+0.06
37.7=2008
39.29:10.07
45.54
12
65.11=0.09
63.66+00.08
72.64
65.0
71.17=0.07
64.27+0.1
29.04+0.04
28.22++0.05
13
77.0110.06
60.08
78.78.10.07
60.1510.055
62.4910.05
56.62.10.2
44.2310.04
38.7910.07
14
77.05+0.009
77.45+00007
67.07+00.09
68.49+0.1
68.4640.12
54.9+0.28
54.44+00.07
59.83+0.14
86.2110.09 15
89.02
94.2510.05
93.8310.05
93.7210.05
90.63110.06
91.77110.05
92.2+00066
82.72+0.02
83.35+00.2
81.71+0002
7.09
8.066+0002
83.09+0.02
79.22+-0.22
77.2140.02
17
60.3620.04
66.88
63.54
63.5810.03
63.37=0.03
62.02
61.63:00.04
64.05=00.03
18
52.66.10.05
64.24
61.62.10.03
61.8310.03
61.9
57.65
60.9110.4
60.57-10004
19
47.38
63.13.10.03
58.4110.00
59.71
58.79
53.64.00.05
60.11
59.01
20
57.96+00007
6..61
49.7940.04
45.57+0004
55.17
59.6+0.07
42.41+00.06
48.41
83.07
82.7410.00
69.1210.04
66.110.04
76.6510.04
83.011.05
72.0210.06
71.1510.05
22
05.8
42.58+0.09
37.48+0.06
36.5+0.06
41.16+0.07
37.73+0.1
34.31+0.06
34.85+0.06
23
33.27+0.14
47.7640009
45.610006
44.28:10.06
48.5210.07
46.52+0.1
41.28110.6
42.6940.05
24
5.48
18.46-10.09
23.88
20.05
L0.48
11.35
15.24
15.34
25
1.52.0.05
17.42-0.008
19.68
20.2
18,29.0.09
4.21
14.44.10.03
13.9
13.4+0.14
33.244+.11
38.3240.08
35.1240.09
44.620.09
28.01
31.59+0.07
29.22+0.07
27
15.8910.07
77.0.10.07
73,910.07
72.5810.07
74.410.08
79.32.10.06
13.97
75.16
53.73+0.03
59.79+00.02
56.92+0.003
56.41+0.02
56.83+0.02
54.61+0.03
57.76+0.002
57.99-0.03
29
75.6610.04
77.96:10.04
70.65+:0.04
68.7110.04
69.7310.04
76.6410.04
63.95.10.04
66.940.04
30
18.3940.1
39.83+L0.09
35.38
35.33+00007
41.48-0.088
27.62-40.12
25.04
26.66+00.04
63.81+0.09
69.99-00.77
68.9540.06
69.3110.07
68.4+0.07
66.51
43.8710.05
44.28+00.66
54.57+0.1 32
56.62#0..1
40.29+0.06
39.92+0007
52.68
57.75+1011
24.68+00.5
26.98++0.05
27

---

Learning Imbalanced Data with Beneficial Label Noise
Table 14: G-mean with KNN classifier on KEEL datasets
Over-sampling
Under-sampling
Dataset
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
16.810.13
39.98:10.08
35.02:10.06
33.6410.06
35.2110.07
22.78:10.11
33.9:10.04
29.95-10.04
0.3+-0.03
17.72-40.00
15.47+0.07
15.52+0.07
11.33+0.08
0.62+0004
16.2+0004
15.69+00.03
35.35=0.15
05.11=0.11
37.61=0.07
36.57+0.08
43.2+0.1
38.32::0114
36.68:0.07
34.93-10.06
78.02.10.06
1..64
7.09
78.1910.055
79.11..05
78.69-10.06
78.83:10.05
78.78-10.05
88.81=0.05
88 08.6
83.0
78.55+0.05
81.38=0.05
86.49=0.06
84.75=0006
77.84=0.07
62.17=0.1
70.18+00.08
64.55+0008
62.69+0.07
64.34+0.08
67.19=0.09
62.77+0006
61.79++0.06
26.92.10.13
34.65.10.08
30.2610.07
30.3910.07
30.4910.07
27.1410.12
28.36.10.04
33.2110.05
79.32+0.07
80.48+0.07
85.69+0.05
8..03+0005
85.45+0.06
83.95+0.07
83.33+0.06
82.19+0.06
72.54+0.05
73.13+0..55
74.5610.05
73.77:0.04
74.3400.4
72.03
70.5410.04
72.52+0.05
66.4+1007 10
69.0+-0.06
68.77-00.06
70.14
69.78
66.71
66.65
16.06
07.84+0.07
48.1440.00
43.93+0.07
44.18=0.007
43.5+0.06
38.22-0007
40.41+:0.08
46.43+00.77
12
66.82 08.8
64.25+0.008
03.29
65.47=00.8
71.97=0.07
65.6=0009
40.94=10.44
40.01
13
78.9110.05
67.410.07
79.76.10.07
64.25110.04
66,210.04
19.88
52.64.10.03
18.06
14
77.38+0.00
77.69
68.46+0.08
69.
69.46+0.11
56.93+0.26
60.03+00.06
63.78+0.11
87.2410.08 15
90.3
94.3810.05
94.05.10.04
93.9110.05
90.98110.06
92.0310.005
92.43:10.06
82.9
83.4+0.02
82.06+0002
80.23+0.02
82.83+0.02
83.22+0.02
79.75+0002
78.2940.02
17
00.61
67.69+0.03
63.83+0.04
64.18210.33
63.83+0.03
62.1=0.04
61.8410.04
64.4620.00
18
52.8
65.51
63.1410.03
63.75100.3
63.62
57.85
62.39
62.75.10.04
19
08.4410.05
64.48
59.5110.04
61.26.10.03
59.98
53.79
61.01
60.92
20
0.00
6..87
52.3+0.04
48.0740.04
56.01
60.92+0.07
47.18+0.04
51.33+00.05
83.32.1.004
82.8310.04
70.9110.04
68.0110.04
77.0910.04
83.1710.05
73.54:10.05
72.7310.04
22
01.13+0.09
03.72+0009
39.69+0.06
38.7740.06
41.99+0.07
41.09+0.09
40.14+00.06
39.82+0.06
23
38.2+0.14
48.45
47.46+0.07
46.01:.06
49.31
48.
47.4+10005
48.01
24
9.2--0.15
10.45-1.11
27.28
24.81
31.51-0.11
17.44
24.3
22.71+10.05
25
2.66
18.92
22.55.10.07
22.99-0.007
18.76.1.099
6.78
24.16.10.04
22.47-10.04
10.3440.19
34.85+0.11
40.83+0.08
37.38+0.09
45.5
35.28
37.91
35.26+0007
27
17.6910.06
17.68
74.43
73.05
74.7210.07
80.06
74.93
75.75
54.17+0.03
61.14+00.03
57.53+0.03
0.04
57.62+0.02
54.75+0.03
60.98+0.02
58.96+00.33
29
76.07-0.03
78.14410.04
71.8640.03
69.8311.04
70.42+0.04
76.82-1004
67.64-10.33
69.79
30
15.6440.11
40.62+00.09
37.77+0.007
37.52+0.07
42.05
33.46
35.17+0.004
36.27+0.03
65.42+00.09
70.98+10.06
70.8740.06
70.410.07
70.0910.06
67.23+0.08
52.9810.04
53.26+00.05
56.51+0.1 32
57.36#0..1
44.41+0.06
43.7440007
54.07+0.09
58.86-1011
35.55+0.04
37.02+0.05
28

---

Learning Imbalanced Data with Beneficial Label Noise
Table 15: AUC with KNN classifier on KEEL datasets
Over-sampling
Under-sampling
Datasct
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
76.65:10.06
87.59:10.05
76.36:10.05
76.22:10.05
74.72:10005
75.1411..05
88.8:10.33
87.88110.04
61.62+00.07
72.77++0.08
63.32+0.07
63.33+0.07
6..68+0006
61.02+0.06
67.22+00.08
66.47+0007
73.9=0.06
80.04:0.06
73.46+0.05
73.11+0005
74.02=0.06
75.72=0007
79.71=0.00
79.37=00005
93.26:10.03
92.75-10.03
9..55.10.03
91.46.10.03
91.4911.03
93.55.10.33
94.37:10.02
93.58-10.03
95.41=0.03
95.87=00.03
94.81+0.03
93.51#0.03
93.79=0.03
95.98=0.03
96.22++0.03
95.48=0.03
90.39+0.04
92.06+00.04
89.44+0.04
88.31=0.04
88.45=0.04
91.58=0.04
93.11++0.03
92.34+0.04
76.7310.07
77.510.07
74.8410.006
74.910.06
75.3910.05
69.66110.05
75.12.10.06
84.29
94.9+0.03
95.52+0.02
95.36+00.03
95.32+0.03
95.54+0.03
94.91+0.04
96.3++0.03
95.1+0.03
86.41+0.00
85.4610.04
86.6610.04
86.15
86.0310.04
84.92=0.04
83,9210.04
84.24410.04
10
82.17=0.05
82.16
82.08+00.05
82.04
82.0+0.05
79.83
80.12
81.85
63.83+0.05
67.2140.05
62.14
61.6+1005
62.16=0.05
61.79=0005
55.33+10006
62.51
12
93.88=0.03
95.36+00.3
95.13+0.03
93.43+0.03
94.87=0.03
93.87=0004
97.56:0.01
96.57=0.01
13
93.4710.05
93.28110.05
93.1910.04
96.3310.02
95.3910.03
87.32.0.04
97.32.0.01
97.02.10.01
14
90.2+0.04
90.37+0004
89.0+0.04
89.07+0.04
89.12+0.04
86.4+0.13
95.45+00.33
93.35+0.05
15
09.57
99.41
99.5910.01
99.5210.01
99.4110.01
99.48110001
99.5810.0
99.57
95.53+0.01
95.74+0.01
95.52+0.01
95.18+0.01
95.32+0.01
95.69
97.6+-0001
96.16+0.01
17
77.72=0.03
79.38
75.78
75.1810.03
75.05=00.33
76.12=00.3
75.35=00.03
76.92-00.3
18
81.74.0.00
82.9410.02
80.7711.02
80.77
80.5710.02
80.83
80,53
19.01
19
81.26.10.02
02.02
79.4710.02
79.36
79.0710.02
79.85
81.42.10.02
79.99 10.02
20
03.3
84.17+00.03
82.44+0.03
80.3+0.03
82.63+0.03
82.79+0.03
83.24+0.04
82.46+0.00
93.8410.02
93.9911..02
92.9410.02
91.8410.02
92.9610.02
93.8311.02
94.3210.02
93.7810..2
22
02.94+0.06
75.18+0.06
72.61+0.05
71.68+0.05
72.9+0.05
72.01+0.06
75.55+00.05
74.21+0.05
23
78.46:0.06
80.9440.05
78.03+0.04
77.1120.44
77.18
80.21
85.6110.03
84.9440.04
24
68.99-10.07
73.35
70.6110.07
70.22
69.37+0.07
69.02
75.72.10.07
10.09
25
63.51
64.99
64.61
64.7.10.07
64.06
63.14
67.07
64.01
73.91
77.12+0.05
74.1+0.07
73.1640.06
74.68
73.81
78.95+00.06
76.07+0..6
27
91.62.10.04
93.15.10.03
92.6100.33
91.4710.03
90.910.03
95.38
97.05
96.3710.02
78.04+0.02
78.67+0.02
74.32+0.02
73.0++.02
73.06+0.02
75.61+0.02
76.02++0.02
75.8+0.02
29
93.53+0.02
95.52-0.002
92.83+0.02
91.400.02
91.2210.02
94.22-10.22
96.67-10.01
96.51
30
78. 14400066
82.78-L0.05
77.35+00.05
+7.29
76.76+00.05
78.05
90.18-10.03
89.79+0.03
96.56+0.03
98.27:10.02
96.48+0.02
96.3910.02
96.3810.03
97.1410.02
98.0910.01
97.71
86.83+0.05 32
88.55+00.0
85.65+0.05
85.42+0.05
86.2+0.05
86.96+00.05
92.84+1.04
91.89+0.04
29

---

Learning Imbalanced Data with Beneficial Label Noise
Table 16: F1 score with CART classifier on KEEL datasets
Over-sampling
Under-sampling
Datasct
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
28.6110.09
35.51110.07
24.22:10.08
23.3610.07
24.12:1.09
26.04:10.08
18,45:10.04
16.03:10.03
6.97+00.8
10.32-+0.06
8.76+-0.07
8.89+0.07
7.08+0.07
6.58+0.06
7.97+0002
7.26++0.02
33.29=0.11
36.29:10.08
28.07:0.009
26.85=00.99
31.6810.1
31.
23.55:0.05
23.77=00.06
0.02
76.71
75.73.10.07
75.55
74.29.10.06
75.61:10.06
75.44.10.07
75.81
73.83=0.07
75.79+0.077
76.27
74.05+00.07
73.94=0.08
71.92=0.08
67.61+0.08
66. 14.00800.0
53.97=0.13
6..244+01
51.71+#012
52.56+0.12
51.27=0.12
55.72-00122
59.2+00099
56.66-#0.09
23.2510.11
32.35.10.09
20.44.0.11
20.27110099
22.05.0.009
24.11
10.35
24.3111.006
05.14+0.06
87.58
85.28+0.06
83.98+0.07
85.35+0.07
78.33+0.11
83.58+0.07
84.74+0.06
70.88
70.07
71.5110.07
7..53
72.1310.06
7..5
68.8710.07
71.13=0.007
10
63.32-+0008
65.684-0.08
63.55+40.08
66.17
65.2440.08
65.11
65.4400.0
64. 04.12-+0.08
37.15+0.07
48.8+0006
37.9440.08
37.56+0.07
38.18
40.92+0007
40.85+10.06
44.37+0.07
12
78.36=0.06
73.98+0.055
71.32+0.06
71.39
70.56+0.06
76.65
33.87=0.09
50.86+0.08
13
82.9
70.2.00.66
63.46.10.06
61.7210.05
63.16
67.22.10.16
58.3210.04
48.7310.06
14
76.99+0.007
74.71+0.008
07.3+0.07
76.97+0008
77.11+0.08
75.18+0.08
46.9+0.08
56.99+0.1
87.6810.006 15
87.86:10.07
92.2710006
02.55
92.1611.05
86.99
82.0210.1
86.41
83.7+0.02
84.6+00.22
83.12+0.02
80.7+0.02
82.45+0.02
82.8+0.03
66.82+0.04
76.2440.03
17
57.01
64.35:10.03
58.38120.04
57.49
57.05+0.04
58.52:0.04
56.46:10.04
60.28=0004
18
51.85
58.73.10.04
53.5110.04
53.66
52.78
53.35.00.04
54.65
57.13
19
51.710.05
59.4.00.04
52.8810.005
52.6110.05
52.45.0.005
53.62.10.04
54.13.1..05
15.18
20
09.01
5..53+00.06
48.76+00.77
45.45+00.6
48.47+0.06
47.45+0.07
27.68+0..55
38.2440.055
73.6610.05
75.1310.05
74.7310.00
72.4110.06
74.1510.055
69.46.10.06
37.7310.07
57.7310.07
22
35.41+0.1
36.02+00.08
31.5440.1
28.9+0.08
30.17+0.09
31.76+0.1
24.05-+0.04
29.58+0.05
23
40.75+0.09
47.78+0.08
42,61+0.09
40.01
41.73+10.09
43.5941.09
32.4110.06
38.9240.07
24
2..6
19.19.0.09
14.33
14.97
17.74.0.1
21.0
9.9-0.02
11.63-0..44
25
11.82.10.09
13.91
11.12.10.07
10.42
12.74.0..08
13.22
12.17
12.92.0.033
0..61
32.78+0.1
27.59+0.11
24.68
28.6740.11
33.57+0.11
23.85+00.4
24.35+0.08
27
72.0410.007
72.59110.07
73.1310.08
70.5210.08
70.5410.08
67.53
68.7710.08
72.7310.00
50.46+0.03
57.79+-0.03
50.98+0.03
50.89++.03
50.8+0.04
51.7440.03
53.51+0.03
52.86+0.03
29
68.7310.05
76.47:10.04
71.24:0.05
69.7910.04
69.7610.05
70.0
62.36110.06
67.41
30
30.4+0.09
36.97-00.08
30.26-10.09
28.75+0.09
28.85
30.76
16.71+0.006
24.04+0..44
66.1910.1
68.31+10.08
68.4810.09
69.24
69.4810.08
61.5610.11
55.91
51.7440.1
32
41.96:10.11
47.48+10.09
44.6740.11
40.48+0.11
45.22+0.11
38.11+0.111
11.3740.06
20.25+0.05
30

---

Learning Imbalanced Data with Beneficial Label Noise
Table 17: G-mean with CART classifier on KEEL datasets
Over-sampling
Under-sampling
Datasct
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
28.9110.09
37.02:10.08
24.49:10.08
23.6410.07
24.3410.09
26.53:10.08
26.61:10.04
25.34:10.04
7.07+00.88
12.23++0.07
9.05+0.07
9.22+0.07
7.23+0008
6.82+0.06
14.67+0.04
15.17+0004
33.71=0.11
37.26:10.09
28.3610.09
27.08
32.23+0.1
32.46=0.11
28.62:1.006
30.74=10.07
7..51
77.06
75.9110.07
7..76
74.49.10006
75.85:10.06
75.81
76.35:10.06
74.25=0.07
76.23+00006
76.63+L0.06
74.39+0.07
74.23+0.08
72.82=0.07
68.85+0008
68. 13.007
56.92+0.13
60.96+#0.1
52
13.25+0.12
51.95=#.12
56.59=0012
60.99:10.09
59.45=00.08
25.63.1.11
34.26.10.09
21.2510..1
20.9310.1
22.7410.09
25.93
21.6910.03
32.6710.055
85.49+0.06
8..8
85.59+0.06
84.28+0.07
85.63+0.07
78.97+0.099
83.89+0.07
85.05+00.66
71.18:0.06
72.02=10.07
71.8110.06
70.8210.06
72.5410.06
72.8
69.4510.07
71.58=0.007
10
03.62-+0.08
65.99-10.08
63.88+00.08
0.07
65.51+0.08
65.62-00007
65.82-00007
64.5+
37.38
49.54
38.1
37.72+0.07
38.3620.08
41.32+0.07
42,43+00.06
45.59+0.07
12
78.88=0.05
74.87
71.95+00006
7.98
71.2440.06
77.19+0.07
44.89+0.07
58.07=0.06
13
84.21 0.04
70.64
66.67.10.04
65.64.1.04
66.710.04
69.37.1..15
64.16.10.03
56.51
14
07.31
75.22+
77.67+0.07
77.37+0.07
77.49+0.08
75.77+0.08
53.31+0.06
60.6+0.08
15
0,09
88.16:10.06
92.4110.06
92.7110.05
92.3410.05
87.68
83.4110.09
86.92-10.08
83.74+0.02
84.7+0002
83.16+0.02
80.74+0.02
8..47+0.02
82.91+0.03
69.36+0.03
77.57+0.03
17
57.1
64.96:10.03
58.46220.04
57.5940.04
57.13+0.04
58.6910.04
56.67:0.04
60.61
18
51.97-10.04
10.44
53.58110.04
53.73.0.05
$2.89
53.54.0.04
55.72.0.04
18.26
19
51.8110.00
60.38
52.98.10.05
52.7110.05
52.5410.05
53.86
55.31
56.34.1.04
20
49.32+0.07
55.0
49.06+00007
45.71+0006
48.71+0.06
47.83+0.07
35.91
43.62+0.04
73.8310.05
75.510.05
74.92.0.006
72.6410.06
74.3110.05
69.98
46.06.10.05
61.4610.06
22
35.73+0.1
36.82+0008
31.91+0.1
29.26+0.08
30.45+0.09
32.25+0.1
32.9+0.05
34.84+0.05
23
41.2+0.09
48.5+10.88
43.01+10.09
40.410.1
42.1610.09
44.25
39.88110.06
43.41
24
2.11
20.09
14.63
15.32+
18.05+40.1
21.76-10.1
19.18-00.04
19.26-10.05
25
12.08
15.06-10.08
11.37-L0.08
10.62
12.92
13.73
21.99
20.6
38.18+0.12
33.67+0.1
28.0740.11
24.9940.12
19.1940.11
34.46+0.11
32.44+0.055
30.63+0.08
27
72.4510.007
10.01
73.55.10.08
70.910.08
71.110.07
70.52.10.06
70.8910.07
73.09
50.5+0.03
58.81+00.03
51.0440..3
50.95++0.03
50.84+0.04
51.87+0.03
55.16-00.03
53.73+0.03
29
68.8610.05
76.95:10.04
71.3610.05
69.93
69.8610.05
70.2+0.05
65.7+00.5
69.63-0.03
30
30.8++0.09
38.13-00.08
30.59-0.09
29.08
29.19+0.1
31.06
27.67+00.06
13.00
66.73+0.1
69.04=1..0
68.9110.00
69.7140.08
70.0110.08
62.22+1.11
60.96100.88
58.0240.08
32
42.54+0.11
48.97+00.9
45,4940.11
41.12+0.11
45.83+0.11
39.04
22.69+10.05
30.39+0.05
31

---

Learning Imbalanced Data with Beneficial Label Noise
Table 18: AUC with CART classifier on KEEL datasets
Over-sampling
Under-sampling
Dataset
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
62.85+0.05
72.47:10.06
61.97:10.05
61.4510.005
60.9810.05
63.12:10.05
74.83:10.05
74.89-10.05
52.45+0.00
57.644+0.06
53.92+0.04
54.09+0.05
52.6+0.04
52.54+0.04
61.73++0.08
61.8+
03.71=0.06
68.64:1.06
61.82+0.05
61.03=00.55
63.05=0.05
65.25=0.06
66.11:0.06
68.44:10.7
0..3
85.27
83.5710.005
83.5710.05
82.46.10004
84.36.10.05
84.64.10.05
85.42.10.04
5
84.23=0.05
86.4+10.5
86.19=00.05
84.76+0.05
84.52=0.06
85.83=0.05
84.06:00.05
84.26=0.05
74.0440.08
79.37+00077
72.69
73.06+0007
72.03+0007
77.02=0.08
82.51+10.06
82.4+0.06
64.23110.09
73.7410.11
63.72.10.09
63.35.1.11
63.7710.1
64.48.10009
55.67
77.43
89.1+0.05
91.81+0.05
89.53+0005
88.7+0.06
89.63+0.05
86.07+0.07
88.26+00005
90.62+0.05
77.11=0.05
77.5+0.05
77,59
76.83+0.05
78.1410.05
7..3
74.7810000
77.03=10.05
10
70.72+40.06
71.97-00.06
70.76++0.06
72.71+0.05
72.04-0.06
70.66
71.37+-0.06
70.23
56.2220.04
62.71+00.05
56.4550.055
56.22+0.05
56.59=0.05
57.06=#0055
53.7
57.83++0.06
12
95.71=0.03
97.0+0.033
96.17.0..3
96.09+0.03
96.11=0.03
96.1=0.03
92.78+00.2
96.42+0.02
13
99.2910.00
98.82.10.0
99.2310.0
99.15.1.001
99.210.0
98.4510.011
99.0110.00
97.55
14
88.28+0000
87.94+00.05
88.79+0.05
88.77+0.05
88.61+0.06
88.6+0005
82,94+0.005
85.71
15
91.5910.04
93.18110.04
94.811004
94.98110.04
94.6110.04
94.411004
93.510.04
93.84.10.55
90.6+0.01
93.01+0.01
91.32+0.011
89.96+0.01
90.41+0.02
91.79+0.01
90.53+0.01
93.48+0.01
17
66.32+0.03
70.74+10.3
67.02+00.03
66.01+0.03
65.810.03
66.59
64.5810.03
67.5+0003
18
66.910.03
10.06
67.99.L0.03
68.0911..3
6..56.0.04
67.8.10.33
68.71
70.83
19
67.2
13.02
68.0.10.04
67.7910.00
67.69
68.49
68.91
19.00
20
0.03
7..59
72.18+0.04
70.06+00.04
71.37+0.04
72.02+00.55
64.33+0.06
73.68+0.04
85.6710.04
88.6110.03
86.2610.04
85.6910.04
85.8310.04
85.66110.04
76.1710.04
86.4310.03
22
63.38+0.05
6.027++0.06
61.89+0.06
60.26+0.05
60.77+0.05
62.2440.06
57.03+0.07
63.97+0.055
23
66.5740.06
72.64+0.06
68.1750.005
66.8910.07
67.3710.06
70.0441.00
69.05:10.06
72.65+10.06
24
60.66
59.76-100.06
56.05
56.64
07.45
60.67-10.06
59.3
11.08
25
53.510..04
L5.84
53.33
52.78.10005
54.03
14.68
57.72.10.06
58.25.0.00 10.07
65.67
65.17
61.0940.06
59.37+0.07
61.2340.06
65.56+0007
66.9910.06
64.76+0.088
27
83.2110.05
84.510.05
84.85100.06
83.0810.05
84.9810.05
91.27.10.04
90.02.10.44
85.93
64.57+0.02
69.38-00.02
64.82+0.022
64.68+00022
64.66+0.03
65.09+0.022
65.1020.02
15.01+
29
82.02+0.00
90.02-10.03
84.4310.00
83.7+...3
83.0310.04
84.2910.03
88.34:10.02
89.59-10.02
30
63.47+0..05
+1.58-00.05
64.711-0.06
63.99+00.55
62.9440.05
65.01+00.05
72.15
78.74+0.06
81.08
87.63+0.00
83.6340.06
84.1110.06
83.9410.06
82.18
93.35+10.04
93.72-10.03
32
70.93+0.06
78.9740.07
74.56+0.07
71.76+0.07
73.45+0.07
7148=00.07
71.71+0.05
81.01+0.05
32

---

Learning Imbalanced Data with Beneficial Label Noise
Table 19: F1 score with MLP classifier on KEEL datasets
Over-sampling
Under-sampling
Datasct
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
26.4710.12
44.35:10.08
37.1110.07
36.8111.066
38.3610.07
32.04:10.13
27. 27.03:10.04
24.62:10.05
0.0+0.0
18.44-0.07
13.8+00.7
12.75+0.07
10.29+0.07
0.0+00.0
10.04+0.03
8.93+-0.02
44.83=0.17
52.22:10.08
48.33=+0.08
45.94
51.6910.09
43.47=10.21
40.43:0.07
36.91=0.00
75.16
76.18:10.06
77.05.10.05
76.63
76.75.10.05
75.48:10.07
76.83:10.00
13.10:5
82.01=0.06
82.46+00.06
81.27
76.19+0.07
76.7=#008
80.06=0.06
78.89+0.006
75.15=0.07
40.44+0.31
66.92+00.08
66.48-10.09
66.17
66.07=0.09
60.85=0.11
61.47+0007
16.008
11.4910.11
31.02:1..1
27.75110.06
26.45.10.07
27.55
20.8910.12
16.94.10.06
25.1710.005
86.35+00.05
86.05+00.66
87.07+0.06
87.02+0.06
88.18+0.05
83.64+0.07
85.72+0.06
87.44+0.05
51.91
71.02::0.08
69.4610.1
72.2810.07
69.7910.1
59.72:10.27
69.4310.07
70.4410.06
10
63.87+-0.07
64.83+-0.06
14.16.00007
65.06+40.07
6.022
63.52-0008
64.27+10.06
65.75
29.54=0.12
50.44+10.07
48.06+0.08
48.31
48.0#0.08
43.26
40.48+0.07
47.47=0.08
12
68.6=0.07
08.055
63.01
62.55+0.05
64.46+0.05
67.3
50.6
45.61
13
19.55
70.58
53.6410.055
55.36.10006
56.7410.05
69.3910.11
51.4810.06
49.38
14
45.53+0..34
70.19+00.09
72.12+0.09
70.46+0.1
71.65+0.11
73.86
58.99+0.1
56.48+0.11
15
94.310.04
95.24110.04
95.0210.03
95.3110.04
95.410.03
93.3410.05
93,3110.055
94.0711.005
82.79+0.02
83.65+0.02
81.57+0002
76.76+0.03
80.04+0.02
84.15
77.06+00.03
77.85+0.02
17
61.88+0.04
67.36110.03
64.76120.04
64.8440.03
64.85+0.03
63.81=0.04
61.75+00.04
64.7940.04
18
71.16.0.04
73.2.10.03
72.2.00.44
71.8710.33
71.46.0..33
71.74.0.04
71.43
70.93
19
62.1110..05
67.23.10.04
66.910.04
66.66.10.04
67.18
62.68
65.18
60.96
20
54.87+0.1
59.41
47.81+0.06
40.61+0.06
50.05
55.88
37.26+00.05
38.22+0.05
78.810.06
79.8:10.05
72.5910.05
63.8110.88
69.0410.1
78.71
59.310.08
63.77110009
22
32.23+0.1
40.73+00099
35.86+0.08
31.63+0.09
38.56+0.1
31.22+0.13
31.61+0006
30.49+0.07
23
14.7340.11
50.9411.008
48.1710.1
47.1340.1
48.4340.09
45.08
41.05:10.06
38.67:0.08
24
0.0.1..0
16.27-L0.08
16.75-10.09
15.27
2.13
1.540.04
12.14
10.20
25
0.0.10.0
18.43
14.18.L0.08
14.0.10.08
13.09
0.1-1.011
15.63
13.42.10.03
8.76+0.15
33.8++0.09
29.75+0.08
39.72+0.1
25.9440.14
30.1=0.06
26.53+00.77
28.76+0.11
27
69.7710.00
71.1410.08
68.4710.00
71,24
70.8410.08
67.36
64.610.07
11.01
55.49+0.03
60.34++0.03
59.53+0.03
59.0++.02
59.16+0.02
58.09+0.03
58.9+0.02
59.75+0.02
29
78.3240.04
78.37:10.04
75.2510.033
73.4710.04
75.53+0.04
78.2310.04
69.7910.04
69.3941.004
30
10.44+0.11
41.68-L0.07
36.45-L0.08
36.55+00007
38.78+00007
19.53+0.11
24.72+0.04
22.67+0.044
60.93+0.11
66.58+0.08
74.58+0.07
74.1910.07
73.7740.08
67.37=1.1
52.9810.08
45.92+00.07
32
49.7+0.16
58.02
46.87+0.1
45.48+0.1
54.62+0.11
49.75=0.115
26.67+0.07
22.41+0.05
33

---

Learning Imbalanced Data with Beneficial Label Noise
Table 20: G-mean with MLP classifier on KEEL datasets
Over-sampling
Under-sampling
Dataset
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
31.8610.13
45.44.10.08
40.6110.07
40.53:10.06
40.2511.07
35.08:10.14
35.94:10.04
34.36:10.04
0.0+0.0
20.99--0.08
15.88
14.55+0.08
10.62+00.8
0.0+0000
17.33+00.04
17.66-+0.04
47.07=0.18
53.11=0.008
49.47=0.08
4..01
52.21
44.74=:.222
46.310.06
43.42=t0.06
7..52.10.07
76.73:10.00
77.5100.55
77.25
77.44.1.005
75.77:10.07
77.63:10.04
77.16:10.05
02.24=0.06
02.73++0.06
81.55
76.7=:0.07
77.11=0.08
80.42=0.06
79.6+0006
76.35=0.07
40.92+0.31
67.82++0.08
67.24+0.08
66.9+1008
66.79=0.09
61.3=0011
63.87+00077
62.48+0.07
26.8810.12
32.611..1
30.15
28.910.07
29.8510.08
25.56.10.13
24.64.10.06
33.2810.055
86.56+00005
86.37+00.06
87.28+0.06
87.22+0.06
88.35+0.05
84.04+0.07
85.92+0.05
87.74+0.05
52.19:0.32
71.64:0.08
70.0910.1
73.0310.07
70.7+0.1
60.0810.27
70.0310.07
70.98
10
64.33+0.00
65.13+10.06
64.42+00.07
65.5140.07
6.052
64.05
64.77+10.06
66. 14.007
34.05+0.11
50.69
48.41
48.85+0008
48.38=0.08
44.05=0008
42.18+0.07
48. 17.008
12
09.45-00066
66.8
66.88+00.55
65.98
68.0940.04
68.24=0.09
57.81=0004
53.89+0.05
13
81.3310.04
70.94
59.4110.04
60.6110.04
61.8710.04
71.22.10.15
57.92.0.04
56.22.1.04
14
45.94+0..4
70.61+0.09
72.67+00009
71.04+0.1
72.16+0.1
74.31+0.11
61.41+00.09
59.95+0.1
15
94.4210.04
90.41
95.1110.03
95.4110.04
95.4910.03
93.52:10.05
93,5910.04
94.28110.05
82.97+0.02
83.69+0.02
82.29+0002
78.08+0.03
80.81+0.02
84.2+0.02
78.03+00.22
19.13+
17
62.1220.04
67.97-10.33
65.02+0..4
65.3+:0.3
65.25+0003
63.95=0004
61.9410.04
65.25=10.04
18
71.28
10.91
72.49.0.04
72.310.03
71.9110.03
72.04
72.0.0.033
71.73
19
62.3110.005
68.17
67.3110.04
67.110.04
67.65
63.07
66.0.10.04
10.04
20
56.8+0009
59.73+0.06
48.89+0.06
42.18+0.06
50.57+0.07
57.06
41,4+0005
43.0++0.05
79.1910.05
79.9410.05
72.9210.05
66.311008
69.3610.1
79.01
61.8210.07
67.7810.08
22
37.12+0.09
41.32+0009
36.55+0.08
32.3+0.1
39.21+0.1
32.61+0.13
35.08+0.07
34.97+0007
23
46.7940.12
51.610.08
48.6510.11
47.5810.1
48.8740.09
46.21=10.12
44.05:10.06
42.7640.07
24
0.0.1..0
18.09-10.09
17.89-00009
16.36
23.63
1.79
19.57
18.63-10.05
25
0.0.10.0
19.33
14.83
14.61
13.910.1
0.111.01
21.75:10.06
20.08
9.76+0.16
36.43+0.1
31.83+0.08
41.08
26.77+0.15
36.99+0.06
33.3+0007
29.85+0.11
27
70.6310.08
71.6610.08
68.9310.007
71.7111.08
71.3210.08
67.92
66.34.0.007
63.68
16.1030.03
60.79+0.03
60.29+00.03
0..34+0.02
60.43+0.02
58.27+0.03
61.04+-0.22
60.76+00.2
29
78.4240.04
78.6:10.04
75.82+0.03
74.14410.04
75.93+0.04
78.3710.04
72.07
71.61
30
12.52+0.15
42.27-00.07
38.54-0.08
38.6440.08
39.47+0.07
21.23
32.73+0.005
31.05
61.89+0.11
67.53+0.08
74.92+0.07
74.56+0007
74.13+0.08
67.93+0.1
58.8440.07
53.74400.6
32
52.58+0.16
58.58+0..1
48.4+0.1
46.77=0.1
55.06+0.11
51.87=0.14
34.5610..6
32.06+00.05
34

---

Learning Imbalanced Data with Beneficial Label Noise
Table 21: AUC with MLP classifier on KEEL datasets
Over-sampling
Under-sampling
Datasct
baseline
LNR
SMOTE
ADASYN
Borderline
OSS
CC
RUS
92.510.02
92.24410.03
89.2410.05
89.12:10.05
88.1810.05
90.38:10.04
91.48:10.04
91.1911.04
79.15+00.06
78.32++0.06
70.78+0.08
7.04
68.96+0.09
78.62+0.06
72.97+0.05
72.24+00.6
91.61=0.03
91.18:00.44
88.810.05
88.2910.005
90.67=0.04
90.64=0.04
90.31+0.03
89.020.00
93.75-10.03
93.68
94.27.10.02
94.04.1002
94.0.10.02
93.0-10.03
94.01
93.83-10.02
94.94=0.03
95.52++0.03
95.35+00.03
94.45+0.03
93.65=0.03
94.71=0003
95,48+00.3
94.91=0.03
91.21=0.05
90.85++0.05
90.46+0.004
90.12++.055
90.28=0005
88.4=0.05
89.54+10.04
89.56=00.44
84.62.0.00
83.95100.06
72.55
70.23
75.8410.08
81.8110.07
70.7510.00
81.56110007
96.62+0.03
96.6+0.03
97.25+00.3
97.21+0.03
97.7+0003
95.76+00.33
95.56+0.04
97.05+00.3
76.43:0.14
82.25::0.06
83,9910.06
84.1610.05
81.8710.08
78.0440.14
81.9410.05
81.9210.05
75.2140005 10
75.25-+0.05
75.26+-0.05
7.035
7..05
73.33+0006
75.56+00.05
74.92-10.06
66.32+0.05
67.9440.05
66.83+0.06
66.1
66.52+0.07
65.88=0.06
56.31
64.09+0.088
12
98. 08.7
08.0
98.93+00011
98.46+0.01
98.73=0.01
98.7=0001
98.7#0.00
98.38=0001
13
98.5610.01
98.57100.01
98.71110.0
97.611.01
98.5310.01
97.1910.03
98.6110.01
98.3910.01
14
13.10+0..33
93.24++0.03
93.73+0.03
03.99+0.03
93.86+0.03
90.97+0.05
92.35+0.03
92.10+0.0M
15
99.8910.0
99.8710.00
99.8610.0
99.8611.00
99.8710.0
99.6411.01
99,8910.0
99.82
12+0.01
98.11+0.01
98.52+0.0
97.86+0.01
97.87+0.01
98.13+0.01
97.94+0.01
98.05+0001
17
79.94=0.03
80.48
78.4310.03
77.87+0.03
77.31
79.05=0003
76.55+0.00
78.07=0.03
18
90.8810.02
90.47.10.02
90.96.10.02
89.94
89,2310.02
90.45.10.22
89.97
89.27-10.03
19
88.2910.02
87.52
88.110.02
87.6710..02
88.08
85.98
86.61
86.21
20
83.13+0.04
82.15+00.04
79.59+0.04
77.41+0044
79.86+0.05
82.64+0.04
76.89+0.04
78.63++0.04
93.1410.03
92.910.03
91.8910.03
90.7411.03
90.1110.04
92.8910.03
90.6310.03
92.6410..03
22
0.01
74.29+0.07
71.19+0.05
68.97+0.07
72.3+0.07
71.76
69.19+0.07
69.25+0006
23
82.02=0.005
81.4340.06
82.210.07
81.7410..7
80.9710.06
78.7910.06
78,35+0.05
77.8810.06
24
71.04.10.07
12.42-10.00
68.1740.08
67.3110.07
6..02
70.46
67.27
65.72-0000
25
63.0
66.21
64.37-L0.07
63.99
60.52
62.66
66.55.10.07
62.410.07
76.47:0.06
78.45+0.06
76.34+0.05
79.14
74.53+0.06
79.95
74.65+00.07
75.44+0.06
27
91.66110.04
91.05.0..05
91.02.10.04
91.1110.55
91.16110.05
90.98
92.3510..04
10,04
7..64+0.02
79.37++0.02
78.49+0.02
77.93+0.02
77.5+0.02
79.43+0.02
77.52+0.02
78.69+0.02
29
97.1310.01
97.26:10.01
96.3111.01
95.56110.01
95.7410.02
96.8410.01
96.5810.01
96.3810.01
30
8.04
86.62+00.04
85.54+0.05
86.08
84.54+0.05
84.0
84.29-0..5
82.39
98.3740.01
98.1640.011
98.440.01
98.2910.01
98.240.01
97.88=0002
97.6810.01
97.59+0.01
92.3410.04 32
92.28
89.45+0.04
90.47=00.3
90.88+0.04
90.58=00.05
88.9440.04
89.41+0.04
35


Reviews:
[{"review_id": "aoD37IMZcJ", "reviewer": "Reviewer_AMhG", "summary": "This paper introduces a novel Label-Noise-Re-balancing (LNR) approach to mitigate the decision boundary bias caused by data imbalance.The numerical experiments in both binary and multi-calss imbalance demonstrated the effeciency of their approach.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Response to ‘Essential References Not Discussed’， Weakness 1 and Question 1**\n\nWe sincerely thank the reviewer for highlighting the need to include recent methodological advancements in our literature review. As suggested, we will carefully revise the 'Related Work' section in our manuscript: \n\n1. **Algorithm-level methods** refine loss functions or training paradigms to improve tail-class accuracy, primarily by decoupling feature learning from classifier training to separately enhance feature representation and classifier fine-tuning. Contrastive learning-based methods—including DRO-LT (2021), TSC (2021), BCL (2022), and SBCL (2023)—leverage contrastive losses during feature learning to boost feature discriminability and model robustness under imbalance. For classifier optimization, margin-based approaches like LDAM-DRW (2019) and τ-norm (2019) employ loss engineering to create larger decision margins for tail classes, while logit adjustment methods such as GCL (2022b) address softmax saturation by expanding the tail classes' embedding space through increased cloud sizes.\n\n2. **Data-level methods** often leverage generative models or autoencoders such as $\\Delta$-encoder (2018), DGC (2020), and RSG (2021) to synthesize few-shot samples. These approaches typically depend on high-quality pre-trained models, which can introduce additional challenges, especially on scarce tail-class data, limiting the ability to generate diverse or meaningful samples. Instead of involving a generative model, Mixup (2018) interpolates features and labels via a fixed mixing ratio $\\lambda$, empirically demonstrating its effectiveness for data augmentation. Building on this, Remix (2020) introduces separate mixing ratios for features ($\\lambda_x$) and labels ($\\lambda_y$) to rebalance class distributions, though it retains random sampling. More recently, SelMix (2024) advanced this direction by selectively sampling pairs for mixing based on the gain on non-decomposable metrics (e.g., recall, G-mean), thereby enabling targeted improvements in specific metrics. However, SelMix's gain matrix relies on a balanced ($\\alpha = 0.95$, where 1 means fully balanced) augmented auxiliary set (5000 for CIFAR). With imbalanced or small auxiliary data, its metric optimization fails to meet theoretical constraints. This limitation is particularly acute in practice where validation data is often scarce and inherently skewed—a gap our method intentionally addresses with a carefully designed noise model, which requires neither feature editing nor a balanced auxiliary dataset.\n\n3. **Multi-expert ensemble methods** (e.g.,  RIDE (2021), TLC (2022), SADE (2023), and BalPoE (2023)) allocate specialized \"experts\" to model head- and tail-class features separately, achieving notable gains. While these approaches fall outside the scope of our work, they highlight the ensemble learning for tackling class imbalance.\n\n**Response to Weaknesses 2-4**\n\n- W2: Regarding the many/medium/few-shot categories, we acknowledge these thresholds vary across datasets. For clarity, we've detailed these specifications in Appendix C.7 and will add explicit cross-references in the main text. We sincerely appreciate you bringing these issues to our attention.\n\n- W3: We have carefully re-examined the definition of False Negatives (FN) as:\n\"A False Negative occurs when a ground-truth positive sample is incorrectly predicted as negative.\"\nBased on this standard definition, the formulation of FN/N in Section 3.2 is mathematically correct.\nShould there be any misunderstanding on our part regarding your specific concern, we would be grateful if you could further clarify your perspective so we can address it precisely.\n\n- W4: All typographical errors identified in the manuscript, including those in Table 2, have now been carefully corrected. We are truly grateful for your meticulous review。\n\n\n**Response to Questions 2-3**\n\n**Q2:** We sincerely appreciate your thoughtful suggestion regarding Algorithm 1. To clarify the notation: $ U \\sim \\mathcal{U}(0,1)$ indicates sampling a number U from the range [0,1], where label flipping occurs when $\\rho(x) > U$. As you insightfully pointed out, this could alternatively be expressed as: $U \\sim \\mathcal{U}(\\rho[x])$. If this alternative formulation better serves clarity, we would be delighted to incorporate this change in the manuscript. \n\n**Q3:** To our knowledge, all class imbalance methods inevitably improve minority class performance at some cost to majority class accuracy. This trade-off is fundamental because:\n- Their strong majority class performance inherently comes at the expense of minority classes\n- Therefore, improving minority class recognition necessarily reduces the \"overprivileged\" majority class performance.\n\nOur fairness analysis (confusion matrices) demonstrates this trade-off: On CIFAR-10, LNR introduced 94 label noises. This exchanged 16 true positives (TP) from head classes for 204 TP gains in tail classes", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes.\n\nNovelty: Instead of relying on traditional data augmentation, the authors propose a novel label-flipping method to improve minority class prediction accuracy in imbalanced datasets.\n\nExperimental Validation: The experiments clearly demonstrate that the proposed method improves the accuracy of minority class prediction in both binary and multi-class settings.", "theoretical_claims": "The paper provides a theoretical justification for how imbalanced data can lead to biased decision boundaries, highlighting a critical issue in classification tasks.", "experimental_designs_or_analyses": "The experimental design is generally strong. However, the definitions of the imbalance ratio categories (Many-shot, Medium-shot, Few-shot) presented in Table 2 are unclear.", "relation_to_broader_scientific_literature": "1) This paper made a valuable contribution by introducing a novel Label-Noise-Re-balancing (LNR) method, which creatively mitigate the challenge of decision boundary bias in imbalanced datasets. \n\n2) This paper theritically demonstarted that the relationship of imbalance ratio and decision boundary bias.\n\n3) Experimental results confirmed the efficacy of the LNR method in enhancing the prediction accuracy for the minority class.", "essential_references_not_discussed": "It appeared that the paper lacks recent related work (especially after 2022), which is essential for placing the contribution in the context of current research.", "other_strengths_and_weaknesses": "Strengths: \n\n1) The paper is well-organized and presented with clear, accessible writing.\n\n2) The LNR method offers a novel approach to addressing data imbalance.\n\n3) The paper provides comprehensive mathematical proofs that elucidate the impact of class imbalance on decision boundary bias, strengthening the theoretical underpinning of the proposed method.\n\nWeaknesses: \n\n1) The paper lacks a comprehensive review of recent (after 2022) literature.This is my main concern. If the authors could resolve this, I would be happy to adjust my rating.\n\n2) There is a lack of clear definition and handling of imbalance ratios, as terms like Many-shot, Medium-shot, and Few-shot are not precisely delineated.\n\n3) Check the defination of FN/N in section 3.2.\n\n4) Typos and Formatting Issues: The paper contains typos (e.g., \"many-shot\" in Table 2).", "other_comments_or_suggestions": "Regarding the organization, it might be beneficial to integrate the \"Motivation and Contributions\" section into the introduction to create a more cohesive narrative flow. This is merely a suggestion for improving readability.", "questions_for_authors": "1) Could the authors identify if there are any latest relevant work (published after 2022) and provide a fair comparison of the proposed LNR method with them  that address data imbalance? \n\n2) Could you explain the algorithm1, why sampling u from random(0,1)?\n\n3)  Although flipping labels from the majority to the minority class might improve minority class accuracy, does it decrease the accuracy of the majority class?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "79T4ttT7jx", "reviewer": "Reviewer_VQ6R", "summary": "The paper introduces a novel method called Label-Noise-based Re-balancing (LNR) to address imbalanced classification problems by incorporating beneficial label noise. This approach involves flipping labels of majority class samples to minority classes to adjust decision boundaries and enhance classifier performance, particularly for minority classes. The authors provide a theoretical analysis focused on binary classification and extend the method to multi-class settings. Experiments on synthetic and real-world datasets demonstrate LNR’s effectiveness and its compatibility with various classifiers.\n\n## update after rebuttal\n\nI believe the authors have adequately addressed my concerns. There are no remaining major concerns from my side, so I have adjusted my score accordingly.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reply to Claims And Evidence and Methods And Evaluation Criteria** \n\nThank you for your thoughtful comments. We appreciate your feedback and would like to clarify that our evidence sufficiently supports our claims.\n\n- Our theoretical analysis focuses on the deviation between the optimal F1 decision boundary and the optimal Bayes decision boundary (rather than other metrics) because F1-score is typically the primary evaluation criterion for imbalanced binary classification tasks, especially when minority-class recognition is crucial, as in fraud detection scenarios.\n\n- Through a comparative analysis of how optimal decision boundaries are affected by both label noise and imbalance ratio, we designed a feature-dependent **asymmetric** label noise model. Our primary theoretical contribution lies in LNR's ability to make the label noise's effect on decision boundaries counterbalance the impact of class imbalance—effectively correcting the boundary shift.\n\n- We would also like to kindly emphasize that our evaluation metrics are comprehensive and **widely accepted**. These include F1, G-Mean, and AUC for binary classification tasks and Many/Medium/Few-shot metrics, which are broadly used for imbalanced multi-class classification and acknowledged by Reviewer k9AK and other reviewers.\n\n- The experimental results in Tables 1-2, along with a comparison against the latest SOTA methods in our response to Reviewer k9AK, provide sufficient evidence of the effectiveness of our approach. In our latest comparison with ReMix/SelMix based on MiSLAS, we also included Expected Calibration Error (ECE) to address potential concerns regarding model calibration when modifying the data. As shown in the results, LNR, due to its zero feature editing of the data, not only enhances performance but also achieves a significantly lower ECE than ReMix.\n\n\n**Reply to Relation To Broader Scientific Literature & Essential References Not Discussed**\n\nWe sincerely appreciate your valuable feedback regarding improvements to our literature review. In response to your suggestion, we will expand the discussion of methods related to our work, including generative and mixup-based approaches, in the revised manuscript. We will revise the literature review as described in our response to Reviewer AMhG for your reference. In our response to Reviewer k9AK, we have included updated comparative results with the recent ReMix and SelMix approaches. We have also clarified the fundamental distinctions between our method and SelMix in greater detail. We sincerely hope that these additional analyses and explanations will help address your concerns.\n\n**Reply to Weaknesses and Questions**\n\n- W1/Q1: We sincerely appreciate the reviewer's insightful suggestion regarding multi-class generalization, which is indeed an important direction for future research. While extending the theory to multi-class settings represents valuable future work, our current study specifically focuses on using theoretical analysis to motivate our carefully designed feature-dependent asymmetric label noise model. \nIn multi-class scenarios where LNR flips labels from a majority class i to a minority class j, our binary classification analysis of decision boundaries remains applicable to understanding the boundary between classes i and j specifically.\n\n- W2/Q2: In our response to Reviewer k9AK, we have added a comparison with SelMix, the latest state-of-the-art metric-optimizing method. Although our method, LNR, **does not assume the availability of a sufficiently large balanced validation set** for metric optimization, its performance remains comparable to that of SelMix. This makes LNR more advantageous in practical applications.\n\n- W3/Q3: \n    * Thank you for your insightful comment. In Section 4.3, we emphasize that LNR mitigates the impact on feature extraction by postponing the introduction of label noise during the fine-tuning stage, thus avoiding any potential threat to feature representations. As such, LNR does not involve risks to feature extraction. We would be happy to include a comparison of feature representations after the noise introduction in the appendix of the revised manuscript to address your concern better.\n\n    * The fairness changes after introducing noise, as demonstrated on the CIFAR-10 confusion matrix, were primarily intended to show that these noises do not disrupt the existing fairness. On the contrary, they contribute to improved model fairness. These conclusions are also evident in the results of the many/medium/few-shot tasks on CIFAR-100. If there are specific fairness metrics you would like us to report, we would be pleased to address them during the rebuttal period.", "claims_and_evidence": "The primary claim is that introducing controlled label noise can mitigate the effects of data imbalance on decision boundaries, thereby improving classifier performance, especially for minority classes. The authors support this with theoretical derivations showing how label noise influences decision boundaries and with experimental results reporting improved metrics such as accuracy and F1-score compared to baseline methods. However, the evidence is insufficiently convincing. The theoretical contributions are limited and not novel, focusing on specific metrics without broader generalization. Empirically, the improvements may be inflated by the choice of metrics that do not fully reflect overall performance, raising doubts about the method’s true effectiveness.", "methods_and_evaluation_criteria": "LNR employs an asymmetric label noise model, where labels of majority class samples are flipped to minority classes based on a flip-rate estimator, typically implemented via a modified MLP classifier. The evaluation uses standard classification metrics like accuracy and F1-score across various datasets and classifiers. However, the choice of these metrics is bad, i.e. they fail to convincingly demonstrate LNR’s superiority, particularly in terms of overall performance. The lack of metrics aligned with methods that directly optimize performance measures of interest further weakens the evaluation framework, suggesting that the reported gains may not generalize beyond the specific experimental setup.", "theoretical_claims": "The paper offers a theoretical analysis for binary classification, illustrating how label noise adjusts decision boundaries to favor minority classes. While this provides some insight, these results are largely known and restricted to two specific metrics, lacking generalization to a broader class of performance measures. Additionally, there is no clear theoretical analysis demonstrating how well LNR optimizes any given metric of interest. This limitation undermines the claim of a significant theoretical contribution, as the analysis does not extend meaningfully to multi-class scenarios or diverse evaluation criteria beyond the binary case.", "experimental_designs_or_analyses": "The experimental design spans synthetic and real-world datasets, testing LNR with multiple classifiers to showcase its versatility. Results indicate performance improvements, particularly for minority classes, and the authors provide a confusion matrix for CIFAR-10-LT to address fairness concerns. However, there is an issue here, the absence of comparisons with methods that directly optimize the performance metrics evaluated in the paper. This omission makes it difficult to assess LNR’s relative effectiveness against state-of-the-art approaches. While the authors claim integration with methods like GCL and LDAM, the lack of direct metric-specific benchmarks limits the empirical rigor and persuasiveness of the findings.", "relation_to_broader_scientific_literature": "The paper discusses traditional resampling techniques and some algorithm-level methods but falls short in engaging with the broader scientific literature. It lacks a comprehensive comparison with recent advancements in imbalanced learning, especially post-2022 methods that might directly optimize the same performance metrics. The authors’ response mentions integration with methods like Mixup and SelMix but does not sufficiently differentiate LNR or contextualize it within the latest data-level or noise-based approaches. This inadequate discussion weakens the paper’s positioning within the field and its claim of opening a new avenue for imbalanced learning.", "essential_references_not_discussed": "Some key references are missing, particularly those related to methods that directly optimize the performance metrics used in the paper’s evaluations (e.g., accuracy, F1-score). Examples might include recent works on metric-specific optimization for imbalanced data, such as advanced re-weighting schemes or generative methods beyond Mixup, which are not cited or compared.", "other_strengths_and_weaknesses": "Strengths:\n* Innovative Concept: The use of beneficial label noise to address imbalance is a creative departure from traditional resampling, avoiding information loss and generative errors.\n\n* Generality: LNR’s compatibility with various classifiers and datasets is a practical advantage.\n\n* Theoretical Insights: The binary classification analysis offers a foundation, even if limited.\n\nWeaknesses:\n* Limited Theoretical Scope: The analysis does not generalize beyond specific metrics or robustly extend to multi-class settings.\n\n* Insufficient Comparisons: The lack of benchmarks against metric-optimizing methods undermines empirical claims.\n\n* feature extraction impacts and fairness under-explored beyond the CIFAR-10-LT confusion matrix.", "other_comments_or_suggestions": "To strengthen the paper, the authors should:\nExpand Theoretical Analysis: Provide a broader analysis covering multiple performance metrics and a clearer link to multi-class optimization.\n\nEnhance Comparisons: Include empirical comparisons with state-of-the-art methods that directly optimize the evaluated metrics, such as recent re-weighting or augmentation techniques.\n\nAddress Feature and Fairness Impacts: Conduct a deeper investigation into how label noise affects feature extraction and class fairness, beyond the provided confusion matrix, possibly with additional datasets or metrics like fairness indices.\n\nUpdate Related Work: Incorporate and discuss post-2022 literature to better situate LNR within the current research landscape.", "questions_for_authors": "Theoretical Generalization: Can you extend the theoretical analysis to demonstrate how LNR optimizes a wider range of performance metrics, particularly for multi-class settings, beyond the binary case?\n\nMetric-Specific Comparisons: Why were methods that directly optimize the evaluated metrics (e.g., F1-score, overall accuracy) not included in the experiments, and how would LNR fare against them?\n\nFeature Extraction and Fairness: How do you ensure that label noise does not disrupt feature extraction or compromise class fairness across diverse datasets, and can you provide additional evidence beyond the CIFAR-10-LT confusion matrix?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uK8QULnH3y", "reviewer": "Reviewer_VaeL", "summary": "In this paper, the authors study the problem of class imbalance. To be specific, they propose using asymmetric label noise in favor of the minority classes to mitigate the bias on the decision boundary between majority and minority classes. To this end, the authors formulate Bayesian optimal decision boundaries in an imbalanced setting for accuracy and F1 scores. Then, they introduce a label noise rebalancing approach based on the cardinalities of samples in classes. The proposed approach is evaluated across a diverse set of problems.\n\n## update after rebuttal\n\nI've read the comments by other reviewers and the rebuttal provided the authors. The rebuttal addressed my concerns and I increased my recommendation accordingly.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reply to Relation To Broader Scientific Literature & Essential References Not Discussed & Weakness 1 and 2**\n\n- We sincerely appreciate you providing these books and the two recent articles—they are very insightful. We recognize that some of these prior studies share similar conclusions with the theoretical portion of our paper, and we will adequately acknowledge these overlaps in our revised manuscript. Additionally, we will supplement our discussion with existing findings on how class imbalance affects decision boundaries, including the articles you kindly pointed out.\n\n- However, we would like to kindly emphasize that the core contribution of our theory lies in proving **the impact of our proposed asymmetric instance-dependent label noise model on decision boundaries under class imbalance**. Like other imbalanced learning methods, the mechanism of how data imbalance biases the classifier's decision boundary serves as the motivation for our theoretical framework. Yet, by introducing asymmetric label noise through our defined beneficial label noise model, we theoretically demonstrate how our beneficial asymmetric noise modifies decision boundaries and empirically validate that this approach effectively and beneficially corrects biased decision boundaries.\n\n\n**Reply to Weakness 3 and 4**\n\n- **W3**: We appreciate your suggestions about our baseline comparisons and relevant works.  \n    * In response to Reviewer AMhG, we have expanded the discussion of related literature (i.e. logit-adjustment approaches and metrics-optimization approaches). \n    * We also included new experimental comparison results with MiSLAS, ReMix, and SelMix (most recent SOTA) in our response to Reviewer k9AK. We further clarified some fundamental differences between our method and SelMix. We hope these revisions and explanations adequately address your concerns, and we are deeply grateful for your insightful feedback.\n\n- **W4**: We sincerely appreciate your suggestion regarding comparative experiments on large-scale datasets. We fully acknowledge that such comparisons could provide a more comprehensive evaluation of our approach. \n     * We are currently conducting additional comparisons on ImageNet with state-of-the-art methods. As these experiments require more time to complete, we will promptly share the results with you should they become available during the rebuttal period.\n    * In addition, we would like to re-emphasize our primary research focus has been on validating the feasibility and effectiveness of feature-dependent asymmetric label noise in addressing the class imbalance, which we have thoroughly demonstrated through extensive experiments on CIFAR-10/100 under both step-wise and long-tailed imbalance scenarios.", "claims_and_evidence": "I would say yes, to a large extent.", "methods_and_evaluation_criteria": "Yes, to a large extent.", "theoretical_claims": "No, since most of these are already known in the literature.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "The paper does not sufficiently position its analysis with prior work providing the same or similar analyses / findings on the impact of class imbalance on the decision boundary.", "essential_references_not_discussed": "The following work which has analyzed the impact of imbalance on the decision boundary:\n\nBishop's book or articles such as \"Adjusting the Outputs of a Classifier to New a Priori Probabilities: A Simple Procedure\", \"Rethinking Class Imbalance in Machine Learning\", \"To be Robust or to be Fair: Towards Fairness in Adversarial Training\".", "other_strengths_and_weaknesses": "Strengths:\n\n+ The proposed approach is novel and well-motivated.\n+ The improvements over the baseline methods are very strong, especially on minority classes.\n+ The paper is well-written and easy to follow.\n\nWeaknesses:\n\n1. \"Motivated by the decision boundary distortion due to class imbalance,\" => But, this is known already in the literature. It is not clear how much of the analysis on the impact of imbalance on the decision boundary is known in the literature (even in textbooks like Bishop's book or articles such as \"Adjusting the Outputs of a Classifier to New a Priori Probabilities: A Simple Procedure\", \"Rethinking Class Imbalance in Machine Learning\", \"To be Robust or to be Fair: Towards Fairness in Adversarial Training\") and how much is new in the paper.\n\n2. Some arguments such as the following are not surprising (and not sure whether it is new?): \"This paper theoretically shows that optimizing for accuracy leads to a decision boundary in binary classification misaligned with metrics like the F1 score.\"\n\n3. The baselines should have included a simple decision-boundary-based approach similar to Reject Option Classification and more advanced and recent logit-adjustment approaches such as DRO-LT.\n\n4. The experimental evaluation should have included more challenging long-tailed datasets such as iNaturalist, ImageNet-LT, Places-LT.", "other_comments_or_suggestions": "None.", "questions_for_authors": "Please see Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Hw8tFazMae", "reviewer": "Reviewer_k9AK", "summary": "This paper proposes a new data-level approach called Label-Noise-based Re-balancing (LNR) to solve the imbalanced learning issue. LNR utilizes the introduction of asymmetric label noise to adjust decision boundaries and improve classifier performance, particularly for minority classes. Unlike existing approaches, LNR highlights the potential of beneficial label noise without introducing generative errors or losing information.\n\n## update after rebuttal\nAfter reading the authors' responses, I have raised my score to 3.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Reply to Information Loss caused by label flipping in Claims And Evidence and Weakness 2**\n- We sincerely appreciate your insight on potential information loss due to label flipping. In LNR, the majority-class samples selected for flipping are primarily outliers that have deeply encroached into the minority-class region beyond the decision boundary. In this way, LNR enriches the minority class while preserving the core distribution of the majority class.\n\n-  Compared to undersampling approaches that discard large numbers of majority class samples (especially in extreme imbalance scenarios), LNR requires significantly fewer data-editing (only 94 label flips on CIFAR10-LT), mitigating information loss.\n\n- The number of flips can be regulated through the threshold $t_{flip}$, which can be optimally selected through cross-validation to avoid unexpected negative impacts on majority class performance. Sensitivity analysis (Appendix C.3) and Tables 1–2 show that with a well-calibrated $t_{flip}$, LNR enhances minority-class performance without compromising overall accuracy—a justified trade-off given the empirical gains. \n\n- In the revised version, we will adjust our claim on \"information loss,\" acknowledging that trade-offs (and information loss) exist but can be well-controlled.\n\n**Reply to comment in Essential References Not Discussed section and Weakness 1**\n- We appreciate your valuable feedback on incorporating more recent literature. We will add discussions on the latest approaches in imbalanced learning (please refer to our response to Reviewer AMhG for details).\n\n- To strengthen our comparative analysis, we have included comparisons with state-of-the-art methods such as MiSLAS, ReMix, and SelMix. However, we encountered some practical challenges in implementation: ReMix does not provide official code, and SelMix lacks reproducible code for its supervised version. Experiments on CIFAR100 and ImageNet require more time for proper adaptation and reproduction due to framework-specific complexities. We will include these results during the rebuttal period to provide a more comprehensive comparison once we have the results. Below are the CIFAR10 (IR=100) results:\n\n|| CIFAR10-LT (IR=100) | | | | |\n|:-:|:-:|:-:|:-:|:-:|:-:|\n||Overall Accuracy|Many-shot|Medium-shot|Few-shot|ECE ↓|\n|MiSLAS-stage2 |82.1|**91.0**|**80.2**|75.7|**3.70**|\n|MiSLAS+ReMix  |82.9| 90.0|79.8|79.8|19.6|\n|MiSLAS+SelMix* |83.3| - | - | - | - |\n|MiSLAS+LNR|**83.4**|87.6| **80.1**|**83.6**|**4.26**|\n\n**$\\bigstar$ It is worth highlighting that SelMix requires an additional balanced validation set of size 5000, while our LNR does not, yet still achieves comparable better performance to SelMix.**\n\n**Reply to Weakness 1**\nWe thank the reviewer for noting our method's and SelMix's conceptual similarities. While both methods share the common goal of improving imbalanced classification, key differences exist:\n\n- SelMix computes a gain matrix using class centroids from a **balanced validation set** to guide its mixup sampling at the **class level**. While this shares some conceptual similarities with our label noise selection process based on feature similarity ranks, LNR differs fundamentally: We **only flip labels** of majority class samples that exhibit **instance-level** feature similarity to minority classes **without requiring any auxiliary data** and **no feature mixing nor sample generating**.\n\n- In LNR, any sample with features similar to those of the minority class may be flipped, **regardless of how far or close the feature centroids of the two classes are**. For instance, in Figure 1, although the feature centroid of majority class 2 is relatively closer to class 9, the samples flipped to class 9 by LNR primarily come from classes 4 and 8, as these show stronger instance-level similarity to the minority class.\n\nNotably, as shown in our CIFAR-10 experiments, LNR achieves comparable performance to SelMix while **eliminating its dependency on balanced validation data**. This makes our approach more practical for real-world scenarios where such **balanced datasets may be unavailable**. \n\nWe will expand our discussion of these comparisons in the revised manuscript (Section 3.2) to better highlight LNR's unique contributions to the field.\n\n**Reply to Weakness 3&4**\n\n- W3: Our KEEL experiments with classic ML methods (KNN, CART) primarily validate our theoretical results and demonstrate the LNR's **model-data-agnostic** advantages for offering higher performance by comparing with classical methods for **tabular** data. The comparison with recent methods is conducted on long-tailed multi-class **image data** using DNN models, where these methods are explicitly targeted. Our results show consistent gains over algorithm-level methods and SOTA performance over data-level methods, highlighting our contributions.\n\n- W4: Kindly refer to the results and analysis of synthetic data in Appendix C.7 (after references), including tables of full results.", "claims_and_evidence": "***Claims***\nThere are two main claims made in this paper:\n1) Existing data-level approaches often lead to information loss or generative errors, and existing algorithm-level approaches are often tailored to specific models or problem settings.\n2) LNR alleviates the issues of information loss and generative errors and can be integrated seamlessly with any classifier or algorithm-level approaches.\n\n***Evidences***\nFor Claim 1, the related work section of this paper describes some existing data-level and algorithm-level approaches, and these descriptions support Claim 1.\nFor Claim 2, the proposed LNR does not generate instances, so it does not lead to generative errors. Experimental results show that LNR can be integrated with existing algorithm-level approaches. However, considering that LNR flips some majority class instances into minority class, I am unsure whether this might lead to information loss of the majority class.", "methods_and_evaluation_criteria": "Yes. The proposed LNR utilizes the introduction of asymmetric label noise to adjust decision boundaries, thereby improving classifier performance. In terms of evaluation criteria, this paper employs the F1 score, G-mean, AUC, and the average accuracy across Many-shot, Medium-shot, Few-shot, as well as the overall dataset. These evaluation metrics are widely used in other studies on imbalanced learning.", "theoretical_claims": "Yes. The paper provides detailed theoretical insights into the impacts of imbalance ratios on decision boundaries. I have reviewed these insights, and they seem to be correct.", "experimental_designs_or_analyses": "Yes. The experimental section partially validates the effectiveness of LNR. However, there are several issues in current experimental section. For example, it lacks recent comparison baselines, and there is no mention of synthetic experiments in the main text.", "relation_to_broader_scientific_literature": "This paper claims that existing data-level approaches often lead to information loss or generative errors, while algorithm-level approaches are often tailored to specific models or problem settings. To address these limitations, it introduces a novel data-level approach called LNR. LNR leverages asymmetric label noise to adjust decision boundaries, effectively mitigating issues of information loss and generative errors.", "essential_references_not_discussed": "Yes. This paper lacks references to some more recent and relevant studies in the field of imbalanced learning. I have listed some of them as follows:\n[1] Remix: Rebalanced Mixup. ECCV 2020.\n[2] Selective Mixup Fine-Tuning for Optimizing Non-Decomposable Objectives. ICLR 2024.", "other_strengths_and_weaknesses": "Strengths:\n1) This paper proposes a new approach called LNR for imbalanced learning. LNR is applicable to both binary and multi-class imbalanced problems and can be seamlessly integrated with any classifier or algorithm-level approach.\n2) This paper provides detailed theoretical insights into the impacts of imbalance ratios on decision boundaries and introduces asymmetric label noise to mitigate these impacts.\n3) This paper validates the effectiveness of LNR on multiple datasets using various evaluation metrics. Experimental results show that LNR outperforms its competitors in several settings.\n\nWeaknesses:\n1) In the field of imbalanced learning, several recent mixup-based approaches [1][2] have been proposed. Similar to this paper, these approaches perform mixing either at the feature level or across labels. From certain perspectives, the LNR proposed in this paper seems to be viewed as a special case of such approaches. However, this paper fails to mention or discuss these approaches.\n2) The paper claims that existing data-level approaches lead to information loss or generative errors, whereas the proposed LNR alleviates these issues. However, if a portion of the majority class instances is flipped into the minority class, why would this process not also cause information loss for the majority class?\n3) The experimental section lacks recent baselines, particularly in the binary classification setting (the latest baseline considered was published in 2017). This makes it challenging to accurately evaluate the contributions of this paper. More recent approaches, such as [1][2], should be included to provide a comprehensive comparison.\n4) The paper frequently mentions “synthetic/simulated and real-world datasets”, yet the experimental section in the main text provides no description of the experiments conducted on synthetic datasets. It is unreasonable to confine these experiments solely to the appendix, as they are necessary for the completeness of the paper and should be mentioned in the main text.\n\n[1] Remix: Rebalanced Mixup. ECCV 2020.\n[2] Selective Mixup Fine-Tuning for Optimizing Non-Decomposable Objectives. ICLR 2024.", "other_comments_or_suggestions": "1) On lines 73-74 of page 2, there are too many references (Goldberger & Ben-Reuven, 2022; Liu & Tao, 2015; ...... ; Zhu et al., 2003) that are unrelated to this paper.\n2) On line 240 of page 5, the end of Lemma 4.1 is missing a period.\n3) Algorithm 1 and Figure 3 are placed too far from the main text where they are mentioned.", "questions_for_authors": "1) In the field of imbalanced learning, several recent mixup-based approaches [1][2] have been proposed. Similar to this paper, these approaches perform mixing either at the feature level or across labels. From certain perspectives, the LNR proposed in this paper seems to be viewed as a special case of such approaches. However, this paper fails to mention or discuss these approaches.\n2) The paper claims that existing data-level approaches lead to information loss or generative errors, whereas the proposed LNR alleviates these issues. However, if a portion of the majority class instances is flipped into the minority class, why would this process not also cause information loss for the majority class?\n3) The experimental section lacks recent baselines, particularly in the binary classification setting (the latest baseline considered was published in 2017). This makes it challenging to accurately evaluate the contributions of this paper. More recent approaches, such as [1][2], should be included to provide a comprehensive comparison.\n4) The paper frequently mentions “synthetic/simulated and real-world datasets”, yet the experimental section in the main text provides no description of the experiments conducted on synthetic datasets. It is unreasonable to confine these experiments solely to the appendix, as they are necessary for the completeness of the paper and should be mentioned in the main text.\n\n[1] Remix: Rebalanced Mixup. ECCV 2020.\n[2] Selective Mixup Fine-Tuning for Optimizing Non-Decomposable Objectives. ICLR 2024.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes a novel Label-Noise-based Re-balancing (LNR) approach to address the challenge of imbalanced learning in classification tasks. The authors theoretically analyze how class imbalance biases decision boundaries and demonstrate that introducing carefully designed asymmetric label noise can correct these biases, improving classifier performance, particularly for minority classes. Unlike traditional resampling methods, LNR avoids information loss and generative errors by flipping labels of majority-class samples that exhibit feature similarity to minority classes. The method is model-agnostic and can be integrated with existing classifiers or algorithm-level approaches. Extensive experiments on synthetic and real-world datasets (including CIFAR-10/100 and KEEL datasets) validate LNR's superiority over baselines.

The paper received generally positive reviews, with reviewers acknowledging its ​​novelty​​, ​​theoretical grounding​​, and ​​empirical effectiveness​​. While initial concerns were raised, the authors' rebuttal and additional experiments convincingly addressed these issues, leading to improved reviewer scores. Thus, I recommend ​​acceptance​​.
Decision: Accept (poster)