An Embedding is Worth a Thousand Noisy Labels
Authors: Francesco Di Salvo
Sebastian Doerrich
Ines Rieger
Christian Ledig
Venue: Accepted by TMLR ()

Abstract: The performance of deep neural networks scales with dataset size and label quality, rendering the efficient mitigation of low-quality data annotations crucial for building robust and cost-effective systems. Existing strategies to address label noise exhibit severe limitations due to computational complexity and application dependency. In this work, we propose WANN, a Weighted Adaptive Nearest Neighbor approach that builds on self-supervised feature representations obtained from foundation models. To guide the weighted voting scheme, we introduce a reliability score $\eta$, which measures the likelihood of a data label being correct. WANN outperforms reference methods, including a linear layer trained with robust loss functions, on diverse datasets of varying size and under various noise types and severities. WANN also exhibits superior generalization on imbalanced data compared to both Adaptive-NNs (ANN) and fixed k-NNs. Furthermore, the proposed weighting scheme enhances supervised dimensionality reduction under noisy labels. This yields a significant boost in classification performance with 10x and 100x smaller image embeddings, minimizing latency and storage requirements. Our approach, emphasizing efficiency and explainability, emerges as a simple, robust solution to overcome inherent limitations of deep neural network training.

Published in Transactions on Machine Learning Research (03/2025)
An Embedding is Worth a Thousand Noisy Labels
Francesco Di Salvo
francesco. shai::ooddddaaaoooououuuuuaaaaa de
zAllab Bambery University of Bamberg, Germany
Sebastian Doerrich
sebastian, sebustian.derrichhuuut----eeee
aAlLab Bambery
University of Bamberg, Germany
Ines Rieger
ines. aueesrregeerummi-bbmberr... de
aAILab Bamberg
University of Bamberg, Germany
Christian Ledig
christiam. Ceaiiimmeeeddguuunaamamaeeeee
aAlLab Bambeery
University of Bamberg, Germany
Reviewed on OpenReview: https: // openreview. net/ hta//forur forum? id= X3gSvQjShh
Abstract
The performance of deep neural networks scales with dataset size and label quality, rendering the efficient mitigation of low-quality data annotations crucial for building robust and cost- effective systems. Existing strategies to address label noise exhibit severe limitations due to computational complexity and application dependency. In this work, we propose WANN, a Weighted Adaptive Nearest Neighbor approach that builds on self-supervised feature rep- resentations obtained from foundation models. To guide the weighted voting scheme, we introduce reliability score y. which measures the likelihood of a data label being correct. WANN outperforms reference methods, including linear layer trained with robust loss func- tions, on diverse datasets of varying size and under various noise lypes and severities. WANN also exhibits superior generalization (ANN) and fixed k-NNs. Furthermore, the proposed weighting scheme enhances supervised dimensionality reduction under noisy labels. This yields a significant boost in classification performance with 10x and 100x smaller image embeddings, minimizing latency and storage requirements. Our approach, emphasizing efficiency and explainability. emerges as a simple, robust solution to overcome inherent limitations deep neural network training. The code is available at
imbalanced data compared to both Adaptive-NNs
1 Introduction
The remarkable results achieved by deep neural networks in a multitude of domains are possible due to ever- increasing computational power. This progress has enabled the development of deeper architectures with stronger learning capabilities. However, these higm-parametric architectures are often characterized data- hungry, because they require large amount of data to generalize effectively. Collecting and annotating such extensive datascts can be both expensive and time-consuming, potentially introducing machine and human errors. In fact, the proportion of corrupted labels in real-world datasets ranges from 8% to 38.5% (Song ett al. 2022). While unsupervised and semi-supervised methods have garnered significant attention within the rescarch community (Chen et al. 2020 Tarvainen & Valpola 2017 Berthelot al., 2019), supervisod methods are still widely employed due to their generally higher performance. In this context, incorrect labels pose significant challenges, including a tendency to memorize label noise, negatively affecting generalization

---

Published in Transactions on Machine Learning Research (03/2025)
capabilities. This issue, demonstrated by Zhang et al (2021) is hindering the adoption of Al systems in safety-critical domains, such as healthcare. Consequently, there is a growing interest in enhancing the robustness of deep models against noisy labels, resulting in five different lines of research (Song et al. 2022]: robust architectures, robust regularization, robust loss function, loss adjustment and sample selection. Nevertheless, as also highlighted by Zhu el al. (2022), all these approaches involve a learning process and, by definition, have limitations in generalizing to diverse datasets or varying noise rates. We address those limitations and the challenges posed due to noisy labels by exploiting feature representations obtained from large pre-trained models, commonly referred to as foundation models. Many of these foundation models are now publicly available and designed for wide range of applications, including image classification, semantic segmentation tasks, and beyond. While their initial focus was primarily on text (Devlin et al. 2019 Brown el al. 2020) and natural images (Dosovitskiy et al.) 2021 Caron et al., 2021 Oquab et al.
2023 Radford et al. 2021), there is currently a focus on the development of open-source foundation models tailored to specific domains, such as healtheare (Tu et al.) 2024 Li ct al. 2023; Zhou al. 2023 Xu et al., 2024), making it possible to translate this paradigm to various applications with little ellort. Specifically, large image-based foundation models are usually trained in sele-super fashion, e.g., by means of contrastive learning (Chen et al.) 22020), representing similar objects closely within the embedding space, and semantically distinct objects further away. Thus, the position of an image in the embedding space can be just as informative as the label, if not more so. Although embedding-space approaches are not new in the literature to process noisy labels, they typically focus on noise detection, through online (Bahri et al. 2020) or offline (Zhu et al.] 2022) methods. However, thanks to the representational power of modern embedding spaces (Radford et al.) 2021 Oquab et al., 2023), we believe that it also possible to provide robust simple k--N methods. This category of approaches not only offers computational efficiency but also enhances explainability. Moreover, the lower reliance on hyperparameters further enhances generalizability. Thus, owing to its simplicity and efficiency, it is able to mitigate some of the limitations associated with the training of deep networks. In this work, we propose a Weighted Adaptive Nearest Neighbor (WANN) approach, illustrated in Figure | which operates on the image embeddings extracted from a pre-trained foundation model. The introduced weighted adaptive voting scheme addresses the challenges posed by large-scale, limited, and imbalanced noisy datasets. Our contributions are:
predictions by building
Formulation of a reliability score (n), measuring the likelihood of a label being correct. This scoring guides the formulation of WANN, a method that determines a neighborhood of the test sample to weight an adaptively determined number of training samples in the embedding space.
Extensive quantitative experiments, confirming that WANN has overall greater robustness compared to reference methods (ANN, fixed k-NN, robust loss functions) across diverse datasets and noise levels, including limited and severely imbalanced noisy data scenarios.
Formulation of the Filtered LDA (FLDA) approach that relies on our samplc-specific reliability scores for higher quality projections. FLDA improves the robustness for dimensionality reduction by filtering out detected noisy samples and improves the classification performances with 10x and 100x smaller image embeddings.
Consequently, through weight embeddings and a simple yet efficient classification algorithm, we demon- strate that working efficiently within the embedding space might constitute a potential paradigm shift, increasing efficiency and robustness while addressing critical limitations related to model training.
2 Related works
Noise robust lcarning Robust learning from noisy labels has received considerable attention in recent years, and multiple deep-learning-basee methods are currently available (Song et. al., 2022). Two popular categories of approaches are sample selection and loss adjustment. Methods based on sample selection aim
to isolate correctly labelled samples from the noisy dataset, often employing stratogics such as multi-network
learning, iterative data filtering, or both HHan et al.) 2018 Jiang al., [0018] Song et al.] 2019: Li et al.
2020 Wei et al. 2020:
Xiao et al. 2023 Zhang et al.] 2024 Fooladgar et. al. 2024). In contrast, loss
2

---

Published in Transactions on Machine Learning Research (03/2025)
Reliability estimation
Inference
k 13,
Class A
Class B
Class C
k=3
3
Test (xT)
15
11
11
kT
arg max 1ggmax(11,1142 (1/1,)
3
13
73
Figure 1: Illustration of the proposed Weighted Adaptive Nearest Neighbor (WANN) algorithm. Initially, a reliability score (1) is computed for each training observation, representing the inverse of the minimum number of samples needed for a correct prediction. During inference, the adaptive neighborhood size (KT) of each test observation (xT) is determined based on the reliability score of its closest training sample (ke 3). A weighted majority vote determines the final label, reducing the impact of noisy labels.
adjustment techniques modify how the loss is computed for every sample, thereby mitigating the impact of label noise (Patrini et al.) 2017 Tanaka et al.) 2018 Liu et al. 2020 Zheng et. al. 2021). However, due to their computational complexity, these models often lack generalizability across datasets or noise settings. Some of the lightweight techniques designed to handle noisy labels fall under the category of robust loss functions. While the Cross Entropy (CE) suffers from overfitting to wrong labels (Zhang el al.. 2021), the Mean Absolute Error (MAE) has been theoretically guaranteed to be noise-tolerant (Ghosh et al. 2017). However, it suffers from severe underfitting in challenging domains. To address this issue and enhance its generalization, the Generalized Cross Entropy (GCE) (Zhang & Sabuncu 2018) was introduced as a generalization of both MAF and CF. While Wang et al. (2019) propose the Symmetric Cross Entropy (SCE), Zhou et al. (2021) overcome the symmetric condition through Asymmetric Loss Functions. Active Passive Losses (APL) (Ma al., (020)) which combines two robust losses to balance between overfitting and underfitting. Recently, Active Negative Losses (ANL) (Ye et al., 2023 2024) replaced the previous passive loss (in API.) with Normalized Negative Loss Function (NNLF). Nevertheless, despite their relatively lower computational complexity compared to multi-network methods, they still inherit challenges related to training neural networks. Some of these challenges include the demand for large datasets, lack of explainability, computational complexity, hyperparameter dependence, and overfitting towards wrong labels.
k-NN for label noise Besides native deep learning-based approaches, certain traditional machine learning methods are robust by design. Notably, -Neeares-NNiggbbos--bee methods have recently gained attention for their utility in filtering out noisy training observations (Reeve & Kabán 2019 Bahri et al. 2020 Kong
et al. 2020 Although the proposed online k-NN approaches can enhance the robust.ess of D trained with label noise, they do not address the fundamental issues associated with DNNs, such as the lack of explainability, data-efficiency, and generalizability. A closely related work (Zhu et al. (022) already em- phasized the potential a training-free cleaning strategy based on k--N with CLIP (Radford et al. 2021 embeddings, enhancing DNNs downstream performance. Furthermore, a concurrent work (Zhu al. demonstrated that a simple noise-agnostic linear probing on high-quality features outperforms a deep net- work trained from scratch, achieving satt--of-the-art results. In contrast, we show that training-free approach can outperform linear probing, with without robust loss functions. Indeed, our work represents broader paradigm shift: rather than training large networks from scratch, we utilize high-quality feature representations from foundation models and employ a simple, interpretable, and training- free classification scheme in the embedding space. This design choice directly addresses practical challenges such as mitigating overfitting to noisy labels, enabling transparent predictions (ef. Figure 2) and simplifying dataset updates. We further enhance classification performance under label noise and label imbalance through the adoption of an adaptive neighborhood for each test. observation.
2024
3

---

Published in Transactions on Machine Learning Research (03/2025)
(a) Wrong lest labels
(b) Ambiguuus test labels
TEST
TOP MOST SIMILAR TRAINING IMAGES
IMAGE
TOP MOST SIMILAR TRAINING IMAGES
testkat/1449ppp 29gg
3rantoogl533..gg boinkog4602eggg
tesUbhipp/778
trinnbid44144nn travanesse..rg trinbro/2194pg
testtruck/3 11688/16888888 1168.ppg
1einttar/67C..ng pri4
prainturr4733pr
test /1788
mwinloog522.ppr
tainkog44666pp png
tu twwntogg1761.pgg
Example of test images and their relative top-3 closest training samples extracted from the STL-10 dataset (Coales al. 2011). Figure 2a shows two test images having wrong labels, while Figure 2b shows ambiguous labels, including multiple known objects within single image. For instance, bird with ship in the background, or a dog fighting with a cat.
Figure 2:
3 Method
At its core, the proposed method relies on adaptive A-NN search over the image embeddings extracted from pre-trained foundation model, using the Euclidean distance as reference distance metric. In particular, we employ the DINOv2 (Oquab et al. 2023) Large backbone, with 14x 14 patches, yielding image embeddings of size 1024. DINOv2 was trained in a self-supervised fashion on 142M images, and the choice of the backbone is further motivated in Scction 4.1 To enhance computational efficiency, we pro-gencrated database of embeddings for all of the evaluated datasets, following the approach adopted by Nakata et al. (2022).
Label reliability score Although k-NN is robust against noisy labels by design, its efficacy may diminish under severe noise conditions. To address this challenge, the introduction of weighting scheme becomes helpful, aiming to mitigate the impact of noisy labels. Intuitively, proper weighting scheme should assign higher weights to samples that are more likely to have a correct label, and lower weights to those potentially mislabeled. In this context, we introduce the concept of label reliability, building upon the work [Sun & Huang (2010). Extending their approach, which inferred an optimal & value for each training observation, determine the reliability (ie.., quality) of the associated label, guiding subsequent weighting scheme. For given training observation (æ;, y:), define the reliability πi the inverse of the minimum number of training samples (neighborhood size) required for a correct k-NN classification. Dropping the index for readability, we express as the function H(x yi X),, where X' represents the training dataset:
H(., y;
(1)
arg min Me MMllKmin.kkma
(2,
Here kimin and kmmx define the search interval for K' E.N. It was previously shown by Zhu al. (2022) that for fixed k-NN approach suitable neighborhood size to detect corrupted labels is 10. Consequently, we set kmin 11 and kmax 51. This allows to establish substantial search space covering wide range of noise levels while ensuring a bascline ability to detect corrupted labels (kmin > 10). During paramcter scarch we increase k' by al each incremental step, ensuring an odd number of samples in the neighborhood while reducing the computational complexity. If there is no k' satisfying Equation we set the optimal = A The correlation between the value and the reliability of the assigned label stems from the main property of the embedding space. The proximity of similar objects suggests that small is should be sufficient for accurate classification. In such cases, the reliability of the data point is higher (k: 4 17 1)-
4

---

Published in Transactions on Machine Learning Research (03/2025)
Conversely, if data point requires large le value for correct classification, approaching reaching kmax: it may signal potential mislabeling (I 1 4). The pseudocode is presented in Algorithm | and it requires three parameters: kmin, kmax, and Draain The latter, Dwain, represents a data structure containing triple (key, z. y) for each train observation. Here, key serves as a unique identifier.
Algorithm reliability Aimax, Derain)
R R+ p PairviseDist(Drninn Dtrain)
D reliability map of key value pairs matrix
for (key. v) in Drrain do for k' kmin Kmin + 2,,... kmax do
+- k k)
k points excluding x
if 9 then R[key| + F break
end if end for
if not key in R. keysys() then R[key] end if end for return R
kmaa
Weighted Adaptive Nearest Neighbor (WANN) Adaptive Nearest. Neighbors algorithms were com-
monly employed for tabular data with a limited number of observations and classes (Geva & Sittc 1991
Wettschereck & Dietterich, 1993 Wang el al., 2006). Following the intuition of Sun & Huang (2010), we use 17 to adaptively determine the neighborhood size for any test observation. In fact, if test sample is close to potentially noisy label (17 4)) larger neighborhood is preferable to reduce variance. Conversely, if test observation is close to a possibly clean label (n 1), small neighborhood increases specificity. Thus, given test observation with nearest training sample (Ia, Un, 7h) (nearest in the embedding space), its test neighborhood size kt will be adaptively defined by means of the reliability score of In (cf. Equation 2
Kt
kn
(2)
TJn
With this notation we formulate an Adaptive Nearest Neighbor (ANN) method as bascline, similar to Sun & Huang (0100):
ANN( arg max- GEC
Σ 1yy NE
KiT
Here, C denotes the set of classes, Nr refers to the adaptive neighborhood determined by the by closest observations, and (.) is the indicator function. Finally, having defined the reliability score 17 = =HH, y: X),, enhance ANN by introducing weighting scheme. Thus, instead of merely taking the most frequent class in the neighborhood, we now associate a weight (i.e., 17) to each training observation. Consequently, training samples with higher reliability 7 will have a greater impact on the final prediction, while those with lower π) (possibly wrong labels) will contribute loss. As such, our Weighted Adaptive Nearest Neighbor approach is formulated as:
WANN
arg max CEC
Σ Th 11
(4)

---

Published in Transactions on Machine Learning Research (03/2025)
Filtered dimensionality reduction k-NN, like several other listance-based machine learning methods, is susceptible to the curse of dimensionality (Kouiroukidis & Evangelidis 2011). Tn high-dimensional spaces, the relative distances between data points become less discriminative, inevitably affecting the algorithm's 's performance. To address this issue, dimensionality reduction techniques can be applied to reduce the dimen- sionality of the feature space with minimal information loss. Common choices for dimensionality reduction include linear methods like Principal Component Analysis (PCA) (Jolliffe 2002) and Linear Discriminant Analysis (LDA) (Belhumeur et al., 1996).
While PCA seeks directions of maximum variance, LDA leverages class labels to maximize the separation between classes and minimize variance within classes, according to the Fisher-Rao criterion. Due to its supervised nature, LDA relies on correct labels, as label noise can result in inaccurate linear mappings. To address this limitation, we employ WANN's reliability score 7) to filter out potentially wrong labels (i.e., 7= m// before performing LDA. We term this approach Filtered LDA (FLDA). Once the projection axes have been obtained through the clean dataset, we subsoquently project both training and test datasots onto those axes.
4 Experiments and results
Initially, in Section
we justily the choice of the DINOv2 Large backbone (Oquab el. al. 2023 Next, in 1.5 we evaluate WANN against robust loss functions on noisy real-world, limited, and medical we assess the robustness of the weighted adaptive neighborhood in the context of
Sections 1.2
data. In Section 1.6 heavily imbalanced datasets. We further demonstrate the effectiveness of the proposed Filtered LDA (FLDA) approach in Section 4.7 Moreover, in Section 4.8 we qualitatively explore the explainability benefits of our approach on noisy datasets. It is important to note that none of the datasets used in our quantitative experiments were part of DINOv2's pretraining data, as detailed in Table 15 of (Oquab et al., 2023). Finally, [ demonstrate the generalizability of WANN utilizing an additional backbone and its robustness with respect to the proposed reliability score, respectively.
Appendix
4.1 Backbone
In order to achieve satisfactory k-NN performance, high-quality feature representation is essential to ensure close proximity of similar objects. To evaluate this, we initially compare the performance of ResNet50 and ResNet101 (He et al., 2016), both pre-trained on ImageNet-lk lkk In the realm of Vision Traanformers,
we explore self-supervised pre-training with MAE (He et al. 2022) (Basc and Large). CLIP pro-training Radford et. al. (2021) (Base and Large), and lastly DINOv2 pre-training (Oquab et al.) 2023) (Base and Large). All pre-trained models are obtained from HuggingFace (timm). We report WANN's classification accuracy on CIFAR-10 and CIFAR-100 (Krizhevsky et al. 2009), utilizing (kmin: kmax) (11,51).
Results Table [ displays the results, clearly confirming DINOv2 as backbone choice. Results further reveal a notable decline in performance for all backbones as the number of classes increases. This trend is expected due to increasing overlap of class-specific feature distributions within the embedding space. Additionally, obtaining ground truth labels for a larger set of visually more ambiguous classes poses more challenges associated with label quality. Notably, comparing MAE ViTs with ResNets, both pre-trained ImageNet, it is possible to observe substantially lower performance of MAE ViTs. Despite the higher number of parameters, the self-supervised pre-training paradigm appears to be not beneficial in this context. Furthermore, the discrepancy between CLIP and DINOv2 can be attributed to the divergent pre-training stralegies: CLIP is trained on pairs of image-text, whereas DINOv2 exclusively relies on image data. Beyond computational efficiency, the DINOv2 framework allows to disregard potential ambiguities introduced by textual descriptions. For this reason, it also required considerably less data, as DINOv2 was trained on 142 millions of images while CLIP on 400 millions of image-text pairs. Thus, in scenarios where text embeddings are not explicitly needed, as in our case, opting for an image-centric training paradigm is beneficial.
6

---

Published in Transactions on Machine Learning Research (03/2025)
Backbone
#Par
#Dim
CIFAR-10
CIFAR-100
ResNet50
26M
2048
84.09
60.31
ResNet101
45M
2048
87.32
64.59
MAE VIT-B/16
86M
768
72.04
45.18
MAE VIT-I/16
303M
1024
82.99
56.18
CLIP VIT-B/16
86M
768
92.24
68.96
CLIP VIT-L/14
304M
1024
95.65
75.69
DINOv2 ViT-B/14
S7M
768
98.70
89.36
DINOv2 VIT-L/14
304M
1024
99.36
91.68
Table 1: WANN's classification performance (accuracy 1) over eight different feature spaces, on CIFAR-10 and CIF,R-100, which contains 10 and 100 classes, respectively. Note that "#Par" denotes the number of parameters and "#Dim" denotes the feature dimension.
4.2 Real-world noisy labels
We compared ANN and WANN with lightweight techniques falling into the category of robust loss functions. As commonly employed for assessing the feature space quality (Oquab et al., 2023). we trained a single linear layer over the image embeddings. The embeddings were normalized using training set statistics, excluding 15% for validation in lincar training. The lincar probing utilizes the Adam optimizer with learning rate of 1x10- for 100 epochs, with early stopping after 5 epochs. These settings were consistently applied to all experiments in this manuscript. Following the publicly available losses and hyperparameters proposed by Ye et al. (2023), we compared: Cross Entropy (CE) as a bascline, Focal Loss (FL), Mean Absolute Error (MAE), Generalized Cross Entropy (GCE), Symmetric Cross Entropy (SCE), Active Passive Losses (NCE-MAF, NCF-RCE,, NFI-RCE)), Asymmetric Losses (NCF-AGCE, NCF-AUT, NCF-A AL.). and Active Negative Losses (ANL-FL.ANL-CE- (ELR) (Liu et al. 2020), widely adopted bascline related to label correction. Since the embeddings are pre-generated, no data augmentation was applied. Both ANN and WANN use (kmin: kimax) = (11,51). We measure the accuracy on two publicly available datasets, namely CIFAR-101 1ON and CIFAR-10ON (Wei et al, 2022), which are "noisy versions" of the established CIFAR datasets. CIFAR-ION contains three human annotations per image, proposing different noisy training splits with varying levels of noise rates (NR). Conversely, CIFAR-100N presents solely one annotation per image, thereby offering one single noisy split. Due to the potential sensitivity of linear training the choice of the random seed, we show the accuracy the mean and standard deviation computed over five runs.
Additionally, we include Early Learning Regularization
Results Table 2 shows the results of the designed experiment for real-world noisy labels. The introduced weighting scheme in WANN substantially enhances robustness compared to ANN, particularly in high-noise conditions. Additionally, WANN exhibits greater advantages compared to robust loss functions in scenarios with limited noise ratios (< 20%), showing a notable lower performance drop than robust loss functions on CIFAR-1ON Aggr and R1, respectively. Active Negative Losses (ANL-CE-ER and ANL-FE) rank among the top robust. loss functions, approaching WANN's performance. Furthermore, while NCF-AGCE exhibits higher accuracy CIFAR-10ON compared to WANN, it notably presents higher performance drop with respect to the clean data (7.15% versus 6.24%), indicating lower robustness.
4.3 Limited noisy data
The inherent limitation of deep models lies in their reliance on a large amount of clean annotated data. In contrast, k-NN models demand less data, exhibiting potential robustness, particularly the presence of label noise. To demonstrate this, two subsets are randomly sampled from CIFAR-10 10, each containing only 50 and 100 samples per class. This experiment compares robust. loss functions against ANN and WANN.

---

Published in Transactions on Machine Learning Research (03/2025)
CIFAR-1ON
CIFAR-100N
Noisy split NR (=)
Clean
Aggr.
R1
R2
R3
Worst 40.21%
Clean
Noisy
9.01%
17.23%
18.12%
17.64%
40.20%
CE
99.40
98.46++0.07
08,29+0.11
98. 38.33=0.11
98,45+0000
03.20
93.38=0.18
03
FL
99.39-0.02
08.3
98. 18.12++0.12
98 21-0.07
98, 10.01
02.85++..18
93.30=0.10
82.03++0.22
MAR
99.4220.0r
99.21 1:000
99.2210.06
29.17
99.2010.00
95.7910.1
93.4020.
86.0819.2
GCE
99.39
99. 1:0.01
99.061
98.9620.00
99.03.1..0
95.
93.4520.0r
85.741..32
SCE
99.42=0.00
99. 0.000
99.11
99.07=0.0s
19.124..05
95.52
93. 73.40.2014
05
ELR
99.44=0.02
99.13.10..10
99.09 .8
99.14
99.141..04
97.72
93.34=0.04
86.10:0.22
NCE-MAE
99.42:0001
99.171:0.02
99.15 04
99.17
99.15:10.05
95.35
93.46 1.446:2000
85.44110.40
NCE-RCE
99.42.00.0 .02
99.21 10.00
99.21 1:0 07
19.15
99. 19..9.s
95.5810 33
03.47=0.00
86.08110.08
NCL
99.42g0.0 02
99.21 10.00
99.21 1:0 07
10.15
19.191o.08
95.5810
93.4620.07
86.05.10.11
NCE-AGCE
99.42=0.02
99.21 +0.00
99.21++0 +0025
99.18-0.03
99.18+o.08
95.93++.45
93.40=o.0r
86.25+0..12
NCE-AUL
99. 99.42:0.00 0022
99.20:40.o
99.21 .0
99.1820.04
99.18
95.77
93.41 20.09
85.81
NCE-AEL
99.41=0.00 .0.
99.14410.01
99.08 o6
99.04.0.0a
99. 09.1120.00
95.33
93.43.0000
84.7910.58
ANL-PL
99.39=0.00 0.04
99.25 +0000
99.21+0.04
99.19=0.0a
99.24 0.000
98.23
90.77=0.20
84.19+0.44
ANL-CE-ER
99.39-0.04
09.24+0.06
99.19+0.10
99.20=0.0a
99.25+0..s
98.28
70.76-0.24
84.98+00.9
ANN
99.37
99.19
99.10
99.06
98.97
95.31
91.99
84.91
WANN
99.36
99.32
99.27
99.19
99.24
97.21
91.68
85.44
Table 2: Accuracy (1) on two clean and noisy datasets (Wi et al. 2022). The two best results are highlighted in bold. Note that NR stands for Noise Rate. CIFAR-1ON includes three annotations per image, and Noisy splil denotes various aggregation strategies. Following the proposed naming convention, Aggr refers to a majority voting, R-i (i (1,2,3) denotes the i-th submitted label for each image, and Worst denotes the selection of only wrong labels, if applicable. Conversely, CIFAR-100N presents just one annotation per image, thereby offering one single noisy split.
Due to the limited dataset size, we further extract an additional noisy validation set of 250 samples (i.e., 25 samples per class). However, WANN does not require this additional data, which often difficult to obtain in limited-data settings.
Artificial noise is introduced in CIFAR-10 using traditional methods: symmetric (Patrini et al. 2017 asymmetric (Scott et al. 2013), and instance-dependent (Xia et al., 2020) noise. Symmetric noise involves the random switching of one label with any other label in the dataset. Asymmetric noise flips a specific label to another fixed, incorrect label (e.g., "deer" mislabeled as "horse"), simulating systematic class confusion. Finally, instance-dependent noise takes into account the relationships between the features within the dataset. Following Zhu et al. (2022), we report. the accuracy on 60% symmetric noise, 30% asymmetric noise, and 40% instance-dependent noise. Following Ye et al. (2023), our asymmetric flips are defined as: TRUCK- > AUTOMOBILE, BIRD AIRPLANE, DEER HORSE, CAT <> DOG. Due to the stochasticity in subsampling and noise generation, we average the results across five seed runs.
Results As shown in Table 3 across all experiments (i.e., two sample sizes and three noise injections each), WANN is clearly superior to both ANN and robust loss functions. The advantage of adaptive methods over robust loss functions particularly evident under heavy noise conditions (e.g., 60% sym.). Although this performance gap narrows slightly as data size increases (f.e., under less challenging conditions), the weight- ing scheme introduced in WANN continues to demonstrate clear benefits over ANN with minimal overhead. Moreover, paired t-test (p 0.05) conducted across five seed runs confirms that WANN is the significantly best method in all but two cases (ef. Table 3). Furthermore, with 100 samples per class, WANN consistently exhibits lower standard deviation across seed runs, thereby confirming its higher robustness against compet- ing methods. Figure 3 illustrates that under asymmetric noise, incorrect labels may cluster closely in the embedding space. This in fact poses challenges for the traditional k-NN but is mitigated by WANN. Notably, while linear methods approach and surpass the performance of ANN with 100 samples per class, they do not perperform WANN.
8

---

Published in Transactions on Machine Learning Research (03/2025)
CIFAR-10 (#50)
CIFAR-10 (#100)
Pattern
Clean
Symmetric 60%
Asymmetric 30%
Instance 40%
Clean
Symmetric 60%
Asymmetric 30%
Instance
NR
40%
CE
95.06 35.06=1.71
68.41
08.77
84.58=1.43
96.98=0.3 00.00
81.17+2.62
90.97+1.48
88.14=1.55
FL
94.66-1.6
68.19+3.24
88.30+1.00
81.41+1.83
96 85
80.33++0.0
89.89+1.00
85.88+1.14
MAE
95.6820
83.9814.41
11
92.41 2
97. 97.64
93.3311
95.5510.74
94.9621
GCE
96.25.1
82.35.13.15
92.22
93.20 20.20
97. 07.48810.0 0.44
91.81 12.00
94.85
94.80.2
SCE
96.05=0
84.52:44.01
92.73+0.00
90.91
97.78
93.01
95.06
95.63=1.088
ELR
95.69=0.1z 73
85.26:t3.99
2.20
90.38
96.93
91.18:14.06
95.32.:0.89
94..55
NCE-MAE
96.19=00m0
83.5413.54
02.84
92.45
97. 97.75.2032
93.02111
94.9611.32
95.1821.00
NCE-RCE
96.00
83.9214.22
93.28
91.1821.13
97. 97.65.20.2
93.3211 84
95.57 0.
95.03
NFL-RCE
96.00
83.9214.24
93.2811.00
91.18
97. 07.65220.2
93.31 .95
95.57
95.03
NCE-AGCE
96.05=0.47
83,98++4.20
93.27+11110
91.01+3.96
97 0..80
93.06++2.1
95.52+0.68
95.06 95.06+1.11
NCE-AUL
96. 00.04
83.84144.26
02.75
90. 25
97. 0..21
91.04::4.5
94.72+00.9
95.61 25
NCE-AEL
96.26.0.05
82
93.4221.16
92.43 1.0
97.3620.31
92.30 81
1.
95.17:20.4
ANL-FL
93.32=2.02
78.54+2..0
91.15#11.78
57.84+1.98
96.90=o.om 5o
87 87.66
92. 1.30
91.30=2.0 66
ANL-CE-ER
95.16=0.99
76.20+0.68
91.16+1.50
+6.799
96.59+1.00 .00
90.36+1106
93.55++166
92.44+0.70
ANN
97.42 aa
92.26 1.00
93.9611.77
92.63
98.24
95.83.10.49
95.6411..4
95.60
WANN
97.2920.38
22.28
95.00
93.77
98.181.
96.79.0.42
96.9240.25
96.84_0.40
Table 3: Accuracy (1) on limited noisy data settings. Both experiments are conducted on CIFAR-10, with 50 noisy samples per class (left) and 100 noisy samples per class (right). We bold the top two methods and underline the significantly best one if their difference is statistically significant (paired I-test, p < 0.05).
4.4 Limited real-world noisy data
We further assess classification performance on four stratified subsets of Animal-10N (Song et al. 2019) consisting of (500, 1000, 2500, 5000} total samples. It is widely adopted dataset for label noise benchmarks (Ye et al. 2023) and includes 50,000 samples across 10 classes, with approximately xs 8% noisy labels. We compare the performance of ANN and WANN against three representative loss functions (cf. Figure 4 As in Section 4.3 solely for linear probing methods, we extract an addilional noisy validation set of 250 samples. Furthermore, we do not tune loss-specific hyperparameters for this dataset; instead, we used those found for CIFAR-10. We report the results obtained across five seed runs.
As shown in Figure while linear probing approaches k-NN methods (de.. ANN and WANN) with larger subsets, its offectiveness diminishes with limited data. Specifically, when reduced to about 50 (noisy) samples per class, linear methods substantially underperform against AMN and WANN, exhibiting a larger 95% confidence interval. This performance gap narrows only when the sample size increased to about 2,500 and 5,000 samples.
Results
95
CLEAN
30% ASYMMETRIC NOISE
93
30
ELR
88
NCE AGCE
AIRPLANE BIRD CAT DOG
ALL
ANN
86
500
1000
5000
Number of samples
Figure t-SNE projection of a CIFAR-10 subset and its noisy (30% asymmetric) counterpart.
Figure Average accuracy (1) and 95% confidence interval on stratified subsets of Animal-1ON.
9

---

Published in Transactions on Machine Learning Research (03/2025)
4.5 Generalization to medical data
To demonstrate the generalization capabilities and elfectiveness of WANN, we provide another pool of experiments on two challenging medical datasets, which are semantically far from the pre-training data of the employod foundation model. Specifically, we used two datascts from the McdMNIST dataset collection (Yang et al., 2023), namely BreastMNIST and DermaMNIST, each with resolution 224x224. BreastMNIST is a binary dataset with only 546 training samples. Given binary nature, we limit our noise injection symmetric noise at 20%, 30%, and 40% noise ratios. DermaMNIST, on the other hand, is multi-class dataset with 7 classes and 7,007 training samples. Thus, apply the same noise settings previously (ef- Table 4.. For asymmetric noise, we shift each class by one in a circular manner. We use the loss-hypepprranctors defined for CIFAR-10 and report average accuracy over five runs.
Results Table presents the results for the medical datasets. Notably, on BreastMNIST, WANN and ANN exhibit the highest accuracy with both clean and noisy training sets. Furthermore, WANN outperforms ANN with increased symmetric noise. While it might be expected that k-NN-based approaches would outperform linear methods a binary problem with small dataset, the weighting scheme in WANN demonstrates clear benefits on DermaMNIST as well, with a significant improvement on instance dependent label noise. In fact, although GCE outperforms ANN with symmetric noise, it does not surpass WANN, which consistently achieves higher accuracy and lower standard deviation, demonstrating greater robust
MNISS
DermaMNIST
Pattern NR
Clean
Symmetric 20%
Symmetric 30%
Symmetric 40%
Clean
Symmetric 60%
Asymmetric 30%
Instance 40%
CE
28
a6.67=a.56
55.38-6.42
52.18-4.00
78. -8.82
60.17+2.02
69.09+1.6a
5-1.57=4.89
FL
67.82+
59.49+4.4
57.05-6.1s 6.11
53.33 6.44
78. (8.80
60.20+3.s8
6..36+1.s1
52.65 62.69-6.10
MAE
66.67 a 26
56.03
53.72 3o
73.38_0.91
63.1613.45
66.55
58. 58.7622.00
GCE
70.13:a.22
61.03
55.64=7.27
51.03=4.82
74. 14.16
67.16:2.a8
68.5241.16
59.53=3.19
SCE
66. 66.5442..4
55.90 50:000=...0
54.10=8.46
52.95=6.15
73.29=1.67
62.34+3.119
70.07 1.55
60.55=2.67
ELR
60.00
67.44_26.16
55.64_6.17
12.18_1.95
70.58_1.41
63.87 12269
67.67 19.55
19.57:110.19
NCE-MAF
68.721
57.5628.13
53.59
53.21 =
73. 10_
61 .1.89 13.23
70.00 1.44
60.61 1.00
NCE-CCE
67. 2.13331
57.8228.77
53.722 35
53.21 20.03
73. 12 20.54
61.67 12.99
07.0411.50
59 199.4820.24
NFL-RCE
36.28813.00
57.562s.08 63
53.72=
53.08_5.84
73. 12 10.54
61.67 22.99
17.04
59 19..4821.24
NCE-AGCE
26.28
58. 72.
53.59=s.50
53.72-6.14
73.85
61.58
66.63
59.48
NCE-AUL
66.67:22.40
7.666
53.72=0.02
32.69-5.54
72.90
61 61.63:12.9
67 1..2
60. 10..87
NCE-AEL
66.6713.64
59.74_7.89
55.77
53.46_5.99
73.52_0.93
61.11 12.75
70.0011.78
59.36=2.75
ANL-FL
69.10++.58
60.26+6.67
55.13=7.96
54.10
61.82-0.25
57 57.46+2.22
1..3
36681+6.55
ANL-CE-ER
67.18+2..5
60.00+7.55
55.64-8.10
53.97-6.50
42.51 1.88
36.20
36,10+1.sa
30.63-2.44
ANN
76.92
7.
69.62.1.70
63.08-2.28
73.77
65.87 20.49
70.1210.78
82.81=1.72
WANN
75.00
74.2
72.69
65.64=1.50
73.07
69.96 20.23
71.42#0.14
67.49=1.20
Table 4: Accuracy (t) on BreastMNIST and DermaMNIST. We bold the top two methods and underline the significantly best one if their difference is statistically significant (paired t-test, < 0.05).
4.6 Long-tailed noisy data
While k-NN inherently robust, the selection of an appropriate value of i is critical and not trivial, especially considering the unknown noise rate and potential class imbalances. In this experiment, we assess the per- formance of ANN and WANN, as compared to two fixed A-NN approaches. As discussed in the method section, both adaptive methods used (Kmin kmax) (11,51). They compared with their respective fixed K-NN counterparts, namely 11-NN and 51-NN. For benchmarking on long-tailed problems, we use CIFAR-10IT and CIFAR-10OLT, following prior studies (Cao el al., 2019: Du et al.) 2023), with imbalance ratios of 1% and 10%, respectively. denotes the ratio between the least and most frequent class, with an exponentially decaying imbalance across all other classes. In CIFAR-IOLT, the majority class has 5000 samples, while the minority class consists of 50 samples. Consequently, in CIFAR-100LT, we have 500 and 50 samples, respectively.
10

---

0.3 Published in Transactions on Machine Learning Research (03/2025)
As in previous experiments, noise is artificially injected. To emphasize the generalizability across different noise patterns and severities, we inject symmetric noise ranging from 20% to 60%, asymmetric noise from 20% to 40%, and instance-dependent noise from 20% to 40%, reporting the accuracy over five runs.
Results The results presented Figure 5 align with our expectations. Under limited noise conditions, lower value of & (k = 11) generally achieves higher performance, while a higher value (A 51) shows greater robustness against more severe noise levels. In contrast, adaptíve methods consistently approach the best performances across various noise rates and patterns. WANN's weighting scheme exhibits higher gains compared to ANN, especially under heavy asymmetric and instance-dependent noise. Additionally, considering all experiments and respective seed runs (due to the dataset variations from the stochastic noise injection), WANN emerges as the significantly best method (Wilcoxon signed-rank test, p < 0.05).
(a) Noisy CIFAR-IOLT (1%)
(b) Noisy CIFAR-1OOLT (10%)
SYMMETRIC
ASYMMETRIC
INSTANCE
SYMMETRIC
ASYMMETRIC
INSTANCE
100
90
Accuracy
90-
Accuracy
85-
80
JUM JM6
AM
0.3 Rate
70
0.2
0.4
0.6
0.2
0.4 0.2
0.3
0.4
Noise Rate
Figure 5: Accuracy of fixed and adaptive k-NN based approaches on CIFAR-IOLT and CIFAR-10OLT, with 1% and 10% imbalance ratios. The imbalance ratio denotes the ratio between the least and the most frequent class, with an exponentially decaying imbalance between all other classes. Notably, WANN is close to the best performance across any noise pattern and severity. Furthermore, WANN is the significantly best method across all experiments and datasets (Wilcoxon signed-rank test, p < 0.05).
4.7 Dimensionality reduction
To showcase the effectiveness of the proposed Filtered LDA (FLDA) approach, we evaluate WANN's classifi- cation accuracy on the linearly projected test set, utilizing the complete projected training set as WANN's support set. Four different projections are employed for this analysis. First, as a baseline, we report classification performances without dimensionality reduction, labeled as "None". Subsequently, we utilize Principal Component Analysis (PCA) with a fixed number of components, specifically 200, explaining an average of 77.19% of the lotal variance across each dataset. Additionally, we compare this with plain Linear Discriminant Analysis (LDA) fed with the entire noisy dataset. Finally, we present the effectiveness of our filtering approach before applying LDA, named FLDA. To this extent, we recall that LDA projects the dataset along C' 1 axes, where C' is the number of classes. Consistently with our previous experimental settings, the accuracy is reported on datasets with 60% symmetric noise, 30% asymmetric noise, and, 40% instance-dependent label noise. The experiments cover three different datasets: CIFAR-10, CIFAR-100, and MNIST (Lecun et al.) 1998). Following Ye et al. (2023), for asymmetric noise on CIFAR-100 we group the 100 classes into 20 super-classes and flip each class within its super-class in a circular fashion, while on MNIST we use the mapping + 1, + 7, 5 ++ 3.
The results are visually reported in Figure On the clean dataset, LDA and FLDA exhibit among the highest gains, substantially reducing the dimensionality and thereby reducing the overall spar- sity. Here, LDA outperforms its filtered counterpart because of the absence of noisy labels. In fact, these performance gains vanish in the presence of label noise, confirming that FLDA is significantly better than all other dimensionality-redueee
Results
methods (Wilcoxon signed-rank test, p<0.05) under varying noise conditions. The reduction in dimensionality not only enhances classification performance but also contributes to faster computational speed and potential storage efficiency.
11

---

Published in Transactions on Machine Learning Research (03/2025)
CITAR1O
CITAR100
MNIST
100
99
92
96
Accurccy
98
94
90
92
97-
88
90-
SYM (0.6) (0.3 INST 10.4)
CLEAN
SYM (0.6)
(0.3) (0.4)
(0.6) ASYM(0.3) (0.4)
None
PCA
B
LDA
FILAA
Figure 6: Classification accuracy of WANN under three dimensionality reduction strategies, evaluated across three datasets and three different noise scttings each. None reflects the performance using the original image embeddings without dimensionality reduction. PCA and LDA show the classification performances on the lincarly projected feature spaces, utilizing the whole noisy training set. Finally, FLDA involves the proposed filtering approach. FLDA is the significantly best dimensionality reduction method across all experiments and datasets (Wilcoxon signed-rank test, p < 0.05).
Notably, using DINOv2 VIT-L as feature extractor, we have an initial embedding size of 1024. Thus, we significantly improved the classification performance with 100x (CIFAR-10, MNIST) and 10x (CIFAR-100) smaller embeddings.
4.8 Explainability benefits
While deep learning methods are often viewed as black boxes with limited explainability, some traditional ma- chine learning approaches inherently allow for a certain level of explainability. Notably, k-NN-based methods provide explainability through the represenlative nature of the neighborhood. Exploring the neighborhood for each prediction provides deeper understanding of the factors contributing to correct erroneous clas- sifications. As illustrated in Figure 2 we present four test samples along with their top 3 closest training examples from STL-10 (Coates et al. 2011), & subset of ImageNet (Deng et al, 2009). In Figure 2a we highlight two test images with clearly incorrect reference labels, supported by clean training samples. Tn contrast, Figure 2igure2bJshove shows two ambiguous test labels, including multiple known objects within a single image, such a bird with a ship in the background or a dog playing with a cat. The pitfalls of ImageNet labels were already rated by Beyer et al. (2020). However, this provides a simple yet effective way to manually inspect potentially wrong labels within our datasets. Indeed, the explainability of based approaches
becomes particularly valuable in the context of active label correction (Rebbapragada et al. 2012 Bernhardt
et al. 2022 where human intervention is employed to re-annotate noisy samples.
5 Discussion and conclusions
The effectiveness of k-NN relies on a representative feature representation, posing challenges for domains not represented during pre-training. However, this limitation can be mitigated by using alter- native feature extractors. This is now possible thanks to the growing availability of such large open-source models. Moreover, while the proposed FLDA significantly enhances classification performance, it requires sufficient sample size to generate reliable projection vectors. Finally, to mitigate the computational demands of k-NN's exhaustive search, established Appreaimale-NN methods (Malkov & Yashunin 2020 Guo et al., 2020 Johnson et al. 2021) provide an effective solution, achieving comparable classification performance with substantially lower computational complexity. Indcod, we could expect a negligible influence on classification performance, as distance metrics inherently only approximate the nearest sample.
Limitations
12

---

Published in Transactions on Machine Learning Research (03/2025)
Side benefits and generalization The efficiency of our classification method, further optimized with dimensionality reduction, enables its deployment with limited resources. Moreover, the reliable feature space of current feature extractors becomes particularly suitable for k-NN-based approaches, possibly generalizing to different domains. For instance, we observed outstanding classification performance and robustness in the medical domain, where interpretability, data scarcity, and data imbalance are still open challenges. Furthermore, the compliance with data regulations, such as the right be forgotten (Mantelero) (2013). is facilitated by the proposed database approach. The flexibility to remove or add data points without extensive re-training or fine-tuning, aligns with privacy guidelines, providing cost-effective solution. Lastly, our method can be readily applied to any domain, tabular or otherwise, as long as can obtaín a meaningful semantic space. While our current work focuses on image embeddings derived from large vision foundation models, the same concept exlends to language, audio, or multimodal embedklnggs, assuming they capture the semantic relationships necessary for defining reliable neighborhoods.
Conclusion We introduce WANN, Weighted Adaptive Nearest Neighbor approach, based on a reliabilily score quantifying the correctness of the training labels. Our experiments demonstrate its higher robustness with respect to label noise, particularly in scenarios with limited labeled data, surpassing traditional models relying on robust loss functions. Additionally, the incorporation of dimensionality reduction techniques enhances its performance under noisy labels, facilitated by the proposed weighted scoring.
Acknowledgments
This study was funded through the Hightech Agenda Bayern (HTA) of the Free State of Bavaria, Germany.
References
Dara Bahri, Heinrich Jiang, and Maya Gupta. Deep k-nn for noisy labels. In International Conference Machine Learning, pp. 540 550. PMLR, 2020.
Peter N. Belhumeur, João P. Hespanha, and David J. Kriegman. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. In Bernard Buxton and Roberto Cipolla (ods.), Computer Vision ECCV '96, pp. 43 58, Berlin, Heidelberg, 1996. Springer Berlin Heidelberg.
Mélanie Bernhardt, Daniel C. Castro, Ryutaro Tanno, Anton Schwaighofer, Kerem C. Tezcan, Miguel Mon- teiro, Shruthi Banmur, Matthew P. Lungren, Aditya Nori, Ben Glocker, Javier Alvarez-Valle, and Ozan Oktay. Active label elcaning for improved dataset quality under resource constraints. Nat Commun, 13 (1):1161, March 2022. ISSN 2041-1723.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Avital Oliver, Nicolas Papernot, and Colin Raffel. Mix- Match: a holistic approach to semi-superuised learning. 2019.
Lucas Beyer, Olivier Hénaff, Alexander Kolesnikov, Xiaohua Zhai, and Aäron van den Oord. Are we done with imagenet? arXiv preprint arXiv:2006.07159, 2020.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Fric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christo- pher Berner, Sam McCandlish, Alec Radford, 1lya Sutskever, and Dario Amodei. Language models are fow-shot learnors. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1807-1901, Lane Red Hook, NY, 2020. Curran Associates, Inc
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with diserldisttibuton-aaaaaa margin loss. In Advonces in Neural Information Processing Systems, volume 32, 2019.
13

---

Published in Transactions on Machine Learning Research (03/2025)
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1600 PMLR, 2020.
Adam Coatos, Andrew Ng, and Honglak Lec. An analysis of singlc-layer networks in unsupervised feature learning. In Proceedings of the fourleenth international conference on artificial intelligence and statistics, pp. JMLR Workshop and Conference Proceedings, 2011.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248- 255, 2009.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirec- tional transformers for language understanding. In Proceedings of the 2019 Conference of the North Amer- ican Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171 4171-4186, 4186, Stroudsburg, PA, USA, 2019. Association for Computational Linguistics.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
Fei Du, Peng Yang, Qi Jia, Fengtao Nan, Xiaoting Chen, and Yun Yang. Global and local mixture consistency cumulative learning for long-tailed visual recognitions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15814-15823, 2023.
Fahimeh Fooladgar, Minh Nguyen Nhat To, Parvin Mousavi, and Purang Abolmaesuni. Manifold dividemix: A semi-supervised contrastive learning framework for severe label noise. In Proceedings of the LEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 4012-4021, 2024.
Shlomo. Geva and Joaquin Sitte. Adaptive nearest neighbor pattern classification. IEEE Transactions on Newral Networks, pp. 118-222, 1991.
Aritra Ghosh, Himanshu Kumar, and P Shanti Sastry. Robust loss functions under label noise for deep neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar. Acceler- ating large-scale inference with anisotropic vector quantization. In International Conference on Machine Learning, pp. 3887-3896. PMLR, 2020.
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor W. Tsang, and Masashi Sugiyama. Co-teaching: robust training of deep neural networks with extremely noisy labels. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, pp. 8536 8546, Red Hook, NY, USA, 2018. Curran Associates Inc.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision Iearners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000 16009, 2022.
Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In International conference on machine learning, pp. 2313-2313. PMLR, 2018.
14

---

Published in Transactions on Machine Learning Research (03/2025)
Jeff Johnson, Matthijs Douze, and Hervé Jégon. Billion-scale similarity search with gpus. JEEE Transactions on Big Dala, 7(3):535-544 2021.
Tan T Jolliffe. Principal component analysis for special types of data. 2002.
Shuyu Kong, You Li, Jia Wang, Amin Rezaci, and Hai Zhou. Knn-enhanced deep learning against noisy labels. CoRR, abs/2002.042244, 2020.
Nikolaos Kouiroukidis and Georgios Evangelidis. The effects of dimensionality curse in high dimensional knn search. In 2011 15th Panhellenic Conference on Informalics, pp- 41 45, 2011.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-basee based learning applied to document recognition. Proceedings of the IEEE, 86(11))2278-25) 2324,, 1998.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoi- fung Poon, and Jianfeng Gao. LLaVA-med: Training large Einguac--and-visin assistant for biomedicine in one day. In Thirly- Thirly-sevenlh Conference on Newral Information Processing Systems Dalasels and Bench- marks Track, 2023.
Jumnan Li, Richard Socher, and Steven C.H. Hoi. Dividemix: Learning with noisy labels as semi-supervised learning. In International Conference on Learning Representations, 2020.
Sheng Liu, Jonathan Niles Niles-Weed, Narges Razavian, and Carlos Ternandez-Granda. Early-learning regular- ization prevents memorization of noisy labels. In Proceedings of the 34th International Conference Newral Information Processing Systems, NIPS '20 Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546..
Xingjun Ma, Hanxun Huang, Yisen Wang, Simone Romano, Sarah Erfani, and James Bailey. Normalized loss functions for deep learning with noisy labels. In International conference on machine learning, pp. 6543-6553, PMLR, 2020.
Yu A. Malkov and D. A. Yashunin. Efficient and robust approximate nearest neighbor search using hicrar- chical navigable small world graphs. IEEE Trans. Pallern Anal. Mach. Intell., 42(4):824 836, apr 2020. ISSN 1162-8888.
A Alessandro Mantelero. The eu proposal for a general data protection regulation and the roots of the 'right to be forgotten'. Computer Law & Security Review, 29(3):229 235, 2013. ISSN 0267 3649.
Kengo Nakata, Youyang Ng, Daisuke Miyashita, Asuka Maki, Yu-Chieh Lin, and Jun Deguchi. Revisiting a kun-based image classification system with high-capacity storage. In European Conference on Compuler Vision, pp. 457-474. Springer, 2022.
Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojeiech Galuba, Mike Rabbat, Mido Assran, Nicolas Ballas, Gabriel Syn- naeve, Ishan Misra, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. Dinov2: Learning robust visual features without supervision, 2023.
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust. to label noise: A loss correction approach. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 1944-1952, 1952, 2017.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamcla Mishkinn Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
15

---

Published in Transactions on Machine Learning Research (03/2025)
Umaa Rebbapragada, Carla E. Brodley, Damien In 2012 IEFE 12th International Conference on Dala Mining, pp. 1080-1085, 2012.
and Mark A. Friedl. Active label correction.
Henry Reeve and Ata Kabán. Fast rates for knn classifier robust to unknown asymmetric label noise. In International Conference on Machine Learning, pp. 5401-54099. PMTR, 2019.
Clayton Scott, Gilles Blanchard, and Gregory Handy. Classification with asymmetric label noise: Consistency and maximal denoising. In Shai Shalev-Shwarde and Ingo Steinwart (eds.), Proceedings of the 26th Annual Conference on Learning Theory, volume 30 of Proceedings of Machine Learning Research, pp. 489-511, 12 14 Jun 2013.
Hwanjun Song, Minscok Kim, and Jac-Gil Lee. SELFIE: Refurbishing unclean samples for robust deep learning. In ICML, 2019.
Hwanjun Song, Minscok Kim, Dongmin Park, Yooju Shin, and Jac-Gil Lec. Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Newral Networks and Learning Systems, 2022.
Shiliang Sun and Rongqing Huang. An adaptive k-ncarest neighbor algorithm. In 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery, volume 1, pp. 91-94, 2010.
Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, and Kiyoharu Aizawa. Joint optimization framework for learning with noisy labels. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5552-55550 5560, 2018.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp. 1195-1204 Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Tao Tu, Shekoofeh Azizi, Danny Driess, Mike Schackermann, Mohamed Amin, Pi-Chuan Chang, Andrew Carroll, Charles Lau, Ryutaro Tanno, Ira Ktena, Anil Palepu, Basil Mustafa, Aakanksha Chowdhery, Yun Liu, Simon Kornblith, David Fleet, Philip Mansfield, Sushant Prakash, Renee Wong, Sunny Vir- mani, Christopher Semturs, S. Sara Mahdavi, Bradley Green, Ewa Dominowska, Blaise Aguera y Arcas, Joclle Barral, Dale Webster, Greg S. Corrado, Yossi Matias, Karan Singhal, Pete Florence, Alan Karthike- salingam, and Vivek Natarajan. Towards generalist biomedical ai. NEJM AI, (ow):13o0300 2024.
Jigang Wang, Predrag Neskovic, and Leon N. Cooper. Improving nearest neighbor rule with simple adaptive distance measure. In Licheng Jiao, Lipo Wang, Xin-bo Gao, Jing Liu, and Feng Wu (eds.), Advances in Natural Computation, pp. 43-46, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg, ISBN 978-3-540-45902-6.
Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo, Jinfeng Yi, and James Bailey. Symmetric cross entropy for robust learning with noisy labels. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 322-330, 2019.
Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. Combating noisy labels by agreement: A joint training method with co-regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition (CVPR), June 2020.
Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: A study using real-world human annotations. In International Conference on Learning Representations, 2022.
Dietrich Wettschereck and Thomas Dietterich. Locally adaptive nearest neighbor algorithms. In Advances in Neural Information Processing Systems, volume 6, Burlington, Massachusotts, 1993. Morgan-Kaufman
Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. Part-dependent label noise: towards instanco-dependent label noise. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, 2020.
16

---

Published in Transactions on Machine Learning Research (03/2025)
Ruixuan Xiao, Yiwen Dong, Haobo Wang, Lei Feng, Runze Wu, Gang Chen, and Junbo Zhao. Promix: com- bating label noise via maximizing clean sample utility. In Proceedings of the Thirly-Secomee International Joint Conference on Artificial Intelligence, IJCAI '23, 2023. ISBN 978-1-956792-03-4
Hanwen Xu, Naoto Usuyama; Jaspreet Bagga, Sheng Zhang, Rajesh Rao, Tristan Naumann, Cliff Wong, Zelalem Gero, Javier González, Yu Gu, Yanbo Xu, Mu Wei, Wenhui Wang, Shuming Ma, Furu Wei, Jianwei Yang, Chunyuan Li, Jianfeng Gao, Jaylen Rosemon, Tucker Bower, Soohee Lee, Roshanthi Weerasinghe, Bill J. Wright, Ari Robicsek, Brian Piening, Carlo Bifulco, Sheng Wang, and Hoifung Poon. A whole-slide foundation model for digital pathology from real-world data. Nature, 2024.
Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
Xichen Ye, Xiaogiang Li, Songmin Dai, Tong Liu, Yan Sun, and Weiqin Tong. Active negative loss functions for learning with noisy labels. In Thirty-sevent seventh Conference Neural Information Processing Systems, 2023.
Xichen Ye, Yifan Wu, Yiwen Xu, Xiaoqiang Li, Weizhong Zhang, and Yifan Chen. Active negative loss: A robust framework for learning with noisy labels. areie preprint arXiv:2412.0237 12.02373, 2024.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(33)107-115, 2021.
Jingfeng Zhang, Bo Song, Haohan Wang, Bo Han, Tongliang Liu, Lei Liu, and Masashi Sugiyama. Badlabel: A robust perspective on evaluating and enhancing label-noise learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(6):4398 4409, 2024. doi: 10.1109 SLU0//TPAMM..22233335525
Zhilu Zhang and Mert R. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In Proceedings of the 32nd International Conference Neural Information Processing Systems, NIPS'18, pp. 8792-8802, Red Hook, NY, USA, 2018. Curran Associates Inc.
Guoqing Zheng, Ahmed Hassan Awadallah, and Susan Dumais. Meta label correction for noisy label learning. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11053 11061, 2021.
Xiong Zhou, Xianming Liu, Junjun Jiang, Xin Gao, and Xiangyang Ji. Asymmetric loss functions for learning with noisy labels. International conference on machine learning, pp. 12846 12856. PMLR, 2021.
Yukun Zhou, Mark A Chia, Siegfried K Wagner, Murat S Ayhan, Dominic J Williamson, Robbert R Struyven, Timing Liu, Moucheng Xu, Mateo G Lozano, Peter Woodward-Court, et al. A foundation model for gencralizable discasc detection from retinal images. Nature, 622(7981):156-163 2023.
Yilun Zhu, Jianxin Zhang, Aditya Gangrade, and Clayton Scott. Label noise: Ignorance is bliss In The Tiigtt- Annual Conference on Neural Information Processing Systems, 2024.
Zhaowei Zhu, Zihao Dong, and Yang Liu. Detocting corrupted labels without training model to prodict. In International conference on machine learning, pp. 27412 27427. PMLR, 2022.
17

---

Published in Transactions on Machine Learning Research (03/2025)
A Generalizability across backbones
This section illustrates two additional experiments using a ViT-L backbone pretrained on ImageNet in a supervised fashion. We compared method with the same subset of loss functions utilized in Section 1.4 (CE, ELR, NCE-AGCE, ANL-FL, ANL-CE-ER). On CIFAR-N (c.f. Table A.1 while WANN shows slightly lower performance than the best-performing robust loss function on this particular backbone, our FLDA approach offers a substantial performance boost, achieving results comparable to or even surpassing those of the robust loss functions.
Additionally, we conducted an experiment analogous Section 4.3 using CIFAR-10 with subsets of 50 and 100 samples per class (c.f. Table A.2p. Notably, when using only 50 samples per class, the adaptive mot substantially outperform the robust loss functions. These findings demonstrate that the advantages of our approach are not merely due to the DINOv2 backbone's inherent strengths but rather underscore the general applicability and robustness of our method.
CIPAR-1ON
CIFAR-10ON
Notay split NR (#)
Clean
Ager,
R1 17.29%
R2
R3
Worst
Clean
Nolay
18.12%
17.64%
40,215
CR
86. 06.5610-00
95. 05-199100008
94 B6.00
05.011:0:10
85-03320
91.35
84 01.00.30333
74-482.0
ELR
00.37-+0.02
05.04400.07 0.007
96. T4++- 14
96. 96.85+0.00
95.71#-0 771--0.00 86
94.26+0038
64.034-0.00
T0.50-0.22 0.20
ACE-AGCE
00.50=00.00 0.00
00.10::0.07
05. 165.00=-..0 0.00
06.98=0.18 10
06. 05.08 0-8==0000 00
08.70=0.25
84.02 04.002-0.11
77.92=0.00
ANL-FL
96.08220.19
95. 25.90020.10
95. 80:200022 23
95-788 05-78220.10
05.455220- 27
94..111..122
78 78-41:200333
T1 T1.24:20.32
AEL-CE-ER
96. 06.18:20.14
95. 78..2220.28
vs. 52:52.00555
95. 6.320.92
95.72220.0 16
94.79-20..15
79. 20.33.20.30
T3.07 0.222.022
WANN
WANN
Table A.I: Accuracy on CIFAR-N datasets utilizing a VIT Large pre-trained on ImageNel. The results are averaged over five seed runs.
CIFAR-10 R- (#55)
CIFAR-10 (#110)
Symmesrio
Symmesrie
Asymmetrio
30%
CR
85-2310 677
91.4629.13
88.074: .45
51 76:71.90
95 851:0 0006
#3.87=1.00 100
90 00.99..1.1 77
86 2.01
RLR
95 85-001:0-3 -4
82 #2-10=11.0 1.65
93 031601...0 63
88 BR 88.2-.15
96 451:0 .19
84.02=1 .80
94.1412.14
9n. 91.1911.881
NCE-AGCE
96.49:20.97
85.94-2 2.62
08. 1.15
00.92.$3.11 5.19
96 06.93+-0.33 0.3
94.71=0.50
05.60
96.40::0.40
ANL-FL
04.39:21.00
78.00=2
33.022+2..7
80. 22.90:22.15
96 20:-0---000 70
80.54=1
02. 02.10:20.00 65
88 89.70-22.24
ANL-CE-ER
94.821:0.89
78.18=3.90
90.05.20.3
13.13-2:4.20
80.54=0.07
92.97:::000
89.20:1
ANN WANN
07:07100114
04.00:10.02
94.48-10.82
08-76111-79
88 70.1
00.74:00.37
02.3744.19
94.47:1.600
97.081:0.15
05.47:10.65
00.92:10.74
09.12
88.602 $6.09911.31
07.10-10.36
922007:11.26
94.56-f-1.55
Table A.2: Accuracy on limited noisy data (50 and 100 samples per class) utilizing a ViT Large pre-trained on ImageNet. The results are averaged over five seed runs.
B Robustness of reliability score
B.1 Signal for linear classifiers
This experiment aims to evaluate the utility and robustness of our reliability score. The hypothesis is the following: if our reliability score is a vobust indicator of label noise, linear layer should be able to utilize this information just as effectively as k-NN. To test this hypothesis, we procomputed the reliability score for each training sample and concatenated it with the respective feature embedding. For test samples, we utilized an average reliability score from their training neighborhood, based on the intuition that the spatial distribution of these scores is both informative and reliable.
As illustrated in Table B.1 for CIFAR-10 and CIFAR- 100, this straightforward approach of augmenting the feature emboddings with the reliability score slightly improves performance while further reducing the variability (lower standard deviation across five seed runs). These results hold promise that the reliability score provides valuable signal that linear classifier can utilize effectively the k-NN approach.
18

---

Published in Transactions on Machine Learning Research (03/2025)
CIFAR-10
CIPAR-100
Symmetric
Axymmatric 30%
Instance 40%6
Clean
Symmetric 80%
Asymmetric 30%
Instance 40%
60%
CB
40 +0000
98.00
96.47 +-0
95. 864-0 72
T8.82=0.31 0.332
60.17++2.00
69.09+1.65
64.57-44.80
Q-CE
26.6 +-0.2
08.10-0.13
96.65 +-0.0-... 8.
96. 08.94++0..3
80.22-1.1 1.71
63.48 +2.01
70. 10.118
80.06+3.22
Table B.1: Accuracy on CIFAR-10 and CIFAR-100 under three label noise settings. We compare the standard performance using the raw embeddings (CE) and the ones concatenated with our reliability score (7-CE). The results are averaged over five seed runs.
B.2 Distribution of the reliability scores
Utilizing a toy dataset consisting of 1, 000 samples and 40% symmetric noise, we illustrate in Figure B.1 (top-left) the real class distribution, (top-right) the noisy class distribution, (bottom-left) the reliability scores assigned to each point, and (bottom-right) the distribution of reliability scores for clean vs. noisy samples. From the bottom-right plot, is possible to observe that clean samples (blue) tend to have higher reliability scores, whereas noisy samples (red) cluster lower or intermediate values. Points near decision boundaries or mislabeled neighbors also receive moderate or low reliability scores. This observation supports our intuition that WANN utilizes local consistency in the embedding space to effectively distinguish clean from noisy samples.
Real class distributian
Nolsy class distrribution (Sym 40%))
Feature
Feature
Feature
Data points colored by reliability score
Reliability scores for Clean vs. Neisy samples
5.09
0.08
0.07
150
Densit
Rdt Eabiiity
50-
0.03
0.02
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
Reature
Reliabilyy Score
Figure B.l: Reliability score distribution a toy dataset with 40% symmetric noise.
19


Reviews:
[{"review_id": "ov3J2ojEP6", "reviewer": "Reviewer_JgxJ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Stregnths: The algorithm is easy to implement and shows suprior performance compared to existing algorithms.\n  \nWeakness: It is unclear why the proposed algorithm is superior even thouhg the idea is simple. Below, I list up my comments and questions.\n\n1. What happens when there is no $k$ satisfying the Eq (1)? I think that this situation could occur frequently for\nnoisy-labeled data in the minor class. I\n\n2. In the last paragraph of page 4, $k$ would be $\\mathcal{H}(x,y,\\mathcal{X})$ (or $eta$). $k$ is an argument used to define $\\mathcal{H}(x,y,\\mathcal{X})$.\n \n3. I understand the idea of using the smallest number of nearest neighbors for correct classification (i.e. k) for measuring the reliability\nbut I am not sure that 1/k is an optimal choice. For me, 1/k seems to be too naive.\n\n4. I understand the intuition of (4) , but it seems too naive. Are there any theoretical justifications for (4)?\nI worry that (4) depends only on the nearest datum but not on multiple nearest neighbors.\n\n5. As far as I understand, 'Filtered dimensionality reduction' is needed since nearest neighbor methods are susceptible to the curse of dimensionality. I wonder how sensitive the nearest neighbor method to the dimension of data when obtaining the reliability scores. I think\nthat nearest neighbor is still sensitive and thus the reliability scores may not be informative when the dimension of data is high.\n\n6. In numerical studies, it seems that WANN outperforms other methods using specially designed loss functions for noisy-labels.\nHowever, I could not understand intuitively why WANN is better? More detailed explanations would be necessary.\n\n7. Is it possible to combine the reliability score and specially designed loss function? For example, we could learn a prediction model\nby minimizing the weighted empirical average of the specially designed loss functions where the weights are proportional to the proposed reliability scores. This idea would be promising since the nearest neighbor method is difficult to be deployed.\n\n8. To justify the proposed method (e.g. explain why WANN is better than existing loss-based methods), I recommend\nthe authors to investigate how WANN works for a simple toy model such as two Gaussian mixture model.\n\n9. I think that WANN can be applied to tabular data without modification. Is there any reason to focus only on classification of embedding vectors?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors proposed a learning algorithm which is robust to noisy labels. The novel part is to propose\nthe reliability score based on the nearest neighbor method on the embedding space.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please, add intuitive explanations why the proposed method is better than existing algorithms.", "broader_impact_concerns": "No concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jnce27Co9e", "reviewer": "Reviewer_y5QB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The approach is straightforward and intuitive.\n- Extensive experiments across diverse settings empirically demonstrate its effectiveness.\n- The method is computationally efficient.\n- The paper is well-written and easy to follow.\n\nWeaknesses (sorted descending by importance):\n- **Comparison to related work:** While the paper builds on well-established nearest-neighbor approaches for noisy label learning, its positioning within the broader literature remains unclear. The authors note that previous k-NN methods have struggled with issues like a lack of explainability, data efficiency, and generalizability. However, the paper does not explicitly detail how their approach overcomes these specific challenges. *Moreover, the work of Zhu et al. (2022), which applied a k-NN strategy on the CLIP model, is highlighted as closely related but not sufficiently differentiated from the current approach.* The paper should clearly outline what makes their method distinct—whether it is through a novel mechanism for computing reliability scores, or improvements in explainability and data efficiency. Without this clear differentiation, it is difficult to evaluate the novelty and significance of the contribution relative to existing methods.\n- **The choice of the foundation model:** The authors employ the DINOv2 model as the backbone, justifying their choice by evaluating the performance of their methods on different backbones. However, since the backbone is a crucial element of the strategy, the choice of DINOv2 appears suboptimal, and additional backbones should be considered. This concern arises from the fact that the DINOv2 paper explicitly optimizes their SSL model for k-NN performance, which might boost the observed performance of the approach explicitly on this backbone. Therefore, to ensure that the results are not merely a byproduct of the backbone's inherent strengths, it is necessary to evaluate the method using at least one other backbone. Additionally, incorporating a backbone for other modalities, such as text, would enhance the study’s generality. Otherwise, the title should be adjusted to reflect the focus on image-based analysis.\n- **End-to-end finetuning:** Since the approach centers on a k-NN method, the model cannot be fine-tuned end-to-end, limiting training to frozen embeddings. This is problematic because, in practical applications, a perfect foundation model is rarely available, and there is often a need to optimize earlier layers. Furthermore, it would be insightful to explore the effectiveness of incorporating the reliability scores with a linear layer. If the reliability score is truly a robust indicator of noisy labels, a linear layer should be able to leverage this information just as effectively as the k-NN approach. Therefore, an experiment combining the reliability scores with a linear classifier would be a valuable addition.\n- **Normalized feature embeddings:** The paper states that embeddings are normalized. However, because the SSL backbone inherently scales features by their importance, normalization might strip away critical information. Did you try experiments using unnormalized embeddings? Preserving the original scaling could potentially lead to better performance.\n- **Hyperparameter tuning:** What is the reason reason that no loss-specific hyperparameters where optimized in 4.4? \n- **Oversized figure 1:** The size of Figure 1 (the GA) appears excessive. I recommend reducing its scale and relocating it to the end of the introduction, as this adjustment would enhance the overall presentation of the paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors introduce a classification algorithm and a dimensionality reduction method designed to mitigate the effects of noisy labels. Their classification approach leverages self-supervised features from the DINOv2 foundation model and employs an adaptive nearest neighbor strategy to compute a reliability score using neighborhood information. Each new instance is then classified based on the reliability score of its closest training neighbor. The dimensionality reduction method is essentially LDA, but they filter out instances with low reliability scores to mitigate the effects of noisy labels.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See Weaknesses above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "EMvsHeXvBr", "reviewer": "Reviewer_XUqU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper explores a novel and interesting direction by addressing the label noise problem at the inference stage rather than during training.\n- The proposed method, WANN, is technically sound and well-motivated.\n- The evaluation is comprehensive, including existing baseline methods, which clearly demonstrate the improvement and validate the effectiveness of the proposed approach.\n\nWeakness:\n- The proposed method lacks novelty, as it can be seen as a weighted KNN approach for self-supervised learning (SSL) models. Existing SSL methods already utilize KNN for inference in classification tasks, and WANN is essentially a weighted extension, which feels somewhat intuitive rather than a fundamentally new contribution. The paper’s presentation does not provide much insight beyond the method itself, making it less engaging for readers.\n- The proposed Filtered LDA also lacks significant novelty, as it is essentially a straightforward combination of filtering with LDA. The contribution feels incremental rather than introducing a new approach.\n- The effectiveness of WANN is heavily dependent on representation quality, particularly when using DINOv2. The empirical improvements rely on this specific feature extractor, raising concerns about the method’s generalizability. The reviewer suggests a deeper analysis of why DINOv2 is necessary and why other methods listed in Table 1 are insufficient. This analysis would strengthen the contribution and provide deeper insights.\n- It would be good to include SimCLR, BYOL, and VICReg in the analysis.\n- All experiments are conducted on small-scale datasets such as CIFAR and MNIST, which limits the ability to assess the generalization of the proposed method. The reviewer suggests including evaluations on larger datasets, such as ImageNet.\n- For real-world analysis, it would be more beneficial to explore some real-world medical datasets, such as the Cholec80 Tool classification [1]. SSL models are also trained for such datasets [2].\n\n[1] Twinanda, Andru P., et al. \"Endonet: a deep architecture for recognition tasks on laparoscopic videos.\" IEEE transactions on medical imaging 36.1 (2016): 86-97.\\\n[2] Zhou, Yuning, et al. \"DDA: Dimensionality Driven Augmentation Search for Contrastive Learning in Laparoscopic Surgery.\" Medical Imaging with Deep Learning 2024.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates the noisy label problem from a novel perspective. Unlike existing works that primarily focus on training robust models, this study explores robust inference using self-supervised learning models. It introduces a method called Weighted Adaptive Nearest Neighbor (WANN), which assigns a reliability score to each training sample, where potentially noisy labels receive lower scores. During inference, predictions are derived from multiple local neighbourhoods, weighted by these reliability scores. Experimental results demonstrate that the proposed method is highly effective, particularly when using DINOv2 as the feature extractor.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the Strengths And Weaknesses section.", "broader_impact_concerns": "There are no ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a method for addressing the label noise problem. Interestingly, they address this problem at the inference stage (rather than during training), by leveraging pretrained foundational models and a weighted nearest-neighbour method.

Reviewers appreciated the experimental results, and the fact that the method is relatively simple to apply, meaning that it can be readily used in practice.

And whilst reviewers appreciated the overall idea of addressing the label noise problem at inferene-time which is quite novel, they also felt that the idea was quite simple and lacked stronger theoretical motivations.

On the balance, the decision is to accept the paper as its strengths outweigh its weaknesses. However, the paper is not being recommended for the ICLR Journal-to-Conference track, or any certifications.
Decision: Accept (poster)