Sample Selection via Contrastive Fragmentation for Noisy Label Regression
Authors: Chris Dongjoo Kim
Sangwoo Moon
Jihwan Moon
Dongyeon Woo
Gunhee Kim
Venue: NeurIPS 2024 poster (2024)
Keywords: Noisy Labels
Regression
Primary area: other
TL;DR: To address the problem of regression with noisy labels, we propose the Contrastive Fragmentation framework to select clean samples by Mixture of Neighboring Fragments and curate four benchmark datasets along with a novel metric, Error Residual Ratio.

Abstract: As with many other problems, real-world regression is plagued by the presence of noisy labels, an inevitable issue that demands our attention. 
Fortunately, much real-world data often exhibits an intrinsic property of continuously ordered correlations between labels and features, where data points with similar labels are also represented with closely related features.
In response, we propose a novel approach named ConFrag, where we collectively model the regression data by transforming them into disjoint yet contrasting fragmentation pairs. 
This enables the training of more distinctive representations, enhancing the ability to select clean samples.
Our ConFrag framework leverages a mixture of neighboring fragments to discern noisy labels through neighborhood agreement among expert feature extractors.
We extensively perform experiments on four newly curated benchmark datasets of diverse domains, including age prediction, price prediction, and music production year estimation.
We also introduce a metric called Error Residual Ratio (ERR) to better account for varying degrees of label noise.
Our approach consistently outperforms fourteen state-of-the-art baselines, being robust against symmetric and random Gaussian label noise.

Sample Selection via Contrastive Fragmentation for Noisy Label Regression
Chris Dongjoo Kim 122 Sangwoo Moon'; Jihwan Moon
Dongyeon Woo Gunhee Kim
Seoul National University, LLG Al Research
[edjkim, sangwoo. moon, jihwan moon, dongyeon. woo ) @iviviiion....... anu ac kr
gurhee@snu.ac
Abstract
As with many other problems, real-world regression is plagued by the presence noisy labels, an inevitable issue that demands our attention. Fortunately, much real-world data often exhibits intrinsic property of continuously ordered correla- tions between labels and features, where data points with similar labels are also represented with closely related features. In response, named ConFrag, where we collectively model the regression data by transforming them into disjoint yet contrasting fragmentation pairs. This enables the training of more distinctive representations, enhancing the ability to select clean samples. Our ConFrag framework leverages mixture of neighboring fragments discern noisy labels through neighborhood agreement among expert feature extractors. We extensively perform experiments six newly curated benchmark datasets of diverse domains, including age prediction, price prediction, and music production year estimation. We also introduce metric called Error Residual Ratio (ERR) to better account for varying degrees of label noise. Our approach consistently outperforms fourteen saute-of-theate baselines, being robust against symmetric and random Gaussian label noise?
propose novel approach
Introduction
Regression is an important task in many disciplines such as finance (Zhang et al. 2017b Wu et al. 2020b). medicine de Vente al. physics [Sia et al. 2020 Doi et al.
Tanaka et al., 2022], economics [Zhang al. [002]] geography [Liu et al. 2023 and more. However, real- world regression labels are prone being corrupted with noise, making it inevitable problem to overcome in practical applications. In previous research, noisy label regression has been studied much in age estimation with noise incurred from Web data crawling [Rothe et al., 2018 Yiming et al.] 2021 Beyond that, the issues of continuous label errors have also been reported in the tasks of object detection [Su et al. 2012 Ma et al. 2022] and pose estimation [Geng and Xia 2014 as well as measurements in hardware systems [Zhou et al. 2012 Zang et al. 2019
2021
2022
The vast amount of noisy label learning research has focused more classification than regression.
Some notable approaches include regularization [Wang et al. 2019 Zhang and Sabuncu, 2018 data
re-weighting [Ren al.) 20188 Shen and 200g training procedures [Jang al. 2018)] transition matrices Yao et al. 2020 Xia et al.) 2020). contrastive learning [Zhang et al. 202 la Li
al. 2022b). refurbishing [Song al. [2199 and sample selection [Lee al.) 2018| Ostyakov et
2018
Particularly, sample selection can be further divided into exploring the memorability of neural
'These authors contributed equally this work.
"The code is available at
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

networks Arpit et al. 2017 Zhang et al. 2017a| and delineating samples via the loss magnitude [Wei] al. 2020].
To the best of our knowledge, there have been three works that address the noisy label problem for
regression with deep learning. Castells al. (2020) propose a weighted loss correction method based
on the small loss assumption. Garg and Manwani (2020) propose an ordinal regression-basee
loss
correction via noise transition matrix estimation. However, they assume that accurate noise rates are
known in prior [Patrini et al. 2017]. which are hard to attain in practice. Yao et al. [2022] extend Mix Zhang et al. 2018 for regression to interpolate the proximal samples in the label space to improve generalization and robustness. Thanks to its regularizing effect, it can aid the noisy label
issue.
In this work, we explore the regression problem with noisy labels, surpassing the scope of previous studies both empirically and methodologically. For evaluation, make three notable contributions. Firstly, recognizing the absence of a standardized benchmark dataset for this task, we take the initiative to curate six balanced real-world datasets. These datasets span diverse domains, encompassing age estimation [Niu et al. 2016 Yiming et al. [202]], music production year estimation [Bertin- Mahieux et al. 201TJ, and clothing price prediction Kimura et al. 202T]. Secondly, we conduct a comprehensive empirical benchmark exercise, evaluating the performance of fourteen baselines, which are carefully selected from various branches of noisy label research extendable to regression tasks, Lastly, we introduce performance measure called Error Residual Ratio (ERR), which accounts for the unique property of regression, where labels exhibit varying degrees of noise severity.
Methodologically, we introduce the ConFrag (Contrastive Fragmentation) framework as a novel approach to address label noise in regression. It is rooted in one fundamental characteristic of regression: the continuous and ordered correlation between the label and feature space. In other words, data points similar in the feature space are likely to have similar labels. The framework begins by partitioning the dataset into smaller segments, referred to as fragments, and pairs the most distant fragments in the label space to form what we call contrastive fragment pairs. Training an expert network on these contrastive fragment pairs aids in generalization due to the distinctive feature matching and conversion of closed-set noise into open-set noise, which is less detrimental for learning. Next, the framework incorporates neighboring relationships by aggregating and reordering the learned features to detect clean samples. This is accomplished through the design of Mixture (Jacobs et al., 1991] of neighboring fragments. Furthermore, we enhance our approach with neighborhood jittering regularization, which strengthens the selection process by improving the data coverage of each expert. This, in turn, leads to improved agreements among neighboring fragments and serves an effective tool for mitigating overfitting. Finally, the contributions of this work can be summarized as follows.
1. We propose novel method named ConFrag (Contrastive Fragmentation) for noisy labeled regression. It leverages the inherent orderly relationship within the label and feature space by employing contrastive fragment pairing and constructs mixture model based on neigh- borhood agreement. This is further enhanced by our neighborhood jittering regularization.
2. We perform one of the most thorough empirical investigations into noisy labeled regression up to date. We assemble six well-balande benchmarks using datasets of AFAD [Niu (Yiming et al.) 2021], ) IMDB-W -IKI [Rothe et al.) 2018), UTK- Face Zhifei et al. 2017]. SHIFTISM [imura et al. 2021] and MSD [Bertin Mahieux et al. 2011], on which we evaluate fourteen baselines. We design a metric termed ERR (Error Residual Ratio), which accounts for the degree of noise severity within the labels, offering a more comprehensive assessment. Our experiments affirm the superiority of ConFrag over
et al. 2016), IMDB-CIes
state-of- the-the-ttt art noisy label learning baselines.
2 ConFrag: Contrastive Fragmentation
In the noisy label regression problem, we are presented with a dataset denoted as D = (X, Y): in each sample (x. y), Rd is input, and y E IR is the observed label, which can be possibly noisy. We use yd to denote the groundtruth label. The objective of ConFrag is to sample a clean subset of the data S C D. By training on S, we aim enhance the performance of the regression model.
An overview of our ConFrag framework is shown in Fig. a). The framework has the following steps. We divide the dataset into what we refer to contrastive fragment pairs ($ 2.1. which collectively

---

Closed
Opet:
MRAE (16)
MRAE (%)
40-
opez-aat out closed-oot
contrustive all-frags
30-
20
20
Itagment e [1-6]
Jc 1.4
S [2.0]
f [3.9
600
1200
100 epoch
noiey samples
(a)
(b)
(c)
Figure (a) An example of t-SNE illustration of contrastive fragment pairing. The data with label noise are grouped into six fragments (f E [1-6]) and formed into three contrastive pairs If [1, 4], [2, 5], [3, 6)). Contrastive fragment pairing transforms some of closed-set noise (whose ground truth is within the target label set) into open-set noise (whose ground truth is not within the label set). For example, in the [1,4] figure, label noise whose ground truth fragment is either I or 4 is closed-set noise, and the others are open-set noise. The t-SNE illustration shows that learned features of open-set noises tend to reside outside the feature clusters of the clean samples. (b) The open-set noise is less harmful with much lower errors (MRAE) in the downstream regression. (c) The contrastive pairing ([1, 4), [2. 5]. [3, 6)) is more effective than using all-fragments together ([1-6), resulting in much lower MRAE scores. All experiments are based IMDB- B-DB-CCeannB with more details in Appendix G.4 G.5
enhance the training of the feature extractors ($ 2.2 We then select clean samples S from dataset D based on neighborhood agreements, utilizing a tragment-basee mixture model ($ 2.3). A regression model is trained the clean samples S. We also propose neighborhood jittering regularizer for further improved training ($ 2.4). ConFrag is noise rate-agnostic unlike prior methods as it operates without knowing pre-defined noise rate.
2.1 Contrastive Fragment Pairing
In order to sample a clean data subset S C D, we need to learn a robust feature that distinguish clean samples from noisy ones, As one theoreti- cal result in [Zhang al. 2023), the cross-entropy
D.
2)
Dg
D= (X. Y
loss used in classification is better for learning high- entropy feature representation than the mean squared loss in regression (see Appendix [.I] for details). Based on this, we start by discretizing the label space into F continuous fragments, transforming the origi- nal regression problem into the multi-class classifica- weights tion one. This transformation harnesses an inherent property of regression: data points with similar la- bels are also represented with closely related features, 3-4. The best perfect matching with the largest minimal edge as acknowledged in prior studies [Gong et al. 2002 weights
1. Dataset partitioning by discretizing the continuous labels
edge weight, Oo
20
Graphical
fragments distance as edge
20
Yang et al. 2022b Yao et al. 2022].
Figure 2: The contrastive fragment pairing algorithm.
However, instead of training single feature extractor on the multi-class classification with F' classes, we construct F/2 maximally contrasting fragment pairs and train a smaller expert feature extractor for each pair. The procedure of contrastive fragment pairing detailed below with an illustration in Fig. 2
1. Divide the range of continuous labels Y into F even number of equal-length fragments. This allows divide the dataset into F' disjoint subsets: D (D, Dr). where each D; contains the data samples whose values are in the i-th fragment label range.
Construct a complete graph g {D, E), where each vertex fragment D;, and each edge weight eij is the distance in the label space between the closest samples of the fragments (D,, Dz).
Compute all possible perfect matchings Monfaredd and Mallik 2016 Gibbons 1985), where every vertex of graph is incident to exactly one edge in the graph.
3

---

4. Find the perfect matching with the largest minimal edge weight: P= arg max min ( where each g is a perfect matching (graph), and v(g) is the set of edge weights in g. Finally, P = {D;; D,),.............. (Da, } constitutes the maximally contrasting pairs of fragments.
Motivation behind contrastive fragment pairing. Formulating the multi-class classification problem into F/2 binary classification problems via contrastive fragment pairing has the following advantages. Firstly, since the distance between fragments in each contrastive fragment pair is large, the feature extractor trained on each contrastive pair can generalize better [Shawe- Taylor and Cristianini 1998 Grønlund et al. 2019 2020]. Fig. Tc) shows the generalization abilities of the expert feature extractors trained on contrastive fragment pairs compared to the single feature extractor trained on all fragments. When using single feature extractor all fragments (all-frags), the samples selected by the feature extractor tend to become more noisy as the feature extractor overfits, causing the regressor to perform worse over time. On the other hand, when using multiple feature extractors trained on contrastive pairs, the performance of the regression model consistently improves, indicating that the learned features are more robust and the selected samples are cleaner. The large distance between fragments also explains why contrastive fragment pairing is superior to other fragment pairings, as shown in 4.3] The analysis of the prediction depth (Baldock et al., 202T] in Appendix suppooss the claim, as it shows that the binary classification on contrastive fragment pairs results in lower prediction depth, leading to better generalization.
Secondly, the contrastive fragment pairing transforms some of closed-ser label noise (whose ground truth is within the label set) into open-sel label noise (whose ground truth is not within the label set), as shown in Fig. aa). Previous works (Wei et al., 2021 Wan et al. 2024] observe that the open-set noise is less harmful than the closed-set noise and may even benefit generalization and robustness against inherent noisy labels. Indeed, in our experiments, we found similar observations where injecting open-set label noise less harmful than closed-set one, as shown in Fig. [b).
The t-SNE visualization in Fig. a also supports this observation. Let f and f8l be fragments that the observed label y and the groundtruth label y respectively belong to. Prior to contrastive fragment pairing, all of the noisy labeled data (f * f8t are closed-set noise as their ground truth fragment ids within the label set (f [1-6)) and their features are located in the feature spaces of incorrect classes within the group. After contrastive fragment pairing, much of these noisy labeled data is transformed to open-set noise (8 & [1,4] while [1, in case of fragment pair [1, 4)), and their learned features tend to reside outside the feature clusters of the clean samples, thus mitigating the adverse effects of the noise.
2.2 Training Feature Extractors for Contrastive Pairs
Once we obtain the contrastive fragment pairs P, we train F/2 number of expert feature extractors on binary classification p(y|a; [z; fz.5) with its respective contrastive pair (D,, D2) E '. where 0t,, denotes the parameter of an expert. That is, it is trained to predict whether a data x is in D; or D,. Later, the feature extractors play crucial role in determining whether a sample (z, y) is clean.
2.3 Mixture of Neighboring Fragments
With the learned expert feature extractors, the next step is to perform sample selection. Given sample (ar y), let f be a fragment close to and ft be its contrasting pair. Intuitively, we consider sample clean if the expert trained on (D,, ; Dy+) strongly predicts that belongs to a fragment f- However, since the expert feature extractor is a binary classifier only trained using a contrasting pair of fragments, we utilize all experts' opinions to obtain a more robust prediction. Specifically, we deem a sample as clean if the experts exhibit a consensus response (Neighborhood Agreement) for fragments close to y (Fragment Prior).
Based on this intuition, we formulate Mixture of Experts (MoE) [Jacobs et al. 1991 model, where the sampling probability of a datapoint (I, 3) is defined as
F
p Di... F; θ) Σ (x
(1)
where & denotes parameters of all feature extractors, py is the fragment prior (mixture weight), and a is the neighborhood agreement (a binary vote of whether x belongs to the fragment f). Based on

---

(a) Overall Framework
(b) Fragmentation! Train Feature Extractor
(c) Selection by Mixture of Neighboring Fragments
Dittase
Naghtte Agreement Ci
p(s)
Irigmencation (32.) step 1) and Contrastive Parrin ($2.1 2-41
Do
Poar
Di
py(y)
02.3
Train Feature Exunctor ($2.2) with Neighborhood Fitturing 2.4)
(d) Neighborhood Jittering
coverage butler
Pa
epoch
75
Dy
epoch
Selection
0LA
opach
Neighboring l'ragments ($2.3)
Uy uniginal coveniee
original
By
Figure 3: Contrastive Fragmentation framework. (a) The overall sequential process of our framework. (b) Shows the fragmentation of the continuous label space obtain contrasting frugment pairs ($ 2. and train feature extractors on them. (c) Sample Selection by Mixture of Neighboring Fragments obtains the selection probability in both prediction and representation perspectives ($ 2.3
(d) Illustration of Neighborhood Jittering ($ 2.4).
the intuition above, (y) large when the fragment close to y, and a E {00, 1) i likely to belong to the fragment f.
Fragment Prior. For sample (ar, y). we compute the prior (y()) of fragment f, using softmax weighting of each fragment f with respect to its relative distance to y:
exp(gr(v))
pf(v)
ΣS exp(gr
where gy()) range(Y)/((u -yl), Y,),, range(Y) = max(Y) - min(Y) is the label range, and Yy is the mean label value of fragment Since range(Y) is constant for a given dataset, () rapidly decreases when the mean value of fragment / is far from in the continuous label space. From the MoE perspective, the fragment prior can be regarded soft gating that depends on y.
Neighborhood Agreement. Given a sample (z, y) and a fragment f. we need to determine whether æ belongs to f. The simplest approach is to use the expert trained using (D,, Dy ) to classify whether belongs to f or f+, where I+t is the contrasting fragment of f. Based on the classification output Mx:;; f,,+) E [J,t+ , we define self-agreement as:
self af
[1(2;; ((s;;
where [A] is the Iverson bracket outputting 1 if A is true, and 0 otherwise. Since training with noisy labels often results in suboptimal calibration [Bae et al.) 2022 Zong et al. 2024), we use discrete classification output for self rather than continuous probabilistic one.
Since the expert f is only trained to discriminate between ff and its contrasting fragment f+, it is better to utilize other experts to obtain more robust prediction. For example, consider contrastive fragment pairs {(1, 4), (2, 5), (3, 6)) as in Fig. 2 If is more likely to belong to fragment 2 than 5, then it should be more likely to belong to 1 than 4 and 3 than Thus, we consider agreement of neighboring fragments Sr. (left) and J'.. (right) to obtain neighborhood agreement (z; Di...F, Q):
solf (2 Di.... +) = of
t of where agb = [aa of
ngb
self V o
self"
(4)
Intuitively, ay is 1 if the fragment f is more likely for than f+ (a = 1) and either f's left or right fragment is more likely for x than its respective contrasting fragment (a =).
In practice, we implement two variants of the agreements Eq.33 using the feature extractor's binary classifier and a K K-nearest neighbor classifier on the learned reature space. These two classifiers respectively consider predictive and representutional aspects of the expert feature extractor and effectively work as ensemble, as shown in Appendix G.7 As result, we compute two versions of sample probability in (T. and use the union of the sampled clean dataset S for training of the regression model. Algorithm in Appendix summarizes the overall procedure.
5

---

Avg. extractor (%)
Sel. (%)
80
80
80-
60-
MRAB (%)
60
60
110 jit tost (cleea)
ERR (%))
80
40-
40
no jit train (nolay)
wi jic test (clesn)
w/ jit train (noisy)
80
40
no jit
jit
20
30
60
90
120
30
60
120
30
60
90
120
epoch
epoch
epoch
(a) Feature extractor acc.
(b) Selection rate / ERR
(c) Regression performance
Figure 4: Jittering analysis. (a) When trained without jittering, feature extractors easily overfit the
noisy training data (yellow-shaded region), while reveering-egularizeed feature extractors robustly learn from the noisy training data. (b) Overfitted feature extractors (yellow-shades region) on noisy samples increase their likelihood, leading to higher selection rate and ERR. It exhibits nearly twice higher ERR (a lower value is better). (c) Most importantly, jittering regularization improves performance in regression.
2.4 Neighborhood Jittering
potential limitation of mixture models is that the individual expert feature extractor may not fully benefit from the full dataset as they model their own disjoint subsets (Dukler et al.) 2023). Our neighborhood jittering mitigates this limitation as a robust regularizer that expands the effective coverage of each contrastive fragment pair during learning. The process is visualized in Fig. 3 d).
We bound the ratio of the jittering buffer range within [0, z where F is the fragment number. For every epoch, we shift the label coverage of each fragment by randomly sampling the value in this range. Jittering leads to a partially overlapping mixture model (Heller and Ghahramani 2007b Hinton| 2002] as some data belong to multiple, neighboring fragments and thus the effective coverage per each expert is expanded. Such regularization inhibits feature extractors from overfitting to potentially noisy samples and promotes learning of more robust features, even those that can be generalizable to overlapping parts of neighboring fragments.
Fig. 4 a) shows that with jittering, the feature extractor exhibits higher accuracy on the clean test data due to its regularization effect. In the sample selection stage (Fig. 4b)), the feature extractor trained without jittering easily overfits the noise, resulting in over-selection and higher ERR ($ 4.2). In contrast, the jittered feature extractor achieves relatively low selection rate with halved ERR, indicating that the noisier samples are filtered out. Better sample selection due to jittering subsequently leads to significantly better performance in regression (Fig. 4c)). In Appendix G.9. we compare neighborhood jittering to other regularizations, demonstrating its efficacy.
3 Related Works
We review prior works on learning with noisy labels and defer a comprehensive survey to Appendix We organize them into those utilizing prediction, representation, and combination of the two.
Prediction-based Methods. This approach has been the focus of much existing research and covers a wide array of topics: (i) the small loss selection by exploring the pattern of memorization in
neural networks (Han al., 2018 Arazo et al.) ] (ii) relying the consistency of predictions to select or refurbish the samples [Liu et al. 2220 Huang et al. 2020]. (iii) estimating the noise distribution (Patrini al., 2017 Hendrycks et al., 2018), (iv) introducing auxiliary parameters or
labels [Pleiss et al. 2020 Hu et al. 2020). (v) using unlabeled data with semi-supervised learn-
ing [i et al. 2020a| Bai al. [021 Karim et al., 2022]. and (vi) designing noise-robust loss function Meeon et al. 2020 Wang et al. 2019].
Representation-baase Methods. This approach has seen recent surge in interest, including (i) clustering based selection Mirzasoleiman et al. 2020 Wu et al.) 2020aa (ii) feature eigendecomposi
tion filtering [Kim et al.)
2021] (iii) using neighbor information to sample and refurbish with clean
validation [Li et al. 2022a Gao et al. 2016), and (iv) generative models of features for sampling [ee
et al. 2019
Combination. Some works have also studied the combination of representation and prediction spaces. Wang et al. 2022 formulate a penalized regression between the network features and the labels for

---

selection, and Ma et al. [0018] use intrinsic dimensionality and consistent predictions to refurbish. Other important approaches include (i) regularization via MixUp (Zhang et al., 2018] along with its regression version Yao et al. 2022], (ii) model-based methods that discourage large parameter
shifts [Hu et al., 2020), and (iii) importance discrimination of parameter updates (Xia et al., 2021].
The majority of previous works have studied noisy labels for classification. Hence, a large portion of these works may not be directly applicable regression tasks due to the restricted usage of class-wise information. In $ $ we empirically compare our method with some of these works that are expandable to the regression task with some or minor technical adaptation.
4 Experiments
We compare ConFrag with fourteen strong baselines adapted for noisy label regression. Due to the scarcity of benchmark datasets, update existing datasets for the study of noisy labels.
4.1 Settings
Curation of Benchmark Datasets. We create six benchmark datasets for noisy labeled regression to
encompass a sufficient quantity of balanced data, span multiple domains, and present a meaningful
level of complexity. (i) Age Prediction from an image is well-studied regression problem [Li
eet al. 2019, Shiin et al. 2022 Lim et al. 2020]. To address this domain, we acquire four datasets of AFAD [Niu |016 IMDB-Clean Yiming al. [021]. IMDB-WIKS [Rothe et al., 2018).
and UTKFaae (Zhifei et al.) [217]]. Notably, IMDB WIKI contains real-world label noise stemming [002]]. We use a ResNet-50 backbone for
from the automatic web crawling process (Yiming et al.
all datasets. (ii) Commodity Price Prediction is a vital real-world task [Wen-Huang et al. 202T]
We opt for the SHIFT15M dataset [Kimura et al. 2021 due to the diversity and scale of this domain. This dataset is provided as the penultimate feature of the ImageNet pre-trained VGG-16 model. Consequently, we use three-layer MLP architecture for all experiments (Papadopoulos et al.) 2022 Kimura et al. [222]] (iii) Music Production Year Estimation uses the tabular MSD dataset (Bertin-Mahieus et al., 2011]. This dataset identified as one of the most intricate and challenging datasets, based on the test R2 score [Grinsztajn et al., [022]. We adopt a tabular ResNet proposed by Gorishniy al. [20T]. The suffix "-B" is appended to the dataset name (e.g., AFAD-B) to indicate that it is curated version of the original dataset. To focus on the noisy label problem, take measures to balance the datasets as elaborated in Appendix F.I
Experimental Design. For all datasets except for IMDB-WIKI-B which contains real-world label noise, we inject symmetric and Gaussian noise into the labels, as done in prior literature (Yao et al., 2022 Yi and Wu [2019 (Wei et al., (020). These types of noise can simulate a low-cost nouman-freeo controlled setting. Symmetric noise mimics randomness such as Web crawling or annotator errors. and Gaussian noise is often used for modeling the regression label noise. While Yao et al. 2022 inject a fixed 30% standard deviated Gaussian noise for every label, we make more realistic by randomizing the standard deviation up to 30% or 50% of the domain's 's range. For our ConFrag experiments, we fix the fragment number (F) as four. See Appendix F33 for further training details.
There are many existing methods of noisy labeled learning for classification. We assess fourteen baselines from the three branches that are naturally adaptable to regression with minor or no updates. (i) Small loss selection: CNLCU-S,H [Xia et al. [022]] Sigua (Han et al.) 2020 SPR (Wang al. [2022]. BMM [Arazo et al. 2019), DY-S [Arazo et al.) 2019]. SuperLoss [Castells
Baselines.
et al. 2020]. (ii) Regularization: C-mixup Yao et al. 2022 |. RDI Hu et al. 2020). CDR (Xia et al.
2021 D2I. [M al. 2018. (iii) Refurbishment: AUX Hu al. 2020), Selfie [Song et al. 2019
Co-Selfie [Song et al. 2019 Appendix F.2 comprehensively details these baselines.
4.2 Evaluation Metrics
We mainly report the Mean Relative Absolute Error (MRAE) following prior works. The MRAE computed as (e/p) 1, where e is the model's Mean Absolute Error (MAE) performance under varying conditions (noise type, severity) and is the noise-free model's MAE. We express MRAES in percentage for better comprehensibility The traditional MAE values are also reported Ap- pendix G.12 In addition, we report the Selection rate (a.k.a prevalence), which is a metric often seen
7

---

Table 1: Comparison of Mean Relative Absolute Error (%) over the noise-free trained model on the AFAD-B, negative value indicates it performs even better than the noise-free model. The results are the mean of three random seed experiments. The best and the second best methods are respectively marked in red and blue. CNLCU-S/H, Co-Selfie, and Co-ConFrag
BMDB-WIKLLL SHIFTI5M-B, and MSD-B datasets. Lower is better.
dual networks to teach each other as done in Han et al. 2018 SPR [Wang et al. 2022) fails to run for SHIFT15M due to excessive memory usage.
AFAD-B
MMDB-Clea-B
HIDB-WIKI-B
symmetric
Gaussian
symmetric
Gaussian
real noise
noise rate
20
40
60
80
30
50
20
40
60
80
30
50
Vanilla
9.37
20.27
30.65
43.09
28.77 39.03
16.18
32.05 53.13 76.35 26.89
50.28
0
CNLCU-S
10.98
20.44
32.44
41.99 30.60
40.66
51.40
66.62
82.83 85.65 83.39 82.10
21.54
CNLCU-H
4.63
16.32 36.01 44.71
35.68 43.64
6.84
31,16 63.08 82.65 46.53 65.24
-2.93
Sigua
5.96
21.09 43.33 49.71 42.52 46.19
9.82
46.17 77.59 85.62 60.97 77.42
1.96
SPR
9.74 18.85 30.43 43.25 28.50 39.69
14.47
32.44 54.88 79.37 25.67 51.05
-0.93
BMM
5.60 15.00 39.15 46.41 30.96
44.00
8.85
21.54
55.57 80.40 24.33 57.21
17.88
DY-S
6.87 15.56 32.24 45.72 24.40
43.41
10.42 21.90
49.94 78.16 24.70 44.56
-3.41
C-Mixup
2.74 14.80 27.17 41.95 24.28 36.91
8.82
27.74 50.87 76.79 21.92 47.04
-5.26
RDI
10.64 21.80 39.32 47.07 37.33 44.41
16.35 29.33 55.91 79.92 25.69 51.35
1.06
CDR
10.26 18.71 32.27 43.38 29.74 39.21 17.47 32.19 54.75 75.45 28.46 51.73
-0.39
D2L
9.43 20.75 31.25 44.50 28.86 40.10 16.94 33.85 55.54 76.28 29.30 52.44
-0.66
AUX
6.15 19.01 31.16 42.83 28.28 39.05 12.58 28.82 52.33 76.75 23.27 49.42
-3.67
Selfie
16.91
25.02 44.18 47.78 46.02 50.73 27.43 53.74 79.38 84.00 60.68 78.03
14.00
Co-Selfie
14.61
22.95
39.79 47.72 41.05 53.00 23.52 50.07 67.42 84.25 52.44 74.73
-0.44
Superloss
7.36
18.24
29.78
44.26 27.59 42.96 23.38
45.41
67.11 80.85 53.88 63.33
-3.58
ConFraag
2.74
8.16
15.91
34.42
17.49
27.31
5.08
12.64
27.26
61.24
15.70
33.36
-3.06
Co-ConFrag
0.54
7.25
16.65
33.93
17.43 28.26
1.50
9.45
28.44
61.36
14.87 35.88
-8.86
SHIFTISM-B
MSD-B
symmetric
Gaussian
symmetric
Gaussian
noise rate
20
40
60
80
30
50
20
40
60
80
30
50
Vanilla
9.11 17.96
27.02
36.34
6.54
15.16
8.23
18.43
31.67
45.85
6.96
15.74
CNLCU-S
12.98
19.42
24.31
34.47
15.33
20.90
0.13
6.04
21.52
46.01
4.75
12.51
CNLCU-H
6.26
12.84
20.04
36.03
8.88
15.65
0.27
4.98
10.32
29.83
5.11
9.22
Sigua
6.94
14.09
26.08
37.03
10.32
17.44
1.29
7.19
17.35
50.87
6.80
12.38
SPR
7.07
18.19
33.39
45.61
5.01
15.36
BMM
6.96
12.42
18.64
26.79
7.58
13.13
3.32
10.30
23.40 43.56
5.29
11.85
DY-S
7.11
11.94
18.85
29.04
6.90
13.50
3.39
8.06 18.65 35.24
4.77
9.83
C-Mixup
9.47
16.15
24.08 34.17
5.88
14.51
3.75
13.13 26.73 40.90
2.96
10.97
RDI
9.91
17.92
26.63 36.29
7.08
15.18
21.04
30.09
38.78 49.49
19.19
27.88
CDR
9.52
17.78
26.97 35.97
7.14
15.17
7.83
17.86
32.83 45.91
6.73
16.92
D2L
9.25
18.03 26.55 36.23
6.34
15.60
7.13
19.96
32.47 46.64
5.51
15.54
AUX
7.74
16.95
26.61 36.47
4.92
14.40
6.12
18.18
31.09 45.70
5.21
15.45
Selfie
4.84
10.22
22.28 38.15
5.51
11.58
1.43
8.40
20.24
45.87
14.37
24.13
Co-Selfie
11.53
16.43
32.08 39.32
13.45
22.33
-0.38
4.41
8.32
35.47
6.78
13.15
Superloss
5.44
12.26
23.23
35.24
5.60
13.28
-0.15
10.68
23.15
45.55
4.35
16.36
ConFrag
2.46
6.18
10.68
19.04
3.66
8.09
0.57
4.94
11.22
23.41
2.39
6.49
Co-ConFrag
0.85
5.52
10.80
18.83
3.03
8.70
-0.65
2.98
8.66
20.53
1.73
6.00
in noisy labeled classification to quantify the coverage the total dataset, |S| where S and DD are the selected and total set, respectively.
Error Residual Ratio. To better assess selection and refurbishment approaches, we introduce a new metric termed Error Residual Ratio (ERR). Unlike classification, noisy labels in regression can show the diverse severity of the noise present in each label y (i.e., various degrees of deviation from the ground truth ). This cannot be addressed when using conventional metries, which are primarily
8

---

ConFragg
CNLCUAH
BMM
DY-S
AUX
Selãie
(%)
(%)
(%)
(%6
ERR (%6
(96)
100
100
100
120-
100
100
50-
60-
80-
100
25
50
100
100
100
25
50
75
100
100
spoch
epoch
spoch
spoch
spoch
spuch
(a) Symmetric 40%
(b) Gaussian 30
Figure 5: Selection/ERR/MRREE comparison between ConFrag and strong baselines of CNLCU-H, We exclude the performance during the warm-up.
BMM, DY-S, AUX and Selfie on
designed for classification and tend to treat all instances of noise equally severe. Our proposed ERR considers the varying severity of noise and is defined as
1/1C| E We 1/ID
y
ERR
(5)
2. 1ya - yal
where C is set of cleaned (selected or refurbished) samples. The numerator is the average cleaned error that serves as an indicator of the precision of the cleaned data, while the denominator is the average dataset eтror that normalizes it for standardized assessment. The ERR. along with the selection rate and regression metrics (e.g., MSE, MRAE), provides a deeper insight into the model performance. Ideally, a method with a high selection rate coupled with low ERR and regression error can be deemed as closer to the upper bound.
4.3 Results and Discussion
Overall performance. Table I compares the MRAE values to the noise-free trained model between ConFrag and the baselines. We evaluate six types of noise: four symmetric and two random Gaussian noises. ConFrag and Co-ConFrag achieve the strongest performance in all experiments compared to the fourteen baselines. Notably, Co-ConFrag mixes co-teaching during the regression learning phase by assuming that S still contains 25% noise. The results on UTKFace-B dataset can be found in Appendix G.I
Selection/ERR/MRAA comparison. Fig. compares ConFrag to five selection and refurbishment baselines of CNLCU-H. BMM, DY-S, AUX, Selfie on MMDB-Clean- using the selection rate, ERR, and MRAE Ideally, model should attain a high selection rate and low ERR. It is worth noting that the relative importance of ERR and selection rate may vary depending on the dataset and the task. ConFrag achieves the lowest ERR while maintaining above-average selection rates, resulting in the best MRAE, Appendix G.10 includes comparison results for all noise types with more baselines.
Fragment pairing. Fig. 6 compares contrastive pairing to alternative pairings using MRAE as a metric. The contrastive fragment pairing demonstrates superior performance to other pairing methods. Notably, the performance is poorest when both the average and minimum distance between fragments are smallest (11, 2]. [3. 4] when F 4, 21.2].
2].2 [3, 4]. [5, 6] when F = 6). While the pairings of [1, 4]. [2. 3) and [1, (2, 5]. 43 have the same average distance between fragments as the contrastive pairings, their minimum distances between fragments are smaller, resulting in poorer performances than contrastive pairings. This result shows the effectiveness of contrastive fragment pairing for selecting clean samples. See Appendix G.4 for more details.
Fragment number. ConFrag introduces a hyperparameter F, the number of fragments. While simply set F 4 for all experiments, we conduct analysis on the effect of using different F, as shown in Fig. b). On SHIFTI5M-B dataset, the performance is relatively stable across different fragment numbers. On IMDB- MLDB-Clean-B,, a small declining trend in performance is observed as the number of fragments increases. This decrease is likely attributed to a finer division of the training data among feature extractors, ultimately leading to overfitting and reduced generalization capabilities. Appendix G.2 provides further analysis of the fragment number.
Ablation analysis on mixture of neighboring fragments. In Table we conduct an ablation analysis of the Mixture of neighboring fragments ($ 2.3) When evaluating neighborhood agreement

---

SHIFTLEM
IMPP Clean D
11.3U341
60
50-
50
23-
80-
NIKEC C4)
MEAE
25-
2w10
SINE 05
20
20
spoad
egac3
Fragment pairing unalysis
Fragment number analysis
Figure 6: Analysis with 40% symmetric noise. (a) Comparison between the proposed contrastive pair- ing and other pairings on IMDB-Clean-B, (b) Comparison between fragment numbers on SHIFT15M- and IMDB-Clean-B.
Table 2: Ablation of Mixture of Neighhoring Frag- Table 3: Parameter size comparison. regres-
ments, MRAE on the IMDB-Clean-B dataset (lower better).
sion: parameters for regression, noise: param- eters mitigate noisy labels, "others": SPR, CDR, D2L, C-Mixup, Sigua, Selfie, BMM, DY-S, Superloss.
symmetric
Gaussian
f
S
40
30
50
regression
noise
total
S S
16.97
17.53
38.18
S" S
22.22
22.77
46.36
RDI
23.9M
47.8M
47.8M
V
S"
14.18
15.84
33.07
CNLCU
47.8M
47.8M
47.8M
V
S"
14.06
16.94
39.85
"others"
23.9M
23.9M
23.9M
V
S" N
13.08
16.18
34.23
V
8
US"
12.64
15.70
33.36
ConFrag
23.9M
22.8M
46.7M
based solely on either the agreement of the current fragment (az")) or the neighboring fragment's agreement (a) the ablation reveals that relying on the current fragment's 's agreement alone (a-gtt) exhibited relatively stronger performance. Nevertheless, this approach still fell short achieving satisfactory level compared to considering both agreements, defined in Eq.
Next, as we consider sample selection based on two variants of agreements, the predictive one utilizing the feature extractor's 's binary classifier and the representational one using K K-nearest neighbors on the learned feature space (referred to as the selected sample sets S" and S respectively), we conduct an ablation study on these selected sample sets. This involves evaluating the results when determining the final selected sample set (S) either individually, the intersection, or at the union of SP and S*. Overall, in line with ConFrag, the union of sets (SP US" proves to be the most effective strategy.
Parameter size comparison. Table compares the number of parameters of ConFrag and baselines on the ResNet-based age prediction datasets. A thorough description of the ConFrag architecture is in Appendix F.3 It worth noting that each of the ConFrag's feature extractors for noise mitigation employs a much fewer number of parameters than the downstream regression task (e.g., 48% in age prediction datasets). The total number of parameters of each method varies, some share parameters for regression as well as noise mitigation while others, such as ConFrag, do not. Nevertheless, ConFrag uses fewer total parameters than CNLCU-H and RDI.
5 Conclusion
To address the problem of noisy labeled regression, we introduce the Contrastive Fragmentation framework (ConFrag). The framework partitions the label space and identifies the most contrasting pairs of fragments, thereby training a mixture of feature extractors over contrastive fragment pairs. This mixture is leveraged for clean selection based on neighborhood agreements. Extensive exper- iments on six curated datasets on three domains with different levels of symmetric and Gaussian noise demonstrate that our framework performs superior selection and ultimately leads better regression performance than fourteen state-of-the-ar models. Given its foundation in the Mixture of Experts model, the parameter size of ConFrag linearly grows with an increase in the number of fragments. We acknowledge this potential avenue for future research.
10

---

Acknowledgement
We express our gratitude the members of the Vision and Learning Lab at Scoul National University for their valuable feedback on the manuscript. In particular, we would like to acknowledge Jaekyeom Kim, Soochan Lee, Jinseo Jeong, Wonkwang Lee, and Sehun Lee. This work was supported by LG Al Research, Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2019-I1191083 SW StarLab), In- stitute of Information & communications Technology Planning & Evaluation (IIPP) grant funded by the Korea government (MSIT) (No. RS-2022-11220155 56, Fundamental research on continual meta-learning for quality enhancement of casual videos and their 3D metaverse transformation), the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 20.30033R11 (023RIA2C2005573) and Institute of Information & communications Technology Planning Evaluation (ITP) grant funded by the Korea government (MSIT) (No. RS-2021-1221343 Ar- tificial Intelligence Graduate School Program (Seoul National University)). Gunhee Kim is the corresponding author.
References
E. Arazo, D. Ortego, Albert, N. E. O'Connor, and K. McGuinness. Unsupervised label noise modeling and loss correction. In ICML, 2019.
D. Arpit, S. Jastrzebski. N. Ballas, D. Krueger, E. Bengio, M. Kanwal, T. Maharaj, A. Fischer,
A. Courville, and Y. Bengio. closer look at memorization in deep networks. In JCML, 2017.
H. Bae, S. Shin, J. Jang, K. Song, and I. Moon. From noisy prediction to true label: Noisy prediction calibration via generative model. In ICMI.,
Y. Bai, Yang. B. Han, Y. Yang. J. Li, Y. Mao, G. Niu, and T. Liu. Understanding and improving early stopping for learning with noisy labels. In NeurlPS, 2021.
R. J. N. Baldock, H. Maennel, and B. Neyshabur. Deep learning through the lens of example difficulty. In NeurlPS, 2021.
T. Baatin-Mahieux, P. W. Ellis, B. Whitman, and P. Lamere. The million song dataset. ISMIR, 2011.
M. Boudiat, J. Rony, I. M Ziko, E. Granger, M. Pedersoli. P. Piantanida. and I. B. Ayed. A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses. In ECCV, 2020.
T. Castells, P. Weinzaepfel, and J. Revaud. Superloss: A generic loss for robust curriculum learning. In NeurlPS, 2020.
C. de Vente, P. Vos, M. Hosseinzadeh. Pluim, and M. Veta. Deep learning regression for prostate cancer detection and grading in bi- parametric mri. IEEE Transactions on Biomedical Engineering, 68(2):374-33) 383, 2021.
H. Doi, Z. Takahashi, H. Yasuoka, Fukuda, and Aoyagi. Regression analysis for predicting the elasticity of liquid crystal elastomers. Scientific Reports, 12(19788), 2022.
Y. Dukler, B. Bowman, A. Achille, A. Golatkar, A. Swaminathan, and S. Soatto. Safe: Machine unlearning with shard graphs. arXiv preprint arXiv: 2304.13169, 2023.
W. Fedus, B. Zoph, and N. Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.0396) 2021.
H. Fu, M. Gong, C. Wang. K. Batmanghelich, and D. Tao. Deep ordinal regression network for monocular depth estimation. In CVPR, 2018.
J. Gao, J. Wang, S. Dai, L. J. Li, and R. Nevatia. Note-renn: Noise tolerant ensemble renn for supiviuper object detection. In ICCV, 2019.
W. Gao, B. Yang, and Zhou. the resistance of nearest neighbor to random noisy labels. arXiv preprint 1rXiv:1607.07533 2016.

---

Z. Gao, S. Cheng, R. He. Z. Xie, H. Zhao. Z. Lu, and T. Xiang. Compressing deep neural networks by matrix product operators. In Physical Review Research, 2020.
Z. Gao, P. Liu, W. X. Zhao, Z. Lu, and J. Wen. Parameter-efficient mixture-of-experts architec- ture for pre-trained language models. In Proceedings of the 29th International Conference on Computational Linguistics, 2022.
B. Garg and N. Manwani. Robust deep ordinal regression under label noise. In Asian Conference Machine Learning, 2020.
X. Geng and Y. Xia. Head pose estimation based on multivariate label distribution. In CVPR, 2014.
A. Gibbons, editor, Algorithmic Graph Theory. Cambridge University Press, London, England, 1985.
Y. Gong, Mori, and F. Tung. Ranksim: Ranking similarity regularization for deep imbalanced regression. In ICML,, 2022.
Y. Gorishniy, I. Rubachev, V. Khrulkov, and A. Babenko. Revisiting deep learning models for tabular data. In NeurlPS, 2021.
L. Grinsztajn, E. Oyallon, and G. Varoquaux. Why do tree-based models still outperform deep learning on tabular data? In NeurlPS Track Datasets and Benchmarks, 2022.
A. Grønlund, L. Kamma, K. G. Larsen, A. Mathiasen, and J. Nelson. Margin- based generalization lower bounds for boosted classifiers. In NeurlPS, 2019.
A. Grønlund, L. Kamma, and K. G. Larsen. Near-tight margin-based generalization bounds for support vector machines. In /CML, 2020.
B. Han, Q. Yao, X. Yu, G. Niu, M. Xu, W. Hu, I. Tsang, and M. Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NeurIPS, 2018.
B. Han, G. Niu, X. Yu, Q Yao, M. Xu, I. W. Tsang, and M. Sugiyama. Sigua: Forgetting may make learning with noisy labels more robust. In ICML, 2020.
J. He, J. Qiu, A. Zeng, Z. Yang, J. Zhai, and J. Tang. Fastmoe: A fast mixture-of-expert training system. arXiv preprint arXiv:2103. 130.:22,03. 2021.
K. A. Heller and 7. A. Ghahramani. Nonparametric bayesian approach to modeling overlapping clusters. In Artificial Intelligence and Statistics, 2007a.
Katherine A. Heller and Zoubin Ghahramani. A nonparametric bayesian approach to modeling overlapping clusters. In Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, volume 2, pages 187-194. PMLR, 2007b.
D. Hendrycks, M. Mazeika, D. Wilson, and K. Gimpel. Using trusted data train deep networks on labels corrupted by severe noise. In NeurlPS, 2018.
G. E. Hinton. Training products of experts by minimizing contrastive divergence. In Neural Computation, 2002.
R. Hirk, K. Homik, and L. Vana. Multivariate ordinal regression models: an analysis of corporate credit ratings. Statistical Methods & Applications, 28:507-539, 2019.
W. Hu, Z. Li, and D. Y. Simple and effective regularization methods for training on noisily labeled data with generalization guarantee. In ICLR, 2020.
Huang, C. Zhang, and H. Zhang. Self-adaptive training: beyond empirical risk minimization. In NeurlPS, 2020.
Z. Huang, J. Zhang, and H. Shan. Twin contrastive learning with noisy labels. In CVPR, 2023.
C. H. Hoi J. Li, C. Xiong. Learning from noisy data with robust representation learning. ICCV, 2021.
12

---

R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton. Adaptive mixtures of local experts. Neural Comput, 3:79-87, 1991.
L. Jiang, Z. Zhou, T. Leung, Li, and Fei-Fei. Mentornet.eemiin data-driven curriculum for very deep neural networks on corrupted labels. In ICML, 2018.
N. Karim, M. N. Rizve, N. Rahnavard, A. Mian, and M. Shah. Unicon: Combating label noise through uniform selection and contrastive learning, In CVPR, 2022.
Kim, J. Ko, Cho, Choi, and S. Yun. Fine samples for learning with noisy labels. In NewrIPS, 2021.
Y. J. Kim, A. A. Awan, A. Muzio, A. Salinas, L. Lu, A. Hendy, S. Rajbhandari, Y. He, and H. H. Awadalla. Scalable and efficient moe training for multitask multilingual models. arXiv preprint arXiv:2109. 10465, 2023.
M. Kimura, T. Nakamura, and Y. Saito. 5m: Multiobjective large-scale fashiondataset with distributional shifts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop, 2021.
D. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. In ICLR, 2015.
S. M. Kye, K. Choi, J. Yi, and B. Chang. Learning with noisy labels by efficient transition matrix estimation combat label miscorrection. In ECCV, 2022.
K. Lee, X. He, L. Zhang, and Yang. Cleannet: Transfer learning for scalable image classfier training with label noise. In CVPR, 2018.
K. Lee, S. Yun, K. Lee, H. Lee, B. Li, and J. Shin. Robust inference via generative classifiers for handling noisy labels. In ICMI., 2019.
D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, and Y. Huang. Gshard: Scaling giant models with conditional computation and automatic sharding. In ICLR, 2021.
M. Lewis, S. Bhosale, T. Dettmers, N. Goyal, and L. Zettlemoyer. Base layers: Simplifying training of large, sparse models. arXiv preprint arXiv:2103. 16716, 2021.
J. Li, R. Socher, and S. C. Hoi. Dividemix: Learning with noisy labels semi-supervised learning. In ICLR, 2020a.
J. Li, C. Xiong, R. Socher, and S. Hoi. Towards nesse-resistant object detection with noisy annotations. arXiv preprint arXiv: 2003.01285, 2020b.
J. Li, G. Li, F. Liu, and Y. Yu. Neighborhood collective estimation for noisy label identification and correction. In ECCV, 2022a.
Li, X. Xia, S. Ge, and T. Liu. Selective-supervised contrastive learning with noisy labels. In CVPR, 2022b.
S. Li, X. Xia, H. Zhang, Y. Zhan, S. Ge, and T. Liu. Estimating noise transition matrix with label correlations for noisy multi-label learning. In NeurlPS, 2022c.
W. Li, J. Lu, Feng, C. Xu, J. Zhou, and Q. Tian. Bridgenet: continuity-aware probabilistic network for age estimation. In CVPR, 2019.
K. Lim, N. H. Shin, Y. Y. Lee, and C. S. Kim. Order learning and its application age estimation. In ICLR, 2020.
C. Liu, K. Wang, H. Lu, Z. Cao, and Zhang. Robust object detection with inaccurate bounding boxes. In ECCV, 2022.
S. Liu, J. Weed, N. Razavian, and C. Fernandez-Granda Early-leaming regularization prevents memorization of noisy labels. NeurIPS, 2020.
Y. Liu, Duffy, G. Dy, and Gaunguly. Explainable deep learning for insights in niño and river flows. Nature Communications, 14(339), 2023.
13

---

I. Loshchilov and F. Hutter. Sgdr: Stochastic gradient descent with warm restarts. In ICLR, 2017.
J. Ma, Y. Ushiku, and M. Sagara. The effect of improving annotation quality on object detection datasets: A preliminary study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop. 2022.
X. Ma, Y. Wang, M. E. Houle, S. Zhou, S. M. Erfani, S. T. Xia, S. Wijewickrema, and J. Bailey. Dimensionnlity-driven learning with noisy labels. In ICML, 2018.
J. Mao, Q. Yu, Y. Yamakata, and K. Aizawa. Noisy annotation refinement for object detection. British Machine Vision Conference, 2021.
Masoudnia and R. Ebrahimpour. Mixture experts: a literature survey. In Artificial Intelligence Review, 2014.
A. K. Menon, A. S. Rawat, S. J. Reddi, and Kumar. Can gradient clipping mitigate label noise? In ICLR, 2020.
B. Mirzasoleiman, K. Cao, and J. Leskovec. Coresets for robust training of neural networks against noisy labels. In NeurlPS, 2020.
K. H. Monfared and S. Mallik. Spectral characterization of matchings in graphs. Linear Algebra and its Applciations, 196(1))234-778) 2016.
Z. Niu, M. Zhou, X. Gao, and G. Hua. Ordinal regression with multiple output enn for age estimation. In CVPR, 2016.
D. Ortego, F. Arazo, P. Albert, N. E. O'Connor, and K. McGuinness. Multi-ohjective interpolation training for robustness to label noise. In CVPR, 2021.
P. Ostyakov, E. Logacheva, R. Suvorov, V. Aliev, G. Sterkin, O. Khomenko, and S. I. Nikolenko. Label denoising with large ensembles heterogeneous neural networks. In ECCV, 2018.
S. Papadopoulos, C. Koutlis S. Papadopoulos, and I. Kompatsiaris. Multimodal auusi--atoreeression forecasting the visual popularity of new fashion products. International Journal of Multimedia Information Retrieval, 11:717-729, 2022.
G. Patrini, A. Rozza, A. Menon, R. Nock, and L. Qu Making deep neural networks robust to label noise: a loss correction approach. In CVPR, 2017.
G. Pleiss, T. Zhang, E. R. Elenberg, and K. Q. Weinberger. Identifying mislabeled data using the area under the margin ranking. NeurIPS, 2020.
S. Reed, H. Lee, D. Anguelov, C. Szegedy, D. Erhan, and A. Rabinovich. Training deep neural networks on noisy labels with bootstrapping. In ICLR workshop, 2015.
M. Ren, W. Zeng, B. Yang, and R. Urtasun. Learning to reweight examples for robust deep learning. In ICML, 2018
R. Rothe, R. Timofte, and L. V. Gool. Deep expectation of real and apparent age from a single image without facial landmarks. International Journal of Computer Vision. 126(2-4):144-155 2018.
Rasmus Rothe, Radu Timofte, and Luc Van Gool. Dex: Deep expectation of apparent age from single image. In Proceedings of the IEEE international conference on computer vision workshops, pages 10-15, 2015.
Amir M Sarfi, Zahra Karimpour, Muawiz Chaudhary, Nasir M Khalid, Mirco Ravanelli, Sudhir Mudur, and Eugene Belilovsky. Simulated annealing in early layers leads to better generalization. In CVPR, pages 20205-20211 2023.
M. Schubert, T. Riedlinger, K. Kahl, D. Kröll, S. Schoenen, S. Šegvić, and M. Rottmann. Identifying label errors in object detection datasets by loss inspection. arXiv preprint arXiv: 2303.062 06999, 2023.
14

---

D. Shah, Z. Y. Xue, and T. M. Aamodt. Label encoding for regression networks. arXiv preprint 2.Xv:22112.01927, 2022.
A. Sharkey and Sharkey. Combining diverse neural nets. In The Knowledge Engineering Review, 1997.
J. Shawe-Taylor and N. Cristianini. Robust bounds generalization from the margin distribution. 1998.
Y. Shen and S. Sanghavi. Learning with bad training data via iterative trimmed loss minimization. In ICMI., 2019.
Y. Shen, R. Ji, Z. Chen, X. Hong, Zheng. J. Liu, M. Xu, and Q. Tian. Noise-aware fully webly supervised object detection. In CVPR, 2020.
N. Shin, S. Lee, and C. Kim. Moving window regression: a novel approach to ordinal regression. In CVPR, 2022.
H. A. Sia, R. Baldrich, M. Vanrell, and D. Samaras. Light direction and color estimation from single image with deep regression, arXiv preprint 10Xi:20009008441, 2020.
H. Song, M. Kim, and J. Lee. Selfie: Refurbishing unclean samples for robust deep learning. In ICML, 2019.
Cory Stephenson, Abhinav Ganesh, Yue Hui, Hanlin Tang, Yuon Chung, et al. On the geometry of generalization and memorization deep neural networks. In ICLR, 2021.
H. Su, J. Deng, and L. Fei-Fei. Crowdsourcing annotations for visual object detection. In HCOMP@AAR AAAI, 2012.
S. Tanaka, N. Kadoya, Y. Sugai, M. Umeda, M. Ishizawa, Y. Katsuta, K. Ito, K. Takeda, and K. Jingu. deep learning- based radiomics approach to predict head and neck tumor regression for adaptive radiotherapy. Scientific Reports, 12(8899), 2022.
Wenhai Wan, Xinrui Wang, Ming-Kun Kun Xie, Shao- Yuan Li, Sheng-Jun Huang, and Songcan Chen. Unlocking the power of open set: A new perspective for open-set noisy label learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 15438-15446. 2024.
Y. Wang, X. Ma, Z. Chen, Y. Luo, J. Yi, and J. Bailey. Symmetric cross entropy for robust learning with noisy labels. In ICCV, 2019.
Y. Wang, X. Sun, and Y. Fu. Scalable penalized regression for noise detection in learning with noisy labels. In CVPR, 2022.
Z. Wang, G. Hu, and Q Hu. Training noise-robust deep neural networks via meta-learning. In CVPR, 2020.
H. Wei, L. Feng, X. Chen, and B. An. Combating noisy labels by agreement: A joint training method with co-regularization. In CVPR, 2020.
Hongxin Wei, Lue Tao. Renchunzi Xie, and Bo An. Open-set label noise improve robustness against inherent label noise. NeurIPS, 34:7978-7999 2021.
Cheng Wen-Huang, Song Sijie, Chen Chieh- Yun, Hidayati Shintami Chusnul, and Liu Jiaying. Fashion meets computer vision: A survey. ACM Computing Surveys, 2021.
Wu, S. Zheng, M. Goswami, D. N. Metaxas, and Chen. A topological filter for learning with label noise. In NeurIPS, 2020a.
Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, and C. Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. KDD, 2020b.
X. Xia, Liu, B. Han, N. Wang, M. Gong, H. Liu, Niu, D. Tao, and M. Sugiyama. Part-dependent label noise: Towards instance-dependent label noise. In NeurIPS, 2020.
15

---

X. Xia, T. Liu, B. Han, C. Gong. N. Wang, Z. Ge, and Y. Chang, Robust early-learning: Hindering the memorization noisy labels. In ICLR, 2021.
X. Xia, Liu, B. Han, M. Gong, J. Yu, G. Niu, and M. Sugiyama. Sample selection with uncertainty of losses for learning with noisy labels. In ICLR, 2022.
S. Yang, E. Yang, B. Han, Y. Liu, M. Xu, G. Niu, and T. Liu. Estimating instance-dependent label-noise transition matrix using dnns. In ICML, 2022a.
Y. Yang, K. Zha, Y. Chen, and H. Wang D. Katabi. Delving into deep imbalanced regression. In ICMI,, 2022b.
H. Yao, Y. Wang, L. Zhang, J. Zou, and C. Finn. C-mixup: Improving generalization in regression. In NeurlPS, 2022.
Y. Yao, T. Liu, B. Han, M. Gong, J. Deng, G. Niu, and M. Sugiyama. Dual t: Reducing estimation error for transition matrix in label-noise learning. NeurlPS, 2020.
K. Yi and Wu. Probabilistic end-to-end noise correction for learning with noisy labels. In CVPR, 2019.
L. Yi. S. Liu, Q. She, A. McLeod, and B. Wang. On learning contrastive representations for learning with noisy labels. In CVPR, 2022.
L. Yiming, S. Jie, W. Yujiang, and P. Maja. Fp-age: Leveraging face parsing attention for facial age estimation in the wild. arXiv, 2021.
E. Yuksel, N. Wilson, and P. D. Gader. Twenty years of mixture of experts. In Transactions on neural networks and learning systems, 2012.
T. Zadouri, A. Üstün, A. Ahmadian, B. Ermis, A. Locatelli, and S. Hooker. Pushing mixture of experts the limit: Extremely parameter efficient moe for instruction tuning. arXiv preprint arXiv:2309.05444, 2023.
S. Zang, M. Ding, D. B. Smith, P. Tyler, T. Rakotoarivelo, and M. Ali Kâafar. The impact of adverse weather conditions on autonomous vehicles: How rain, snow, fog, and hail affect the performance of a self-driving car. IEEE Vehicular Technology Magazine, 14:103- 2019.
K. Zha, P. Cao, Y. Yang, and D. Katabi. Supervised contrastive regression. arXiv preprint arXiv:2200.01 189, 2022.
C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals. Understanding deep learning requires rethinking generalization. In ICLR, 2017a.
H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lapez-Paz. mixup: Beyond empirical risk minimization. In ICLR, 2018
I. Zhang, C. Aggarwal, and G. Qi. Stock price prediction via discovering multi- multi-frequency trading patterns. In KDD, 2017b.
Zhang, L. Yang. M. B. Mi, X. Zheng, and A. Yao. Improving deep regression with ordinal entropy. In ICLR, 2023.
X. Zhang, Z. Liu, K. Xiao, T. Shen, J. Huang, W. Yang, D. Samaras, and X. Han. Codim: Learning with noisy labels via contrastive semi-supervised learning. arXiv preprint arXiv: 2111. 1652, 2021a.
Y. Zhang, H. Sun, G. Gao, L. Shou, and D. Wu. Developing spatio-temporal approach to predict economic dynamics based on online news. Scientific Reports, 12(16158), 2022.
Z. Zhang and M. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In NeurlPS, 2018.
7. Zhang, Y. Lin, Z. Liu, Li, M. Sun, and Zhou. Moefication: Conditional computation of transformer models for efficient inference. arXiv preprint arXiv:2110.01786, 20111.
16

---

Z. Zhifei. S. Yang, and Q. Hairong. Age progression regression by conditional adversarial autoencoder. In CVPR, 2017.
Hattie Zhou, Ankit Vani, Hugo Larochelle, and Aaron Courville. Fortuitous forgetting connectionist networks. In ICLR, 2022.
M. Zhou, Y. Xu, L. Ma, and S. Tian. On the statistical errors of radar location sensor networks with built-in wi-fi gaussian linear fingerprints. Sensors (Basel, Switzerland), 12:3605 3626, 2012.
Z. Zhu, J. Wang, and Y. Liu. Beyond images: Label noise transition matrix estimation for tasks with lower-quality features. In ICML, 2022.
Chen-Chen Zong, Ye-Wen Wang, Ming-Kun Xie, and Sheng-Jum Huang. Dirichlet-based prediction calibration for learning with noisy labels. Proceedings of the AAAI Conference on Artificial Intelligence, 17245411775411726 2024.
B. Zoph, I. Bello, S. Kumar, N. Du, Y. Huang, J. Dean, N. Shazeer, and W. Fedus. Designing effective sparse expert models. arXiv preprint 2XXi:22022.08066, 2022.
S. Zuo, X. Liu, J. Jiao, Y. J. Kim, H. Hassan, R. Zhang, T. Zaho, and J. Gao. Taming sparsely activated transformers with stochastic experts. CoRR CoRR:2110.04200 2021.
17

---

A Appendix: Table of Contents
The Appendix enlists the following additional materials.
I. Limitations.
B
II. Broader Impacts. S
III. Theory of ConFrag.
í. Classification versus Regression for Feature Learning D.1
ii. Fragmentation and Neighborhood Jittering D.2
iii. Prediction Depth Analysis D.3
IV. Extended Related Work.
i. Continuously Ordered Correlation of Labels and Features E.1
ii. Noisy Label in Object Detection
B.2
iii. Transition Matrix based Methods
iv. Combination with Contrastive Learning
V. Experiment Details.
i. Dataset Curation Details
Baseline Details
iii. ConFrag Training Details
iv. Random Gaussian Noise
F.4
Computation Resource F.5
VI. Extended Results & Analyses. $
i. UTKFace Results
ii. Fragment Number Analysis
2.2
iii. Hyperparameter Analysis
iv. Fragment Pairing Analysis
G.4
V. Closed-Set versus Open-Set Noise
G.5
vi. Analysis of Samples on the Bounday versus Center of Fragment G.6
vii. Ablation & Combination Analysis G.7
viii. Discretized Baselines G.8
ix. Comparison of Neighborhood Jittering and Other Regularization Methods G.9
X. Extended Selection Rate/ERR/MRAE Comparison and Analysis G.10
xi. Variance Across Random Seeds G.
xii. Standard Mean Absolute Error G.12
VII. ConFrag Pseudo Code (Algorithm)
B Limitation
A key limitation of ConFrag lies in its foundational reliance on the Mixture of Experts (MoE) model [Jacobs al. 199T). Specifically, integrating MoEs with deep learning introduces notable scalability challenges, both compurationally and in memory usage (Zuo et al. 202T Zoph et al. 2022 Zhang al., 20216). To address the memory concern, ConFrag currently employs more compact feature extractors. Nevertheless, a prominent inefficiency stems from expert redundancy in MoEs' parameters [uoo et al., (202T]. Some approaches mitigate this include distilling into sparse MoE models, employing pruning, and subsequently compressing to decrease parameter size [Kim] al., 2023 |Fedus al.) [22T]. There are also emerging strategies centered on parameter sharing. leveraging matrix product operators (MPO) decomposition [Gao et al.) 2020 2022] and parameter- efficient fine-tuning [Zdouui et al. 2023). Of these, we believe the avenue parameter sharing holds special promise when combined with ConFrag; the inherent positive feature correlation in regression problems amplifies the advantages of this approach. Also, as in MoEs, ConFrag introduces new hyperparameter, the number of experts (the number of fragments F in ConFrag's case).
18

---

In its current form, ConFrag facilitates simultaneous training of both the feature extractors and the subsequent regression task, either on a per-batch or per-epoch basis. However, a wealth of research exists that could further optimize ConFrag's scalability. These span from improving training
efficiency (He al. 2021 Zoph et al. 2022 Lepikhin et al. 2021 Lewis et al. 2021 to enhancing
inference capabilities Zhang et al. 2021b Fedus et al. 2021
C Broader Impacts
In the era of deep learning, the need for large datasets increases, yet it is expensive to obtain large dataset with high-quality annotated labels. An alternative solution to collect labels using automated labeling methods, such as web crawling. However, these methods inevitably introduce noisy labels.
This work proposes method for mitigating the negative effect of such label noise in regression, which can save time and money spent on collecting high-quality labels for many applications, bringing positive impact science, society, and economy. However, since the method reduces the need for accurate labeling, it may have potential negative effect on the salaries of label workers.
D Theory of ConFrag
We present several theoretical justifications that enhance the performance of ConFrag.
D.1 Classification versus Regression for Feature Learning
During the learning process, deep neural networks aim maximize the mutual information be- tween the learned representation, denoted as Z, and the target variable, denoted as Y. The mutual information between these two variables can be defined as T(Z: Y) ( ( A high value of T(Z;Y Y) is indicative of a high marginal entropy HZ)). Achieving this dual objective is accomplished in classification (Boudiat et al. 2020
However, Zhang et al. [2023) have shown that regression primarily focuses on minimizing H Y) while disregarding H (Z). This results in relatively lower marginal entropy for the learned represen- tation Z and ultimately leads to performance deficits in comparison to classification.
To experimentally show that this theoretical result also applies to ConFrag, we replace classification- based expert feature extractor learning with regression- based one, where each expert feature extractor is trained with regression loss on its respective fragment pair dataset. We name this variant ConFrag- R. In ConFrag-R R, self-agreement defined using distances to the mean of each fragment in the contrasting pair (+):
self = [18 151;
18yt h(2; ,
(6)
where Y, is the average of fragment f's labels, and (-)(-) is the regression function output. As in ConFrag, ConFrag-R also utilizes K-nearest neighbor-based classification for computing another variant of self-agreement. The results in Table show that using classification for feature learning outperforms using regression (ConFrag-RY) in all datasets.
D.2 Fragmentation and Neighhorhood Jittering
ConFrag operates by partitioning data samples into fragments and leveraging trained feature extrac- tors for sample selection through collective modeling. We conceptualize this as a Mixture-of-Eiee -f-peeets (MoE) model, wherein individual experts specialize in specific problem subspaces through data partitioning (Yuksel et al. 2012 Masoudnia and Ebrahimpour 2014). MoEs possess theoretically advantagcous properties with respect to computational scalability and reduction of output vari- ance Yuuksel et al.) 2012). contributing to the enhancements observed in ConFrag. It is noteworthy that since each network trained distinct training set, MoE effectively mitigates concurrent failures, thereby preventing error propagation among networks and ultimately improving the general- ization performance of ConFrag as well (Sharkey and Sharkey 1997].
Additionally, our Neighborhood Jittering leads to a Partially Overlapping Mixture Model (Heller and Ghahramani 2007a], theoretically enabling the modeling of significantly richer and more
19

---

Table 4: Comparison between ConFrag and ConFrag-R: Mean Relative Absolute Error (%) to the noise-free trained model on the AFAD-B, LMDB-Clean- SHIFT15M-B. and MSD-B dataset. Lower is better. The results the of three random seed experiments.
AFAD-B
IMDB Clean-B
symmetric
Gaussian
symmetric
Gaussian
noise rate
20
40
60
80
30
50
20
40
60
80
30
50
Vanilla
9.37
20.27
30.65
43,09
28.77
39.03
16.18
32.05
53.13
76.35
26,89
50.28
ConFrag-R
4.97
13.93
27.85
37.19
21.93
33.90
8.74
22.73
44.29
68. 14
21.74
46.93
ConFrag
2.74
8.16
15.91
34.42
17.49
27.31
5.08
12.64
27.26
61.24
15.70
33.36
SHIFTISM-B
MSD-B
symmetric
Gaussian
symmetric
Gaussian
noise rate
20
40
60
80
30
50
20
40
60
80
30
50
Vanilla
9.11
17.96
27.02
36.34
6.54
15.16
8.23
18.43
31.67
45.85
6.96
15.74
ConFrag-R R
4.18
9.59
16.21
25.76
4.96
10.90
0.77
5.68
13.63
30.05
2.79
6.87
ConFrag
2.46
6.18
10.68
19.04
3.66
8.09
0.57
4.94
11.22
23.41
2.39
6.49
intricate hidden representations by accommodating multi-cluster membership, ultimately enhancing the selection and overall performance of ConFrag.
D.3 Prediction Depth Analysis
Prediction depth [Baldock et al. 2021] of an example refers to the earliest layer where the layer-wise K-nearest neighbor probes of the layer and all the subsequent layers the same the model prediction. In other words, low prediction depth means that the example is easily distinguishable in early layers. For example, prediction depth of zero means that data be predicted at the input level only based on its distances to other data. Low prediction depth is positively correlated with better prediction consistency, lower learning difficulty, and larger margin. Due these traits, some previous works aim at reducing the prediction depth during training for better generalization performance [Zhou al. 2022 Sarti et al. 2023)
While prediction depth is initially designed as a measure of example difficulty, the mean prediction depth of the dataset can also be used as measure of dataset difficulty (Baldock et al. 2021 Also, since early layers generalize while later layers memorize deep learning [Stephenson et al. 2021 the low mean prediction depth of dataset means it is more generalizable since fewer examples require memorization.
Table 5: Comparison of mean prediction depths of feature extractor learning tasks for all- all-frag, contrastive fragment pairing, and alternative fragmentation pairings when F 4.
IMDB-Clean -lean-B
Fragment pairing
No noise
Symmetric 40%
All-frag ((1-4))
6.6291
7.2452
Contrastive
[1, 3]
3.7752
4,8116
pairing
[2, 4]
3.7028
4.7869
Alternative
[1, 4]
3,0822
4.3307
pairing 1
[2, 3]
4.8215
5.3978
Alternative pairing 2
[1,
4.8738
5,4741
[3. 4]
4.7304
5.1828
20

---

Table 6: Comparison of mean prediction depths of feature extractor learning tasks for all- all-frag,
contrastive fragment pairing, and alternative fragmentation pairings when
IMDB-Clean
Fragment pairing
No noise
Symmetric 40%
All-frag ((1-6))
7.2689
7.8102
[1. 4]
3.8635
4.8706
Contrastive pairing
[2, 5]
3.7668
4.6382
[3, 6]
3.6840
4.5026
[1, 2]
4.8979
5,4790
Alternative pairing
[3, 4]
5,0770
5.2453
[5, 6]
4.8010
5.0605
[1. 6]
2.9685
4.1188
Alternativ pairing 2
[2,
3.7668
4.6382
[3,4]
5,0770
5.2453
Tab. 6 show the mean prediction depth of all samples for each feature extractor learning task, with and without noise. The prediction depth for each task measured using ResNet- trained for 20 epochs, at which the model achieves more than 99% training accuracy. Following BBllock et al. [2021]. we use
30 for K -nearestt neighbor probe and did not use data augmentation during training. The tables show that the binary classification tasks from contrastive pairing achieve much lower prediction depths than the multi-class classification tasks using fragments (All-frag). Also, the mean and maximum prediction depths of contrastive pairing lower than those of alternative pairings, explaining why contrastive pairing outperforms alternative pairings as shown in $ $4.3) Note that the mean prediction depths of the binary classification tasks correlate with the distance between fragments: the larger the distance, the lower the prediction depth tends to be.
Fig. 7 8 show the distribution of prediction depth for each task when F 4 and F = 6. Without noise, about 40% of data in each contrastive fragment pair has a prediction depth of zero, indicating that two fragments are already much separated at input level. Even with 40% symmetric noise, more than 25% of data in each contrastive fragment pair has a prediction depth zero. Meanwhile, the most frequent prediction depth when using all fragments is nine, which means that most data can only predicted at the last hidden feature level.
While the prediction depths of alternative pairings are lower than those of using all fragments, we observe that they tend to perform worse as shown in Appendix G.4 We suspect that this is due the limitation of mixture models that the individual expert feature extractor may not fully benefit from the full dataset they model their own disjoint subset.
no noise
4, symmeizic
0.6
[1.3]
[1.3]
all frag
trag
Probability
0.38
0.2
0.18
4.8
Prodiction Degth
Prediction Dopth
Figure 7: Probability of prediction depth for examples in contrastive pairing and all all-frag (F 4).
21

---

F- no noise
F- 6. symmeuric 40%
[1.4]
[2,51
[2,5]
0.
all-trag
all trag
Probabliny
Probabiiing
00
30
Prodiction Degth
Prediction Dupth
Figure 8: Probability of prediction depth for examples contrastive pairing and all-frag (F = 6).
E
Extended Related Work
E.1
Continuously Ordered Correlation of Labels and Features
One distinctive characteristic of regression problems is their continuous label space, implying a high likelihood of correlation between regions within the feature and label spaces (Yng al. 2022b
Gong et al. 2022 Zha et al. 2022].
Recent research has extensively explored these characteristics, encompassing issues such label
imbalance Yang et al.]
2022b [Gong et al. 2022]. age estimation [Li et al. 2019]. contrastive
learning [Zha et al.
2022 and mixup regularization Yao et al. 2022
Yang et al. 2022b propose label and feature distribution smoothing based on their similarity, while Gong et al. [022 introduce regularization term aimed at aligning the rankings of feature-space space and label-space neighbors. Zha et al. [2022] employ supervised contrastive learning with pairing technique based on label distances in mini-batches To adapt MixUp [Zhang et al. 2018 for
regression tasks, Yao et al. [022 recommend interpolating proximal samples within the label space
with higher probability.
Ordinal regression, also known as ranking learning, pertains to predicting ordinal labels based on input data. It is noteworthy that ordinal regression methods are adaptable for regression tasks due to the inherent numerical ordering within scalar label spaces. Past studies in ordinal regression have successfully addressed various regression challenges. including facial age estimation [Niu et al.
2016 Shin et al. 2022). monocular depth estimation [Fu et al.) 2018), and credit rating [Hirk et al. 2019 Some of these methods share common characteristics with approach, they discretize
continuous labels, effectively converting regression tasks into classification problems [Niu et al.
2016 Fu al. 2018 Shah et al. 2022 Within the framework of ordinal regression, Garg and Manwani 2020 propose loss correction method by estimating the noise transition matrix.
It is important to note that among the previously mentioned methods, only Yao et al. [2022] and Garg and Manwani (020) can effectively address noisy label regression problems without the need for additional techniques. Additionally, Wang et al. [2022 enhance the scalability of their approach by grouping dissimilar classes within the feature space. Our work considers the continuity of labels
and features and their correlation in fragmenting and grouping data. This approach allows each
component to learn distinguishable features and improve sample selection capabilitics.
E.2
Noisy Label in Object Detection
Due to the abundance of research on object detection tasks, with bounding box localization being a prominent example of regression tasks, we have explored the issue of noisy regression within the context of object detection. In particular, obtaining accurate annotations for object detection is resource-intensive task, often constrained by limited time, small number of annotators, or reliance on machine- -eneraatd annotations. These constraints frequently result label noise, represented as incorrect class assignments inaccurate bounding box locations.
Various strategies have been developed to address the issue of noisy labels in object detection. To correct inaccurate bounding box locations, Li et al. 2020b leverage the discrepancy between two
22

---

classification heads by emphasizing the objectness of the region. Liu et al. [2022 generates object bags using the classifier as guidance, Mao al. [02TT] employs center center-matching correction, and Schubert et al. [023] drop instances with high region proposal loss on an instance-wise basis. In scenarios where image-level annotations are available, Gao et al. [2019] employs ensemble learning with two classification heads and distillation head, while[Shen et al. [2020] decomposes the problem into foreground and background noise, employing residual learning and breging-mixup learning.
We also explored the possibility of applying object detection techniques to noisy labeled regression. However, our analysis revealed that these methods are not well-suited for the broader regression task. Specifically, [Liu et al. [2022), Schubert et al. [2023], Mao et al. [202T) utilize region proposal networks generate bounding box proposals. They leverage these proposals selectively choose clean labels or re- re-weight the training samples. However, because this approach necessitates an auxiliary model the proposal generation process, it cannot be directly applied in the context of regression tasks.
Additionally, et al. 2020b], Liu al. [022]. Schubert et al. 20033] Gao et [2019] employ the object detector's 's classifier to update or assess the quality of bounding boxes. By evaluating the confidence consistency of the bounding box through the classification output, this approach helps mitigate the impact of noisy labels. However, implementing a similar approach in the context of regression tasks would require the inclusion of an auxiliary co-trained task.
E.3 Transition Matrix based Methods
Methods based on transition matrices constitute one of the primary approaches for addressing the issue of noisy labels.
Driven by the observation that the clean class posterior, denoted as pz)) can be inferred from the transition probability and the noisy class posterior, p(y x) (ylly")) ((y""))(("(("")) the modification of the loss function enables the construction of a risk-consistent estimator using the estimated transition matrix Yao et al. 2020).
There are many approaches aiming to enhance the estimation of the transition matrix. These include factorizing it into the product of two matrices by introducing an intermediate class Yao et al.) 2020). training the Bayes label transition network (Yang et al.] 2022a], learning the transition matrix within meta-learning framework (Wang et al. 2020), flonnwweighing less informative features based on f-mutual information [Zhu et al.) 2022]. and adopting a two-head architecture. The latter involves a noisy classifier for simultaneous transition matrix estimation and a clean classifier for statistically
consistent training [Ke et al. 2022).
Moreover, Xia et al. 20200] explores the utilization of part-dependent transition matrices, combining them to approximate the instance- instance-dependent transition matrix.
In an extended context, Li et al. [2022c] broadens the problem to include noisy multi-labe learning and suggests considering label correlations.
E.4 Combination with Contrastive Learning
Incorporating unsupervised learning methods proves effective in alleviating label noise, prompting the integration noisy label mitigation techniques with unsupervised learning, particularly contrastive learning.
Zhang et al. 1 show that the combination of contrastive loss and semi-supervised loss yields successful mitigation of the noisy label problem.
Beyond the application of contrastive learning, other approaches involve selecting confidence pairs and confidence samples [Li et al. 2022b). leveraging clean probability estimation derived from the relationship between representation clusters and labels (Huang al.) [202). employing class prototypes for relation between representations and labels Ortego al. 2021].
loss (. Li 2021], and implementing labelng based on the
Additionally, an approach introduces a contrastive regularization function aimed at preventing adverse effects stemming from noisy labels (Yi et al. 2022
23

---

Table 7: Dataset Statistics on the six newly curated balanced datasets for regression: AFAD-
NNuu et al., 2016], IMDB-Clean-B (Yiming al. 2021]. AMDB-WIKI-D [Rothe et al. 2018 UTKFaae-B Zhifei et al. 2017]. SHIFT15M-B
Kimura et al. 202T]. MSD-B
et al.
2011
Datasct
range
train
valid
test
total
AFAD-B
[15, 40]
27647
1627
3252
32526
[15, 661
44200
2600
5200
52000
AMDB-WIKT-L
[15, 651
42500
2500
5000
50000
UTKFace-B
[1. 70]
10467
386
787
11640
SHIFTISM-B
[0. 40000]
273417
16080
32180
321677
MSD-B
[1956, 2010]
25218
1512
2970
29700
F Experiment Details
F. Dataset Curation Details
Table provides comprehensive overview of the statistics for the six benchmark datasets meticu- lously curated for the task of noisy label regression. Detailed descriptions of the dataset tailoring process are presented below for clarity.
Age prediction datasets are harmonized by achieving a balance across distinct age values. This equilibrium is established using bin sample count threshold (clip value) of 1000 for IMDB-Clean-B B and IMDB-WIKI-B, 1251 for AFAD-B, and 200 for UTKFace-B. Image inputs are resized to dimensions of (128 x 128). For the regression task, we consistently employ a ResNet-50 backbone across all models.
AFAD-B, IMDB-WIKI-B, UTKFace-B); These datasets
Achieving data balance in this dataset involves a two-step process. First, the label space is binned based maximum price of Y40000 are clipped to remove outliers. The binning threshold is set at 16084 sample counts to further ensure balanced representation. To standardize the label currency, it is pegged to the U.S. dollar, referencing exchange rates from 2010 to 2020, which coincides with the period when the original clothing item data is collected. Notably, this dataset is provided the penultimate feature of the ImageNet pre-trained VGG-16 model. Consequently, we opt for a three-layer MLP architecture with hidden layer size of [2048, 1024, 512], aligning with recommendations from Papadopoulos et al. 2002 and Kimura et al. [2021].
SHIFT15M-B:
price threshold of *2000. Subsequently, data points exceeding the
MSD-B: Achieving balance in the Million Song Dataset involves setting threshold of 550 samples per year. For all regression models in this context, we adopt a regression backbone rooted in the tabular ResNet structure proposed by Gorishniy et al. [202]] featuring hidden dimension of 467.
Licenses of existing datasets. IMDB- Clean dataset Yiming al. 2021] is under MIT license SHIFTISM dataset
Kimura et al. |021 is under CC BY-NC 4.0 and MIT license f MSD (Bertin- Mahieux et al. 2011 1] song year prediction dataset is under CC BY 4.0 license UTKFace dataset
Zhifei et al. 2017 is available for non-commercial research purposes only AMDB-WIK dataset
2018 is available for academic research purpose only Unfortunately, the license of AFAD Niu et al. 2016 dataset could not be found
Rothe et al. 2015
F22 Baselines Details
While numerous branches of noisy labeled learning have been explored for classification tasks, our focus in this study centers on the challenging domain of noisy label regression. To comprehensively investigate this task, we have conducted an extensive review of the various branches and have selected
24

---

a set of fourteen baselines that are adaptable to regression. It is worth noting that C-Mixup Yao al. 2022] was originally proposed as a regression baseline. In the following section, we provide an overview of these selected baselines, offering a broad coverage of diverse approaches to address the noisy label regression problem. Additionally, we present detailed descriptions of the experimental settings for each baseline.
1. D2L [Ma et al. 2018 for intrinsic dimension exploration. Following the paper, we set k = 20 and m 10 for Local Intrinsic Dimensionality (LID) estimation and set the LID
estimation window as five following the official implementation.
2. CDR [Xia et al. 2021] for model weight parameter selection, and RDI (Hu et al. 2020 for regularizing the paramter distance from the initialization. At RDI, we use search space
X E [0.25, 0.5, 1, 2, 2,4, 8).
3. C-Mixup Yao et al. [022] for regularization via continuous mixup. C-Mixup-batch is used in all experiments because of the excessive memory requirement for pairwise distance matrix
P. We set the beta distribution variable a as 1.5. The bandwidth variable a is searched over
[0.01, 0.1, 1], following Yao al.] (2022).
4. SELFIE [Song al. 2019 and AUX (Hu et al. 2020 for refurbishing. To apply SELFIE to the continuous label, we redefine the concept of uncertainty F(x;; v) and refurbished labels
yeefurb with the mean and standard deviation.
(HH(())
F(x;9) (max (V) - min (Y)) <6
grefurb (g))((g))
(8)
where (z) is the prediction history of x: from before q epochs, is the uncertainty threshold.
For SELFIE, we train 1/4 of the total training epochs for the warm-up phase, following Song
et al. 2019). . The variable q is searched over half of the warm-up epochs and around. The
variable is searched over [0.05, 0. 0.15, 0.20), following [Song et al. (2019).
For AUX [Hu al. 2020 we regularize the auxiliary variable by weight decay 0.0005,
reducing the weight by 0.1 at 1/2 and 3/4 of the total training epochs. The learning rate of the auxiliary variable is set to 0.1 and 0.01. The variable X is searched over [0.25, 0.5, 1, 2,
4, 8).
5. SPR [Wang et al. 2022 performs penalized regression for selection. It requires some adaptation to regression by ignoring the f penalty as there is no longer a lincarity gap between the scalar output and the final fully connected layer that requires reducing. Also, we use our fragmentation splits (4,8) to bin the regression data for SPR's parallel optimization.
6. Sigua (Han et al. 2020 and CNLCU-S/H (Xia et al. 2022 for small loss selection. For Sigua, we use dt)) E 0..3, 0.4] and 7 = 0.01 and set T. as 5% of the total training epochs. For CNLCU-S/H. we search and Tmin in [0.01, 0.1, 1, 10] and set T3 as 5%.
7. BMM (Arazo et al. 2019 for selection based on beta mixture model fitting on the loss distribution. BMM does hard sampling and trains using the selected samples. DY-S is a dynamic soft loss. We implemented two versions; the first uses convex combination as in Reed et al. [2015] ((1 - w) 17 - wÿ)². Second, instead of bootstrapping, we dynamically weight the loss using the BMM probability to create cost-sensitive loss, (1 w... The w is the mixture clean probability, ý is the model prediction, ÿ is the assigned noisy label, and l is the loss.
8. SuperLoss [Castells et al., 2020) regularization parameter À is searched over [0.01, 0.1, 1, 10] while uses an exponential running average with a fixed smoothing parameter a = 0.9.
9. [Incompatible] CRUST [Mirzasoleiman al. 2020 for clean coreset selection. It aims to select coreset based on class-wisely gradient clustering. For regression, we initially viewed all data as a single class and proceeded with coreset selection, but the results were unsatisfactory. Therefore, we report results based only on the discretized version, demonstrating comparable performances. We select 1/2 the total dataset as a coreset. The distance threshold in calculating clusters is searched over [1, 2, 4].
25

---

noisy label
noisy label
clean Inbel
Gaussian 30
Gaussian 50
Figure 9: Random Gaussian Noise. (a) Gaussian noise injected from the uniformly sampled random standard deviation between 30]. (b) Gaussian noise injected from uniformly sampled random standard deviation between [1, 50).
10. [Incompatible] OrdRegr Garg and Manwani 2020] for loss correction. Since no official implementation is provided, implemented with cross-entropy loss for ordinal regression. Importantly, we failed to find accurate noise rate estimation using their suggested methods. Even when considering the transition matrix with the actual noise rate, the loss correction algorithm proved ineffective in our benchmark tests.
F33
ConFrag Training Details
ConFrag employs the Cosine Annealing Learning rate [Loshchilov and Hutter 2017 with minimum learning rate of Mmmn = O. The optimization is carried out using the Adam optimizer (Kingma and Ba 2015). For the K K-nearest neighbors- based prediction, we experiment with various values of K, specifically choosing from the set [3, 5, 7]. The number of fragments, denoted as F, remains constant four throughout all the experiments. To determine the buffer range for jittering, we conduct search over values within the range [0, 0.05, 0.1).
Some dataset-specific hyperparameters exist:
Age prediction task datasets, IMDB Clean-B Yiming et al., 2021 AFAD-B NNu et al. 2016), IMDB-WIKI-B WIKI-B [Rothe et al. 2018 and UTKFace-B Zhifei et al. 2017 J. train for 120 epochs with a learning rate of 0.001. Each feature extractor employs the ResNet-18 18 architecture, which contains only 48% of the parameters found in ResNet-50, the architecture utilized for the regressor.
Clothing price estimation task dataset SHIFTI5M-B Kimura et al. 202 trains for 40 epochs with learning rate of 0.0001. MLP with hidden dimensions [1024, 512, 256] is deployed for feature extractors, and the parameter size is 44% of the regressor.
Music year production task dataset MSD-B MeerinnMMhiieu et al. 2011 trains for 20 epochs with a learning rate of 0.0001. Similar the regression backbone, the feature extractor model is the tabular ResNet structure structure/Gorishhny et al. 2021], and the hidden dimension reduced 256.
F.4 Random Gaussian Noise
Fig. illustrates application of random Gaussian noise within the label space IMDB-Clean-
B Yiming et al. 2021]. The procedure for injecting noise is akin to the approach employed by
Yao et al. [2022]. where Gaussian noise applied to every unique label within the training samples. Specifically, Yao et al. 20022 sets the standard deviation of the Gaussian noise fixed 30% of
26

---

the range of the label space corresponding to the dataset. In contrast, our noise injection method introduces an element of stochasticity, allowing for variable levels of deviation for each unique label.
To achieve this variability, we employ uniform sampling from the minimum and maximum values specific each label's domain. For instance, in the context of an age prediction task, we assume minimum and maximum values of and 100, respectively. However, in cases where the label domain lacks clarity (e.g., for variable like 'price'), we utilize the minimum and maximum label values provided by the dataset itself.
It is important to highlight that baselines with known noise rates, such CNLCU-S/H, Sigua, and Selfie, are incapable of dealing with Gaussian noise. Given that these baselines employ a heuristic approach to control selection rates through (1 noise rate), they prove ineffective when exposed to Gaussian noise, as it introduces noise to all samples, thereby resulting in a nearly 100% noise rate. Hence, create soft noise rate be used by them for selection. This is done by calculating updated noise rate, assuming that the Gaussian noise injected samples that fall within an acceptable variance of the original ground-truth label are clean (the acceptable variance is set equal the label length/size of single fragment).
Computation Resource
For implementation, we use Python 3.9 and PyTorch 1.13.1. All experiments are conducted using NVIDIA Quadro 6000 24GB RAM GPUs. The required computation time for experiments differs depending on the dataset. Below, we report the computation time for each dataset.
AFAD-B. On the AFAD-B dataset, the average computation time of ConFrag and Co-ConFrag are approximately 2.5 GPU hours and 5 GPU hours, respectively. The computation time of baselines ranges from 1.25 GPU hours to 4 GPU hours. About 650 GPU hours were required to produce the
AFAD-B part of Tab.
IMDB-Clean-B. On the IMDB-Clean-B dataset, the average computation time of ConFrag and Co Co-ConFrag approximately 3.5 GPU hours and 5.5 GPU hours, respectively. The computation time of baselines ranges from GPU hours to GPU hours. About 970 GPU hours were required to produce the IMDB-Clean's part of Tab.
SHIFTISM-B.
the SHIFT15M- dataset, the average computation time of ConFrag and Co- ConFrag are approximately GPU hour and 1.2 GPU hours, respectively. The computation time of baselines ranges from GPU minutes to GPU hour. About 100 GPU hours were required to produce the SHIFTI5M-B part of Tab.
MSD-B. On the MSD-B dataset, the average computation time of ConFrag and Co-ConFrag approximately 4 GPU minutes and GPU minutes, respectively, The computation time of baselines ranges from 1 GPU minute GPU minutes. About 9 GPU hours were required to produce the MSD-B part of Tab.
Additionally, further computation was required for analysis and experiments in
4.3 and Appendix G
G Extended Results & Analysis
We conduct supplementary experiments and analyses of UTKFaae dataset, parameter sizes, fragment numbers (F), other hyperparameters (K, J), fragment pairing, and the impact of closed-set and open-set noise. Furthermore, we present ablation analyses, comparisons with discretized basclines, baseline performance evaluations considering Selection rate and ERR, variance assessments, and the obtained MAE results.
G.1 UTKFace Results
Table 8 presents comparison of MRAE values between various baseline methods and our proposed approach on the balanced UTKFaaee dataset [Zhifei et al. 2017). UTKFace-B. We experiment under four different symmetric noise conditions of symmteric 20%, 40%, 60% and 80% noise rates. Both ConFrag and Co-ConFrag demonstrate superior performance across all experiments when compared to the fourteen baseline methods.
27

---

Table 8: Comparison of MRAE (%) UTKFace-B datasets with symmetric noise.
UTKFaae-B
symmetric
20
40
60
80
Vanilla
37.59
53.96
82.88
115,49
AUX
25.53
48.01
84.78
118.31
BMM
49.21
76.25
88.87
139.01
CDR
32.87
50.56
83,41
121.36
C-Mixup
17.76
34.00
74.29
117.68
CNLCU-H
5.75
20.75
43,44
121.55
CNLCU-S
20.29
36.69
43.44
121.55
D2L
38.28
51.22
99.19
122.50
DY-
22.73
31.07
58.05
113.21
RDI
43.49
49.44
75.67
122.40
Selfie
30.47
44.62
99.67
130.97
Sigua
10.86
19.58
52.37
128.06
SPR
28.05
48.07
85.18
120.58
Superloss
9.12
22.10
55.78
115.78
ConFrag
6.28
14.30
34.09
83,03
Co ConFrag
-2.22
8.88
25,46
74.78
F=4
F=6
F=8
F=10
vanilla
100
70
Selection rale (%)
80
80
MRAD (5)
50
ERR
40
30-
40
20
20
10
25
50
75
100
25
50
75
100
25
50
75
100
spoch
epoch
epoch
Figure 10: Fragment number analysis compares the Selection rate, ERR and MRAE on IMDB- Clean-B with symmetric 40% noise.
G.2 Fragment Number Analysis
In Fig. 10 and II we undertake an examination of various fragment numbers within the context of symmetric 40% noise, using the IMDB-Clean-B and SHIFTISM-B datasets as benchmarks. Our evaluation criteria encompass the Selection rate, Error Residual Rate (ERR), and Mean Relative Absolute Error (MRAE). The number of fragments is chosen from F E [4, 6, 8, 10). To address scenarios with smaller fragment number, examine cases where F or 2. When F 2, fragment/ ff that satisfies self-agreement (Eq.3) does not meet the criteria for neighbor-agreemen ngh (a, in Eq. as the agreement relies on comparing the distribution of fragment f and its contrasting pair ft. Consequently, the unified neighborhood agreement (Eq. 4 consistently yields value of 0. On the other hand, defining a contrasting pair is not feasible when F 1. Instead, we present a plot of the vanilla baseline to illustrate the when F without utilizing ConFrag.
The results reveal that the MRAE of the vanilla model initially decreases during the early epochs learns patterns from clean samples. However, as the model starts to memorize noisy samples, the MRAE degrades. In contrast, ConFrag consistently mitigates the impact of noisy samples across all plots (F E [4, 6, 8, 10)) when compared the vanilla bascline.
28

---

F-4
F-6
F=8
F-10
vanilla
100
30-
Selection rale (%)
90
25
ERR (%)
80
MRAI (%)
80
70
60
40
30
10
20
30
10
20
epoch
opach
epach
Figure 11: Fragment number analysis compares the Selection rate, ERR and MRAE on SHIFTI5M- with symmetric 40% noise.
We also observe declining trend in performance as the number of fragments increases in the case of IMDB-Clean-B. In contrast, SHIFT15M-B exhibits relatively stable performance across different fragment numbers. This decrease in performance with an increased number of fragments is likely attributed to finer division of the training data among feature extractors, ultimately leading to overfitting and reduced generalization capabilities.
80
70
80
65-8
ERR (%)
70-
MRAE (%)
60
Selection
40
55-
50-
50-
K=
40-
25
100
75
25
50
epoch
epoch
epoch
Figure 12: Hyperparameter K analysis compares the Selection rate, ERR and MRAE on IMDB-
Clean-B with symmetric 40% noise.
30
70
60-
25
Selection rate
65
ERR (%0)
MRAB (96)
20
60-
15
K-3
K=5
10-
45-
10
20
30
10
10
20
30
epoch
apech
epach
Figure 13: Hyperparameter K analysis compares the Selection rate, ERR and MRAE on SHIFT15M-B with symmetric 40% noise.
G.3
Hyperparameter Analysis
The hyperparameter K used for K-nearest neighbor classification when assessing self/neighhor agreement from a representational perspective. As shown in Fig. 12 with an increase in the value
29

---

ontrocter stounig (W6
80
80-
75-
Seledio catt (5)
70-
00
70-
2RR (8)
80
60
5.-
100
005-
1-1100
25
50
75
100
25
25
25
spoct
opoch
spoch
spoch
IMDB-Cleam B with symmetric 40% noise
20g ottrattr aorurer 180
90-
20-
Sootyttie rate M6B
70-
80
80
66-
60
ERR
70-
MRAE
en
60-
10
40-
25
00
80
50
100
(b) IMDB-Clean-B -B with symmetric 60% noise
Figure 14: Hyperparameter J analysis compares the average accuracy of feature extractors, the Selection rate, ERR and MRAE on IMDB-Clean-B with symmetric 40%, 60% noise.
ottrattor a00ATa 660
77
80
36
Joc.15
76-
25-
Selrccton rato
72-
30
60
EAR
60-
20
30
68-
30-
50
10
10
20
10
10
10
opoch
epoch
opoch
opoch
(a) SHIFT15M-R with symmetric 40% noise
100 oftrerbir accurt (36)
30-
3wo.25
80-
72-
Seleetonn mhe
ERR (N)
68
50
66-
60
60
1.0
20
10
1.0
<poch
(b) SHIFTISM- with symmetric 60% noise
Figure 15: Hyperparameter J analysis compares the average accuracy of feature extractors, the
Selection rate, ERR and MRAE on SHIFT1SM-B with symmetrie 40% 60% noise.
of K, the criteria for agreement become more stringent. Consequently, as the value of K increases, greater number confident samples are selected, resulting in a reduction in the Selection rate and ERR.
The hyperparameter J controls the buffer range for jittering, which, turn, determines the level of regularization applied via neighborhood jittering. Increasing the value of J results in stronger regularization, effectively preventing overfitting. However, excessive regularization, as observed when J 0.10, may result in adverse effects during training. Specifically. in Fig. [4[a). the feature extractors exhibit similar convergence patterns when J = 0.05 or 0.10. Consequently, comparable performance is observed in Selection Rate and AE. Yet, in Fig. (4(b). the ERR of
== 0.05 smaller than that of J = 0.10, leading to improved MRAE performance for J 0.05. Similar effects are observed in the SHIFTI5M dataset, depicted in Fig. 15 (SHIFTI5M-B).
30

---

contrastive
all-frags
[1.6..2.51(3.41
[1.2..3.411.88]
80
90-
50-
MRAE (%6
60
ERR (%)
40
Selection
70-
40
50-
20
20
25
50
100
25
50
75
100
25
5.0
75
100
epoch
epoch
epoch
Figure 16:
Fragment pairing analysis compares contrastive pairings [2, (3,6)) all-
fragments (1, 2, 4, 6)). and alternative pairing methods ([1, 2]. [3, 4), [5, 6] and [1, 6), [2, 4) on IMDB-Clean-B with 40% symmetric noise when = 6. For feature extractor, all-fragments use ResNet-34, while other pairing methods use ResNet-18 backbones.
all-trags
[1,41/2.3]
[1,2]_334]
60-
80
90
50-
ERR (%)
MRAE (%6)
Selection
40
30-
20-
20-
23
50
75
100
23
50
75
100
25
50
73
100
opach
opoch
epoch
Figure 17: Fragment pairing analysis compares contrastive pairings (11,3, [2, 4)). all-fragments
([1, 2, 4)), and alternative pairing methods (11, 2), [3, 4] and [1, 4], [2, 3) IMDB-Clean-B with 40% symmetric noise when F = 4. For feature extractor, fll-fragments use ResNet-34 while other pairing methods use ResNet-18 18 backbones.
G.4 Fragment Pairing Analysis
In Fig. [c). we offer deeper insights into our approach by comparing contrastive fragment pairing (11, 4]. 2, 5], [3, 6)) against all- all-fragments 211, 4, 5, 6). In Fig. G(a), show the importance of contrastive fragment pairing by comparing contrastive fragment pairing to alternative pairings. In Fig. 16 present the extended results with Selection rate, ERR, and MRAE alongside other pairing methods.
The experiments involve training the feature extractors using either contrastive fragment pairing, all-fragments, or alternative pairings. Notably, a single feature extractor is employed for all fragments, whereas the fragment pairing (contrastive alternative) uses smaller feature extractor for each individual pair. Subsequently, sample selection is executed in accordance with the Mixture of Neighboring Fragments approach ($ 2.3
In an optimal selection algorithm, the Selection rate should approach 100 noise rate(%), with ERR and MRAE minimized. Across all evaluation metrics, the contrastive fragment pairing demonstrates superior performance compared other methods. important to highlight that performance
poorest when the pairing is least distinguishable (1,2), [3. 4], 6) when F -6, [1.2,, [3, 4) when
F = 4) and moderate when the pairing partially distinguishable (1, 6), [2, 5), [3. 4] when F [1, 4], [2, 3] when F = 4).
31

---

(a) all.fragmeent((011222....
50
(b) neighhar fragments([O,11
(c) neighbor fragments((2.3)
S
(d) contrasting fragments(0 (,2]]
(e) contrasting fragments<(1,]]
77
partial contrusting fragments((0.3]
partial contrasting fragments
Figure 18: Detailed Representation Depiction. detailed comparison of the effect of fragment pairings via (-SNE visualization of the penultimate features from the feature extractors. The experi- ments are based on IMDB-Clean-B.
Furthermore, in Fig. 18 we utilize t-SNE to compare the feature extractors trained using contrastive pairing, alternative pairinss, and all-fragments. The visual comparison validates that representations trained with contrastive pairs exhibit significantly more distinguishable features.
G.5 Closed-Set versus Open-Set Noise
To explore the impact of closed-set and open-set noisy samples, depicted in Fig, [[b) in the main manuscript, we conducted an analysis of Selection rate, ERR, and MRAE performance while gradually introducing closed-set set and open-set noisy samples into the IMDB MMBB-Clean- -B dataset. Our study employs the MMDB-Clean dataset, comprising fixed set of clean samples that represent 40% of the total dataset, alongside varying amounts of noisy samples. These noisy samples are classified into two distinct categories; closed-set and open-set noise [Wei et al. 2021 Wan et al. 2024]. For example, consider example with fragments, whose contrastive fragment pairs are {113), (2, 4)). When training feature extractor on binary classification between fragment I and 3, noisy sample
32

---

set
Selection (%)
70
closed-snt
35
ERR (%)
MRAB (%)
30
65
50
25
60
45
20
400
1200
800
400
1200
number noisy samples
number of noisy samples
numbor notay samples
Figure 19:
Closed-see/open-sett noise analysis displays the selection, ERR and MRAE when closed-set open-set noisy samples injected into the clean dataset. The experiments based on IMDB-Clean-B.
Table 9: Difference of selection rate and ERR between the samples at the boundary and center of fragments
Selection rate
20%
40%
60%
80%
boundary
79.55%
65.31%
55.98%
65.96%
center 80.18%
66.86%
54.64%
61.16%
difference
0.63%
1.55%
1.34%
4.80%
ERR
boundary 31.90%
39.01%
55.44%
82.26%
center 30.16%
36.63%
50.42%
82.91%
difference 1.74%
2.38%
5.02%
0.65%
whose ground truth fragment is either 2 or 4 but mislabeled as fragment id either or is an open-set noisy sample. On the other hand, noisy sample whose ground truth fragment id is but mislabeled as (and vice versa) a closed-set noisy sample.
Fig. 19
demonstrates that closed-set noisy samples have considerably more adverse impact on ERR and MRAE compared to open-set noisy samples. Our contrastive fragment pair-based learning approach is advantageous in this regard, as it introduces open-set noisy samples in lieu of many closed-set noisy samples, thereby facilitating learning with reduced interference.
G.6 Analysis of Samples on the Bounday versus Center of Fragments
Table presents comparative analysis of the selection rate and error reduction rate (ERR) between samples located at the boundary and the center of fragments across eight experimental configurations. The results indicate average difference of 2.29% in selection rates and 2.43% in ERR between the two groups. These findings substantiate the robustness of ConFrag's 's sample selection process, demonstrating consistent performance irrespective of the sample's positional location within the fragment.
G.7 Ablation & Combination Analysis
In Table I0 we present a comprehensive study comparing the performance of Cross-Entropp (CE) and Symmetric Cross Entropy (SCE) [Wang et al. 2019] losses in various ablation and combination experiments conducted on the IMDB-Clean- dataset Yiming et al.) 2021], considering scenarios with 40% symmetric noise and two variations of Gaussian random noise, each having maximum standard deviation of 30 and 50.
Firstly, we illustrate the impact of jittering regularization through ablation on each of the losses. No- tably, jittering regularization emerges as crucial component of ConFrag's performance, preventing the model from overfitting the noisy labels.
33

---

Table 10: Ablation and Combination Analysis. The values are mean relative absolute enror to the noise-free trained model on the IMDB-Clean-B [Yiming et al. 2021] dataset, and lower values indicate better performances. The results are the mean of three random seed experiments.
ablation and combinations
IMDB-Clean-B
symmetric
Gaussian 30
feat. ext. loss
backhone
jitter Co-teaching
40
50
CE
ResNet-18
18.90
21.77
39.78
CE
ResNet-18 18
12.64
15.70
33.36
CE
ResNet-34
13.44
16.06
31.00
CF
ResNet-18
<
9.45
14.87
35.88
SCE
ResNet-18
18.37
20.80
38.10
SCE
ResNet-18
16.84
20.07
38.18
SCE
ResNet-34
14.97
18.95
36.12
SCE
ResNet-18
V
13.19
18.32
41.02
Table 11: Discretized Baseline Analysis. Mean Relative Absolute Error to the noise-free model of discretized versions of strongly performing models on the dataset. Lower better.
(imiig et al. 2021
IMDB-Clene
symmetric
Gaussian 30
noise rate (%)
50
CNLCU-S-D
Xia ct al.
2022
55.71
64.71
79.59
CNLCU-S-D
mixup
Kia al.
2022
55.14
67.17
81.32
CNLCU-I--D
Xia et al.
2022
37.76
51.36
76.40
CNLCU-I-DD mixup
Xia
2022
65.32
67.31
84.22
Sigua-D Han al. 2020
56.17
61.67
66.08
Sigua-D + mixup Han ct al.
2020
33.55
29.33
49.44
BMM-D Arazo et al. 2019
33.86
30.27
50.05
MD-DYR-SH-D AAazo al.
2019
33.89
31.18
51.23
CRUST-D Mirzasolerman al.
2020
33.86
30.27
50.47
CRUST-D mixup Mirzasolerman cf al.
2020
32.33
30.50
50.27
Selfie-D [Song et al.
2019
31.50
24.86
47.46
Selfie-D + mixup Song et al.
2019
35.33
28.02
46.42
Co-Selfie-D [Song et 2019
30.20
26.36
49.61
Co-Selfie-D mixup Song
2019
33.18
28.28
52.20
ConFrag (Ours)
12.64
15.70
33.36
Co-ConFrag (Ours)
9.45
14.87
35.88
The next ablation experiment entails replacing the ResNet-18 architecture of the feature extractors with ResNet-34. The performance is enhanced when trained with SCE but decreases when trained with just CE. This suggests that ConFrag could potentially benefit from a more powerful architecture, but it is not necessity.
significant advantage of ConFrag lies in its compatibility with other approaches. We showcase its performance when combined with additional technique: Co-teaching [Han et al. 2018), which is also employed by CNLCU and Co-Selfie in our baseline. Co-teaching involves training the regression model while heuristically assuming that 25% of the original noise still exists in the data (e.g., 40% original noise implies an assumption of 10% noise during Co-teaching regression). Empirical observations reveal that Co-teaching consistently provides significant benefits.
Upon comparing and SCE for feature extractor training loss, we observe that CE, when combined with jitter regularization, synergizes better to exhibit much stronger performance compared to SCE.
34

---

80
Selection (%)
80
CE Mixup
75
70
50
MRAB (39
60-
SCB
Mixup
CE
Jittering
65-
CE + Mixup
ERR (%)
75
40-
60
SCT
Ang
SCE Mixup
littering
50-
20-
30
60
120
30
120
90
epoch
opoch
opoch
(a) Feature extractor accuracy
(b) Selection rate ERR
/RR
(c) Regression performance
Figure 20: Comparison of regularization methods. Compared to other regularization methods, neighborhood jittering demonstrates superior performance in (a) feature extractor test accuracy, (b) ERR, and (c) performance in regression. The analysis is conducted on IMDB- 1-BB-Clean-B with symmetric 40% noise.
G.8 Discretized Baselines
In Table 11 we present a discretized version of several strong baselines, including Sigua (Han et al.
2020J. CNLCU [ia al. 2022). BMM [Arazo al. [201]] Selfie/Co-Selfie (Song al., 2019].
MD-DYR-SH Arazo et al.
2019,, and CRUST Mirzasoleiman et al. 2020).
The discretization process aligns with fragmentation approach used for ConFrag. We obtain selected samples at the end of every epoch to independently train the regression model. Additionally, we report performance with mixup [Zhang al. 2018), technique that proves beneficial for some baselines like Sigua [Han et al. 2020].
Notably, most baselines exhibit deterioration in performance following discretization. However, Selfic/Co-Selfie [Song et al. 2019] stands out as the exception, showing an improvement in perfor- mance after discretization. Interestingly, Sigua is the sole method that benefits from mixup [Zhang
ct al. 2018 training.
G.9
Comparison of Neighborhood Jittering and Other Regularization Methods
In Fig. 20 we compare neighborhood jittering with other regularization methods that can be applied classification-base feature extractors (SCE with weight decay (Wang al. (019), mixup (Zhang et al. 2018), and their combinations). In conclusion, neighborhood jittering exhibits the strongest performance in feature extractor test accuracy, ERR, and MRAE, among other regularization methods. It is observed that ERR and MRAE improve in line with the performance of the feature extractor.
G.10 Extended Selection Rate/ERR/MRRA Comparison and Analysis
In addition to presenting the Selection rate, ERR and MRAE for symmetric 40%, Gaussian 30, and Gaussian 50 noise experiments on the IMDB-Clean-B dataset in the main manuseript, we have included results for all noise types, along with additional baselines (CNLCU-H, Sigua, BMM, DY-S, AUX, Selfie, Coselfie), in both Fig. and
and Fig.
As mentioned in $ 4.2 the ideal scenario for selection and refurbishment methods involves achieving high selection rate while maintaining low ERR. resulting in a reduced MRAE. We examine the relationship between the selection rate, ERR, and MRAE based on Fig. 21.b). As training progresses, ConFrag and other selection methods (CNLCU-H, Sigua, BMM, DY-S) approach the ideal condition, resulting in an improving trend in MRAE ConFrag, in particular, comes closest to the ideal scenario, resulting in superior MRAE performance.
The most unfavorable scenario arises when there a low selection rate coupled with high ERR. Selfie exemplifies the scenario in Fig. bT],), which is connected to relatively worse MRAE.
The scenarios of the low selection rates with low ERR and the high selection rates with high ERR can be further examined using CNLCU-H and BMM. CNLCU-H demonstrates superior selection quality
35

---

in terms of ERR, while BMM exhibits a higher quantity in the selection rate. This quality/quantity trade-off is linked to the observation that CNLCU-H and BMM show similar MRAE performance in Fig. 2T b). Additionally, Fig. 22 a) reveals that the selection rate gap widens, while the ERR gap narrows when compared Fig. bb). This is associated with BMM outperforming CNI.CU-H in terms of the MRAE.
It's important to note that, rather than employing the selection rate and ERR indicators for MRAE, these metrics offer valuable insights when assessing selected or refurbished samples directly independent of any potential regularizing effects introduced by the underlying regression model.
In addition, upon a detailed analysis of the figures, it becomes evident that Co-ConFrag consistently achieves the lowest ERR across wide range of noise types. Notably, it maintains a Selection rate of above 40% while maintaining low ERR even in the presence of severe noise conditions, which leads outstanding MRAE performance.
Conting
CNUCU-H
BMM
AUX
Co-Setto
CoCCnFin
Sigra
100
200-
100-
Selection rale (96
80-
12EE (%)
150-
MRAE (5)
60-
80-
100-
40-
70-
20-
20
60
100
120
100
120
40
100 120
spoch
epoch
egoch
(a) Symmetric 20%
100
150-
Selectins rate (%)
125-
80-
ERA 05)
100-
MRAE (%3
80-
60-
75-
40-
50-
20-
100 120
25
20
40
100
120
db
160 120
spoch
epoch
(b) Symmetric 40%
100
100
Selecttoon rate (%)
90-
E3R (96)
80-
MRAE (80
100-
80-
70-
60-
80-
40-
40
50-
20
40
100
120
20
40
60
16o
120
40
60
100
120
opoch
epoch
epoch
(c) Symmetric 60%
100
100-
Selectine. sate (%)
140-
80-
Oroep!
L100 (K)
90-
MRAE (96)
120-
60-
100-
80-
80-
70
60-
20
00
60
90
100
120
20
40
60
80
100
120
20
40
60
80
100
120
spoch
spoch
egoch
(d) Symmetric 80%
Figure 21: Selection, ERR and MRAE comparison of ConFrag, Co-ConFrag and filter- ing/refurbishment baselines on noise, repectively.
with symmetric (0%(a), 40%(b). 60%(c) and 80%(d)
36

---

ConFiig
CNLCU-I
BMM
AUX
Co-Selie
Co-ConFreg
Sigua
100
Solerton rate: (90
90-
80
100-
MRAE
80-
70-
60
60-
80-
40-
20-
50
80
120
100
120
BO
100 120
spoch
epuch
epoch
(a) Gaussian
1004
Solerton rate: (9)
120-
100-
ENR
100
MRAE
80-
60-
80-
60
40-
80 100 120
100
120
90
100 120
spoch
epoch
epoch
(b) Gaussian
Figure 22: Selection, ERR and MRAE comparison of ConFrag, Co-ConFrag and filter- ing/refurbishment baselines on IMDB-Clean- with Gaussian 30(a) and Gaussian 50(b) noise, repectively.
G.11 Variance Across Random Seeds
In Fig. 23 we plot the variance of three unique random seed experiments all six noise types (symmetric 20%/40%%/60%%808% Gaussian 30/50) on the IMDB-Clean-B dataset. To declutter the graph, we compare it against the top best-performing baselines under each noise type.
Tab.
12
13
show the main experimental results of Tab. [ with standard deviation.
G.12
Standard Mean Absolute Error
In Tables 14 15
we report the standard mean absolute error (along with standard deviation) within the respective label ranges for each dataset.
37

---

ConFing
ConFing
50
M-iixpp
100-
BMM
CNLCU-H
50-
10
40-
20-
20-
120
epoch
epach
(6) Symmetric 20%
Symmetric 40%
50-
ConFing
DY-S
55-
Mixup
75-
60-
70-
ConPrag
65-
30
60
90
120
60
120
epoch
epoch
(c) Symmetric
Symmetric
ConFing
C-Mixup
AUX
MRAE
50
CooFrag
DY-S
C-Misup
90
120
opoch
spoch
(c) Gaussian
30
Gaussian
50
Figure 23: Variance Analysis of three unique random seed experiments on IMDB-Clean-B, The top two best-performing baselines under each noise type are reported.
38

---

Table 12: Mean Relative Absolute Error (%) and its standard deviation to the noise-free trained model on the AFAD-B, IMDB-Clean and value indicates performs even better than the noise-free model. The results are the mean of three random seed experiments. Number in parenthesis indicates standard deviation. The best and the second best methods are respectively marked in red and blue. CNLCU-S/H, Co-Selfie, and
datasets. Lower is better. A negative
Co-ConFrag dual networks to teach each other done in Han al. 2018).
AFAD-B
IMDB-Clean-B
IMDB-WIKT-R
symmetric
Gaussian
symmetric
Gaussian
real nnise
noise
20
40
30
50
20
40
60
30
50
Vanilla
9.37
20.27
30.65
43.09
28.77
39.03
16.18
32.05
53.13
76.35
26.89
50.28
00.00
(0.72)
(0.93)
(1.15)
(45.96)
(1.57)
(4.32)
(1.60)
(0.20)
(0.93)
(1.29)
(2.45)
(9.07)
(00.00)
CNLCU-S
20.44
66.62
82.83
(0:42)
(3.60)
(0.18)
(46.23)
(1.11)
(5.33)
(2.03)
(2.74)
(2.82)
(0.86)
(10.54)
(4.38)
(1.28)
CNLCU-H
16.32
36.01
-2.93
(0.77)
(1.51)
(3,39)
(28
(3.08)
(2.79)
(0.64)
(1,29)
(2.01)
(1.65)
(5.60)
(7.16)
(0.82)
Sigun
21.09
42.52
46.
77.59
60.97
77.42
1.96
(1.43)
(2.15)
(2.12)
(51.69)
(3.47)
(3.85)
(0.54)
(9.52)
(2.01)
(0.94)
(19.19)
(2.07)
(1.65)
18.85
54.88
25.67
51.05
-0.33
SPR
(0.53) (1,27)
(0.47)
(43.74)
(1.24)
(6.72)
(1.06)
(0.49)
(1.76)
(1.30)
(1.12)
(10.31)
(1.61)
15.00
30,96
21.54
55.57
57.21
17.88
BMM
(0.68) (1,91) (3.16)
(44.77)
(6.89)
(2.11)
(1.53)
(2.29)
(8.88)
(3.18)
(1.49)
(16.74)
(2.05)
DY-S
15.56
32.24
45.72
21.90
49.94
78.16
24.70
44.56
-3.1
(2.22)
(3.26)
(4.14)
(33.46)
(0.68)
(4.87)
(0.96)
(1.58)
(0.26)
(0.77)
(2.23)
(10.04)
(0.86)
C-Mixup
14.80
27.17
41.95
27.74
50.87
76.79
21.92
47.04
-526
(0.74) (0.16) (0.77)
(38.53)
(2.29)
(7.88)
(0.25)
(0.46)
(1.28)
(0.86)
(0.96)
(10.33)
(0.52)
RDI
21.80
39.32
47.07
37.33
29.33
55.91
51.35
1.06
(0.41)
(0.66)
(0.76)
(49.28)
(1.51)
(4.41)
(1.09)
(1.98)
(0.60)
(0.69)
(1.75)
(12,37)
(0.67)
32.19
54.75
40.39
CDR
(1.20) (1.03) (0.66)
(45.06)
(0.41)
(6.15)
(1.11)
(1.54)
(1.72)
(0.89)
(2.89)
(7.38)
(1.28)
D2L.
20.75
55.54
(0.41) (2,51) (0.29)
(45.37)
(1.01)
(5.73)
(1.34)
(2.21)
(0.96)
(1.00)
(3.73)
(9.20)
(0.82)
AUX
$2.33
(0.17) (0.71)
(0.50)
(44.84)
(1.70)
(4.81)
(0.66)
(1.35)
(0.82)
(1.08)
(1.78)
(9.86)
(0.72)
Selfie
(4.09)
(3.42)
(0.48)
(42.55)
(5.90)
(4.93)
(15.12)
(2.07)
(0.08)
(0.28)
(5.42)
(3.46)
(11.45)
Ce-Sel
23.52
50.07
(2,66)
(1.03)
(1.33)
(35.29)
(6:05)
(15.38)
(12.18)
(11.39)
(3.08)
(0.59)
(8.15)
(6.99)
(5.19))
Superloss
67.11
(2.02)
(1.38)
(1.64)
(40.38)
(1.09)
(5.80)
(4.49)
(3.14)
(1.88)
(1.66)
(16.22)
(8.88)
(1.46)
ConFrag
2.74 8.16
15.91
34.42
17.49
27.31
5.08
12.64
27.26
61.24
15.70
33.36
-3,06
(0.95)
(0.45)
(0.39)
(22.14)
(1.12)
(5.31)
(0.36)
(1.94)
(2.25)
(1.42)
(1.43)
(10.14)
(1.25)
Co-ConFrag
0.54
7.25
16.65
33.93
17:43
28.26
1.50
9.45
28.44
61.36
14.87
35,88
-8.86
(0.77) (0.59)
(0.14)
(18.48)
(0.99)
(2.45)
(0.71)
(0.62)
(3.09)
(3.14)
(0.23)
(11.44)
(0.83)
39

---

Table 13: Mean Relative Absolute Error (%) and its standard deviation the noise-free trained model on the SHIFTI5M-B and MSD-B datasets. Lower is better. A negative value indicates it performs even better than the noise-free model. The results are the mean three random seed experiments. Number in parenthesis indicates standard deviation. The best and the second best methods are respectively marked in red and blue. CNLCU-S/H, Co-Selfie, and Co-ConFrag use dual networks teach each other as done in Han al. [2018). SPR (Wang al. 2022 fails for SHIFT15M-B due to excessive memory consumption.
SHIFTISM-B
MSD-B
symmetric
Gaussian
symmetric
Gaussian
noise rate
20
40
60
80
30
50
20
40
60
80
30
50
9.11
17.96
27.02
36.34
6.54
15.16
8.23
18.43
31.67
45.85
6.96
15.74
Vanilla
(0.56)
(1,50)
(0.96)
(0.08)
(1.11)
(0.90)
(0.18)
(2.47)
(3.51)
(0.36)
(0.65)
(3.03)
12.98
19.42
24.31
34.47
15.33
20.90
0.13
6.04
21.52
46.01
4.75
12.51
CNLCU-S
(0.15)
(0.39)
(0.70)
(0.17)
(1.09)
(0.34)
(1.18)
(0.31)
(3.61)
(1.51)
(0.92)
(1.08)
6.26
12.84
20.04
36.03
8.88
15.65
0.27
4.98
10.32
29.83
5.11
9.22
CNLCU-H
(0.43)
(0.55)
(0.42)
(0.80)
(0.36)
(0.19)
(0.19)
(0.94)
(1.46)
(1.33)
(0.31)
(0.39)
6.94
14.09
26.08
37.03
10.32
17.44
1.29
7.19
17.35
50.87
6.80
12.38
Sigua
(0.13)
(0.17)
(0.29)
(2.84)
(0.77)
(0.70)
(0.21)
(0.64)
(3.30)
(3.33)
(1.44)
(0.10)
7.07
18.19
33.39
45.61
5.01
15.36
SPR
(-)
(-)
(1.30)
(1.32)
(3.17)
(2.11)
(0.31)
(2.15)
6.96
12.42
18.64
26.79
7.58
13.13
3.32
10.30
23.40
43.56
5.29
11.85
BMM
(0.52)
(1.16)
(0.78)
(1.04)
(1.24)
(0.55)
(0.64)
(1.85)
(2.31)
(1.95)
(0.63)
(0.77)
7.11
11.94
18.85
29.04
6.90
13.50
3.39
18.65
35.24
4.77
9.83
DY-S
(0.25)
(0.74)
(0.10)
(1.48)
(0.97)
(0.95)
(1.07)
(1.19)
(3.50)
(1.83)
(1.28)
(1.33)
C-Mixup
16.15
24.08
34.17
5.88
14.51
3.75
13.13
26.73
40.90
2.96
10.97
(0.23)
(0.83)
(0.32)
(0.40)
(1.03)
(0.98)
(0.83)
(2.34)
(2.16)
(2.07)
(0.17)
(0.38)
9.91
17.92
26.63
36.29
7.08
15.18
21.04
30.09
38.78
49.49
19.19
27.88
RDI
(0.45)
(0.27) (0.36)
(0.67)
(0.85)
(0.84)
(0.09)
(0.22)
(0.98)
(1.12)
(0.69)
(1.60)
9.52
17.78
26.97
35.97
7.14
15.17
7.83
17.86
32.83
45.91
6.73
16.92
CDR
(1.17) (0.83)
(0.36)
(0.73)
(1.01)
(0.72)
(0.91)
(3.23)
(2.36)
(0.74)
(0:49)
(1.55)
9.25
18.03
26.55
36.23
6.34
15.60
7.13
19.96
32.47
46.64
5.51
15.54
D2L
(0.26)
(1.47)
(1.13)
(0.66)
(0.69)
(1.58)
(0.37)
(1.08)
(2.21)
(2.56)
(0.76)
(2.05)
7.74
16.95
26.61
36.47
4.92
14.40
6.12
18.18
31.09
45.70
5.21
15.45
AUX
(0.33)
(1.03)
(0.30)
(0.50)
(1.11)
(0.94)
(0.88)
(1.55)
(3.07)
(1.43)
(0.28)
(1.78)
4.84
10.22
22,28
38.15
5.51
11.58
1.43
8.40
20.24
45.87
14.37
24.13
Selfie
(0.77)
(0.71) (2.82)
(0.42)
(0.97)
(0.45)
(0.24)
(1.30)
(4.61)
(2.88)
(3.28)
(3.41)
11.53
16.43
32.08
39.32
13.45
22.33
-0.38
4.41
8.32
35.47
6.78
13.15
Co-Selfie
(0.84)
(0.62) (0.64)
(0.54)
(0.74)
(0.85)
(0.12)
(0.68)
(1.40)
(0.57)
(1.70)
(1.60)
5.44
12.26
23.23
35.24
5.60
13.28
-0.15
10.68
23.15
45.55
4.35
16.36
Superloss
(1,03)
(1.48)
(1.89)
(0.28)
(1.28)
(0.67)
(0.29)
(2.10)
(3.15)
(6.77)
(0.74)
(2.99)
2,46
6.18
10.68
19.04
3.66
8.09
0.57
4.94
11.22
23.41
2.39
6.49
ConFrag
(0.42)
(0.45)
(0.65)
(0.63)
(0.37)
(0.05)
(0.43)
(0.34)
(1.38)
(2.00)
(0.84)
(1.90)
0.85
5.52
10.80
18.83
3.03
8.70
-0.65
2.98
8.66
20.53
1.73
6.00
Co-ConFrag
(0.31) (0.66)
(0.43)
(0.41)
(0.94)
(0.46)
(0.72)
(0.66)
(0.36)
(2.46)
(1.02)
(1.07)
40

---

Table 14: Standard Mean Absolute Error and its standard deviation the noise-free trained model on the AFAD-B, IMDB-Clean-B and IMDB-WI WIKI-B datasets. Lower is better. The results are the mean of three random seed experiments. Number in parenthesis indicates standard deviation. The best and the second best methods are respectively marked in red and blue. CNLCU-S/H, Co-Selfie, and Co-ConFrag use dual networks to teach each other as done in Han et al. 2018
AFAD-B
IMDB-Clean-B
IMDB IMDB-WIKI-B WIKI-B
symmetric
Gaussian
symmetric
Gaussian
real noise
noise rate
20
40
60
80
30
50
20
40
60
80
30
50
Vanilla
4.75
5.22
5.68
6.22
5.59
6.04
8.11
9.22
10.70
12.32
8.86
10.50
7.23
(0.02)
(0.03)
(0.05)
(0.06)
(0.08)
(0.18)
(0.09)
(0.05)
(0.12)
(0.06)
(0.13)
(0.60)
(0.09)
CNLCU-S
4.82
5.23
5.75
6.17
5.67
6.11
10.57
11.64
12.77
12.97
12.81
12.72
8,78
(0.14)
(0.01)
(0.09)
(0.07)
(0.22)
(0.15)
(0.20)
(0.15)
(0.04)
(0.78)
(0.25)
(0.16)
CNLCU-H
4.55
5.05
5,91
6,29
5.89
6.24
7.46
9.16
11.39
12.76
10.24
11.54
7,01
(0.05)
(0.05)
(0.16) (0.08)
(0.15)
(0.11)
(0.08)
(0.12)
(0.14)
(0.18)
(0.43)
(0.47)
(0.02)
4.60
5:26
6,23
6,50
6.19
6.35
7.67
10.21
12.40
12.96
11.25
12.39
7.37
Sigua
(0.07)
(0.10) (0.07)
(0.04)
(0.17) (0.17)
(0.04)
(0.67)
(0.08)
(0.04)
(1.37)
(0.19)
(0.12)
SPR
4.77
5.16
5.67
6,22
5,58
6.07
8.00
9.25
10.82
12.53
8.78
10.55
7.16
(0.04) (0.04) (0.03)
(0.06)
(0.07) (0.28)
(0.09)
(0.02)
(0.08)
(0.12)
(0.05)
(0.70)
(0.03)
BMM
4.59
5,00
6.04
6,36
5.69
6.26
7.60
8.49
10.87
12.60
8.68
10.98
8.52
(0.03) (0.09) (0.12)
(0.06)
(0.31) (0.09)
(0.11)
(0,20)
(0.68)
(0.21)
(0.12)
(1.16)
(0.13)
4.64
5.02
5.74
6.33
5.40
6.23
7.71
8,51
10.47
12.44
8.71
10.10
6.98
DY-S
(0.09) (0.12) (0.24)
(0.16)
(0.05)
(0.20)
(0.11)
(0.13)
(0.04)
(0.05)
(0.14)
(0.68)
(0.07)
C-Mixup
4,46
4,99
5.52
6.17
5.40
5.95
7.60
8,92
10.54
12.35
8.52
10.27
6.84
(0.04) (0.02) (0.05)
(0.07)
(0.12)
(0.34)
(0.06)
(0.04)
(0.06)
(0.12)
(0.04)
(0.69)
(0.04)
RDI
4.81
5.29
6.05
6.39
5.97
6.27
8.13
9.03
10.89
12.57
8.78
10.57
7,30
(0.02) (0.04)
(0.02)
(0.02)
(0.09)
(0.18)
(0.04)
(0.19)
(0.04)
(0.11)
(0.10)
(0.84)
(0.04)
4.79 5.16
5.75
6,23
5.64
6.05
8.20
9.23
10.81
12.25
8.97
10.60
7.20
CDR
(0.04) (0.03) (0.03)
(0.10)
(0.03)
(0.26)
(0.08)
(0.16)
(0.16)
(0.02)
(0.16)
(0.48)
(0.01)
4.75
5.24
5.70
6,28
5.60
6.09
8.17
9,35
10.86
12.31
9.03
10:65
7.18
D2L
(0.03) (0.10)
(0.03)
(0.11)
(0.03)
(0.24)
(0.05)
(0.11)
(0.09)
(0.01)
(0.21)
(0.62)
(0.12)
AUX
5.57
7,86
9.00
10.64
12.35
(0.02) (0.05)
(0.04)
(0,07)
(0.09)
(0.20)
(0.01)
(0.11)
(0.11)
(0.13)
(0.13)
(0.66)
(0.05)
Selfie
5.08
5.43
6.26
6.42
6.34
6.55
8.90
10.74
12.53
12.85
11.22
12.43
8.24
(0.19) (0.16)
(0.01)
(0.28)
(0.23)
(1.07)
(0.10)
(0.07)
(0.05)
(0.39)
Co-Selfie
4.98
5.34
6.07
6.42
6.13
6.65
8.63
10:48
11.69
12.87
10.65
12.20
7.20
(0.13)
(0.06)
(0.08)
(0.08)
(0.28)
(0.68)
(0.88)
(0.84)
(0.19)
(0.03)
(0.61)
(0.45)
(0.46)
Superloss
4.66
5.14
5.64
6.27
5.54
6.21
8.62
10.16
11.67
12.63
10.75
11.41
6.97
(0.07)
(0.05)
(0.05)
(0.05)
(0.05)
(0.25)
(0.27)
(0,25)
(0.19)
(0.15)
(1.14)
(0.57)
(0.05)
ConFrag
4,46
4.70
5.04
5.84
5.10
5.53
7.34
7.87
8.89
11.26
8.08
9.31
7,00
(0.04)
(0.02)
(0.03)
(0.05)
(0.06)
(0.22)
(0.04)
(0.15)
(0.12)
(0.04)
(0.07)
(0.68)
(0.13)
Co-ConFrag
5.57
7.09
7.64
8.97
11.27
8.02
9.49
(0.05)
(0.04)
(0.01)
(0.09)
(0.06)
(0.11)
(0.08)
(0.05)
(0.20)
(0.16)
(0.05)
(0.76)
(0.06)
41

---

Table 15: Standard Mean Absolute Error and its standard deviation to the noise-free trained model on the CHIFTI5M-B and MSD-B datasets. Lower is better. The results are the mean of three random seed experiments. Number in parenthesis indicates standard deviation. The best and the second best methods are respectively marked in red and blue. CNLCU-S/HL Co-Selfie, and Co-ConFrag use dual networks to teach each other as done in Han et al.] (2018). SPR [Wang et al. 2022 fails to run for SHIFT15M-B due excessive memory consumption.
1HIFT15M-B
MSD-B
symmetric
Gaussian
symmetric
Gaussian
noise rate
20
80
50
40
60
80
30
50
7.47
8.08
8.70
9.34
7.30
7.89
.5918
.6475
.7199
.7974
.5848
.6328
Vanilla
(0.02)
(0.07)
(0.08)
(0.04)
(0.05)
(0.08)
(.0024)
(.0112)
(.0171)
(0001))
(.0031)
(.016))
7.74
8.18
8.51
9.21
7.90
8.28
.5475
.5798
.6644
.7983
.5727
.6511
CNLCU-S
(0.03)
(0.04)
(0.01)
(0.05)
(0.04)
(0.02)
(.0068)
(,0010)
(.016))
(.0052)
(.0033)
(.0035)
7.28
7.73
8.22
9.32
7.46
7.92
.5483
.5740
.6032
.7098
.5747
.5972
CNLCU-H
(0.03)
(0.04) (0.01)
(0.05)
(0.01)
(0.03)
(.0034)
(.0027)
(.0055)
(C0065)
(0004)
(0019)
7.32
7.81
8.64
9.39
7.56
8.04
.5588
.5861
.6416
.2488
.5899
.6145
Sigua
(0.03)
(0.03)
(0.06)
(0.24)
(0.02)
(0.08)
(0005)
(.0024)
(.014))
(0150)
(.0062)
(0026)
.8544
.6462
.7293
.7961
.5741
.6308
SPR
(-)
(.0059)
(.0048)
(.0147)
(0113)
(.0028)
(01119)
7.33
7.70
8.13
8.68
7.37
7.75
.5649
.6031
.6747
.7849
.5757
16116
BMM
(0.03)
(0.05)
(0.03)
(0.08)
(0.06)
(0.07)
(.0044)
(.0081)
(.0099)
(.0033)
(.0016)
(.025))
7.34
7.67
8.14
8.84
7.32
7.77
.653
.5908
.6487
.7394
.5728
6005
DY-S
(0.02)
(0.03)
(0.04)
(0.10)
(0.03)
(0.10)
(.0072)
(.0040)
(.0164)
(.0102)
(.0045)
(.009))
C-Mixup
7,50
7.95
8.50
9.19
7.25
7.84
5673
.6885
.6929
.7704
5630
6067
(0.03)
(0.03)
(0.03) (0.05)
(0.04)
(0.08)
(.004))
(.1107)
(,009)
(0135)
(0023)
(0028)
7,53
8.08
8.67
9.33
7.33
7.89
6618
.7113
.7588
8174
6517
6992
RDI
(0.02)
(0.02) (0.05) (0.03)
(0.02)
(0.08)
(.000))
(.0040)
(,0050)
(0003)
(0005))
(0092)
7.50
8.07
8.70
9.31
7.34
7.89
5896
.6444
.7262
.7978
5836
.6393
CDR
(0.05)
(0.03) (0.05) (0.02)
(0.04)
(0.08)
(.005))
(.0160)
(.0116)
(.0062)
(0003)
(0089)
7.48
8.08
8.67
9.33
7.28
7.92
5857
.6559
.7243
8018
.5769
.6317
D2L
(0.03) (0.07) (0.07) (0.01)
(0.01)
(0.13)
(Q021)
(.0049)
(0016)
(0112) (.0030)
(0106)
7.38
8.01
8.67
9.35
7.19
7.83
5802
.6462
.7167
.7966
.5753
.6112
AUX
(0.01) (0.04)
(0.04)
(0.04)
(0.04)
(0.08)
(.0027)
(,0097)
(.0152)
(.0067)
(.0010)
(0089)
7.18
7.55
8.37
9.46
7.23
7.64
5546
.927
.6574
.7976
.653
6787
Selfie
(0.03)
(0.03)
(0.19) (0.02)
(0.04)
(0.05)
(.0020)
(.0046)
(.0230)
(.1174)
(.0190)
(0115)
Co-Selfie
7.64
7.97
9.05
9.54
7.77
8,38
5447
.7099
5923
.7407
.5399
.6887
(0.03)
(0.01)
(0.01) (0.01)
(0.02)
(0.08)
(.0026)
(.0019)
(.0088)
(.0023)
(.0105)
(.084)
Superloss
7.22
7.69
8.44
9.26
7.23
7.76
.4600
.6052
.6733
.7959
.5706
.6362
(0.06)
(0.07)
(0.09)
(0.06)
(0.06)
(0.06)
((0036)
(.0104)
(0153)
(.0405)
(00055)
(0140)
7.02
7.27
7.58
8.15
7.10
7.40
$4999
.5338
.6081
.6477
.5988
.822
ConFrag
(0.01)
(0.02)
(0.01)
(0.03)
(0.01)
(0.04)
(.0035)
(.0039)
(.0051)
(.0098)
(.0050)
(0084)
6.91
7.23
7.59
8.14
7.06
7.44
.532
.5631
.941
.590
.5622
S796
Co-ConFrag
(0.01) (0.01)
(0.01)
(0.06)
(0.04)
(0.06)
(0055))
(.0018)
(.0009)
(.0129)
(.0051)
(0044)
42

---

Algorithm 1 Contrastive Fragmentation
Input: Train data D {X, Y}, Fragment number F, KNN parameter K, Jitter
Total epochs N
θ (0..0... ),,) [feature extractors)
Φ RandomI Init() (regression model)
D I...F = Pragmentation(D) 1$ 2.1
P ContrastivePuiring ID... 2.1 2-4) for to N do
train feature extractors pittte
htwebbbboooo/ttttrinng./o for (Deither Djitter) in pjitter do
(neighborhood jittering ($ 2.40
Dijtee =Dj Diitter U Dijitter
train p(f: Dister)
end for
initialize S, S, S =1. 0.0 (selected samples)
# obtain SP for (x, y) in D do
for f to F do
calculate Py(3) (fragment prior (Eq. 2
# use two types classification for neighborhood agreement
calculate a² DI...F, θ) (predictive neighborhood agreement (Eq. 4h)
calculate 0² (x;; D Di...F, θ) (representational neighborhood agreement (Eq.
end for
p (s(5, DI...F; () :) Σ p)(W) DI...F, Q) (pred. sample probability (Eq.
p' (s(,,, DI... -)=0==10 O). (x(1)) DI...F O) (repr. sample probability (Eq.
sample {WP, w²} ~ uniform(0,
if (s y, D r) n² then
S 35=5" U (x.
S"
end if
if p' y, DI...;; 9) > "" then =" U (2,))
end if end for
union filtered samples (SP,S") S = SP U S"
# train regression model
+ TrainOneEpoch((;
end for
43

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The contributions of the paper outlined in the introduction and match the experimental results of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the work discussed in Appendix B
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
44

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Detailed information required to reproduce the main experimental results are provided in $ 4.1 and Appendix
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
45

---

Answer: [Yes]
Justification:
We provide open access to code for reproducing the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
httpi///giiee///oo/eoee Code://odeeSuumm ssionE oollollcyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https) wttp:c///ppbcicgiiieeeeoceeeeeee /tt.pp/cc/ouuliic///////////ccc/
//nips cc/
CooeeeooeeSSae ubmi issssssss aesionPoolise
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental setting is presented in $ 4.1 with the full details provided in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Appendix G. contains main experimental results with standard deviation calculated over three random seeds, along with visualizations of the variance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
46

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Appendix F.3 provides the computation resource used and time of execution.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: All authors read and confirm that the research in the paper conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The broader societal impacts of the work is described in Appendix C
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
47

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper proposes general machine learning method and thus the paper poses no such risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper properly cites the original assets such datasets, and Appendix provides the licenses of existing dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paapeettsppprrssseeee paperswws thoode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
48

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The provided code includes documentation on training and dataset construc-
tion.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
49


Reviews:
[{"review_id": "i6ggEwbKxm", "reviewer": "Reviewer_T5Bx", "summary": "The paper introduces an innovative approach for the collective modeling of regression data, grounded in the idea that similar labels often correspond to shared significant features. The authors convert the data into separate, yet juxtaposed fragment pairs, employing a combination of adjacent fragments to detect noisy labels. This detection is achieved via a mutual agreement within the prediction and representation spaces.", "questions": "- If I understand correctly, you can use the method for generative tasks as well. Did you consider it? if it is not possible, can you please elaborate?\n- Did you study classification tasks? If yes, how did it perform?", "limitations": ".", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The method seems technically solid and correct\n- To the best of my knowledge, this paper presents a novel method that has not introduced in the past\n- The paper presents a robust evaluation setup", "weaknesses": "I think that the paper is overwhelmed with its presentation of too many details, which can move to the appendix to let the reader focus on the important parts. That being said, this is the author's decision, and I didn't consider it in my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W. Paper is overwhelmed with its presentation of too many details, which can move to the appendix to let up the reader focus on the important parts.**\n\nWe sincerely thank the reviewer for the suggestion. In order to balance the clarity and the detail, we will move some overly detailed procedure of contrastive fragment pairing in Section 2.1 (line 107-116) to Appendix, and make the motivation part (line 117-144) more concise. To improve clarity better, we will include a figure illustrating the overall framework of ConFrag, highlighting the online nature of the filtering and training processes.\n\n**Q1. Extension to generative tasks**\n\nConFrag learns to select samples $(x, y)$ that are better aligned and use them for training regression tasks (i.e., learning $P(y|x)$). As the reviewer suggested, the selected samples can also be used for conditional generation tasks (i.e., learning $P(x|y)$). \n\nTo verify this, we train a continuous conditional GAN model (SVDL+ILI) [4] for 40k steps using (1) clean IMDB-Clean-B dataset, (2) IMDB-Clean-B with 40% symmetric noise, and (3) samples selected by ConFrag on the noisy dataset. To measure the condition alignment and the quality of generated samples, we use MAE and intra-fragment FID on 52,000 generated images. Specifically, intra-fragment FID is the average of FID values measured for images in each fragment. We use $F=4$ as done in ConFrag. For both evaluation metrics, lower is better. The results show that sample selection by ConFrag improves both the condition alignment and the quality of conditionally generated images.\n\n|  | intra-fragment FID (F=4) | MAE |\n|---|:---:|:---:|\n| Clean (0% noise) | 14.42 | 10.419 |\n| Symmetric 40% noise | 15.998 | 13.569 |\n| ConFrag | 15.244 | 10.348 |\n\n\n**Q2. Extension to classification tasks**\n\nAs suggested by the reviewer, it is possible to apply our contrastive fragmentation approach to classification tasks by making the following adjustments:\n\n1. We set each class as a fragment (i.e., the number of fragments $F$ = the number of classes)\n2. We measure the distance between fragments/classes. Possible metrics include euclidean distance between the GloVe embedding, CLIP text cosine similarity of each class, or CLIP image similarity using the samples of each class. The distance metric is used for constructing contrastive pairs and defining two neighboring fragments for each fragment.\n3. We redefine the fragment prior (Equation 2) using the distance between classes. \n\nWe evaluated the extended ConFrag on CIFAR-10 classification with 20/40/60% symmetric noise ratios using selection ratio and precision as metrics, where precision is defined as P(clean|selected). We found that samples selected by the modified ConFrag are cleaner than the original dataset (i.e., precision higher than 1 - noise ratio), showing the potential of extending ConFrag to classification tasks. \n\n| Noise | Selection ratio | Precision |\n|---|:---:|:---:|\n| 20% | 0.8191 | 0.8871 |\n| 40% | 0.7353 | 0.7359 |\n| 60% | 0.6702 | 0.5243 |\n\nHowever, since ConFrag is designed to leverage the characteristics of noisy regression tasks as mentioned in the general response, and thus noisy classification is out of our main focus, its performance does not reach the level of state-of-the-art sample selection approaches designed for noisy classification tasks. Additionally, we found that the choice of the distance metric has a significant effect on sample selection performance, indicating that the design of a better distance metric is necessary for successful extension of our approach to noisy classification tasks.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E0iXDUZNHa", "reviewer": "Reviewer_1GFp", "summary": "This paper addresses the issue of label noise in regression tasks. The proposed method partitions the data and trains several binary classifiers for the most distant partition pairs. Noisy data samples are detected using the voted probability of all classifiers. The method outperforms other baselines on several public datasets across different domains, as measured by a newly proposed metric.", "questions": "In addition to addressing the weaknesses mentioned above, I hope the authors can answer these questions:\n\n1. Is there a systematic way to determine the optimal number of fragments?\n\n2. The paper states, \"If x is more likely to belong to fragment 2 than 5, then it should be more likely to belong to 1 than 4 and 3 than 6.\" Are there limitations to this statement? Would the proposed method still work if this relationship doesn't hold?", "limitations": "Yes, the limitations and broader impacts are addressed.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The intuition behind the method is sound. It aims to identify samples whose y values are most misaligned with their x values, based on the weighted combined opinion of all classifiers.", "weaknesses": "1. The focus on regression tasks is limiting. Given that the proposed method transforms regression tasks into classification tasks, it could potentially be extended to address data noise in classification tasks using the same intuition.\n\n2. The paper lacks a detailed discussion of the types of noise detected and not detected. It's unclear whether this method can detect close-set noise for individual classifiers or if it's more effective at identifying noisy samples at the center or boundaries of the fragments.\n\n3. There is no ablation study on the amount and type of noise. The paper doesn't address whether the method is effective for all noise ratios or explore the typical types and ratios of label noise in real-life scenarios. It's unclear how the method performs with varying levels of noise (e.g., 1%, 0.5%, 5% of noise data or noise strength).\n\n4. The paper doesn't consider scenarios where the noise degree is so small that most noisy samples cannot be transferred from closed-set to open-set noise. It's unclear whether increasing the number of fragments would be beneficial in such cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1. Focus on regression is limiting. ConFrag transforms regression tasks into classification tasks; it could potentially be extended to address noise in classification**\n\nWe believe that noisy regression is an important task on its own! However, most noisy label learning research focuses on classification tasks. Regression assumes a continuously ordered relationship between features and labels, a premise well-supported in existing research [5, 6, 1, 7], whereas classification is categorical rather than ordered. Our experiments demonstrate that many noisy classification approaches do not perform well in noisy regression tasks mainly due to the fundamental difference between regression and classification.\nWe employ cross-entropy loss (classification) as a surrogate objective to enhance the robust feature extraction, which in turn improves sample selection used to solve the regression task. This approach has been both theoretically and empirically validated thanks to better feature learning [8], as detailed in Appendix D.1. Table 2 compares regression-trained experts (ConFrag-R) and classification-trained experts (ConFrag), showing the superiority of using surrogate classification objectives. \nThe potential for extending our approach to classification exists, but it is out-of-scope of this work.\n\n**W2. Detected noise analysis, specifically, close-set noise detection and noise in the center or boundaries of the fragments.**\n  \n**Close-set noise detection.** Fig R1 shows the selection rate of closed-set noise on the IMDB-Clean-B dataset with symmetric 40% noise. As training progresses, they become less likely to be selected, showing ConFrag’s ability to detect closed-set noise.\n\n**Boundary, Centre noise.** We compare the selection rate and the error reduction rate (ERR) between the samples at the boundary and center of fragments. Table R1 shows that the average difference of the selection rates and ERR between two groups is 2.29% and 2.43%, respectively. These results confirm that ConFrag consistently performs robust sample selection regardless of the sample's position.\n\n**[W3,4] Do additional ablation on the amount and strength of noise (with emphasis on 1%, 0.5%, 5% noise) and also evaluate on real-life noise. unclear whether increasing the number of fragments would be beneficial in extreme small noise.**\n\nWe perform thorough experiments on the diverse amount and strength of noise following previous research on noisy regression [1, 9, 10, 11]. It involves the ablation on symmetric noise levels of 20/40/60/80% and Gaussian noise with a standard deviation of 30/50%. Notably, we are the only work that tests on both symmetric and Gaussian noise! The results presented in Section 4.3 show that ConFrag is effective across all of these noise ratios and strengths.\n\nIn Table R2, we analyze ConFrag using tiny noise amounts (0.5/1/5% symmetric noise) and strengths (Gaussian 2/5%) on IMDB-Clean-B. \n\nFor noise ratios of 0.5/1/5%, ConFrag achieves a very low ERR, indicating that it can detect noisy samples regardless of the noise ratio. However, when the noise strength is very small (Gaussian 2/5%), ConFrag is not able to filter them effectively. We acknowledge this as a limitation of our method. \nSince ConFrag primarily focuses on sample selection, it can be easily reinforced by other techniques during the training process to mitigate the effects of tiny noise strengths. Indeed, in the Gaussian 2/5% experiments, integrating C-Mixup or Co-teaching with our ConFrag improves the performance by 4.74% and 4.51%, respectively. Regarding the reviewer’s question on whether increasing the number of fragments can be beneficial when the noise strength is small, please refer to our answer to Q1.\nLast but most importantly, we evaluate the performance of ConFrag on real-world noise, which includes many types of noise with varying strengths. Table R4 reports the results on a version of the IMDB dataset with real-world noise [3], IMDB-org-B. The results show that the vanilla version of our method performs on par with other best-performing baselines on real-world datasets. Importantly, since ConFrag only concerns the sample selection process, it can be easily integrated with other noisy label methods. By combining our approach with other techniques, ConFrag outperforms all baselines by a non-trivial margin. Demonstrating the practical effectiveness of our approach.\n\n**Q1. A systematic way to determine the fragment number?**\n\nAnalysis in Appendix G.2 shows that fixing the fragment number $F$ to 4 yields the best performance for the IMDB-Clean-B and SHIFT15M-B datasets, as it is a non-sensitive hyperparameter to the performance. This setting was subsequently used in all our experiments. The analysis further shows that given a large enough dataset, the sensitivity to the number of fragments decreases.\n\nFiner fragmentation can improve detection of tiny strength noise but increases the risk of overfitting due to fewer samples per task. In larger datasets, this risk is mitigated because the data size per fragment remains large enough, allowing ConFrag to benefit from finer fragmentation. Based on these insights, we recommend starting with $F=4$ and incrementally increasing it until performance deteriorates, especially for larger datasets.\n\n**Q2. Limitations to the statement, \"If x is more likely to belong to fragment 2 than 5, then it should be more likely to belong to 1 than 4 and 3 than 6.\"? Can it hold without this relationship?**\n\nThe statement is highly related to the fundamental characteristic of regression: the continuous and ordered correlation between the label and feature space [5,6,1,7]. Since our approach is rooted in this characteristic, its effectiveness can be compromised if it does not hold. However, we reemphasize that this is a common characteristic found in most regression tasks, and prior works on regression with imperfect data build their methods on top of this characteristic [5,6,1].", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gYUTBB8tAX", "reviewer": "Reviewer_wYcT", "summary": "This paper aims at addressing noisy labels in real-world regression problems and propose the ConFrag method. ConFrag transforms regression data into disjoint fragmentation pairs to enhance the distinction between clean and noisy samples. It leverages neighboring fragments and expert feature extractors to identify noisy labels through neighborhood agreement. The approach is validated through experiments on diverse datasets and introduces a new metric, Error Residual Ratio (ERR), which shows its consistent superiority over fourteen existing methods.", "questions": "Since contrastive fragment pairing transforms some closed-set label noise into open ones, could this nature be considered when designing the data selection metric to choose open noise labels?", "limitations": "Adequately addressed", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1.\tThis method leverages the inherent orderly relationship between the label and feature space to identify noisy labels.\n2.\tFour newly curated benchmark datasets and a metric are constructed for conducting experiments.", "weaknesses": "1.\tThe motivation behind the idea should be stated more clearly in the introduction section. Since there is a close connection between labels and features, it is necessary to clarify why contrastive fragment pairing is introduced and whether the design of the data selection metric considers the connection.\n2.\tCertain statements in the paper require further clarity. For instance, the meaning of \"f\" in \"all of the noisy labeled data (f…)\" needs explicit clarification upon first mention, although the meaning of the symbol can be inferred in subsequent discussions. Additionally, the phrase \"we employ neighboring relations in fragments to leverage the collective information learned\" in the introduction warrants elaboration.\n3.\tIt is novel to create four curated benchmarks. However, it is recommended to also validate your method on the existing noisy regression datasets used in other papers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1. The motivation behind the idea should be stated more clearly in the introduction section. Since there is a close connection between labels and features, it is necessary to clarify why contrastive fragment pairing is introduced and whether the design of the data selection metric considers the connection.**\n\nWe appreciate the feedback and suggestions! To clarify the motivation behind our idea, we will elaborate on the rationale for contrastive fragment pairing. Our method addresses the challenges of learning from noisy data by employing distinctive feature matching, which improves generalization [12,13]. Additionally, this approach helps convert closed-set noise into open-set noise, which is less harmful to the learning process [14].\nWe will also explain that our data selection approach, the mixture of neighboring fragments (Section 2.3), considers the correlation between labels and features that the samples with close label distances are likely to have similar features. This correlation is essential for effective fragmentation and sample selection using the mixture of neighboring fragments.\n\n**W2. Certain statements in the paper require further clarity. For instance, the meaning of \"f\" in \"all of the noisy labeled data (f…)\" needs explicit clarification upon first mention, although the meaning of the symbol can be inferred in subsequent discussions. Additionally, the phrase \"we employ neighboring relations in fragments to leverage the collective information learned\" in the introduction warrants elaboration.**\n\nWe will make sure to include that $f$ means the index of the fragment that the data is assigned to upon the first mention! We will also elaborate the phrase \"we employ neighboring relations in fragments to leverage the collective information learned,\" explaining that it utilizes a mixture model [15] to achieve probabilistic consensus in both prediction and representation spaces.\n\n**W3. The performance on existing noisy regression datasets?**\n\nWe appreciate your recognition of our four curated benchmarks, covering age, price, and music production year prediction tasks. As suggested by the reviewer, we further evaluate the performance of our approach and baselines on two noisy regression datasets from existing noisy regression literature. The first dataset is from SuperLoss [9, 11], which injects 0.2, 0.4, 0.6, and 0.8 symmetric noise into the UTKFace dataset. The second dataset is the IMDB-org-B dataset, a real-world noise dataset studied in [16, 3, 17]. The results can be found in Table R4. We perform on par with or better than the baselines on both datasets as shown in the table below.\n\n**Q1. Could the nature of closed-set noise transformation into open-set noise be used advantageously during the data selection metric?**\n\nThank you for the insightful question. As illustrated in Fig. 2(a), transforming closed-set noise into open-set noise can also be seen as converting it into anomalies or outliers. By leveraging the extensive literature on anomaly and outlier detection techniques, we could effectively handle open-set noise to enhance the data selection process.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gXlfHMjc6E", "reviewer": "Reviewer_2i51", "summary": "This paper targets the noisy label regression problem. Inspired by the classification loss which helps get good representation, they first propose to discretise continuous label space into pieces which thus divides data into disjoint fragments. Then they look at the F/2 maximally contrasting fragment pairs on which the binary classifiers are built. The main learning objective is MoE where a neighborhood agreement is applied to enhance the decision consistency. And the representation and prediction are used to select clean samples and then train the regressor.", "questions": "1. When illustrating the motivation of contrastive fragment paring, you pointed out the advantages could be better generalization and robustness. Regarding the generalization, were you referring to its performance on clean-label data training? Then is Fig. 2(c) on noisy or clean data? I felt confused when I read this part. \n2. It seems that the proposed approach to construct F/2 maximally contrasting pairs of fragments ignored the order relationship between fragments that has been thought crucial to regression tasks [Yang et al., 2022b]. Can you provide some evidence if you do not think so?\n3. The ratio of noise labels is important according to literature. Since the proposed method does not mention it in methodology, were you suggesting that the proposed method could be noisy ratio unaware and applicable to any level of noisy ratio? \n4. Clean samples selection strategy from line 185-188 looks intuitive. You can certainly consider two views somehow complementary when necessary, but your methodology did not touch the representation learning.\n5. Zhang et al., 2023 in line 91 on page 3 seems a strong baseline, which compute distance on representation. May I know why it is not included?", "limitations": "My major concern is that the proposed pair construction strategy is interesting but not well motivated by noisy label topic, which makes me doubt which components contribute much to this research problem. Also the connection with recent work lack deep discussion.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The presentation is overall good and the idea of constructing maximally contrasting pairs is interesting.\n2. The fundamental works in regression have been discussed and many of them are compared detailedly.\n3. I think MoE is also a good choice to integrate all classification learners.", "weaknesses": "1. Some illustration is not clear for me.  \n2. Some baselines which takes the order information should be discussed or compared, because binary classification ignores such information which however was thought important in previous regression works.\n3.The technical contributions and novelty are not well highlighted in the main body of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[Limitations, Q2]. Pair construction strategy is interesting but not well motivated by the noisy label topic casting concerns regarding contribution. The maximally contrasting pairs of fragments ignores order relation.**\n\nWe would like to clarify that ConFrag is strongly motivated by the topic of noisy label regression. \n\n**Focus on noisy label regression.** A method tailored for noisy regression problems must primarily focus on detecting noise with high severity, rather than treating all noise as equal. The components of our method such as maximally contrasting fragment pairing and fragment prior, which are based on the distance in continuous and ordered label space, are designed to prioritize filtering out such high severity noise. \n\n**Order Relation.** Our framework leverages ordinal relations by training on contrastive fragmented pairs to learn robust features. These features are then aggregated and ordered. Clean sample selection is done through the mixture of neighboring fragments which ensures the integrity of the learned order relations. \n\n**W2. Discuss or compare baselines considering ordinality since binary classification ignores ordinality which however was thought important in previous regression works + Limitations state that “the connection with recent work lacks deep discussion.”.**\n\nWe discuss additional works that consider order information in Appendix E.1. Furthermore, we evaluate noisy regression baselines that account for ordinality in the introduction (ln 35-42), specifically C-Mixup [1] and OrdRegr [10]. OrdRegr is a noise transition matrix based loss correction method which requires noise rate estimation. Even when provided with ground truth noise in synthetic settings, the algorithm was highly ineffective. For example, in IMDB symmetric 40% noise experiments, OrdRegr performed at least 31.86% worse than other baselines in terms of MRAE. Therefore although we have implemented both methods, we only report C-Mixup results. We will ensure that OrdRegr results are included in the manuscript as well.\n\nTo the best of our knowledge, the most recent published related work is \"Robust Classification via Regression for Learning with Noisy Labels.\" which improves the classification by reformulating it as a regression problem. We will include this to references. If there are any other works we have overlooked, kindly let us know and we will make sure to review and include it.\n\nAdditionally, we would like to reemphasize that our framework utilizes ordinal relations by training binary classifiers on contrastive fragmented pairs to develop robust features. These features are then aggregated and ordered. Clean sample selection is performed through the mixture of neighboring fragments, ensuring the preservation of the learned order relations.\n\n**Q1. You pointed out the advantages could be better generalization and robustness. Regarding the generalization, were you referring to its performance on clean-label data training? Then is Fig. 2(c) on noisy or clean data?**\n\nGeneralization in the motivation of contrastive fragment pairing refers to the generalization of feature extractors trained on noisy datasets, as obtaining robust and generalizable features is crucial for sample selection in noisy label settings. Fig. 2(c) shows that under symmetric 40% label noise, training expert feature extractors on contrastive fragment pairs is better for sample selection (and thus better regression performance) than training a single feature extractor on all fragment because they are less prone to overfitting and learn more generalizable features. We will also make it clear that the results in Fig. 2(c) are on noisy data.\n\n**Q3. Is the proposed method noisy ratio unaware and applicable to any level of noisy ratio?**\n\nThe reviewer correctly noted that knowing the ratio of noisy labels beforehand is beneficial but challenging to estimate in practice. We address this point at the start of Section 2, line 86, by stating, 'ConFrag is noise rate-agnostic unlike prior methods as it operates without knowing a pre-defined noise rate.' This highlights that our method does not require prior knowledge of the noise ratio.\n\n**Q4. Clean samples selection strategy from line 185-188 looks intuitive. You can certainly consider two views somehow complementary when necessary, but your methodology did not touch the representation learning.**\n\nWe use the term 'representation' to indicate that our fragment pair trained binary classifiers automatically learn data representations. This means they can be considered representational learners or feature extractors. However, we acknowledge that 'representation learning' often refers to techniques like self-supervised learning. We will clarify this distinction in the manuscript.\n\n**Q5. Zhang et al., 2023 in line 91 on page 3 seems a strong baseline, which compute distance on representation. May I know why it is not included?**\n\nOrdinalEntropy [8] proposes a regularizer that learns higher entropy features by increasing the distance between representations while preserving the ordinality via weighting the representation and target space distances. Since it doesn’t tackle the noisy regression problem directly on its own, it was not included as a baseline in the manuscript. However, it is surely something we can try to mix together with other baselines as well as our method to enhance the ordinality and to better learn the high-entropy features. \nResults in Table R3 show that combining OrdinalEntropy with vanilla, baselines, and our method shows a slight drop in performance. \n\n**W1. some unclear illustrations**\n\nwe will enhance Figure 2(a) to aid the understanding. If there are any other illustrations that require attention we would be happy to revise!\n\n**W3. clearly state technical contributions**\n\nWe will highlight the technical contributions and novelty better in the manuscript, especially within the introduction.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Global Response

We thank the reviewers for their insightful comments and acknowledgment. We appreciate the recognition that our approach is technically solid and correct (Reviewer T5Bx), our presentation is commendable (Reviewer T5Bx, 2i51), and our extensive comparison of regression works is thorough (Reviewer 2i51). Additionally, we value the positive feedback on our robust evaluation setup and metrics (Reviewer T5Bx, RwYcT) and the inclusion of four additional benchmark datasets (Reviewer wYcT). We are grateful to all four reviewers for recognizing the novelty, soundness, intuition, and conceptual strength of our approach.

As a global response, we would like to highlight the following.

## Characteristics of Noisy Regression
Noisy regression problems have two distinct characteristics that distinguish them from noisy classification tasks: continuously ordered correlation between labels and features and varying degrees of noise strengths. We emphasize that these two characteristics are utilized for designing the key components of ConFrag.

**Ordered relations** 

ConFrag leverages ordinal relations by training on contrastive fragmented pairs to learn robust features. These features are then aggregated and ordered. Clean sample selection is done through the mixture of neighboring fragments which ensures the integrity of the learned order relations.

**Focus on noisy label regression** 

A method tailored for noisy regression problems must primarily focus on detecting noise with high severity, rather than treating all noise as equal. The components of our method such as maximally contrasting fragment pairing and fragment prior, which are based on the distance in continuous and ordered label space, are designed to prioritize filtering out such high severity noise. 

## Additional Experiments
In this rebuttal, we present the results of ConFrag on two additional noisy label regression datasets suggested by Reviewer RwYcT. Please refer to W3.

**Additional Real-World Dataset**

We’d like to emphasize that ConFrag deals with the sample selection process, and thus it can be easily integrated with other noisy label methods. Our approach combined with regularization methods such as C-Mixup [1] and Co-Teaching [2] significantly outperforms all baselines on the IMDB-Org-B [3] dataset. Specifically, our method achieves improvements of 3.3% and 4.82% in Mean Relative Absolute Error (MRAE), respectively. This result demonstrates the practical effectiveness of our approach. 


We attach a pdf file containing figures and tables. For figures and tables whose index starts with ‘R’, please refer to the attached file.

**Global References**

[1] Yao et al., C-mixup: Improving generalization in regression. ICML 2022

[2] Han et al., Co-teaching: Robust training of deep neural networks with extremely noisy labels. NeurIPS 2018

[3] Lin et al., FP-Age: Leveraging Face Parsing Attention for Facial Age Estimation in the Wild. TIP 2022

[4] Din et al., Continuous Conditional Generative Adversarial Networks: Novel Empirical Losses and Label Input Mechanisms. TPAMI 2023

[5] Yang, et al. Delving into deep imbalance regression. ICML 2022

[6] Gong, et al. Ranksim: Ranking similarity regularization for deep imbalanced regression. ICML 2022

[7] Zha, et al. Supervised contrastive regression. 2022

[8] Zhang, et al. Improving deep regression with ordinal entropy. ICLR 2023

[9] Castells et al., SuperLoss: A Generic Loss for Robust Curriculum Learning. NeurIPS 2020.

[10] Garg and Manwani. Robust deep ordinal regression under label noise. ACML 2020

[11] Wu et al., discrimLoss: A Universal Loss for Hard Samples and Incorrect Samples Discrimination. TMM 2024.

[12] Grønlund et al., Margin-based generalization lower bounds for boosted classifiers. NeurIPS 2019

[13] Grønlund et al., Near-tight margin-based generalization bounds for support vector machines. ICML 2020

[14] Wei et al., Open-set label noise can improve robustness against inherent label noise. NeurIPS 2021

[15] Jacobs et al., Adaptive mixtures of local experts. Neural Computation 1991

[16] Dornaika et al., Robust regression with deep CNNs for facial age estimation: An empirical study. Expert Systems with Applications 2020

[17] Zha et al., Rank-N-contrast: Learning continuous Representations for Regression. NeurIPS 2023.

Meta-review: This paper proposes a novel algorithm for regression with label noise. The proposed method partitions the data and trains several binary classifiers for the most distant partition pairs. Noisy data samples are detected using the voted probability of all classifiers. The method outperforms other baselines on several public datasets across different domains, as measured by a newly proposed metric. Given that the robust representation learning idea makes intuitive sense and that the reviewers are generally happy with the evaluation, I suggest the paper to be accepted. I have concerns on how the method would scale up to more regression targets but that could be left for future work. Authors shall incorporate reviewers' feedback on the presentation in the next version.
Decision: Accept (poster)