Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning (2024)

Abstract: Parameter-efficient fine-tuning (PEFT) has enabled the efficient optimization of cumbersome language models in real-world settings. However, as datasets in such environments often contain noisy labels that adversely affect performance, PEFT methods are inevitably exposed to noisy labels. Despite this challenge, the adaptability of PEFT to noisy environments remains underexplored. To bridge this gap, we investigate various PEFT methods under noisy labels. Interestingly, our findings reveal that PEFT has difficulty in memorizing noisy labels due to its inherently limited capacity, resulting in robustness. However, we also find that such limited capacity simultaneously makes PEFT more vulnerable to interference of noisy labels, impeding the learning of clean samples. To address this issue, we propose Clean Routing (CleaR), a novel routing-based PEFT approach that adaptively activates PEFT modules. In CleaR, PEFT modules are preferentially exposed to clean data while bypassing the noisy ones, thereby minimizing the noisy influence. To verify the efficacy of CleaR, we perform extensive experiments on diverse configurations of noisy labels. The results convincingly demonstrate that CleaR leads to substantially improved performance in noisy environments1.

heading: 1 Introduction
text: The ever-growing size of pre-trained language models (PLMs) has presented significant challenges in adapting these models to desired tasks. In response to this practical limitation, parameter-efficient finetuning (PEFT) has emerged as a promising strategy for real-world environments. Instead of finetuning all weights, PEFT optimizes only a minimal set of parameters (e.g., biases (Zaken et al., 2022), adapters (Houlsby et al., 2019), prompts
* These authors contributed equally to this work. 1Our code is available at https://github.com/
yeachan-kr/clear
(Liu et al., 2022b), or low-rank matrices (Hu et al., 2022)), thereby drastically cutting down the computation and storage costs. Such efficiency has led PEFT methods to become the preferred standard approaches for applying PLMs in real-world contexts, such as federated learning (Kim et al., 2023; Liao et al., 2023) and continual learning (Ermis et al., 2022; Razdaibiedina et al., 2022).
While PEFT enables the efficient optimization of PLMs in real-world settings, datasets in such environments often contain noisy labels (i.e., incorrectly-labeled samples) (Jia et al., 2019; Alt et al., 2020), which adversely affects the generalization capabilities of PLMs (Wu et al., 2022). Given such distinct characteristics of the practical environments, PEFT methods are inevitably exposed to noisy labels during the optimization phase. Despite this significant challenge, there is a lack of prior research on the general adaptability of PEFT methods to noisy label learning (NLL) scenarios.
In this work, we bridge this research gap by exploring PEFT under noisy environments. Our results reveal that PEFT struggles in memorizing noisy labels due to its inherently limited capacity, which interestingly provides robustness2 to noisy labels. However, we also find that such limited capacity simultaneously makes PEFT more susceptible to interference of noisy labels, which impedes learning ability for clean samples, potentially leading to sub-optimal performance. This characteristic markedly contrasts with the behaviors in full finetuning, presenting the necessity of PEFT that steers its limited learning capacity towards clean samples.
In response, we propose Clean Routing (CleaR), a novel routing-based PEFT approach that adaptively activates PEFT modules. Our main strategy is to preferentially expose PEFT modules to correctly-labeled samples, while bypassing PEFT
2Following (Wang et al., 2021), we define robustness as the preservation of the performance under noisy labels.
5922
weights for noisy samples, thereby minimizing the detrimental impact of noisy ones. To this end, CleaR estimates the probability that a given sample is correctly labeled through the lens of training dynamics. These probabilities are then conditioned to draw the routing decision such that potentially clean samples are more encouraged to route through the PEFT modules. The independent sampling across layers enables the fine-grained optimization for the PEFT modules. Consequently, engaging PEFT primarily with clean samples minimizes the influence of noisy samples on PEFT.
The CleaR approach is designed to be modelagnostic, allowing us to integrate the concept of CleaR with various types of PEFT methods. To evaluate the effectiveness of CleaR-based PEFT methods, we conduct comprehensive experiments across diverse configurations, such as noise rate and noise type. Furthermore, we explore whether existing robust methods can be enhanced by seamlessly incorporating the CleaR approach. In summary, the contributions of this paper include the following:
• We first explore the effectiveness of PEFT methods in the context of noisy environments, providing a comprehensive analysis of their robustness and limitations.
• We propose CleaR, a novel PEFT approach that adaptively activates the PEFT modules to improve generalization capability while minimizing the influence of noisy samples.
• We demonstrate that CleaR-based PEFT methods achieve superior performance across various NLP tasks even under heavy noise conditions, thereby pushing the boundaries of robustness and generalization.

heading: 2 Investigation of PEFT on Noisy Labels
text: In this section, we systematically investigate PEFT methods in the presence of noisy labels.
Noisy environment Following the previous work (Wu et al., 2022), we simulate the noisy environment by randomly flipping the given labels. Specifically, we employ the SST-5 dataset with a symmetric noise rate of 60% (i.e., 60% of the training set contains incorrect labels). Note that the test set is not corrupted to confirm the generalization ability of the trained model. The detailed process is described in Appendix E.
PEFT methods We analyze the three representative types of PEFT methods (Chen et al., 2022) with the full fine-tuning: LoRA (Hu et al., 2022) that adds trainable decomposition matrices; BitFit (Zaken et al., 2022) that trains only biases; Prompt Tuning (Liu et al., 2022b) that appends learnable embeddings to the input of each layer3.
Observations Figure 1 presents the evaluation results of the PEFT methods alongside the full finetuning. The accuracy results (first row in the figure) show that PEFT methods reveal superior robustness to the full fine-tuning, even though all methods suffer from performance degradation. To gain further insights into the behavior of each method, we include the training loss for both clean4 and noisy samples (second low in the figure), which enables to analyze the learning capacity on clean and noisy samples (Arazo et al., 2019). These results show that PEFT methods have difficulty in memorizing the noisy samples, which interestingly contributes
3The setups for each PEFT methods are described in §D 4In this paper, we use the terms clean and correctly-labeled interchangeably to refer samples with ground truth annotation.
to the robustness. However, it is evident that PEFT methods also face challenges in learning from clean samples (i.e., a large gap in losses of clean samples when exposed to the noisy dataset). The result implies that PEFT, which inherently has limited capacity (Ding et al., 2023), is more vulnerable to the interference of noisy labels, impairing its learning ability on clean data. This limitation can lead to sub-optimal performance, underscoring the need for PEFT methods that can steer its limited capacity toward clean samples.

heading: 3 CleaR: PEFT with Clean Routing
text: In this section, we elaborate Clean Routing (CleaR) in detail. The core idea is to adaptively activate the PEFT modules to circumvent the detrimental effect from noisy labels. To achieve this, CleaR estimates the probability whether a given sample is correctly labeled, leveraging the distinct training dynamics between clean and noisy samples. With these probabilities, CleaR steers the potentially clean samples to route through PEFT modules, whereas the noisy ones are directed to bypass PEFT modules, thereby minimizing their influence on the PEFT. To improve the routing stability of CleaR, we also introduce consistency regularization for PEFT modules. Figure 2 illustrates the overall procedures of CleaR.

heading: 3.1 Parameter-Efficient Fine-Tuning Modules
text: We start by defining PEFT modules in CleaR. Since CleaR is designed to be module-agnostic, diverse PEFT methods can be seamlessly integrated with CleaR method. To showcase such applicability, we consider four representative PEFT modules (i.e., Adapter, BitFit, Prompt-tuning, and LoRA), which are commonly employed within NLP community5. While these PEFT modules have distinct characteristics, they can be succinctly represented as additional parameters δ added to the
5The detailed illustrations of these modules on CleaR are shown in the Appendix §D (Figure 6)
pre-trained weights θ of PLMs (Ding et al., 2023). More specifically, since PEFT modules are uniformly distributed across all layers, we represent these modules as a set of additional parameters δ = {δ(1), δ(2), . . . , δ(L)}, where L is the number of layers. Let the prediction involving these PEFT modules be denoted as f(x, δ + θ), the objective with an arbitrary loss function L can be formulated as follows:
min δ
L(x) = L(f(x, δ + θ), y), (1)
where x and y denote the training sample and the given label, respectively. Note that the PEFT modules and the task-specific classifier are only updated during training.

heading: 3.2 Routing PEFT Modules
text: Building upon these PEFT modules, we introduce a clean routing scheme that adaptively activates modules according to the estimated probability that a given sample is correctly labeled.
Estimating clean probability for routing To derive the clean probability for each sample, we leverage the distinct learning patterns when learning with clean and noisy samples: deep networks prefer to learn clean samples first before fitting noisy ones (Arazo et al., 2019). Namely, noisy samples tend to have a higher loss than clean samples in the early training stage. This enables to distinguish potentially clean samples from the datasets based on loss deviation (Jiang et al., 2018; Han et al., 2018). Taking advantage of such phenomena, we adopt the widely-used Gaussian Mixture Model (GMM) in noise label learning (Li et al., 2020; Qiao et al., 2022), in which the probability of samples being clean is estimated by the per-sample loss.
Based on the estimated mixture models, we compute the clean probability p using the posterior probability, i.e., p(g|ℓ) where ℓ is the loss for the training sample, and g is the Gaussian component
with a smaller mean (i.e., smaller loss). Specifically, we first train our model for the k epochs warm-up to measure the loss of samples, and then we estimate the clean probability for each training sample on every subsequent epoch. It is noteworthy that we leverage training losses obtained from the previous epoch to estimate clean probabilities, thereby reducing the additional computational cost from redundant forward passes.
Sampling routing decision Once the clean probability is estimated, the PEFT modules are stochastically routed across the transformer layers. To derive the routing decision (i.e., routing through PEFT or bypassing PEFT), we sample the decision from a Bernoulli distribution with the estimated clean probability p:
r ∼ BERNOULLI(γp), (2)
where r is an independent Bernoulli random variable with a probability γp of being 1 and a probability 1− γp of being 0. The coefficient γ ∈ [0, 1] limits the range of clean probability, setting its upper bound at γ. This coefficient plays a role in preventing over-reliance on the estimated probability, considering that small-loss samples might still contain noisy samples (i.e., high clean probability despite being noisy samples) (Li et al., 2020). Crucially, this routing decision is independently made at each layer, allowing for the fine-grained differentiation of each sample’s influence based on its probability of being clean. For example, if the clean probability is 70% and the number of layers is 10, seven PEFT modules are activated in average6.
Activating PEFT based on the decision The routing decisions across different layers are then applied to all PEFT modules. Formally, let the hidden states in the l-th layer be denoted as h(l), and the hidden state in the next layer is derived as follows:
h(l+1) = { Trans(l)(h(l), δ(l) + θ(l)), if r(l) = 1 Trans(l)(h(l), θ(l)), if r(l) = 0 (3) where δ(l) and θ(l) represent PEFT module and pre-trained parameters in the l-th layer, respectively, and r(l) indicates the routing decision on the
6While different positions of the PEFT could have varying effects on the prediction (Chen et al., 2023), in this work, we focus solely on the clean probability as a trigger to activate the PEFT. We reserve further exploration of this for future work.
layer. Trans(l)(·) denotes the function of the transformer block. Through the above routing decision, CleaR activates a subset of PEFT modules, i.e., δr = {δ(l)|δ(l) ∈ δ, r(l) = 1}, on each forward pass. This routing scheme ensures that PEFT modules are favorably activated for potentially clean samples and deactivated for noisy ones, thereby reducing the influence of noisy samples.
CleaR in inference phase As clean routing only performs in training, we need to decide the routing strategy during inference. To make the most of well-trained PEFT modules and ensure consistency with training, we empirically set the routing probability to the upper bound, i.e., p = 1.0 in Eq. (2), and observe that it works well in practice.

heading: 3.3 Consistency Regularization for CleaR
text: While the routing scheme effectively mitigates the influence of noisy labels, model predictions may end up being overly diverse due to varying activations with each forward pass, potentially resulting in training instability. To address this issue, we introduce a consistency regularization to minimize the model variability. Considering that guiding the model to adhere to past predictions can enhance the stability and consistency of training (Shen et al., 2022; Xu et al., 2023), we regulate the model by minimizing the distance between its current and previous predictions. Specifically, we make ensemble predictions from multiple forwards to reduce predictive variance and increase stability:
fens(x, δ̄r + θ) = 1
N
N∑
k=1
f(x, δ̄r,k + θ), (4)
where N is the number of forwards, and δ̄r,k represents activated PEFT modules in the k-th forward of the previously trained model. It is noteworthy that, for computational efficiency, we reuse the predictions, which were previously used for fitting GMM. With the derived predictions, the model with CleaR is optimized with the following loss:
min δr L(x) = LCE(f(x, δr + θ), y)
+ λLCE(f(x, δr + θ), fens(x, δ̄r + θ)). (5)
where LCE(·) indicates the cross-entropy loss, and λ is a coefficient to control the strength of the regularization.

heading: 4 Experiments
text: We demonstrate the efficacy of CleaR across diverse configurations of noisy environments.

heading: 4.1 Configurations of Noisy Labels
text: To comprehensively assess our method diverse scenarios characterized by different noisy labels, we evaluate each baseline against three distinct types of noisy labels: symmetric, asymmetric, and instance-dependent. We provide detailed descriptions of each noisy label type and the methodology for constructing noisy label datasets in §E.

heading: 4.2 Baselines and Implementations
text: Following the previous works, we use the BERTbase and BERT-large model (Devlin et al., 2019). Building on this PLM, we mainly compare CleaR with the full fine-tuning and widely-used PEFT methods including Adapter (Houlsby et al., 2019), LoRA (Hu et al., 2022), Prompt tuning (Liu et al., 2022b), and BitFit (Zaken et al., 2022). For a fair comparison, we utilize the same settings for PEFT modules (e.g., bottleneck dimension, prompt length) for baselines and our CleaR. The detailed implementations are represented in §F. Additionally, we compare CleaR with the existing NLL methods to confirm the competitiveness of the proposed method in §5.3.

heading: 4.3 Evaluation Metric
text: Building upon previous work (Wu et al., 2022), we assess each baseline model using two metrics: the instantaneous peak accuracy and the average accu-
racy across the last few epochs. The former metric evaluates the model’s generalization performance, while the latter reflects its stability. Consequently, a smaller gap between these two metrics indicates a more effective model to noisy labels. For all CleaR models, we report the average performance on 10 different seeds considering their stochasticity.

heading: 4.4 Sentiment Analysis
text: We first evaluate baselines in sentiment analysis due to the inherent subjectivity of this task, which often results in noisy labels. Following the previous work, we use the Standard Sentiment Treebank (SST-5) dataset (Socher et al., 2013). For the levels of noisy labels, we scale the symmetric noise from 20% to 60% and asymmetric noise from 10% to 40%, respectively.
Table 1 presents the evaluation results for the task. As observed in the previous analysis, PEFT exhibits better robustness than full fine-tuning across different types and levels of noisy labels. Notably, CleaR-based methods show substantial improvement compared to PEFT methods on both metrics. This demonstrates that CleaR enhances the generalization capability of PEFT (i.e., improved peak accuracy) while maintaining or even strengthening its robustness (i.e., reduced gaps between peak and average accuracy).

heading: 4.5 Intent Detection
text: We further evaluate CleaR on intent detection. Given that the task is typically employed in conversational systems, the query usually consists of only a few words (e.g., 6 to 12 words) (Casanueva
et al., 2020). Such brevity can amplify ambiguity, potentially leading to noisy annotations. We utilize the BANKING77 (Casanueva et al., 2020) that encompasses 77 fine-grained intent categories. For the levels of noisy labels, we scale the symmetric noise from 20% to 60% and asymmetric noise from 10% to 40%, respectively.
Table 2 presents the evaluation results for intent detection. Similar to the previous task, PEFT achieves superior robustness to noisy labels compared to full fine-tuning by showing higher average accuracy even on the highest noisy levels. Meanwhile, CleaR consistently outperforms both the PEFT and full fine-tuning methods, demonstrating its efficacy to improve the robustness. It is noteworthy that in certain setups, while PEFT shows better average accuracy, it yields slightly lower
peak accuracy due to its limited capacity to memorize clean samples (e.g., asymmetric noise levels between 20% and 40%). Interestingly, CleaR significantly boosts the peak accuracy of each PEFT variant, achieving similar or even better peak accuracy than full fine-tuning. These results again verify that CleaR successfully mitigates the limited memorization of PEFT methods, thereby leading to better generalization and robustness.

heading: 5 Analysis
text: To make a more comprehensive analysis of our CleaR, we designed a series of fine-grained experiments aimed at addressing the following research questions (RQs): • RQ1. How does each component within CleaR
contribute to its overall performance? (§5.1) • RQ2. Can CleaR be integrated with other meth-
ods for learning with noisy labels? (§5.2) • RQ3. Can CleaR be combined with other noisy
label learning methods? (§5.3) • RQ4. Does CleaR offer improvements under
more realistic noisy label scenarios? (§5.4) • RQ5. Can CleaR be generalized to the large-
sized model? (§C)

heading: 5.1 Ablation Studies on CleaR (RQ1)
text: We perform ablation studies to investigate the contributions of each component and routing strategy in CleaR. Table 3 presents the ablation results.
Routing and Regularization As shown in the upper part of Table 3, omitting routing and consistency regularization largely affects both peak
and average accuracy. The results demonstrate that each component is essential for improving generalization and robustness. In particular, we observe that the routing mechanism plays a significant role.
Routing strategy To further investigate the proposed routing scheme, we compared it with three different schemes: (i) Random Routing, where PEFT modules are activated randomly; (ii) Noisy Routing, where PEFT modules are activated in proportion to the noisy probability, representing the opposite approach of CleaR; and (iii) Deterministic Routing, where the noisy samples are filtered out by leveraging the estimated probability based on GMM, and PEFT modules are only deterministically activated on remaining samples.
As shown in the lower part of Table 3, adopting Random and Noisy Routing substantially decreases both metrics, indicating that favoring clean sam-
ples in the routing is indeed beneficial to achieve both generalization and robustness ability. We then compare our Clean Routing with the Deterministic Routing. We observe that Deterministic Routing exhibits inferior performance than our Clean Routing. Moreover, Figure 3 illustrates the differences in accuracy on clean and noisy samples between Clean Routing and Deterministic Routing. It reveals that Deterministic Routing leads to increased memorization of noisy samples compared to our stochastic Clean Routing, which is attributed to lower performance. These results emphasize the significance of stochastic routing in Clean Routing, which can differentiate samples in a more finegrained manner.

heading: 5.2 Memorization Effects (RQ2)
text: To confirm whether CleaR mitigates the underfitting problems of PEFT methods on clean samples, we compare the ratio of memorizing clean and noisy samples after fine-tuning. Figure 4 shows the comparison. Notably, we observe that adopting CleaR into PEFT methods largely increases the memorization of clean samples, while preserving or even reducing the memorization of noisy samples. These results verify that CleaR successfully mitigates the underfitting of PEFT methods on the clean dataset by favorably activating the PEFT modules on potentially clean samples.

heading: 5.3 CleaR on Different NLL Methods (RQ3)
text: We compare CleaR with existing NLL methods to demonstrate its applicability. We consider three approaches: Co-teaching (Han et al., 2018), SELC
(Lu and He, 2022), and STGN (Wu et al., 2022)7. Table 4 showcases the comparison and integration results with CleaR and Adapter. We observe that NLL methods can be enhanced with CleaR, as it can be seamlessly integrated by adding PEFT modules. The results show that while combining NLL methods with Adapter brings marginal improvement, adopting CleaR leads to a marked enhancement across both metrics. This suggests that CleaR can place the current state-of-the-art NLL methods on a more solid footing by enjoying the robustness and improved generalization ability of PEFT.

heading: 5.4 Instance-dependent Label Noise (RQ4)
text: To investigate CleaR in more realistic settings, we evaluate CleaR in scenarios where noisy labels arise from input features. Table 5 presents the evaluation results on SST-5 with the instancedependent noise. Similar to the other noise settings, CleaR consistently yields the best performance on both peak and average accuracy across different noise ratios, highlighting the validity of CleaR in addressing feature-dependent noise.

heading: 6 Related Work

heading: 6.1 Robustness of PEFT Methods
text: The well-known beneficial properties of PEFT are its generalization capabilities in low-data environments (Liu et al., 2022a; Zaken et al., 2022) and stability (Houlsby et al., 2019; Sung et al., 2021). These studies have demonstrated that full finetuning suffers from overfitting on small datasets,
7The detailed settings are included in the Appendix §G
whereas PEFT methods achieve superior performance in few-shot settings due to the regularization effect on the pre-trained models (Fu et al., 2023). Regarding model calibration, recent work has shown that preserving the pre-trained features through the PEFT methods improves the model calibration by preventing overfitting (He et al., 2023; Ding et al., 2023). Following the robustness on the overfitting, several studies have demonstrated that PEFT methods also being less suffered from catastrophic forgetting (He et al., 2021), and their robustness against forgetting is further evidenced in continual learning scenarios (Ermis et al., 2022).
Previous studies have broadened the understanding of PEFT in diverse aspects. Building upon this research direction, we explore its robustness to noisy labels, which remains a challenging problem within the deep learning community. Therefore, exploring PEFT methods in this context could provide valuable insights into their practical applicability.

heading: 6.2 Noisy Label Learning in NLP
text: Noisy labels are inevitably introduced on largescale datasets (Jia et al., 2019; Wu et al., 2023). To mitigate the influence of noisy labels in text classification, Jindal et al. (2019) have proposed a noise transition matrix on top of the classifier, learning the transition distribution of noisy labels, and Wang et al. (2023) have tackled noisy labels in classification tasks with supplemental guidance from large language models (e.g., ChatGPT). In addition, Zhuang et al. (2023) have utilized the dynamics of features with generative models during training to learn the transition matrix for noisy labels. For named-entity recognition, Meng et al. (2021) have introduced a self-training method based on the contextualized augmentations, and Zhou and Chen (2021) have proposed a co-regularization in which multiple models teach each other to avoid negative memorization. In entity linking, Le and Titov (2019) have explored an interpretable method to identify clean samples on the premise that interpretable samples tend to be clean. Recently, Wu et al. (2022) have proposed a noise-robust optimization by introducing stochastic tailor-made gradient noise.
While these works have achieved robustness to noisy labels, these works typically consider the scenarios that the entire parameters in PLMs are optimized during fine-tuning, which is challenging due to the huge number of parameters. To the best of our knowledge, we are the first to explore
the PEFT methods for NLL. Given that CleaR is built upon PEFT methods, it can enjoy the benefits of training efficiency and robustness, which is favorable in the real-world environment.

heading: 7 Conclusion
text: In this work, we have explored whether the PEFT methods can be generalized to noisy environments. Interestingly, we have found that, while the limited capacity of PEFT allows the robustness to noisy labels, it can also act as a double-edged sword that hinders learning even with clean samples. In response, we have proposed CleaR, a novel routing-based PEFT approach that adaptively activates the PEFT modules by considering the probability of samples being clean. Extensive experiments have convincingly demonstrated the efficacy of CleaR across diverse configurations of noisy labels. Moreover, our in-depth analysis has demonstrated that CleaR effectively mitigates the underfitting on clean samples of PEFT methods.

heading: 8 Limitation
text: While we have shown that CleaR successfully improves the effectiveness of PEFT on various NLL scenarios, there exists a few limitations.
Exploration on Different Architectures Our efforts have been focused on improving the efficacy of PEFT for encoder models, aligning with previous studies (Wu et al., 2022). Therefore, the applicability of CleaR methods to different architectures (e.g., decoder, encoder-decoder models) remains under-explored in this work. Nevertheless, based on recent evidence suggesting that routing-based PEFT methods can be effectively generalized to various architectures beyond encoder models (Wang et al., 2022; Choi et al., 2023), we believe that CleaR is expected to work well within other architectures. We leave the exploration of this direction as promising future work.
Computational Overheads The adaptive routing mechanism in CleaR could potentially introduce computational overhead in two main areas: (i) determining the probability of each sample, and (ii) executing the PEFT routing. However, the overhead for (i) can be mitigated by caching the samples’ losses during the training phase, eliminating the need for separate procedures to compute the training loss. As for (ii), in comparison to other routing-based methods cited as (Choi et al., 2023)
that employ parameterized routers, the additional computational costs in CleaR are negligible, as the router in CleaR is non-parametric, and decisions are made through sampling, which streamlines the process.

heading: Acknowledgment
text: This work was supported by the Basic Research Program through the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2021R1A2C3010430) and Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.RS-2019II190079, Artificial Intelligence Graduate School Program (Korea University)).

heading: A PEFT Analysis on Different Dataset
text: We further conducted the same analysis on a different dataset to validate the generality of our observations. Specifically, we examined PEFT methods on the BANKING77 (Casanueva et al., 2020) dataset, which focuses on an intent detection task. The results are presented in Figure 5. Similar to our analysis of the SST-5 dataset, we observed a similar trend: (i) PEFT methods exhibit greater robustness than full fine-tuning, and (ii) its limited memorization on noisy label attributes to robustness, although they also inhibit memorization even on clean samples. These findings further support our observations and analysis regarding the robustness of PEFT methods to noisy labels.

heading: B Experiments on Additional Datasets
text: To further demonstrate the broad applicability of our proposed method, we have evaluated the proposed methods on the TREC and 20Newsgroups datasets. Table 6 shows the evaluation results on the two datasets with 60% symmetric noises. We can observe the similar performance trend with the existing benchmarks, further underscoring the general applicability of the proposed method.

heading: C CleaR on Larger Model
text: To evaluate how CleaR performs as the model evolves, we compare our CleaR with baselines on BERT-Large, which is 3× larger than the basesized model. Table 7 represents the evaluation results on SST-5 with different levels of symmetric noise. We observe CleaR still outperforms baselines on both peak and average accuracy by a large margin. These results demonstrate that our CleaR allows us to improve the model performance on noise label settings regardless of the model sizes.

heading: D CleaR on PEFT methods
text: In this paper, we apply CleaR to the existing PEFT methods. Specifically, we select the widely-used and different types of PEFT methods, which include Adapter (Houlsby et al., 2019), Prompt Tuning (Li and Liang, 2021), BitFit (Zaken et al., 2022), and LoRA (Hu et al., 2022). For each PEFT methods, we follow the commonly-used setup, and the ratio of trainable parameters are listed in Table 8. We name the adopted version of each method as CleaRAdapter, CleaRPrompt, CleaRBitFit,
and CleaRLoRA. For each method, we provide the graphical procedure in Figure 6, and its details are as follows:
CleaRAdapter Adapter (Houlsby et al., 2019) employs a bottleneck architecture, comprising downand up-projection matrices with non-linearity applied to the bottleneck feature. These adapters are positioned above the multi-head self-attention and position-wise feed-forward layers. In the CleaR method, we group these two adapters and determine their usage through a sampling process.
CleaRPrompt We adopt P-Tuning v2 (Liu et al., 2022b) as a representative method for prompt tuning. This approach simply appends trainable prompts to the original input embeddings and fine-tunes these appended prompt embeddings during the fine-tuning phase. In the CleaR method, we group these prompt embeddings and decide
whether or not the prompts are appended to the input. As a result, in CleaRPrompt, the input length can vary based on the routing decision.
CleaRBitFit Different to other PEFT methods that introduce trainable weights (e.g., adapters, prompts), BitFit (Zaken et al., 2022) fine-tunes the existing biases in the PLMs. To apply routing to BitFit, we store the pre-trained biases, and the routing mechanism determines whether the training model utilizes the pre-trained (fixed) biases or the trainable biases.
CleaRLoRA LoRA (Hu et al., 2022) employs a bottleneck architecture similar to the adapter. However, the bottleneck in LoRA is situated within the weights for the attention matrices. In the CleaR method, we decide whether the linear weights for the attention mechanism (e.g., query, key, and value weights) incorporate trainable weights or not.

heading: E Detailed Process for Generating Noisy Labels
text: We detail the process for generating noisy labels when the noise rate p is given.
• Symmetric noise: To generate this noise, we create the noise transition matrix T ∈ RC×C where C is the number of classes. We then set a value of p to its diagonal elements and distribute the remaining probability (1 − p) to other non-diagonal elements. Based on the probability in the matrix, we flip the labels in training samples. • Asymmetric noise: Similar to the symmetric noise, we create the noise transition matrix T ∈ RC×C where C is the number of classes. We then set a value of p to its diagonal elements and assign the remaining probability (1 − p) to the next element of the diagonal values to implement single-flip noise (Qiao et al., 2022). Based on the probability in the matrix, we flip the labels in training samples. • Instance-dependent noise: To generate instance-dependent noise, we first pre-train the classifier on the original dataset. We then select the two classes, which are the most confident u and the second most confident classes s, and calculate the distance of decision boundaries between the classes, i.e., [fu(x) − fs(x)]2. Based on the distance, we define the noise function as τ = −12 [fu(x)− fs(x)]2 + 12 . A smaller distance between the classes results in a larger flipping probability to the second most confident class s. Lastly, to control the degree of noisy labels, we multiply τ by a certain constant factor such that the final proportion of noise matches the pre-defined noise probability.
F Implementation Details and Setups
In this section, we detail to implement the baselines and our CleaR on various tasks.
PEFT implementation. We compare our CleaR with four strong baselines, which include Adapter (Houlsby et al., 2019), LoRA (Hu et al., 2022), BitFit (Zaken et al., 2022), and Prompt-tuning (Liu et al., 2022b). Specifically, we set the bottleneck dimension r for the Adapter and LoRA as 16 and 4, respectively. For LoRA, we only apply LoRA weights on query and value attention weights. Moreover, we fine-tune all bias parame-
ters in transformer blocks for BitFit. For Prompttuning, we set the fixed length as 20 for prompts in each transformer layer, following P-tuning v2 (Liu et al., 2022b).
Hyper-parameters. For the hyper-parameters to fine-tune in CleaR, we select the best warmup epoch in [3, 10] and clean probability weights γ in [0.5, 1] corresponding to each PEFT method and task. We also use the number of forward N = 5 for constructing ensemble predictions on consistency regularization. we use the consistency regularization coefficient λ = 1 for all experiments. For other settings, we use Adam optimizer (Kingma and Ba, 2015) with β1 = 0.9 and β2 = 0.999. We also train all models using a batch size of 32 and sweep the learning rates in {1e-4, 2e-4, 3e-4, 4e-4, 5e-4} for PEFT methods. For full fine-tuning, we select the best learning rates in {1e-5, 2e-5, 3e-5, 4e-5, 5e-5}. All models are fine-tuned for 20 epochs.
Hardware Details. We train all our models using four RTX 3090 GPUs. We utilize mixedprecision training (Micikevicius et al., 2018) to expedite the training procedure. All the implementations are performed using the PyTorch framework.

heading: G Details for other NLL Methods
text: We compare our CleaR with the following NLL methods:
Co-teaching. Co-teaching (Han et al., 2018) trains two models simultaneously and lets each model select clean training samples (i.e., smallloss instances) for training each other model. Coteaching framework gradually drops the noisy samples to prevent overfitting. To this end, they require an estimation of the noise level (τ ), warm-up steps (Tk), and coefficient (c). We set the noise level τ as the ground truth noise ratio. We vary the warm-up steps Tk in {1500, 2000, 2500, 3000, 3500}, and select the best coefficient c from search range of [1, 2] for each fine-tuning method.
SELC. SELC (Lu and He, 2022) trains the model using ensemble prediction based on historical model outputs to correct the noisy labels. Specifically, they first train their models with given labels until the turning point T , which represents the model would start overfitting on noisy levels. And then they combine the given labels with ensemble predictions with momentum α as the target. We estimate the turning point T by leveraging met-
rics, following (Lu and He, 2022), corresponding to each fine-tuning method. We also select the best momentum α by searching parameters from [0.5, 1].
STGN. STGN (Wu et al., 2022) trains the model by reducing the disturbance on correct samples and increasing the perturbation on corrupted ones. Specifically, they utilize the standard deviation σf to perturb the gradient of loss and forgetting events threshold λf to separate corrupted data from correct ones. For full fine-tuning, we use the same setup in (Wu et al., 2022). For adapter-based tuning, we set σmax = 2σf and select the σf = 7e−4 and λf = 4.
We implement other NLL methods based on standard BERT with reference to their public code and make comparisons under the same setting. For other hyper-parameters (i.e., learning rate, and warmup epochs), we select the optimal values by searching on the same parameter space with baselines and CleaR.

heading: Future Work
text: We leave the exploration of this direction as promising future work. Computational Overheads The adaptive routing mechanism in CleaR could potentially introduce computational overhead in two main areas: (i) determining the probability of each sample, and (ii) executing the PEFT routing. However, the overhead for (i) can be mitigated by caching the samples’ losses during the training phase, eliminating the need for separate procedures to compute the training loss. As for (ii), in comparison to other routing-based methods cited as (Choi et al. , 2023)
that employ parameterized routers, the additional computational costs in CleaR are negligible, as the router in CleaR is non-parametric, and decisions are made through sampling, which streamlines the process