Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning (2024)

Abstract: Parameter-efficient fine-tuning (PEFT) has enabled the efficient optimization of cumbersome language models in real-world settings. However, as datasets in such environments often contain noisy labels that adversely affect performance, PEFT methods are inevitably exposed to noisy labels. Despite this challenge, the adaptability of PEFT to noisy environments remains underexplored. To bridge this gap, we investigate various PEFT methods under noisy labels. Interestingly, our findings reveal that PEFT has difficulty in memorizing noisy labels due to its inherently limited capacity, resulting in robustness. However, we also find that such limited capacity simultaneously makes PEFT more vulnerable to interference of noisy labels, impeding the learning of clean samples. To address this issue, we propose Clean Routing (CleaR), a novel routing-based PEFT approach that adaptively activates PEFT modules. In CleaR, PEFT modules are preferentially exposed to clean data while bypassing the noisy ones, thereby minimizing the noisy influence. To verify the efficacy of CleaR, we perform extensive experiments on diverse configurations of noisy labels. The results convincingly demonstrate that CleaR leads to substantially improved performance in noisy environments

## introduction the ever-growing size of pre-trained language models (plms) has presented significant challenges in adapting these models to desired tasks. in response to this practical limitation, parameter-efficient finetuning (peft) has emerged as a promising strategy for real-world environments. instead of finetuning all weights, peft optimizes only a minimal set of parameters (e.g., biases @xcite , adapters @xcite , prompts @xcite , or low-rank matrices @xcite ), thereby drastically cutting down the computation and storage costs. such efficiency has led peft methods to become the preferred standard approaches for applying plms in real-world contexts, such as federated learning @xcite and continual learning @xcite . while peft enables the efficient optimization of plms in real-world settings, datasets in such environments often contain noisy labels (i.e., incorrectly-labeled samples) @xcite , which adversely affects the generalization capabilities of plms @xcite . given such distinct characteristics of the practical environments, peft methods are inevitably exposed to noisy labels during the optimization phase. despite this significant challenge, there is a lack of prior research on the general adaptability of peft methods to noisy label learning (nll) scenarios. in this work, we bridge this research gap by exploring peft under noisy environments. our results reveal that peft struggles in memorizing noisy labels due to its inherently limited capacity, which interestingly provides robustness foot_0 to noisy labels. however, we also find that such limited capacity simultaneously makes peft more susceptible to interference of noisy labels, which impedes learning ability for clean samples, potentially leading to sub-optimal performance. this characteristic markedly contrasts with the behaviors in full finetuning, presenting the necessity of peft that steers its limited learning capacity towards clean samples. in response, we propose clean routing (clear), a novel routing-based peft approach that adaptively activates peft modules. our main strategy is to preferentially expose peft modules to correctly-labeled samples, while bypassing peft weights for noisy samples, thereby minimizing the detrimental impact of noisy ones. to this end, clear estimates the probability that a given sample is correctly labeled through the lens of training dynamics. these probabilities are then conditioned to draw the routing decision such that potentially clean samples are more encouraged to route through the peft modules. the independent sampling across layers enables the fine-grained optimization for the peft modules. consequently, engaging peft primarily with clean samples minimizes the influence of noisy samples on peft. the clear approach is designed to be modelagnostic, allowing us to integrate the concept of clear with various types of peft methods. to evaluate the effectiveness of clear-based peft methods, we conduct comprehensive experiments across diverse configurations, such as noise rate and noise type. furthermore, we explore whether existing robust methods can be enhanced by seamlessly incorporating the clear approach. in summary, the contributions of this paper include the following: ## investigation of peft on noisy labels in this section, we systematically investigate peft methods in the presence of noisy labels. noisy environment following the previous work @xcite , we simulate the noisy environment by randomly flipping the given labels. specifically, we employ the sst-5 dataset with a symmetric noise rate of 60% (i.e., 60% of the training set contains incorrect labels). note that the test set is not corrupted to confirm the generalization ability of the trained model. the detailed process is described in appendix e. ## clear: peft with clean routing in this section, we elaborate clean routing (clear) in detail. the core idea is to adaptively activate the peft modules to circumvent the detrimental effect from noisy labels. to achieve this, clear estimates the probability whether a given sample is correctly labeled, leveraging the distinct training dynamics between clean and noisy samples. with these probabilities, clear steers the potentially clean samples to route through peft modules, whereas the noisy ones are directed to bypass peft modules, thereby minimizing their influence on the peft. to improve the routing stability of clear, we also introduce consistency regularization for peft modules. figure 2 illustrates the overall procedures of clear. ## experiments we demonstrate the efficacy of clear across diverse configurations of noisy environments. ## analysis to make a more comprehensive analysis of our clear, we designed a series of fine-grained experiments aimed at addressing the following research questions (rqs): • rq1. how does each component within clear contribute to its overall performance? ( §5.1) • rq2. can clear be integrated with other methods for learning with noisy labels? ( §5.2) • rq3. can clear be combined with other noisy label learning methods? ( §5.3) • rq4. does clear offer improvements under more realistic noisy label scenarios? ( §5.4) • rq5. can clear be generalized to the largesized model? ( §c) ## conclusion in this work, we have explored whether the peft methods can be generalized to noisy environments. interestingly, we have found that, while the limited capacity of peft allows the robustness to noisy labels, it can also act as a double-edged sword that hinders learning even with clean samples. in response, we have proposed clear, a novel routing-based peft approach that adaptively activates the peft modules by considering the probability of samples being clean. extensive experiments have convincingly demonstrated the efficacy of clear across diverse configurations of noisy labels. moreover, our in-depth analysis has demonstrated that clear effectively mitigates the underfitting on clean samples of peft methods.