Mitigate Negative Transfer with Similarity Heuristic Lifelong Prompt Tuning (2024)

Abstract: Lifelong prompt tuning has significantly advanced parameter-efficient lifelong learning with its efficiency and minimal storage demands on various tasks.Our empirical studies, however, highlights certain transferability constraints in the current methodologies: a universal algorithm that guarantees consistent positive transfer across all tasks is currently unattainable, especially when dealing dissimilar tasks that may engender negative transfer.Identifying the misalignment between algorithm selection and task specificity as the primary cause of negative transfer, we present the Similarity Heuristic Lifelong Prompt Tuning (SHLPT) framework. This innovative strategy partitions tasks into two distinct subsets by harnessing a learnable similarity metric, thereby facilitating fruitful transfer from tasks regardless of their similarity or dissimilarity. Additionally, SHLPT incorporates a parameter pool to combat catastrophic forgetting effectively. Our experiments shows that SHLPT outperforms state-of-the-art techniques in lifelong learning benchmarks and demonstrates robustness against negative transfer in diverse task sequences.

## introduction drawing on the remarkable capacity of humans to amass new knowledge throughout their lifetime, lifelong learning (ll) systems aim to emulate this progressive learning trajectory by sequentially mastering various tasks, each contributing to the system's cumulative knowledge base. however, it is not trivial for deep learning models to achieve this ideal due to inherent challenges. these include the need to (1) avoid catastrophic forgetting -where the acquisition of new information can lead to the erosion of previously learned knowledge, and to recent advancements in lifelong learning of language model (lm) have integrated the concept of prompt tuning to enhance its capabilities. these approaches maintain the pre-trained model's parameters, while training a small set of additional prompts to adapt the model to various downstream tasks. the efficiency and lightweight nature of prompt tuning align well with the demands of ll, sidestepping the need for the onerous storage of entire model versions for every new task. this technique facilitates the model to accumulate knowledge over time, adapt flexibly to new tasks, and recall how to perform on older tasks with the aid of task-specific prompts. moreover, substantial ef-fort is dedicated to the transfer of knowledge from past tasks. this includes methods such as prompt concatenation @xcite , parameter sharing @xcite , and weighted summation @xcite , which are pivotal in ensuring that knowledge is effectively retained and utilized throughout the lifelong learning process. current transfer learning methods commonly presuppose that earlier tasks can positively impact succeeding tasks. @xcite leverages prompts from previous tasks in the process of learning new ones, and similarly, @xcite utilizes prompts from former tasks as a foundation to harness prior knowledge. nevertheless, we suggest that when there is a considerable dissimilarity between tasks, these approaches don't consistently assure positive transfer; on occasions, they might even provoke negative transfer. this phenomenon is vividly illustrated by the empirical analysis in to address the issue of inconsistent knowledge transfer, we introduce a new approach to lifelong prompt tuning, which we refer as to shlpt (similarity heuristic lifelong prompt tuning). first, we construct a prompt pool for learned tasks, thereby reducing the risk of forgetting. then, we segment our knowledge transfer module into three components: (1) assessing the similarity between the current task and previous tasks, (2) categorizing the previous tasks into similar and dissimilar subsets, and (3) applying different transfer algorithms to each subset accordingly. for the initial step, we calculate an attention-weighted combination of past prompt embeddings and incorporate this into the current task's prompt. during optimization, the model assigns higher attention scores to tasks that are more beneficial. then, we utilize this attention score as task similarity metric and split task set accordingly. for tasks deemed similar, we integrate their parameters to provide the current task with an optimized starting point. conversely, for tasks that are dissimilar, we introduce a variety of innovative regularization techniques aimed at guiding the pre-trained model towards accessing a broader range of knowledge. this nuanced approach allows the model to better adapt to each new task while preserving and effectively utilizing the knowledge from all previous learning experiences without negative transfer. in this paper, we make several notable contributions: (1) we address the novel challenge of mitigating negative transfer and facilitating knowledge transfer from dissimilar tasks in lifelong learning, which is particularly important in sequences of low-similarity tasks. (2) we present shlpt, an innovative lifelong prompt tuning technique that reduces forgetting and enables knowledge transfer across tasks with varying degrees of similarity. our extensive experiments demonstrate that shlpt surpasses existing methods on benchmark datasets. (3) we introduce a challenging benchmark characterized by low task similarity, which typically results in increased negative transfer. our approach exceeds the performance of previous state-of-the-art methods in this context. ## related work lifelong parameter efficient tuning. parameter efficient tuning tunes a subset of parameters of pretrained language models, and can largely reduce the computation costs and memory usage @xcite @xcite @xcite . for lifelong prompt tuning, lfpt5 @xcite uses distillation loss and a generative replay to learn a continuous prompt. progressive prompts @xcite learns a new prompt for every task, and progressively concatenate it with previous prompts. lpt @xcite employs a trainable binary mask on the overall prompt to selectively choose parameters for different tasks. l2p @xcite initializes a prompt pool and selects a certain number of prompts from it for each task. through this approach, parameters among different tasks can be shared and isolated simultaneously. base on this parameter pool architecture, coda-prompt @xcite employs an attention-based prompt selection strategy, diana @xcite and hide-prompt @xcite decompose prompts into a hierarchical structure. in addition to lifelong learning on prompt tuning, o-lora @xcite introduces orthogonal regularization to lora, facilitate learning in mutually orthogonal subspaces for different tasks; classic @xcite adds task masks to the adapter layer and utilizes contrastive loss to transfer knowledge between similar tasks. similarity heuristic methods. similarity heuristic lifelong learning methods use task similarity to identify which tasks can transfer knowledge and minimize interference from dissimilar tasks. cat @xcite ) compares the performance between transfer model and reference model to determine whether use the transfer model. b-cl @xcite and ctr @xcite utilize capsule network and routing algorithm to cluster similar tasks along with their shareable features. classic @xcite creates views form the hidden space information of previous similar tasks, and uses the contrastive loss to help current tasks to learn shared knowledge. ## method to this end, we propose shlpt (similarity heuristic lifelong prompt tuning), which intends to achieve the following goals for a robust lifelong learning system: (1) avoid forgetting previous knowledge, (2) transfer knowledge from both similar and dissimilar tasks, (3) reduce the probability of negative transfer during the transfer stage. shlpt is composed of the following three structures: (1) a prompt pool preserving previous task's prompts {𝑃 1 , 𝑃 2 , ..., 𝑃 𝑡 } to avoid forgetting of past learning in a lifelong learning context. during the testing phase, identification of the task is necessary. (2) a task similarity estimator which can estimate the similarity between current tasks and previous tasks. we operate under the premise that "the more similar the tasks, the more effective the knowledge transfer." based on estimated similarity, the collection of previous prompts is further categorized into two subsets: prompts from similar tasks 𝑃 1 𝑠 , 𝑃 2 𝑠 , ..., 𝑃 𝑚 𝑠 , and those from dissimilar tasks 𝑃 1 𝑑 , 𝑃 2 𝑑 , ..., 𝑃 𝑛 𝑑 . ## conclusion in this paper, we introduce a novel method shlpt for lifelong language learning. shlpt aims to alleviate negative transfer by learning task similarity in one step and employing suitable transfer method for different old task scenarios (similar or dissimilar). experimental results show shlpt outperforms baseline methods via better knowledge transfer on two existing benchmarks and our custom negative transfer benchmark.