LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5

Existing approaches to lifelong language learning rely on plenty of labeled data for learning a new task , which is hard to obtain in most real scenarios . Considering that humans can continually learn new tasks from a handful of examples , we expect the models also to be able to generalize well on new few-shot tasks without forgetting the previous ones . In this work , we define this more challenging yet practical problem as Lifelong Few-shot Language Learning ( LFLL ) and propose a unified framework for it based on prompt tuning ( PT ) of T5 . Our framework called LFPT5 takes full advantage of PT ’ s strong few-shot learning ability , and simultaneously trains the model as a task solver and a data generator . Before learning a new domain of the same task type , LFPT5 generates pseudo ( labeled ) samples of previously learned domains , and later gets trained on those samples to alleviate forgetting of previous knowledge as it learns the new domain . In addition , a KL divergence loss is minimized to achieve label consistency between the previous and the current model . While adapting to a new task type , LFPT5 includes and tunes additional prompt embeddings for the new task . With extensive experiments , we demonstrate that LFPT5 can be applied to various different types of tasks and significantly outperform previous methods in different LFLL settings . 1 INTRODUCTION . A hallmark of human intelligence is that they can learn new tasks quickly by leveraging previously acquired knowledge from other related tasks , and they do so without forgetting prior knowledge . However , despite the monumental success of deep learning in recent years , models face challenges to retain and accumulate knowledge when learning new tasks due to the shift of data distribution – they run into the overfitting issue when the data for the new task is small and they forget prior knowledge , a phenomenon known as catastrophic forgetting ( McCloskey & Cohen , 1989 ) . Researchers in Lifelong Learning ( Thrun & Mitchell , 1995 ) have proposed a number of methods to alleviate the above issues with machine learning . When it comes to language , earlier approaches to Lifelong Language Learning ( LLL ) merely focus on a single type of NLP tasks ( Wang et al. , 2019 ; d ’ Autume et al. , 2019 ) ; see ( Biesialska et al. , 2020 ) for a survey . In contrast , humans can easily handle tasks that vary with respect to not only domain but also task type ( Figure 1 ) . More recent methods attempt to learn from different types of tasks . These include LAMOL ( Sun et al. , 2019 ) and its improvements ( Chuang et al. , 2020 ; Sun et al. , 2020 ; Kanwatchara et al. , 2021 ) . Despite the effectiveness of these methods in LLL , there are several limitations . First , they all assume plenty of training data for every task which is hard to acquire in most real scenarios as getting large labeled datasets is often expensive and time-consuming . Second , they mainly consider tasks from the decaNLP challenge ( McCann et al. , 2018 ) that can be easily framed as question answering ( Kumar et al. , 2016 ) , paying little attention to sequence labeling tasks such as Name Entity Recognition ( NER ) . Finally , they fine-tune the entire model for all tasks ignoring the possibility of negative transfer ( Lopez-Paz & Ranzato , 2017 ) between different types of tasks . Our work in this paper aims to address these limitations of LLL . We focus on a more challenging yet more practical problem where the model needs to generalize well on new few-shot tasks without forgetting the previous ones . We regard this as Lifelong Few-shot Language Learning ( LFLL ) and investigate three different kinds of tasks : sequence labeling tasks , text classification tasks and text generation tasks . Based on the strong few-shot learning ability of prompt tuning ( Lester et al. , 2021 ) of T5 ( Raffel et al. , 2019 ) , we propose a unified framework for LFLL , named LFPT5 ( Lifelong Fewshot Language Learning with Prompt Tuning of T5 ) . Specifically , we reframe all types of tasks into a text-to-text format ( Figure 2 ) . To continually learn new domains of a task , we simultaneously train the prompt embeddings designed for this task type as a task solver and a data generator keeping the backbone T5 frozen . When LFPT5 goes about learning a new domain , it first generates pseudo labeled samples of previously learned domains , which are then combined with the new domain training data to alleviate catastrophic forgetting . To achieve label consistency between the previous and the current model , LFPT5 also minimizes a KL divergence loss . For the adaptation from one task type to another , LFPT5 includes additional prompt embeddings for the new task , and tunes them similarly . In this way the learning of new tasks minimally affects previously acquired knowledge , mitigating the catastrophic forgetting problem . In the whole learning process , the pre-trained T5 acts as a meta-learned model ( Brown et al. , 2020 ) that is kept frozen , while the tunable soft prompt acts as a task or domain adaptation model . In summary , our main contributions are : • To the best of our knowledge , we are the first to consider LFLL , a challenging yet practical problem . We propose LFPT5 , a unified framework for LFLL based on prompt tuning of T5 . LFPT5 can generalize well on various new few-shot tasks without severe forgetting of previously acquired knowledge , which can be seen as an important step towards general language intelligence . • With extensive experiments and analysis , we demonstrate that LFPT5 outperforms previous baselines by a large margin . We have open-sourced our code base at https : //github.com/ qcwthu/Lifelong-Fewshot-Language-Learning . 2 RELATED WORK . 2.1 LIFELONG LEARNING . In lifelong learning ( LL ) , the model is expected to learn sequentially from a stream of tasks with different data distributions . The main problem in LL is catastrophic forgetting ( McCloskey & Cohen , 1989 ) – the model forgets previously acquired knowledge after learning a new task . Prior approaches to LL can be divided into three categories . First , architecture-based methods dynamically adjust the model architecture to learn new knowledge while preventing the forgetting of previously learned tasks ( Chen et al. , 2015 ; Rusu et al. , 2016 ; Mallya et al. , 2018 ) . Second , regularization-based methods constrain the update of parameters that are important to the learned tasks to retain previous knowledge ( Li & Hoiem , 2017 ; Kirkpatrick et al. , 2017 ; Aljundi et al. , 2018 ) . Third , memorybased methods keep a number of key samples from previous tasks in memory to alleviate forgetting ( Lopez-Paz & Ranzato , 2017 ; Chaudhry et al. , 2018 ; d ’ Autume et al. , 2019 ) . These methods for LL mostly focus on tasks of the same type ( referred as domains in this work ) . Recently , Sun et al . ( 2019 ) proposes LAMOL , a general framework designed for lifelong language learning ( LLL ) , where the model needs to continually learn from different domains as well as different types of NLP tasks . 2.2 FEW-SHOT LEARNING . Few-shot learning ( FL ) aims to learn tasks with a few labeled examples . Due to the scarcity of labeled training data , FL faces the problem of over-fitting . Existing methods to overcome over- fitting include : ( i ) model-based methods that explore how to reduce the hypothesis space of the fewshot task ( Triantafillou et al. , 2017 ; Hu et al. , 2018 ) , ( ii ) data-based methods that try to augment additional data to the few-shot set ( Benaim & Wolf , 2018 ; Gao et al. , 2020b ) , and ( iii ) algorithmbased solutions that aim to improve strategies for searching for the best hypothesis . Recently , a new paradigm introducing prompts achieves promising results for few-shot language learning as shown by GPT-3 ( Brown et al. , 2020 ) , PET ( Schick & Schütze , 2020 ) and LM-BFF ( Gao et al. , 2020a ) . 2.3 PROMPT-BASED LEARNING . Brown et al . ( 2020 ) first show that a GPT-3 frozen model can achieve impressive few-shot results through manually designed prompts that provide a natural language description of the task . Since then many efforts have been made on prompt-based learning ( PL ) . In general , PL modifies the original input , often adding a task-specific template or prompt , which usually contains some unfilled slots to let a pre-trained language model probabilistically generate a textual response , from which the final model output can be derived ( Liu et al. , 2021b ) . The ongoing research on PL has explored ( i ) methods of prompt designing , including discrete prompts ( Schick & Schütze , 2020 ; Shin et al. , 2020 ; Tam et al. , 2021 ) and continuous or soft prompts ( Li & Liang , 2021 ; Liu et al. , 2021c ; Lester et al. , 2021 ) , ( ii ) applications of PL ( Han et al. , 2021 ; Ben-David et al. , 2021 ; Ding et al. , 2021 ) , and analysis of prompt-based learning ( Liu et al. , 2021a ; Le Scao & Rush , 2021 ; Zhong et al. , 2021 ) . Summary . Existing work in lifelong language learning aims to learn from a stream of NLP tasks with plenty of training data , while the research in few-shot learning explores how to generalize well on few-shot tasks . In contrast , we focus on a more challenging yet more practical problem lifelong few-shot language learning ( LFLL ) , where the model is expected to continually learn from a stream of few-shot tasks while avoiding overfitting on the new task and forgetting of previously acquired knowledge . We regard LFLL as an important step towards general language intelligence and propose LFPT5 which takes full advantage of the strong few-shot learning ability of prompt tuning . 3 METHODOLOGY . In this section , we first formally define the LFLL problem with the two different adaption dimensions of domains and tasks , and then illustrate how we reframe all types of tasks considered in this work into a text-to-text format in T5 . Finally , we present the details of our framework LFPT5 . 3.1 PROBLEM FORMULATION . As shown in Figure 1 , we identify two different dimensions of LFLL : learning of new tasks that are of the same type but potentially of different domains ( STDD ) , and learning of new tasks that are of different types ( DT ) . Specifically , STDD involves learning from a stream of domains D = ( D1 , . . . , Dn ) that belong to the same type of few-shot task T , such as NER learning from CoNLL03 ( Sang & De Meulder , 2003 ) and subsequently from OntoNotes ( Hovy et al. , 2006 ) . Each task domain Dk has its own training set Sktrain , validation set Skvalid , and test set Sktest . After the training on Sktrain , the model is expected to perform well on all the k domains that it has learned so far and will be evaluated with the same evaluation metric ( s ) on the combined test set Ŝktest = ∪ki=1Sitest . Different from STDD , in the DT dimension , the model is expected to continually learn from a sequence of different types of few-shot tasks T = ( T 1 , . . . , T m ) , such as learning of NER ( sequence labeling ) , then text classification , and subsequently text summarization ( generation ) . After learning of T k , the model will be evaluated on the test set Sitest of every learned task T i separately for 1 ≤ i ≤ k as the evaluation metrics for different kinds of tasks might be different . In both dimensions of LFLL , we assume that the validation set Skvalid has the same size as the fewshot training set Sktrain , that is , |Skvalid| = |Sktrain| . The set up of using a few-shot validation set is aligned with the overall goal of generalizing well on new tasks with limited labeled data .

Summary: The paper defines the problem of lifelong few-shot language learning (LFLL) where the goal is to continuously learn a model with new few-shot tasks without forgetting the previous ones. Towards this problem, the paper introduces a prompt tuning-based framework that augments the pre-trained language model (T5) with continuous prompts (trainable prompt embeddings). Prompts are simultaneously optimized for task solving and data generation. During continual learning of the new domains, pseudo-examples of the previously seen domains are generated for episodic rehearsal and further KL-based consistency regularization is implemented to prevent drifting of the model. Lastly, for new tasks, separate prompts are appended with the existing prompts to enable the transfer of knowledge from previous domains/ tasks. The main contribution of this paper is to showcase the effectiveness of the recently proposed prompt tuning-based method for lifelong language learning. Empirically, this paper reports superior results over EWC and MAS methods when evaluated on text classification, NER, and summarization tasks.