LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5

Existing approaches to lifelong language learning rely on plenty of labeled data for learning a new task , which is hard to obtain in most real scenarios . Considering that humans can continually learn new tasks from a handful of examples , we expect the models also to be able to generalize well on new few-shot tasks without forgetting the previous ones . In this work , we define this more challenging yet practical problem as Lifelong Few-shot Language Learning ( LFLL ) and propose a unified framework for it based on prompt tuning ( PT ) of T5 . Our framework called LFPT5 takes full advantage of PT ’ s strong few-shot learning ability , and simultaneously trains the model as a task solver and a data generator . Before learning a new domain of the same task type , LFPT5 generates pseudo ( labeled ) samples of previously learned domains , and later gets trained on those samples to alleviate forgetting of previous knowledge as it learns the new domain . In addition , a KL divergence loss is minimized to achieve label consistency between the previous and the current model . While adapting to a new task type , LFPT5 includes and tunes additional prompt embeddings for the new task . With extensive experiments , we demonstrate that LFPT5 can be applied to various different types of tasks and significantly outperform previous methods in different LFLL settings . 1 INTRODUCTION . A hallmark of human intelligence is that they can learn new tasks quickly by leveraging previously acquired knowledge from other related tasks , and they do so without forgetting prior knowledge . However , despite the monumental success of deep learning in recent years , models face challenges to retain and accumulate knowledge when learning new tasks due to the shift of data distribution – they run into the overfitting issue when the data for the new task is small and they forget prior knowledge , a phenomenon known as catastrophic forgetting ( McCloskey & Cohen , 1989 ) . Researchers in Lifelong Learning ( Thrun & Mitchell , 1995 ) have proposed a number of methods to alleviate the above issues with machine learning . When it comes to language , earlier approaches to Lifelong Language Learning ( LLL ) merely focus on a single type of NLP tasks ( Wang et al. , 2019 ; d ’ Autume et al. , 2019 ) ; see ( Biesialska et al. , 2020 ) for a survey . In contrast , humans can easily handle tasks that vary with respect to not only domain but also task type ( Figure 1 ) . More recent methods attempt to learn from different types of tasks . These include LAMOL ( Sun et al. , 2019 ) and its improvements ( Chuang et al. , 2020 ; Sun et al. , 2020 ; Kanwatchara et al. , 2021 ) . Despite the effectiveness of these methods in LLL , there are several limitations . First , they all assume plenty of training data for every task which is hard to acquire in most real scenarios as getting large labeled datasets is often expensive and time-consuming . Second , they mainly consider tasks from the decaNLP challenge ( McCann et al. , 2018 ) that can be easily framed as question answering ( Kumar et al. , 2016 ) , paying little attention to sequence labeling tasks such as Name Entity Recognition ( NER ) . Finally , they fine-tune the entire model for all tasks ignoring the possibility of negative transfer ( Lopez-Paz & Ranzato , 2017 ) between different types of tasks . Our work in this paper aims to address these limitations of LLL . We focus on a more challenging yet more practical problem where the model needs to generalize well on new few-shot tasks without forgetting the previous ones . We regard this as Lifelong Few-shot Language Learning ( LFLL ) and investigate three different kinds of tasks : sequence labeling tasks , text classification tasks and text generation tasks . Based on the strong few-shot learning ability of prompt tuning ( Lester et al. , 2021 ) of T5 ( Raffel et al. , 2019 ) , we propose a unified framework for LFLL , named LFPT5 ( Lifelong Fewshot Language Learning with Prompt Tuning of T5 ) . Specifically , we reframe all types of tasks into a text-to-text format ( Figure 2 ) . To continually learn new domains of a task , we simultaneously train the prompt embeddings designed for this task type as a task solver and a data generator keeping the backbone T5 frozen . When LFPT5 goes about learning a new domain , it first generates pseudo labeled samples of previously learned domains , which are then combined with the new domain training data to alleviate catastrophic forgetting . To achieve label consistency between the previous and the current model , LFPT5 also minimizes a KL divergence loss . For the adaptation from one task type to another , LFPT5 includes additional prompt embeddings for the new task , and tunes them similarly . In this way the learning of new tasks minimally affects previously acquired knowledge , mitigating the catastrophic forgetting problem . In the whole learning process , the pre-trained T5 acts as a meta-learned model ( Brown et al. , 2020 ) that is kept frozen , while the tunable soft prompt acts as a task or domain adaptation model . In summary , our main contributions are : • To the best of our knowledge , we are the first to consider LFLL , a challenging yet practical problem . We propose LFPT5 , a unified framework for LFLL based on prompt tuning of T5 . LFPT5 can generalize well on various new few-shot tasks without severe forgetting of previously acquired knowledge , which can be seen as an important step towards general language intelligence . • With extensive experiments and analysis , we demonstrate that LFPT5 outperforms previous baselines by a large margin . We have open-sourced our code base at https : //github.com/ qcwthu/Lifelong-Fewshot-Language-Learning . 2 RELATED WORK . 2.1 LIFELONG LEARNING . In lifelong learning ( LL ) , the model is expected to learn sequentially from a stream of tasks with different data distributions . The main problem in LL is catastrophic forgetting ( McCloskey & Cohen , 1989 ) – the model forgets previously acquired knowledge after learning a new task . Prior approaches to LL can be divided into three categories . First , architecture-based methods dynamically adjust the model architecture to learn new knowledge while preventing the forgetting of previously learned tasks ( Chen et al. , 2015 ; Rusu et al. , 2016 ; Mallya et al. , 2018 ) . Second , regularization-based methods constrain the update of parameters that are important to the learned tasks to retain previous knowledge ( Li & Hoiem , 2017 ; Kirkpatrick et al. , 2017 ; Aljundi et al. , 2018 ) . Third , memorybased methods keep a number of key samples from previous tasks in memory to alleviate forgetting ( Lopez-Paz & Ranzato , 2017 ; Chaudhry et al. , 2018 ; d ’ Autume et al. , 2019 ) . These methods for LL mostly focus on tasks of the same type ( referred as domains in this work ) . Recently , Sun et al . ( 2019 ) proposes LAMOL , a general framework designed for lifelong language learning ( LLL ) , where the model needs to continually learn from different domains as well as different types of NLP tasks . 2.2 FEW-SHOT LEARNING . Few-shot learning ( FL ) aims to learn tasks with a few labeled examples . Due to the scarcity of labeled training data , FL faces the problem of over-fitting . Existing methods to overcome over- fitting include : ( i ) model-based methods that explore how to reduce the hypothesis space of the fewshot task ( Triantafillou et al. , 2017 ; Hu et al. , 2018 ) , ( ii ) data-based methods that try to augment additional data to the few-shot set ( Benaim & Wolf , 2018 ; Gao et al. , 2020b ) , and ( iii ) algorithmbased solutions that aim to improve strategies for searching for the best hypothesis . Recently , a new paradigm introducing prompts achieves promising results for few-shot language learning as shown by GPT-3 ( Brown et al. , 2020 ) , PET ( Schick & Schütze , 2020 ) and LM-BFF ( Gao et al. , 2020a ) . 2.3 PROMPT-BASED LEARNING . Brown et al . ( 2020 ) first show that a GPT-3 frozen model can achieve impressive few-shot results through manually designed prompts that provide a natural language description of the task . Since then many efforts have been made on prompt-based learning ( PL ) . In general , PL modifies the original input , often adding a task-specific template or prompt , which usually contains some unfilled slots to let a pre-trained language model probabilistically generate a textual response , from which the final model output can be derived ( Liu et al. , 2021b ) . The ongoing research on PL has explored ( i ) methods of prompt designing , including discrete prompts ( Schick & Schütze , 2020 ; Shin et al. , 2020 ; Tam et al. , 2021 ) and continuous or soft prompts ( Li & Liang , 2021 ; Liu et al. , 2021c ; Lester et al. , 2021 ) , ( ii ) applications of PL ( Han et al. , 2021 ; Ben-David et al. , 2021 ; Ding et al. , 2021 ) , and analysis of prompt-based learning ( Liu et al. , 2021a ; Le Scao & Rush , 2021 ; Zhong et al. , 2021 ) . Summary . Existing work in lifelong language learning aims to learn from a stream of NLP tasks with plenty of training data , while the research in few-shot learning explores how to generalize well on few-shot tasks . In contrast , we focus on a more challenging yet more practical problem lifelong few-shot language learning ( LFLL ) , where the model is expected to continually learn from a stream of few-shot tasks while avoiding overfitting on the new task and forgetting of previously acquired knowledge . We regard LFLL as an important step towards general language intelligence and propose LFPT5 which takes full advantage of the strong few-shot learning ability of prompt tuning . 3 METHODOLOGY . In this section , we first formally define the LFLL problem with the two different adaption dimensions of domains and tasks , and then illustrate how we reframe all types of tasks considered in this work into a text-to-text format in T5 . Finally , we present the details of our framework LFPT5 . 3.1 PROBLEM FORMULATION . As shown in Figure 1 , we identify two different dimensions of LFLL : learning of new tasks that are of the same type but potentially of different domains ( STDD ) , and learning of new tasks that are of different types ( DT ) . Specifically , STDD involves learning from a stream of domains D = ( D1 , . . . , Dn ) that belong to the same type of few-shot task T , such as NER learning from CoNLL03 ( Sang & De Meulder , 2003 ) and subsequently from OntoNotes ( Hovy et al. , 2006 ) . Each task domain Dk has its own training set Sktrain , validation set Skvalid , and test set Sktest . After the training on Sktrain , the model is expected to perform well on all the k domains that it has learned so far and will be evaluated with the same evaluation metric ( s ) on the combined test set Ŝktest = ∪ki=1Sitest . Different from STDD , in the DT dimension , the model is expected to continually learn from a sequence of different types of few-shot tasks T = ( T 1 , . . . , T m ) , such as learning of NER ( sequence labeling ) , then text classification , and subsequently text summarization ( generation ) . After learning of T k , the model will be evaluated on the test set Sitest of every learned task T i separately for 1 ≤ i ≤ k as the evaluation metrics for different kinds of tasks might be different . In both dimensions of LFLL , we assume that the validation set Skvalid has the same size as the fewshot training set Sktrain , that is , |Skvalid| = |Sktrain| . The set up of using a few-shot validation set is aligned with the overall goal of generalizing well on new tasks with limited labeled data .

Summary: This paper proposes to leverage prompt tuning on large pre-trained language models to achieve Lifelong Few-shot Language Learning (LFLL). Lifelong learning and few-shot learning have been considered different ML paradigms. The authors argue that with the emergence of modern large-scale pre-trained language models, AI models now have the capability to achieve these 2 at the same time. This work attempts to formally define the LFLL problem and benchmarked a strong pre-trained LM (T5) on 3 tasks (NER, text classification, and summarization) over 9 different datasets (defined as domains in LFLL).