LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5
Authors: Chengwei Qin
Shafiq Joty
Venue: ICLR 2022 Poster (2022)
Keywords: lifelong few-shot language Learning
prompt tuning
pseudo samples
knowledge distillation

Abstract: Existing approaches to lifelong language learning rely on plenty of labeled data for learning a new task, which is hard to obtain in most real scenarios. Considering that humans can continually learn new tasks from a handful of examples, we expect the models also to be able to generalize well on new few-shot tasks without forgetting the previous ones. In this work, we define this more challenging yet practical problem as Lifelong Few-shot Language Learning (LFLL) and propose a unified framework for it based on prompt tuning of T5. Our framework called LFPT5 takes full advantage of PT's strong few-shot learning ability, and simultaneously trains the model as a task solver and a data generator. Before learning a new domain of the same task type, LFPT5 generates pseudo (labeled) samples of previously learned domains, and later gets trained on those samples to alleviate forgetting of previous knowledge as it learns the new domain. In addition, a KL divergence loss is minimized to achieve label consistency between the previous and the current model. While adapting to a new task type, LFPT5 includes and tunes additional prompt embeddings for the new task. With extensive experiments, we demonstrate that LFPT5 can be applied to various different types of tasks and significantly outperform previous methods in different LFLL settings.

Published as a conference paper at ICLR 2022
LFPT5: A UNIFIED FRAMEWORK FOR LIFELONG FEW-SHOT LANGUAGE LEARNING BASED ON PROMPT TUNING OF T5
Chengwei Qin and Shafiq Joty**
Nanyang Technological University
Salesforce Research
{chengwei w0w.gggwii03ee.e..mm 0030e ntu, srjotyentu).edu.ss
ABSTRACT
Existing approaches lifelong language learning rely plenty of labeled data for learning a new task, which is hard to obtain in most real scenarios. Consid- ering that humans can continually learn new tasks from handful of examples, we expect the models also to be able to generalize well on new few-shot tasks without forgetting the previous ones. In this work, we define this more challeng- ing yet practical problem as Lifelong Few-shot Language Learning (LFLL) and propose unified framework for based on prompt tuning (PT) of T5. Our frame- work called LFPTS takes full advantage of PT's strong few-shot learning ability, and simultaneously trains the model as a task solver and data generator. Before learning
domain of the same task type, LFPT5 generates pscudo (labeled) samples of previously learned domains, and later gets trained on those samples to alleviate forgetting of previous knowledge it learns the new domain. In addi- tion, KL. divergence loss is minimized to achieve label consistency between the previous and the current model. While adapting to a new task type, LFPT5 in- cludes and tunes additional prompt embeddings for the new task. With extensive experiments, we demonstrate that LFPT5 can be applied various different types of tasks and significantly outperform previous methods in different LFLL settings.
INTRODUCTION
A hallmark of human intelligence is that they can learn new tasks quickly by leveraging previously acquired knowledge from other related tasks, and they do so without forgetting prior knowledge. However, despite the monumental success of deep learning in recent years, models face challenges to retain and accumulate knowledge when learning new tasks due to the shift of data distribution they into the overfitting issue when the data for the new task is small and they forget prior knowledge, a phenomenon known as catastrophic forgetting (MeCloskey & Cohen 1989).
Researchers in Lifelong Learning (Thrun & Mitchell 1995) have proposed number of methods to alleviate the above issues with machine learning. When it comes to language, earlier approaches to Lifelong Language Learning (LLL) merely focus
single type of NLP tasks (Wang al.) 2019
d'Autume et al. 2019); see (Biesialska et al. (020) for survey. In contrast, humans can easily handle tasks that vary with respect to not only domain but also task type (Figure Tp. More recent methods attempt to learn from different types of tasks. These include LAMOL (Sun et al. 2019) and its improvements (Chuang et al., 2020 Sun et al.] 20200 Kanwatchara et al. 2021). Despite the effectiveness of these methods in LLL, there are several limitations. First, they all assume plenty of training data for every task which is hard to acquire in most real scenarios as getting large labeled datasets is often expensive and time-consuming. Second, they mainly consider tasks from the decaNLP challenge (McCann et al. 2018) that can be easily framed as question answering (Kumar et al. 2016), paying little attention sequence labeling tasks such Name Entity Recognition (NER). Finally, they fine-tune the entire model for all tasks ignoring the possibility of negative
transfer (Lopez-Paz & Ranzato 2017) between different types of tasks.

---

Published as a conference paper at ICLR 2022
Our work in this paper aims address these limitations of LLL. We focus more challenging yet more practical problem where the model needs to generalize well on new sho-shot tasks without forgetting the previous ones. We regard this Lifelong Few-shot Language Learning (LFLL) and investigate three different kinds of tasks: sequence labeling tasks, text classification tasks and text generation tasks.
Based on the strong few-shot learning ability of prompt tuning (Lester et al. 2021) of TS (Raf- Tel et al. 2019). we propose unified frame- work for LFLL, named LFPTS (Lifelong Few- shot Language Learning with Prompt Tuning of T5). Specifically, we reframe all types of tasks into a text-to-text format (Figure 2 To contin- ually learn new domains of task, we simulta- neously train the prompt embeddings designed for this task type as a task solver and a data gen- erator keeping the backbone T5 frozen. When LFPTS goes about learning a new domain, first generates pseudo labeled samples of pre- viously learned domains, which are then com- bined with the new domain training data al- leviate catastrophic forgetting. To achieve label consistency between the previous and the cur- rent model, LFPT5 also minimizes a KL diver- gence loss. For the adaptation from one task type to another, LFPTS includes additional prompt embeddings for the new task, and tunes them similarly. In this way the learning new tasks minimally affects previously acquired knowledge, mitigating the catastrophic forgetting problem. In the whole learning process, the pre-tree pre-trained T5 acts as meta-learned model (Brown et al. 2020) that is kept frozen, while the tunable soft prompt acts task domain adaptation model. In summary,
Domain
NER
Fales Compunte
4GHen
Anik
Adthiia
Classification
Type
CINDY
MaV
Riw
Xuan
Summarization
NEWE
Question Answering
Figure
Two different dimensions lifelong lan- guage learning. The horizontal axis (Domain) indicates tasks of the same type (e.g. NER), whereas the vertical axis (Task) indicates different kinds of tasks.
main contributions are:
To the best our knowledge, we are the first to consider LFLL, challenging yet practical prob- lem. We propose LFPT5, unified framework for LFLL based on prompt tuning of T5. LFPTS can generalize well on various new few-shot tasks without severe forgetting of previously acquired knowledge, which can be seen an important step towards general language intelligence.
With extensive experiments and analysis, we demonstrate that LFPTS outperforms previous base- lines by a large margin. We have open-sourced our code base at gcwthu/Li few.ng--o-g-eoo wwwpggg-ewwhh----eeeee Lanquage---e Learning
2
RELATED WORK
2.1
LIFELONG LEARNING
In lifelong learning (LL), the model is expected to learn sequentially from a stream of tasks with dif- ferent data distributions. The main problem in LL is catastrophic forgetting (MeCloskey & Cohen 1989) the model forgets previously acquired knowledge after learning new task. Prior approaches to LL can be divided into three categorics. First, architecture-bate
methods dynamically adjust the model architecture to learn new knowledge while preventing the forgetting of previously learned tasks (Chen et al. 2013 Rusu et al. 2016 Mallya et al. 2018). ). Second, regularization- based meth- ods constrain the update of parameters that important to the learned tasks to retain previous knowledge (Li & Hoiem] 2017: Kirkpatrick et al. 2017 Aljundi et al.) 2018). Third, memory- based methods keep a number key samples from previous tasks in memory to alleviate forgetting [Lopez-Paz & Ranzato 2017 Chaudhry et al. 2018 d' Autume et al. 2019). These methods for LL (2019) proposes LAMOL, a general framework designed for lifelong language learning (LLL), where the model needs to continually learn from different domains as well as different types of NL.P tasks.
mostly focus on tasks of the same type (referred as domains in this work). Recently, /Sun
2.2 FEW-SHOT LEARNING
Few-shot learning (FL) aims to learn tasks with a few labeled examples. Due to the scarcity of labeled training data, FL. faces the problem of over-fitting. Existing methods overcome over-
2

---

Published as a conference paper at ICLR 2022
fitting include: (i) model-based methods that explore how to reduce the hypothesis space of the few- shot task (Triantafillou et al., 2017: Hu et al.] 2018), (ii) data-bacc basa- methods that try to augment additional data to the few-shot set (Benaim & Wolf 2018: Gao et al.] [222b),, and (iii) algorithm- based solutions that aim to improve strategies for searching for the best hypothesis. Recently, a new paradigm introducing prompts achieves promising results for few-shot language learning as shown by GPT-3 [Brown et al., 2020), PET (Schick & Schütze 2020) and LM-BFF (Gao et al. 2020a).
2.3
PROMPT-BASED LEARNING
Brown et al. (2200) first show that GPT-3 frozen model can achieve impressive few-shot results through manually designed prompts that provide a natural language description of the task. Since then many efforts have been made prompt-based learning (PL.. In general, PL modifies the original input, often adding a task-specific template or prompt, which usually contains some unfilled slots let a pre-trained language model probabilistically generate a textual response, from which the final model output can be derived (Liu et al. (2021b). The ongoing research on PL has explored (i) methods of prompt designing, including discrete prompts (Schick & Schütze [020; [Shin al.)
2020
Tam et al. 2021) and continuous or soft prompts (Li & Liang. |02
Liu et al. c0 Lester
jt al. 2021), D. (ii) applications of Pt. (Han al.
2021
Ben-David et al. 2021
Ding et al. |02] ) and
analysis of prompt-based learning (Liu et al.) 2021a Le Scao & Rush 2021
Zhoog et al. 2021).
Summary, Existing work in lifelong language learning aims to learn from a stream of NLP tasks with plenty of training data, while the research in few-shot learning explores how to generalize well on few-shot tasks. In contrast, we focus on a more challenging yet more practical problem lifelong few-shot language learning (LFLL), where the model is expected to continually learn from stream of few-shot tasks while avoiding overfitting on the new task and forgetting of previously acquired knowledge. We regard LFL.L an important step towards general language intelligence and propose LFPTS which takes full advantage of the strong few-shot learning ability of prompt tuning.
3 METHODOLOGY
In this section, we first formally define the LFLL problem with the two different adaption dimensions of domains and tasks, and then illustrate how we reframe all types of tasks considered in this work into a text-to-text format in T5. Finally, we present the details of our framework LFPT5.
3.1 PROBLEM FORMULATION
As shown in Figure we identify two different dimensions of LFLL: learning of new tasks that are of the same type but potentially of different domains (STDD), and learning of new tasks that are of different types (DT). Specifically, STDD involves learning from a stream of domains D (D) (D...... Dn) that belong to the same type of few-shot task T, such as NER learning from CoNLLO3 (ang De Meulder 2003) and subsequently from OntoNotes (Hovy et al.. 2006). Each task domain DDd has its own training set Sha validation set S% and test set S After the training on S the model is expected to perform well on all the k domains that it has learned so far and will be evaluated with the same evaluation metric(s) on the combined test set S U/= L
Different from STDD, in the DT dimension, the model is expected to continually learn from a sequence of different types of few-shot tasks = (T1 ,..,, TT),, such as learning of NER (sequence labeling), then text classification, and subsequently text summarization (generation). After learning of T the model will be evaluated on the test Si of every learned task 7 separately for 1 i < k as the evaluation metrics for different kinds of tasks might be different.
In both dimensions of LFLL, we assume that the validation shot training set She that is, Is aligned with the overall goal of generalizing well on new tasks with limited labeled data.
Sk Valid has the same size as the few- |5 The set up of using few-shot validation set is
3.2 LIFELONG FEW-SHOT LANGUAGE LEARNING WITH PROMPT TUNING OF T5 (LFPTS)
Without loss of generality, let Dusk denote the training dataset for any new few-shot task and D Dpe denote a large-scale pre- treiiraining dataset. Our goal is to learn a model ø for the task. Formally,
arg max log pl (6) Drask, Dpre) R arg max [log (rack, 0) + log (ogg(000 ((
(1)
3

---

Published as a conference paper at ICLR 2022
NER
Classification
Summarization
Daro for And salter
Dota reck sulver
Date for fosk selver
Original dats
bepal:
Original dats
Sepas to alock Ocigaat incetort.
Original dkie
lapas piis moco
Ueppe The Dook poace
Iapa: Tims ne CTD ptAiple
haper The leei unm myfiing
leper Ie acrmmt
D.C
M
bapac (13,) and
Jipet:CCDi disli Orspert seriiiinng
512344
lapuc GEN, cars)
Auplee ci Tie
betwreffthe wd
Pate for grosesstr
Dist far gearronor
There for grosranar
Figure 2: Task formulation for Named Entity Recognition (NER), classification and summarization.
where 9 is prior pre-trained model, more specifically, point estimate of the pre- pre-trained model (see A.1 The adaptation task for LFLL thus boils down to solving: arg max, log pl o Puskk 9). Tradi- tronally, this has been done through fine-tuning A. However, fine-tuning the entire model effectivelyy on small few-shot tasks could be challenging and may lead to overfitting (Howard & Ruder 2018).
Brown et al. 2020) show that large- pre-trained model (a frozen GPT-3) can act black-box meta-learner (Chen et al. 2017) and yield impressive few-shot performance via manually designed prompts constructed with task descriptions and some canonical examples. As model size continues to increase (often in billions), it is indeed more appealing to have single generalist model to perform multiple different tasks simultancussly rather than having a separate copy for each task. However, as Lester et al. (2021) pointed out manual prompt engineering may have several key limitations including the human labor involved in the design process which can also be subjective and error- prone, and its rigidness with respect to the maximum sequence length supported by the model. Furthermore, the manual design assumes knowing the task in advance, which limits its applicability to lifelong learning where the next task to learn may not be known in advance.
In our work for LLFL,, we adopt the idea of prompt tuning proposed by Lester et al. (022I) We freeze the pre-trained model 0 and prepend series of tunable tokens P. parameterized by Φ (namely, prompt embeddings), to the input sequence and optimize log p(o) Pask f) through gradient descent. We use T5 (Raffel et al. 2019) the pre-trained meta model, and the prompt embeddings are initialized with the embeddings drawn from the vocabulary T5.
Prompt tuning is simple yet effective approach for learning many tasks it only requires learning a small number of prompt embeddings for each task. In addition, as the prompt embeddings can condense the signal from the training data and exploit the huge amount of meta knowledge contained in the frozen T5 model, prompt tuning also shows impressive results in few-shot learning. These two advantages naturally make prompt tuning a good choice for LFL.
3.2.1 TASK FORMULATION & ADAPTATION
We consider three typical task types in NLP: sequence labeling (e.g., NER), text classification and text generation (e.g., summarization). Inspired by (Raffel et al. 2019 Lester et al., 2021), we re- frame all tasks into a text-to-text format as shown Figure 2 We denote the input text as X and the output text as Y. The training objective for a task with dataset Dinkk = (X), Yi). .......... (X,, Y.)):
Clask = log p(ol Dizsk: 0) = Σ log logr(Y(( 0)
(2)
Where P are the prompt tokens pre-pended to the input and 6 denote their embeddings. Wang al. 2019) show that memory-based methods where the model preserves some key samples from previous tasks in memory to overcome forgetting, are more effective for lifelong learning in NLP than the other two kinds, architecture and regularization based methods (I.).. Instead of using an external memory module, we tune our task prompts such that the model simultaneously acts as task solver and a generator. The generation capability allows the model to generate pseudo samples of previously learned tasks that the current model can use "refresh" its prior task knowledge.
When training as a task solver, the model learns to decode the output text (Y) after reading the original input text (X). We call this input-output format TASK format. For sequence labeling, the output text is split into segment-labee pairs by a special token ":', and the text segment and its label in pair separated by another special token T. For classification, a natural language description as the output text, e.g., converting the review score 5 into 'wonderful' for sentiment analysis. For text generation, simply use the target text the output text.
convert the original label into
4

---

Published as a conference paper at ICLR 2022
Genenating
Learning, from affferent devailss
Pow
treining
dyfirost tyes
Typel
Type2
Type3
Bi
diregenc
pre-taltodd (iaore)
predratin
presrulard (froow
13 (freccs)
Aonule?
Poodo
Sonal skse
Driade Ale
dhomas
dhreppene
dee
Alc
pre-truised 13 (hoonn)
presrained T5 (frggey
predraiteed 13 (fhoomn)
Donaler
nober & generate
Demalel sufeer generater
safter & geserator
Figure 3: Illustration of the learning process of LFPTS for different task domains and task types. For learning new domains, LFPTS simultancously trains the prompt embeddings as task solver and data generator. When domain comes, it first generates pseudo samples of previous domains which will be combined with new data for training mitigate the forgetting of learned knowledge. achieve label consistency between the previous and current model. To learn new task type, LFPTS includes and tunes additional prompt embeddings for the new task while keeping the previous embeddings frozen.
KL. divergence loss is also optimized to
When training as data generator, the model learns to generate X as well as Y given a task-specinf generation token as input; we call this GEN format. We use different generation tokens for different types of tasks and different domains to guide the model to generate pseudo samples for a specific task, such as GGEN_nerl' for CONLL NER, "GEN_ner2" for OntoNotes NER and 'GEN_ claass for AGNews classification. In addition, we insert one special token "ppli___" between X and Y. During inference, the generated pseudo samples which do not contain this special token are discarded. The data generation language modeling (LM) loss can be expressed as:
C Σ
log
Where G is a spek-sp generation token added the prompt P. The training objective with the TASK and LM losses becomes: L. = ti + Am Clm. where Amm is the weight of the LM loss.
Figure 3 iillustrates the complete learning process of LFPT5 for new domains and task types.
Adapting New Domains Before learning on a new domain D LFPTS first generates pseudo
samples (X, Y) of previous domains D ..... D using the corresponding generation token in the imput prompt, which will be replayed later to alleviate forgetting of learned knowledge. To achieve label consistency on the pseudo samples, we also minimize a KL divergence loss between
the previous and current models for the output tokens. More formally,
CKL
(VL. (,)1:, X.)
(4)
where m is the number of pseudo samples. t is the number of tokens in Y/. V is the T5 vocabulary and øf the prompt embeddings of the previous model.
The overall loss that LFPT5 optimizes for adapting to new domains is: L.g = Crack + Cim Au k where Au is the weight of KL divergence loss.
Adapting New Task Types In order to learn a new task type Ta while not forgetting the acquired 1, we include an additional set of prompt tokens for the new task and fine-tune their embeddings while keeping the old ones frozen. This indeed an instance of dynamically expandable network (Yoon et al.) 2017). where each task type has its own dedicated prompt. The embedding of the new prompt tokens can initialized with the learned embeddings from previous task to avoid forgetting and to have better task prior. We define 300 tunable tokens per task prompt, meaning that we only add about 0.04% of the parameters of the pretrained T5 when learning new task type. Compared with previous lifelong learning frameworks which fine-tune the entire model for all tasks ignoring the negative transfer between different types of tasks, LFPT5 shows significant superiority, and it can also achieve better results than multitask learning ($4.))
knowledge of previous tasks T..... Tk-1,
5

---

be used compare the Published as a conference paper at ICLR 2022
4 EXPERIMENTS
4.1 EXPERIMENT SETUP
Tasks, Datasets and Metries Three different types of tasks are investigated in our work: NER as an instance of sequence labeling, text classification, and summarization as an instance of text gener- ation. For NER, we use CONLLO03 (Sang & De Meulder 2003) and OntoNotes (Hovy et 2006) as different domains. For classification, we conduct experiments on four different datasets/domains: AGNews for news classification (Zhang et al., 2015). Amazon Review for sentiment analysis (McAuley et al., 2015), DBPedia for Wikipedia article classification into topics (Lehmann et al.) 2013), and Yahoo for QA categorization (Zhang et al. 2015). The datasets for summarization in- clude CNNDM containing CNN/DM news (Nallapati et al. 2016). WikiHow containing how-to instructions (Koupace& & Wang 2018 and Xsum containing BBC news (Narayan et al. 2018).
We conduct 16-shot learning for NER and classification based on Gao et al. (2020a), Lee. there are 16 samples per class in the training and validation sets. For summarization, we sample 64 examples for training and validation per domain (see for details). For pseudo data, LFPT5 generates 2 samples per learned class for NER and classification, and 4 samples per learned domain for summarization. The evaluation metries of NER, classification and summarization are F1, accuracy and ROUGE scores, respectively. As the task order and few-shot data might influence the performance, we run every experiment times with different random seeds and report the mean results.
Methods Compared We use Ta-Large as the backbone model and compare our LFPT5 with the following methods in the experiments for learning new domains of a task:
Fine-tuning (FT) tunes the whole T5 model during the I.FLL process. We include this method as fine-tuning is still the dominant paradigm in NL..
Prompt tuning (PT) continually tunes the prompt embeddings while learning different do- mains. PT does not include LM and KL objectives and does not generate pseudo samples.
EWC KKrrkpttrick et al., 2017) and MAS (Aljundi et al., 2018) are two regularization-basee lifelong learning methods requiring no extra memory. They constrain the update of parameters that are important to the learned tasks to retain previous knowledge. We apply these two methods both PT and FT, and get four distinct methods: EWC-PT, MAS-PT, EWC-FT and MAS-FT.
Prompt tuning with real data (PT-R) selects the same number of randomly selected real samples from the learned domains the generated pseudo samples in LFPTS. These samples are used as memory data which is replayed during the learning of the new domain. PT-R resembles a 'real' memory-basee based LFLL model with prompt tuning and its performance quality of the pseudo samples generated by LFPT5.
Multitask prompt tuning (MT-PT) simultaneously trains on all the domains together with the combined data. It serves as an upper bound for LFPTS which can use only the new domain data.
In addition, we report experiments with a different backbone model (T-Basc),, different numbers of few-shot data and different number of pseudo samples in Appendix A.4 A.5 and A.6 respectively.
For adapting to new task types, we compare LFPT5 with multitask fine-tuning (MT-FT), MT-PT and AdapterFuse Fusion (Pfeiffer et al. 2021) which learns task-specific composition of adapters from previous tasks.
4.2 SINGLE TASK RESULTS
To assess the learning ability of prompt tuning, we first compare single task few-shot results for T5 fine-tuning (T5-FT), T5 prompt tuning (TS-PT) and BERT-Large fine-tuning on NER and classifica-
NER
Text classification
Method
CoNLL03
OntoNotes
AGNews
Amazon
DBPedia
Yahoo
SoTA (full-shot)
94.6
92.07
95.55
65.83
99,38
77.62
BERT-Large
62.67+1134
63.55
82.33+1.60
40.47++1.90
97. 0.61
59.97
TS-FT
53.74=1.20
55.15
83.17:42.0
48.80
98.19 19.19
50.07 84
TS-PT
68.40+1.24
61.23+2.14
$5.33+1.05
43.73
97.36
65.67
Table 1: Results on single few-shot tasks on NER (F1 score) and text classification (accuracy).

---

Published as a conference paper at ICLR 2022
tion in Table | while Figure 4 shows the comparison between TS-FT and T5-PT on summarization. We also report the state-of thte--of-hhe-rt art (SoTA) results for the original full-shot training for each task.
We can that the performance of T5-PT is quite good compared with BERT-Large and T5-FT. T5- FT overfits on several few-shot tasks (CoNLLO3, OntoNotes and Yahoo) and achieves poor re- sults. PT significantly improves these results as it requires to tune only the prompt embed- dings. particular, T5-PT achieves better results than fine-tuned BERT- in all cases except OntoNotes NER. Similarly, on summarization, T5-PT achieves better performance than T5-FT in all measures across the datasets except ROUGE- I WikiHow. These results suggest that PT has the potential for LFLL if we can solve the catas- trophic forgetting problem well.
60-
500
SOTA ROUGE-1
200
BY ROUGE-1
PT ROUGE
SOTA ROUGE-
ROIGE-1
PT ROUGGE-
50
SOTA ROUGE-
BY ROUGE-L
PT RORGGEE
30
20
0
CNNDM
Wikillow
Xsum
Figure 4: Results for T5 prompt tuning (PT) and T5
fine-tuning (FT) on summarization (ROUGE scores).
4.3 RESULTS FOR LEARNING NEW DOMAINS
NER The LFLL results on the NER domains are shown in Table 2 We report the final Fl score on the whole test set after learning all domains. We observe that EWC and MAS achieve slightly better results than simply fine-tuning the parameters, meaning the catastrophic forgetting problem is still severe. LFPTS outperforms these two regularization-base lifelong learning methods by a large margin, which demonstrates the superiority of our method.
Method
Fine-tuning
EWC- Fine-tuning
MAS-Fine-tuning
Prompt tuning- Real
MT-Prompt tuning
F1 43.07
43.53
43.63+1.5
48.72+0.9
54.32=0.88
Method
Prompt tuning
EWC Prompt tuning
MAS Prompt tuning
LFPTS
F1
44.34.0.40
44.68+14
45. 45.09
47.59
Table Fl score on the whole test set after learning NER domains (CoNLLO33, OntoNotes).
Comparing the results of PT- and FT-based methods, we can find that PT-based methods show better performance, which can be interpreted by two factors: (i) PT has stronger ability than FT for few- shot learning of new domains. (ii) The knowledge of the two domains is not so difficult transfer from one to the other as there are some overlaps between the label spaces. So even if PT needs to continually learn knowledge from different domains with much fewer tunable parameters than FT, it can successfully do so and outperform FT. PT-R performs better than I.PT5, which means that the quality of generated pseudo samples could be further improved. In addition, there is performance gap between LFPTS and MT-PT, indicating there still remains room for improvement.
Method
Fine-tuning
EWC- Fine-tuning
Mis-Fine-u
Prompt Rening-Real
MT-Prompt tuning
Accuracy
40.11
40.60=s.02
40.79
67.23=1.8
76.08
Method
Prompt tuning
EWC Prompt tuning
MAS Prompi tuning
LFPTS
Accuracy
28.47
29.09
29.46ga.97
$2.71
Table 3: Accuracy on the whole set after learning all domains (AGNews, Amazon, DBPedia, Yahoo).
Text Classification Table 3 shows the classification results on the whole test set after learning the four domains. We can see that LFPTS achieves significant improvements compared with previous lifelong learning methods. For text classification, a significant difference from NER is that FT-based methods show much better performance than Pa-based methods. We analyse the reasons as follows. The label space of the four domains is quite different, which makes it hard to transfer knowledge across different domains. So retaining and accumulating knowledge during the learning of different domains is pretty challenging for the PT-based methods they have only few tunable parame- ters. Acquiring of new information can easily cause forgetting of previously learned knowledge. Compared with PT, there are much more tunable parameters in FT, improving its ability to accom- modate knowledge from different domains. Even though LFPTS is based on PT, it can overcome such limitations by learning remember consistently from its own generated pseudo samples.
In Appendix A.3 we additionally evaluate how LFPT5 performs compared to the baselines for a large number of different tasks (domains) by considering 5 NLI tasks and combine them with the
7

---

Published as a conference paper at ICLR 2022
original 4 classification tasks to form a longer sequence of 9 classification tasks. These results verify that LFPTS performs much better than previous baselines when learning from many tasks.
Fine-tuning
EWC- Fine-tuning
MAS-Fine-tuning
Prompe tuning-Real Real
MT-Prrompt taning
A-RG
a5
g1
15.76
17.48
19.78
Method
Pronupt taning
EWC-Prompt tuning
MAS- PoompP tuning
1.PTS
A-RG
15.67
15.85 .035
15.79 19.79
17.05 Lo.2
Table 4: Average of ROUGE-I, ROUGE-2 and ROUGE-L scores (A (A-RG) on the whole test set after learning all domains (CNNDM, WikiHow, XSum).
Summarization For summarization, we find that the generated pscudo summaries (that follow the generated pseudo source documents) are often ambiguous. This could be because summarization has a large search space and is often underconstrained task for the model as showed by Kryscinski et al. (2019). As the leading three sentences (a.k.a. Lead-3) already construct strong baseline for summarization (especially for news articles), use the leading three sentences of the generated document as its summary to form the pseudo data. From the results in Table we can see that PT- based methods achieve similar performance FT-based methods. This is different from NER and text classification, showing that the difficulty of transferring knowledge across different domains in summarization might be between that of NER and classification. Here also LFPT5 outperforms previous lifelong learning methods by a large margin.
Summary LFPTS achieves much better performance than previous lifelong learning methods on three different types of tasks, which verifies its effectiveness and strong generalization ability.
4.4 RESULTS FOR LEARNING NEW TASK TYPES
To investigate LFPT5's performance on learning new task types, we consider two different variants: (i) LFPT5 with FKT initializes the prompt embeddings of one task using the prompt embeddings of the previously learned task, which we regard as forward knowledge transfer (FKT), and (ii) LFPTS w.o. FKT initializes the prompt embeddings of every task with the embeddings drawn from the vocabulary of T5. For these experiments, we use CoNLLO3 for NER, AGNews for text classification and CNNDM for summarization. From the results in Table 5
we can observe the following:
Both variants of LFPT5 can achieve better performance than MT-FT and MT-PT. Multitask learn- ing simultaneously trains all tasks together. The learning of one task might cause negative effect on the learning of others. In contrast, LFPT5 variants include and tune additional prompt embed- dings for new types of tasks which avoids the negative cross-task knowledge transfer.
LFPTS performs better than AdapterFusion (Pfeiffer et al., 2021) which demonstrates its superi- ority. Moreover, LFPT5 is much more surameter-efficien than AdapterFusion. In the course of learning these three different task types, AdapterFusion introduces about 21.72% of the parame- of the pretrained T5, while LFPT5 only adds about 0.12%.
Comparing the two variants of LFPT5, the effect of forward knowledge transfer can be positive or negative, depending on the tasks. The forward knowledge transfer between classification and summarization is positive. However, they have negative effect NER; transferring knowledge from them to NER or from NER to them negatively affect the learning of the new task.
Task Order
Method
(i) Summ Class-NER
(ii) Class-NER-Sun Summ
(iii) VER-Summa Class
Multitask fine-tuning
23.24, 78.25, 57.81
81.50, 58.28, 21.28
50.21, 22.49, 82.25
Multitask prompt tuning
24.16, 85.50, 50.80
82.75, 65.31, 23.36
62.83, 11.51, 83.25
AdapterFusion LFPTS w.o. FKT
22.26, 83.25, 55.62
81.25, 63.19, 22.37
62.99, 21.20, 82.50
25.48. 84.75, 63.28
83.25, 67.66, 23.68
66.65, 22.97, 84.50
LFPTS with FKT
25.48, 86.00, 62.44
83,25, 65.01, 24.92
66.65, 22.80, 84.25
Table 5: Results for learning three different task types: NER (CoNLL), Classification (AGNews) and Summa- rization (CNNDM). The tasks are presented in three different orders with different few-shot samples (results shown in the same order). The metrics reported are Fl for NER, accuracy for Classification and Average- ROUGE for Summarization.
8

---

Published as a conference paper at ICLR 2022
Domain Order
Method
())
(i)
(ii)
Average
Dm-Amo Yahoo-ACi (Avg.)
DB-Amazon-AGG
(Avg.)
Yahon-Amazon-4e AG--B (Aug.)
Prompt tuing
00.00,
07.00,0...9
81.71 (18.88)
00.00, 00.00, 52.57, 64.57 (24.85)
00.71, 00.00, 00.00. 97.86 (41.67)
28.47 a..00
PWCppppppp tuning
00.00. 00.00, 10.86,
(2.11(11979))) (19.79)
00.00. 00.00. 56.14, 68. (26.36)
00.00. 00.00. 00:00. 96,93 (11.12)
29.09
MAS-Promph tuming
00.00, 00.00, 12.57, 83.86 (20.4)
00.00, 00.00, 58,00, (0, 65,71 (26.24)
00.29, 00.00 00.57, 97.86 (41.70)
29. 29.462x007
Fine- tuning
(32.48)
21.36,
71.29 (37.09)
17,80,
e7.77
CWVC-Fine-une F-nn---tuning
38.21, 00.20, 21.00, 86,29 (39.00)
21.07, 01.00, 59.85 86, 68. (37.97)
00.14, 12.80, 05 86, 98.07 (14.82
10.60
MAS-Fine tuning
37.29. 00.60, 15.71, 83.20 (36.91)
20.79, 00.40, 61.14, 14, 67.00 (36.06)
01.00. 11.40, 27.57, 98.07 (49. 39)
40.70 40.79g5.00
Prompt tuning Real
85.79. 31.20, 43,14, 82.00 (67.67)
82.57. 37.40, 67.29, 29, 64.43 (4.
43:71,24.5 24,20, 58.71, 94.29 (65. .9)
67. 67.23, 67.23.2.56
LFPTS
32.13,
(47.64)
67.43 (52.58)
10.57,
59.69 39,56, (57.91)
52.71 24.39
MT MT--Prompt tuinn
95.000 47,40, 62.29, 75.57 (76.73)
93.57, 47.80, 73.86, 65.57 (76.52)
61.71, 43.00, 74.71, 93.21 (75.00)
76.08 60.77
Table 6: Text classification accuracy on the whole test set for three runs with different domain order.
5 ANALYSIS
Influence of Domain Order To evaluate the influence of domain orders when LFPT5 is learning different task domains, we show the results of three runs with different domain order on the clas- sification task in Table 6 We can see that the order of domains influences the performance of all methods a lot. For example, PT can achieve 41.67 accuracy on the third run while the accuracy of the first run is only 18.88. This phenomenon indicates that the difficulty of transferring knowledge from one domain to another might be quite different from that of the opposite transfer direction. Though the performance is affected by the order, LFPTS outperforms previous regularization-base lifelong learning methods by a large margin for all different orders (see Appendix A.8 for more analysis).
0.02
0.04
0.10
0.20
0.40
A-RG 16.27 16.41
17.05 0.82
17.11 10.30
16.97
16.26_oo
Table 7: Average Rouge (A-RG) score of LFPTS with different Aua on summarization.
Influence of KL Loss
To investigate the influence of the label consistency loss LKL (Eq. we conduct experiments with different Au on summarization. From the results in Table we observe that the model achieves the best A-RG score of 17.11 1 with Au = 0.10 and score of 16.26 with Ap=- 0.40. The performance of the variant without CKI. (i.e., Au 0) is worse than the performance of all other variants except the variant with An = 0.40 (too large), which demonstrates the effectiveness of CkI.
Quality of Pseudo Samples We show a few pseudo samples generated by LFPTS in Fig- 3 We can observe that LFPT5 can generate high-quality pseudo samples which are useful for remembering previous knowledge. How- ever, as shown in the right part of the figure, the label of generated data could also be incorrect, which explains the performance gap between LFPT5 and PT-R. In addition, there are several obvious errors, e.g., the pseudo data might not have the "_split_.' token or belong to the re- quired domain. We can automatically discard these samples. We believe that exploring methods to generate more reliable pseudo data should be a quite promising research direction in LFLL.
Harb-qually Dats
Lovegaally Dase
besutifol the wanderts
Nike, Namnuung unvell nrw mobil pleam The GonglelEh mobile plane, has beee ferigned with Nokia fann is miidd split
Euctnaaing.
FRANCISCO!
We We lacutlion
location
Figure 5: Examples of generated pseudo samples for text classification (lop) and NER (bottom).
Abbreviation Variations When learning NER, LFPT5 as a task solver needs to generate the en- tities in the original input (Figure 2). We observe an entity error related to abbreviation during the generation, such as generating 'the United States' while the original entity is 'U.S.'. This kind of error unfairly penalizes LFPT5's is F1 score, but it also indicates that T5 does not just copy words from the original input but thinks about the relevant knowledge and expresses it in its own way.
CONCLUSION
In this work, we introduce LFPTS, a unified framework for lifelong few-shot language learning (LFLL) where the model needs to generalize well on various new few-shot tasks without forgetting previous acquired knowledge. Extensive experimental results and analysis show that LFPTS can easily adapt to new types of tasks new domains while retaining the knowledge of learned tasks, which we regard as an important step towards general language intelligence. In the future, we would like to investigate ways improve the quality of generated pseudo samples.
9

---

Published as a conference paper at ICLR 2022
REFERENCES
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pp. 139-154, 2018.
Eyal Ben-David, Nadav Oved, and Roi Reichart. Pada: A prompt-based autoregressive approach for adaptation unseen domains. arXiv preprini arXiv:2 102. 12206, 2021.
Sagie Benaim and Lior Wolf. One-shot unsupervised cross domain translation. arXiv preprint wwXi::/806.0602, 2018.
Magdalena Bicsialska, Katarzyna Biesialska, and Marta R Costa-jussà, Continual lifelong learning in natural language processing: A survey, arXiv preprint arXiv:2012.0982. 2020.
Tom В Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners arXiv preprint arXiv:2005.1416. 2020.
Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint urXiv:1812.00420, 2018.
Tianqi Chen, Ian Goodfellow, and Jonathon Shlens. Net2net: Accelerating learning via knowledge transfer. arXiv preprint arXiv: /5/ / 1,05411, 2015.
Yutian Chen, Matthew W. Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P. Lilli- crap. Matt Botvinick, and Nando de Freitas. Learning learn without gradient descent by gra- dient descent. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning. volume 70 of Proceedings of Machine Learning Research, httthssss/pppseeeeeeeeee
pp. 748-756, PMLR, 06-11 Aug 2017. 1e.n17
ml mir.press/v/o.
Yung-Sung Chuang, Shang- Yu Su, and Yun-Nung Chen. Lifelong language knowledge distillation. arXiv preprint arXiv:2010.02112 2020.
Cyprien de Masson d' Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. Episodic memory in lifelong language learning. arXiv preprint 1rXiv:1906.010766 2019.
De Marneffe, Mandy Simons, and Judith Tonhauser. The commitmentbank: In- vestigating projection in naturally occurring discourse. In proceedings of Sinn und Bedeutung, volume 23, pp. 107-124, 2019.
Ning Ding, Yulin Chen, Xu Han, Guangwei Xu, Pengjun Xie, Hai-Tao Zheng, Zhiyuan Liu, Juanzi Li, and Hong-Gee Kim. Prompt-learning for fine-grained entity typing. arXiv preprint arXiv:2108. 10604, 2021.
Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012. 2020a.
Tianyu Gao, Xu Han, Ruobing Xie, Zhiyuan Liu, Fen Lin, Leyu Lin, and Maosong Sun. Neural snowball for few-shot relation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 7772-7779, 2020b.
Xu Han, Weilin Zhao, Ning Ding. Zhiyuan Liu, and Maosong Sun. Ptr: Prompt tuning with rules for text classification. arXiv preprint arXiv:2 1259, 2021.
Eduard Hovy, Mitch Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. Ontonotes: the 90% solution. Proceedings of the human language technology conference of the NAACL, Companion Volume: Short Papers, pp. 57-60, 2006.
Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 228-339, Mclbourne, Australia, July 2018. Association for Com- putational Linguistics. doi: 10.18653/v1/P1 1031. URL https: http//aaaannttgy..oooomo
P 1031
10

---

Published as a conference paper at ICLR 2022
Zikun Hu, Xiang Li, Cunchao Tu, Zhiyuan Liu, and Maosong Sun. Few-shot charge prediction with discriminative legal attributes. In Proceedings of the 27th International Conference on Computa- tional Linguistics, pp. 487-498, 2018.
Kasidis Kanwatchara, Thanapapas Horsuwan, Piyawat Lertvittayakumjon, Boonserm Kijsirikul, and Peerapon Vateekul. Rational lamol: A rationale-based lifelong learning framework. In Pro- ceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference Natural Language Processing (Volume I: Long Papers), pp. 2922-2922222 2021.
Tushar Khot, Ashish Sabharwal, and Peter Clark. Seitail: A textual entailment dataset from science question answering. In I'hirty-Second AAAI Conference on Artificial Intelligence, 2018.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabssa-Barwinska. et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3511-352, 2017.
Mahnaz Koupaee and William Yang Wang. Wikihow: A large scale text summarization dataset. arXiv preprint 1rXiv 2018.
Wojeiech Kryscinski, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, and Richard Socher. Neural text summarization: A critical evaluation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-UCNLP), pp. 540-551, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 1L118663/11DDI910551
URL https:
7/aclanthology; ow1p://-100/011111111111111
Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, and Richard Socher. Ask me anything: Dynamic memory networks for natural language processing. In International conference on machine learning, pp. 1378-1387. PMLR, 2016.
Teven Le Scao and Alexander M Rush. How many data points is prompt worth? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2627-2636, 2021.
Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer, et al. Dbpedia-a large- scale, multilingual knowledge base extracted from wikipedia. Semantic web, 6(2):167-195, 2015.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104 (atl::2144.0000 1r4.v22104.08611, 006691, 2021.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.000 0000000,,,,,, 2021.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pallern analysis and machine intelligence, 40(12):2935-294) 2017.
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt-3? arXiv preprint arXiv:2101.06804 2021a
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language pro- cessing. arXiv preprint arXiv:2107.13886 2021b.
Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. Gpt understands, too. arXiv preprint arXiv: 10385, 2021c.
David Lopez- Paaez-Paz Paz and Mare Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30:6467-6476, 2017.

---

Published as a conference paper at ICLR 2022
Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting single network multi- ple tasks by learning to mask weights. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 67-82, 2018.
Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Beraardi, Roberto Zam- parelli, et al. A sick cure for the evaluation of compositional distributional semantic models. In Lrec, pp. 216-223. Reykjavik, 2014.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-based rec- ommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pp. 43 52, 2015.
Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. The natural language decathlon: Multitask learning as question answering. arXiv preprint 1806vv8806.088
2018.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, al. Abstractive text summarization using sequence-to-sequeee rmns and beyond. arXiv preprint arXiv:1602.06025. 2016.
Shashi Narayan, Shay B Cohen, and Mirella Lapata. Don't give me the details, just the sum- mary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.081 08745, 2018.
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapter- Fusion: Non-destructive task composition for transfer learning. In Proceedings of the 16th Con- ference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 487-503, Online, April 2021. Association for Computational Linguistics. doi: 10. 18633v1v/v 2021.eacl-man
URL httpt:tt//// www.aaaannnoooooooooooooooooo org /elggg/2221cc2 2tt///2211...ac... -ew.lall-aas 39
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:19I0. 10683, 2019.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.0 1et..6606.467,, 2016.
Erik Sang and Fien De Meulder. Introduction to the con1l-2003 shared task: Language- independent named entity recognition. arXiv preprint cs/0306050, 2003.
Timo Schick and Hinrich Schütze. Exploiting cloze questions for few shot text classification and natural language inference. arXiv preprint arXiv:2001.07677 2020.
Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In International Conference on Machine Learning, pp. 4596-4604, PMLR, 2018.
Taylor Shin, Yasaman Razeghi, Robert Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.159800 2020.
Fan-Keng Sun, Cheng-Has Ho, and Hung- Yi Lee. Lamol: Language modeling for lifelong language learning. arXiv preprint 19X9.03900.03322 2019.
Jingyuan Sun, Shaonan Wang, Jiajun Zhang, and Chengqing Zong. Distill and replay for contin- ual language learning. In Proceedings of the 28th International Conference on Computational Linguistics, pp. 3569-3579, 2020.
Derek Tam, Rakesh Menon, Mohit Bansal, Shashank Srivastava, and Colin Raffel. Improving and simplifying pattern exploiting training. arXiv preprint arXiv:2 1955, 2021.
12

---

Published as a conference paper at ICLR 2022
Sebastian Thrun and Tom M. Mitchell.
Lifelong robot learning.
Robotics and Au-
tonomous Systems, 15(1):25-46, 1995.
ISSN 0921-8890. doi: http://doi.or/// 101660921-89901950000044 URL bttpp::/wwil sci encedirect com com/science/ The Biology and Technology Intelligent Au-
tonomous Agents.
Eleni Triantafillou, Richard Zemel, and Raquel Urtasun. Few-shot learning through an information retrieval lens, 2017.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.0744 2018.
Hong Wang, Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Shiyu Chang, and William Yang Wang. Sen- tence embedding alignment for lifelong relation extraction. arXiv preprint 2019.
Adina Williams, Nikita Nangia, and Samuel R Bowman. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv: 2017.
Jachong Yoon, Eunho Yang. Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01477 2017.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text clas- sification. Advances in neural information processing systems, 28:649-651 2015.
Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein. Meta-tuning language models to answer prompts better. arXiv preprint arXiv:2104.046770 2021.
A APPENDIX
A.1 DERIVATION OF EQUATION
Assuming D Ppee I. we can write:
J. [log [log ((8)Pask 0) + log p(O) )d0
arg max log Deak Dpee)= = arg max
arg max log 8) logg(()Dppe))
= with point estimate of
A.2 DERIVATION OF EQUATION 2
Ls - log ppppp((uss Diak, 0) = Log(p(DDnn Daskk ( 0) p(o)
log (P, á, 0) assuming uniform p(0).
A.3 LEARNING FROM A LARGE NUMBER OF DIFFERENT DOMAINS
To evaluate whether our method can perform better than the baselines when learning from large number of different domains, consider NLI tasks (GLUE-NNLI (Williams et al., 2017). Scitail (Koot et al. 2018), SICK Marelli et al. 2014), SuperGLUE-CB (De Marneffe et al. 2019) and GLUE-RTE Wang et al. 2018)) as classification and combine them with the original 4 classification tasks to form a long sequence of 9 classification tasks. We evaluate LFPT5, MAS- HAS-Prompp tuning and MAS-Fine-tunin on this long sequence. The accuracy after learning all tasks is shown in Table g From the results, we can observe that LFPT5 still performs much better than previous basclines when learning from large number of tasks.
13

---

Published as a conference paper at ICLR 2022
Method
LFPT5
MAS-Promp tuning
MAS-Fine-tuning
Accuray (%)
43.98 12.68
9.37
34.37
Table 8: Accuracy (%) of different methods after learning all 9 domains.
A.4 DIFFERENT BACKBONE MODEL
To investigate how the model scale affects the LFLL capability, we compare the performance of LFPT5, EWC-Prommp tuning and EWW-Fine-tuning summarization using T5-Base backbone. From the results in Table | we can observe that LFPTS performs much better than EWC-Prompi tuning. However, it is slightly worse than EWW-Fine-tuning. This consistent with the finding in Lester et al. (202T) that prompt tuning performs better when applied to larger pretrained language
models.
Method
LFPTS
EWC-Prompt tuning EWC-Fine-tuning
A-RG
10.93
13.05 +1.23
11.35
Table A-RG score different methods with T5-Bas backbone on summarization.
A.5 DIFFERENT NUMBERS OF FEW-SHOT DATA
We conduct experiments to compare the performance of LFPTS, EWC-Prompt tuning and EWC- Fine tuming with different numbers (16, 32) of few-shot data on summarization. The A-RG score shown in Table 10 From the results, we can see that LFPT5 consistently outperforms previous baselines with different numbers of few-shot samples.
Few-shot Number
LFPT5
EWC- EWC-Prompt tuning
EWC-Fine-tuning
10.44 15.58
14.16.0.42
13.75 12.35
32
14.30
14.78
Table 10: A-RG score of different methods with different numbers (16, 32) of few-shot data on summarization.
A.6 INFLUENCE THE NUMBER PSEUDO SAMPLES
As generating pseudo samples is feasible and cheaper, we can use any number of pseudo samples. We conduct experiments on summarization to analyze the influence of different numbers of pseudo samples. From the results in Table [ we can find that increasing the number of pseudo samples will not always improve the performance. The model achieves the best A-RG score 17.05 with pseudo samples.
A.7 MULTIPLE PROMPTS MULTITASK PROMPT TUNING
We use a single prompt for multitask prompt tuning in Table 5 which is different from LFPTS in model capacity. To better support our claim, we conduct multitask prompt tuning experiments using the same number of tunable tokens as LFPT5 (multiple prompts). The tunable tokens are shared among all three tasks. From the results Table [2] we can see that LFPT5 performs better than both multitask prompt tuning methods.
A.8 ANALYSIS ON PT-BASED EWC AND MAS
From the results in Table 6 we can observe that performance on previous tasks for PT-based EWC and MAS is almost 0. There could be two reasons:
There are only few tunable parameters prompt tuning, which is difficult for retaining and accumulating knowledge. So the learning of new knowledge from different domains more
14

---

Published as a conference paper at ICLR 2022
Number
8
16
A-RG
17.05 16.2820.95
16.80
16. 16.45
16.79=0.ed
Table 11: A-RG score of LFPTS with different numbers of pseudo samples summarization.
Task Order
Method
i) Summ-Class-NER
(ii) Class-NER NER--umm Summ
(iii
NER-Summ- Classs
Multitask prompt tuning (single prompt)
14.16, 85.50, 50.80
82.75, 65.31, 23.36
62.83, 11.51, 83.25
Multitask prompt tuning (multiple prompts)
21.01, 82.25, 58.59
83.00. 64,83, 22.73
63.23, 21.41. 84.00
LFPT5 w.o. FKT
25.48. 84.75, 63.28
83.25. 67.66. 23,68
66.65. 22.97. 84.50
LFPT5 with FKT
25.48. 86,00, 62.44
83.25. 65.01, 24.92
66.65. 22.80, 84.25
Table 12: Comparison results of single prompt multitask prompt tuning, multiple prompts multitask prompt tuning and LFPTS.
likely to cause the forgetting of previously learned knowledge. LFPT5 utilizes pseudo samples to alleviate this problem.
Few-shor language learning is more challenging, The model training is already sub-optimal even without lifelong learning. So the performance is relatively low.
A.9 DATASETS DETAILS
There are 4 and 18 classes in CoNLLO3 and OntoNotes, respectively. And the number of classes in AGNews, Amazon, DBPedia and Yahoo is 4, 5, 14 and 10, respectively. We sample 16 examples per class, which means that there least 64 samples in the training and validation sets. Therefore, we sample 64 examples for the training and validation set per domain (dataset) for summarization.
A. PARAMETER SETTINGS
We use the Adafactor (Shazeer & Stern 2018) optimizer with a learning rate of 0.5. For NER, we set Am and ARI to 0.10 and 0.03, respectively. For text classification, we adopt 0.25 and 0.01 for the loss weights Am and Axa. respectively. We set 0.10 for Am and 0.04 for ARa when learning summarization. Hyperparameter search is done the validation sets when comparing the single task few-shot results in Section 4.2
For EWC and MAS based methods, we conduct hyper-parameter search and report the optimal results. For AdapterFusion (Peiiffer et al.) 202I), we adopt the implementation from AdapterHub and use the default adapter settings for T5. The default bottleneck reduction factor is 16, i.e. the bottleneck size is 64. We adopt a learning rate of le-4 with Adam W and linear learning rate decay following the orginal AdapterFusion paper. All other and evaluation interval) are the same as LFPT5.
settings (such as batch size
15


Reviews:
[{"review_id": "fZXAtwibeLE", "reviewer": "Reviewer_PawA", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n=================================\nStrengths:\n  - Combining lifelong and few-shot learning is a new setting.\n  - Experiments contain 3 different tasks and each has datasets from different domains.\n  - Experiments are well-designed: many baselines are implemented to compare proposed method with traditional lifelong learning methods.\n  - Paper is well-written.\n\n=================================\nWeaknesses:\n  - The actual method is simple combination of existing ideas.\n  - The number of tasks and domains is minimal setting. To really become a benchmark to measure the progress of LFLL, more tasks/datasets will be needed.\n\n\n=================================\nAdditional Suggestions/Questions:\n  - Question Answering and NLI are also very important NLP problems and many datasets exists from different domains. Perhaps the authors can consider adding them to setup a more comprehensive benchmark.\n  - Perhaps adding more pre-trained LMs such as GPT-2 and different sizes of T-5. The community will be interested how the model type / scale affect the LFLL capability.\n  - How will the pseudo data generation amount affect the learning / forgetting performance? Since the data is generated by T-5, couldn't we generate as much as we want? Maybe adding another ablation on this would be a good idea.\n\n\n", "summary_of_the_paper": "This paper proposes to leverage prompt tuning on large pre-trained language models to achieve Lifelong Few-shot Language Learning (LFLL). Lifelong learning and few-shot learning have been considered different ML paradigms. The authors argue that with the emergence of modern large-scale pre-trained language models, AI models now have the capability to achieve these 2 at the same time. This work attempts to formally define the LFLL problem and benchmarked a strong pre-trained LM (T5) on 3 tasks (NER, text classification, and summarization) over 9 different datasets (defined as domains in LFLL).", "summary_of_the_review": "Overall I'm leaning positive about this paper. The proposed LFLL problem, in my opinion, is very important for the community to think about: how to adapt large-scale pre-trained LM to different tasks/domains with few samples? This paper sets up a strong baseline and benchmark for research along this direction. Although prompt tuning, KL regularization, and pseudo data generation are existing ideas, combining them and formulate this LFLL problem/benchmark is non-trivial contribution. I think this work has potential, and if the authors can try to add more tasks/datasets/baselines, it may create more impact to the community.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "_GDdjlacobG", "reviewer": "Reviewer_jNL2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper has several strengths that make it a promising submission:\n\n1. The idea of using prompt-tuning for life-long language learning is a promising direction and the methods proposed in the paper constitute a novel and effective way in bridging the gap between these 2 methodologies. The discussion on few-shot learning also clearly demonstrated the difference of the proposed methods and a major collection of prior work using prompt-tuning.\n\n2. The methods proposed in the paper are effective in utilizing the assumption of language modeling, such that tasks are formed as prompt-label pair sequences, and autoregressive sequence generation is used to anchor the model to previously learnt knowledge.\n\n3. The experimental setup is representative of actual uses of life-long language learning, with 3 different tasks included. Comparisons to fine-tuning, prompt tuning, and their combinations with elastic weight consolidation (EWC) and memory-aware synapses (MAS) are solid choices to demonstrate the effectiveness of the proposed methods. In most of these comparisons, the proposed methods achieved better results.\n\nThere are a few weaknesses that I hope the paper can address to improve the scoring:\n\n1. In the paragraph below equation 4, the loss form is an addition of 3 values: 1) the task loss L^{task}; 2) the language modeling loss L^{lm}; 3) label consistency loss L^{KL}. Both 1) and 2) are reasonable, but 3) is new in the paper and it is unclear why it is absolutely necessary. In particular, the paper should include an ablation study on a range of \\lambda_{KL}, not just a comparison between models with and without L^{KL}.\n\n2. Some statistics that are useful for understanding the propose methodology are missing in the experimental section: 1) it is probably beneficial to include some numbers on the state-of-the-art results on methods for the original tasks and datasets. They may not be fair statistics in a life-long scenario, but they are helpful for the readers to understand where the methods stand in a larger picture of machine learning methods. 2) For the domain order experiments in section 5, it could be useful to also include accuracy for each of the datasets separately. This will provide the readers with additional knowledge on the relationship between task ordering and individual task performance.\n\nFinally, some suggestions to the paper, but these will not affect the scoring:\n\n1. Remove the quote in page 1.\n\n2. The paper should include a brief introduction to the 2 regularization-based life-long learning methods that it compares with. These are elastic weight consolidation (EWC) and memory-aware synapses (MAS).\n\n3. Try to reduce the use of abbreviations in tables. For example, by typing fine-tuning and prompt-tuning instead of FT and PT, it could make the table much easier to read without the need to refer to the paper text.", "summary_of_the_paper": "The paper extended the use of prompt tuning to the scenario of life-long language learning. To prevent catastrophic forgetting, the paper proposes to generate pseudo samples as part of the training data when adapting to a new domain, and adding new set of prompt tokens when adapting to a new task. Using the T5 model, the paper demonstrated the effectiveness of the proposed methods using text classification, named entity recognition (NER), and summarization tasks. The method achieved better results for these tasks using the proposed life-long learning mechanisms, compared to fine-tuning, prompt tuning, and their combinations with regularization-based life-long learning methods such as elastic weight consolidation (EWC) and memory-aware synapses (MAS).", "summary_of_the_review": "Good novelty in extending prompt-tuning to life-long language learning using the generative properties of language models. Good experiments demonstrating the effectiveness of the methods in preventing catastrophic forgetting. Some parts of the method are not sufficiently studied in the experiments.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "KgLblNRI8H", "reviewer": "Reviewer_9Wdo", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n- The paper attempts to solve an important problem of LFLL in the context of the pre-trained Transformers. With the rise of larger models, it is important to consider the scalability of these models for lifelong learning. Given that this paper builds upon the recent developments in parameter-efficient transfer learning, the number of parameters in the proposed prompt tuning-based method would not grow dramatically with the number of tasks.\n- Empirical results showcase that the proposed approach is superior to existing methods like EWC and MAS when evaluated on the sequence of 2-4 domains/tasks.\n\n\nWeaknesses:\n\n- For the LFLL problem, this paper proposes to use an existing prompt tuning-based transfer learning method where the underlying model is frozen and a few additional task/domain-specific parameters are learned. Furthermore, it repurposes existing lifelong learning methods like deep generative replay [1], distillation-based methods [2], dynamically expandable network [3] as a unified framework when applied to prompt tuning. Thus, the contributions of this paper are marginally novel/ significant.\n- Another major limitation of this paper is that it completely ignores comparison with other parameter-efficient transfer learning methods like adapters [4]. Concretely, AdapterFusion [3] work proposes to learn a task-specific composition of adapters from previous tasks. How does the prompt tuning-based method compare with adapter fusion when applied to the LFLL problem? Is there a consensus that prompt tuning-based methods are better than adapters for few-shot learning? \n- By definition, the lifelong learning paradigm deals with a large number of tasks in sequence. However, the paper evaluates the proposed approach on the sequence of 2-4 tasks. So it is unclear whether the gains persist as we increase the number of tasks to 10-15 (towards a more realistic evaluation setup). As we increase the number of tasks, does it enable positive forward/ backward transfer or increase the chance of negative transfer?\n- Overall, the paper is written in good style. However, there are key experimental details that are missing. This paper defines prompts as a series of tunable tokens but then it is unclear how many tokens are defined per task prompt. This is important because as we scale the number of tasks, the input sequence length scales linearly, and with that self-attention scales quadratically. This issue limits the practical applicability of the prompt tuning-based method for lifelong learning.\n- What does it mean to use validation sets for hyper-parameter selection? Ideally, during lifelong learning, we do not know the sequence of tasks apriori. It is unfair to freeze a particular task order, then search for optimal hyper-parameters, and then report results on multiple runs with the same task order. At least, report the results on unseen task orders? Or search on a subset of disjoint tasks [5]. The paper should consider updating their results with a fair hyper-parameter selection strategy.\n\nAdditional clarification question(s):\n- The paper trains a single prompt for both task solving and data generation. It is unclear whether this is an optimal strategy when it comes to generating pseudo-samples? Did the paper try learning a separate prompt for task solving and data generation? Does it help in addressing the quality of generated pseudo-samples? Furthermore, how effective is the current pseudo-sample generation strategy? How many samples are discarded to get valid samples?\n- There are a few hyper-parameters that are set without any justification: 16-shot learning? How does the method perform with varying the number of training examples? How does the performance change with varying the number of pseudo-samples? Does it hurt if we have more pseudo-samples (generating them is feasible and cheaper)?\n- In terms of the multi-task experiments (MT-PT), do we have separate prompts or a single prompt being trained with all data? If it is a single prompt then is it fair to compare it with LFPT5 with more capacity? If there are multiple prompts then how does one ensure that there are dedicated prompts for each task?\n\n[1] Sun, Fan-Keng, Cheng-Hao Ho, and Hung-Yi Lee. \"LAMOL: LAnguage MOdeling for Lifelong Language Learning.\" International Conference on Learning Representations. 2020.\n\n[2] Li, Zhizhong, and Derek Hoiem. \"Learning without forgetting.\" IEEE transactions on pattern analysis and machine intelligence 40.12 (2017): 2935-2947.\n\n[3] Pfeiffer, Jonas, et al. \"AdapterFusion: Non-Destructive Task Composition for Transfer Learning.\" Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021. \n\n[4] Houlsby, Neil, et al. \"Parameter-efficient transfer learning for NLP.\" International Conference on Machine Learning. PMLR, 2019.\n\n[5] Chaudhry, Arslan, et al. \"Efficient Lifelong Learning with A-GEM.\" International Conference on Learning Representations. 2018.\n", "summary_of_the_paper": "The paper defines the problem of lifelong few-shot language learning (LFLL) where the goal is to continuously learn a model with new few-shot tasks without forgetting the previous ones. Towards this problem, the paper introduces a prompt tuning-based framework that augments the pre-trained language model (T5) with continuous prompts (trainable prompt embeddings). Prompts are simultaneously optimized for task solving and data generation. During continual learning of the new domains, pseudo-examples of the previously seen domains are generated for episodic rehearsal and further KL-based consistency regularization is implemented to prevent drifting of the model. Lastly, for new tasks, separate prompts are appended with the existing prompts to enable the transfer of knowledge from previous domains/ tasks. The main contribution of this paper is to showcase the effectiveness of the recently proposed prompt tuning-based method for lifelong language learning. Empirically, this paper reports superior results over EWC and MAS methods when evaluated on text classification, NER, and summarization tasks.", "summary_of_the_review": "Overall, I rate this paper marginally below the acceptance threshold. It is interesting to see the efficacy of prompt tuning for lifelong language learning. However, with the current experiments, it is unclear how they compare with adapters? Also given this paper unifies existing approaches and adapts them for prompt tuning setup, it is marginally significant/ novel. There are several open questions (see weakness section + additional clarification questions) that need to be answered before understanding the overall benefits of the proposed framework. I am looking forward to the author's response in the rebuttal period and will consider updating my scores accordingly.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "2VIxSZ4FXGu", "reviewer": "Reviewer_Ap2b", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nStrengths\n- Few-shot setting and lifelong learning are important settings.\n- I can't give any more strengths at this time because I had a hard time understanding the paper.\n\nWeaknesses (reasons to reject)\n- The main contribution of saying that LFPT5 \"can be seen as a vital step towards general language intelligence\" is surely an overclaim, right? If I ask 100 researchers, how many of them will see this paper as a vital step towards general language intelligence?\n- I did not understand why the proposed method is restricted to the few-shot setting. Do gains go away in the full data setting?\n- The experiments are OK but not probably not good enough to support the general claims about lifelong learning. E.g., it would be great to show more than 3 tasks.\n- I had a hard time understanding the paper, see below (and I will remove this comment from reasons to reject once it is clarified):\n\t- The key part that I did not understand is why prompt tuning isn't already a sufficient life-long learning method. My understanding of prompt tuning is that no model weights are modified, only the prompt is optimized, and optimized prompts are stored for each task. Then given a new task, a new prompt can be trained and stored. This works because the size of prompts is many orders of magnitude smaller than the size of the model. Maybe another way of asking this question is, how is the following different from regular prompt tuning: \"while adapting to a new task type, LFPT5 includes and tunes additional prompt embeddings for the new task\"?\n\t- Another part I do not understand is why pseudo samples need to be generated. Why can't you just use the prior examples?\n\t- I am missing why this idea of retraining on previous domains each time is a good one. Let's say we're up to 1000 tasks. How much weight does the new task get in this mixture? Won't this take a long time?\n\nWeaknesses (not reasons to reject)\n- The introduction could make it more clear why the second two limitations of LLL are indeed limitations. Why NER in particular, instead of other tasks that haven't been explored thoroughly? The negative transfer point needs some more evidence as well, as I believe the decaNLP challenge showed positive transfer, right?\n\nMinor comments\n- Is the finetuned BERT-Large a few-shot finetuned or with the full dataset?\n- PT is not defined at first use in the abstract\n- All the acronyms are pretty confusing\n- Typo: review socre 5", "summary_of_the_paper": "This paper explores lifelong few-shot language learning. Their framework trains the model as a task solver and data generator. They use pseudo data for new domains, and additional prompt embeddings for new tasks.", "summary_of_the_review": "This paper tackles an important task. I didn't understand why their method specifically applies to the few-shot scenario. I also struggled to understand why the proposed method solves lifelong learning over regular prompt tuning. One sentence in the paper seems like an overclaim to me.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work defines the new problem of lifelong few-shot language learning where the goal is to continually learn new few-shot tasks and use those to benefit future tasks while not forgetting previous tasks. With larger models, this is an important goal due to the cost of updating and retraining these models. The work also shows superiority to existing approaches like EWC and MAS. After the author's rebuttal, the experimental section is also thorough with evaluation on a good range of tasks and approaches such as adapters showing good results. While this setting appears simpler than the full lifelong-learning setting and the approach combines existing ideas, this work's contribution to the definition and thinking about this problem is valuable. However, the authors should more clearly state the advantages of their approach vs standard prompt tuning (with an emphasis of benefiting future tasks) since two reviewers seem caught up on this point. The other two reviewers comments were addressed by the rebuttal as they stated in their comments.
Decision: Accept (Poster)