CITB : A Benchmark for Continual Instruction Tuning (2023)

Abstract: Continual learning (CL) is a paradigm that aims to replicate the human ability to learn and accumulate knowledge continually without forgetting previous knowledge and transferring it to new tasks. Recent instruction tuning (IT) involves fine-tuning models to make them more adaptable to solving NLP tasks in general. However, it is still uncertain how instruction tuning works in the context of CL tasks. This challenging yet practical problem is formulated as Continual Instruction Tuning (CIT). In this work, we establish a CIT benchmark consisting of learning and evaluation protocols. We curate two long dialogue task streams of different types, InstrDialog and InstrDialog++, to study various CL methods systematically. Our experiments show that existing CL methods do not effectively leverage the rich natural language instructions, and fine-tuning an instruction-tuned model sequentially can yield similar or better results. We further explore different aspects that might affect the learning of CIT. We hope this benchmark will facilitate more research in this direction.

## introduction recent studies have shown that multi-task instruction tuning (it) makes language models better zeroshot learners @xcite @xcite @xcite . it fine-tunes pre-trained language models (plms) on various tasks with natural language instructions (fig. 1 ) and can achieve remarkably well generalization to unseen tasks. despite their impressive performance, these instruction-tuned plms still fall short on domainspecific tasks due to the limited exposure to relevant knowledge and vocabulary from the training corpus @xcite . moreover, plms are static after deployment, and there is no mechanism to update themselves or adapt to a changing environment @xcite . continual learning (cl) aims to enable information systems to learn from a continuous data stream across time @xcite . therefore, it is promising to leverage cl for instruction-tuned plms to continually adapt to new domains and tasks without costly re-training. despite its importance, it is non-trivial to alleviate catastrophic forgetting, a phenomenon in which previously learned knowledge or abilities are degraded due to overwritten parameters @xcite . moreover, enabling knowledge transfer is also essential since many tasks are similar and have common knowledge @xcite . unfortunately, there is little work on applying cl for it and has only been explored in rather specific settings. @xcite continually fine-tune a t0 @xcite on eight new tasks with memory replay to avoid forgetting. despite effectiveness, they need to store large number of instances per task in memory, which is too costly when scaling to a larger number of tasks. in addition, they do not study knowledge transfer between tasks. @xcite propose to use history task instructions to reduce forgetting and enable knowledge transfer. however, they do not compare with commonly adopted cl methods, which makes the effectiveness of other cl methods unknown. moreover, they only evaluate the model on the newly learned tasks while largely ignoring previously learned tasks during the multi-task training stage (fig. 1 ). they also overlook the intrinsic ability of the instruction-tuned model on unseen tasks. lastly, both of them use different evaluation metrics and setups, which creates an obstacle to comparing different techniques and hinders the development of this field. to this end, we first formulate this practical yet under-explored problem as continual instruction tuning (cit). then, we propose a first-ever benchmark suite to study cit systematically. our benchmark, citb, consists of both learning and evaluation protocol and is built on top of the recently proposed superni dataset @xcite . we create two cit task streams: instrdialog stream, which consists of 19 dialogue-related tasks spanning three categories; instrdialog++ stream, which includes all the tasks in instrdialog stream and 19 additional tasks selected from broad categories and domains. using the two long task streams, we implement various cl methods to study forgetting and knowledge transfer under the setup of cit. we find that directly fine-tuning an instruction-tuned model sequentially yields competitive performance with existing cl methods. with further investigation, we find that rich natural language instructions enable knowledge transfer and reduce forgetting, which is barely fully leveraged by current cl methods. we conduct comprehensive experiments to explore what effects the learning of cit. we hope our citb benchmark will serve as a helpful starting point and encourage substantial progress and future work by the community in this practical setting. to summarize, our main contributions are: ## preliminaries instruction tuning (it). following previous studies @xcite @xcite , each task @xmath0, y t @xmath1, which can be split into the training d t train , validation d t dev and test sets d t test . each instance is filled into an instruction template such that different tasks can be transformed into a unified text-to-text format (fig. 2 ). it aims to learn a model f : i t × x @xmath2. in general, the model is first trained on a mixture of tasks ( t seen ) and then evaluated for its zero-shot generalization ability on held-out tasks (t unseen ), where t seen ∩ t @xmath3. the model is expected to learn to follow instructions via the training tasks and then solve new tasks with only the help of task instructions. ## continual instruction tuning benchmark in this section, we first formalize the cit problem (fig. 1 ). then, we present the learning and evaluation protocol of our framework citb. lastly, we describe the data for creating the benchmark. ## experiments using our citb benchmark, we conduct experiments on various popular cl methods of different kinds. we describe our experiment setups and compared methods in this section. ## results and analysis in this section, we report the performance of various baselines discussed in §5.2 on our benchmark. ## conclusion in this work, we establish a benchmark for continual instruction tuning, with two 19 and 38 long task streams to be learned sequentially. we implement and compare various continual learning methods of different types using the benchmark to study their effectiveness under this new domain. we conduct extensive ablation studies to analyze the lack of current practices, and propose a future direction.