Rational LAMOL : A Rationale-based Lifelong Learning Framework (2021)

Abstract: Lifelong learning (LL) aims to train a neural network on a stream of tasks while retaining knowledge from previous tasks. However, many prior attempts in NLP still suffer from the catastrophic forgetting issue, where the model completely forgets what it just learned in the previous tasks. In this paper, we introduce Rational LAMOL, a novel end-to-end LL framework for language models. In order to alleviate catastrophic forgetting, Rational LAMOL enhances LAMOL, a recent LL model, by applying critical freezing guided by human rationales. When the human rationales are not available, we propose exploiting unsupervised generated rationales as substitutions. In the experiment, we tested Rational LAMOL on permutations of three datasets from the ERASER benchmark. The results show that our proposed framework outperformed vanilla LAMOL on most permutations. Furthermore, unsupervised rationale generation was able to consistently improve the overall LL performance from the baseline without relying on human-annotated rationales.

## introduction the grounds of lifelong learning (ll) stem from the ability of humans to continually acquire, consolidate, and transfer knowledge and skills throughout their lifespan. this ability is also important for real-world natural language processing (nlp) applications, where autonomous agents are required to interact with users from various domains through continuous streams of information and language * equal contributions † corresponding author semantic drifts occur over time. the existing dominant paradigm for machine learning, however, is isolated learning @xcite . while isolated learning has shown some successes in a variety of domains, their applicability remains limited to the assumption that all samples are available during the learning phase. when a stream of tasks are trained sequentially, machine learning and neural network models face catastrophic forgetting or interference @xcite . this occurs due to the non-stationary data distribution that biases the model. we focus on lifelong language learning (lll), which is lifelong learning on a stream of nlp tasks. to the best of our knowledge, the grounds of lll are left largely underexplored. lamol is an lll general framework that has garnered recent interest due to its simplicity @xcite . in particular, lamol transforms all nlp tasks into the question answering (qa) format according to @xcite and generates pseudo-samples of old tasks using its language modeling (lm) capability to refresh the learned knowledge. however, there is still a gap between the performance of lamol and the result of multi-task learning which is generally considered as the upper bound of lll performance. this indicates that only pseudo-samples generation may not be sufficient to prevent catastrophic forgetting. in this paper, we improve existing lll strategies by proposing rational lamol, a rationale-based lifelong learning framework which equips the original lamol with critical freezing @xcite to further prevent catastrophic forgetting. particularly, we devise an algorithm to identify critical components in transformer-based language models using rationales, and the selected compo-nents will be frozen to maintain learned knowledge while being trained on a new task. the contributions of our paper are listed below: we evaluated rational lamol on six task order permutations of three datasets from the eraser benchmark @xcite . the results show that our proposed framework outperformed the original lamol on five out of the six permutations, achieving average improvements of 1.83% with a lower standard deviation of 4.57%. moreover, using unsupervised rationale generation instead of human rationales also yielded competitive performance, achieving average improvements of 2.67% from original lamol. ## background and related work in this section, we briefly introduce the concept of lifelong learning, catastrophic forgetting, and component freezing which are relevant to the core idea of rational lamol. we also briefly summarize prominent researches related to rationales. ## methodology we introduce rational lamol and its detailed implementation in this section. as rational lamol is based from lamol @xcite , we briefly explain lamol in section 3.1. then we introduce the core lifelong learning framework of rational lamol in section 3.2. this is followed by two proposed enhancements including critical component identification and unsupervised rationale generation, detailed in section 3.3 and 3.4, respectively. ## results and discussion this section reports the performance of rationale lamol and compares it with lamol as the baseline as well as multitask learning, which is considered as the upper bound of ll. we also analyze the effect of each component in the proposed framework. ## conclusion to effectively retain learned knowledge in ll for nlp tasks, we proposed rational lamol, a learning framework that uses rationales to identify and freeze the most critical components of the model while being trained on a new task. we showed that rational lamol is able to outperform lamol by a significant margin. furthermore, our framework can be applied to any nlp datasets by leveraging unsupervised rationale generation, eliminating the need for human rationales while maintaining comparable improvements. overall, rational lamol bridges the gap between ll in nlp with model understanding through rationales, exhibiting potential for a true lifelong language learning as well as limiting catastrophic forgetting.