Continual Machine Reading Comprehension via Uncertainty-aware Fixed Memory and Adversarial Domain Adaptation (2022)

Abstract: Continual Machine Reading Comprehension aims to incrementally learn from a continuous data stream across time without access the previous seen data, which is crucial for the development of real-world MRC systems. However, it is a great challenge to learn a new domain incrementally without catastrophically forgetting previous knowledge. In this paper, MA-MRC, a continual MRC model with uncertainty-aware fixed Memory and Adversarial domain adaptation, is proposed. In MA-MRC, a fixed size memory stores a small number of samples in previous domain data along with an uncertainty-aware updating strategy when new domain data arrives. For incremental learning, MA-MRC not only keeps a stable understanding by learning both memory and new domain data, but also makes full use of the domain adaptation relationship between them by adversarial learning strategy. The experimental results show that MA-MRC is superior to strong baselines and has a substantial incremental learning ability without catastrophically forgetting under two different continual MRC settings.

## introduction recently, machine reading comprehension (mrc) has attracted wide attention and achieved remarkable success when solving specific tasks in stationary environments, such as answering factual questions with wikipedia articles or answering narrative questions with web search logs @xcite @xcite . however, the answering scenario changes over time in real-world applications. for example, the dialog system should continuously adapt to new user requirements @xcite . in this paper, we focus on one of the most typical scenario changes for mrc tasks: the domain data shift. existing * hua xu is the corresponding author. stationary-trained mrc systems are usually trained with in-domain data but are applied to new domain data @xcite . therefore, it is necessary to build a non-stationary mrc that continually learns with incremental domain data. we formulate such a challenging problem as continual mrc task, which is required to incrementally learn over sequential domain data and perform well on all seen domain data. figure 1 illustrates the incremental learning and testing processing. in this scenario, the mrc system can only trained on the latest coming domain data without access the previous seen data. to tackle this issue, if we directly fine-tune the mrc model on each new incoming domain, the performance on earlier domains will significantly drop @xcite . another naive approach is to retrain the whole mrc model from scratch, but it is costly and time-consuming. hence it is a great challenge for incrementally learning without largely forgetting previously acquired knowledge. existing studies for continual mrc can mainly be divided into three categories. the first class is model expansion techniques that design domainindividual classifier for each in coming domain @xcite . however, it is expensive and unpractical in real-world. the second class borrows the regularization idea, which utilizes an ad-ditional loss term to aid knowledge consolidation when learning new domains. for example, @xcite added a penalty that restricts the change of important parameters to prevent forgetting previous knowledge. the third class is episodic memory based methods. for example, de masson d' @xcite introduced a key-value memory module that stored previously seen examples for sparse experience replay and gradient-based local adaptation. @xcite proposed templatebased never-ending kb-qa that learned new templates by capturing new syntactic structures with a semantic similarity function between questions and user feedback. however, the above methods still have limitations, mainly including two aspects. on the one hand, to prevent catastrophic forgetting, these methods only design consistent constraints of model output or gradient for previous and new domains, while ignoring the domain adaptation relationship between them. however, transfer learning can help the mrc model generalize to other domains. on the other hand, the memory update strategy for continual mrc is limited. some previous work stores fixed examples for each incoming domain. it greatly grows the number of samples kept in memory and leads to expensive costs. other methods that limit the maximum number of memory for all seen domains usually update the memory by random sampling, ignoring the forgotten degree of different samples. in fact, the continual model should pay more attention to samples that are more likely to be forgotten. to handle the above limitations, this paper proposes ma-mrc, an incremental model that solves continual mrc task via uncertainty-aware fixed memory and adversarial domain adaptation. concretely, ma-mrc 1) introduces a fixed-size memory to store a small number of samples in previous domain, which are later periodically replayed when learning new domain; and 2) updates the memory with an uncertainty-aware strategy that takes the forgotten degree of previous data into account; 3) leverages an adversarial learning strategy to make full use of the domain adaptation relationship between different domains with a domain discriminator, so as to help generalize and avoid overfitting very small memorized examples. the intuition behind this is to mimic the human learning process that replays the memory and adapts to new domains. the key contributions of this work are: (1) this paper proposes a continual mrc model, ma-mrc, which learns new domain data incrementally. (2) applying uncertainty-aware memory and adversarial learning and to mrc model contributes to strong incremental learning ability. (3) the experimental results on two different continual mrc settings indicate that ma-mrc obtaines good incremental learning ability without largely forgetting and significantly outperforms strong baselines. ## conclusion in this paper, an incremental learning mrc model with uncertainty-aware fixed memory and adversarial domain adaptation, ma-mrc, is proposed for continual mrc and alleviating catastrophically forgetting. inspired by the human learning process, there are two main ideas of ma-mrc: a memory that stores a small number of samples in previous seen domain data and always focuses on what the model most needs to replay; and adversarial learning the domain adaptation in a two-player game to learn better transfer representations between previous and current domain data. experimental results show that the proposed ma-mrc can achieve a good continuous learning performance without catastrophically forgetting under cda-c and cda-q settings. in the future, we would like to explore a more effective sampling strategy, domain adaptation strategy, and balance training strategy for multiple objectives to enhance the continual mrc model.