Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020)

Abstract: Continual relation learning aims to continually train a model on new data to learn incessantly emerging novel relations while avoiding catastrophically forgetting old relations. Some pioneering work has proved that storing a handful of historical relation examples in episodic memory and replaying them in subsequent training is an effective solution for such a challenging problem. However, these memory-based methods usually suffer from overfitting the few memorized examples of old relations, which may gradually cause inevitable confusion among existing relations. Inspired by the mechanism in human long-term memory formation, we introduce episodic memory activation and reconsolidation (EMAR) to continual relation learning. Every time neural models are activated to learn both new and memorized data, EMAR utilizes relation prototypes for memory reconsolidation exercise to keep a stable understanding of old relations. The experimental results show that EMAR could get rid of catastrophically forgetting old relations and outperform the state-of-the-art continual learning models.

## introduction relation extraction aims at detecting relations between entities from text, e.g., extracting the relation "the president of " from the given sentence "newton served as the president of the royal society", which could serve as external resource for various downstream applications @xcite @xcite ., * indicates equal contribution † corresponding author 2018 ). the conventional re methods @xcite @xcite mostly focus on recognizing relations for a fixed pre-defined relation set, and cannot handle rapidly emerging novel relations in the real world. some researchers therefore explore to detect and learn incessantly emerging relations in an open scenario. as shown in figure 1 , their efforts can be formulated into a two-step pipeline: a straightforward solution is to store all historical data and re-train models every time new relations and examples come in. nevertheless, it is computationally expensive since relations are in sustainable growth. moreover, the huge example number of each relation makes frequently mixing new and old examples become infeasible in the real world. therefore, storing all data is not practical in continual relation learning. in view of this, the recent preliminary work @xcite indicates that the main challenge of continual relation learning is the catastrophic forgetting problem, i.e., it is hard to learn new relations and meanwhile avoid forgetting old relations, considering memorizing all the data is almost impossible. recent work @xcite @xcite has shown that the memory-based approaches, maintaining episodic memory to save a few training examples in old tasks and re-training memorized examples during training new tasks, are one of the most effective solutions to the catastrophic forgetting problem, especially for continual learning in nlp scenarios @xcite . however, existing memory-based models still suffer from an overfitting problem: when adapting them for continual relation learning, they may frequently change feature distribution of old relations, gradually overfit a few examples in memory, and finally become confused among old relations after long-term training. in fact, these memory-based methods are similar to long-term memory model of mammalian memory in neuroscience @xcite . although researchers in neuroscience are not clear about secrets inside the human brain, they reach a consensus that the formation of long-term memory relies on continually replaying and consolidating information @xcite @xcite , corresponding to the episodic memory and memory replay in continual learning models. yet later work @xcite @xcite in neuroscience indicates that reactivation of consolidated memory triggers a reconsolidation stage to continually maintain memory, and memory is easy to be changed or erased in this stage. to apply some reconsolidation exercises can help memory go through this stage and keep long-term memory stable. intuitively, the ex-isting memory-based models seem like continual memory activation without reconsolidation exercises, and thus become sensitive and volatile. inspired by the reconsolidation mechanism in human long-term memory formation, we introduce episodic memory activation and reconsolidation (emar) to continual relation learning in this paper. more specifically, when training models on new relations and their examples, we first adopt memory replay to activate neural models on examples of both new relations and memory, and then utilize a special reconsolidation module to let models avoid excessively changing and erasing feature distribution of old relations. as the core of relation learning is to grasp relation prototypes rather than rote memorization of relation examples, our reconsolidation module requires models to be able to distinguish old relation prototypes after each time memory is replayed and activated. as compared with pioneering explorations to improve episodic memory replay @xcite , with toughly keeping feature distribution of old relations invariant, emar is more flexible in feature spaces and powerful in remembering relation prototypes. we conduct sufficient experiments on several re datasets, and the results show that emar effectively alleviates the catastrophic forgetting problem and significantly outperforms the stateof-the-art continual learning models. further experiments and analyses indicate the reasons for the effectiveness of emar, proving that it can utilize a few examples in old tasks to reconsolidate old relation prototypes and keep better distinction among old relations after long-term training. ## related work the conventional re work, including both supervised re models @xcite @xcite @xcite @xcite @xcite @xcite and distantly supervised models @xcite @xcite @xcite @xcite , focuses on extracting predefined relations from text. yet in the real world, new relations are rapidly emerging, and it is impossible to train models with a fixed dataset once to cover all relations. hence, some researchers pay their attention to relation learning in various open scenarios, in order to detect and learn relations without pre-defined relation sets. as we introduced before, learning incessantly emerging relations consists of two important steps: open relation learning and continual relation learning. there have been many efforts for open relation learning, including pattern extraction @xcite @xcite @xcite @xcite @xcite , relation discovery @xcite , relation clustering @xcite @xcite , and data collection @xcite @xcite @xcite @xcite . however, for continual relation learning, there are still only some preliminary explorations for it. following continual learning setting 1 @xcite in machine learning, @xcite first explore continual relation learning. existing continual learning methods focus on three research directions: (1) consolidation-based methods @xcite @xcite @xcite which consolidate the model parameters important to previous tasks and reduce their learning weights; (2) dynamic architecture methods @xcite @xcite which dynamically expand model architectures to learn new tasks and ef-1 some work names it lifelong or incremental learning. fectively prevent forgetting old tasks. yet model size growing dramatically with increasing tasks makes these methods unsuitable for nlp applications; (3) memory-based methods @xcite @xcite @xcite remember a few examples in old tasks and continually learn them with emerging new tasks to alleviate catastrophic forgetting. among these methods, the memorybased methods have been proven to be the most promising for nlp tasks, including both relation learning @xcite and other nlp tasks @xcite . inspired by reconsolidation in human memory formation, we introduce episodic memory activation and reconsolidation (emar) to alleviate the overfitting problem of the existing memory-based methods and better learn relations continually. ## conclusion and future work to alleviate catastrophically forgetting old relations in continual relation learning, we introduce episodic memory activation and reconsolidation (emar), inspired by the mechanism in human long-term memory formation. compared with existing memory-based methods, emar requires models to understand the prototypes of old relations rather than to overfit a few specific memorized examples, which can keep better distinction among relations after long-term training. we conduct experiments on three benchmarks in relation extraction and carry out extensive experimental results as well as empirical analyses, showing the effectiveness of emar on utilizing memorized examples. for future work, how to combine open relation learning and continual relation learning together to complete the pipeline for emerging relations still remains a problem, and we will continue to work on it.