Continual relation learning aims to continually train a model on new data to learn incessantly emerging novel relations while avoiding catastrophically forgetting old relations . Some pioneering work has proved that storing a handful of historical relation examples in episodic memory and replaying them in subsequent training is an effective solution for such a challenging problem . However , these memorybased methods usually suffer from overfitting the few memorized examples of old relations , which may gradually cause inevitable confusion among existing relations . Inspired by the mechanism in human long-term memory formation , we introduce episodic memory activation and reconsolidation ( EMAR ) to continual relation learning . Every time neural models are activated to learn both new and memorized data , EMAR utilizes relation prototypes for memory reconsolidation exercise to keep a stable understanding of old relations . The experimental results show that EMAR could get rid of catastrophically forgetting old relations and outperform the state-of-the-art continual learning models . The code and datasets are released on https://github.com / thunlp/ ContinualRE . Relation extraction aims at detecting relations between entities from text , e.g. , extracting the relation " the president of " from the given sentence " Newton served as the president of the Royal Society " , which could serve as external resource for various downstream applications ( Dong et al . , 2015 ; Xiong et al . , 2017 ; Schlichtkrull et al . , 2018 ) . The conventional RE methods ( Riedel et al . , 2013 ; Zeng et al . , 2014 ; Lin et al . , 2016 ) mostly focus on recognizing relations for a fixed pre-defined relation set , and can not handle rapidly emerging novel relations in the real world . Some researchers therefore explore to detect and learn incessantly emerging relations in an open scenario . As shown in Figure 1 , their efforts can be formulated into a two-step pipeline : ( 1 ) Open Relation Learning extracts phrases and arguments to construct patterns of specific relations , and then discovers unseen relation types by clustering patterns , and finally expands sufficient examples of new relation types from large-scale textual corpora ; ( 2 ) Continual Relation Learning continually uses those expanded examples of new relations to train an effective classifier . The classifier is trained on a sequence of tasks for handling both existing and novel relations , where each task has its own relation set . Although continual relation learning is vital for learning emerging relations , there are rare explorations for this field . A straightforward solution is to store all historical data and re-train models every time new relations and examples come in . Nevertheless , it is computationally expensive since relations are in sustainable growth . Moreover , the huge example number of each relation makes frequently mixing new and old examples become infeasible in the real world . Therefore , storing all data is not practical in continual relation learning . In view of this , the recent preliminary work ( Wang et al . , 2019 ) indicates that the main challenge of continual relation learning is the catastrophic forgetting problem , i.e. , it is hard to learn new relations and meanwhile avoid forgetting old relations , considering memorizing all the data is almost impossible . Recent work ( Shin et al . , 2017 ; Kemker and Kanan , 2018 ; Chaudhry et al . , 2019 ) has shown that the memory-based approaches , maintaining episodic memory to save a few training examples in old tasks and re-training memorized examples during training new tasks , are one of the most effective solutions to the catastrophic forgetting problem , especially for continual learning in NLP scenarios ( Wang et al . , 2019 ; d'Autume et al . , 2019 ) . However , existing memory-based models still suffer from an overfitting problem : when adapting them for continual relation learning , they may frequently change feature distribution of old relations , gradually overfit a few examples in memory , and finally become confused among old relations after long-term training . In fact , these memory-based methods are similar to long-term memory model of mammalian memory in neuroscience ( McClelland et al . , 1995 ; Bontempi et al . , 1999 ) . Although researchers in neuroscience are not clear about secrets inside the human brain , they reach a consensus that the formation of long-term memory relies on continually replaying and consolidating information ( Tononi and Cirelli , 2006 ; Boyce et al . , 2016 ; Yang et al . , 2014 ) , corresponding to the episodic memory and memory replay in continual learning models . Yet later work ( Nader et al . , 2000 ; Lee et al . , 2004 ; Alberini , 2005 ) in neuroscience indicates that reactivation of consolidated memory triggers a reconsolidation stage to continually maintain memory , and memory is easy to be changed or erased in this stage . To apply some reconsolidation exercises can help memory go through this stage and keep long-term memory stable . Intuitively , the ex-isting memory-based models seem like continual memory activation without reconsolidation exercises , and thus become sensitive and volatile . Inspired by the reconsolidation mechanism in human long-term memory formation , we introduce episodic memory activation and reconsolidation ( EMAR ) to continual relation learning in this paper . More specifically , when training models on new relations and their examples , we first adopt memory replay to activate neural models on examples of both new relations and memory , and then utilize a special reconsolidation module to let models avoid excessively changing and erasing feature distribution of old relations . As the core of relation learning is to grasp relation prototypes rather than rote memorization of relation examples , our reconsolidation module requires models to be able to distinguish old relation prototypes after each time memory is replayed and activated . As compared with pioneering explorations to improve episodic memory replay ( Chaudhry et al . , 2019 ; Wang et al . , 2019 ) , with toughly keeping feature distribution of old relations invariant , EMAR is more flexible in feature spaces and powerful in remembering relation prototypes . We conduct sufficient experiments on several RE datasets , and the results show that EMAR effectively alleviates the catastrophic forgetting problem and significantly outperforms the stateof-the-art continual learning models . Further experiments and analyses indicate the reasons for the effectiveness of EMAR , proving that it can utilize a few examples in old tasks to reconsolidate old relation prototypes and keep better distinction among old relations after long-term training . To alleviate catastrophically forgetting old relations in continual relation learning , we introduce episodic memory activation and reconsolidation ( EMAR ) , inspired by the mechanism in human long-term memory formation . Compared with existing memory-based methods , EMAR requires models to understand the prototypes of old relations rather than to overfit a few specific memorized examples , which can keep better distinction among relations after long-term training . We conduct experiments on three benchmarks in relation extraction and carry out extensive experimental results as well as empirical analyses , showing the effectiveness of EMAR on utilizing memorized examples . For future work , how to combine open relation learning and continual relation learning together to complete the pipeline for emerging relations still remains a problem , and we will continue to work on it .

Challenge: Storing histories of examples is shown to be effective for continual relation learning however existing methods suffer from overfitting to memorize a few old memories.
Approach: They propose a human memory mechanism inspired by memory activation and reconsolidation method which aims to keep a stable understanding of old relations.
Outcome: The proposed method mitigates catastrophic forgetting of old relations and achieves state-of-the-art on several relation extraction datasets showing it can use memorized examples.