SlowGuess/ACL_2020_OCR huggingface dataset:
(1) FewRel (Han et al., 2018b). FewRel is a RE dataset that contains 80 relations and 56,000 examples in total. We follow the settings from Wang et al. (2019) to make FewRel a continual learning benchmark: FewRel is split into 10 clusters of relations, leading to 10 tasks and each relation just belongs to only one task. Each example in these tasks is related to a relation and a candidate set of 10 randomly selected relations for evaluation.
(2) SimpleQuestions (Bordes et al., 2015). SimpleQuestions (SimpleQ) is a knowledge base question answering dataset that contains 108,442 questions, and Yu et al. (2017) construct a relation detection dataset based on it, where questions are linked to relations. Like FewRel, we follow the settings from Wang et al. (2019): SimpleQ is split into 20 clusters of relations to construct 20 tasks. As each question in SimpleQ has been related to a candidate set for evaluation, we do not randomly sample candidate sets again for SimpleQ.
(3) TACRED (Zhang et al., 2017). TACRED is a RE dataset that contains 42 relations and 21, 784 examples. Similar to FewRel, we also split TACRED into 10 clusters of relations to construct 10 tasks, and randomly sample candidate relation sets consisting of 10 relations for each examples. Considering there is a special relation "n/a" (not available) in TACRED, we filter out these examples with the relation "n/a" and use the left examples for continual TACRED.
(a) FewRel
(b) SimpleQuestions
(c) TACRED
Figure 3: Changes in accuracy  $(\%)$  with increasing tasks through the continual learning process.
FewRel	SimpleQ	TACRED
10	25	50	10	25	50	10	25	50
W	A	W	A	W	A	W	A	W	A	W	A	W	A	W	A	W	A
EWC	-	-	-	-	21.3	24.4	-	-	-	-	63.9	62.5	-	-	-	-	14.5	14.5
AGEM	29.0	34.0	33.8	39.0	41.2	47.5	69.1	66.1	72.2	69.2	76.2	73.1	14.7	14.5	15.0	15.5	15.7	16.0
EMR	42.0	54.1	49.0	60.5	53.6	65.1	81.5	77.4	84.9	81.0	86.9	82.9	21.8	26.5	25.7	31.6	28.7	35.6
EA-EMR	49.0	61.2	54.9	66.4	59.1	69.9	83.3	78.7	86.4	82.0	87.9	83.5	23.0	30.0	27.7	37.0	30.5	40.5
EMAR	53.8	69.1	62.5	74.9	66.0	77.9	80.9	78.7	84.6	81.4	85.2	83.7	31.0	36.3	37.8	48.5	44.5	54.4
Table 2: Accuracy (%) of models with different memory sizes. All the results come from our implemented models.
We use two evaluation settings including whole performance, which calculates the accuracy on the whole test set of all tasks, and average performance, which averages the accuracy on all seen tasks. After having seen all tasks, we use the final whole performance and average performance to evaluate the overall performance of continual relation learning. As average performance highlights the performance of handling catastrophic problem, and thus it is the main metric to evaluate models.
As the task sequence has influence on final model performance, we implement the baseline models by ourselves based on the toolkit released by Wang et al. (2019). For fair comparison, we unify the random seeds in our experiments completely consistent with the seeds in Wang et al. (2019), so that the task sequence can be completely consistent with Wang et al. (2019). For other settings, such as hidden embedding dimension and pre-trained input embeddings, we also follow the settings in Wang et al. (2019).
We evaluate our model and several baselines on the benchmarks, and select two theoretical models to measure the lower and upper bounds: (1) Lower Bound, which continually fine-tunes models for each new task without memorizing any historical examples; (2) Upper Bound, which remembers all examples in history and continually re-train models with all data. In fact, this model serves as the ideal upper bound for the performance of continual relation learning; (3) EWC (Kirkpatrick et al., 2017), which adopts elastic weight consolidation to add special  $L_{2}$  regularization on parameter changes. Then, EWC uses Fisher information to measure the parameter importance to old tasks, and slow down the update of those parameters important to old tasks; (4) EMR (Parisi et al., 2019), a basic memory-based method, which memorizes a few historical examples and simply conduct memory replay. Every time a new task comes in, EMR mixes memorized examples and new examples together to fine-tune models; (5) GEM (Lopez-Paz and Ranzato, 2017), an extension of EMR, which adds a constraint on directions of new gradients to make sure that optimization directions do not conflict with gradients on old tasks; (6) AGEM
$^{2}$ https://github.com/hongwang600/Lifelong_Relation_Detection
(a) EA-EMR (step-1)
(b) EA-EMR (step-4)
(c) EA-EMR (step-7)
(d) EA-EMR (step-10)
(e) EMAR (step-1)
(f) EMAR (step-4)
(g) EMAR (step-7)
Figure 4: A visualization of features learnt by EA-EMR and EMAR at different training steps on FewRel. For each image, we use the support vector machine to acquire its best linear boundary and draw it as the blue line.
(h) EMAR (step-10)
Step-1	Step-4	Step-7	Step-10
EA-EMR	98.8	65.0	78.8	73.8
EMAR	92.5	75.0	87.5	80.0
Table 3: Classification accuracy  $(\%)$  based on the features learnt by EA-EMR and EMAR in Figure 4.
(Chaudhry et al., 2019), the extension of GEM, which takes the gradient on sampled memorized examples from memory as the only constraint on the optimization directions of the current task; (7) EA-EMR (Wang et al., 2019), which introduces memory replay and embedding aligned mechanism to enhance previous tasks and mitigate the embedding distortion when trained on new tasks. EA-EMR is also an extension of EMR, and the state-of-the-art on continual relation learning.
4.4 Overall Results
Table 1 shows the overall performance on three benchmarks under two different settings. From the table, we can see that (1) our proposed EMAR significantly outperforms other baselines and achieves state-of-the-arts almost in all settings. On the SimpleQ dataset, the performance of EMAR is close to EA-EMR and EMR. The reason is perhaps that the SimpleQ benchmark is over simple (even the weakest Lower Bound achieves relatively high results close to Upper Bound). On other benchmarks, EMAR outperforms all the baseline models with a large margin, showing the
superiority of our proposed episodic memory activation and reconsolidation mechanism. (2) There is still a huge gap between our model and the upper bound. It indicates there remains lots of things to be explored in continual relation learning.
To further investigate how accuracy changes while learning new tasks, we show the average performance of models at each step in Figure 3. As shown in the figure, we can observe that: (1) With increasing numbers of tasks, the performance of all the models decreases in some degree. This indicates that catastrophically forgetting old relations is inevitable, and it is indeed one of the major difficulty for continual relation learning. (2) The memory-based methods significantly outperform the consolidation-based method, which demonstrates the memory-based methods could alleviate the problem of catastrophic forgetting to some extent. (3) Our proposed EMAR achieves a much better results compared to state-of-the-art model EA-EMR. It shows the effectiveness of our memory reconsolidation, and further indicates understanding relation prototypes is more important and reasonable than rote memorization of examples.
4.5 Effect of Memory Size
Memory size indicates the number of remembered examples for each task. In this section, we investigate the effect of memory size for the performance of baselines and our proposed model. We compare three memory sizes: 10, 25 and 50. As ex
6436
isting work does not report the results with different memory size, we re-implement baseline models by ourselves in this experiment. The results are shown in Table 2. We can find that: (1) With the increasing memory size, the performance of all models improves respectively, which shows that the memory size is one of the key factor determining the performance of continual relation learning models. (2) On both FewRel and TACRED, our EMAR keeps performing the best under different memory sizes, and even achieves comparable results with other models of larger memory sizes. It indicates adopting relation prototypes in EMAR is a more effective way to utilize memory compared with existing memory-based methods.
4.6 Effect of Prototypes and Reconsolidation
To show the effectiveness of prototypes and reconsolidation, we give a case study demonstrating the changing of feature spaces learnt by EA-EMR and EMAR (ours). We sample two relations from the training set and 40 examples per relation from the test set. Then we train EA-EMR and EMAR with the sampled training data respectively and visualize the changes of the sampled 40 instances in the feature spaces at different steps.
From Figure 4, we can see that EMAR learns better features of instances after multi-step training: the embedding space of EMAR is more sparse and features from two relations are more distinguishable. On the other hand, the features learnt by EA-EMR become more dense with increasing steps, thus harder to classify.
This phenomenon is mainly due to the different approaches of constraining features used by EA-EMR and EMAR. The  $L_{2}$  regularization used in EA-EMR for keeping the instance distribution of old relations leads to higher density in the feature space and smaller distances between different relations after several training steps. On the contrary, EMAR avoids models from forgetting previous relations by relation prototypes. Compared with EA-EMR, using prototypes for reconsolidation is a more flexible constraint, allowing EMAR to utilize larger feature spaces for representing examples and prototypes.
To quantitatively analyze the case, we use the support vector machine to acquire linear boundaries for each image in Figure 4 and list the classification results in Table 3. The quantitative results in the table show that embeddings learnt by
EMAR achieve better classification performance, which further supports our above observations.
To alleviate catastrophically forgetting old relations in continual relation learning, we introduce episodic memory activation and reconsolidation (EMAR), inspired by the mechanism in human long-term memory formation. Compared with existing memory-based methods, EMAR requires models to understand the prototypes of old relations rather than to overfit a few specific memorized examples, which can keep better distinction among relations after long-term training. We conduct experiments on three benchmarks in relation extraction and carry out extensive experimental results as well as empirical analyses, showing the effectiveness of EMAR on utilizing memorized examples. For future work, how to combine open relation learning and continual relation learning together to complete the pipeline for emerging relations still remains a problem, and we will continue to work on it.
This work is supported by the National Key Research and Development Program of China (No. 2018YFB1004503) and the National Natural Science Foundation of China (NSFC No. 61732008, 61772302). Tianyu Gao is supported by 2019 Tencent Rhino-Bird Elite Training Program and Tsinghua University Initiative Scientific Research Program.
Cristina M Alberini. 2005. Mechanisms of memory stabilization: are consolidation and reconsolidation similar or distinct processes? Trends in neurosciences, 28(1):51-56.
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. 2018. Memory aware synapses: Learning what (not) to forget. In Proceedings of ECCV, pages 139-154.
Gabor Angeli, Melvin Jose Johnson Premkumar, and Christopher D Manning. 2015. Leveraging linguistic structure for open domain information extraction. In Proceedings of ACL-IJCNLP, pages 344-354.
Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019. Matching the blanks: Distributional similarity for relation learning. In Proceedings of ACL, pages 2895-2905.
6437
Michele Banko, Michael J Cafarella, Stephen Soderland, Matthew Broadhead, and Oren Etzioni. 2007. Open information extraction from the web. In Proceedings of IJCAI, pages 2670-2676.
Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks, 5(2):157-166.
Bruno Bontempi, Catherine Laurent-Demir, Claude Destrade, and Robert Jaffard. 1999. Time-dependent reorganization of brain circuitry underlying long-term memory storage. Nature, 400(6745):671.
Antoine Bordes, Nicolas Usunier, Sumit Chopra, and Jason Weston. 2015. Large-scale simple question answering with memory networks. arXiv preprint arXiv:1506.02075.
Richard Boyce, Stephen D Glasgow, Sylvain Williams, and Antoine Adamantidis. 2016. Causal evidence for the role of rem sleep theta rhythm in contextual memory consolidation. Science, 352(6287):812-816.
Razvan Bunescu and Raymond Mooney. 2007. Learning to extract relations from the web using minimal supervision. In Proceedings of ACL, pages 576-583.
Arslan Chaudhry, Marc'Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. 2019. Efficient lifelong learning with a-gem. In Proceedings of ICLR.
Tianqi Chen, Ian Goodfellow, and Jonathon Shlens. 2016. Net2Net: Accelerating learning via knowledge transfer. In Proceedings of ICLR.
Lei Cui, Furu Wei, and Ming Zhou. 2018. Neural open information extraction. In Proceedings of ACL, pages 407-413.
Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. 2019. Episodic memory in lifelong language learning. In Proceedings of NIPS.
Luciano Del Corro and Rainer Gemulla. 2013. Clausie: Clause-based open information extraction. In Proceedings of WWW, pages 355-366.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171-4186.
Li Dong, Furu Wei, Ming Zhou, and Ke Xu. 2015. Question answering over Freebase with multicolumn convolutional neural networks. In Proceedings of ACL-IJCNLP, pages 260-269.
Hady Elsahar, Elena Demidova, Simon Gottschalk, Christophe Gravier, and Frederique Laforest. 2017. Unsupervised open relation extraction. In Proceedings of ESWC, pages 12-16.
Oren Etzioni, Michael Cafarella, Doug Downey, Ana Maria Popescu, Tal Shaked, Stephen Soderland, Daniel S Weld, and Alexander Yates. 2005. Unsupervised named-entity extraction from the web: An experimental study. AI, 165:91-134.
Anthony Fader, Stephen Soderland, and Oren Etzioni. 2011. Identifying relations for open information extraction. In Proceedings of EMNLP, pages 1535-1545.
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra. 2017. PathNet: Evolution channels gradient descent in super neural networks. ai.google.
Tianyu Gao, Xu Han, Ruobing Xie, Zhiyuan Liu, Fen Lin, Leyu Lin, and Maosong Sun. 2020. Neural snowball for few-shot relation learning. In Proceedings of AAAI.
Matthew R Gormley, Mo Yu, and Mark Dredze. 2015. Improved relation extraction with feature-rich compositional embedding models. In Proceedings of EMNLP, pages 1774-1784.
Xu Han, Pengfei Yu, Zhiyuan Liu, Maosong Sun, and Peng Li. 2018a. Hierarchical relation extraction with coarse-to-fine grained attention. In Proceedings of EMNLP, pages 2236-2245.
Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2018b. FewRel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In Proceedings of EMNLP, pages 4803-4809.
Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of ACL, pages 541-550.
Ronald Kemker and Christopher Kanan. 2018. *Fearnet: Brain-inspired model for incremental learning*. In Proceedings of ICLR.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of NAS, pages 3521-3526.
Jonathan LC Lee, Barry J Everitt, and Kerrie L Thomas. 2004. Independent cellular processes for hippocampal memory consolidation and reconsolidation. Science, 304(5672):839-843.
Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. TPAMI, 40(12):2935-2947.
6438
Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In Proceedings of ACL, pages 2124-2133.
ChunYang Liu, WenBo Sun, WenHan Chao, and Wanxiang Che. 2013. Convolution neural network for relation extraction. In Proceedings of ICDM, pages 231-242.
Xialei Liu, Marc Masana, Luis Herranz, Joost Van de Weijer, Antonio M Lopez, and Andrew D Bagdanov. 2018. Rotate your networks: Better weight consolidation and less catastrophic forgetting. In Proceedings of ICPR, pages 2262-2268.
Yang Liu, Furu Wei, Sujian Li, Heng Ji, Ming Zhou, and WANG Houfeng. 2015. A dependency-based neural network for relation classification. In Proceedings of ACL-IJCNLP, pages 285-290.
David Lopez-Paz and Marc'Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In Proceedings of NIPS, pages 6467-6476.
Diego Marcheggiani and Ivan Titov. 2016. Discrete-state variational autoencoders for joint discovery and factorization of relations. TACL, 4:231-244.
Mausam, Michael Schmitz, Stephen Soderland, Robert Bart, and Oren Etzioni. 2012. Open language learning for information extraction. In Proceedings of EMNLP-CoNLL, pages 523-534.
Mausam Mausam. 2016. Open information extraction systems and downstream applications. In Proceedings of IJCAI, pages 4074-4077.
James L McClelland, Bruce L McNaughton, and Randall C O'Reilly. 1995. Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. *Psychological review*, 102(3):419.
Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of ACLIJCNLP, pages 1003-1011.
Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using LSTMs on sequences and tree structures. In Proceedings of ACL, pages 1105-1116.
Karim Nader, Glenn E Schafe, and Joseph E Le Doux. 2000. Fear memories require protein synthesis in the amygdala for reconsolidation after retrieval. Nature, 406(6797):722.
Ndapandula Nakashole, Martin Theobald, and Gerhard Weikum. 2011. Scalable knowledge harvesting with high precision and high recall. In Proceedings of WSDM, pages 227-236.
Thien Huu Nguyen and Ralph Grishman. 2015. Relation extraction: Perspective from convolutional neural networks. In Proceedings of the NAACL Workshop on Vector Space Modeling for NLP, pages 39-48.
Patrick Pantel and Marco Pennacchiotti. 2006. *Espresso: Leveraging generic patterns for automatically harvesting semantic relations*. In *Proceedings of COLING*, pages 113-120.
German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. 2019. Continual lifelong learning with neural networks: A review. Neural Networks.
Fabio Petroni, Luciano Del Corro, and Rainer Gemulla. 2015. CORE: Context-aware open relation extraction with factorization machines. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1763-1773.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017. iCaRL: Incremental classifier and representation learning. In Proceedings of CVPR, pages 2001-2010.
Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010. Modeling relations and their mentions without labeled text. In Proceedings of ECML-PKDD, pages 148-163.
Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M Marlin. 2013. Relation extraction with matrix factorization and universal schemas. In Proceedings of NAACL, pages 74-84.
Ellen Riloff, Rosie Jones, et al. 1999. Learning dictionaries for information extraction by multi-level bootstrapping. In Proceedings of AAAI, pages 474-479.
Mark Bishop Ring. 1994. *Continual learning in reinforcement environments*. Ph.D. thesis, University of Texas at Austin, Austin, Texas 78712.
Hippolyt Ritter, Aleksandar Botev, and David Barber. 2018. Online structured laplace approximations for overcoming catastrophic forgetting. In Proceedings of NIPS, pages 3738-3748.
Benjamin Rozenfeld and Ronen Feldman. 2008. Self-supervised relation extraction from the web. Proceedings of KAIS, 17(1):17-33.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. 2016. Progressive neural networks. ai.google.
Cicero dos Santos, Bing Xiang, and Bowen Zhou. 2015. Classifying relations by ranking with convolutional neural networks. In Proceedings of ACLIJCNLP, pages 626-634.
6439
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In Proceedings of ESWC, pages 593-607.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. In Proceedings of NIPS, pages 2990-2999.
Yusuke Shinyama and Satoshi Sekine. 2006. Preemptive information extraction using unrestricted relation discovery. In Proceedings of NAACL-HLT, pages 304-311.
Richard Socher, Brody Huval, Christopher D Manning, and Andrew Y Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of EMNLP, pages 1201-1211.
Gabriel Stanovsky and Ido Dagan. 2016. Creating a large benchmark for open information extraction. In Proceedings of EMNLP, pages 2300-2305.
Fan-Keng Sun, Cheng-Hao Ho, and Hung-Yi Lee. 2019. LAMAL: Language modeling is all you need for lifelong language learning. arXiv preprint arXiv:1909.03329.
Sebastian Thrun and Lorien Pratt. 2012. Learning to learn. Springer Science & Business Media.
Giulio Tononi and Chiara Cirelli. 2006. Sleep function and synaptic homeostasis. *Sleep medicine reviews*, 10(1):49-62.
Hong Wang, Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Shiyu Chang, and William Yang Wang. 2019. Sentence embedding alignment for lifelong relation extraction. In Proceedings of NAACL-HLT, pages 796-806.
Ruidong Wu, Yuan Yao, Xu Han, Ruobing Xie, Zhiyuan Liu, Fen Lin, Leyu Lin, and Maosong Sun. 2019. Open relation extraction: Relational knowledge transfer from supervised data to unsupervised data. In Proceedings of EMNLP-IJCNLP, pages 219-228.
Chenyan Xiong, Russell Power, and Jamie Callan. 2017. Explicit semantic ranking for academic search via knowledge graph embedding. In Proceedings of WWW, pages 1271-1279.
Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, and Zhi Jin. 2015. Classifying relations via long short term memory networks along shortest dependency paths. In Proceedings of EMNLP, pages 1785-1794.
Guang Yang, Cora Sau Wan Lai, Joseph Cichon, Lei Ma, Wei Li, and Wen-Biao Gan. 2014. Sleep promotes branch-specific formation of dendritic spines after learning. Science, 344(6188):1173-1178.