Interpretability Rules: Jointly Bootstrapping a Neural Relation Extractorwith an Explanation Decoder (2021)

Abstract: We introduce a method that transforms a rule-based relation extraction (RE) classifier into a neural one such that both interpretability and performance are achieved. Our approach jointly trains a RE classifier with a decoder that generates explanations for these extractions, using as sole supervision a set of rules that match these relations. Our evaluation on the TACRED dataset shows that our neural RE classifier outperforms the rule-based one we started from by 9 F1 points; our decoder generates explanations with a high BLEU score of over 90%; and, the joint learning improves the performance of both the classifier and decoder.

## introduction information extraction (ie) is one of the key challenges in the natural language processing (nlp) field. with the explosion of unstructured information on the internet, the demand for high-quality tools that convert free text to structured information continues to grow @xcite @xcite . the past decades have seen a steady transition from rule-based ie systems @xcite to methods that rely on machine learning (ml) (see related work). while this transition has generally yielded considerable performance improvements, it was not without a cost. for example, in contrast to modern deep learning methods, the predictions of rule-based approaches are easily explainable, as a small number of rules tends to apply to each extraction. further, in many situations, rule-based methods can be developed by domain experts with minimal training data. for these reasons, rule-based ie methods remain widely used in industry @xcite . in this work we demonstrate that this transition from rule-to ml-based ie can be performed such that the benefits of both worlds are preserved. in particular, we start with a rule-based relation ex-traction (re) system @xcite and bootstrap a neural re approach that is trained jointly with a decoder that learns to generate the rules that best explain each particular extraction. the contributions of our idea are the following: ## related work relation extraction using statistical methods is well studied. methods range from supervised, "traditional" approaches @xcite to neural methods. neural approaches for re range from methods that rely on simpler representations such as cnns @xcite and @xcite to more complicated ones such as augmenting rnns with different components @xcite , combining rnns and cnns @xcite , and using mechanisms like attention @xcite or gcns @xcite . to solve the lack of annotated data, distant supervision @xcite is commonly used to generate a training dataset from an existing knowledge base. @xcite address the inherent noise in distant supervision with an entity attention method. rule-based methods in ie have also been extensively investigated. @xcite developed a system that learns extraction patterns using only a pre-classified corpus of relevant and irrelevant texts. @xcite proposed a unsupervised method for discovering inference rules from text based on the harris distributional similarity hypothesis @xcite . @xcite introduced a rule language that covers both surface text and syntactic dependency graphs. @xcite further show that converting rule-based models to statistical ones can capture some of the benefits of both, i.e., the precision of patterns and the generalizability of statistical models. interpretability has gained more attention recently in the ml/nlp community. for example, some efforts convert neural models to more interpretable ones such as decision trees @xcite . some others focus on producing a post-hoc explanation of individual model outputs @xcite . inspired by these directions, here we propose an approach that combines the interpretability of rule-based methods with the performance and generalizability of neural approaches. ## approach our approach jointly addresses classification and interpretability through an encoder-decoder architecture, where the decoder uses multi-task learning (mtl) for relation extraction between pairs of named entities (task 1) and rule generation (task 2). figure 1 summarizes our approach. ## conclusion we introduced a strategy that jointly bootstraps a relation extraction classifier with a decoder that generates explanations for these extractions, using as sole supervision a set of example patterns that match such relations. our experiments on the tacred dataset demonstrated that our approach outperforms the strong rule-based method that provided the training patterns by 9 f1 points, while decoding explanations at over 90% bleu score. further, we showed that the joint training of the classification and explanation components performs better than training them separately. all in all, our work suggests that it is possible to marry the interpretability of rule-based methods with the performance of neural approaches. daojian zeng, kang liu, siwei lai, guangyou zhou, and jun zhao. 2014. relation classification via convolutional deep neural network. in proceedings of coling 2014, the 25th international conference on computational linguistics: technical papers, pages 2335-2344. dongxu zhang and dong wang. 2015. relation classification via recurrent neural network. arxiv preprint arxiv:1508.01006. yuhao zhang, peng qi, and christopher d. manning. 2018. graph convolution over pruned dependency trees improves relation extraction. in proceedings of the 2018 conference on empirical methods in natural language processing, pages 2205-2215, brussels, belgium. association for computational linguistics. yuhao zhang, victor zhong, danqi chen, gabor angeli, and christopher d. manning. 2017. positionaware attention and supervised data improve slot filling. in proceedings of the 2017 conference on empirical methods in natural language processing (emnlp 2017), pages 35-45. peng zhou, wei shi, jun tian, zhenyu qi, bingchen li, hongwei hao, and bo xu. 2016. attention-based bidirectional long short-term memory networks for relation classification. in proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: short papers), pages 207-212, berlin, germany. association for computational linguistics.