Relabel the Noise: Joint Extraction of Entities and Relations via Cooperative Multiagents (2020)

Abstract: Distant supervision based methods for entity and relation extraction have received increasing popularity due to the fact that these methods require light human annotation efforts. In this paper, we consider the problem of shifted label distribution, which is caused by the inconsistency between the noisy-labeled training set subject to external knowledge graph and the human-annotated test set, and exacerbated by the pipelined entity-then-relation extraction manner with noise propagation. We propose a joint extraction approach to address this problem by re-labeling noisy instances with a group of cooperative multiagents. To handle noisy instances in a fine-grained manner, each agent in the cooperative group evaluates the instance by calculating a continuous confidence score from its own perspective; To leverage the correlations between these two extraction tasks, a confidence consensus module is designed to gather the wisdom of all agents and re-distribute the noisy training set with confidence-scored labels. Further, the confidences are used to adjust the training losses of extractors. Experimental results on two real-world datasets verify the benefits of re-labeling noisy instance, and show that the proposed model significantly outperforms the state-of-the-art entity and relation extraction methods.

## introduction the extraction of entities and relations has long been recognized as an important task within natural language processing, as it facilitates text understanding. the goal of the extraction task is to identify entity mentions, assign predefined entity types, and extract their semantic relations from text corpora. for example, given a sentence "washington is the president of the united states of america", an extraction system will find a president of relation between person entity "washington" and country entity "united states of america". a major challenge of the entity and relation extraction task is the absence of large-scale and domain-specific labeled training data due to the expensive labeling efforts. one promising solution to address this challenge is distant supervision (ds) @xcite , which generates labeled training data automatically by aligning external knowledge graph (kg) to text corpus. despite its effectiveness, the aligning process introduces many noisy labels that degrade the performance of extractors. to alleviate the introduced noise issue of ds, extensive studies have been performed, such as using probabilistic graphical models @xcite , neural networks with attention @xcite and instance selector with reinforcement learning (rl) @xcite . however, most existing works overlooked the shifted label distribution problem @xcite , which severely hinders the performance of ds-based extraction models. specifically, there is a label distribution gap between ds-labeled training set and human-annotated test data, since two kinds of noisy labels are introduced and they are subject to the aligned kg: (1) false positive: unrelated entity pair in the sentence while labeled as relations in kg; and (2) false negative: related entity pair while neglected and labeled as none. existing denoising works assign low weights to noisy instances or discard false positives while not recovering the original labels, leaving the shifted label distribution problem unsolved. moreover, most denoising works assume that the target entities have been extracted, i.e., the entity and relation extraction is processed in a pipe-lined manner. by extracting entities first and then classifying predefined relations, the entity extraction errors will be propagated to the relation extractor, introducing more noisy labels and exacerbating the shifted label problem. besides, there are some correlations and complementary information between the two extraction tasks, which are under-utilized but can provide hints to reduce noises more precisely, e.g., it is unreasonable to predict two coun-try entities as the relation president of. in this paper, to reduce the shifted label distribution gap and further enhance the ds-based extraction models, we propose a novel method to re-label the noisy training data and jointly extract entities and relations. specifically, we incorporate rl to re-label noisy instances and iteratively retrain entity and relation extractors with adjusted labels, such that the labels can be corrected by trial and error. to leverage the correlations between the two extraction tasks, we train a group of cooperative multiagents to evaluate the instance confidence from different extraction views. through a proposed confidence consensus module, the instances are re-labeled with confidence-scored labels, and such confidence information will be used to adjust the training loss of extractors. finally, the performances of extractors are refined by exploring suitable label distributions with iterative re-training. empirical evaluations on two real-world datasets show that the proposed approach can effectively help existing extractors to achieve remarkable extraction performance with noisy labels, and the agent training is efficient with the help of correlations between these two extraction tasks. ## related works many entity and relation extraction methods have been proposed with the pipelined fashion, i.e., perform named entity recognition (ner) first and then relation classification. traditional ner systems usually focus on a few predefined types with supervised learning @xcite . however, the expensive human annotation blocks the large-scale training data construction. recently, several efforts on ds and weak supervision (ws) ner extraction have been made to address the training data bottleneck @xcite . for relation extraction, there are also many ds methods @xcite @xcite @xcite and ws methods @xcite @xcite to address the limitation of supervised methods. our method can be applied for a large number of those extractors as a post-processing plugin since the ds and ws usually incorporate many noises. a recent work crossweigh @xcite ) estimates the label mistakes and adjusts the weights of sentences in the ner benchmark conll03. they focus on the noises of supervised "gold standard" labels while we focus on the noises of automatically constructed "silver standard" labels. moreover, we deal with the noises by considering the shifted label distribution problem, which is overlooked by most existing ds works. in @xcite , this issue is analyzed and authors improve performance significantly by using the distribution information from test set. in this paper, we propose to use rl to explore suitable label distributions by re-distributing the training set with confidencescored labels, which is practical and robust to label distribution shift since we may not know the distribution of test set in real-world applications. another extraction manner is joint extraction, such as methods based on neural network with parameter sharing @xcite , represen-sentence 1, false negative, label: (bashardost[/person], none, ghazni[/location]) entity extractor relation extractor entity agents relation agent confidence consensus bashardost, an ethnic hazara, was born in ghazni province to a family of government employees. bashardost[/person] ghazni[/location] country of birth 0.772 0.729 0.896 positive (0.799) sentence 2, false positive, label: (profilin[/protein], pos action physical, actin[/protein]) acanthamoeba profilin affects the mechanical properties of nonfilamentous actin. profilin[/none] actin[/protein] none 0.373 0.791 0.236 negative (0.533) the binary selection strategy is also adopted in a related study, reinforced co-training @xcite , which uses an agent to select instances and help classifiers to form auto-labeled datasets. an important difference is that they select unlabeled instances while we evaluate noisy instances and relabel them. more recently, hrl @xcite uses a hierarchical agent to first identifies relation indicators and then entities. different from using one task-switching agent of this work, we leverage a group of multiagents, which can be a pluggable helper to existing extraction models. ## conclusions to deal with the noise labels and accompanying shifted label distribution problem in distant supervision, in this paper, we propose a novel method to jointly extract entity and relation through a group of cooperative multiagents. to make full use of each instance, each agent evaluates the instance confidence from different views, and then a confidence consensus module is designed to re-label noisy instances with confidences. thanks to the exploration of suitable label distribution by rl agents, the confidences are further used to adjust the training losses of extractors and the potential harm caused by noisy instances can be alleviated. to demonstrate the effectiveness of the proposed method, we evaluate it on two real-world datasets and the results confirm that the proposed method can significantly improve extractor performance and achieve effective learning.