Adaptive Ordered Information Extraction with Deep Reinforcement Learning (2023)

Abstract: Information extraction (IE) has been studied extensively. The existing methods always follow a fixed extraction order for complex IE tasks with multiple elements to be extracted in one instance such as event extraction. However, we conduct experiments on several complex IE datasets and observe that different extraction orders can significantly affect the extraction results for a great portion of instances, and the ratio of sentences that are sensitive to extraction orders increases dramatically with the complexity of the IE task. Therefore, this paper proposes a novel adaptive ordered IE paradigm to find the optimal element extraction order for different instances, so as to achieve the best extraction results. We also propose an reinforcement learning (RL) based framework to generate optimal extraction order for each instance dynamically. Additionally, we propose a co-training framework adapted to RL to mitigate the exposure bias during the extractor training phase. Extensive experiments conducted on several public datasets demonstrate that our proposed method can beat previous methods and effectively improve the performance of various IE tasks, especially for complex ones.

## introduction information extraction (ie) has been studied extensively over the past few decades @xcite . with the rapid development of pre-trained language models, simple ie tasks such as named entity recognition @xcite have been well solved. however, complex ie tasks with multiple elements to be extracted such as relation extraction @xcite and event extraction @xcite still need further exploration. traditional ie methods always follow a static extraction manner, i.e. with a pre-defined fixed † corresponding authors. 1 resources of this paper can be found at https:// github.com/ez-hwh/autoextraction ## related work pipeline information extraction (ie) methods split the extraction process into several sub-tasks and optimize each of them. they rely on the task definition so the framework varies for different ie tasks. for relation extraction task, @xcite @xcite gradually extract subject, relation and object from the sentence in different order. for event extraction task, @xcite @xcite first recognize the event type and trigger and then extract the arguments with sequence tagging model or machine comprehension model. joint ie methods combine two or more extraction processes into one stage. graph-based methods are the mainstream joint ie framework. they recognize the entity or text span and build a graph with co-reference, relation @xcite , entity similarity @xcite or sentence (co-occurrence) @xcite . through information propagation on the graph, they better encode the sentence and document and then decode the edge to build the final sub-graph. generation-based ie methods are another paradigm for joint extraction, @xcite for relation extraction task, @xcite @xcite for event extraction, @xcite for unified information extraction, they all serialize the structured extraction results into sentence or pre-defined template in a fixed order. apart from works above, @xcite propose one-stage joint extraction framework that decode the subject and object simultaneously. recently, reinforcement learning (rl) has been applied to ie task. information extraction was augmented by using rl to acquire and incorporate external evidence in @xcite . @xcite both train a rl agent for instance selecting to denoise training data obtained via distant supervision for relation extraction. @xcite utilizes a hierarchical reinforcement learning framework to improve the connection between entity mentions and relation types. @xcite first considers extraction order of relational facts in a sentence and then trains a sequence-to-sequence model with rl.