T ext2 E vent: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction (2021)

Abstract: Event extraction is challenging due to the complex structure of event records and the semantic gap between text and event. Traditional methods usually extract event records by decomposing the complex structure prediction task into multiple subtasks. In this paper, we propose Text2Event, a sequence-to-structure generation paradigm that can directly extract events from the text in an end-to-end manner. Specifically, we design a sequence-to-structure network for unified event extraction, a constrained decoding algorithm for event knowledge injection during inference, and a curriculum learning algorithm for efficient model learning. Experimental results show that, by uniformly modeling all tasks in a single model and universally predicting different labels, our method can achieve competitive performance using only record-level annotations in both supervised learning and transfer learning settings.

## introduction event extraction is an essential task for natural language understanding, aiming to transform the text into event records @xcite . for example, in figure 1 , mapping "the man returned to los angeles from mexico following his capture tuesday by bounty hunters." into two event records {type: transport, trigger: returned, arg1 role: artifact, arg1: the man, arg2 role: destination, arg2: los angeles, ... } and {type: arrest-jail, trigger: capture, arg1 role: person, arg1: the man, arg2 role: agent, arg2: bounty hunters, ... }. event extraction is challenging due to the complex structure of event records and the semantic gap between text and event. first, an event record contains event type, trigger, and arguments, which form a table-like structure. and different event types have different structures. for example, in figure 1 , transport and arrest-jail have entirely different structures. second, an event can be expressed using very different utterances, such as diversified trigger words and heterogeneous syntactic structures. for example, both "the dismission of the man" and "the man departed his job" express the same event record {type: end-position, arg1 role: person, arg1: the man}. currently, most event extraction methods employ the decomposition strategy @xcite @xcite @xcite @xcite , i.e., decomposing the prediction of complex event structures into multiple separated subtasks (mostly including entity recognition, trigger detection, argument classifica-tion), and then compose the components of different subtasks for predicting the whole event structure (e.g., pipeline modeling, joint modeling or joint inference). the main drawbacks of these decomposition-based methods are: (1) they need massive and fine-grained annotations for different subtasks, often resulting in the data inefficiency problem. for example, they need different finegrained annotations for transport trigger detection, for person entity recognition, for transport.artifact argument classification, etc. (2) it is very challenging to design the optimal composition architecture of different subtasks manually. for instance, the pipeline models often lead to error propagation. and the joint models need to heuristically predefine the information sharing and decision dependence between trigger detection, argument classification, and entity recognition, often resulting in suboptimal and inflexible architectures. in this paper, we propose a sequence-tostructure generation paradigm for event extraction -text2event, which can directly extract events from the text in an end-to-end manner. specifically, instead of decomposing event structure prediction into different subtasks and predicting labels, we uniformly model the whole event extraction process in a neural network-based sequence-tostructure architecture, and all triggers, arguments, and their labels are universally generated as natural language words. for example, we generate a subsequence "attack fire" for trigger extraction, where both "attack" and "fire" are treated as natural language words. compared with previous methods, our method is more data-efficient: it can be learned using only coarse parallel text-record annotations, i.e., pairs of sentence, event records , rather than fine-grained token-level annotations. besides, the uniform architecture makes it easy to model, learn and exploit the interactions between different underlying predictions, and the knowledge can be seamlessly shared and transferred between different components. furthermore, we design two algorithms for effective sequence-to-structure event extraction. first, we propose a constrained decoding algorithm, which can guide the generation process using event schemas. in this way, the event knowledge can be injected and exploited during inference on-thefly. second, we design a curriculum learning algorithm, which starts with current pre-trained language models (plms), then trains them on simple event substructure generation tasks such as trigger generation and independent argument generation, finally trains the model on the full event structure generation task. we conducted experiments foot_0 on ace and ere datasets, and the results verified the effectiveness of text2event in both supervised learning and transfer learning settings. in summary, the contributions are as follows: ## text2event: end-to-end event extraction as controllable generation given the token sequence @xmath0, ..., x |x| of the input text, text2event directly generate the event structures @xmath1, ..., e |e| via an encoderdecoder architecture. for example, in figure 1 , text2event take the raw text as input and output two event records including {type: transport, trigger: returned, arg1 role: artifact, arg1: the man, ...} and {type: arrest-jail, trigger: capture, ..., arg2 role: agent, arg2: bounty hunters, ...}. for end-to-end event extraction, text2event first encodes input text, then generates the linearized structure using the constrained decoding algorithm. in the following, we first introduce how to reformulate event extraction as structure generation via structure linearization, then describe the sequence-to-structure model and the constrained decoding algorithm. the man returned to los angeles from mexico following his capture tuesday by bounty hunters. ## learning this section describes how to learn the text2event neural network in an end-toend manner. our method can be learned using only the coarse parallel text-record annotations, i.e., pairs of sentence, event records , with no need for fine-grained token-level annotation used in traditional methods. given a training dataset @xmath2, y 1 ), ...(x |d| , y |d| )} where each instance is a sentence, event records pair, the learning objective is the negative log-likelihood function as: to address the above challenges, we employ a curriculum learning @xcite strategy. specifically, we first train plms using simple event substructure generation tasks so that they would not overfit in non-semantic indicators; then we train the model on the full event structure generation task. substructure learning. because event representations often have complex structures and their token sequences are different from natural language word sequences, it is challenging to train them with the full sequence generation task directly. therefore, we first train text2event on simple event substructures. specifically, we learn our model by starting from generating only "(label, span)" substructures, including "(type, trigger words)" and "(role, argument words)" substructures. for example, we will extract substructure tasks in figure 2c in this stage as: (transport returned) (artifact the man) (arrest-jail capture), etc. we construct a sentence, substructures pair for each extracted substructures, then train our model using the loss in equation 4. full structure learning. after the substructure learning stage, we further train our model for the full structure generation task using the loss in equation 4. we found the curriculum learning strategy uses data annotation more efficiently and makes the learning process more smooth. ## experiments this section evaluates the proposed text2event model by conducting experiments in both supervised learning and transfer learning settings. ## conclusions in this paper, we propose text2event, a sequence-to-structure generation paradigm for event extraction. text2event directly learns from parallel text-record annotation and uniformly models all subtasks of event extraction in a sequence-to-structure framework. concretely, we propose an effective sequence-to-structure network for event extraction, which is further enhanced by a constrained decoding algorithm for event knowledge injection during inference and a curriculum learning algorithm for efficient model learning. experimental results in supervised learning and transfer learning settings show that text2event can achieve competitive performance with the previous sota using only coarse text-record annotation. for future work, we plan to adapt our method to other information extraction tasks, such as n-ary relation extraction. liberal event extraction and event schema induction. in proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), pages 258-268, berlin, germany. association for computational linguistics. lifu huang, heng ji, kyunghyun cho, ido dagan, sebastian riedel, and clare voss. 2018. zero-shot transfer learning for event extraction. in proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pages 2160-2170, melbourne, australia. association for computational linguistics. ruihong huang and ellen riloff. 2012. modeling textual cohesion for event extraction. in proceedings of the twenty-sixth aaai conference on artificial intelligence, aaai'12, page 1664-1670. aaai press. heng ji and ralph grishman. 2008. refining event extraction through cross-document inference. in proceedings of acl-08: hlt, pages 254-262. association for computational linguistics. nal kalchbrenner and phil blunsom. 2013. recurrent continuous translation models. in proceedings of the 2013 conference on empirical methods in natural language processing, pages 1700-1709, seattle, washington, usa. association for computational linguistics. fayuan li, weihua peng, yuguang chen, quan wang, lu pan, yajuan lyu, and yong zhu. 2020. event extraction as multi-turn question answering. in findings of the association for computational linguistics: emnlp 2020, pages 829-838, online. association for computational linguistics. qi li, heng ji, yu hong, and sujian li. 2014. constructing information networks using one single model. in proceedings of the 2014 conference on empirical methods in natural language processing (emnlp), pages 1846-1851, doha, qatar. association for computational linguistics. qi li, heng ji, and liang huang. 2013. joint event extraction via structured prediction with global features. in proceedings of the 51st annual meeting of the association for computational linguistics (volume 1: long papers), pages 73-82, sofia, bulgaria. association for computational linguistics. shasha liao and ralph grishman. 2010. using document level cross-event inference to improve event extraction. in proceedings of the 48th annual meeting of the association for computational linguistics, pages 789-797, uppsala, sweden. association for computational linguistics. hongyu lin, yaojie lu, xianpei han, and le sun. 2018. nugget proposal networks for chinese event detection. in proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1565-1574, melbourne, australia. association for computational linguistics. ying lin, heng ji, fei huang, and lingfei wu. 2020. a joint neural model for information extraction with global features. in proceedings of the 58th annual meeting of the association for computational linguistics, pages 7999-8009, online. association for computational linguistics.