N eural G ibbs S ampling for J oint E vent A rgument E xtraction (2020)

Abstract: Event Argument Extraction (EAE) aims at predicting event argument roles of entities in text, which is a crucial subtask and bottleneck of event extraction. Existing EAE methods either extract each event argument roles independently or sequentially, which cannot adequately model the joint probability distribution among event arguments and their roles. In this paper, we propose a Bayesian model named Neural Gibbs Sampling (NGS) to jointly extract event arguments. Specifically, we train two neural networks to model the prior distribution and conditional distribution over event arguments respectively and then use Gibbs sampling to approximate the joint distribution with the learned distributions. For overcoming the shortcoming of the high complexity of the original Gibbs sampling algorithm, we further apply simulated annealing to efficiently estimate the joint probability distribution over event arguments and make predictions. We conduct experiments on the two widely-used benchmark datasets ACE 2005 and TAC KBP 2016. The Experimental results show that our NGS model can achieve comparable results to existing state-of-the-art EAE methods. The source code can be obtained from https://github.com/THU-KEG/NGS .

## introduction event argument extraction (eae) is a crucial subtask of event extraction, which aims at predicting entities and their event argument roles in event mentions. for instance, given the sentence "fox's stock price rises after the acquisition of its entertainment businesses by disney", the event detection (ed) model will first identify the trigger word "acquisition" triggering a transfer-ownership event. then, with the trigger word and event type, 's stock price rises after the acquisition of its by . the eae model is required to identify that "fox" and "disney" are event arguments whose roles are "seller" and "buyer" respectively. as ed is well-studied in recent years @xcite @xcite , eae becomes the bottleneck and has drawn growing attention. ## related work event extraction (ee) aims to extract structured information from plain text, which is a challenging task in the field of information extraction. ee consists of two subtasks, one is event detection (ed) to detect words triggering events and identify event types, the other is event argument extraction (eae) to extract argument entities in event mentions and identify event argument roles. as ee is important and beneficial for various downstream gibbs sampling prior neural model prior distribution … p p (x i |o) prior state initialization x 2 x n … x 1 conditional neural model conditional distribution … sampling process . x (t) 1 x (t) 2 x (t) n x (t) i . t step state t+1 step state x (t+1) i . x (t) 1 x (t) 2 x (t) n . p c (x (t+1) i |x (t) i , o) i ⇠ max(p c (x (t+1) i |x (t) i , o)) 1/c p n @xmath0, o)) 1/c simulated annealing decrease c 's stock price rises after the acquisition of its entertainment businesses by . transfer-ownership fox disney buyer seller artifact fox 's stock price rises after the acquisition of its entertainment businesses by disney. text result converge initialization nlp tasks, e.g., question answering @xcite , information retrieval @xcite , and reading comprehension @xcite , it has attracted wide attentions recently. ed has been well-studied by the previous works due to its simple and clear definition, including feature-based and rule-based methods @xcite @xcite @xcite @xcite @xcite @xcite , neural methods @xcite @xcite @xcite @xcite , the methods with external heterogeneous knowledge @xcite @xcite @xcite . some advanced architectures, such as graph convolutional networks @xcite and adversarial training @xcite , have also been applied recently. as ed models has achieved relatively promising results, the more difficult eae becomes the bottleneck of ee, and have drawn growing research interests. the early works @xcite @xcite @xcite focus on designing hand-crafted features and heuristic rules to extract event arguments, which suffer from the problem of both implementation complexity and low recall. as the rapid develop-ment of neural networks, various neural methods have been proposed, such as utilizing convolutional models @xcite , utilizing recurrent models @xcite , and finetuning pre-trained language model bert @xcite . as compared with the early featurebased and rule-based methods, neural methods automatically represent sentence semantics with lowdimensional vectors, and independently determine argument roles with the vectors, leading to getting rid of designing sophisticated features and rules. recently, some works adopt some advanced techniques to further improve eae models in different scenarios, including zero-shot learning @xcite , multi-modal integration @xcite , cross-lingual @xcite , end-to-end @xcite , and weak supervision @xcite . the current methods for eae have achieved some promising results. however, they focus on independently handling each argument entity to predict its role. because of ignoring to capture rich correlated knowledge among event arguments, the above-mentioned methods are easy to trap in a local optimum and make some inexplicable mistakes. inspired by some methods in named entity recognition @xcite and relation extraction @xcite , some recent proactive works view eae as a sequence labeling problem. following the methods for sequence labeling problem @xcite , these sequential eae models @xcite adopt conditional random field (crf) with the viterbi algorithm @xcite , and unintentionally consider the correlation of event arguments. limited by the markov property, the linear-chain crf sequentially considers the correlation between two adjacent event arguments, which cannot adequately handle the complex situation in eae that each argument and any other arguments may be correlated. to this end and inspired by some proactive works @xcite , we adapt gibbs sampling @xcite for eae to perform approximate inference from the joint distribution. moreover, we incorporate simulated annealing @xcite to accelerate the sampling process, leading to an effective and efficient method. ## conclusion and future work in this paper, we propose a novel neural gibbs sampling (ngs) method to adequately model the correlation between event arguments and argument roles, which combines the advantages of the gibbs sampling method to model the joint distribution among random variables and the neural network models to automatically learn the effective representations. considering the shortcoming of high complexity of gibbs sampling algorithm, we further apply simulated annealing to accelerate the whole estimation process, which lead our method to being both effective and efficient. the experimental results on two widely-used real-world datasets show that ngs can achieve comparable results to existing state-of-the-art eae methods. the empirical analyses and ablation studies further verify the effectiveness and efficiency of our method. in the future: (1) we will try to extend ngs to other tasks and scenarios to evaluate its general effectiveness of modeling the latent correlations. (2) we will also explore more effective and simple methods to consider the correlations.