Effective Use of Graph Convolution Network and Contextual Sub-Tree for Commodity News Event Extraction (2021)

Abstract: Event extraction in commodity news is a less researched area as compared to generic event extraction. However, accurate event extraction from commodity news is useful in abroad range of applications such as under-standing event chains and learning event-event relations, which can then be used for commodity price prediction. The events found in commodity news exhibit characteristics different from generic events, hence posing a unique challenge in event extraction using existing methods. This paper proposes an effective use of Graph Convolutional Networks(GCN) with a pruned dependency parse tree, termed contextual sub-tree, for better event ex-traction in commodity news. The event ex-traction model is trained using feature embed-dings from ComBERT, a BERT-based masked language model that was produced through domain-adaptive pre-training on a commodity news corpus. Experimental results show the efficiency of the proposed solution, which out-performs existing methods with F1 scores as high as 0.90. Furthermore, our pre-trained language model outperforms GloVe by 23%, and BERT and RoBERTa by 7% in terms of argument roles classification. For the goal of re-producibility, the code and trained models are made publicly available.

## introduction world events such as geo-political and macroeconomic-related events have been shown to impact commodity prices in both short-term and longterm @xcite . generally, events found in commodity news articles can be categorized into geo-political, macro-economic, supplydemand related, and commodity price movements. commodity news is a valuable source of information to extract and mine for such events. accurate figure 1 : an example of a sentence from a piece of commodity news foot_2 , loaded with three events (trigger words are in bold): (1) crude oil inventory increase, (2) oversupply and (3) crude oil price decrease. three event arguments that are of the same entity type are highlighted in color. as defined in ace (automatic content extraction) program foot_1 , the event extraction task is made up of two subtasks: (1) event trigger extraction (identifying and classifying event triggers) and ( 2 ) event argument extraction (identifying arguments of event triggers and labeling their roles). in this work, we perform event extraction on the commodity news dataset introduced in @xcite . figure 1 shows a sample sentence from this dataset. to illustrate the task of event extraction, consider the example in figure 1 and the corresponding information tabulated in figure 2 : the example in figure 1 is shown here with annotation provided in @xcite ; trigger word is underlined and in bold, entity type for each entity mention is shown in blue below its respective word in biotagging format while event argument role for each entity is shown in light yellow above the words; arches link argument to its trigger word. event entity mention argument role trigger: u.s. supplier soared crude item stockpiles attribute 1.350 million barrels difference event type: december reference point movement_ 200 million barrels initial value up_gain 438.9 million barrels final value more than 50% none the types of events found in commodity news are vastly different from generic events in ace2005 4 . below is a list of unique characteristics of these events: with these challenges in mind, this paper proposes a solution applying graph convolutional networks (gcn) over contextual sub-tree for the task of event extraction. the contextual sub-tree is a dependency parse tree uniquely pruned that provides not just dependency path information but also offpath information. the off-path information adds more context to the existing dependency path between two nodes; hence it is termed contextual sub-tree. the contributions of this work are as follows: (1) we show that a domain-adaptive pre-trained language model, combert, can yield promising performance over generic pre-trained language model when fine-tuned on event extraction tasks, and (2) we propose an effective usage of a graph neural network, in the form of a gcn with contextual sub-tree that outperforms other existing approaches. the usefulness is particularly apparent in the subtask of event argument classification. ## commodity news dataset the dataset introduced in @xcite is a collection of annotated commodity news articles where its annotation is based on standards introduced by canonical programs such as ace and tac-kbp foot_3 . figure 2 captures in graphical form, the annotatations found in the dataset: (1) entity mentions -both named and nominal (entity type shown in blue under the sentence in bio-tagging format); (2) event trigger words (word underlined and in bold); and (3) argument roles (labels are in light yellow above the sentence) with arches linking each argument to the event it belongs. this dataset consists of 21 entity types, 18 event types, and 19 argument role types (each event type has its own set of argument roles. event types are defined based on ravenpack's foot_4 event taxonomy, out of which 18 event types are chosen. in terms of size, this dataset contains 8,850 entity mentions and 3,949 events. ## proposed solution in reference to the list of unique characteristics of events found in commodity news (listed in section 1), this paper proposes a solution using graph convolutional networks (gcn) with contextual sub-tree for effective event extraction in commodity news. syntactic dependency graphs represent sentences as directed trees with head-modifier dependency arcs between related words. each word in such graphs is surrounded by its direct syntactic governor and dependent words (the neighbors), over which convolution operations can be performed on the most relevant words and avoid the modeling of unrelated words. rather than using the full dependency parse tree, we propose to use a uniquely pruned dependency tree that is made up of the shortest path between two nodes (in our case -the trigger candidate and entity mention) and additional off-path nodes. the off-path nodes are included to provide additional contextual information. the resulting tree is termed contextual sub-tree. ## experiment setup parameter settings. the data is split into 70% for training and 30% for testing. for all the experiments, the word embedding is of size 768 dimensions (same as bert-base-cased) while 50 dimensions for the other two embeddings -postag embedding and entity-type embedding. for the gcn module, we use a two-layer gcn (@xmath0. the model is trained using the cross-entropy loss function and adam optimizer. 6 results and analysis ## conclusion this paper presents a graph-based deep learning framework to extract commodity-related events. this framework is tailored for the purpose of commodity news event extraction based on the new commodity news dataset introduced by @xcite . our method addresses specific challenges exhibited by the characteristics of this dataset, in particular: (1) sentences containing lots of numerical information such as price, percentage of change, and dates, (2) entities of similar type playing distinctly different argument roles, and (3) the need for arguments extraction to disambiguate the identified events. the proposed solution uses a graph convolutional network with a contextual sub-tree to extract events effectively. experimental results demonstrate that the proposed solution outperforms existing solutions with higher f1 scores, particularly in argument role classification. with accurate event extraction from commodity news, the extracted information can be used for other downstream tasks such as learning event chains and event-event relations that can be further exploited for commodity price prediction.