Unregulated C hinese-to- E nglish Data Expansion Does NOT Work for Neural Event Detection (2022)

Abstract: We leverage cross-language data expansion and retraining to enhance neural Event Detection (abbr., ED) on English ACE corpus. Machine translation is utilized for expanding English training set of ED from that of Chinese. However, experimental results illustrate that such strategy actually results in performance degradation. The survey of translations suggests that the mistakenly-aligned triggers in the expanded data negatively influences the retraining process. We refer this phenomenon to “trigger falsification”. To overcome the issue, we apply heuristic rules for regulating the expanded data, fixing the distracting samples that contain the falsified triggers. The supplementary experiments show that the rule-based regulation is beneficial, yielding the improvement of about 1.6% F1-score for ED. We additionally prove that, instead of transfer learning from the translated ED data, the straight data combination by random pouring surprisingly performs better.

## introduction we tackle ed, a task of recognizing trigger words (triggers for short) that signal different types of events @xcite . for example, the trigger "meeting" in (1) signals the contact-meet event. trigger: meeting type: contact-meet the current study of ed, in general, utilizes neural classification models for determining the event types in the word-by-word manner, including the predefined ace 1 event types and non-trigger type. correspondingly, supervised learning is applied to pursue the semantic-level distributed representations of words, so as to provide perceptible evidence for decoding event types. expanding the training data is able to enhance the ed-oriented neural classification models. it is because there are a larger amount of knowledge and diverse pragmatic phenomena can be introduced into the supervised learning process, as claimed in different tasks of natural language processing. we childishly embrace this method, conducting expansion by translating chinese ed corpus and pouring translations into the english training set, where word alignment @xcite is used for designating triggers in the translations. what is beyond our expectation is that, however, cross-language data expansion actually results in performance degradation. we survey the translations which were used for expansion, and observe that a large number of low-quality instances were involved. the survey also suggests that the falsified triggers in such instances probably misled neural models during supervised learning. we provide an example in figure 1 where chinese event mention is taken from the publicly-shared ace corpus of chinese ed, and word alignment is carried out for trigger designation in english translation, as well as transmission of class label information. it can be found that the trigger "to" is mistakenly designated and labeled for event class (viz., transport) due to inexact alignment, and undoubtedly it will cause severe misleading (in terms of the high occurrence frequency of "to" in english). to overcome the issue, we explore a variety of easily-accessible rules to purify the translated ed instances (section 2). using the purified instances as external data, we conduct retraining and transfer learning to strengthen the baseline ed models, including the ones which are grounded on basic and large roberta @xcite , respectively (section 3). experimental results (section 4) show that 1) the rule-based regulation helps to avoid performance degradation and yields substantial improvements, and 2) conventional expansion by combining datasets is beneficial while, on the contrary, transfer learning is less useful. we overview the related work in section 5 before concluding this paper (section 6). ## rule-based purification against trigger falsification we apply google translation toolkit foot_0 for translating chinese event mentions into english, and use simalign @xcite to pursue the alignment between triggers in chinese mentions and words in the corresponding translations. the aligned words are designated as triggers of translations and assigned with the manually-labeled event types in chinese corpus. word alignment unavoidably falsify triggers in the translations. therefore, we explore five heuristic rules to regulate the falsified triggers. unbinding prepositions it has been exhibited in figure 1 that some prepositions (e.g., "to") are mistakenly designated as triggers due to inexact alignment, i.e., a chinese trigger is aligned to the constituent that contains both verb and preposition. the number of prepositions that serve as triggers in translations is up to 326, occupying 8% of all the designated triggers. in the cases, we unbind verbs from prepositions, and designate the latter as non-trigger words. unbinding participles in some cases, a single chinese trigger is aligned to the present or pastparticiple phrase, where the participle that stands for an attributive is redundant for signaling a certain event type and, more seriously, it is common and generally leads a variety of word senses. for example, the past-participle "opened" in (2) is redundant. there are 38 participles found to be mistakenly designated as triggers, occupying about 1% of all the designated triggers. we repeal the designation. binary-choice exclusion occasionally, a single english word is aligned with a pair of chinese words, including not only a non-trigger word but trigger. for example, both the non-trigger word "提 出" (i.e., "bring") and trigger "上 诉" ("lawsuit") are aligned to the english word "appealing" in (3). in the cases, we exclude the non-trigger type, but instead merely assign the concrete event type (such as sue in ( 3 )) to the aligned english word. there are 58 binary-choice cases occurred in the translations, occupying 1.4% of all the designated triggers. translation: we are appealing chinese trigger: 上诉; type: sue alignment: (提出)(上诉)="appealing" correcting far-fetched triggers before alignment, some chinese triggers are segmented into formal characters or the ones holding less senses. as a result, the chinese triggers are easily aligned with function words (prepositions and conjunctions) instead of content words in english. grounded on the alignment results, the trigger designation method produces a series of far-fetched triggers. for example, the chinese trigger "身中" (i.e., "injured") in (4) is mistakenly segmented into the characters "身" (i.e., body) and "中" ("in"), and the aligned preposition "in" is designated as the injury trigger. the number of english prepositions and conjunctions that were designated as triggers is up to 226, occupying 5.5%. we correct the errors by designating them as non-trigger words. translation: fired 80 pepper bombs at him, with about 57 (in) his body chinese trigger: 身中; type: injury alignment: (身中)="in" skipping the omissions a large number of chinese triggers fail to be aligned with any english word. for example, although the chinese trigger "启用" is semantically equivalent to the english word "opened" in (5), the alignment is neglected. this results in the omission of triggers in translations. more seriously, the omitted triggers will be designated as non-trigger word, and thus mislead classification models during training. therefore, we skip the mentions in which trigger omission occurs. there are 426 cases of trigger omission found in the designation process, occupying 10.4% of all the chinese triggers. translation: the remodeled service center was completed and (opened) in the morning chinese trigger: 启用; type: start-org alignment: (启用)="none" we use pretrained language models for ed, including roberta-base and roberta-large @xcite . roberta-base is constructed by 12 transformer layers @xcite , each of which contains a 12-head attention network and 768 hidden states. roberta-large is constructed by 24 transformer layers, each of which contains a 16head attention network and 1,024 hidden states. the input of both roberta models is a sentence no matter whether it appears as an event mention containing triggers. the maximum input length is set to 256 tokens, and padding is used if the input sentence fails to reach the length (section 4.2 presents other hyperparameters). the initial word embeddings are obtained using look-up tables, and they are slightly strengthened by elementwise fusion with position embeddings. besides, both roberta models are connected with a linear fully-connected layer and softmax layer @xcite . for each word in the input sentence, the roberta models conduct 34-class classification, towards not only the predefined 33 ace event types but non-trigger type. we intend to enhance the classification models by transfer learning @xcite and data expansion @xcite , using the translated chinese ed corpus (mt-ed for short) as the external data. the aforementioned rule-based purification is utilized for regulating mt-ed. the considered models in experiments are as below: baselines the baselines denotes the robertabased classifiers which are merely trained on the original training set. such a training set contains ed instances that were split from the english corpus of the publicly shared ace-2005 tasks. edt transfer learning is used to enhance the roberta-based classifiers. we first train the classifiers on mt-ed, and then train them on the original training set. within the double-stage training process, the parameters obtained in the first stage (on mt-ed) are transferred to the second stage (on the original set). we refer the classifiers to edts. ede we use mt-ed to expand the original training set by straight pouring, without any additional handling. using the expanded data set, we train the roberta-based classifiers from scratch. we refer the obtained classifiers to edes. ## related work conventional ed models rely heavily on elaborate feature engineering, such as that of contextindependent features @xcite , as well as cross-event @xcite and cross-entity @xcite statistical features. in order to pursue the perception of deep event semantics, the current study concentrates on the utilization of neural networks, designing and developing a series of reliable neural ed models, including those which are grounded on cnn (nguyen and grishman, 2015), dmcnn @xcite , @xcite , gan @xcite , gcn @xcite and vae @xcite . recently, the pretrained language models like bert @xcite , roberta @xcite and ad-dmbert @xcite are used, yielding substantial improvements. data-driven enhancement strategies have been explored for ed, most of which are implemented by data augmentation. @xcite produce new ed instances by entity replacement. it is potentially effective to enhance entity-aware neural encoders for detecting events that hold entities. tong et al. (2020) ## conclusion we use cross-language data expansion to enhance neural ed models. experimental results demonstrate that unregulated data expansion yields less improvement or even causes performance degradation. by contrast, data purification by simple heuristic rules produces substantial performance gains. in addition, it is proven that data expansion contributes more to ed then transfer learning. conducting multilingual data expansion potentially contributes to the enhancement of ed models. it is because diverse pragmatics in different languages and exotic event knowledge are informative for versatile encoding. however, it is challenging due to the lack of shareable purification rules among different languages for trigger alignment. therefore, we will develop an automatic purification model that generalize well in different languages, where the encoding of syntactic information and reinforcement learning will be used.