Soft Syntactic Reinforcement for Neural Event Extraction (2025)

Abstract: Recent event extraction (EE) methods rely on pre-trained language models (PLMs) but still suffer from errors due to a lack of syntactic knowledge. While syntactic information is crucial for EE, there is a need for effective methods to incorporate syntactic knowledge into PLMs. To address this gap, we present a novel method to incorporate syntactic information into PLM-based models for EE, which do not require external syntactic parsers to produce syntactic features of task data. Instead, our proposed soft syntactic reinforcement (SSR) mechanism learns to select syntax-related dimensions of PLM representation during pretraining on a standard dependency corpus. The adapted PLM weights and the syntax-aware representation then facilitate the model’s prediction over the task data. On both sentence-level and document-level EE benchmark datasets, our proposed method achieves state-of-the-art results, outperforming baseline models and existing syntactic reinforcement methods. To the best of our knowledge, this is the first work in this direction. Our code is available at https://github.com/Anran971/sre-naacl25 .

## introduction event extraction (ee), which obtains poly-element structured information regarding what happens from unstructured text, is a challenging task in natural language processing. among ee literature, syntax information has been found beneficial for the task @xcite @xcite . syntactic features such as dependency trees provide clues for the models to better learn the interrelations between candidate trigger words and respective entities in sentences. with the advent of large pre-trained language models (plms), there has been a notable shift towards neural ee methods @xcite , which have demonstrated impressive performance gains. as shown in figure 1 , prior to plm, the traditional approach to ee relies on a "word embedding + representation learning" paradigm (red dotted lines), in which syntactic annotation is added to enhance the system via syntactic feature embedding. in other words, separate embeddings for words and syntactic features are trained and then combined for task-specific representation learning and prediction. the latest ee studies based on plm replace the "word embedding + representation learning" structure with a powerful language model (the red block). with sufficient selfsupervised pre-training, these plm-based models significantly improve task-specific representation learning and lead to new state-of-the-art results in ee. however, qualitative analyses reveal that plmbased ee models still make certain errors due to a deficiency in syntactic structure knowledge @xcite for event extraction. we empirically found that the conventional figure 2 : our method relies on dependency parse trees, a classical structure to represent syntax. take the sentence "last monday, a 19-year-old extremist detonated a 30-kilo bomb near a military jeep, injuring three soldiers." as an example, its dependency parse tree can be converted to a syntactic depth vector or a syntactic distance matrix. in this work, we introduce a novel approach to syntactically enhance plm-based ee models, using a soft syntactic reinforcement (ssr) mechanism. without the need for syntactically annotating task data, our approach leverages a general-purpose syntactic resource, the penn tree bank (ptb) dependency dataset, for reinforcing implicit syntactic knowledge learned by plms. the ssr mechanism learns transformation matrices to activate syntaxrelevant dimensions for emergent syntactic structures in a plm and then uses the enhanced syntactic representations for event prediction. to the best of our knowledge, this is the first work in this direction. our results suggest that syntactic reinforcement mechanisms are important for event extraction research and contribute to the understanding of utilizing innate syntactic representation in plms. our contributions are three-fold: ## related work event extraction is challenging as it concerns more than just binary relations. among efforts to tackle the task on the sentence level, syntactic knowledge has always been relevant @xcite . early approaches to ee are feature-based methods (e.g., @xcite ), in which syntactic features such as dependency tree parsing play a significant role. they are replaced by representation-based methods, which rely on powerful pre-trained language representations, from word2vec to large language models @xcite . existing studies that explore syntactic enhancement can be largely divided into three categories: (1) using syntactic knowledge as auxiliary information for encoding, e.g., @xcite ; @xcite , (2) proposing novel architectures using parsed dependency, e.g., @xcite ; @xcite , (3) using syntactic trees to build graphs and performing graph neural network learning, e.g., @xcite ; @xcite ; lai et al. (2020); cui et al. (2020) ; @xcite . however, most of these studies rely on external parsers, which lead to error propagation foot_0 . recently, probing studies on plms have found that syntactic structures can be emergent and intrinsic to language representation @xcite @xcite . this indicates a potential alternative approach to the existing ee studies. in this paper, we propose a soft syntactic reinforcement mechanism to reinforce intrinsic syntactic knowledge learning. we use a humanannotated, task-independent parse tree bank for pre-training, instead of parsed dependency results of the task dataset from a trained parser. document-level ee, with argument entities scattering across sentences, has been formulated as converting the text containing event information into structured event description based on predefined templates, which was first defined as template filling in the muc paradigm @xcite . state-of-the-art models include the following: mmr @xcite ) is a multi-turn and multi-granularity, machine reading comprehensionbased model. nst @xcite proposes a sequence tagging model that uses a gate mechanism to merge sentence and paragraph representations. tempgen @xcite frames the task as a template generation task, incorporating a copy mechanism that takes the top-k important cross-attentions as copy distributions into bart. ricb @xcite leverages redundant information and closed boundary loss. recent work in event extraction has also highlighted the potential of generative methods, particularly in few-shot and zero-shot learning scenarios @xcite @xcite . however, their performance in fully supervised event extraction, both at the sentence and document levels, often lags behind that of existing extractive methods. this is due to challenges including the complexity of accurately generating event structures and the tendency for generative models to produce hallucinations, raising concerns about their reliability in practical applications @xcite . in this work, we adopt the extractive approach and our model outperforms the state-ofthe-art generative methods @xcite @xcite @xcite . ## proposed approach in this section, we first introduce the intuition of intrinsic syntactic encoding (section 3.1). then, we describe the proposed soft syntactic reinforcement (ssr) mechanism (section 3.2), and the syntax-reinforced event (sre) model that incorporates the ssr mechanism (section 3.3). ## case studies to better understand how our proposed ssr mechanism helps ee, we present two case studies based on the ace05 and the muc-4 development set, which are shown in ## conclusion in this paper, we propose a soft syntactic reinforcement mechanism for neural event extraction. to our best knowledge, this is the first work exploring mechanisms that enhance syntactic knowledge intrinsically captured by plms. experiments on both sentence-level and document-level ee benchmark datasets show that our proposed method achieves state-of-the-art results and significantly improves recall for document-level ee.