Retrieve-and-Sample: Document-level Event Argument Extraction via Hybrid Retrieval Augmentation (2023)

Abstract: Recent studies have shown the effectiveness of retrieval augmentation in many generative NLP tasks. These retrieval-augmented methods allow models to explicitly acquire prior external knowledge in a non-parametric manner and regard the retrieved reference instances as cues to augment text generation. These methods use similarity-based retrieval, which is based on a simple hypothesis: the more the retrieved demonstration resembles the original input, the more likely the demonstration label resembles the input label. However, due to the complexity of event labels and sparsity of event arguments, this hypothesis does not always hold in document-level EAE. This raises an interesting question: How do we design the retrieval strategy for document-level EAE? We investigate various retrieval settings from the input and label distribution views in this paper. We further augment document-level EAE with pseudo demonstrations sampled from event semantic regions that can cover adequate alternatives in the same context and event schema. Through extensive experiments on RAMS and WikiEvents, we demonstrate the validity of our newly introduced retrieval-augmented methods and analyze why they work.

## introduction transforming the large amounts of unstructured text on the internet into structured event knowledge is a critical, yet unsolved goal of nlp, especially when addressing document-level text. documentlevel event argument extraction (document-level eae) is the process of extracting informative event kernels from a document, which benefits many downstream applications, e.g., information retrieval, question answering, and event graph reasoning. figure 1 event arguments and identify the roles they take: the fsb (preventer), 53 young men (transporter), the illegal prayer hall (origin), syria (destination). retrieval-augmented methods have recently been successfully applied to many nlp tasks, e.g., dialogue response generation @xcite @xcite , machine translation @xcite @xcite and information extraction @xcite @xcite . these methods retrieve additional knowledge from various corpora to augment text generation, which allows models to (a) explicitly acquire prior external knowledge in a non-parametric manner, leading to great flexibility. (b) regard the retrieved reference instances as cues to generate text and learn by analogy. these retrieval-augmented methods use similarity-based retrieval, which is based on a simple hypothesis @xcite : the more x r (retrieved demonstration) resembles x (original input), the more likely y r (demonstration label) resembles y (input label), so it will help the generation. this hypothesis is intuitive: similar input results in similar output for most tasks @xcite . for example, in language modeling, dickens is the author of and dickens wrote will have essentially the same distribution over the next word. however, in document-level eae, x r resembles x cannot guarantee the equivalent distribution of y r and y in label space. in a document, only a few words are event arguments, while other distracting context can mislead similarity-based retrieval and cause demonstration label y r deviate from input label y. furthermore, document-level eae should predict not only the argument entity but also the correspondence between arguments and roles, which makes it challenging to find a demonstration with an identical event label to the original input. according to our statistics on rams dataset @xcite , only 16.51% of instances can recall a sample with the same event schema through similarity-based retrieval. this raises an interesting question: since document-level eae doesn't satisfy the hypothesis of similarity-based retrieval, how do we design the retrieval strategy for document-level eae? in this paper, we explore various retrieval settings. first, if similar documents cannot guarantee the same distribution of event labels, does it make sense to pursue x r to be similar to x in retrieval process? to answer this, we first retrieve x r , close to x in input space, as discrete demonstration to keep contextual semantic consistency (setting 1); then, since the essence of the above hypothesis is to pursue y r resembles y, why don't we directly retrieve y r similar with y as the reference? so we recall y r , close to y in label space, as discrete demonstration to alleviate the difficulty of learning the complex event pattern of y (setting 2); to find depth cues to guide the model, we want a demonstration that has equal distribution with input document in both input and label space. intuitively, it is impossible to retrieve the ideal demonstration in discrete space, so we try to sample a cluster of pseudo demonstrations in continuous space instead. recent works @xcite have shown that the vectors in an adjacency region can easily cover adequate alternatives of the same meaning. inspired by this intriguing observation, we sample pseudo demonstrations from the intersection of the adjacent regions of x and y, thus preserving both context and event schema consistency with the input (setting 3). we present a systematic evaluation for analyzing various retrieval settings and observe that given a document, (1) context-consistency retrieval (setting 1) helps the model identify the argument span more accurately than setting 2. this suggests that in-distribution demonstration contexts can contribute to performance gains by improving the ability to recognize argument spans; (2) schema-consistency retrieval (setting 2) makes the generated role labels more accurate than setting 1, which indicates that conditioning on the label space contributes to better performance by alleviating the difficulty of learning the complex event pattern; and (3) adaptive hybrid retrieval (setting 3) has achieved state-of-the-art (sota) performance among all generation-based baselines, indicating that this setting can generate diverse and faithful pseudo demonstrations with consistency in both input space and label space. overall, the contributions can be summarized as follows: training corpus encoder decoder encoder encoder decoder top-k training corpus input sequence target sequence encoder encoder decoder decoder encoder encoder decoder top-k retrieved doc (a) context-consistency retrieval-augmented doceae training corpus encoder decoder encoder retrieved label top-k target sequence input sequence (b) schema-consistency retrieval-augmented doceae transportperson preventer transporter origin destination [sep] ..., the fsb special forces detained 53 young men, ..., @xmath0... in syria input sequence the fsb [preventer] 53 young men [transporter] the illegal prayer hall [origin] syria [destination] target sequence encoder decoder training corpus retrieved doc encoder gaussian sampling top-k repeat k times (c) adaptive hybrid retrieval-augmented doceae for retrieval-augmented document-level eae, we first retrieve the top-k potentially helpful demonstrations (discrete or continuous), then fuse them into the decoder to generate role records (a sequence of (a, r) pairs). in the following, we first introduce how to reformulate document-level eae as retrieval-augmented generation (rag), then describe various retrieval settings. ## experiments we evaluate our model's performance on two commonly used document-level eae benchmarks and compare it to prior works. then we conduct additional analytical experiments on how the demonstration retrieval design affects performance. ## conclusion in this paper, we explore how to design retrievalaugmented strategy for document-level eae from the input and label distribution views. and our introduced retrieval strategies can recall demonstrations that can be helpful to demonstrate how the model should solve the task. we further propose a novel adaptive hybrid retrieval augmentation paradigm to generate the reference vectors as depth cues to improve the analogical capability of model. through extensive experiments on rams and wikievents datasets, we demonstrate the validity of our newly introduced retrieval-augmented models. in the future, we plan to adapt our method to other document-level extraction tasks, such as document-level relation extraction.