M ail E x: Email Event and Argument Extraction (2023)

Abstract: In this work, we present the first dataset, MailEx, for performing event extraction from conversational email threads. To this end, we first proposed a new taxonomy covering 10 event types and 76 arguments in the email domain. Our final dataset includes 1.5K email threads and ~4K emails, which are annotated with a total of ~8K event instances. To understand the task challenges, we conducted a series of experiments comparing three types of approaches, i.e., fine-tuned sequence labeling, fine-tuned generative extraction, and few-shot in-context learning. Our results showed that the task of email event extraction is far from being addressed, due to challenges lying in, e.g., extracting non-continuous, shared trigger spans, extracting non-named entity arguments, and modeling the email conversational history. Our work thus suggests more future investigations in this domain-specific event extraction task.

## introduction email has been one of the most widely used communication mediums, especially in professional and work environments. as the number of email users continues to rise, service providers are constantly looking for ways to improve user experience. with advancements in machine learning and natural language processing, email platforms have introduced a range of features aimed at helping users manage their inboxes and automate tasks @xcite @xcite , concurrent with research on request identification @xcite , intent identification @xcite , meeting summarization @xcite , task management @xcite , etc. however, most of the existing work touches only one specific aspect of email information and thus cannot connect with other relevant tasks. for instance, in @xcite even after identifying emails containing requests, users will still need to manually search through the lengthy emails to find the actual request and then manage the tasks with separate tools. on the other hand, existing exploration on the email data can cover only a subset of potential events in email communications (e.g., requests or to-do tasks), whereas there are many others that also commonly happen and need proper management (e.g., delivery of data or information). to facilitate more comprehensive downstream tasks on email data, in this paper, we introduce the task of event extraction on email threads. event extraction @xcite , or ee, is the task of extracting a specific occurrence of an event and its arguments. it is an important step for downstream tasks such as information retrieval, question answering, knowledge base population, etc. ee has long been studied in the context of news articles @xcite @xcite . as far as we know, there has not been such a dataset in the email domain. on the other hand, ee on email threads brings unique challenges such as performing information extraction in a conversational setting and needing to handle much longer and more verbose arguments, which cannot be studied with existing datasets. to fill this gap, we first developed a taxonomy to describe events and arguments in email data. specifically, we designed a set of 10 event classes and 76 arguments using the speech act theory of @xcite . our event classes cover proposals, amendments, and deliveries of actionable events on meetings, actions, and data items (section 2). unlike existing ee datasets, each trigger in the email ee task is described with one verb and one noun act (e.g., deliver verb data noun ), and the arguments are often long-span, non-named entities (e.g., a description of the meeting agenda), which make the task much more challenging. based on this taxonomy, we then proposed a new dataset, mailex, which consists of 1.5k email threads and ∼4k emails annotated with ∼8k events. the dataset achieves a substantial agreement between annotators. comparing three approaches, i.e., fine-tuned sequence labeling based on bert @xcite , fine-tuned generative ee based on bart @xcite , and in-context learning using gpt-3.5, 2 we analyze the challenges in trigger and argument extraction. our results highlight the need for advancements in handling non-continuous, shared triggers and long-span, non-named entity arguments while emphasizing the importance of effectively modeling email history. moreover, the incontext learning of gpt-3.5 yields much worse performance, suggesting the challenge of this domainspecific task in the few-shot setting. ## conclusion in this paper, we have proposed a new task of extracting events and their arguments from conversational email data. to motivate future research in this direction, we also present a new dataset mailex, including a new taxonomy to describe common events mentioned in emails. we also conduct a series of evaluations on mailex, concluding that email ee is far from being addressed and more advanced methodologies are needed.