Focus on the Action: Learning to Highlight and Summarize Jointly for Email To-Do Items Summarization (2022)

Abstract: Automatic email to-do item generation is the task of generating to-do items from a given email to help people overview emails and schedule daily work. Different from prior research on email summarization, to-do item generation focuses on generating action mentions to provide more structured summaries of email text. Prior work either requires large amount of annotation for key sentences with potential actions or fails to pay attention to nuanced actions from these unstructured emails, and thus often lead to unfaithful summaries. To fill these gaps, we propose a simple and effective learning to highlight and summarize framework (LHS) to learn to identify the most salient text and actions, and incorporate these structured representations to generate more faithful to-do items. Experiments show that our LHS model outperforms the baselines and achieves the state-of-the-art performance in terms of both quantitative evaluation and human judgement. We also discussed specific challenges that current models faced with email to-do summarization.

## introduction automatic email to-do generation is the task of summarizing to-do items from given emails @xcite to help people overview overwhelming numbers of emails they receive every day @xcite and schedule their daily work. unlike prior research on emails such as generating email conversation summarization @xcite @xcite , keyword extraction @xcite @xcite and subject line generation @xcite , to-do items generation focuses more on action mentions to provide more structured summaries from email communications, which requires identifying important tasks to be performed among all the action items and aligning them with the right users @xcite . to tackle these challenges in to-do item generation, a two-stage framework @xcite is generally utilized to first extract the commitment sentence which is the most related to the to-do item and then generate to-do items based on those selected sentences. despite the effectiveness, several limitations exist: (1) extra annotations are usually needed for the first stage @xcite to learn the classifiers which require extra cost/expertise and are often hard to obtain in low-resourced settings; (2) any information loss in the identification stage might lead to bigger noises in the to-do generation stage; (3) directly extracting actions from less-structured text usually leads to unfaithful summaries @xcite that mismatch the relations between users and actions. to fill in these gaps, in this work, we propose a learning to highlight and summarize model, where we learn the important sentence identification module and to-do summarization module concurrently in an end-to-end manner to focus on the most salient actions, as well as to incorporate structured action representations to generate more faithful todos. one example is shown in figure 1 . specifically, we use an unsupervised approach to extract the salient sentences and actions by comparing them to the ground truth to-do. we extract action triplets from the text and construct an action graph to encode the structural information in the unstructured text. during training, the model learns to generate to-do items and identify highlights jointly. during prediction, the model utilizes the predicted highlights by modifying the attention distribution accordingly. furthermore, as extrinsic hallucinations involving named entities are common in automatic summarization tasks @xcite , we propose a perturbation technique based on person names to reduce the extrinsic hallucinations. we conduct experiments on the smarttodo corpus @xcite and show that our lhs model outperforms strong baselines on to-do item generation, demonstrating the effectiveness of this joint model on learning to highlight and summarize. by evaluating our lhs model on a different email corpus, we demonstrate that it generalizes well on a zero-shot condition. conversation summarization automatic to-do generation is also similar to the task of conversation summarization in the way that the input data contains both the current email in which the user promises to take an action and the previous email to which the user is replying. for extractive conversation summarization @xcite , statistical machine learning methods such as skip-chain crfs @xcite , svm with lda models @xcite , and multi-sentence compression algorithms @xcite have been used. when key information is scattered in multiple sentences, such methods had trouble in succinctness, fluency and naturalness @xcite . abstractive conversation summarization methods are more effective in these circumstances. these methods design hierarchical models @xcite , incorporate commonsense knowledge @xcite , or leverage conversational structures like dialogue acts (goo and chen, 2018), key point sequences @xcite , topic segments @xcite and stage developments @xcite . some recent research has also utilized structural information to detect important content in conversations @xcite @xcite . however, recent methods that explicitly identify salient content has not been used jointly with conversation summarization models. summarization tasks on email several summarization-like tasks have been proposed such as email text summarization @xcite @xcite , keyword and action extraction @xcite @xcite , subject line generation @xcite and to-do generation task @xcite . these previous works often deploy a two-stage framework that first extracts salient sentences from the email and then generates summary based on extracted sentences. however, any information loss in the first stage might lead to bigger noises in the to-do generation stage, and directly extracting actions from less-structured text might lead to unfaithful summaries @xcite . to fill these gaps, we propose to explicitly model actions within sentences in the email in a structured way, by using action graphs to model the actions, introducing an auxiliary classifier to identify the highlights, and further combining these with a decoder that attends to both the encoding of input tokens and extracted actions for to-do generation. ## methods to generate faithful to-do items that correlate actions with correct users from emails foot_0 , we propose to encode both text-level and action-level information as guidance signals. the architecture of lhs model is shown in figure 2 . we first extract the action mention ("who-doing-what" triples @xcite ) from the emails and construct an action graph (section 3.1), then we encode text and actions separately (section 3.2) and use a multi-scale decoder that attends to both email and action graphs to generate the to-do item (section 3.4). besides directly incorporating different levels of information, we further design an end-toend framework to utilize guidance signals such as important sentences and actions to help summarization models focusing more on the key to-dos by jointly learning a highlight classifier (section 3.3) with the summarization model and view the predicted results as priors to attention distributions in the decoder (section 3.4). ## conclusion in this work, we propose a simple yet effective learning to highlight and summarize framework (lhs) to learn to identify salient text and actions from both email text and the constructed action graph, and generate faithful to-do items jointly. experiments show that the proposed model outperforms the baselines significantly and achieves stateof-the-art performance in both quantitative evaluation and human judgement. we further demonstrated that our lhs framework generalized well to out-of-domain conditions. in the future, we plan to extend the email to-do item generations to other domains such as generating multiple to-do action items from conversation or meeting threads.