Incomplete Utterance Rewriting with Editing Operation Guidance and Utterance Augmentation (2024)

Abstract: Although existing fashionable generation methods on Incomplete Utterance Rewriting (IUR) can generate coherent utterances, they often result in the inclusion of irrelevant and redundant tokens in rewritten utterances due to their inability to focus on critical tokens in dialogue context. Furthermore, the limited size of the training datasets also contributes to the insufficient training of the IUR model. To address the first issue, we propose a multi-task learning framework EO-IUR (Editing Operation-guided Incomplete Utterance Rewriting) that introduces the editing operation labels generated by sequence labeling module to guide generation model to focus on critical tokens. Furthermore, we introduce a token-level heterogeneous graph to represent dialogues. To address the second issue, we propose a two-dimensional utterance augmentation strategy, namely editing operation-based incomplete utterance augmentation and LLM-based historical utterance augmentation. The experimental results on three datasets demonstrate that our EO-IUR outperforms previous state-of-the-art (SOTA) baselines in both open-domain and task-oriented dialogue.

## introduction to express concisely and conveniently in multiturn dialogues, speakers tend to use incomplete utterances which usually omit (i.e., ellipsis) or refer back (i.e., coreference) to the history of dialogue context. @xcite reported that more than 70% of utterances exhibited the phenomena of coreference and ellipsis, particularly in pro-drop languages like chinese. this presents a significant challenge for the application of natural language understanding, particularly in the context of virtual assistants and customer support systems @xcite . to address this * corresponding author @xcite utterance who is tolstoy? he is the author. he tolstoy is the author of anna karenina. furthermore, the majority of annotated iur datasets are relatively small in size (e.g., the size of cqr @xcite is only 0.64k). despite the existence of numerous data augmentation methods in nlp, they are designed for ordinary documents. the structure of dialogue is more complex and personal pronouns appear with greater frequency. however, there is no research on data augmentation in iur. to address the first issue, we propose a multi-task learning framework, eo-iur (editing operation-guided incomplete utterance rewriting), that incorporates editing operation labels generated by the sequence labeling model to guide the generation model. this allows the decoder of the generation model to focus on the critical tokens in historical utterances and incomplete utterances, with the use of four types of defined editing operations. furthermore, we introduce a token-level heterogeneous graph to represent dialogues, which enables the model to learn the syntactic structure corresponding to the omitted tokens and the relationships between coreference-related tokens. to address the second issue, we propose a twodimensional utterance augmentation strategy for iur, namely editing operation-based incomplete utterance augmentation and llm-based historical utterance augmentation. these are used to augment incomplete utterances and the historical utterances, respectively. the experimental results on two chinese datasets and one english dataset show that our proposed model outperforms several sota baselines significantly. ## conclusion in this paper, we introduce the editing operation labels generated by the editing operation labeling to guide the generation model to focus on critical tokens. furthermore, we introduce a token-level heterogeneous graph to learn the syntactic structure and the relationships among coreference-related tokens. we also propose a two-dimensional utterance augmentation strategy to further boost the model. the experimental results on three datasets show that our eo-iur outperforms previous sota baselines significantly. in the future, we intend to employ large language models (llms) to facilitate utterance generation and to reduce the number of redundant tokens in rewritten utterances.