Mining Clues from Incomplete Utterance: A Query-enhanced Network for Incomplete Utterance Rewriting (2022)

Abstract: Incomplete utterance rewriting has recently raised wide attention. However, previous works do not consider the semantic structural information between incomplete utterance and rewritten utterance or model the semantic structure implicitly and insufficiently. To address this problem, we propose a QUEry-Enhanced Network(QUEEN) to solve this problem. Firstly, our proposed query template explicitly brings guided semantic structural knowledge between the incomplete utterance and the rewritten utterance making model perceive where to refer back to or recover omitted tokens. Then, we adopt a fast and effective edit operation scoring network to model the relation between two tokens. Benefiting from extra information and the well-designed network, QUEEN achieves state-of-the-art performance on several public datasets.

## introduction multi-turn dialogue modeling, a classic research topic in the field of human-machine interaction, serves as an important application area for pragmatics @xcite and turing test. the major challenge in this task is that interlocutors tend to use incomplete utterances for brevity, such as referring back to (i.e., coreference) or omitting (i.e., ellipsis) entities or concepts that appear in dialogue history. as shown in no, smith does not care about the type of cuisine. 不，史密斯不关心菜肴的类型。 early works use coreference resolution methods @xcite to identify the entity that a pronoun refers to. however, they ignore the more common cases of ellipsis. so the text generation-based methods @xcite are introduced to generate the rewritten sequence from the incomplete sequence by jointly considering coreference and ellipsis problems. though effective, generation models neglect a key trait of the iur task, where the main semantic structure of a rewritten utterance is usually similar to the original incomplete utterance. so the inherent structure-unawareness and uncontrollable feature of generation-based models impede their performances. for semantic structure-aware methods, @xcite utilize an edit operation matrix (e.g., substitution, insertion operations) to convert an incomplete utterance into a complete one. they formulate this task as a semantic segmentation problem with a cnn-based model @xcite on the matrix to capture the semantic structural relations between words implicitly. @xcite attempt to add additional semantic information to language models @xcite by annotating semantic role information but it is time-consuming and costly. @xcite propose a semi-autoregressive generator using a tagger to model the some considerable overlapping regions between the incomplete utterance and rewritten utterance, yet only implicitly learn the difference between them. although these methods maintain some similarities between the incomplete utterance and the rewritten utterance (i.e., the overlap between them), it is difficult for these methods to explicitly model the semantic structure, especially the difference between the two utterances, ignoring the information in the incomplete utterance, such as which tokens are more likely to be replaced and which positions are more likely to require the insertion of new tokens. therefore, there are still limitations of existing methods for iur task, especially in jointly considering coreference and ellipsis cases and better utilizing semantic structural information. this paper proposes a simple yet effective query-enhanced network (queen) to solve the iur task. queen jointly considers coreference and ellipsis problems that frequently happen in multi-turn utterances. specifically, we propose a straightforward query template featuring two linguistic properties and concatenate this query with utterances as input text. this query explicitly brings semantic structural guided information shared between the incomplete and the rewritten utterances, i.e., making model perceive where to refer back to or recover omitted tokens. we regard the rewritten utterance as the output from a series of edit operations on the incomplete utterance by constructing a token-pair edit operation matrix, which attempts to model the the overlap between the incomplete utterance between the rewritten utterance. different from @xcite , we adopt a well-designed edit operation scoring network on the matrix to perform incomplete utterance rewriting, which is faster and more effective. queen brings semantic structural information from linguistics into the model more explicitly and avoids unnecessary overheads of labeled data from other tasks. experiments on several iur benchmarks show that queen outperforms previous state-ofthe-art methods. extensive ablation studies also confirm that the proposed query template makes key contributions to the improvements of queen. ## conclusion we propose a simple yet effective query-enhanced network for iur task. our well-designed query template explicitly brings semantic structural information to improve the ability to predict correct edit operations between incomplete utterance and complete one. experiments have shown that our model with this well-designed query achieves promising results than previous methods.