Incomplete Utterance Rewriting as Semantic Segmentation (2020)

Abstract: Recent years the task of incomplete utterance rewriting has raised a large attention. Previous works usually shape it as a machine translation task and employ sequence to sequence based architecture with copy mechanism. In this paper, we present a novel and extensive approach, which formulates it as a semantic segmentation task. Instead of generating from scratch, such a formulation introduces edit operations and shapes the problem as prediction of a word-level edit matrix. Benefiting from being able to capture both local and global information, our approach achieves state-of-the-art performance on several public datasets. Furthermore, our approach is four times faster than the standard approach in inference.

## introduction a dramatic progress has been achieved in singleturn dialogue modeling such as open-domain response generation @xcite , question answering @xcite , etc. by contrast, multi-turn dialogue modeling is still in its infancy, as users tend to use incomplete utterances which usually omit or refer back to entities or concepts appeared in the dialogue context, namely ellipsis and coreference. according to previous studies, ellipsis and coreference exist in more than 70% of the utterances @xcite , for which a dialogue system must be equipped with the ability of understanding them. to tackle the problem, early works include learning a hierarchical representation @xcite and concatenating the dialogue utterances selectively @xcite . recently, researchers focus on a more explicit and explainable solution: the task of incomplete utterance rewriting (iur, also known as context rewriting) (kumar and joshi, 2016; su et al., 2019; liu et al., * work done during an internship at microsoft research. ## related work the most related work to ours is the line of incomplete utterance rewriting. recently, it has raised a large attention in several domains. in question answering, previous works include non-sentential utterance resolution using the sequence to sequence based architecture @xcite , incomplete follow-up question resolution via a retrieval sequence to sequence model @xcite and sequence to sequence model with a copy mechanism @xcite . in conversational semantic parsing, @xcite proposed a novel approach which considers the structures of questions, while substitute insert none 𝑥 1 𝑥 2 𝑥 3 𝑥 4 𝑥 5 𝑥 6 𝑥 7 𝑥 8 … 𝑥 𝑁 𝐱 𝑐 1 𝑐 2 𝑐 3 𝑐 4 𝑐 5 𝑐 6 𝑐 7 𝑐 8 𝑐 9 𝑐 10 … 𝑐 𝑀 𝐜 𝐱 * = (𝑥 1 , 𝑐 2 , 𝑐 3 , 𝑐 4 , 𝑥 4 , 𝑥 5 , 𝑥 6 , 𝑐 6 , 𝑐 7 , 𝑐 9 , 𝑐 10 , 𝑥 7 , 𝑥 8 , … , 𝑥 𝑁 ) liu et al. (2019a) imposed an intermediate structure span and decomposed the incomplete utterance rewriting into two sub-tasks. in dialogue generation, pan et al. (2019) presented a cascaded model which first picks words from the context via bert, and then combines these words to generate the rewritten utterance, and su et al. (2019) distinguished the weights of context utterances and the incomplete utterance using a hyper-parameter λ. different from all of them, we formulate the task as a semantic segmentation task. our work is also closely related to coreference resolution. it is an active task that has been studied years, and deep learning based methods have achieved state-of-the-art performance via the paradigm of scoring span or mention pairs @xcite @xcite . researchers also explored to use unsupervised contextualized representations to enhance the coreference resolution. @xcite applied spanbert @xcite to enhance the span representation in coreference resolution, and @xcite formulated coreference resolution as query-based span prediction and employed spanbert to solve it as a machine reading task. the above works only focus on coreference resolution, while our work deals with coreference and ellipsis under a unified approach. from the perspective of the methodology, our work is correlated with directions of edit-based text generation and semantic similarity measurement. @xcite proposed a prototype-thenedit paradigm for open-domain response generation, while malmi et al. (2019) cast text generation as a text editing task and tackled it with a word-level edit matrix 𝑥 1 𝑥 2 𝑥 3 𝑥 4 𝑥 5 𝑥 6 𝑥 7 𝑥 8 … 𝑥 𝑁 𝑐 1 𝑐 2 𝑐 3 𝑐 4 𝑐 5 𝑐 6 𝑐 7 𝑐 8 𝑐 9 𝑐 10 … 𝑐 𝑀 conv conv pool conv conv pool conv conv deconv conv conv deconv conv conv feedforward skip connect skip connect context layer segmentation layer feature map matrix embedding bilstm 𝑐 1 𝑐 2 𝑐 3 𝑐 4 … 𝑐 𝑀 𝑥 1 𝑥 2 𝑥 3 𝑥 4 𝑥 5 𝑥 𝑁 … image pixel-level mask substitute insert none grass cat tree sky encoding layer 𝑐 𝑚 𝑥 𝑛 element cos bilinear similarity functions sequence tagging approach. our work is different from theirs since we model the editing process between two sentences as a semantic segmentation task. as for semantic similarity measurement, similar to us, both @xcite and @xcite used convolutional neural networks to capture similarities between sentences. ## incomplete utterance rewriting as semantic segmentation in this section, we will have a glance at the fundamental idea behind our approach: incomplete utterance rewriting as semantic segmentation. in a multi-turn dialogue, given the context utterances (x 1 , • • • , x t-1 ) and the incomplete utterance x t , iur is to rewrite x t to x * t using contextual information, where x * t has the same meaning with x t . the rewritten utterance x * t has self-contained semantics and can be understood solely. to produce x * t , our approach formulates the problem as a semantic segmentation task. concretely, we concatenate all the context utterances to produce an m -length word sequence @xmath0, c 2 , • • • , c m ). to separate context utterances in different turns, we insert a special word [s] between each context utterance. meanwhile, the incomplete utterance is denoted by @xmath1, x 2 , • • • , x n ). as mentioned, the rewritten utterance x * can be obtained by editing the incomplete utterance x with in-dialogue words (i.e. words in c). to model edit operations between x and c, we define a m × n matrix y , where the entry y m,n indicates the edit type between c m and x n . there are three kinds of edit types: substitute means replacing the span in x with the corresponding context span in c; insert aims to insert the context span before a certain token in x, and none represents no operation. for example, as shown schematically in figure 1 , we can edit x by replacing (x 2 , x 3 ) with (c 2 , c 3 , c 4 ) and insert (c 6 , c 7 , c 9 , c 10 ) before x 7 . it is notable that we append a special word [e] to x, to enable insert take place after x. more concrete examples can be found in section 5.3. then, we propose to emit such a matrix y in a way analogous to the task of semantic segmentation. specially, we build a m × n feature map via capturing the word-to-word relevance between c and x. taking the feature map as an image, the output word-level edit matrix y is parallel to the pixel-level mask in semantic segmentation, which bridges iur and semantic segmentation. such a formulation comes with several key advantages: (i) easy: compared with traditional methods generating the rewritten utterance from scratch, such a formulation introduces edit operations to lower the difficulty of generation; (ii) fast: these edits are predicted concurrently, so our model naturally enjoys the fast inference speed than conventional models which decode word by word; (iii) transferable: taking the formulation as a bridge between iur and semantic segmentation, one can transfer empirical models from the community of semantic segmentation with ease. ## methodology as shown in figure 2 , our approach firstly obtains the word-level edit matrix through three neural layers. then based on the word-level edit matrix, it applies a generation algorithm to produce the rewritten utterance. since the model yields a ushaped architecture (illustrated later), we name our approach as rewritten u-shaped network (run). ## experiments in this section, we conduct thorough experiments to demonstrate the superiority of our approach. ## conclusion & future work in this paper, we present a novel and extensive approach which formulates the incomplete utterance rewriting as a semantic segmentation task. on top of the formulation, we carefully design a u-shaped rewritten network, which outperforms existing baselines significantly on several datasets. in the future, we will investigate on extending our approach to more areas.