Context-Aware Tracking and Dynamic Introduction for Incomplete Utterance Rewriting in Extended Multi-Turn Dialogues (2024)

Abstract: Incomplete utterance rewriting (IUR) aims to reconstruct the utterance with omitted information and pronouns to be standalone and complete based on the context. The existing works predominantly focus on simple ellipsis and coreference problems in brief multi-turn dialogues. But in actual scenarios: 1) the context of the dialogues frequently comprises multiple similar candidates for ellipsis and coreference resolution, pouring to confuse. 2) the number of turns tends to be more extensive, while the content with various topics also grows more complex. This paper proposes a novel method called CaT to address these issues. In particular, we first devise a tacker model, distilled from GPT4-turbo, to adopt Context Tracking that dynamically updates a list of key phrases turn by turn, as accurate candidates for ellipsis and coreference resolution. Second, we further present the Dynamic Context Introduction mechanism to filter irrelevant preceding contexts that are not relied on by any element within the key phrase list to condense extended dialogues. Comprehensive experiments indicate that our solution provides a significant improvement over the existing baselines, and achieves state-of-the-art on three benchmarks.

## introduction incomplete utterance rewriting (iur) serves as a vital component in multi-turn dialogue systems, which reconstruct the current utterance by integrating the omitted information and resolving coreference based on the context to ensure its completeness. currently, iur is widely employed in prevalent tasks such as information retrieving @xcite , question answering @xcite , and web searching @xcite . with the advent of large language models (llms) (ouyang et al., 1 https://github.com/ygxw0909/cat figure 1 : an example of iur in a product consulting and sales scenario. within, green denotes the user's input, while blue signifies the salesperson's. the full dialogue is upon 35 turns. 2022; openai, 2023) and breakthroughs in multiturn dialogue-oriented tasks, iur has received increasing attention in recent years. in contrast to the brief (in less than 10 turns on average) and simple (usually centering around a single topic) dialogue observed between users and systems, which is primarily focused on by current iur researches as the benchmark @xcite @xcite @xcite , customer service and sales scenarios often entail more complex dialogues between users and salespersons, as shown in figure 1 . these scenarios present two principal challenges: a) complex ellipsis and coreference problems delivered by the diverse dialogue content and topics. for instance, according to the demand for reimbursement, several insurance products are mentioned throughout the dialogues, which may lead to confusion in figuring out which product is the user inquiring about. b) extended turns, at times, reaching upwards of hundreds, encompassing multiple distinct topics. within the dialogue in figure 1 , the topic transitions from consulting the function of medical insurance to recommending various products for purchase. some of the existing methods @xcite @xcite apply cross attention between the current utterance and its context to gain the rewriting matrix, using it for token-level actions (such as insertion, deletion, etc.). however, these methods are limited to copying tokens from the context, lack the flexibility for modifications, and also struggle with lengthy contexts. meanwhile, many other methods @xcite @xcite decompose the rewriting task into action prediction and span prediction, performing sequence tagging for action selection in each token of the current utterance and predicting candidate spans in the context. these methods do not adequately address the issue of ellipsis and coreference resolution among various similar candidates. @xcite jointly optimizes important tokens picking and generative utterances rewriting, but the design of the picker is too straightforward and still ignores the challenges. to handle the challenges, this paper proposes a novel method called cat (context-aware tracking and dynamic introduction), adopting context tracking (ct) and dynamic context introduction (dci) for resolve ellipsis and coreference, and requisite irrelevant context filtering: 1) context tracking. to facilitate effective tracking and extracting the key content of complex dialogues that the current utterance depends on, we leverage a trainable tracker and propose context tracking to perpetually maintain a key-phrase list (kp list) for ellipsis and coreference resolution. specifically, an empty kp list is initialized at first, as the dialogue advances, the tracker dynamically updates the kp list turn by turn. the updates include: a) add when new key phrases occur. b) delete when the topic shifts and key phrases are not dependent anymore. c) reserve when the previous key phrases may still be mentioned. 2) dynamic context introduction. in order to further filter out the turns which irrelevant to the current utterance, while ensuring the contextual integrity required for rewriting, the dynamic context introduction mechanism is applied. upon the update of the key phrases list, each key phrase is tagged with the index of the most recent turn it is mentioned. based on that, the mini-mal dependency context is obtained by filtering out the part preceding the furthest turn tagged in the key phrases list. ultimately, the rewriter takes the current utterance, kp list, as well as the minimal dependency context as input to generate the final complete utterance. to fully evaluate our proposed cat, we collect the real dialogues between users and salespersons from our platform with manual annotation and construct a new challenging benchmark insqr. experimental results across three datasets prove the effectiveness of our method. our contributions are summarized as follows: ## preliminaries given a dialogue context @xmath0, c 2 , ..., c | c|} and a following incomplete utterance @xmath1, u 2 , ..., u |u | }, where c @xmath2, c 2 , ..., c |c i | } is the utterance of the i-th turn, u j and c k are the tokens of the utterance. the goal of iur is to learn a mapping function, ## experiments datasets to comprehensively evaluate our method, we used the following three datasets covering both english and chinese: 1) insqr is a challenging chinese iur dataset constructed by the real dialogues between customers and sales collected from alipay 2 platform, as well as the manual annotation to provide the gold utterance. within, the longest dialogues can reach 100 turns, and the average turns is about 30, which is much longer than the existing datasets. the detailed comparison is shown in ## related work early works on iur primarily combine sequenceto-sequence based model with a copy mechanism to fetch the relevant information in the context @xcite @xcite . due to the strong representation capabilities of pre-trained models @xcite , run @xcite introduces a rewriting matrix derived from the embedding of context and utterance to adjudicate on a token-level whether insertion or replacement operations should be carried out. srl @xcite pre-identifies the candidate lexicon for the subject, predicate, and object within the context, and incorporates these as extra features into the encoder to achieve better performance. rast @xcite formulates the iur as sequence labeling, effectuating the rewrite by predicting actions (insertion, deletion, and none) and the spans of the context separately. hct @xcite enhances the action predictor of the previous work to a rule predictor, optimizing for the generation of modified words not exactly present in the context. queen @xcite proposed a query template that explicitly brings guided semantic structural knowledge between the incomplete utterance and the rewritten utterance. jet @xcite jointly optimizes important token picking and rewritten utterances generation @xcite , but the design of the picker is too straightforward to perform accurate recall in extended dialogues. @xcite employs continued pre-training on the t5 @xcite model to enhance its performance on iur, while @xcite and @xcite further devise fine-grained subtasks and refined model architectures based on the edit matrix. however, the existing benchmarks targeted by all the work referenced above are limited to simple ellipsis and coreference problems, while none of them pay attention to the challenges mentioned in section 1. ## conclusion in this paper, we presented a novel method for incomplete utterance rewriting among extended dialogues with complex content, called cat which is a component of a context tracker and a generative rewriter. context tracking is adopted by the tracker to maintain a list turn by turn, which includes the key phrase mentioned in the previous context that may still be referred to in the current utterance, as candidates to promote the performance of ellipsis and coreference resolution. furthermore, the dynamic context introduction is designed to filter the irrelevant previous turns and retain the minimal dependency context for the rewriter. extensive experiments indicate that cat achieves state-ofthe-art results on three benchmarks compared with the existing baselines. in future work, we will expand our work on a wider range of scenarios, and also explore the optimization of the tracker.