Envisioning Future from the Past: Hierarchical Duality Learning for Multi-Turn Dialogue Generation (2023)

Abstract: In this paper, we define a widely neglected property in dialogue text, duality, which is a hierarchical property that is reflected in human behaviours in daily conversations: Based on the logic in a conversation (or a sentence), people can infer follow-up utterances (or tokens) based on the previous text, and vice versa. We propose a hierarchical duality learning for dialogue (HDLD) to simulate this human cognitive ability, for generating high quality responses that connect both previous and follow-up dialogues. HDLD utilizes hierarchical dualities at token hierarchy and utterance hierarchy. HDLD maximizes the mutual information between past and future utterances. Thus, even if future text is invisible during inference, HDLD is capable of estimating future information implicitly based on dialogue history and generates both coherent and informative responses. In contrast to previous approaches that solely utilize future text as auxiliary information to encode during training, HDLD leverages duality to enable interaction between dialogue history and the future. This enhances the utilization of dialogue data, leading to the improvement in both automatic and human evaluation.

## introduction dialogue generation achieves great success in recent years. in this task, a dialogue session lasts for multiple turns, and the goal is to predict the response to the previous context. researchers have explored various aspects of dialogue generation models including context-aware @xcite , persona-based @xcite and knowledgegrounded @xcite , etc. because dialogue generation is a context-aware process, there is a large body of literature on the context modeling among all research lines. in the pilot studies, contexts are modeled as the concatenation of previous utterances @xcite intra-duality. figure 1 : (a) inter-duality: the dialogue is divided into two parts. according to the first two utterances, we can infer that the follow-up part is related to the scenic spot commodities. similarly, based on the last two utterances, we can infer that the previous conversation is about tourism. (b) intra-duality: in an utterance, given the blue part, people know that the follow-up part is the reason. given the pink part, people know that the previous tokens may explain the result of the argument. 2015), organized in a non-hierarchical way. a dialogue is later considered to have hierarchical characteristics in which both token semantics within an utterance and logic between utterances need consideration @xcite . the hierarchical context modeling among utterances can be further improved with a global hidden variable @xcite indicating topic, emotion, or persona. at the same time, in addition to modeling text at multiple hierarchies, some researchers have also committed to mining the intensive interaction among the previous utterances sufficiently @xcite @xcite . in summary, most context modeling approaches focus on the previous utterances and ignore the followup utterances. in recent years, researchers @xcite @xcite realize that a response of high quality should not only be related to dialogue history but also connect the follow-up parts. however, they only treat utterances after the target response (i.e., dialogue future) as auxiliary information to encode during training, which violates a human cognitive ability in daily conversations: given a dialogue segment, people can infer future contents; also, people can imagine what the interlocutors discussed before. in the whole process, there are interactions between visible dialogue parts and human-envisioned parts. we name the property in dialogue texts that humans take advantage of to infer unknown previous or follow-up parts as duality foot_0 . inadequate consideration of duality ignores the interaction between dialogue history and the future, resulting in underutilized future information. duality in dialogues has two key characteristics. firstly, as a property in dialogues, duality can be viewed from a hierarchical viewpoint. at token hierarchy, intra-utterance duality (abbreviated as intra-duality) is reflected in that preceding words and the subsequent words can infer each other. similarly, at utterance hierarchy, inter-utterance duality (abbreviated as inter-duality) is reflected in that utterances before and after the target response can infer each other. figure 1(a ) and (b) illustrate dualities at two hierarchies. secondly, due to the contextaware nature of dialogues, intra-duality does not play a positive role alone and must be tightly integrated with inter-duality. in this paper, from the point of view of dialogue duality, we propose the hierarchical duality learning for dialogue (hdld) in order to fully exploit dialogue text for better generation. in hdld, a dialogue session is divided into two parts: past and future. at the utterance hierarchy, there is a forward generation model and a backward generation model working in a dual learning framework. the forward model takes the past utterances as context and predicts utterances in the future part. predicted outputs can be used as context for the backward model to back-predict the past utterances. the backward model performs a dual process. to integrate dualities at two hierarchies, hdld employs a cross-hierarchy distillation mechanism: guided by both future and past contexts, we distill both context-related and intra-dual knowledge from the ground truth, then we use the distilled knowledge to train the models. through this cross-hierarchy distillation mechanism, hdld couples hierarchies tightly, improving generation quality at multiple granularities. we design a two-stage joint optimization of the forward and the backward model that maximizes the mutual information between the past and future. thus, during inference even if future information is invisible, hdld is capable of envisioning future information implicitly only based on the dialogue history and generates both coherent and informative responses. we conduct multi-turn dialogue generation experiments on two public datasets, dailydialog @xcite and opensubtitles @xcite . compared with previous works that also consider future utterances, exploiting dialogue duality helps better utilize dialogue data, leading to the improvement in both automatic and human evaluation. to sum up, our contributions can be summarized as follows: ## method we first present the preliminary in § 3.1. then, we introduce a two-stage optimization that exploits dualities at two hierarchies in § 3.2, and some important details during training and inference in § 3.3. ## experimental setups baselines we compare hdld with baselines in two groups. the first group introduces future utterances to enhance the model: nexus @xcite introduces a code space learned from the whole dialogue and samples from the code space during testing to estimate future context. regdg @xcite ) trains a teacher model that has access to the whole dialogue to teach a student model which only takes past utterances as input. prophetchat @xcite first predicts two steps forward, generating multiple responses and future utterances, which are used to squeeze the final response along with the dialogue history. the second group considers the relationship among tokens in an utterance comprehensively: d2gpo @xcite augments the training with a data-dependent gaussian prior distribution, which is generated in pre-procession based on the training set. adalabel @xcite introduces an auxiliary decoder that uses a bidirectional attention to dynamically estimate a token distribution at each time step. besides, we also compare to hred @xcite , a pilot work for hierarchical context modeling, and transformer @xcite , the backbone of adalabel, d2gpo and hdld. datasets we conduct multi-turn dialogue generation experiments on two public datasets: daily-dialog @xcite and opensubtitles @xcite . dailydialog contains high quality multi-turn dialogues collected from daily conversations while opensubtitles contains dialogues from movies. as a preprocessing step, we exclude dialogue sessions shorter than 4 utterances from the dataset, and we create samples where the contexts consist of no more than 100 tokens. we split a pre-processed dialogue into two parts: dialogue history and future. baselines that utilize future utterances have access to two parts and those do not utilize future utterances only take the historical part as contexts. in this way, we ensure all experiments have access to the same amount of training samples, which is a fair setting. due to our unique task settings, we cannot directly cite any numbers from other papers and we reproduced every baseline. detailed datasets statistics are in hyper-parameters we use grid-search to select the best hyper-parameters. the search ranges for learning rate and batch size are {0.00006, 0.00008, 0.00010, 0.00012} and {64, 128, 256, 520, 720, 1000}, respectively. the adopted learning rate for hdld is 0.00008 and the mini-batch size is 128 on dailydialog and 720 on opensubtitles. as for other important settings: the warmup steps are 4000. we use adam optimizer with β = (0.9, 0.98). both attention dropout and activation dropout are 0.1. we select the best parameters based on perplexity on the validation set. all experiments adopt greedy decoding. early stop patience is 10 epochs on dailydialog and 5 on opensubtitles. some baseline-specific hyper-parameters not mentioned follow the original papers. the training is performed on two a40 gpus. evaluation metrics we consider three automatic evaluation metrics: bleu @xcite , distinct (dist) @xcite , and bleurt @xcite . bleu score measures the word overlap between generated responses and the ground truth. dist score measures the ratio of unique n-grams in the generated responses. because these two metrics are only sensitive to lexical variation, we evaluate bleurt, an advanced learned semantic-sensitive evaluation metric. for human evaluation, we recruit five evaluators to manually judge 500 samples from all models in blind testing. all responses are re-capitalized and de-tokenized fairly. evaluators rate samples from three aspects: readability (read.): how readable the response is; coherency (coh.): whether a response is coherent with the context; and informative (info.): whether the response is informative enough. in each aspect, the evaluator can score at '0' for bad, '1' for borderline, or '2' for good. the salary for each evaluator is 1 dollar per 10 samples. to give a fair salary, we first evaluate 50 samples by ourselves, calculate the time and effort, and set this amount (samples evaluated by ourselves are just for evaluating the salary, which is not given to evaluators and not reported in the final results). ## results comparison with prior works human evaluation results are shown in ## conclusion this paper first augments the dialogue generation task with hierarchical duality in dialogue text which was widely ignored. we propose a hierarchical dual learning framework hdld to utilize both interduality and intra-duality, and couple them tightly through the cross-hierarchy distillation mechanism. both automatic experiments and human judgment demonstrate the effectiveness of hierarchical duality in generating dialogues, which helps hdld outperform strong baselines. in the future, more under-utilized properties except for duality in dialogues are to be explored. we will further explore the hierarchical characteristics in natural language.