Envisioning Future from the Past: Hierarchical Duality Learning for Multi-Turn Dialogue Generation
Year: 2023

Abstract:
In this paper, we define a widely neglected property in dialogue text, duality, which is a hierarchical property that is reflected in human behaviours in daily conversations: Based on the logic in a conversation (or a sentence), people can infer follow-up utterances (or tokens) based on the previous text, and vice versa. We propose a hierarchical duality learning for dialogue (HDLD) to simulate this human cognitive ability, for generating high quality responses that connect both previous and follow-up dialogues. HDLD utilizes hierarchical dualities at token hierarchy and utterance hierarchy. HDLD maximizes the mutual information between past and future utterances. Thus, even if the future text is invisible during inference, HDLD is capable of estimating future information implicitly based on dialogue history and generates both coherent and informative responses. In contrast to previous approaches that solely utilize future text as auxiliary information to encode during training, HDLD leverages duality to enable interaction between dialogue history and the future. This enhances the utilization of dialogue data, leading to the improvement in both automatic and human evaluation.

heading: 1 Introduction
text: Dialogue generation achieves great success in recent years. In this task, a dialogue session lasts for multiple turns, and the goal is to predict the response to the previous context. Researchers have explored various aspects of dialogue generation models including context-aware (Tian et al., 2017), persona-based (Qian et al., 2018) and knowledgegrounded (Zhao et al., 2020; Tao et al., 2021), etc. Because dialogue generation is a context-aware process, there is a large body of literature on the context modeling among all research lines.
In the pilot studies, contexts are modeled as the concatenation of previous utterances (Sordoni et al.,
∗Equal contribution. †Corresponding author: Rui Yan (ruiyan@ruc.edu.cn).
2015), organized in a non-hierarchical way. A dialogue is later considered to have hierarchical characteristics in which both token semantics within an utterance and logic between utterances need consideration (Serban et al., 2016). The hierarchical context modeling among utterances can be further improved with a global hidden variable (Serban et al., 2017) indicating topic, emotion, or persona. At the same time, in addition to modeling text at multiple hierarchies, some researchers have also committed to mining the intensive interaction among the previous utterances sufficiently (Tao et al., 2019; Zhou et al., 2018; Wu et al., 2017). In summary, most context modeling approaches focus on the previous utterances and ignore the followup utterances. In recent years, researchers (Shen et al., 2018; Feng et al., 2020; Liu et al., 2022) realize that a response of high quality should not only be related to dialogue history but also connect the follow-up parts. However, they only treat
7382
utterances after the target response (i.e., dialogue future) as auxiliary information to encode during training, which violates a human cognitive ability in daily conversations: given a dialogue segment, people can infer future contents; also, people can imagine what the interlocutors discussed before. In the whole process, there are interactions between visible dialogue parts and human-envisioned parts. We name the property in dialogue texts that humans take advantage of to infer unknown previous or follow-up parts as duality1. Inadequate consideration of duality ignores the interaction between dialogue history and the future, resulting in underutilized future information.
Duality in dialogues has two key characteristics. Firstly, as a property in dialogues, duality can be viewed from a hierarchical viewpoint. At token hierarchy, intra-utterance duality (abbreviated as intra-duality) is reflected in that preceding words and the subsequent words can infer each other. Similarly, at utterance hierarchy, inter-utterance duality (abbreviated as inter-duality) is reflected in that utterances before and after the target response can infer each other. Figure 1(a) and (b) illustrate dualities at two hierarchies. Secondly, due to the contextaware nature of dialogues, intra-duality does not play a positive role alone and must be tightly integrated with inter-duality.
In this paper, from the point of view of dialogue duality, we propose the Hierarchical Duality Learning for Dialogue (HDLD) in order to fully exploit dialogue text for better generation. In HDLD, a dialogue session is divided into two parts: past and future. At the utterance hierarchy, there is a forward generation model and a backward generation model working in a dual learning framework. The forward model takes the past utterances as context and predicts utterances in the future part. Predicted outputs can be used as context for the backward model to back-predict the past utterances. The backward model performs a dual process. To integrate dualities at two hierarchies, HDLD employs a cross-hierarchy distillation mechanism: guided by both future and past contexts, we distill both context-related and intra-dual knowledge from the ground truth, then we use the distilled knowledge to train the models. Through this cross-hierarchy distillation mechanism, HDLD couples hierarchies tightly, improving generation quality at multiple
1Our definition of ‘duality’ is different from that in papers on dual learning (He et al., 2016). We discuss details in section 2.2
granularities. We design a two-stage joint optimization of the forward and the backward model that maximizes the mutual information between the past and future. Thus, during inference even if future information is invisible, HDLD is capable of envisioning future information implicitly only based on the dialogue history and generates both coherent and informative responses.
We conduct multi-turn dialogue generation experiments on two public datasets, DailyDialog (Li et al., 2017) and OpenSubtitles (Lison and Tiedemann, 2016). Compared with previous works that also consider future utterances, exploiting dialogue duality helps better utilize dialogue data, leading to the improvement in both automatic and human evaluation. To sum up, our contributions can be summarized as follows: • We first define the dialogue duality, which was widely neglected in previous works. We propose the hierarchical duality learning framework for dialogue (HDLD) to augment multi-turn dialogue generation by exploiting duality. • To realize HDLD, We design a cross-hierarchy distillation mechanism to couple hierarchies, and a joint optimization of two models that can be theoretically proved to augment generation. • We show the effectiveness of HDLD in generating coherent and informative responses.

heading: 2 Related Work

heading: 2.1 Dialogue Generation with Future Utterances
text: In open-domain dialogue generation, the focus of this paper, most models are context-aware, predicting the target response based on previous context representations. With the development of this field, researchers began to pay attention to the future context: NEXUS (Shen et al., 2018) introduces a posterior distribution for estimating the future information as a hidden variable and decodes responses accordingly. RegDG (Feng et al., 2020) trains a teacher model with both past and future utterances and transfers the whole-context knowledge to a student model which has only access to past utterances. ProphetChat (Liu et al., 2022) first predicts two utterances forward, generating multiple responses and future utterances. With a selector, the best response and future utterance are selected. A final response is squeezed by both history and future. Compared with HDLD, there is inadequate consideration of dialogue duality in these works.
They ignore the explicit interaction between the dialogue history and future, still resulting in underutilization of future information.

heading: 2.2 Dual Learning
text: Dual learning is proposed to boost neural machine translation (He et al., 2016). In the dual learning framework, one agent represents the model for the primal task, which aims to build a mapping from domain X to domain Y , and the other agent represents the model for the dual task, which aims to build a mapping from domain Y back to domain X . Based on the training feedback signals generated from each other, two models are updated iteratively until convergence. By leveraging the primal-dual structure of two tasks, the training and inference of the primal and dual tasks are improved.
Many dialogue generation methods (Yang et al., 2018; Cui et al., 2019; Li et al., 2021) have mentioned ‘duality’ in their paper but note that there are differences in our definition of ‘duality’ from theirs. Duality describes the relationship between dual and primal tasks (e.g., English-to-Chinese translation and Chinese-to-English translation) in previous works. By contrast, in this paper, duality refers to a unique dialogue property that humans can take advantage of to infer dialogue future (or history) according to dialogue history (or future).
It is important to note the distinction between duality and "bi-directional" models such as BiRNN (Schuster and Paliwal, 1997) or MLM like BERT (Devlin et al., 2019). While these models extract features in two directions, they are not "dual." In these models, the backward features are typically merged with the forward features, rather than actively enhancing forward feature extraction as in dual learning approaches.

heading: 3 Method
text: We first present the preliminary in § 3.1. Then, we introduce a two-stage optimization that exploits dualities at two hierarchies in § 3.2, and some important details during training and inference in § 3.3.

heading: 3.1 Preliminary
text: Given a multi-turn dialogue D = {u1, . . . , un} with n > 2 utterances. An utterance ui = {w1i , w2i , . . . , w |ui| i } consists of |ui| tokens. The embeddings of ui is ei with d dimension. Take t-th utterance as the dividing point, we divide D into two parts: the past contexts
P = {x1:t|x1:t = {u1, . . . , ut} , t ∈ [1, n− 1]} and the reversed future contexts F = {yn:t+1|yn:t+1 = {un, . . . , ut+1} , t ∈ [1, n− 1]}. In HDLD, there are a forward generation model fxy: P → F and its dual generation model, a backward model fyx : F → P. In each model, there are one transformer encoder and two transformer decoders. At the utterance hierarchy, fxy takes a x1:t ∈ P as context to predict utterances in yn:t+1, and fyx takes the corresponding yn:t+1 ∈ F as context to predict utterances in x1:t, which forms the horizontal cycle in Figure 2. Two decoders in each model process in dual directions to extract intra-duality features in utterances, and cross-hierarchy distillation integrates hierarchies: guided by both future and past contexts (top-down), we distill knowledge that is both context-related and intra-dual. Distilled knowledge is used to teach models (bottom-up), which forms the vertical cycle in Figure 2. Figure 3 shows the HDLD architecture. In the following description, we remove the subscript of x or y when there is no ambiguity.

heading: 3.2 Two-Stage Optimization
text: As a dual learning framework, our ultimate goal is to obtain the forward model fxy which generates response based on dialogue history while the fyx is auxiliary. Given any aligned x ∈ P and y ∈ F, to leverage inter-duality, we apply the two models consecutively, completing the horizontal dual cycle. This process yields four outputs: x′ = fyx(y), y′ = fxy(x), x′′ = fyx(y′), and y′′ = fxy(x′). Reconstruction errors between x′′ and x, and between y′′ and y reflect how well the horizontal dual cycle is completed, i.e., the utility of inter-duality. We
minimize reconstruction errors by minimizing the objective:
− ∑
x∈P log p(x′′ = x|x; fxy, fyx)
− ∑
y∈F log p(y′′ = y|y; fyx, fxy).
(1)
However, it is challenging to directly optimize the final output x′′ and y′′, which is hindered by the error accumulation in practice. Thus, we find the upper bound of Eq. 1 which is more practical. Take the logarithmic probability − log p(x′′ = x|x; fxy, fyx) as an example, we have:
− log p(x′′ = x|x; fxy, fyx) = − log ∑
y′ p(x′′ = x, y′|x, fxy, fyx)
= − log ∑
y′ p(x′′ = x|x, y′, fxy, fyx)p(y′|x; fxy, fyx)
≤ − ∑
y′ p(y′|x, fxy) log p(x′′ = x|y′, fyx).
(2)
The inequality is derived from two facts: (1) x′′ is only directly related to y′ and fyx and y′ is only directly related to x and fxy and (2) the concavity of logarithm. Therefore, we can derive the upper bound of Eq. 1:
− ∑
x∈P log p(x′′ = x|x; fxy, fyx)
− ∑
y∈F log p(y′′ = y|y; fyx, fxy)
≤ − ∑
x∈P
∑ y′∈F p(y′|x, fxy) log p(x′′ = x|y′, fyx)
− ∑
y∈F
∑ x′∈P p(x′|y, fyx) log p(y′′ = y|x′, fxy).
(3)
We can optimize the upper bound by simultaneously minimizing each probability term within it. This means that in order to exploit inter-duality, it is not necessary to complete the horizontal dual cycle by consecutively applying two models and directly optimizing the final outputs. Instead, we can optimize the intermediate outputs, which divides the training into two stages. In the first stage, fxy generates y′ and fyx generates x′. We fit y′ and x′ to their ground truth y and x, respectively. In the second stage, fxy predicts y′′ and fyx predicts x′′, which fit to y and x again, respectively. Formally,
we minimize the negative log-likelihoods:
Linter = E [ − log p ( y′ = y|x, fxy )]
+ E[− log p ( y′′ = y|x′, fxy ) ] + E[− log p ( x′ = x|y, fyx ) ] + E[− log p ( x′′ = x|y′, fyx ) ].
(4)
Another advantage of the two-stage optimization is that it allows for easier consideration of intraduality by assigning the top-down and bottom-up parts of the cross-hierarchy distillation to each stage. We delve into the detailed implementation of Eq. 4 in each stage.
Stage One. In the first stage, we apply both models once, and utilize top-down distillation to distill integrated hierarchical duality knowledge. However, reconstructing the entire dialogue history x and future response y can be challenging due to their often lengthy nature, consisting of multiple sentences. To address this challenge, we relax the constraint and allow each model to reconstruct only one subsequent utterance in the other part of the dialogue. For example, in the case of fxy, x1:t serves as the context and ut+1 is the target, while for fyx, yn:t+1 acts as the context and ut is the target. Both models encode their respective contexts and pass the encoder output to their respective main decoders. The generated responses from the main decoders in the first stage are denoted as u
′ t+1 and
u ′ t. To optimize fxy and fyx, we fit their outputs to the ground truth ut+1 and ut by minimizing the negative log-likelihoods:
Lmain-dec = E [− log p(ut+1|x1:t)] + E [− log p(ut|yn:t+1)] . (5)
The dual-view decoder in each model operates on the same encoder outputs as the main decoder but processes them in a reverse direction, which enables us to leverage intra-duality, as the cross attention mechanism in decoders limits tokens from attending to their subsequent tokens. To achieve this, we flip the attention mask used in the main decoder along the main diagonal to create the mask for the dual-view decoder. The generated responses from the dual-view decoders in the first stage are represented as ũ
′ t+1 and ũ ′ t. Figure 3(a) provides
an illustration of the behaviors of both the main decoder and the dual-view decoder.
Next, we introduce the top-down part of the cross-hierarchy distillation. Denote the mixture of outputs from the main decoder and the dual-view
decoder as intra-dual features I:
Ix = e ′ t+1 ⊗ σ(WIe ′ t+1) + ẽ ′ t+1 ⊗ σ(W̃I ẽ ′ t+1), Iy = e ′ t ⊗ σ(WIe ′ t) + ẽ ′ t ⊗ σ(W̃I ẽ ′ t),
(6) where WI , W̃I ∈ Rd×d, the ⊗ denotes Hadamard product, and σ denotes the sigmoid function. e·· denotes hidden state embeddings of the corresponding u··. The representations of the entire dialogue, denoted as ED, play a crucial role in distilling both context-related and intra-dual knowledge from the intra-dual features I using the multi-head attention mechanism. In this mechanism, I serves as both the key and value, while ED acts as the query. The output of the multi-head attention, denoted as A, is then mapped to match the vocabulary size |V |, resulting in the final knowledge representation K. The detailed process is illustrated in Figure 3(c), where the linear layer weight is represented by WA of size R|V |×d:
Ax = softmax( EDI T x√ d )Ix,
Ay = softmax( EDI T y√ d )Iy,
Kx = softmax(WAAx),
Ky = softmax(WAAy).
(7)
Stage Two. In the second stage, we use predicted responses in stage one to construct pseudo contexts and based on which, two models are applied again to complete the horizontal dual cycle. Meanwhile, the bottom-up distillation is carried out to complete the vertical dual cycle.
As shown in Figure 3(b), we substitute e′t and e′t+1 obtained in the previous stage for et and et+1 in embedded x1:t and yn:t+1, respectively. By utilizing e′t and e ′ t+1 to construct pseudo contexts, we preserve the gradient flow, allowing the joint optimization of the two models. These modified context representations can be referred to as embedded pseudo contexts, denoted as x
′ 1:t and y ′ n:t+1.
Next, fxy and fyx predict each one’s target again based on respective pseudo contexts. We denote the generated responses from main decoders in stage two as u
′′ t+1 and u ′′ t , respectively.
Instead of using discrete ground truth sentences, we employ more informative intra-dual knowledge Kx and Ky to guide the decoding process of the two models. To enhance the suitability of the knowledge as labels, we further augment the target word probability in the soft distributions Kx
and Ky, following the method proposed by (Wang et al., 2021). The guidance of the decoding process is achieved through bottom-up distillation, where we minimize the Kullback-Leibler divergence (KL) (Kullback and Leibler, 1951) between the outputs of the main decoders in both stages and the knowledge representation:
Ldistill = 1 |P| ∑
Kx
(KL[Kx||p(u ′ t+1|x1:t)]
+ KL[Kx||p(u ′′ t+1|x ′ 1:t)])
+ 1 |F| ∑
Ky
(KL[Ky||p(u ′ t|yn:t+1)]
+ KL[Ky||p(u ′′ t |y ′ n:t+1)]).
(8)
Moreover, we optimize both the dual-view decoder outputs and knowledge to be similar to ground truth in semantics: Ldual-dec = E [ − log p(ũ′t+1 = ut+1|x1:t) ]
+ E [ − log p(ũ′t = ut|yn:t+1) ] ,
Lknow = E [ − log p(Kx = ut+1|ED, e ′ t+1, ẽ ′ t+1) ]
+ E [ − log p(Ky = ut|ED, e ′ t, ẽ ′ t) ] .
(9) For clear demonstration, we use red arrow to indicate all loss calculations in Figure. 3.

heading: 3.3 Training and Inference
text: During training, the training loss function L is defined as follows:
L = Lmain-dec+Ldual-dec+Lknow+Ldistill. (10) During inference, only the encoder and the main decoder of fxy remain active. Therefore, HDLD does not incur any additional inference costs than a vanilla transformer. It is feasible because to consider the mutual information I(P,F) between P and F:
I(P,F) = − ∑
x∈P
∑ y∈F p(x, y) log p(x, y)
+ ∑
x∈P
∑ y∈F p(x, y) log p(x|y)
+ ∑
x∈P
∑ y∈F p(x, y) log p(y|x)
(11)
In Eq. (11), p(x, y) is constant and is decided by data, and HDLD maximizes p(x|y) and p(y|x) according to Eq. (4) through the joint optimization.
Thus, Eq. 11, the mutual information between dialogue history and future is maximized. Given the past context, HDLD is able to implicitly envision future information with the maximized mutual information.

heading: 4 Experimental Setups
text: Baselines We compare HDLD with baselines in two groups. The first group introduces future utterances to enhance the model: NEXUS (Shen et al., 2018) introduces a code space learned from the whole dialogue and samples from the code space during testing to estimate future context. RegDG (Feng et al., 2020) trains a teacher model that has access to the whole dialogue to teach a student model which only takes past utterances as input. ProphetChat (Liu et al., 2022) first predicts two steps forward, generating multiple responses and future utterances, which are used to squeeze the final response along with the dialogue history.
The second group considers the relationship among tokens in an utterance comprehensively: D2GPO (Li et al., 2020) augments the training with a data-dependent Gaussian prior distribution, which is generated in pre-procession based on the training set. AdaLabel (Wang et al., 2021) introduces an auxiliary decoder that uses a bidirectional attention to dynamically estimate a token distribution at each time step. Besides, we also compare to HRED (Serban et al., 2016), a pilot work for hierarchical context modeling, and Transformer (Vaswani et al., 2017), the backbone of AdaLabel, D2GPO and HDLD.
Datasets We conduct multi-turn dialogue generation experiments on two public datasets: DailyDialog (Li et al., 2017) and OpenSubtitles (Lison and Tiedemann, 2016). DailyDialog contains high quality multi-turn dialogues collected from daily conversations while OpenSubtitles contains dialogues from movies. As a preprocessing step, we exclude dialogue sessions shorter than 4 utterances from the dataset, and we create samples where the contexts consist of no more than 100 tokens. We split a pre-processed dialogue into two parts: dialogue history and future. Baselines that utilize
future utterances have access to two parts and those do not utilize future utterances only take the historical part as contexts. In this way, we ensure all experiments have access to the same amount of training samples, which is a fair setting. Due to our unique task settings, we cannot directly cite any numbers from other papers and we reproduced every baseline. Detailed datasets statistics are in Table 1.
Hyper-parameters We use grid-search to select the best hyper-parameters. The search ranges for learning rate and batch size are {0.00006, 0.00008, 0.00010, 0.00012} and {64, 128, 256, 520, 720, 1000}, respectively. The adopted learning rate for HDLD is 0.00008 and the mini-batch size is 128 on DailyDialog and 720 on OpenSubtitles. As for other important settings: the warmup steps are 4000. We use Adam optimizer with β = (0.9, 0.98). Both attention dropout and activation dropout are 0.1. We select the best parameters based on perplexity on the validation set. All experiments adopt greedy decoding. Early stop patience is 10 epochs on DailyDialog and 5 on OpenSubtitles. Some baseline-specific hyper-parameters not mentioned follow the original papers. The training is performed on two A40 GPUs.
Evaluation Metrics We consider three automatic evaluation metrics: BLEU (Papineni et al., 2002), Distinct (Dist) (Li et al., 2016), and BLEURT (Sellam et al., 2020). BLEU score measures the word overlap between generated responses and the ground truth. Dist score measures the ratio of unique n-grams in the generated responses. Because these two metrics are only sensitive to lexical variation, we evaluate BLEURT, an advanced learned semantic-sensitive evaluation metric.
For human evaluation, we recruit five evaluators to manually judge 500 samples from all models in blind testing. All responses are re-capitalized and de-tokenized fairly. Evaluators rate samples from three aspects: readability (Read.): how readable the response is; coherency (Coh.): whether a response is coherent with the context; and informative (Info.): whether the response is informative enough. In each aspect, the evaluator can score at ‘0’ for bad, ‘1’ for borderline, or ‘2’ for good.
The salary for each evaluator is 1 dollar per 10 samples. To give a fair salary, we first evaluate 50 samples by ourselves, calculate the time and
effort, and set this amount (samples evaluated by ourselves are just for evaluating the salary, which is not given to evaluators and not reported in the final results).

heading: 5 Results
text: Comparison with Prior Works Table 2 shows the automatic evaluation results. HDLD outperforms all baselines on all metrics. HDLD doubles scores on most metrics of baselines using no future information, i.e., Transformer and HRED, and we gain improvement by a large margin than other strong baselines. Thanks to exploited interduality, HDLD connects the past and future contexts so it matches references best (highest BLEU scores). Related to both the past and future contexts, there are few general and meaningless responses so HDLD achieves the highest diversity scores. Except above lexical metrics, we also achieve the highest semantic-sensitive BLEURT scores. From automatic evaluation, our motivation of making responses meaningful and specific by utilizing hierarchical duality is proved effective.
Human evaluation results are shown in Table 4. The highest readability shows that intra-duality in an utterance is leveraged and thus responses are more natural. The highest coherency and informative scores demonstrate that HDLD makes full use of dialogue texts and has the best context modeling capability, resulting in responses connecting and related to both past and future contexts. Overall, our model generates the highest-quality responses.
Ablation Study We study three variants of HDLD by adding following approaches to a vanilla transformer: (1) +Inter.: utilizing inter-duality, i.e., let two transformer models form a horizontal dual cycle; (2) +Intra.: utilizing intra-duality, i.e., adding a dual-view decoder in a vanilla transformer to obtain knowledge for teaching the model itself; (3) +Inter. + Intra.: combining previous two approaches but do not combine two hierarchies with cross-hierarchy distillation; In variant (1), there is no way to distill knowledge so we minimize the negative log-likelihood in two stages. Table 3 shows results. On two datasets, the inter-duality approach improves performance and outperforms the first group baselines except ProphetChat in Table 2. By only using intra-dual features, or by combining two duality approaches without cross-hierarchy distillation, most metrics are negatively affected. It reveals that without inter-duality, intra-duality does
not work well in dialogues, which confirms our motivation of coupling hierarchies. Finally, HDLD significantly improves the performance on almost all metrics. The study reveals that (1) coupled hierarchical duality is effective, and (2) cross-hierarchy distillation is necessary and non-trivial.
Further, we provide some cases to demonstrate the impact of dualities. For inter-duality, we demonstrate a case to illustrate the effectiveness of HDLD in Table 5. Although only the past context is visible to HDLD, it still generates the response which connects the past and the future utterances best. On the contrary, responses from baselines are either unrelated to the dialogue topic or can not lead to future conversation. For intra-duality, we visual-
ize trained embedding layers of Transformer, Transformer with inter-duality approach, and HDLD in Figure 4. Comparing three sub-figures, we find that without considering intra-duality, token embeddings tend to be optimized to be indistinguishable. The result is: representations are similar whatever the inputs are, so the model prefers outputting certain specific tokens, which makes responses less diverse. Only exploiting inter-duality has little effect on token representations which are fine-grained and of low-hierarchy. By contrast, in HDLD, tokens are much more distinguishable. Because semantic information can be represented better, the context modeling ability is improved and generated responses are more diverse and meaningful.
Analysis of Length Intuitively, the ratio between the length of the past context and that of the future context has an impact on the performance. We analyze this ratio. In the above experiments, we randomly divide the dialogue session into two parts to obtain as many training samples as possible. In this study, we keep the number of training samples constant by reusing randomly divided training data while the ratio constraint is satisfied by masking redundant utterances. Because dialogues in OpenSubtitles are too short to support this ablation, we only report results on DailyDialog in Figure 5. We report the variation of four metrics and for ease of comparison, we also report the performance of AdaLabel, RegDG, and D2GPO as dashed horizontal lines. It is clear that random data division performs best. Although constraining the ratio to constant hurts performance, it still outperforms all baselines. According to this study, in different dialogue sessions, responses depend to a different degree on the contexts of the past and the future. Therefore, it is better to fully utilize data without such a ratio constraint.

heading: 6 Conclusion
text: This paper first augments the dialogue generation task with hierarchical duality in dialogue text which was widely ignored. We propose a hierarchical dual learning framework HDLD to utilize both interduality and intra-duality, and couple them tightly through the cross-hierarchy distillation mechanism. Both automatic experiments and human judgment demonstrate the effectiveness of hierarchical duality in generating dialogues, which helps HDLD outperform strong baselines. In the future, more under-utilized properties except for duality in dialogues are to be explored. We will further explore the hierarchical characteristics in natural language.
Limitations
Since there are two transformer-based models in HDLD, the major limitation of HDLD is the training cost. It costs twice training time than most baselines. Here is the training time comparison on OpenSubtitles:
Model Minutes Per Epoch
AdaLabel 7.13 D2GPO 7.16 RegDG 7.83
Ours 14.32
To address this limitation, a potential improvement direction is to fuse the inter-duality approach into one transformer, e.g., sharing parameters between the fxy and fyx. However, we find the significant performance drop when doing so. In the future, the trade-off between performance and cost needs more attention. Besides, like most generative models, if there is malicious information in data, there is no guarantee to avoid bad responses to users, which is a potential risk that needs to pay attention.

heading: Acknowledgement
text: This work was supported by National Natural Science Foundation of China (NSFC Grant No. 62122089), Beijing Outstanding Young Scientist Program NO. BJJWZYJH012019100020098, and Intelligent Social Governance Platform, Major Innovation & Planning Inter-disciplinary Platform for the "Double-First Class" Initiative, Renmin University of China.

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Following instructions, we add Limitations after Conclusion.

heading: 3 A2. Did you discuss any potential risks of your work?
text: In Limitations.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: The main claims in the paper are stated in the abstract and in the introduction.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? We use open source code and public dataset.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: We have cited all datasets we use. We have cited open pre-trained models.

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: We discuss in appendix A.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. In appendix B, we report the dataset statistics.
C 3 Did you run computational experiments?

heading: In Experiments.

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? In terms of parameters, we report the bone architecture of most baselines: transformer with 6 encoder layers and 6 decoder layers, which has certain amount of parameters. In appendix A, we report GPU version. In Limitations, we report computational budget.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? In 4.1.2, appendix A and appendix C, we discuss experimental setup in detail.

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? We report p-value in t-test and kappa value of human evaluation agreement.

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? In 4.1.4, we report evaluation metrics. In appendix A, we report our script is based on NLTK.
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? In 4.1.4.

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? In 4.1.4, we summarized three aspects of evaluation instructions. Before human evaluation, we have de-tokenized and re-capitalized the outputs for a fair and solid evaluation, and thus the instructions are relatively concise. In appendix D, we discuss more detials on human evaluation.

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? In appendix D, we discuss these.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: In the future, more under-utilized properties except for duality in dialogues are to be explored. We will further explore the hierarchical characteristics in natural language. Limitations
Since there are two transformer-based models in HDLD, the major limitation of HDLD is the training cost. It costs twice training time than most baselines. Here is the training time comparison on OpenSubtitles:
Model Minutes Per Epoch
AdaLabel 7. 13 D2GPO 7. 16 RegDG 7. 83
Ours 14. 32
To address this limitation, a potential improvement direction is to fuse the inter-duality approach into one transformer, e. g. , sharing parameters between the fxy and fyx. However, we find the significant performance drop when doing so. In the future, the trade-off between performance and cost needs more attention. Besides, like most generative models, if there is malicious information in data, there is no guarantee to avoid bad responses to users, which is a potential risk that needs to pay attention