Multijugate Dual Learning for Low-Resource Task-Oriented Dialogue System
Year: 2023

Abstract:
Dialogue data in real scenarios tend to be sparsely available, rendering data-starved endto-end dialogue systems trained inadequately. We discover that data utilization efficiency in low-resource scenarios can be enhanced by mining alignment information uncertain utterance and deterministic dialogue state. Therefore, we innovatively implement dual learning in task-oriented dialogues to exploit the correlation of heterogeneous data. In addition, the one-to-one duality is converted into a multijugate duality to reduce the influence of spurious correlations in dual training for generalization. Without introducing additional parameters, our method could be implemented in arbitrary networks. Extensive empirical analyses demonstrate that our proposed method improves the effectiveness of end-to-end task-oriented dialogue systems under multiple benchmarks and obtains state-of-the-art results in low-resource scenarios.

heading: 1 Introduction
text: With the emergence of dialogue data (Zhang et al., 2020b), and the evolution of pre-trained language models (Qiu et al., 2020), end-to-end task-oriented dialogue (TOD) systems (Su et al., 2022; Lee, 2021; Tian et al., 2022) gradually replaced the previous modular cascading dialogue systems (Gao et al., 2018). The end-to-end TOD system adopts a uniform training objective, preventing the error propagation problem in pipelined dialogue systems (Gao et al., 2018). Nonetheless, the end-to-end paradigm requires more training data to perform better (Su et al., 2022). Meanwhile, TOD data is enormously expensive to annotate (Budzianowski et al., 2018) as it simultaneously contains dialogue state tracking, dialogue action prediction, and response generation. It is also expensive to annotate large amounts of complicated dialogue data for
∗Corresponding Author.
each emerging domain (Mi et al., 2022). Therefore, improving data utilization efficiency in lowresource scenarios becomes critical for end-to-end TOD.
Previous approaches (Zhang et al., 2020b; Su et al., 2022) improve the transferability of models on downstream tasks and capacity to handle small samples by conducting self-supervised or semisupervised further-pretraining (He et al., 2022) of models on data from additional dialogue domains. However, these further pre-trains on million-level datasets may require hundreds of GPU hours and are resource-constrained. Then on specific downstream dialogue tasks, a unified multi-task generative paradigm (Lee, 2021; Su et al., 2022) was applied to end-to-end dialogue tasks. Although this generative approach demonstrates better generalization and outcomes, we argue that heterogeneity
11037
and duality between data are ignored. Here, heterogeneity refers to the formative discrepancy between uncertain, unstructured discourse (e.g., user utterances and system responses) and deterministic, structured dialogue states. Accordingly, the underlying alignment information and knowledge contained within the heterogeneous data is not fully exploited in the above approach.
To address the above challenges, we propose an innovative multijugate dual learning framework in TOD (MDTOD). Contrary to previous work on reconstructing user discourse based on belief states (Sun et al., 2022; Chen et al., 2020), we observed that modeling the duality between user utterance and system responses can further uncover alignment information of entities between user utterance, system responses and dialogue states. Specifically, the model is required to reconstruct the user discourse based on the dialogue state, and also to deduce the user utterance backwards based on the system response. Consequently, the model can further learn the mapping relationship between the heterogeneous information, and improve the performance of the end-to-end TOD system in lowresource scenarios.
However, proper dual training increases the likelihood of the model learning spurious data correlations. It is evidenced by the fact that comparable model performance can be attained using only highfrequency phrases as the training set (Yang et al., 2022). As a result, the model does not generalize well to test samples with significant expression variations or domain differences, as illustrated in Figure 1. To accomplish this, we expand the oneto-one dual learning paradigm to multijugate dual learning by capitalizing on the property of semantic representation variety. Given a deterministic dialog state as a constraint (Hokamp and Liu, 2017), a specific user utterance (system response) is rewritten into multiple utterances (responses) with the same semantics but various expressions utilizing decoding methods such as beam search or random sampling. Consequently, the richer representation of information permits the spurious correlation of shallow statistical patterns acquired by the model to be effectively mitigated, thereby enhancing the model’s generalization (Cui et al., 2019).
Our proposed method exploits the entity alignment information among heterogeneous data by designing a dual learning task; it also mitigates the phenomenon of false correlations and increases
the generalization capacity of models via rephraseenhanced multijugate dual learning. As a result, the method does not introduce any additional trainable model parameters. It can be directly integrated into end-to-end TOD systems in arbitrary low-resource scenarios as a training approach to increase data utilization efficiency. We show the effectiveness of our method in several task-oriented datasets, including MultiWOZ2.0 (Budzianowski et al., 2018), MultiWOZ2.1 (Eric et al., 2020), and KVRET (Eric et al., 2017). We also demonstrate the advantages of our approach in low-resource scenarios. All code and parameters will be made public.
Our primary contributions are summarized below:
• A novel, model-independent, dual learning technique intended for low-resource end-toend TOD systems is presented that can be incorporated directly into the training of any TOD system.
• To address the issue of spurious correlations impacting the generalization of models, a paradigm of paraphrase-enhanced multijugate dual learning is presented.
• We empirically evaluate the technique on several datasets, achieving competitive results without introducing extra model parameters or further pre-training and state-of-the-art results in low-resource circumstances.

heading: 2 Related Work

heading: 2.1 Task-Oriented Dialogue Systems
text: TOD aims to complete user-specific goals via multiple turns of dialogue. Prior work focused mainly on TOD subtasks based on the pipeline paradigm (Gao et al., 2018), but it was prone to error propagation between modules. Therefore, recent research has attempted to model dialogue tasks from an endto-end generation approach. DAMD (Zhang et al., 2020a) generates the different outputs of a conversation process via multiple decoders and expands multiple dialogue actions dependent on the dialogue state. A portion of the study (Hosseini-Asl et al., 2020; Yang et al., 2020; Peng et al., 2021) models the individual dialogue tasks in the TOD as cascading generation tasks using GPT2 (Radford et al., 2019) of the decoder architecture as the backbone network. Multi-task approaches (Lin et al., 2020; Su et al., 2022; Lee, 2021) utilizing
encoder-decoder architectures such as T5 (Raffel et al., 2020) or BART (Lewis et al., 2020) exist for modeling dialogue sub-tasks as sequence-tosequence generating tasks.
Although the methods mentioned above use a uniform end-to-end approach to model TOD, none performs well in low-resource scenarios. To this end, we devise a rephrase-enhanced multijugate dual learning to exploit the entity alignment information more adequately and to obtain more robust performance.

heading: 2.2 Dual Learning for Generation
text: Dual learning aims to utilize the paired structure of data to acquire effective feedback or regularization information, thus enhancing model training performance. Dual learning was initially introduced in unsupervised machine translation (He et al., 2016) and combined with reinforcement learning to optimize two agents iteratively. DSL (Xia et al., 2017) then extended dual learning to supervised settings to take advantage of pairwise relationships of parallel corpora. Similar work (Guo et al., 2020) employs cycle training to enable unsupervised mutual generation of structured graphs and text. MPDL (Li et al., 2021) expands the duality in dialogue tasks to stylized dialogue generation without the parallel corpus. A portion of the work (Sun et al.,
2022; Chen et al., 2020) integrates the idea of duality into the dialogue state tracking. Some of the work (Zhang et al., 2018; Yang et al., 2018; Cui et al., 2019) introduces dual learning in dialogue generation to enhance responses’ diversity, personality, or coherence. However, each method mentioned above requires multiple models or combines reinforcement learning and dual modeling, considerably increasing the task’s complexity and training difficulty.
In contrast to previous work, our proposed multijugate dual learning objectives share the same model parameters. It does not require modifications to the original training objectives of the maximum likelihood estimation, making training more straightforward and more readily applicable to other tasks.

heading: 3 Methodology

heading: 3.1 End-to-End Task-Oriented Dialogue System
text: Typically, end-to-end TOD systems consist of subtasks such as dialogue state prediction and response generation (Lee, 2021). End-to-end TOD systems typically model the several subtasks of the dialogue process as sequence generation tasks to facilitate the unification of model structure, and training objectives (Hosseini-Asl et al., 2020). Denote the
TOD dataset as DTOD = {Diali,DB}Ni=1, where DB is the database. In a multi-turn dialogue Diali, where the user utterance in the t-th turn is Ut, and the system response is Rt, the dialogue history or dialogue context can be expressed as follows:
Ct = [U0, R0, · · · , Ut−1, Rt−1, Ut]. (1)
After that, the model generates the dialogue state Bt based on the previous dialogue context Ct:
LB = N∑
i=1
Ti∑
t=1
− logPθ(Bt|Ct), (2)
where N represents the total number of sessions in the dataset, Ti symbolizes the total number of turns per session and θ denotes an arbitrary generation model. The system then searches the database with the criterion Bt and retrieves the database result Dt. Then, the TOD system generate the response Rt based on the context Ut, dialogue state Bt and database query result Dt for each round:
LR = N∑
i=1
Ti∑
t=1
− logPθ(Rt|Ct, Bt, Dt). (3)
Finally, a human-readable response text containing the entity is obtained by combining the belief state and the search results from the database.

heading: 3.2 Multijugate Dual Learning
text: This section describes how to design dual learning objectives in the training process of TOD. Also, we expound how to construct multijugate dual learning by paraphrasing user utterances and system responses with representational diversity based on deterministic dialogue states.

heading: 3.2.1 Dual Learning in TOD
text: We define the deterministic dialogue state St = [Bt;Dt] consisting of two informational components: the belief state Bt and the database query results Dt.
As illustrated in Figure 2, dialogue states can be viewed as information with a unique manifestation of determinism (Zhang et al., 2020a) without regard to the order of dialogue actions. Utilizing dialogue state as a constraint, the natural language of context and response could be viewed as data with different representations of uncertainty. Therefore, we designed the dual task in TOD to learn
the mapping relationship between the utterance of linguistic forms and dialogue state representation.
Let fcb : Ct 7−→ Bt denote the forward learning objective of generating belief states according to the context referred to by Eq.2, and fbc : Bt 7−→ Ct denote the reverse learning objective of reconstructing the context according to the belief states, then the dual learning task between user utterance and dialogue state is defined as maximizing the following logarithmic probability:
log ∑
i∈N
∑ t∈Ti Pθ(S i t |Cit ; fcb)(Cit |Sit ; fbc). (4)
Similarly, let fcr : Ct 7−→ Rt, frc : Rt 7−→ Ct denote the dual learning task between the dialogue context Ct and the system response Rt:
log ∑
i∈N
∑ t∈Ti Pθ(R i t|Cit ; fcr)(Cit |Rit; frc). (5)
Accordingly, the loss function of the total dual learning objective is the sum of the above two components:
LDual = E i∼N t∼Ti −(logPθ(Sit , Rit|Cit ; fcr, fcb)
+ logPθ(C i t |Sit ; fbc) + logPθ(Cit |Rit; frc)). (6)
Furthermore, the two dual learning objectives share a set of model parameters in a multi-task paradigm, thus ensuring knowledge transfer between the dual tasks.

heading: 3.2.2 Construction of Multijugate Relations
text: Dual learning enhances data usage efficiency by acquiring additional entity alignment information between heterogeneous data, but it does not lessen the effect of spurious correlations on model generalization. Leveraging the deterministic properties of dialogue states and the uncertainty of linguistic representations, we expand the original one-toone dual learning to multijugate dual learning by paraphrases. Theoretically, several semantically identical but inconsistently expressed contexts or system responses exist for a deterministic dialogue state. Consequently, given (St, Ct) or (St, Rt), we rephrase the context Ct and the response Rt restricted by the entities in dialogue state St with the following constraint generation method:
C̃t ∼ P(Ct, St), R̃t ∼ P(St, Rt). (7)
Specifically, we utilize an off-the-shelf paraphrasing model with the dialogue context Ct as the model input. Also the value in the dialogue state St will be treated as a constraint to limit the decoding. Then, beam search is employed in generation to obtain K different contexts C̃t or responses R̃t as the result of paraphrase generation.
Moreover, since the context Ct of the current turn depends on the dialogue history (· · · , Ct−1, St−1, Rt−1) of the previous turn, rewriting the context or responses of each turn results in a combinatorial explosion. Therefore, a heuristic was adopted whereby the dialogue context Ct and system response Rt would only be rewritten once every dialogue turns. The method for producing the final paraphrase is:
C̃ijt ∼ N∑
i=1
Ti∑
t=1
M∑
j=1
P(Cijt , Sijt ), (8)
R̃ijt ∼ N∑
i=1
Ti∑
t=1
M∑
j=1
P(Sijt , Rijt ), (9)
where M represents the number of single samples to be rewritten. In practice, as the proportion of training data increases, the number of M decreases. In addition, paraphrasing was preferred over word substitution or addition/deletion-based techniques (Wei and Zou, 2019) because word substitution is based on a particular probability of word-level alterations, preventing the modification of phrases with false correlation. Moreover, section 4.4.3 approved paraphrasing produces more diverse and high-quality augmented content, alleviating the risk of spurious relevance more effectively.

heading: 3.2.3 Multijugate Dual Learning for Training
text: By acquiring paraphrase-enhanced samples, the original one-to-one dual learning can be augmented with multijugate dual learning, allowing the model to completely leverage the entity alignment information between heterogeneous data while maintaining appropriate generalization. The overall framework of our method is illustrated in Figure 2. Consequently, the final loss function for multijugate dual learning of TOD is as follows:
L̃Dual = E i∼N t∼Ti j∼M −(logPθ(Sijt , Rijt |Cijt ; fcr, fcb)
+ logPθ(C ij t |Sijt ; fbc)(Cijt |Rijt ; frc)).
(10)

heading: 4 Experiments
text: In the context of an end-to-end dialogue scenario, we examine the comprehensive performance of multijugate dual learning on several dialogue datasets, including performance on dialogue state tracking and end-to-end task completion. In addition, evaluation studies were conducted in a scenario with limited resources to assess how effectively dual learning utilizes the knowledge contained within the data. In addition, the impact of several dual learning components and rewriting procedures on the method’s overall performance is investigated.

heading: 4.1 Datasets and Evaluation Metrics
text: MultiWOZ2.0 (Budzianowski et al., 2018), MultiWOZ2.1 (Eric et al., 2020), and KVRET (Eric et al., 2017), three of the most extensively investigated datasets in the task-oriented dialogue domain, were analyzed. MultiWOZ2.0 is the first proposed dialogues dataset across seven domains, and MultiWOZ2.1 is the version with several MultiWOZ2.0 annotation problems fixed. Following earlier research, we simultaneously evaluate both datasets to assess the robustness of the model against mislabeling. KVRET is a multi-turn TOD dataset containing three domains: calendar scheduling, weather query, and navigation. Detailed statistics of the three datasets are illustrated in Table 7.
For the selection of metrics under the end-to-end dialogue task, we use the standard and widely used Inform, Success, BLEU, and Combined score, where Inform measures whether the system’s responses refer to the entity requested by the user, Success measures whether the system has answered all of the user’s requests, BLEU measures the quality of the model generation. The Combined score indicates the overall performance of the taskoriented system. It is calculated using the formula: Combined Score = (Inform + Success) * 0.5 + BLEU. For the dialogue state tracking task, the Joint Goal Accuracy (JGA) is applied to quantify the fraction of total turns where the model predicts that all slots in one turn are correct.

heading: 4.2 Baselines
text: We did comparison experiments with the following potent baselines. (1) DAMD (Zhang et al., 2020a): addresses the one-to-many issue in dialogue by extending dialogue states to many system actions. (2) SimpleTOD (Hosseini-Asl et al., 2020): A language model serves as the foundation for end-to-end TOD tasks by generating sequential dialogue states, dialogue actions, and dialogue responses. (3) DoTS (Jeon and Lee, 2021): tackles the problem of higher memory consumption owing to lengthy conversation histories by reducing the context and adding domain states as contexts. (4) SOLOIST (Peng et al., 2021): further pre-training on heterogeneous dialogue data and transfer learning for dialogue tasks downstream. (5) MinTL: employs a copy method to carry over past dialogue states and introduces Levenshtein belief spans to generate a minimal amount of dialogue states efficiently. (6) UBAR (Yang et al., 2020): considers belief states, system actions, and system responses as dialogue contexts, hence optimizing the utilization of the dataset’s content. (7) PPTOD (Su et al., 2022): A T5-based backbone network with additional pre-training on numerous dialogue datasets and simultaneous multitasking of several dialogue tasks with prompt learning. (8) MTTOD (Lee, 2021): Using T5 as the backbone model, two decoders were employed to create dialogue states and system responses, and an additional span prediction task was introduced on the encoder side. (9) BORT (Sun et al., 2022): utilizing denoised reconstruction to recover noisy dialogue states and system responses.

heading: 4.3 Overall Results

heading: 4.3.1 Performance in Low-resource Setting
text: MultiWOZ To investigate the generalizability of multijugate dual learning with limited resources, we assessed the model on the MultiWOZ2.0 dataset for dialogue sizes of 5%, 10%, and 20%. As shown in Table 1, MDTOD received the highest combined score compared to baselines for all data sizes. MDTOD obtains a 1.67-point improvement in the combined score at 5% of the training data compared to the previous best result. Our strategy produces the highest results for Inform and Success, which are task completion metrics, when applied to 10% and 20% of the data, respectively. In addition, our method obtains highly competitive results compared to PPTOD with additional dialogue data for pre-training and MTTOD with 50% more parameters. Thus, the results above imply that paraphrasing augmented multijugate dual learning that leverages implicit information embedded within the data is more effective in settings with limited resources. KVRET We also evaluate the impact of multijugate dual learning on the performance improvement of TOD on the KVRET dataset. We use T5-base as the backbone network, where T5+DL indicates the addition of dual learning on T5 and MDTOD indicates the combination of multijugate dual learning on T5. From the experimental results in Table 2, it can be concluded that after applying the dual learning objective under the low resource setting, the model achieves a significant improvement in Success when given different proportions of training samples, indicating that the dual learning can
further learn the alignment information between entities and thus improve the success rate of the task. Meanwhile, T5+DL achieves higher values on BLEU with different proportions of training data, indicating that the dual learning objective between user utterance and system response is also beneficial for improving the quality of text generation. In addition, MDTOD with multijugate dual learning achieves better results, indicating that controlled rephrasing can further enhance the effect of dual learning.

heading: 4.3.2 Dual Learning in Dialogue State Tracking
text: To further investigate the effectiveness of the dual learning task between user utterance and dialogue state on the gain of TOD in multijugate dual learning, we conducted experiments on the MultiWOZ2.0 dataset for dialogue state tracking in low-resource scenarios. We set four different quantitative training sizes of 1%, 5%, 10% and 20% to represent different degrees of low-resource scenarios.
We can infer from the experimental results in Table 3 that MDTOD had the greatest accuracy at three different magnitudes, 5%, 10%, and 20%. MDTOD is lower than PPTOD at 1% magnitude
due to that PPTOD performs further pre-training on a large amount of additional dialogue data and thus can achieve relatively better results in extremely low-resource scenarios. Conversely, MDTOD does not perform any additional pre-training, but still achieves the highest accuracy in the case of the other three magnitudes of data, indicating that multijugate dual learning between user utterances and dialogue states is an important component that makes the overall approach effective.

heading: 4.4 Analysis

heading: 4.4.1 Dismantling multijugate dual learning
text: To investigate the effect of different dual learning components and paraphrase augmentation on the proposed technique, we conducted ablation experiments by omitting various components using a 10% data size setting. In Table 4, Para represents the approach of paraphrase augmentation, DU-DL represents dual learning between context and dialogue state, and RU-DL indicates dual learning between context and system response.
As shown in Table 4, the model’s performance decreases slightly when only dual learning is retained and the paraphrase enhancement is removed, indicating that multijugate dual learning can partially mitigate the overfitting problem caused by pairwise learning and thereby improve the model’s generalization capability. Among the various dual
learning components, removing dual learning between context and system responses resulted in a 1.87-point performance decrease, indicating that fully exploiting the implicit alignment information between context and system responses was more effective at enhancing the model’s overall performance. Additionally, deleting both dual learning components resulted in a 2.02 points decrease in the combined score, demonstrating that both dual learning objectives are effective for this strategy.

heading: 4.4.2 Mitigating Spurious Correlation for Generalization
text: This section explores the generalizability of dual learning across domains when different numbers of paraphrases are tested, i.e., on a domain that does not appear in the training process, to examine the effect of rephrasing enhanced multijugate dual learning for mitigating spurious correlations of entities and improving generalization. In the In-Car dataset, we explore the ability of MDTOD to generalize to both the scheduling and weather domains separately.
The Goal Score is calculated as (inform + success) * 0.5 to signify task accomplishment. As indicated in Table 5, the model exhibits some improvement in task completion rate and text generation performance in both new domains when using rephrased augmented multijugate dual learning. Further, when the number of paraphrases is 2, a boost of 4.21 points is obtained on the Goal Score compared to no additional rephrasing mechanism. This improvement indicates that the multiple conjugations further alleviate the shallow spurious correlations among entities captured by the model, thus improving the task completion rate.

heading: 4.4.3 Effect of Different Paraphrases
text: To investigate the impact of various rephrasing techniques on the construction of multijugate dual learning, we examined the impact of easy data aug-
mentation (EDA) (Wei and Zou, 2019), synonym replacement (SYN), and paraphrasing (PARA) to generate augmented data with limited resources. As demonstrated in the upper part of Figure 3, both PARA and EDA demonstrate minor improvements as the number of augmented data increases, with PARA exceeding EDA. The results indicate that PARA generates higher-quality augmented data, whereas SYN increases noise.
The results in Figure 3 indicate that increasing the number of PARA leads to an increase in the completion rate of dialogue goals. In contrast, EDA and SYN provide a minor boost or decrease in the model’s performance. This analysis reveals that a rephrasing strategy enables better discourse rewriting under dialogue state constraints, alleviating the spurious correlation issue and enhancing the model’s generalizability.

heading: 5 Conclusion
text: We propose a novel multijugate dual learning for task-oriented dialogues in low-resource scenarios. Exploiting the duality between deterministic dialogue states and uncertain utterances enables the entity alignment information in heterogeneous data to be fully exploited. Meanwhile, paraphraseenhanced multijugate dual learning alleviates the spurious correlation of shallow pattern statistics. Experiments on several TOD datasets show that the proposed method achieves state-of-the-art results in both end-to-end response generation and dialogue state tracking in low-resource scenarios.
Limitations
Multijugate dual learning improves the model’s performance in TOD tasks in low-resource scenarios, but the introduction of the dual training objects increases the required graphics memory and training steps. In addition, the rephrasing mechanism necessitates an additional paraphraser to rewrite the training samples; hence, the number of training samples increases according to the number of paraphrases. Despite this, we find that the higher training cost associated with multijugate dual learning is preferable to employing a large quantity of dialogue data for further pre-training or manually labeling data.
Considered from a different angle, the scenario described above presents possibilities for future research, such as the development of higher-quality rephrasing algorithms to filter the augmented text. In the meantime, multijugate dual learning is a learning objective between structured and unstructured texts. Therefore it may be extended to any task involving heterogeneous data, such as generative information extraction, and data-to-set generation.

heading: Acknowledgements
text: This work was supported by the National Key Research and Development Program of China (No.2020AAA0108700) and National Natural Science Foundation of China (No.62022027).

heading: B Experiments with Full Training Data
text: B.1 End-to-End Evaluation
Table 9 demonstrates that, given the entire dataset, our proposed technique beats the comparable baselines on both datasets. The combined score on MlutiWOZ2.1 has increased by 1.48 points compared to the previous highest result. Notably, our approach does not use more dialogue data for further pre-training, nor does it introduce additional param-
1https://huggingface.co/tuner007/pegasus_ paraphrase.
Generation-based Methods
eters or use a more powerful pre-training model for dialogue. Despite this, Dual-Dialog earns the highest results, proving that dual learning can more thoroughly exploit the information included in the original data and enhance the performance of taskoriented dialogue systems despite the vast amount of data. Our proposed strategy likewise achieves the greatest BLEU on MultiWOZ2.0, showing that the quality of the model’s generated responses has been substantially enhanced.
B.2 Dialogue State Tracking
To further investigate the influence of bipartite modeling between uncertain user utterances and deterministic belief states in dual learning on TOD systems, we compared MDTOD with different generating paradigm baselines while performing the belief state tracking task. According to Table 8, MDTOD obtained up-to-date results for both datasets in the belief state tracking challenge. On MultiWOZ 2.0 and 2.1, our suggested technique achieves a 0.41 JGA improvement above the previous highest BORT and MTTOD. Dual learning between dialogue states and user utterances can learn entity alignment information in the data, resulting in improved performance in belief state tracking.

heading: C Case Analysis
text: We present partial selections of paraphrases in Table 10 to demonstrate the effect of the rephraser.
As shown in the first example, when the constraints are set to the entities "hail" and "los angeles", the rephraser still produces paraphrases that are fluent and satisfy the constraints.
In addition, we illustrate a sample of the dialog generated by MDTOD in Table 11 . The dialogue begins with the user seeking an Indian restaurant in the center of town, and the model correctly extracts the values of the slots "food" and "area". When the conversation proceeds to turn 2, MDTOD generates more belief states than oracle’s belief states, but the model generates the correct results. The reason is that there are some labeling errors in MultiWOZ2.0, while MDTOD can still generate correct belief states, which shows the robustness of MDTOD. When the conversation progressed to turn 5, MDTOD still predicted the correct belief state despite the user changing the reservation time from 13:30 to 12:30, indicating that the model understood the semantic meaning of the current input sentences rather than simply repeating the belief state from the previous turn.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 6

heading: 3 A2. Did you discuss any potential risks of your work?
text: Section 6

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 3,4

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 3,4

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Section 3,4

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 4.1

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Section 4.1

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 4.1
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4.1
C 3 Did you run computational experiments? Section 4.3, 4.4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section A.1
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section A.1

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4.3, 4.4

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4.1
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Considered from a different angle, the scenario described above presents possibilities for future research, such as the development of higher-quality rephrasing algorithms to filter the augmented text. In the meantime, multijugate dual learning is a learning objective between structured and unstructured texts. Therefore it may be extended to any task involving heterogeneous data, such as generative information extraction, and data-to-set generation