Mars: Modeling Context & State Representations with Contrastive Learning for End-to-End Task-Oriented Dialog
Year: 2023

Abstract:
Traditional end-to-end task-oriented dialog systems first convert dialog context into belief state and action state before generating the system response. The system response performance is significantly affected by the quality of the belief state and action state. We first explore what dialog context representation is beneficial to improving the quality of the belief state and action state, which further enhances the generated response quality. To tackle our exploration, we propose Mars, an end-to-end task-oriented dialog system with two contrastive learning strategies to model the relationship between dialog context and belief/action state representations. Empirical results show dialog context representations, which are more different from semantic state representations, are more conducive to multi-turn task-oriented dialog. Moreover, our proposed Mars achieves state-of-theart performance on the MultiWOZ 2.0, CamRest676, and CrossWOZ1.

heading: 1 Introduction
text: Task-oriented dialog system (Zhang et al., 2020c) aims to assist users in completing some specific tasks such as table reservations, hotel reservations, ticket booking, and online shopping. Traditional task-oriented dialog system has been built through dialog state tracking (Lee et al., 2019; Wu et al., 2019), dialog policy (Schulman et al., 2017; Takanobu et al., 2019) and natural language generation (Wen et al., 2015) tasks. dialog state tracking transfers dialog context to belief state, which is the structured semantic state capturing the whole dialog context information. The belief state is used for the dialog system to query the database to obtain matched entities. Dialog policy selects an action state, a semantic state guiding
∗Corresponding author: baojunwei001@gmail.com 1The code is available at https://github.com/
hpsun1109/Mars.
the dialog system to generate a system response based on the current dialog context and database information. System response is generated through a natural language generation task.
With the widespread application of large-scale pre-training models (Devlin et al., 2019; Radford et al., 2019; Raffel et al., 2020), researchers gradually focus on the end-to-end task-oriented dialog system (Lin et al., 2020; Hosseini-Asl et al., 2020; Yang et al., 2021), which converts the whole dialog context into system response through multi-task training. Generally, an end-to-end task-oriented dialog modeling task is formulated as a cascaded generation problem (Su et al., 2021). Before generating a system response, the end-to-end task-oriented dialog system must first transfer dialog context into belief and action states, respectively. The quality of the belief state and action state greatly influence on the end-to-end task-oriented dialog performance2.
In this paper, we explore what dialog context representation is beneficial to improving the quality of the belief/action state, which further enhances the generated response quality. As illustrated in Figure 1, dialog context is recursively hybrid of previous dialog context and semantic states3,
2The detailed analysis is provided in Appendix C. 3These previous semantic states are helpful references for
11139
i.e., belief and action states, for multi-turn dialog. Intuitively, the representation of dialog context {Ct−1, Ut−1}, which is more similar with that of semantic states Bt−1/At−1, is beneficial to generate semantic states of the turn t−1. However, if their representations are too similar, there may be information redundancy in the representation of dialog context {Ct, Ut} in turn t, as shown in Figure 1. Thus we raise another conjecture: whether representations of dialog context, which are more different from that of semantic states, are more conducive to multi-turn task-oriented dialog?
To tackle our conjectures, we propose Mars, an end-to-end task-oriented dialog system with two contrastive learning strategies, i.e., pair-aware context&state and group-aware context&state contrastive learning, to model the relationship between dialog context and semantic states from two different levels. Specifically, (1) the pairaware context&state contrastive learning strategy focuses more on narrowing the gap in the continuous representation space between dialog context and corresponding semantic states for the same dialog turn. This strategy aims to obtain a continuous representation of the dialog context that is semantically more consistent with that of its semantic states. (2) Group-aware context&state contrastive learning strategy enlarges the overall continuous representation margin between dialog context and semantic states. The meaning behind this is to make representations between dialog context and semantic states more different. Extensive experiments and analysis on the response generation and dialog state tracking tasks verify our raised conjectures and the effectiveness of Mars. Mars achieves state-of-the-art performance on the MultiWOZ 2.0, CamRest676, and CrossWOZ. Moreover, Mars achieves remarkable performance in the low-resource scenario. Finally, we perform detailed error analysis and visualization to better apply our proposed Mars to real-world scenarios.
This paper primarily makes the following contributions: (1) We explore what dialog context representation is beneficial to improving taskoriented dialog performance. (2) We propose two contrastive learning strategies to model the relationship between dialog context and semantic state representations. (3) Empirical results show Mars achieves state-of-the-art performance on the MultiWOZ 2.0, CamRest676, and CrossWOZ.
the generation of the current turn (Yang et al., 2021).

heading: 2 Related Work
text: End-to-end task-oriented dialog systems (Lei et al., 2018; Zhang et al., 2020a,b) are established via copy-augmented seq2seq learning (Gu et al., 2016). Zhang et al. (2020b) proposes a multi-action data augmentation method to improve the diversity of generated system responses. Large-scale pretrained language models, including BERT (Devlin et al., 2019), GPT-2 (Radford et al., 2019), T5 (Raffel et al., 2020), and UniLM (Dong et al., 2019), have been demonstrated effective for improving the performance of task-oriented dialog systems (Hosseini-Asl et al., 2020; Peng et al., 2021; Lin et al., 2020; Yang et al., 2021; Jeon and Lee, 2021; He et al., 2022) on MultiWOZ 2.0 (Budzianowski et al., 2018), a large-scale English multi-domain task-oriented dialog dataset. Recently, auxiliary tasks and auxiliary dialog corpora have been introduced to further improve dialog modeling ability. MTTOD (Lee, 2021) introduces a span prediction task to enhance the natural language understanding performance. BORT (Sun et al., 2022) proposes reconstruction strategies to alleviate the error propagation problem. PPTOD (Su et al., 2021) proposes a dialog multi-task pre-training strategy to model task completion from auxiliary heterogeneous dialog corpora. GALAXY (He et al., 2022) introduces a dialog act prediction task to explicitly learn dialog policy from auxiliary dialog corpora.
Recently, contrastive Learning (He et al., 2020; Chen et al., 2020; Grill et al., 2020; Chen and He, 2021) has attracted much attention in the computer vision community and has been applied to natural language processing to enhance sentence representation learning (Fang and Xie, 2020; Wu et al., 2020; Yan et al., 2021; Gao et al., 2021; Giorgi et al., 2021). In contrast, we propose contrastive learning strategies to model the relationship between dialog context and semantic state representations for task-oriented dialog. In addition, we don’t introduce data augmentation methods, which are used in most contrastive learning works.

heading: 3 Task-Oriented Dialog Framework
text: Generally, an end-to-end task-oriented dialog modeling task is formulated as a cascaded generation problem (Su et al., 2021). Before generating a system response, the end-to-end task-oriented dialog system would transfer dialog
context into belief state and action state, respectively. Belief state is a semantic state of dialog context, including dialog domain, slot name, and slot value. Action state is a semantic state of system response, including dialog domain, dialog act, and slot name. For example, the belief state is ‘[attraction] type theatre’, and the action state is ‘[attraction] [inform] name area’.
We construct an end-to-end task-oriented dialog system via the seq2seq framework, including one shared encoder and two different decoders, as illustrated in Figure 2. One shared encoder encodes dialog context, one decoder decoderb(·) decodes belief state, and another decoder decodera(·) decodes action state and system response. Consider a dialog in turn t, dialog history Ct, which contains dialog information for all previous turns, is formulated as {Ct−1, Ut−1, Bt−1, DBt−1, At−1, Rt−1}, where U represents the user utterance, B represents the belief state, DB represents the database state, A represents the action state, and R represents the system response.
For end-to-end dialog modeling, a belief state is first generated. The dialog history Ct and the current user utterance Ut are firstly encoded into hidden representation Hcb through the shared encoder, and the belief state Bt is generated through the belief state decoder:
Hcb = encoder(Ct, Ut),
Bt = decoderb(Hcb). (1)
The dialog state tracking process is optimized by
minimizing the following objective function:
LB = −logP (Bt|Ct, Ut). (2)
We use the generated belief state Bt to query the specific database to achieve the database state DBt, which means the amount of matched entities.
As described by MTTOD (Lee, 2021), the second decoder would be used to generate action state and system response simultaneously. The combination of the dialog history Ct, the current user utterance Ut, and the database state DBt are encoded into hidden representation Hca through the shared encoder. The action state At and system response Rt are generated in turn through the action state decoder:
Hca = encoder(Ct, Ut, DBt),
At, Rt = decodera(Hca). (3)
Therefore, the action state and response generation process is optimized by minimizing the following objective function:
LAR = −logP (At, Rt|Ct, Ut, DBt). (4)
In summary, the entire end-to-end task-oriented dialog system can be optimized by minimizing:
Lall = LB + LAR. (5)

heading: 4 Methodology
text: To tackle our conjectures and enhance the relationship modeling between dialog context and corresponding semantic state representations of the task-oriented dialog system described in Section 3, we propose two contrastive learning methods: pair-aware context&state and groupaware context&state contrastive learning. Figure 3 illustrates the architecture of a task-oriented dialog system with our proposed methods. Generally, for any contrastive learning method, contrastive learning objective functions Lbscl and Lascl are added for dialog state tracking and response generation tasks, respectively, to enhancing the relationship modeling between dialog context and semantic state representations during end-to-end dialog training. The general objective function can be reformulated as follows:
Lall = LB′ + LAR′ , LB′ = LB + λ1Lbscl,
LAR′ = LAR + λ2Lascl, (6)
where λ1 and λ2 are hyper-parameters that adjust the weight of the objective functions.
Shared Encoder
Belief State Decoder
Shared Encoder
Action State Decoder
𝑩𝒕
𝑫𝑩𝒕
𝑯𝒄𝒃 𝑯𝒄𝒂
Query
[attraction] type theatre a [attraction] type college a [restaurant] name la margherita
Shared Encoder
𝑯𝒃𝒃
𝑩𝒕
𝓛𝒃𝒔𝒄𝒍
Shared Encoder
𝑨𝒕
𝑯𝒂𝒂
𝓛𝒂𝒔𝒄𝒍
Can you find a theatre to go to in town? I am looking for a college type attraction. I am looking for a restaurant called la margherita.
[attraction] [request] area a [attraction] [select] area [inform] type choice [restaurant] [inform] food price area [general] [reqmore] Can you find a theatre to go to in town? [db_3] I am looking for a college type attraction. [db_3] I am looking for a restaurant called la margherita. [db_1] 2 3 1 2 3 1 3 2 1 3 2 1
negative sample
positive sample dialog context
semantic state (belief /action state)
𝑪𝒕 , 𝑼𝒕 ,𝑫𝑩𝒕𝑪𝒕 , 𝑼𝒕
𝑨𝒕 , 𝑹𝒕
Group-aware Context&State
2
3
1 1
2
3
Pair-aware Context&State 2
3
1 1
2
3
Figure 3: Illustration of the task-oriented dialog system with contrastive learning strategies. We take dialog turn t = 0 and batch size N = 3 as an example.

heading: 4.1 Pair-aware Context&State Contrastive Learning
text: To achieve dialog context representation, which is semantically more consistent with its semantic state representation, we propose a pair-aware context&state contrastive learning strategy (Mars-P) to close the continuous representation gap between dialog context {Ct, Ut} and corresponding semantic states, including belief state Bt and action state At, for the same dialog turn.
We consider the dialog context {Ct, Ut} and the belief state Bt from the same dialog to be as consistent as possible in the representation space, while the dialog context is as far away from other belief states as possible. As illustrated in Figure 3, the source continuous representation of the dialog context ‘can you find a theater to go to in town?’ should be similar to that of the belief state ‘[attraction] type theatre’ rather than other belief states ‘[attraction] type college’ and ‘[restaurant] name la margherita’.
Specifically, the belief state Bt would be encoded into a hidden representation Hbb through the shared encoder:
Hbb = encoder(Bt). (7)
For every dialog context input in a batch, we treat the corresponding belief state from the same dialog as a positive sample and other belief states and dialog contexts in the same batch as negative samples. Therefore, this dialog model is optimized by minimizing the objective function:
Lbscl ≜ Lbscl_P
= −log e cos(Hicb,H i bb)/T
N∑ k=1 k ̸=i ecos(H i cb ,Hk cb )/T + N∑ k=1 ecos(H i cb ,Hk bb )/T
,
(8)
where cos(·) denotes the cosine similarity function. T is a temperature hyperparameter. N is the batch size. In a batch, H icb denotes the ith dialog context hidden representation after average pooling, and Hkbb denotes the kth belief state hidden representation after average pooling.
During response generation, we would close the continuous representation gap of dialog context {Ct, Ut, DBt} and action state At. As illustrated in Figure 3, the source continuous representation of the user utterance ‘i am looking for a restaurant called la margherita.’ and database information ‘[db1]’ should be similar to that of the action state ‘[restaurant] [inform] food price area [general] [reqmore]’ rather than other action states ‘[attraction] [request] area’ and ‘[attraction] [select] area [inform] type choice’. Specifically, the action state At would be encoded into a hidden representation Haa through the shared encoder:
Haa = encoder(At). (9)
For every dialog context input in a batch, we treat the corresponding action state from the same dialog as a positive sample and other action states and dialog contexts in the same batch as negative samples. Therefore, this dialog model is optimized by minimizing the objective function:
Lascl ≜ Lascl_P
= −log e cos(Hica,H i aa)/T
N∑ k=1 k ̸=i ecos(H i ca,H k ca)/T + N∑ k=1 ecos(H i ca,H k aa)/T
,
(10)
where H ica denotes the ith dialog context hidden representation after average pooling, and Hkaa denotes the kth action state hidden representation after average pooling.

heading: 4.2 Group-aware Context&State Contrastive Learning
text: To explore whether representations of dialog context, which are more different from that of semantic states, are more conducive to multiturn task-oriented dialog, we propose a groupaware context&state contrastive learning strategy (Mars-G). Takes turn t as an example, MarsG enlarges the overall continuous representation margin between dialog context and semantic states, regardless of the pairing relationship between specific dialog context, e.g., {Ci, Ui}, and semantic states, e.g.Bi/Ai (turn i = 0, ..., t). The meaning behind is to make representations between dialog context and semantic states more different, which makes it easy to distinguish dialog context {Ci, Ui} and the corresponding semantic states Bi/Ai (turn i = 0, ..., t) inside the entire dialog context {Ct+1, Ut+1} and achieve much richer dialog context representations.
Specifically, for every dialog context input, we treat all semantic states in the same batch as negative samples and any one dialog context in the same batch as a positive sample. Besides, considering that every dialog input contains a unique context, narrowing the in-batch context distance makes it hard to distinguish different contexts, which may be counterproductive to deriving the context representation. To resolve such an issue, we also select the rest in-batch dialog context inputs except the positive one as negative samples for every dialog context input. Therefore, the contrastive learning objective function can be reformulated as:
Lbscl ≜ Lbscl_G
= −log e cos(Hicb,H j cb )/T
N∑ k=1 k ̸=i ecos(H i cb ,Hk cb )/T + N∑ k=1 ecos(H i cb ,Hk bb )/T
,
(11)
Lascl ≜ Lascl_G
= −log e cos(Hica,H j ca)/T
N∑ k=1 k ̸=i ecos(H i ca,H k ca)/T + N∑ k=1 ecos(H i ca,H k aa)/T
,
(12)
where Hjcb and H j ca denote the jth (j ̸= i) dialog context hidden representations after average pooling.

heading: 5 Experiments

heading: 5.1 Datasets and Evaluation Metrics
text: We conduct experiments on three task-oriented dialog datasets: MultiWOZ 2.0 (Budzianowski et al., 2018), CamRest676 (Wen et al., 2017), and CrossWOZ (Zhu et al., 2020). MultiWOZ 2.0 (Budzianowski et al., 2018) and CamRest676 (Wen et al., 2017) are English taskoriented dialog datasets. CrossWOZ (Zhu et al., 2020) is a Chinese multi-domain task-oriented dialog dataset. A detailed description of the datasets is provided in Appendix A.
We test our proposed Mars on two benchmark task-oriented dialog tasks: end-to-end dialog modeling response generation and dialog state tracking. We evaluate the performance of response generation on MultiWOZ 2.0 and CamRest676. Inconsistencies exist between previous taskoriented dialog works in data preprocessing and evaluation metrics on MultiWOZ 2.0 (Nekvinda and Dušek, 2021). To fairly compare our experiments with previous work, we use the preprocessing strategy (Zhang et al., 2020b) and the standalone standardized evaluation script released by Nekvinda and Dušek (2021). We follow the automatic evaluation metrics to evaluate the response quality for task-oriented dialog system on MultiWOZ 2.0. Inform rate measures whether a dialog system has provided an accurate entity; Success rate measures whether a dialog system has provided an accurate entity and answered all requested information; BLEU score (Papineni et al., 2002), which is computed with references, which have been obtained from the delexicalized MultiWOZ 2.2 span annotations, measures the fluency of the generated response; Combined score, which is calculated by (Inform + Success) × 0.5 + BLEU , measures the overall quality of the dialog system. Moreover, we use the Act F1 to measure the accuracy of generated action states. To make our experiments comparable with previous work (Zhang et al., 2020a; He et al., 2022)
on CamRest676, we use the same pre-processing strategy and use Inform rate, Success F1, BLEU score, and Combined score, which is computed by (Inform + SuccessF1) × 0.5 + BLEU , to evaluate the response quality for the task-oriented dialog system. The success rate whether if the system answered all requested information to assess recall, while Success F1 balances recall and precision.
We evaluate the performance of dialog state tracking on MultiWOZ 2.0 and CrossWOZ. We use the joint goal accuracy to measure the accuracy of generated belief states.

heading: 5.2 Settings
text: We use a pre-trained T5 language model (Raffel et al., 2020) to initialize the dialog system based on the HuggingFace Transformers library (Wolf et al., 2020) and follow the settings of Lee (2021). We select T5-small (Raffel et al., 2020) for MultiWOZ 2.0 and CamRest676 and T5-base-Chinese (Raffel et al., 2020; Zhao et al., 2019) for CrossWOZ. The batch size is 8. The AdamW optimizer (Loshchilov and Hutter, 2019) optimizes the model parameters with linear learning rate decay. The initial learning rate is 0.0005, and the ratio of warm up is 0.2. The hyper-parameters λ1 and λ2 are set to 1 and 0.1, respectively. T is set to 0.1 for Mars-P, and T is set to 0.5 for Mars-G. The hyper-parameter analysis is provided in Appendix E. We train all dialog systems on one NVIDIA A100 GPU for 10 epochs and select the checkpoint model with the best performance on the validation dataset. One model is trained for approximately five hours. In addition, the model is trained for 20 epochs for the low resource scenarios. The description of baseline systems is provided in Appendix B.
Another baseline is the general architecture of a task-oriented dialog system, as illustrated in Figure 2.

heading: 5.3 Main Results
text: The detailed inform rates, success rates, BLEU scores, combined scores, act F1 scores, and joint goal accuracies for end-to-end task-oriented dialog models on the MultiWOZ 2.0 benchmark are presented in Table 1. Our re-implemented baseline system performs comparable with PPTOD (Su et al., 2021), and our proposed Mars-P and Mars-G outperform our re-implemented baseline system by 4.5 and 7.2 combined scores. Moreover, Mars-G, which doesn’t use auxiliary corpora, substantially outperforms the previous state-of-theart GALAXY (He et al., 2022) and MTTOD (Lee, 2021) by 3.2 combined scores, achieving the state-
of-the-art performance in terms of inform rate, success rate, BLEU score, and combined score. In addition, Mar-G achieves the highest joint goal accuracy among the end-to-end task-oriented dialog systems, outperforming BORT (Sun et al., 2022) by 1.1 points. Compared with the baseline system, Mars-P and Mars-G achieve a better act F1 score. This demonstrates our proposed contrastive learning could effectively improve the quality of the belief state and action state, which further improves the generated response quality. Regarding the two proposed methods, Mars-G performs better than MarsP. Figure 4 displays the visualization of dialog context and semantic state representations using t-sne. Compared with the baseline system, MarsP could achieve dialog context representation that is semantically more consistent with its semantic state representation while Mars-G could make representations between dialog context and semantic states more different. These verify dialog context representations, which are more different from semantic state representations, are more beneficial to achieving task completion of task-oriented dialog. Further dialog context representation analysis is provided in Appendix D.
Table 2 presents the performance of taskoriented dialog systems on the CamRest676. Mars-G outperforms the previous state-of-the-art GALAXY (He et al., 2022) by 1.7 combined scores, achieving the state-of-the-art performance in terms of success F1, BLEU score, and combined score.
Table 3 reports the dialog state tracking performance on CrossWOZ. Mars-P and MarsG substantially outperform the previous state-of-
the-art GEEX (Li et al., 2021) by 4.6 and 5.1 points, achieving 59.3 and 59.8 joint goal accuracy. This further indicates that our proposed contrastive learning strategies could improve belief state learning ability, and Mars has good generalization ability. In addition, we provide an example to visualize our proposed Mars-G’s dialog state tracking process in Appendix F.

heading: 5.4 Ablation Study
text: Table 4 shows the performance of the different components of Mars-P and Mars-G. Both state modules of Mars-P and Mars-G could improve the performance of the dialog system. Regarding two modules of contrastive learning strategies MarsP and Mars-G, the action state module performs better than the belief state module by 1.7 and 1.6 combined scores, respectively, because the quality of the action state has a more direct impact on the response generation quality and action state module could improve action state learning ability. Moreover, the combination of both modules can complement each other to further improve endto-end dialog modeling performance. The further ablation analysis is provided in Appendix G.

heading: 5.5 Dialog Turn Analysis
text: To better assess the effectiveness of our proposed contrastive learning strategies, we investigate the performance (inform rate and success rate) of MarsG and the baseline system on the test set with respect to different dialog turns. Specifically, we divide each test set into four groups according to the dialog turn. As shown in Figure 5, MarsG is superior to the baseline system in every dialog turn group. This indicates our proposed contrastive learning strategies are beneficial to taskoriented dialog modeling. Especially, as dialog turn increases, the performance of the baseline system decreases rapidly, and the performance gap between the baseline system and our proposed Mars-G is increased. Because the baseline system is hard to model long-range semantic dependencies to generate inaccurate semantic states and system responses. In contrast, Mars-G enhances the relationship modeling between dialog context and semantic state representations and achieves better dialog context representations to capture longrange semantic dependencies in the long dialog turns.

heading: 5.6 Low Resource Scenario Analysis
text: To investigate the performance of task-oriented dialog systems in the low resource scenario, we choose 5%, 10%, 20%, and 50% of training dialog sessions to do stimulated experiments on the MultiWOZ 2.0. Considering the inconsistency of data distribution with different random seeds in the stimulated low resource scenario, we re-implement all baseline systems with the same random seed to ensure the consistency of data distribution. In addition, we train all dialog systems five times with different random seeds and report the average scores in Table 5. The detailed results of five runs are provided in Appendix H. As shown in Table 5, PPTOD achieves the best performance in the extreme low-resource scenario (5% training data) because auxiliary corpora used in PPTOD have many similar dialog sessions with MultiWOZ
2.0 and this benefits PPTOD in the stimulated lowresource scenario. In contrast, Mars-G doesn’t use auxiliary corpora to improve performance in the low-resource scenario. Apart from this, Mars-G substantially outperforms all baseline systems in other low-resource scenarios. Moreover, Mars-G trained on the 50% training data performs better than some baseline systems such as MinTL and UBAR trained on all training data, as shown in Table 1. These further demonstrate that Mars-G is robust, achieving comparable performance in the low-resource scenario.

heading: 5.7 Error Analysis
text: To better apply our proposed Mars-G to real-world scenarios, we perform error analysis based on inform rate (informable slot) and success rate (requestable slot). In detail, we randomly extract 40 inaccurate dialog sessions from the MultiWOZ 2.0 testing set, respectively. The detailed domain distribution and primary reason distribution of informable slot errors are presented as shown in Figure 6. Given that there is no database in the taxi domain, the informable slots are consistently judged to be correct. The error rate of the dialogs in the hotel and restaurant domains is very high because some informable slots in these two
domains are often mispredicted, such as ‘type’ in the hotel domain. As illustrated in Figure 6(b), 64 percent of dialog informable slot errors are caused by the inaccurate belief states and action states, and the noisy dialog annotations generate 32 percent. 4 percent of that are caused by automatic evaluation scripts and are judged accurately by human evaluation. The detailed requestable slot error analysis and more examples are provided in Appendixes I and J, respectively. In the future, we will focus on solving errors caused by the inaccurate dialog/ action states to better apply MarsG to real-world scenarios.

heading: 6 Conclusion
text: This study explores what dialog context representation is beneficial to improving task-oriented dialog performance. Specifically, we propose two contrastive learning strategies to explicitly model the relationship between dialog context and semantic state representations, achieving better task completion of a task-oriented dialog system. Extensive experiments and analysis demonstrate that dialog context representations that are more different from semantic state representations are more beneficial to multi-turn task-oriented dialog. Moreover, our proposed Mars achieves state-of-the-art performance on three datasets.
Limitations
The training process of Mars needs to rely on manually annotated belief states and action states as semantic states to explicitly model the relationship between dialog context and semantic state representations through contrastive learning methods. We propose Mars in the research community and hope it can be better applied to real-world scenarios in the industry. However, the annotated data is expensive, which makes our methods have some limitations in the landing process of real scenarios. In the future, to better apply our proposed Mars to real-world scenarios, we will introduce semi-supervised methods to reduce the dependence on annotated dialog corpus.

heading: Acknowledgments
text: This work was supported by the National Key R&D Program of China under Grant No. 2020AAA0108600.

heading: A Datasets
text: MultiWOZ 2.0 (Budzianowski et al., 2018) is a large-scale English multi-domain task-oriented dialog dataset containing 8438, 1000, and 1000 dialog sessions for training, validation, and testing datasets. It consists of seven domains: attraction, hotel, restaurant, taxi, train, hospital, and police. CamRest676 (Wen et al., 2017) is a small-scale English restaurant-domain dataset, which is split 3/ 1/ 1 for training, validation, and testing datasets. CrossWOZ (Zhu et al., 2020) is a large-scale Chinese multi-domain task-oriented dialog dataset containing 5012, 500, and 500 dialog sessions
for training, validation, and testing datasets. It comprises five domains: attraction, restaurant, hotel, taxi, and metro.

heading: B Baselines
text: Sequicity (Lei et al., 2018), DAMD (Zhang et al., 2020b), and LABES (Zhang et al., 2020a) are copyaugmented GRU-based end-to-end task-oriented dialog systems. Bidirectional auto-encoding language model BERT (Devlin et al., 2019) is used for the context encoder in DoTS (Jeon and Lee, 2021). Unidirectional auto-regressive language model GPT-2 (Radford et al., 2019) is used in AuGPT (Kulhánek et al., 2021), SOLOIST (Peng et al., 2021), and UBAR (Yang et al., 2021). Seq2seq language model T5 (Raffel et al., 2020) is used in MinTL (Lin et al., 2020), PPTOD (Su et al., 2021), and MTTOD (Lee, 2021). The unified language model UniLM (Dong et al., 2019) is used in GALAXY (He et al., 2022). In addition, auxiliary task-oriented dialog corpora are used to pre-train in AuGPT (Kulhánek et al., 2021), SOLOIST (Peng et al., 2021), PPTOD (Su et al., 2021), and GALAXY (He et al., 2022). TRADE (Wu et al., 2019), BARTCSP (Moradshahi et al., 2021), and GEEX (Li et al., 2021) are some additional dialog state tracking models.

heading: C States Analysis
text: To investigate the impact of belief state and action state on the performance of end-toend task-oriented dialog, we empirically conduct preliminary experiments on MultiWOZ 2.0 (Budzianowski et al., 2018). As shown in Table 6, the system using ground truth belief state and action state substantially outperforms the traditional end-to-end task-oriented dialog systems, achieving performance comparable to reference in terms of task completion. This demonstrates that the quality of belief state and action state greatly
Euclidean distance
influence on the end-to-end task-oriented dialog performance.

heading: D Dialog Context Representation Analysis
text: To further analyze dialog context and semantic state representations of Mars-P and Mars-G, we would measure the similarity of continuous encoder representation between dialog context and corresponding belief/action state on the MultiWOZ 2.0 test set, as illustrated in Figure 7. Table 7 shows the average L2-normalized Euclidean distance between dialog context and corresponding belief/action state representations. Table 8 shows the Euclidean distance between the centroids of these two L2-normalized representation spaces. The centroid is the average of all the points in the representation space. T5 denotes the result before training on the MultiWOZ 2.0. We find the distance between dialog context and corresponding semantic state representations changes a little before and after training. Mars-P achieves a smaller distance, thus obtaining a continuous representation of the dialogue context that is semantically more consistent with its semantic state representation. The distance of Mars-G is enormous, demonstrating Mars-G achieves more diverse dialog context representations, different from semantic state representations.

heading: E Hyper-parameter Analysis
text: We empirically investigate how the hyperparameters λ and T for both modules of Mars-G affect the performance of task-oriented dialog on the MultiWOZ 2.0, respectively. The selection of λ influences the role of the contrastive learning objective function across the entire task-oriented
Model Context&Belief State Context&Action State
Model Context&Belief State Context&Action State
dialog training process. As Figure 8 shows, λ ranging from 0.01 to 5 nearly all improve task-oriented dialog performance. This indicates our proposed Mars-G is robust and effective. When λ = 0.1, w/ ASC achieves the best performance. When λ = 1, w/ BSC achieves the best performance. The selection of T affects the differentiation of hard negative samples. The smaller the value of T is, the more attention is paid to distinguishing complex negative samples. As shown in Figure 9, combined scores increase for almost all T values ranging from 0.01 to 10, and the best performance is achieved when T = 0.5 for both modules of Mars-G.
F Visualization
We provide an example to visualize the dialog state tracking process of our proposed Mars-G and baseline system. The cross-attention weights
between dialog context and generated belief states from the last layer of the transformer decoder stack are shown in Figures 10 and 11. Compared with the baseline system, Mars-G could achieve more accurate attention weights. The slot ‘arrive 09:00’ assigns high attention weights for the user utterance ‘09:00’ and previous belief state ‘arrive 09:00’. Similarly, the slots ‘destination mumford theatre’ and ‘departure wagamama’ accurately give high attention weights for the corresponding user utterance. The visualization further demonstrates that Mars-G could achieve more reasonable dialog context representation to generate accurate belief states.

heading: G Further Ablation Analysis
text: To get a more complete picture of the effectiveness of Mars-P, we introduce a similarity strategy (Mars-variant). We use the cosine similarity function to narrow the distance between the continuous representation of dialog contexts and semantic states for the same dialog session to model the relationship between dialog context and corresponding semantic state representations. We don’t distinguish the continuous representation of dialog context and states for different dialog sessions. As shown in Table 9, Marsvariant outperforms the baseline system by 3.7
combined scores, indicating the effectiveness of the relationship modeling between dialog context and corresponding semantic representations. In addition, Mars-variant underperforms Mars-P by 0.8 combined scores. This demonstrates that distinguishing the continuous representation of dialog context and states for different dialog sessions is beneficial for dialog modeling.

heading: H Low Resource Scenario Results
text: We train all dialog systems five times with different random seeds in the low resource scenario. The detailed results of 5 runs are provided in Table 10.

heading: I Requestable Slot Error Analysis
text: Considering the inclusion relationship of the two metrics described in Section 5.1, we select dialog sessions with the wrong success rate and accurate inform rate for success rate error analysis. The detailed domain distribution and primary reason distribution of requestable slot errors are presented as shown in Figure 12. The error rate of the dialogs in the taxi and train domains is very low because requestable slots in these two domains are few and simple. For example, the requestable slot in the taxi domain only has ‘phone’. The error rate of the dialogs in the attraction domain is very high. As illustrated in Figure 12(b), 77.5 percent of dialog requestable slot errors are caused by the noisy dialog annotations and automatic evaluation scripts. 15 percent of generated system responses are acceptable. When users request some information about something and do not ask for a specific requestable slot, Mars-G generates system responses that lack some requestable slots such as ‘postcode’ and ‘address’. In addition, MarsG requests users some other useful information instead of providing booked reference directly. We think system responses generated by Mars-G in both cases are reasonable. Inaccurate action states cause 7.5 percent of dialog requestable slot errors.

heading: J Examples for Error Analysis
text: Tables 11 - 19 show several examples generated by Mars-G for detailed error analysis. As shown in Table 11, Mars-G generates the inaccurate belief state ‘food jamaican’ rather than ‘food italian’, leading to the informable slot error. Table 12 shows that Mars-G generates the inadequate action state, not including the slot name ‘name’, leading to the informable slot error. Table 13 shows that the informable slot error is caused by automatic evaluation. Mars-G provides the accurate response in turn 7. However, the automatic evaluation script estimates the wrong active domain ‘[taxi]’ rather than ‘[attraction]’ from the belief state. The informable slot error in Table 14 is caused by noisy dialog annotations. The informable slot ‘pricerange moderate’ does not appear in the conversation.
As shown in Table 15, Mars-G generates the inaccurate action state ‘[request] people’ provided in the belief state ‘people 1’, leading to the requestable slot error. Table 16 shows that the requestable slot error is caused by automatic evaluation. Mars-G provides the accurate response in turn 2, while the automatic evaluation script
only determines if it offers a reference in turn 3. We think system responses generated by MarsG in Tables 17 and 18 are acceptable. When users request some information about something and do not ask for a specific requestable slot, Mars-G generates system responses, lack of the requestable slot ‘postcode’, as shown in Table 17. In addition, Mars-G requests users whether to make a reservation instead of providing booked references directly, as shown in Table 18. Noisy dialog annotations cause the requestable slot error in Table 19. The requestable slot ‘postcode’ does not appear in the conversation.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Yes. In Section Limitations
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Yes. In Section 1, we list all the contributions.

heading: 7 A4. Have you used AI writing assistants when working on this paper?
text: Left blank.
B 3 Did you use or create scientific artifacts? Left blank.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Yes. In Section 5.2

heading: 7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: No. The datasets are public data.

heading: 7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 7

heading: 7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No. We only use public data.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Yes. in Section 5.1.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Yes. in Section 5.1.
C 3 Did you run computational experiments? Left blank.

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Yes. in Section 5.2.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Yes. in Section 5.2 and Appendix E.

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Yes. in Section 5.2.

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Yes. in Section 5.2.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: In the future, to better apply our proposed Mars to real-world scenarios, we will introduce semi-supervised methods to reduce the dependence on annotated dialog corpus