M ars: Modeling Context & State Representations with Contrastive Learning for End-to-End Task-Oriented Dialog (2023)

Abstract: Traditional end-to-end task-oriented dialog systems first convert dialog context into belief state and action state before generating the system response. The system response performance is significantly affected by the quality of the belief state and action state. We first explore what dialog context representation is beneficial to improving the quality of the belief state and action state, which further enhances the generated response quality. To tackle our exploration, we propose Mars, an end-to-end task-oriented dialog system with two contrastive learning strategies to model the relationship between dialog context and belief/action state representations. Empirical results show dialog context representations, which are more different from semantic state representations, are more conducive to multi-turn task-oriented dialog. Moreover, our proposed Mars achieves state-of-the-art performance on the MultiWOZ 2.0, CamRest676, and CrossWOZ.

## introduction task-oriented dialog system @xcite aims to assist users in completing some specific tasks such as table reservations, hotel reservations, ticket booking, and online shopping. traditional task-oriented dialog system has been built through dialog state tracking @xcite , dialog policy @xcite and natural language generation @xcite tasks. dialog state tracking transfers dialog context to belief state, which is the structured semantic state capturing the whole dialog context information. the belief state is used for the dialog system to query the database to obtain matched entities. dialog policy selects an action state, a semantic state guiding context to state represents dialog state tracking and dialog policy tasks. the previous dialog context {c t-1 , u t-1 } is included in the dialog context {c t , u t } of turn t. the database state is omitted for clarity. the dialog system to generate a system response based on the current dialog context and database information. system response is generated through a natural language generation task. with the widespread application of large-scale pre-training models @xcite @xcite , researchers gradually focus on the end-to-end task-oriented dialog system @xcite @xcite , which converts the whole dialog context into system response through multi-task training. generally, an end-to-end task-oriented dialog modeling task is formulated as a cascaded generation problem @xcite . before generating a system response, the end-to-end task-oriented dialog system must first transfer dialog context into belief and action states, respectively. the quality of the belief state and action state greatly influence on the end-to-end task-oriented dialog performance foot_0 . in this paper, we explore what dialog context representation is beneficial to improving the quality of the belief/action state, which further enhances the generated response quality. as illustrated in figure 1 , dialog context is recursively hybrid of previous dialog context and semantic states 3 , i.e., belief and action states, for multi-turn dialog. intuitively, the representation of dialog context {c t-1 , u t-1 }, which is more similar with that of semantic states b t-1 /a t-1 , is beneficial to generate semantic states of the turn t -1. however, if their representations are too similar, there may be information redundancy in the representation of dialog context {c t , u t } in turn t, as shown in figure 1 . thus we raise another conjecture: whether representations of dialog context, which are more different from that of semantic states, are more conducive to multi-turn task-oriented dialog? to tackle our conjectures, we propose mars, an end-to-end task-oriented dialog system with two contrastive learning strategies, i.e., pair-aware context&state and group-aware context&state contrastive learning, to model the relationship between dialog context and semantic states from two different levels. specifically, (1) the pairaware context&state contrastive learning strategy focuses more on narrowing the gap in the continuous representation space between dialog context and corresponding semantic states for the same dialog turn. this strategy aims to obtain a continuous representation of the dialog context that is semantically more consistent with that of its semantic states. (2) group-aware context&state contrastive learning strategy enlarges the overall continuous representation margin between dialog context and semantic states. the meaning behind this is to make representations between dialog context and semantic states more different. extensive experiments and analysis on the response generation and dialog state tracking tasks verify our raised conjectures and the effectiveness of mars. mars achieves state-of-the-art performance on the multiwoz 2.0, camrest676, and crosswoz. moreover, mars achieves remarkable performance in the low-resource scenario. finally, we perform detailed error analysis and visualization to better apply our proposed mars to real-world scenarios. this paper primarily makes the following contributions: (1) we explore what dialog context representation is beneficial to improving taskoriented dialog performance. (2) we propose two contrastive learning strategies to model the relationship between dialog context and semantic state representations. (3) empirical results show mars achieves state-of-the-art performance on the multiwoz 2.0, camrest676, and crosswoz. ## related work end-to-end task-oriented dialog systems @xcite are established via copy-augmented seq2seq learning @xcite . @xcite proposes a multi-action data augmentation method to improve the diversity of generated system responses. large-scale pretrained language models, including bert @xcite , @xcite , t5 @xcite , and unilm @xcite , have been demonstrated effective for improving the performance of task-oriented dialog systems @xcite @xcite @xcite on multiwoz 2.0 @xcite , a large-scale english multi-domain task-oriented dialog dataset. recently, auxiliary tasks and auxiliary dialog corpora have been introduced to further improve dialog modeling ability. mttod @xcite introduces a span prediction task to enhance the natural language understanding performance. bort @xcite proposes reconstruction strategies to alleviate the error propagation problem. pptod @xcite proposes a dialog multi-task pre-training strategy to model task completion from auxiliary heterogeneous dialog corpora. galaxy @xcite introduces a dialog act prediction task to explicitly learn dialog policy from auxiliary dialog corpora. recently, contrastive learning @xcite @xcite has attracted much attention in the computer vision community and has been applied to natural language processing to enhance sentence representation learning @xcite @xcite @xcite . in contrast, we propose contrastive learning strategies to model the relationship between dialog context and semantic state representations for task-oriented dialog. in addition, we don't introduce data augmentation methods, which are used in most contrastive learning works. ## task-oriented dialog framework generally, an end-to-end task-oriented dialog modeling task is formulated as a cascaded generation problem @xcite . before generating a system response, the end-to-end task-oriented dialog system would transfer dialog 𝑨 𝒕 , 𝑹 𝒕 shared encoder belief state decoder shared encoder action state decoder 𝑪 𝒕 , 𝑼 𝒕 𝑩 𝒕 𝑫𝑩 𝒕 𝑪 𝒕 , 𝑼 𝒕 , 𝑫𝑩 𝒕 𝑯 𝒄𝒃 𝑯 𝒄𝒂 query 𝑪 𝒕 : can you find a theatre to go to in town? [attraction] type theatre [db_3] [attraction] [request] area sure, do you have an area of town you would like to visit? 𝑼 𝒕 : no, i am not concerned about that. 𝑩 𝒕 : [attraction] type theatre 𝑫𝑩 𝒕 : [db_3] 𝑨 𝒕 : [attraction] [inform] name area 𝑹 𝒕 : [value_name] is in the [value_area]. figure 2: illustration of a general task-oriented dialog system. for clarity, we take dialog turn @xmath0. c t is formulated as {u 0 , b 0 , db 0 , a 0 , r 0 }. we construct an end-to-end task-oriented dialog system via the seq2seq framework, including one shared encoder and two different decoders, as illustrated in figure 2 . one shared encoder encodes dialog context, one decoder decoder b (•) decodes belief state, and another decoder decoder a (•) decodes action state and system response. consider a dialog in turn t, dialog history c t , which contains dialog information for all previous turns, is formulated as {c t-1 , u t-1 , b t-1 , db t-1 , a t-1 , r t-1 }, where u represents the user utterance, b represents the belief state, db represents the database state, a represents the action state, and r represents the system response. for end-to-end dialog modeling, a belief state is first generated. the dialog history c t and the current user utterance u t are firstly encoded into hidden representation h cb through the shared encoder, and the belief state b t is generated through the belief state decoder: the dialog state tracking process is optimized by minimizing the following objective function: we use the generated belief state b t to query the specific database to achieve the database state db t , which means the amount of matched entities. as described by mttod @xcite , the second decoder would be used to generate action state and system response simultaneously. the combination of the dialog history c t , the current user utterance u t , and the database state db t are encoded into hidden representation h ca through the shared encoder. the action state a t and system response r t are generated in turn through the action state decoder: therefore, the action state and response generation process is optimized by minimizing the following objective function: in summary, the entire end-to-end task-oriented dialog system can be optimized by minimizing: to tackle our conjectures and enhance the relationship modeling between dialog context and corresponding semantic state representations of the task-oriented dialog system described in section 3, we propose two contrastive learning methods: pair-aware context&state and groupaware context&state contrastive learning. figure 3 illustrates the architecture of a task-oriented dialog system with our proposed methods. generally, for any contrastive learning method, contrastive learning objective functions l bscl and l ascl are added for dialog state tracking and response generation tasks, respectively, to enhancing the relationship modeling between dialog context and semantic state representations during end-to-end dialog training. the general objective function can be reformulated as follows: shared encoder belief state decoder shared encoder action state decoder 𝑩 𝒕 𝑫𝑩 𝒕 𝑯 𝒄𝒃 𝑯 𝒄𝒂 query [attraction] type theatre a [attraction] type college a [restaurant] name la margherita shared encoder 𝑯 𝒃𝒃 𝑩 𝒕 𝓛 𝒃𝒔𝒄𝒍 shared encoder 𝑨 𝒕 𝑯 𝒂𝒂 𝓛 𝒂𝒔𝒄𝒍 can you find a theatre to go to in town? i am looking for a college type attraction. i am looking for a restaurant called la margherita. [attraction] [request] area a [attraction] [select] area [inform] type choice [restaurant] [inform] food price area [general] [reqmore] can you find a theatre to go to in town? [db_3] i am looking for a college type attraction. [db_3] i am looking for a restaurant called la margherita. [db_1] 2 3 1 2 3 1 3 2 1 3 2 1 negative sample positive sample dialog context semantic state (belief /action state) 𝑪 𝒕 , 𝑼 𝒕 , 𝑫𝑩 𝒕 𝑪 𝒕 , 𝑼 𝒕 𝑨 𝒕 , 𝑹 𝒕 group-aware context&state 2 3 1 1 2 3 pair-aware context&state 2 3 1 1 2 3 figure 3 : illustration of the task-oriented dialog system with contrastive learning strategies. we take dialog turn @xmath1.