A Sequence-to-Sequence Approach to Dialogue State Tracking (2021)

Abstract: This paper is concerned with dialogue state tracking (DST) in a task-oriented dialogue system. Building a DST module that is highly effective is still a challenging issue, although significant progresses have been made recently. This paper proposes a new approach to dialogue state tracking, referred to as Seq2Seq-DU, which formalizes DST as a sequence-to-sequence problem. Seq2Seq-DU employs two BERT-based encoders to respectively encode the utterances in the dialogue and the descriptions of schemas, an attender to calculate attentions between the utterance embeddings and the schema embeddings, and a decoder to generate pointers to represent the current state of dialogue. Seq2Seq-DU has the following advantages. It can jointly model intents, slots, and slot values; it can leverage the rich representations of utterances and schemas based on BERT; it can effectively deal with categorical and non-categorical slots, and unseen schemas. In addition, Seq2Seq-DU can also be used in the NLU (natural language understanding) module of a dialogue system. Experimental results on benchmark datasets in different settings (SGD, MultiWOZ2.2, MultiWOZ2.1, WOZ2.0, DSTC2, M2M, SNIPS, and ATIS) show that Seq2Seq-DU outperforms the existing methods.

## introduction a task-oriented dialogue system usually consists of several modules: natural language understanding (nlu), dialogue state tracking (dst), dialogue policy (policy), and natural language generation (nlg). we consider dst and also nlu in this paper. in nlu, a semantic frame representing the content of user utterance is created in each turn * the work was done when the first author was an intern at bytedance ai lab. @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite slots: "depart": starting city for the trip. ## related work there has been a large amount of work on task-oriented dialogue, especially dialogue state tracking and natural language understanding (eg., @xcite @xcite ). ## our approach our approach seq2seq-du formalizes dialogue state tracking as a sequence to sequence problem using bert and pointer generation. as shown in figure 2 , seq2seq-du consists of an utterance encoder, a schema encoder, an utterance schema attender, and a state decoder. in each turn of dialogue, the utterance encoder transforms the current user utterance and the previous utterances in the dialogue into a sequence of utterance embeddings using bert; the schema encoder transforms the schema descriptions into a set of schema embeddings also using bert; the utterance schema attender calculates attentions between the utterance embeddings and the schema embeddings to create attended utterance and schema representations; finally, the state decoder sequentially generates a state representation on the basis of the attended representations using lstm and pointer generation.