Dialogue Meaning Representation for Task-Oriented Dialogue Systems (2022)

Abstract: Dialogue meaning representation formulates natural language utterance semantics in their conversational context in an explicit and machine-readable form. Previous work typically follows the intent-slot framework, which is easy for annotation yet limited in scalability for complex linguistic expressions. A line of works alleviates the representation issue by introducing hierarchical structures but challenging to express complex compositional semantics, such as negation and coreference. We propose Dialogue Meaning Representation (DMR), a pliable and easily extendable representation for task-oriented dialogue. Our representation contains a set of nodes and edges to represent rich compositional semantics. Moreover, we propose an inheritance hierarchy mechanism focusing on domain extensibility. Additionally, we annotated DMR-FastFood, a multi-turn dialogue dataset with more than 70k utterances, with DMR. We propose two evaluation tasks to evaluate different dialogue models and a novel coreference resolution model GNNCoref for the graph-based coreference resolution task. Experiments show that DMR can be parsed well with pre-trained Seq2Seq models, and GNNCoref outperforms the baseline models by a large margin.The dataset and code are available at https://github.com/amazon-research/dialogue-meaning-representation

## introduction a task-oriented dialogue (tod) system aims to serve users to accomplish tasks in specific domains with interactive conversations. the modeling of converting natural language semantics in their conversational context into a machine-readable structured representation, also known as the dialogue meaning representation, is at the core of tod system research. a meaning representation framework amazon-research/dialogue-meaning-representation. ## dialogue meaning representation this section describes the structure of the dmr graph, the domain-agnostic ontology, and the representation of general semantics. ## related work there is a rising interest in developing more flexible representation for tod other than the slots representation @xcite . in this section, we briefly introduce them and compare the most related works with dmr. amr and dialogue-amr using amr for semantic parsing has been studied from a very early time @xcite . there are several customer: oh, sorry, i want no toppings, instead order me a soda. that's all. t9 customer: emmm, please add salted butter and onions on it. customer: i want a veg out and one green strip, each with 4 slices. customer: okey, one sandwich. t7 t5 customer: hello, good morning! (v1 / openinggreeting) t1 agent: hello, what would you like to order today? t2 t3 agent: awesome! would you like to have some sandwich? i must say we have some sizzling sandwiches... t4 agent: i'll take that order. how about some toppings for sandwich? t6 agent: perfect! is there anything else that you'd like to order? t8 (v1 / orderintent :order-item (v2 / and :op1 (v3 / veg out || pizza :quant (v4 / a || quantity) :mod (v5 / 4 slices || unit)) :op2 (v6 / green strip || pizza :quant (v7 / one || quantity) :mod (v8 / 4 slices || unit)))) (v1 / and :op1 (v2 / confirmationintent) :op2 (v3 / orderintent :order-item (v4 / sandwich || sandwich :quant (v5 / one || quantity)))) (v1 / orderintent :order-item (v2 / reference :refer t:5 n:v4 :ingredient (v3 / and :op1 (v4 / salted butter || ingredient) :op2 (v5 / onions || veggies)))) (v1 / orderintent :order-item (v2 / and :op1 (v3 / reference :refer t:5 n:v4 :ingredient (v4 / reference||toppings :refer (v5 / and :op1 t:7 n:v4 :op2 t:7 n:v5) :polarity -)) :op2 (v6 / soda || drinkitem :quant (v7 / a || quantity)))) figure 3: a dialogue example taken from fast food domain. the customer turns are annotated with dmrs. works that apply amr to dialogue systems. bai et al. (2021) model dialogue state with amr for chit-chat. bonial et al. (2019) extended amr for human-robot dialogues, and further formalize it as dialogue-amr @xcite . dialogue-amr represents both the illocutionary force and the propositional content of the utterance. compared to these works, dmr focuses on tod specifically with extended node types for tod description. dmr is intent-centric, and only captures semantics defined in the ontology of the intents. further, the design of inheritance hierarchy aims at a better domain generalization to support a broad range of applications. amr and dialogue-amr are closely related to dmr, so we have more detailed comparisons in appendix b to show the differences between them. compositional intent-slot some recent works focus on the compositional intent-slot framework, such as top @xcite , sb-top (aghajanyan et al., 2020) and treedst @xcite . these formalizations are much more powerful than the flat intent-slot schema. generally, they focus less on how to get representations for different domains and give fewer descriptions on how nodes/edges are connected. on the contrary, the design which contains both domain-agnostic and domain-specific parts allows dmr to be applied and extended to different domains while assuring the maintenance of the representation structure in the meantime. from this point of view, dmr is designed to provide services to different businesses. this different focus of the application scenarios marks the key difference between dmr and these compositional intent-slot representations. programs many efforts have been devoted to explore the representation in programs @xcite @xcite . though powerful expressiveness, they are hard in annotation which makes it limited in proposing large-scale dialogue datasets. recently, some works such as smcalflow (semantic machines et al., 2020) and toc @xcite and thingtalk @xcite proposes to use executable dialogue states for tod. to this end, the representation itself is also a specially-designed programming language. while executing, both database operations and response generation are performed by the program at the same time. dmr keeps the dialogue state architecture, and leaves the implementation of business logic to the user applications. ## data we use the fast food domain data from the multi-dogo dataset proposed by @xcite . we annotate all the customers' utterances with the redefined ontology. we call the annotated dataset dmr-fastfood. this dataset contains 7k annotated dialogues, and each dialogue has 18.5 turns on average, which is much more than other datasets. further, there are 7k references, 16k conjunctions and 557 negations annotated. the annotation process, statistics and comparison with related datasets are described in appendix a. ## nlu with dmr as nlu tasks of the flat intent-slot representation including intent classification and slot filling, nlu tasks under the dmr framework are to extract dmr graphs from the customer's utterances. given a customer's utterance x i and the dialogue context (x 0 , • • • , x i-1 ), the nlu tasks are to predict the dmr graph g i . in this section, we introduce nlu tasks with dmr and proposed models. ## experiments we report experiments for dmr parsing and coreference resolution separately, and the combined results on the complete dmr graphs. further, we analyze the key factors that affect the model performance for the two tasks. the hyperparameters used and training details are listed in appendix d. ## conclusion in this paper, we focus on the representation with the expression ability of both complex compositional semantics and task-oriented semantics and propose dmr which is capable of representing complex linguistic constructions with a high transferability across domains. moreover, we design the inheritance hierarchy which allows reusing, extending and inheriting node types to enable dmr scale to different domains easily. we annotated a large dataset on fast-food ordering domain, named dmr-fastfood, to incentive research on semantics parsing, which contains more than 70k utterances annotated with rich linguistic semantics. we conduct experiments on dmr parsing and coreference resolution tasks. experimental results show that pre-trained seq2seq models could improve the dmr parsing results. we also propose a graphbased model, gnncoref, for the coreference resolution on dmrs.