Improving Sequence-to-Sequence Semantic Parser for Task Oriented Dialog (2020)

Abstract: Task Oriented Parsing (TOP) attempts to map utterances to compositional requests, including multiple intents and their slots. Previous work focus on a tree-based hierarchical meaning representation, and applying constituency parsing techniques to address TOP. In this paper, we propose a new format of meaning representation that is more compact and amenable to sequence-to-sequence (seq-to-seq) models. A simple copy-augmented seq-to-seq parser is built and evaluated over a public TOP dataset, resulting in 3.44% improvement over prior best seq-to-seq parser (exact match accuracy), which is also comparable to constituency parsers’ performance.

## introduction today, most virtual assistants like alexa and siri are task oriented dialog systems based on gus architecture @xcite . they parse users' utterances to semantic frames composed of intents and slots. an intent normally represents a web api call to some downstream domain application to fulfill certain task. slots correspond to parameters required in web api calls. in this paper, the task of parsing utterances to semantic frames is called task oriented parsing @xcite . many prior work @xcite concentrate on parsing single-intent requests in which one utterance contains only one intent and its slots. @xcite proposes a hierarchical top representation to model the nested requests: one utterance contains multiple recursive intents and their slots. figure 1 .a shows an example of the hierarchical top representation, which is called base representation in this paper. other than expressiveness, base representation also enjoys the easy annotation, efficient parsing and low adoption barrier in practice. two types of models have been employed to perform top tasks: seq-to-seq models, and constituency parsing models @xcite . it has been reported that the latter consistently outperforms the former, probably because constituency parsing algorithms are dedicated to serving tree-based representation by design, while seq-to-seq architecture are purposed to serve more generalized form of representations such as graph and logical form @xcite . in this paper we introduce a compact top representation, which has fewer tokens than base presentation. further, we build a simple seq-to-seq model with attention-based copy mechanism to evaluate the effectiveness of the compact representation. experimental results on a public top dataset show that this approach can significantly improve seq-to-seq parser's inference performance and close its gap to current constituency parsers, who cannot handle the new top representation. ## representation in base representation, words are terminals, and intents and slots are nonterminals. the root node is an intent, and an intent is allowed to be nested inside a slot. in addition, base representation ## data the top dataset foot_0 is introduced in the work of @xcite , and it covers two domains: navigation and events. the utterances contain three types of queries: navigation, events and navigation to events. there are total 44783 annotated utterances with 25 intents and 36 slots. each utterance is annotated with a hierarchical meaning representation. about 30% of records have nested requests. among these data, the median depth of the trees is 2.54, and median length of the utterances is 8.93 tokens. in this work, we remove the records that have in:unsupported intent from the dataset. after this, the dataset has 28414 training records, 4032 validation records and 8241 test records, identical to @xcite . original dataset uses base representation, and we convert them to lotv representation and compact representation. average token lengths of lotv and compact representations are 17 and 12; their vocabulary sizes are 60 and 93 respectively. ## model we use a simple seq-to-seq with attention neural architecture to frame the top problem. encoder is one-layer bi-directional recurrent neural network with lstm @xcite . the final output hidden states of both directions are concatenated and projected to the first input state of decoder through a linear layer. in decoder, attention and output token at time step t are computed as below: where 𝑦 is output token, ℎ, 𝑐 are hidden state and context, α is attention score, 𝑎 is attention, 𝑜 is combined output. 𝑊 ($$)*+, and 𝑊 1 are trainable parameters. to better predict the word indexes in compact representation, we implement an attention-based copy mechanism, introduced by @xcite . first, we define the largest word index (utterance length) as system parameter and expand the decoder's vocabulary to include all word indexes from zero to the largest word index; then we modify the formula (6) to directly add the attention score α to compute the output tokens as below: we call the original model (without copy mechanism) as vanilla seq-to-seq, and the model with copy mechanism as copy-augmented seq-toseq. in this paper, we make two hypotheses: 1. top parsers should benefit the shorten parses of compact representation and produce better inductive bias than lotv representation despite the increase of token vocabulary size; 2. copy mechanism should boost the prediction performance of seq-to-seq model. ## conclusions in this paper, we propose a compact representation for top, which is more friendly to seq-to-seq parsers and demonstrates better performance than base representation and lotv representation. it opens up another door to improve the semantic parsing for task oriented dialog.