Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog (2021)

Abstract: Semantic parsing using sequence-to-sequence models allows parsing of deeper representations compared to traditional word tagging based models. In spite of these advantages, widespread adoption of these models for real-time conversational use cases has been stymied by higher compute requirements and thus higher latency. In this work, we propose a non-autoregressive approach to predict semantic parse trees with an efficient seq2seq model architecture. By combining non-autoregressive prediction with convolutional neural networks, we achieve significant latency gains and parameter size reduction compared to traditional RNN models. Our novel architecture achieves up to an 81% reduction in latency on TOP dataset and retains competitive performance to non-pretrained models on three different semantic parsing datasets.

## introduction advances in conversational assistants have helped to improve the usability of smart speakers and consumer wearables for different tasks. semantic parsing is one of the fundamental components of these assistants and it helps to convert the user input in natural language to a structure representation that can be understood by downstream systems. majority of the semantic parsing systems deployed on various devices, rely on server-side inference because of the lower compute/memory available on these edge devices. this poses a few drawbacks such as flaky user experience with spotty internet connectivity and compromised user data privacy due to the dependence on a centralized server to which all user interactions are sent to. thus, semantic parsing on-device has numerous advantages. for the semantic parsing task, the meaning representation used decides the capabilities of the system built. limitations of the representation with one intent and slot labels were studied in the context of nested queries and multi turn utterances in @xcite and @xcite . new representations were proposed to overcome these limitations and sequence-to-sequence models were proposed as the solution to model these complex forms. but using these new models in realtime conversational assistants still remains a challenge due to higher latency requirements. in our work, we propose a novel architecture and generation scheme to significantly improve the end2end latency of sequence-to-sequence models for the semantic parsing task. due to the autoregressive nature of generation in sequence-to-sequence semantic parsing models, the recurrence relationship between target tokens creates a limitation that decoding cannot be parallelized. there are multiple works in machine translation which try to solve this problem. these approaches relax the decoder token-by-token generation by allowing multiple target tokens to be generated at once. fully non-autoregressive models @xcite @xcite and conditional masked language models with iterative decoding @xcite @xcite are some of them. to enable non-autoregressive generation in semantic parsing, we modify the objective of the standard seq2seq model to predict the entire target structure at once. we build upon the cmlm (conditional masked language model) @xcite and condition the generation of the full target structure on the encoder representation. by eliminating the recurrent relationship between individual target tokens, the decoding process can be parallelized. while this drastically improves latency, the representation of each token is still dependent on previous tokens if we continue to use an rnn architecture. thus, we propose a novel model architecture for semantic parsing based on convolutional networks @xcite to solve this issue. our non-autoregressive model achieves up to an 81% reduction in latency on the top dataset @xcite , while achieving 80.23% exact match accuracy. we also achieve 88.16% exact match accuracy on dstc2 @xcite and 80.86% on snips @xcite which is competitive to prior work without pretraining. to summarize, our two main contributions are: ## method in this section, we propose a novel, convolutional, non-autoregressive architecture for semantic parsing. while non-autoregressive decoding has been previously explored in machine translation, we describe how it can be applied to semantic parsing with several critical modifications to retain performance. we then describe our convolutional architecture. by incorporating these advances together, our approach achieves both high accuracy and efficient decoding. the task is to predict the semantic parse tree given the raw text. we use the decoupled representation @xcite , an extension of the compositional form proposed in @xcite for task oriented semantic parsing. decoupled representation is obtained by removing all text in the compositional form that does not appear in a leaf slot. efficient models require representations which are compact, with least number of tokens, to reduce number of floating point operations during inference. decoupled representation was found to be suitable due to this. figure 1 shows the semantic parse for a sample utterance. our model predicts the serialized representation of this tree which is ## results we show that our proposed non-autoregressive convolutional architecture for semantic parsing is competitive with auto-regressive baselines and word tagging baselines without pre-training on three different benchmarks and reduces latency up to 81% on the top dataset. we first compare accuracy and latency, then discuss model performance by analyzing errors by length, and the importance of knowledge distillation. we do our analysis on the top dataset, due to its inherent compositional nature, however we expect our analysis to hold for other datasets as well. non-compositional datasets like dstc2 and snips can be modeled by word tagging models making seq2seq models more relevant in the case of compositional datasets. ## conclusion this work introduces a novel alternative to autoregressive decoding and efficient encoder-decoder architecture for semantic parsing. we show that in 3 semantic parsing datasets, we are able to speed up decoding significantly while minimizing accuracy regression. our model is able to generate parse trees competitive with state of the art autoregressive models with significant latency savings, allowing complex nlu systems to be delivered on edge devices. there are a couple of limitations of our proposed model that naturally extend themselves to future work. primarily, we cannot support true beam decoding, we decode a single prediction for each length prediction however there may exist multiple beams for each length prediction. also for longer parse trees and more complex semantic parsing systems such as session based understanding, our nar decoding scheme could benefit from multiple iterations. lastly, though we explored models without pre-training in this work, recent developments show the power of leveraging pre-trained models such as roberta and bart. we leave it to future work to extend our non-autoregressive decoding for pre-trained models.