Improving Top-K Decoding for Non-Autoregressive Semantic Parsing via Intent Conditioning (2022)

Abstract: Semantic parsing (SP) is a core component of modern virtual assistants like Google Assistant and Amazon Alexa. While sequence-to-sequence based auto-regressive (AR) approaches are common for conversational SP, recent studies employ non-autoregressive (NAR) decoders and reduce inference latency while maintaining competitive parsing quality. However, a major drawback of NAR decoders is the difficulty of generating top-k (i.e., k-best) outputs with approaches such as beam search. To address this challenge, we propose a novel NAR semantic parser that introduces intent conditioning on the decoder. Inspired by the traditional intent and slot tagging parsers, we decouple the top-level intent prediction from the rest of a parse. As the top-level intent largely governs the syntax and semantics of a parse, the intent conditioning allows the model to better control beam search and improves the quality and diversity of top-k outputs. We introduce a hybrid teacher-forcing approach to avoid training and inference mismatch. We evaluate the proposed NAR on conversational SP datasets, TOP & TOPv2. Like the existing NAR models, we maintain the O(1) decoding time complexity while generating more diverse outputs and improving top-3 exact match (EM) by 2.4 points. In comparison with AR models, our model speeds up beam search inference by 6.7 times on CPU with competitive top-k EM.

## introduction neural sequence models are widely used for the task of conversational semantic parsing, which converts natural language utterances to machineunderstandable meaning representations. recent approaches @xcite @xcite @xcite combine transformer-based sequence models @xcite and pointer generator networks @xcite et @xcite . a vast majority of semantic parsing approaches @xcite @xcite @xcite employ autoregressive (ar) decoders to generate structured output frames for the quality of output parses. during the ar decoding, output tokens are generated sequentially, conditioned on all previously generated tokens. as a result, the decoding time (i.e. inference latency) increases linearly with the decoding length. this not only limits the capacity to accommodate larger language models but may also degrade the user experience of intelligent conversational assistants (e.g. google assistant, amazon alexa) due to the high inference latency. on the contrary, non-autoregressive (nar) decoders are capable of parallel decoding and thus allow much faster inference. compared to ar decoders, which perform o(n) decoding steps, the nar decoding is typically achieved in either o(1) @xcite or o(log(n)) @xcite steps. a recent study @xcite built nar parsers and reduced the latency up to 81% compared to ar parsers. another study by @xcite pro-posed a nar semantic parser called span pointer network and cut the latency by 8.5-10x on cpu while achieving comparable performance to their ar benchmarks. all these studies suggest that the nar decoders have significant latency benefits. although the recent nar semantic parsers have decreased the performance gap from ar parsers on the exact match metric, one of the main limitations of nar models still remain unsolved: beam search and top-k outputs. ar models leverage beam search algorithms @xcite , which are especially effective at producing top-k outputs @xcite . this is possible since the beam search allows ar models to dynamically sort out lessprobable candidate parses and adjust their decoding lengths. as a result, ar models are capable of generating diverse high-quality output parses. on the other hand, the existing nar parsers @xcite cannot employ the ar beam search as the output tokens of the nar parsers are generated independently from each other. instead, the existing nar parsers perform a beam search by generating k candidate frame lengths. for each frame length, a single parse is produced, resulting in a total of k parses. however, we find that the existing nar beam search tends to produce duplicates of the most probable output parse. as exemplified in the ability to generate diverse top-k parses is important for modern conversational assistants. this work focuses on improving the quality of top-k outputs of nar conversational semantic parsing. our idea is to leverage the fact that the toplevel intent of a semantic parse mainly determines the syntax and semantics of the parse. this is the key driving point of the classic intent classification and slot tagging @xcite based parsers, which were widely used traditional semantic parsers. we build our model upon the existing query: avoid bridges on my route parses from the proposed nar (1a) [in:update_directions [sl:path_avoid [in:get _location [sl:category_location bridges]]]] (2a) [in:get_directions [sl:path_avoid [in:get _location [sl:category_location bridges]]]] parses from the baseline nar (1b) [in:update_directions [sl:path_avoid [in:get _location [sl:category_location bridges]]]] (2b) [in:update_directions [sl:path_avoid [in:get _location [sl:category_location bridges nar (i.e., frame length conditioned nar), but decouple the prediction of the top-level intent from the output. we leverage the conditional dependencies of the frame length and parse on the top-level intent for diverse top-k outputs. we evaluate the proposed nar model on two datasets: top @xcite and topv2 @xcite and demonstrate that we further close the gap between ar and nar parsers. most importantly, we show that the intent conditioning improves the quality and diversity of the top-k parses and the total inference time of our nar beam search is 6.7x times shorter than the ar baseline on cpu. ## proposed approach to mitigate the aforementioned limitation of the baseline nar, we propose intent-conditioned nonautoregressive neural semantic parser. our approach is motivated by traditional semantic parsers such as @xcite , which performs an intent classification followed by slot tagging. another motivation comes from an observation that the top-level intent of the parse largely governs the syntax and semantics of the rest of the parse. our idea is to decouple the top-level intent prediction from the output prediction as shown in figure 1 . this builds explicit dependencies of the frame length and the rest of the frame on the toplevel intent. as a result, the joint probability of output and frame length is expressed as follows. ## results we show that the proposed intent-conditioned nar further closes the gap between ar and nar sp. most importantly, our model greatly improves the quality of top-k output parses while maintaining the o(1) decoding time of the baseline nar. ## conclusion we present a novel non-autoregressive neural semantic parser: the intent-conditioned nar for improved top-k decodings. the proposed model addresses the main limitation of the recent nar models (i.e., the limited beam search capability) by decoupling the prediction of the top-level intent from output. this builds explicit dependencies of frame lengths and output parses on top-level intents and helps nar semantic parsers to better control beam search. the proposed nar model further closes the performance gap from ar models by improving the quality and diversity of top-k outputs. we highlight that the decoding time complexity is o(1), regardless of output length or beam width.