Treepiece: Faster Semantic Parsing via Tree Tokenization (2023)

Abstract: Autoregressive (AR) encoder-decoder neural networks have proved successful in many NLP problems, including Semantic Parsing – a task that translates natural language to machine-readable parse trees . However, the sequential prediction process of AR models can be slow. To accelerate AR for semantic parsing, we introduce a new technique called TreePiece that tokenizes a parse tree into subtrees and generates one subtree per decoding step. On TOPv2 benchmark, TreePiece shows 4.6 times faster decoding speed than standard AR, and comparable speed but significantly higher accuracy compared to Non-Autoregressive (NAR).

## introduction autoregressive (ar) modeling @xcite is a commonly adopted framework in nlp where the next prediction is conditioned on the previously generated tokens. this paper focuses on ar approach for semantic parsing @xcite , an nlp task that converts a natural language utterance to a machine-interpretable symbolic representation called logical form. the sequence of actions to derive a logical form is isomorphic to a directed tree and often referred to as a parse tree @xcite . the runtime latency of ar linearly correlates to the output length and could result in low inference speed @xcite . non-autoregressive (nar) modeling @xcite @xcite , on the other hand, is able to produce outputs in parallel and reduce latency by an order of magnitude @xcite . however, nar performs considerably worse than its ar counterparts without extra training recipes @xcite @xcite . the quality benefits of ar models therefore motivates us to improve their speed, rather than exploring nar.