X 2 P arser: Cross-Lingual and Cross-Domain Framework for Task-Oriented Compositional Semantic Parsing (2021)

Abstract: Task-oriented compositional semantic parsing (TCSP) handles complex nested user queries and serves as an essential component of virtual assistants. Current TCSP models rely on numerous training data to achieve decent performance but fail to generalize to low-resource target languages or domains. In this paper, we present X2Parser, a transferable Cross-lingual and Cross-domain Parser for TCSP. Unlike previous models that learn to generate the hierarchical representations for nested intents and slots, we propose to predict intents and slots separately and cast both prediction tasks into sequence labeling problems. After that, we further propose a fertility-based slot predictor that first learns to detect the number of labels for each token, and then predicts the slot types. Experimental results illustrate that our model can significantly outperform existing strong baselines in cross-lingual and cross-domain settings, and our model can also achieve a good generalization ability on target languages of target domains. Furthermore, we show that our model can reduce the latency by up to 66% compared to the generation-based model.

## introduction virtual assistants can perform a wide variety of tasks for users, such as setting reminders, searching for events, and sending messages. task-oriented compositional semantic parsing (tcsp) which comprehends users' intents and detects the key information (slots) in the utterance is one of the core components in virtual assistants. existing tcsp models highly rely on large amounts of training data that usually only exist in high-resource domains and languages (e.g., english), and they 1 the code will be released in https://github.com/ zliucr/x2parser . generally fail to generalize well in a low-resource scenario. given that collecting enormous training data is expensive and time-consuming, we aim to develop a transferable model that can quickly adapt to low-resource target languages and domains. the traditional semantic parsing can be treated as a simple joint intent detection and slot filling task @xcite @xcite , while compositional semantic parsing has to cope with complex nested queries, which requires more sophisticated models. current state-ofthe-art tcsp models @xcite are generation-based models that learn to directly generate the hierarchical representations which contain nested intent and slot labels. 2 we argue that the hierarchical representations are relatively complex, and the models need to learn when to generate the starting intent or slot label, when to copy tokens from the input, and when to generate the end of the label. hence, large quantities of training data are necessary for the models to learn these complicated skills @xcite , while they cannot generalize well when large datasets are absent @xcite . moreover, the inference speed of generation-based models will be greatly limited by the output length. in this paper, we propose a transferable crosslingual and cross-domain parser (x2parser) for tcsp. instead of generating hierarchical representations, we convert the nested annotations into flattened intent and slot representations (as shown in figure 2 ) so that the model can learn to predict the intents and slots separately. we cast the nested slot prediction problem into a special sequence labeling task where each token can have multiple slot labels. to tackle this task, our model first learns to predict the number of slot labels, which helps it capture the hierarchical slot information in user queries. then, it copies the corresponding hidden state for each token and uses those hidden states to predict the slot labels. for the nested intent prediction, we cast the problem into a normal sequence labeling problem where each token only has one intent label since the nested cases for intents are simpler than those for slots. compared to generation-based models @xcite , x2parser simplifies the problem by flattening the hierarchical representations and tackles the task in a non-autoregressive way, which strengthen its adaptation ability in lowresource scenarios and greatly reduce the latency. as shown in figure 1 , we conduct experiments on three low-resource settings: cross-lingual, crossdomain, and a combination of both. results show that our model can remarkably surpass existing strong baselines in all the low-resource scenarios by more than 10% exact match accuracy, and can reduce the latency by up to 66% compared to generation-based models. we summarize the main contributions of this paper as follows: ## task decomposition in this section, we first introduce the intuition of decomposing the compositional semantic parsing into intent predictions and slot filling. then, we describe how we construct intent and slot labels. ## x2parser the model architecture of our x2parser is illustrated in figure 3 . to enable the cross-lingual ability of our model, we leverage the multilingual pre-trained model xlm-r @xcite as the sequence encoder. let us define @xmath0, x 2 , ..., x n } as the user utterance and @xmath1, h 2 , ..., h n } as the hidden states (denoted as emb in figure 3 ) from xlm-r. ## conclusion in this paper, we develop a transferable and nonautoregressive model (x2parser) for the tcsp task that can better adapt to target languages and domains with a faster inference speed. unlike previous tcsp models that learn to generate hierarchical representations, we propose to decompose the task into intent and slot predictions so as to lower the difficulty of the task, and then we cast both prediction tasks into sequence labeling problems. after that, we further propose a fertility-based method to cope with the slot prediction task where each token could have multiple labels. results illustrate that x2parser significantly outperforms strong baselines in all low-resource settings. furthermore, our model is able to reduce the latency by up to 66% compared to the generation-based model.