Multi-level Alignment Pretraining for Multi-lingual Semantic Parsing (2020)

Abstract: In this paper, we present a multi-level alignment pretraining method in a unified architecture formulti-lingual semantic parsing. In this architecture, we use an adversarial training method toalign the space of different languages and use sentence level and word level parallel corpus assupervision information to align the semantic of different languages. Finally, we jointly train themulti-level alignment and semantic parsing tasks. We conduct experiments on a publicly avail-able multi-lingual semantic parsing dataset ATIS and a newly constructed dataset. Experimentalresults show that our model outperforms state-of-the-art methods on both datasets.

## model in this section, we will first briefly introduce the basic sequence-to-sequence (s2s) model as our baseline model. then, we introduce the architecture of our multi-level alignment pretraining for multi-lingual semantic parsing (masp) model. ## experiment we conduct our experiments on two datasets, atis and mlsp. ## related work semantic parsing, as an important task in natural language understanding, has attracted significant attention in the research and industry. recently, various semantic parsing models have been proposed such as @xcite @xcite @xcite @xcite . @xcite propose a combinatory categorical grammar induction technique for semantic parsing. @xcite use grammar and syntax information to improve semantic parsing models. @xcite apply a transfer learning method to semantic parsing. @xcite propose a tree-based decoder to model structure of logical forms. @xcite translate the decode process as a sequence of actions with a sequence-to-sequence model. recently, @xcite propose a two-stage model to decode the logical form with the help of sketches, which contain structure and predicates in logical forms. in multi-lingual semantic parsing, @xcite use majority voting ensemble method to combine outputs from parsers for certain languages to apply on multi-lingual semantic parsing. @xcite use a sequence-to-sequence model to map the questions in the source language into decompositional semantic representations in the target language. in susanto and lu (2017)'s work, they propose a combination method to combine questions in different language simultaneously for multi-source input and achieve promising improvement on atis @xcite . they also explore different architectures for single-source input without their combination mechanism. @xcite propose a method to learn a cross lingual representation and use it in their semantic parsing model @xcite . ## conclusion in this paper, we propose a multi-lingual semantic parsing model, which is first pretrained using a multilevel alignment mechanism, and then we jointly train the multi-lingual semantic parsing and multi-level alignment tasks. most existing multi-lingual semantic parsing datasets are based on specific domain, to better evaluate our method on open domain, we annotate a relative large scale multi-lingual semantic parsing dataset on open domain. experimental results on atis and our dataset show the effectiveness and robustness of our model.