Cross-lingual Back-Parsing: Utterance Synthesis from Meaning Representation for Zero-Resource Semantic Parsing (2024)

Abstract: Recent efforts have aimed to utilize multilingual pretrained language models (mPLMs) to extend semantic parsing (SP) across multiple languages without requiring extensive annotations. However, achieving zero-shot cross-lingual transfer for SP remains challenging, leading to a performance gap between source and target languages. In this study, we propose Cross-Lingual Back-Parsing (CBP), a novel data augmentation methodology designed to enhance cross-lingual transfer for SP. Leveraging the representation geometry of the mPLMs, CBP synthesizes target language utterances from source meaning representations. Our methodology effectively performs cross-lingual data augmentation in challenging zero-resource settings, by utilizing only labeled data in the source language and monolingual corpora. Extensive experiments on two cross-language SP benchmarks (Mschema2QA and Xspider) demonstrate that CBP brings substantial gains in the target language. Further analysis of the synthesized utterances shows that our method successfully generates target language utterances with high slot value alignment rates while preserving semantic integrity. Our codes and data are publicly available at https://github.com/deokhk/CBP.

## introduction semantic parsing (sp) is the task of converting natural language utterances into meaning representations such as sql or python code. with numerous english parsing datasets available, recent studies have enabled applications ranging from natural language interfaces for databases to code generation @xcite . despite sp's practicality, extending it beyond english is challenging. manually annotating examples for other languages is very costly, and relying on machine translation is often impractical due to the complex slot alignment step after translation @xcite . 1 our codes and data are publicly available at https:// github.com/deokhk/cbp . ## methodology overview in this study, we synthesize target language utterances u tgt from source language meaning representation mr src to enhance the performance of sp models that convert u tgt into mr tgt . 2 cbp consists of two components: (1) an utterance generator (section 3.1) that synthesizes utterances in the target languages from source mr; the language identity switch operation alters the encoder output representation of the generator to reflect the source language identity, ensuring that the generator's decoder always receives the encoder representation with the source language identity regardless of the input language. we then train the utterance generator to produce output sequences in the target language using the modified encoder representation, while integrating languagespecific adapters @xcite into the transformer decoder. this training enables the adapter to prompt the generator to produce utterances in different languages while maintaining the same meaning from a given representation. then, we remove low-quality data from the synthesized utterances using the filtering mechanism. by re-parsing the generated utterances, we measure round-trip consistency @xcite to determine whether it accurately maps back to the input meaning representation used during generation. this data filtration process improves the quality of the synthesized data. ## results and analysis in tables 2 and 3 , we compare the performance of cbp against competitive baselines on the mschema2qa and xspider benchmarks. cbp improves the average em score on mschema2qa by 3.2%, with significant improvements of 8.8% in turkish and 5.0% in german, compared to the zero-shot method without data augmentation. similarly, on xspider, our method enhances chinese performance by 4.7% in em and 3.8% in ts. the filtering mechanism proves essential for our method, as evidenced by the significant drop in performance in its ablation (w/o filtering). remarkably, despite operating under the zero-resource setting, our method outperforms all baseline models on the mschema2qa dataset and even surpasses de-r 2 +translation-p, the state-of-the-art in the literature on the xspider dataset. these results highlight the effectiveness and practicality of cbp in cross-lingual sp. additionally, we find that gpt-3.5-turbo exhibits different performance trends on the two datasets. on mschema2qa, gpt-3.5-turbo performs poorly, indicating that in-context learning with english examples alone is insufficient to learn the dataset's domain-specific grammar. this highlights the practicality of zero-shot cross-lingual transfer through fine-tuning. conversely, on xspider, where the model has pre-trained knowledge about text-to-sql @xcite , gpt-3.5-turbo shows strong performance, surpassing ours in ts. however, our backbone model, mt5-large (1.2b parameters), is notably more parameter-efficient and cost-effective than gpt-3.5-turbo. ## conclusion we present cross-lingual back-parsing (cbp), a novel data augmentation methodology aimed at enhancing zero-shot cross-lingual transfer for semantic parsing. leveraging the representation geometry of multilingual pretrained language models, our method enables data augmentation in zero-resource settings. our experiments on two cross-lingual semantic parsing benchmarks demonstrate that cbp significantly improves performance, underscoring its effectiveness and practical applicability. while we focus on semantic parsing, we believe that cbp has the potential to be applied to other cross-lingual generation tasks in zero-resource settings. future work will investigate the application of our method to tasks such as cross-lingual text style transfer @xcite .