CLASP : Few-Shot Cross-Lingual Data Augmentation for Semantic Parsing (2022)

Abstract: A bottleneck to developing Semantic Parsing (SP) models is the need for a large volume of human-labeled training data. Given the complexity and cost of human annotation for SP, labeled data is often scarce, particularly in multilingual settings. Large Language Models (LLMs) excel at SP given only a few examples, however LLMs are unsuitable for runtime systems which require low latency. In this work, we propose CLASP, a simple method to improve low-resource SP for moderate-sized models: we generate synthetic data from AlexaTM 20B to augment the training set for a model 40x smaller (500M parameters). We evaluate on two datasets in low-resource settings: English PIZZA, containing either 348 or 16 real examples, and mTOP cross-lingual zero-shot, where training data is available only in English, and the model must generalize to four new languages. On both datasets, we show significant improvements over strong baseline methods.

## introduction and related work semantic parsing (sp) is the task of mapping a natural language sentence to a structured representation of its meaning. sp enables conversational agents to handle requests such as ordering pizza, creating reminders, and playing music. a bottleneck to developing sp models is their reliance on a large amount of human annotated training data, which is difficult and expensive to curate (particularly for multilingual settings) due to the complexity of the annotation task (section 2). while large language models (llms) perform well at sp given limited data @xcite , they are unsuitable for runtime systems which require low latency. data augmentation (da) is a common approach to mitigating data scarcity, and recently llms are shown to excel at in-context @xcite training data generation for sentence-level tasks ( @xcite @xcite . fine-tuned llms can also generate data for english slot tagging @xcite and multilingual intent classification and slot tagging @xcite ). as we discuss in section 2, sp poses unique challenges for da, and remains relatively under-explored in the field. prior work is either limited to heuristic re-combination of the training data @xcite or else assumes the availability of large-scale unannotated natural data @xcite . furthermore, there is a gap in the literature on multilingual da input: large pizza with extra cheese and pineapple hold the ham and two sprites please ## clasp methods to address the challenge of maintaining text-label agreement when generating sp training data, we propose clasp (cross-lingual data augmentation for semantic parsing). clasp consists of four methods for prompting llms to generate training data, either in the same language [sl] or cross-lingually [cl]: (1) rs: replace slots, generate text [sl]; (2) ts: translate slots, generate text [cl]; (3) gb: generate both parse and text [sl]; and (4) tb: translate both parse and text [cl]. ## conclusion and future work we have demonstrated clasp, a simple method to generate synthetic training data for multi-lingual semantic parsing by prompting a frozen large language model. in very low-resource (@xmath0, on two datasets covering five languages, we show significant improvements over strong baseline methods. in future work, we would like to evaluate on more languages and datasets, combine our method with criss style pre-training, and extend our method to more tasks such as textto-sql and code generation.