Cross-Lingual Data Augmentation For T hai Question-Answering (2023)

Abstract: This paper presents an innovative data augmentation framework with data quality control designed to enhance the robustness of Question Answering (QA) models in low-resource languages, particularly Thai. Recognizing the challenges posed by the scarcity and quality of training data, we leverage data augmentation techniques in both monolingual and cross-lingual settings. Our approach augments and enriches the original dataset, thereby increasing its linguistic diversity and robustness. We evaluate the robustness of our framework on Machine Reading Comprehension, and the experimental results illustrate the potential of data augmentation to effectively increase training data and improve model generalization in low-resource language settings, offering a promising direction for the data augmentation manner.

## introduction question answering (qa) systems are algorithms developed to answer questions posed in a natural language format accurately. a primary task in qa is machine reading comprehension (mrc), which aims to extract answers from text passages given a question. previous works demonstrate that improving the performance of mrc increases the accuracy in real-world applications, i.e., conversational chatbots @xcite @xcite . while the performance of qa systems in english is largely considered to be a solved problem, it is still an open problem in low-resource languages, i.e., thai. recent works in qa demonstrate that the performance gap in qa systems for thai and english is wide. for example, the baseline performance of english on the xquad @xcite dataset has an f1 score of 83.5, while only 42.7 on thai. moreover, only five of the research works @xcite @xcite @xcite has been published on qa for the thai language within the last five years. such disparity has led to a significant gap in the capabilities of nlp systems between high-resource languages and low-resource ones @xcite @xcite . the robustness of nlp applications in low-resource languages leaves much to be desired due to the lack of extensive and diverse language data that covers all aspects of the language. existing literature offers several methods to mitigate the problem of robustness specifically for low data in qa, such as transfer learning @xcite , back translation @xcite , and the use of multilingual language models @xcite . however, these techniques present drawbacks, e.g., transfer learning's success hinges largely on the relatedness of the source and target languages. in addition, multi-lingual models are impacted by the imbalanced data distribution and often do not perform as well as monolingual models @xcite . for example, wangchanberta @xcite , a monolingual roberta-based @xcite model trained explicitly on the thai language, outperforms multilingual models (xlm-r @xcite and mbert @xcite ). additionally, there is a noticeable lack of studies focusing on these methods in the context of the thai language, rendering the extension of such strategies unclear. the issue of robustness and generalization also remains largely unaddressed in recent literature, thereby leaving a significant gap in the field. to improve performance in a low-resource setting, we propose an automatic framework for improving out-of-distribution robustness in qa. our framework integrates back translation, word replacement, large language model (llm) automated paraphrase generation, and llm automated gram-mar correction to construct a 10-way parallel corpus. this corpus features multiple varied sentence formulations to encourage robustness and generalization. while our framework heavily leverages machine translation (mt), which allows our augmentations to leverage the vast library of english augmentations thereby improving the robustness of thai qa, it is our quality control system that sets our work apart. by rigorously removing noisy samples from the data-filtering the distances between the semantic representations of the augmented and the original data-the system ensures that the dataset obtained contains an optimal signal-to-noise ratio. as shown in figure 1 , the proposed framework works as follows. firstly, we aggregate, clean and normalize our datasets: tydiqa @xcite , xquad @xcite , iapp wiki qa @xcite , and thai qa @xcite . then, we translate all questions into english and backtranslate to thai using google translate. next, the gpt-3.5-turbo-0301 model is used in the third stage for grammar correction and paraphrasing of these translated questions. in the fourth stage, the quality controlled paraphrase generation (qcpg) @xcite model generates additional paraphrases for the translated questions, which are then translated back to thai. in addition, we leverage wordnet, thai2fit, thai2transformers, and large thai word2vec (ltw2vec) for wordreplacement on the thai questions without backtranslation. lastly, we utilize our quality control mechanism to filter noisy augmented samples from our corpora. the end corpora is a versatile, 10-way parallel corpus, ready for use in the mrc task. we evaluate the effectiveness of our framework compared to common augmentations without any cross-lingual augmentations on standard qa datasets for the mrc task. specifically we test for out-of-distribution by using completely different mrc datasets for evaluation. for more information please refer to our evaluation card in the appendix. the experimental results demonstrate that our framework significantly enhances the robustness and generalization of thai qa systems. for example, we improve the performance (exact match/f1) of wangchanberta on tydiqa and xquad datasets from 39.46/54.87 and 34.92/48.80, to 42.76/56.51 and 35.25/49.43, respectively. our design analysis of the quality control mechanism demonstrates that our mechanism removes noisy data from augmentation schemes and decreases the training time of mrc from 80 to 50 minutes (∼38.25% faster). we summarize the contribution of our work as follows: ## experimental results we demonstrate the experimental results of the machine reading comprehension task in out-of-domain settings in section 5.1. section 5.2 demonstrates the analysis of the harmonic distance method on the performance and training speed efficiency. in addition, we present error analysis by comparing various augmentation texts with the original text in section 5.3. ## conclusion and future work we present an automatic framework for improving robustness in qa tasks. the experimental results demonstrate that our cross-lingual augmentation improved the performance of thai qa more consistently than monolingual augmentations. moreover, we present the quality control for data augmentations, which decrease the training time and maintain the performance with only 20% of total augmented data. for future work, we would like to explore the application and task of our augmentation and quality control approaches to improve the generalization, such as conversational chatbots @xcite @xcite and retrieval qa @xcite .