Data Augmentation for Low-Resource Italian NLP: Enhancing Semantic Processing with DRS
Authors: Muhammad Saad Amin, Luca Anselma, Alessandro Mazzei (Università di Torino, 2024)
Topic: Language Models

Abstract
Discourse Representation Structure (DRS), a formal meaning representation, has shown promising results in semantic parsing and natural language generation tasks for high-resource languages like English. This paper investigates enhancing the application of DRS to low-resource Italian Natural Language Processing (NLP), in both semantic parsing (Text-to-DRS) and natural language generation (DRS-to-Text). To address the scarcity of annotated corpora for Italian DRS, we propose a novel data augmentation technique that involves the use of external linguistic resources including: (i) WordNet for common nouns, adjectives, adverbs, and verbs; (ii) LLM-generated named entities for proper nouns; and (iii) rule-based algorithms for tense augmentation. This approach not only increases the quantity of training data but also introduces linguistic diversity, which is crucial for improving model performance and robustness. Using this augmented dataset, we developed neural semantic parser and generator models that demonstrated enhanced generalization ability compared to models trained on non-augmented data. We evaluated the effect of semantic data augmentation using two state-of-the-art transformer-based neural sequence-to-sequence models, i.e., byT5 and IT5. Our implementation shows promising results for Italian semantic processing. Data augmentation significantly increased the performance of semantic parsing from 76.10 to 90.56 (+14.46%) F1-SMATCH score and generation with 37.79 to 57.48 (+19.69%) BLEU, 30.83 to 40.95 (+10.12%) METEOR, 81.66 to 90.97 (+9.31%) COMET, 54.84 to 70.88 (+16.04%) chrF, and 88.86 to 92.97 (+4.11%) BERT scores. These results demonstrate the effectiveness of our novel augmentation approach in enhancing semantic processing capabilities for low-resource languages like Italian.

Introduction
The field of Natural Language Processing (NLP) has seen significant advancements in recent years, particularly in semantic processing tasks. These tasks, which include semantic parsing and natural language generation, often rely heavily on parallel corpora --- datasets that align text in one language with its semantic representation or with text in another language AUTHOR. For languages with rich linguistic resources, such as English, the availability of large-scale parallel corpora has facilitated rapid progress in semantic processing AUTHOR. However, for many languages, including Italian, the scarcity of such resources poses a significant challenge to advancing semantic NLP capabilities AUTHOR. Italian presents unique challenges and opportunities. %in the domain of NLP. While Italian shares some structural similarities with English, it possesses distinct linguistic features that complicate NLP tasks. These include a more flexible word order, a rich system of verb conjugations, and the presence of grammatical gender for nouns, adjectives, and articles. In the context of NLP and Natural Language Generation (NLG), Italian has seen moderate progress. However, compared to high-resource languages like English, Italian still lacks extensive task-specific datasets, particularly in areas requiring deep semantic understanding. This deficiency is especially pronounced in tasks involving formal semantic representations such as Discourse Representation Structures (DRS) AUTHOR. While Italian is not typically classified as a low-resource language in general NLP terms, it can be considered as such in the specific domain of semantic processing, especially when dealing with formal semantic representations. This status is characterized by: (i) Named Entities: Italian naming conventions differ from those in English, requiring adaptation in entity recognition tasks; (ii) Syntactic Structure: Although Italian follows the SVO structure like English, it allows for greater flexibility, posing challenges, especially in parsing tasks; (iii) Grammatical Gender: The presence of grammatical gender in Italian adds complexity to tasks such as coreference resolution and agreement in the generated text. These linguistic features, combined with the limited availability of semantically annotated corpora, position Italian as a challenging language for advanced semantic NLP tasks. [ht] \centering \includegraphics[width=0.50\textwidth]{drs-shapes.pdf Data augmentation (DA), a technique widely used in machine learning to increase the size and diversity of training datasets, has shown promise in addressing resource scarcity in NLP AUTHOR. For semantic tasks involving DRS, DA presents unique challenges due to the need to preserve semantic equivalence while introducing linguistic variety. In the context of Italian semantic processing, traditional augmentation techniques such as random word insertion, deletion, substitutions or back-translation have limited applicability due to the scarcity of Italian-specific semantic resources AUTHOR. This necessitates innovative approaches that can leverage resources from high-resource languages while maintaining the integrity of Italian linguistic structures. Given the challenges outlined, this study aims to develop a novel cross-lingual DA technique for Italian, specifically tailored for DRS-based semantic parsing and generation tasks. {While word substitution techniques are established in DA literature, our approach introduces an innovative cross-lingual framework that leverages the language-neutral nature of DRS. The method uniquely bridges the resource gap between high-resource and low-resource languages by temporarily transforming Italian examples into English, enabling access to rich lexical resources like WordNet, before converting back to Italian. This cross-lingual approach leverages the universal semantic representations of the DRS to enable more advanced data transformation approaches than Italian resources alone would allow, which is particularly advantageous given the limited availability of Italian-specific semantic datasets (see Table~tab:dataset-split for Italian examples). This paper makes the following key contributions: \item A novel cross-lingual augmentation methodology that leverages English WordNet to enhance Italian semantic datasets. \item Empirical evidence demonstrating the effectiveness of this augmentation technique in improving performance scores for both DRS parsing and generation tasks in Italian. \item A detailed analysis of how cross-lingual augmentation affects the handling of Italian-specific linguistic features in semantic processing. \item Insights into the scalability and potential applications of this approach to other low-resource languages in the domain of semantic NLP. The remaining paper is organized as follows: Section~sec:background provides an overview of DRS. %related work in cross-lingual DA and semantic processing for low-resource languages. Section~sec:ita-data-aug details semantic DA for Italian with a focus on named entities, lexical, and grammatical data transformation techniques. Section~sec:experimentation presents our experimental implementation, implications of our results and findings, and their broader impact on the field. Finally, Section~sec:conclusion concludes the paper, addresses certain limitations, and outlines directions for future research. %and Related Work

Conclusion
This study has successfully developed and evaluated a novel cross-lingual DA technique for Italian, specifically tailored for DRS-based semantic parsing and generation tasks. Our research has made significant improvements in addressing the challenges faced by low-resource languages in advanced NLP tasks. The proposed augmentation methodology, leveraging English WordNet to enhance Italian semantic datasets, has demonstrated remarkable effectiveness. Empirical evidence shows substantial improvements in performance scores for both DRS parsing and generation tasks in Italian. Notably, our approach achieved a 90.56% SMATCH F1 score for parsing and significant enhancements across all generation metrics (BLEU: 57.48, METEOR: 40.95, COMET: 90.97, chrF: 70.88, BERT-Score: 92.97) on the G+S+C dataset, surpassing both baseline models and previous state-of-the-art results. Our detailed analysis reveals that data augmentation positively affects the handling of Italian-specific linguistic features in semantic processing. The improvements observed across various augmentation strategies indicate enhanced capability in managing syntactic flexibility and grammatical nuances in Italian. This suggests a successful transfer of semantic knowledge through the lens of Italian DRS. \\ \\ \noindent Limitations:\\ Despite our results approach the performance metrics of English---a rich resource language, there remains a gap that future research could address. {For example, the original sentence ``Tom è piuttosto scarso a tennis.'' (``Tom is rather poor at tennis.'') becomes ``Bob era piuttosto ricco con i single.'' (``Bob was sort of rich at singles.'') While this method introduces linguistic diversity, it can result in less coherent sentences in some cases, as seen in this example. Such limitations are common with cross-lingual augmentation strategies through back-and-forth language translations, which focus on lexical variation over syntactic coherence. Future refinement, such as filtering improbable substitutions or adding human validation, could help ensure more consistent logicality in cross-lingual semantic tasks. We thank ``High-Performance Computing for Artificial Intelligence (HPC4AI) at the University of Turin'' for providing GPU support AUTHOR. \appendix ~\newpage