Mapping Sentiments: A Journey into Low-Resource L uxembourgish Analysis (2024)

Abstract: Sentiment analysis (SA) plays a vital role in interpreting human opinions across different languages, especially in contexts like social media, product reviews, and other user-generated content. This study focuses on Luxembourgish, a low-resource language critical to Luxembourg’s identity, utilizing advanced deep learning models such as BERT, RoBERTa, LuxemBERTand LuxGPT-2. These models were enhanced with transfer learning, active learning strategies, and context-aware embeddings, enabling effective Luxembourgish processing. These models further improved with context-aware embeddings and were able to accurately detect sentiments, categorizing news comments into positive, negative, and neutral sentiments. Our approach highlights the significant role of human-in-the-loop (HITL) methodologies, which refine model accuracy by aligning automated analyses with human judgment. The findings indicate that LuxembBERT, especially when enhanced with the HITL method involving feedback from 500 and 1000 annotated sentences, outperforms other models in both binary (positive vs. negative) and multi-class (positive, neutral, and negative) classification tasks. The HITL approach not only refined model accuracy but also provided substantial improvements in understanding and processing sentiments and sarcasm, often challenging for automated systems. This study establishes the basis for future research to extend these methodologies to other underresourced languages, promising improvements in Natural Language Processing (NLP) applications across diverse linguistic landscapes.

## introduction sentiment analysis (sa), a key branch of nlp, automates the extraction of opinions, emotions, and attitudes from texts concerning various entities such as products and organizations @xcite . emerging in the early 2000s and also referred to as opinion mining or sentiment mining, this field primarily aims to classify texts as positive, negative, or neutral @xcite . specialized forms detect whether texts are hateful or offensive @xcite , and address social issues such as racism using data from social media @xcite . given the significant influence of social media on political elections and marketing, sa has become critically important for businesses and governments @xcite . in the area of sa, researchers have explored a range of methods, including both supervised and unsupervised techniques, all showing promising results (e.g., @xcite ). early studies indicate that unsupervised models, which use sentiment dictionaries, grammatical analysis, and sentence structure patterns with manually created rules, can perform just as well as traditional supervised methods, such as support vector machines (svms) and naïve bayes classifiers @xcite . this exploration sets the stage for addressing the impact of data scarcity in low-resource languages. the significant challenges of applying advanced sa techniques become evident in the context of low-resource languages like luxembourgish. despite the superior performance of dl models over traditional methods, their dependency on extensive labeled datasets remains a major hurdle, given the high costs and time required for data annotation @xcite . to mitigate these challenges, methods such as transfer learning and active learning have been introduced to offer viable solutions (e.g., @xcite ). consider luxembourg, a trilingual nation of over 590,000 residents, home to the largest population of luxembourgish speakers. recognized as the national language in 1984, luxembourgish is integral to the nation's identity and essential for communication within the country. originally a central franconian dialect, luxembourgish has evolved into an independent language, becoming essential for all forms of communication within the country. in contexts where all participants are fluent, switching to french or german is generally avoided @xcite . the term "low-resource language" refers to languages that lack substantial annotated or digital data @xcite . in the nlp field, there has been a significant increase in methods targeting these lowresource languages, broadening the applicability of language models to a more diverse set of languages. traditional static word embeddings do not capture contextual variations that influence meaning, which is essential for accurately understanding and analyzing language. to address this limitation, contextaware embeddings like bert (bidirectional encoder representations from transformers) @xcite , robustly optimized bert approach (roberta) @xcite , and gpt have been developed. these models offer dynamic, contextual representations that adapt based on the surrounding text, thus providing a more precise reflection of subtle sentiment expressions within texts @xcite . by understanding the specific context in which words are used, these advanced models significantly enhance the accuracy of sa, making them indispensable in extracting true sentiment from complex language constructions, such as irony or sarcasm, commonly found in social media and other digital communications. further, human-in-the-loop (hitl) is particularly valuable as it allows systems to adjust to real-world variables and user-specific needs that may not be fully anticipated at the time of a model's initial training. for instance, in sa, hitl can be instrumental in refining the understanding and classification of language used in different contexts, such as irony or cultural-specific expressions that automated systems might misinterpret @xcite . this approach not only enhances the performance and trustworthiness of ai systems but also enables them to become more aligned with human values and ethics, a critical consideration as ai becomes more pervasive in everyday life @xcite . ## results and discussion in binary classification tasks, roberta outperforms bert, achieving higher accuracy (59% vs. 57%), precision (66% vs. 60%), and recall (65% vs. 62%). this reflects roberta's ability to efficiently differentiate between positive and negative classes. luxembert, while showing comparable recall, experiences a slight drop in precision and accuracy, indicating sensitivity due to its architectural differences. luxgpt-2, with significantly lower scores in accuracy (49%) and precision (59%), suggests potential configuration deficiencies for binary tasks. the multi-class classification results reveal that luxembert excels, particularly in a multilingual setting, achieving the highest metrics across accuracy (67%), precision (73%), and recall (72%). this indicates its ability to handle more complex label distinctions. roberta maintains robust performance, although it slightly lags behind luxem-bert, highlighting the subtle differences in its processing capabilities. in contrast, luxgpt-2 shows considerable underperformance in this scenario, indicating that enhancements may be necessary to improve its adaptability to multi-class contexts. the iterative training process of luxembert with human annotations under the hitl methodology demonstrates significant improvements, with accuracy increasing to 70% and 75% with 500 and then 1000 sentences, respectively. this gradual improvement underscores the value of integrating human feedback into the training process, enhancing both the accuracy and reliability of the model. following the incorporation of human feedback, significant performance improvements were observed across the models. detailed metrics such as accuracy, precision, and recall for each model iteration are summarized in as detailed in one of the primary challenges was the scalability of human annotations, as recruiting enough native speakers trained in linguists was time-consuming and costly. despite efforts to mitigate bias, the dominance of certain dialects within the annotated data occasionally skewed the model's performance on regional variations of luxembourgish. our findings emphasize the role of hitl methodologies in enhancing the performance of sa models for low-resource languages, with potential applications in content moderation, customer service bots, and social media analytics for improved accuracy and cultural relevance @xcite . integrating hitl-enhanced models with multilingual platforms like google translate or customer relationship management (crm) systems could further enhance adaptability and accuracy across different languages and dialects. this approach not only improves technological inclusivity for low-resource language speakers but also underscores the need for ethical considerations in handling sensitive sentiment data. the broader impact of our research extends to enhancing the digital inclusion of minority language speakers by developing technology that accurately understands and processes their language. however, as we collect and use sensitive sentiment data, it is imperative to adhere to stringent data privacy laws and ethical guidelines to protect individual privacy and prevent biases that could inadvertently arise from data misinterpretation. despite bert's good cross-lingual performance on high-resource languages, it struggles with low-resource languages, indicating a need for more efficient pretraining techniques or more data @xcite . ## limitations and challenges a limitation of our study is the dependency on a sufficient number of trained human annotators, which poses scalability challenges. this dependency could limit the application of our methods in larger-scale environments or in cases where such resources are scarce. furthermore, the iterative training process, while effective, requires substantial computational resources, which may not be feasible in all application scenarios. future research should, therefore, aim to optimize these processes to balance accuracy with operational efficiency better. ## conclusion and future work this study introduces a method that can be useful for low-resource languages by adopting an existing model to a new context. training a new model for a specific language can be expensive and timeconsuming, taking days or even weeks, depending on the available computing power. in our research, we focused on luxembourgish, a language considered resource-scarce, comparing various bert-based models alongside a hitl strategy. to address the scarcity of data, we implemented hitl strategy, which demonstrated improvements in sa of news comments. while not all models showed statistical significance, the hitl approach on luxembert model consistently outperforms the bert-based alternatives. future work should consider integrating semi-supervised learning techniques to better use unlabeled data. this approach could potentially expand the model's training dataset without requiring extensive human annotation. additionally, applying the hitl methodology to other under-resourced languages could provide insights into the generalizability of this method across diverse linguistic landscapes.