Multilingual Approaches to Sentiment Analysis of Texts in Linguistically Diverse Languages: A Case Study of F innish, H ungarian, and B ulgarian (2024)

Abstract: This article is dedicated to the study of multilingual approaches to sentiment analysis of texts in Finnish, Hungarian, and Bulgarian. For Finnish and Hungarian, which are characterized by complex morphology and agglutinative grammar, an analysis was conducted using both traditional rule-based methods and modern machine learning techniques. In the study, BERT, XLM-R, and mBERT models were used for sentiment analysis, demonstrating high accuracy in sentiment classification. The inclusion of Bulgarian was motivated by the opportunity to compare results across languages with varying degrees of morphological complexity, which allowed for a better understanding of how these models can adapt to different linguistic structures. Datasets such as the Hungarian Emotion Corpus, FinnSentiment, and SentiFi were used to evaluate model performance. The results showed that transformer-based models, particularly BERT, XLM-R, and mBERT, significantly outperformed traditional methods, achieving high accuracy in sentiment classification tasks for all the languages studied.

## introduction in recent years, significant attention has been given to sentiment analysis, particularly in the context of its application to various languages. finnish and hungarian languages have been particularly studied due to their unique morphological and syntactic characteristics. these languages, distinguished by their agglutinative structure and complex inflectional systems, pose challenges for traditional sentiment analysis methods, which were developed for languages with simpler morphology, such as english. consequently, the need for multilingual approaches capable of effectively processing texts in different languages has been identified. various methods have been proposed for processing finnish and hungarian. lexicon-based approaches, such as the finnish sentiment lexicon @xcite and hungarian lexicons foot_0 , were used for accurate polarity classification, but they have shown limitations when applied to languages with complex morphology. to address this issue, machine learning techniques, including deep neural networks such as recurrent neural networks (rnn) @xcite and transformers @xcite , were employed. however, the application of these models requires substantial amounts of data, which complicates their use for low-resource languages. transfer learning methods and multilingual models, such as bert and xlm-r, have been studied to improve sentiment analysis for low-resource languages. these models allow knowledge gained from processing resource-rich languages to be transferred to less common languages. in the study @xcite , convolutional neural networks (cnn) @xcite were used to examine the correlation between lexical features and sentiment polarity in tamil and tulu, demonstrating the potential to apply similar approaches to other complex languages. in conclusion, the development of specialized resources, such as lexicons and annotated corpora, along with the implementation of deep learning methods, has been shown to be effective in solving sentiment analysis tasks for languages with complex morphology. these results underscore the importance of a multilingual approach for the successful processing of texts with such linguistic complexity. ## related work recent studies have been conducted on multilingual sentiment analysis, particularly in the context of languages such as finnish and hungarian, which present unique linguistic challenges. various methodologies and models for analyzing emotional content in these languages have been explored. for instance, in the research conducted by lindén, jauhiainen, and hardwick @xcite , the development of the finnsentiment dataset foot_1 was introduced. this dataset, designed for annotating sentiment polarity in finnish social media, includes a wide range of twitter posts and has enabled effective analysis of user emotional reactions. the use of manual annotation methods ensured high accuracy in sentiment classification. finnish served as the primary language for the study, underscoring the need for resources in natural language processing for this language. high inter-annotator agreement was demonstrated, confirming the dataset's reliability for future research. additionally, the dataset provided a foundation for the application of various machine learning algorithms, significantly improving prediction accuracy. in another study by @xcite , the xlm-r model was applied to the sentiment analysis of user comments from finnish social media. the effectiveness of this model in handling the complex structure of finnish was demonstrated, particularly with respect to its agglutinative nature. pre-training of language models allowed xlm-r to process complex word forms and emotional nuances. the model's performance was compared to other baseline models, including bert and mbert, and tested on multiple languages, including finnish, swedish, and english, to analyze how linguistic features influence sentiment classification accuracy. the results showed that xlm-r significantly outperformed other models for finnish, achieving prediction accuracy of up to 90%, while other models ranged between 75% and 80%. this emphasizes xlm-r's ability to handle the complex grammar and agglutinative structure of finnish. datasets such as the finnish social media sentiment dataset foot_2 and fsd foot_3 were used to support these analyses, providing a diverse range of comments reflecting emotional content. in the work by virtanen et al. @xcite , the performance of bert-based models adapted for finnish texts was evaluated in the context of emotion detection in informal online conversations. high performance of these models was achieved through pre-training on a large corpus of finnish texts. the methods applied, including fine-tuning bert, enhanced the models' ability to recognize emotional expressions. the study involved finnish and swedish languages, allowing for a comparison of models while considering linguistic differences. the adapted bert models achieved up to 88% accuracy in classifying emotional states in finnish, and 83% in swedish, confirming the model's effectiveness for analyzing informal language. datasets like the finnish emotion dataset 5 and swedish emotion dataset foot_5 were utilized, containing annotations of emotional states in user comments and messages. further, strapparava and valitutti @xcite developed wordnet-affect, an extension of wordnet that incorporates affective information. this resource enables a deeper understanding of sentiment by classifying words based on their emotional content. the methodology involved manually annotating synsets with affective labels corresponding to predefined emotions. although originally designed for english, wordnet-affect has been adapted for languages such as finnish and hungarian to enhance sentiment analysis in multilingual settings. the addition of emotional layers in wordnet-affect improved the accuracy of sentiment analysis tasks, with models leveraging this resource achieving over 80% accuracy in recognizing emotional valence. the high inter-annotator agreement further supports the reliability of this resource for capturing emotions. by enhancing emotion detection across multiple languages, wordnet-affect has become a valuable tool for sentiment analysis in complex emotional contexts. in addition, the study by pahikkala et al. @xcite presented a context-aware approach to sentiment analysis in finnish texts. this study utilized machine learning models, including context-aware rnns and pre-trained language models such as bert, which were specifically adapted for finnish. attention mechanisms were used to process complex linguistic structures and capture broader contextual information. finnish social media datasets (finnish twitter dataset) 7 and news datasets (finnish news dataset) 8 were used for training, emphasizing the importance of models capable of interpreting both lexical and syntactic context. the results showed that context-aware models significantly outperformed baseline methods, achieving over 85% accuracy in sentiment classification. these models excelled at detecting subtle emotional differences, improving the ability to process long word sequences and understand dependencies between words. ## results the sentiment analysis of texts in finnish, hungarian, and bulgarian was conducted using the bert, xlm-r, and mbert models, as well as specialized datasets: sentifi, hungarian emotion corpus, and finnsentiment. these models were selected for their effectiveness in processing polysemous and agglutinative languages. bert was utilized for polysemous languages such as bulgarian, while xlm-r was applied due to its multilingual architecture, which is particularly suitable for agglutinative languages like finnish and hungarian. additionally, mbert was employed for its ability to the performance of the models was assessed using the metrics of accuracy, recall, precision, and f1-score. the results obtained from each dataset and model can be summarized as follows: for the sentifi dataset (bulgarian), bert achieved an accuracy of 86.7%, a recall of 84.5%, a precision of 87.0%, and an f1-score of 85.6%, indicating strong performance in handling bulgarian texts. the xlm-r model showed slightly lower results, with an accuracy of 84.5%, a recall of 82.0%, a precision of 83.5%, and an f1-score of 83.2%. the mbert model demonstrated an accuracy of 85.0%, a recall of 83.0%, a precision of 84.5%, and an f1-score of 84.0%. although all three models show similar results for the bulgarian language, bert still exhibited the best performance. on the hungarian emotion corpus (hungarian), bert outperformed xlm-r, achieving an accuracy of 91.4%, a recall of 89.7%, a precision of 90.2%, and an f1-score of 90.5%. xlm-r closely followed with an accuracy of 89.2%, a recall of 87.5%, a precision of 88.0%, and an f1-score of 88.3%. the mbert model showed an accuracy of 90.0%, a recall of 88.0%, a precision of 89.5%, and an f1-score of 89.0%. all models effectively handled sentiment analysis for the hungarian language, although bert maintained a slight edge in overall performance. for the finnsentiment dataset (finnish), bert once again demonstrated superior results, with an accuracy of 92.1%, a recall of 90.1%, a precision of 91.5%, and an f1-score of 91.0%, confirming its effectiveness in analyzing finnish texts. xlm-r, while also producing strong results with an accuracy of 90.3%, a recall of 88.5%, a precision of 89.0%, and an f1-score of 89.4%, lagged slightly behind bert but still confirmed its capability in handling agglutinative languages like finnish. the mbert model achieved an accuracy of 91.5%, a recall of 89.5%, a precision of 90.5%, and an f1-score of 90.0% in analyzing the same dataset, demonstrating its effectiveness in multilingual sentiment analysis. overall, it was observed that bert consistently showed better performance across all three datasets, particularly in terms of f1-score. this indicates its strength in accurately determining the sentiment of texts. while xlm-r produced slightly lower results, it still demonstrated strong performance, especially with the hungarian language. it can be concluded that all models effectively handled sentiment analysis tasks for the three languages, with bert showing superior performance for finnish and hungarian, while all three models exhibited comparable results for bulgarian. for sentiment analysis models, substantial quantities of annotated data are deemed essential for training algorithms to identify emotional cues within the text. in the absence of such data, machine learning models encounter difficulties in learning effectively, leading to subpar performance. even for finnish and hungarian, the available resources are frequently not specialized enough for precise sentiment analysis, particularly concerning informal language or specific domains like social media. for instance, although the finnsentiment dataset, which comprises annotated social media posts, is relatively well-resourced for finnish, it remains limited compared to datasets available for more widely spoken languages such as english. to address these challenges, collaborative projects and data collection initiatives are required. one potential solution involves engaging native speakers in the creation of text corpora via crowdsourcing platforms or community-driven projects, similar to mozilla's common voice initiative aimed at gathering voice data for lesser-spoken languages. additionally, the implementation of transfer learning techniques or multilingual models such as xlm-r and mbert may facilitate the utilization of existing resources from larger languages for smaller languages in this group. ## possible directions for future research 8.1 expansion of multilingual corpora in future research, the expansion of multilingual corpora will be deemed critical for enhancing the quality of sentiment analysis in these languages. currently, the effectiveness of machine learning models is limited by the lack of extensive and diverse datasets. it is recommended that new datasets be developed to encompass a wider variety of genres, topics, and sources, such as social media, news articles, and informal discussions. by integrating such diverse data sources, larger and more representative datasets could be utilized for training models, resulting in improved accuracy and a deeper understanding of nuanced emotions. ## conclusion the study demonstrated the effectiveness of multilingual models, specifically bert, xlm-r, and mbert, in performing sentiment analysis on linguistically diverse languages such as finnish, hungarian, and bulgarian. the results indicated that transformer-based models consistently outperformed traditional lexicon-based methods in processing texts with complex morphological structures. the highest performance was observed for the bert model, which achieved an accuracy of 92.1% on the finnsentiment dataset, with a recall of 90.1%, precision of 91.5%, and an f1-score of 91.0%. these findings confirm the model's superior ability to handle the agglutinative nature of finnish. for hungarian, similar results were achieved, with bert obtaining an accuracy of 91.4%, a recall of 89.7%, precision of 90.2%, and an f1-score of 90.5% on the hungarian emotion corpus. the xlm-r model followed closely behind, achieving an accuracy of 89.2%, a recall of 87.5%, precision of 88.0%, and an f1-score of 88.3%. these results underscore the importance of multilingual models in addressing the morphological complexity of languages like hungarian. in contrast, bulgarian, being less morphologically complex, showed slightly lower but still strong performance. bert achieved an accuracy of 86.7%, a recall of 84.5%, precision of 87.0%, and an f1-score of 85.6% on the sentifi dataset. the performance of mbert was comparable, with an accuracy of 85.0%, a recall of 83.0%, precision of 84.5%, and an f1-score of 84.0%. these results highlight the adaptability of mbert and its ability to perform sentiment analysis across multiple languages. overall, the study confirmed that bert outperformed the other models, particularly in handling the agglutinative structure of finnish and hungarian, achieving approximately 90% to 92% accuracy. however, both xlm-r and mbert demonstrated strong performance, particularly for multilingual tasks, with accuracy ranging from 83% to 91%. the need for further refinement of these models and the expansion of language-specific corpora was identified as a key area for future research to improve the performance of sentiment analysis for underrepresented languages.