COVID -19-related N epali Tweets Classification in a Low Resource Setting (2022)

Abstract: Billions of people across the globe have been using social media platforms in their local languages to voice their opinions about the various topics related to the COVID-19 pandemic. Several organizations, including the World Health Organization, have developed automated social media analysis tools that classify COVID-19-related tweets to various topics. However, these tools that help combat the pandemic are limited to very few languages, making several countries unable to take their benefit. While multi-lingual or low-resource language-specific tools are being developed, there is still a need to expand their coverage, such as for the Nepali language. In this paper, we identify the eight most common COVID-19 discussion topics among the Twitter community using the Nepali language, set up an online platform to automatically gather Nepali tweets containing the COVID-19-related keywords, classify the tweets into the eight topics, and visualize the results across the period in a web-based dashboard. We compare the performance of two state-of-the-art multi-lingual language models for Nepali tweet classification, one generic (mBERT) and the other Nepali language family-specific model (MuRIL). Our results show that the models’ relative performance depends on the data size, with MuRIL doing better for a larger dataset. The annotated data, models, and the web-based dashboard are open-sourced at https://github.com/naamiinepal/covid-tweet-classification .

## introduction the covid-19 pandemic has caused a global rise in social media users who express their opinions and share information on various topics related to the pandemic. public health organizations and relevant agencies could analyze the social media data for early warning on potentially new virus variants based on symptoms discussion, for understanding the impact of various intervention measures, the efficacy of vaccination programs, etc. social media data analysis can help develop strategies for combating the pandemic @xcite , and improve the efficiency of the health industry @xcite @xcite @xcite . several studies performed sentiment analysis of tweets to understand people's views towards the pandemic @xcite @xcite . since sentiment analysis provides limited coarse level information, recently there is an interest in building tools for early warning and topic-level discourse analysis. most notably, the world health organization (who) tracks internet discourse by examining global pandemic-related twitter data and news using tools like covid-19 news map foot_0 and ears foot_1 . although a large fraction of the global population uses local languages in social media, most of these tools are limited to english or anglo-european languages. for instance, the who ears works in only nine languages being piloted in 30 countries. in recent years, there has been a growing interest in building multi-lingual language models, building low-resource language datasets, and exploring nlp methods with smaller language models and smaller data @xcite @xcite . nepali is a low-resource language where there is still a large gap in terms of advances, data availability, and the development of nlp tools. while there has been some work on lowresource languages for sentiment analysis in lowresource languages @xcite including nepali @xcite , to our knowledge there is no work on covid-19 tweet topics classification for discourse analysis in the nepali language. figure 1: the web app dashboard shown above uses infographics viz. bar graphs and line charts to track the trend of various topics. we can filter the tweets based on time and topics to make analysis easier. additionally, we have incorporated humans in the loop by developing an administrator interface to validate the predicted tweet labels from the model and proofread the validated ones. in this work, we propose a new dataset, deep learning classification models based on multilingual language models, and an interactive dashboard for incremental learning and visualization of covid-19 tweets topic classification in the nepali language. figure 1 shows a snapshot of our dashboard. in addition to visualizing topics classification in real-time, the dashboard can be used to manually verify the ml model's prediction, correct the predictions to annotate more data, and re-train the model via gui for improvement as more data becomes available. the followings are our contributions to the scientific community. ## discussion and conclusion we have developed an online platform to gather, analyze, and categorize tweets about covid-19 in nepali, written in the devanagari script. we selected eight topics pertinent to online discussions in nepal based on the various categories of online discourse established by the who. seven different people have annotated 12, 241 tweets from our 10 https://kathmandupost.com/health/2021/09/2 0/all-students-above-18-to-be-jabbed-with-cov id-19-vaccine 11 https://english.onlinekhabar.com/nepal-cov id-19-third-wave-signs.html dataset. we arrived at our best model architecture using muril @xcite as the encoder and a batch normalization @xcite layer immediately before the final output layer after fine-tuning several hyperparameters and utilizing two encoder backbones, i.e., mbert @xcite and muril @xcite . the web app dashboard uses infographics like bar graphs and area charts to track the development of online discussions. we can filter the tweets based on time and topics to make analysis easier. additionally, we developed an administrator interface to validate the predicted tweet labels from the model and proofread the validated ones. although our dataset is not large, it may be valuable for transfer learning and semi-supervised learning @xcite . our dataset can help to make multi-lingual datasets more inclusive and the models trained on them more robust. translating and transliterating to and from our dataset can help in augmentation in various settings.