The Best of both Worlds: Dual Channel Language modeling for Hope Speech Detection in low-resourced K annada (2022)

Abstract: In recent years, various methods have been developed to control the spread of negativity by removing profane, aggressive, and offensive comments from social media platforms. There is, however, a scarcity of research focusing on embracing positivity and reinforcing supportive and reassuring content in online forums. As a result, we concentrate our research on developing systems to detect hope speech in code-mixed Kannada. As a result, we present DC-LM, a dual-channel language model that sees hope speech by using the English translations of the code-mixed dataset for additional training. The approach is jointly modelled on both English and code-mixed Kannada to enable effective cross-lingual transfer between the languages. With a weighted F1-score of 0.756, the method outperforms other models. We aim to initiate research in Kannada while encouraging researchers to take a pragmatic approach to inspire positive and supportive online content.

## introduction the last decade has seen a drastic increase in social media users, owing primarily to easier access to the internet as a result of global modernization @xcite . as a result of the surge, several minority groups have turned to social media for support and reassurance. this, however, poses a serious risk to adolescents and young adults who are avid internet users. social media apps like facebook, twitter, and youtube have become an essential part of their daily lives @xcite . certain ethnic groups or individuals are victims of social media manipulation to foster destructive or disruptive behaviour, which is a common scenario in cyberbullying (abaido, 2020). however, these systems ignore potential biases in the dataset on which they are trained and may harm a specific group of social media users, frequently leading to gender/racial discrimination among its users @xcite . as a result, there is a need to detect hope speech in social media. several marginalised groups seek comfort and assistance from social media content that they can relate to and empathise with others' situations @xcite . this type of speech is essential for everyone because it encourages people to improve their quality of life by taking action. hope speech aims to inspire people suffering from depression, loneliness, and stress by providing assurance, reassurance, suggestions, and support @xcite . because most social media in multilingual communities still revolve around english, the phenomenon of codemixing is common. according to studies, codemixing is an essential component of social media in multilingual countries @xcite . kannada (iso 639-3:kan) is one of india's lowresource dravidian languages. dravidian languages are spoken by over 200 million people, mostly in southern india and northern sri lanka @xcite . the language is primarily spoken by people in karnataka, india, and it is also recognised as an official language of the state @xcite . kannada script, also known as catanese, is an alphasyllabary of brahmic scripts that evolved into the kadamba script @xcite . kannada has over 43 million speakers 1 . however, as previously stated, kannada is a language with limited resources due to a lack of language technologies. our work aims to detect hope speech in lowresourced code-mixed languages. we develop models on hope speech detection in low-resourced kannada. we propose that a language model would learn effectively with the help of the parent translations. we make use of translations with google translate api and experiment with several multilingual language models to find the best performing model. we define dual channel language model as a model that uses two translations, namely, codemixed kannada and english. we present dc-lm, (dual-channel language model) based on the architecture of bert that uses the translation of the dataset as additional input for training, performing better in contrast to the typical fine-tuned multilingual bert. we perform a comprehensive analysis of our models on the dataset along with a thorough error analysis on its predictions on the dataset. ## related work researchers have worked on extracting data from social media, particularly from user comments on youtube, facebook, and twitter @xcite . most information extracted from social media does not adhere to grammatical rules and is written in code-mixed, or non-native scripts, as is common among users from multilingual countries @xcite . people can communicate on social media without face-to-face interaction, but they are prone to misunderstandings because they do not consider the perspectives of others. there have been few previous efforts on hope speech identification, with the only dataset contribution being (chakravarthi, 2020), a large multilingual corpus manually annotated for english, tamil, and malayalam, with around 28k, 20k, and 10k comments, respectively. several researchers have worked to promote positivity on social media by developing and analysing systems that filter out malignancy on social media by focusing on very specific events such as crisis and war @xcite , intercountry social media dynamics @xcite , and protests @xcite . the authors conducted a shared task on hope speech detection for comments scraped from youtube in these languages to encourage more research into hope speech for english, @xcite . the organ-isers of the collaborative task used the hopeedi (chakravarthi, 2020) multilingual hope speech dataset. in malayalam @xcite , finetuning a pretrained xlm-roberta model resulted in the best-weighted f1-score of 0.854. in tamil @xcite , an ensemble of synthetically generated code-mixed data for training ulm-fit, baseline-knn, and a fine-tuned roberta achieved the best score of 0.61. the authors fed the combination of pretrained xlm-r and tf-idf vectors as inputs to an inception block, leading to a weighted f1-score of 0.93 @xcite . ## dataset we use the code-mixed kannada hope speech dataset @xcite . the dataset has two labels, namely hope and not-hope. hope speech: ## methodology we perform extensive analysis on the kannada hopespeech dataset using a variety of classifiers, ranging from simple machine learning algorithms to complex deep learning algorithms. to tabulate our results, we employ the scikit-learn library @xcite . we conduct our experiments in the manner described below. we ran an average of 5 runs on each model to tabulate the results. we avoid using stopwords or other lemmatisation techniques because kannada is a morphologically rich language. for machine learning algorithms, we used the scikit-learn library. we used the pytorch implementation of the pretrained language models available on huggingface transformers foot_1 . we finetuned the models on google colaboratory foot_2 for its easier access to gpu resources and user interface. ## results and discussion the results of experiments carried out for classifying hope speech with various models are listed in we use four language models for the dualchannel lm, listed in surprisingly, the monolingual bert (only english) performed worse than some machine learning algorithms in terms of precision, recall, and f1 scores. we believe this is due to the dataset's characteristics. ## conclusion a surge in the active users on social media has inadvertently increased the amount of online content available on social media platforms. there is a need to motivate positivity and hope speech in platforms to instigate compassion and assert reassurance. in this paper, we work on kanhope, a manually annotated code-mixed data of hope speech detection in an under-resourced language, kannada, consisting of 6,176 comments crawled from youtube and propose dc-lm, a dual-channel bert-based model that uses the best of both worlds: codemixed kannada-english and translated english texts. several pretrained multilingual and monolingual language models were analysed to find the best approach that yields a tremendous weighted f1-score. we have also trained the dataset on preliminary machine learning algorithms to baseline for future work on the dataset. we believe that this dataset will expand further research into facilitating positivity and optimism on social media. we have developed several models to serve as a benchmark for this dataset. we aim to promote research in kannada.