dhivya-hope-detection@ LT - EDI - EACL 2021: Multilingual Hope Speech Detection for Code-mixed and Transliterated Texts (2021)

Abstract: In this paper we work with a hope speech detection corpora that includes English, Tamil, and Malayalam datasets. We present a two phase mechanism to detect hope speech. In the first phase we build a classifier to identify the language of the text. In the second phase, we build a classifier to detect hope speech, non hope speech, or not lang labels. Experimental results show that hope speech detection is challenging and there is scope for improvement.

## introduction artificial intelligence models are criticized for their bias against the protected classes @xcite . these biases are shown to arise from data or the model itself. there are several efforts taken to mitigate bias from the data and model perspectives @xcite @xcite . @xcite present a bias alleviation mechanism evaluating the performance following a cross-domain approach. hope speech detection is the task of automatically detecting web content that may play a positive role in diffusing hostility on social media triggered by heightened political tensions during a conflict @xcite . we hypothesize that hope speech detection datasets could be used in evaluating the aforementioned bias alleviation mechanisms. these datasets and mechanisms could help in building ai systems that are diverse and inclusive. additionally, with divisiveness spread across social media platforms, identifying hope speech and enhancing them would help mitigate divisiveness and animosity. english god accepts everyone. hope the tech industry is tough for everyone. non hope casa. la. femmenwest. not eng. tamil g..... semester exam ah pathi video uploade pannunga hope background ெமாக்ைகயா irruku non hope love u mg bye.. bye... take care!! not tam. malayalam surya good. കഴിഞ് ഞ കാലം ഓക് കുന് നതാണ് ഏറ് റവും വലിയ കാര് യം. god bless u hope കമ് മി ആണല് േല?? non hope i am made in india. in this paper, we describe our approach on the hope detection shared task. we work with a multilingual corpora aiming to detect hope speech. we follow a two phase approach to accomplish the task. in the first phase, we identify the language of the input text. in the second phase we classify if the text is hope speech or not. we begin with describing the corpora, then analyze the datasets, explain the experimental setup, and finally discuss the results. ## data we work with the hope speech detection corpora from chakravarthi (2020). unlike most other corpora that target english texts, this corpora focuses on diversity and inclusion including two dravidian languages apart from english. the corpora contains hope speech detection datasets in three languages (i) english, (ii) tamil, and (iii) malayalam. the datasets are generated from youtube comments, and manually labeled for the three labels hope speech, non hope speech, and not lang. the hope speech label and the non hope speech label are self-explanatory. the not lang label indicates that the youtube comment does not belong to the specific language. that is, the datasets include not english, not tamil, and not malayalam instances depending on the language of the dataset they belong. this label becomes important as the tamil and malayalam dataset instances are generated by social media users who are usually bilingual (english and their mother tongue). throughout this paper, we use the label not lang to refer that the text does not belong to the specific language. ## analysis the english dataset includes 28,451 instances (hope: 2,484; non hope: 25,940; not lang: 27), the tamil dataset includes 20,198 instances (hope: 7,899; non hope: 9,816; not lang: 2,483), and the malayalam dataset includes 10,705 instances (hope: 2,052; non hope: 7,765; not lang: 888). find more about the statistics of the corpora in the original paper (chakravarthi, 2020). analyzing the datasets reveal that there are several tamil and malayalam instances that include code-mixed or english transliterated texts. for instance, the non hope example for tamil dataset in figure 1 presents wordclouds for the hope speech instances in the three datasets english, tamil, and malayalam. as we can see, the english hope wordcloud (left) has hopeful words like good, teach, etc. interstingly, the tamil wordcloud (center) is filled with english words and tamil words translitered to english like pathukonga, neenga, etc. tamil social media users tend to use code-mixed (tamil and english) and english transliterations of tamil texts, causing this behavior. in case of malayalam, the wordcloud(right) includes both english and malayalam words. figure 2 presents wordclouds for the non hope speech instances in the three datasets english, tamil, and malayalam. unsurprisingly, the wordclouds for non hope speech instances include words with negative connotations. it is to be noted that tamil social media users have more english influence than malayalam social media users, despite both languages being dravidian. ## experiments as the number of not lang labels are considerably less than (english: 0.1%, tamil: 12%, malayalam: 11%) the other two labels, we specifically target to identify not lang labels by building a dedicated classifier. thus we follow a two-phase approach to detect hope speech. we argue a two-phase language identification approach might be helpful as the datasets include a not lang label despite corresponding to a specific language. in phase 1, we identify the language of the text using a language detector. in phase 2, we use the results from phase 1 in addition to other features, to identify hope speech or not using a hope detector. the architecture is described in figure 1 . ## results we present test results for english, tamil, and malayalam datasets in regarding the tamil dataset, the performance on all the three labels are comparable (f1: .52 vs. .66 vs. .54). note that the tamil dataset includes several code-mixed and transliterated texts. this phenomena can be attributed to why the classifier struggles in identifying the correct label. regarding the malayalam dataset the performance of the label hope speech is worse than the others. additionally, it is relatively easier to predict not lang labels in malayalam. the experimental results show detecting hope speech is difficult regardless of the language. even in the english dataset where there are no transliterations or code-mixing, the classifier struggles. while we infer that hope detection is a difficult task, code-mixing and transliterations in tamil and malayalam increases the complexity of the problem. chakravarthi and muralidaran (2021) describe the results and techniques from the other participants of the hope speech detection shared task. ## conclusion this paper describes the shared task on hope speech detection it targets to detect hope speech from a multilingual corpora. the corpora includes datasets in three languages english, tamil, and malayalam. first we conduct an analysis over the corpora finding codemix and transliterated texts in the tamil and malayalam datasets. next, we build a two phase mechanism to identify hope speech. in the first phase we detect the language of the text. in the next phase, we classify the text into hope speech, non hope speech, or not lang. finally, we discuss the results concluding that hope detection is a challenging task.