MUCS @ LT - EDI - EACL 2021: C o H ope-Hope Speech Detection for Equality, Diversity, and Inclusion in Code-Mixed Texts (2021)

Abstract: This paper describes the models submitted by the team MUCS for “Hope Speech Detection for Equality, Diversity, and Inclusion-EACL 2021” shared task that aims at classifying a comment / post in English and code-mixed texts in two language pairs, namely, Tamil-English (Ta-En) and Malayalam-English (Ma-En) into one of the three predefined categories, namely, “Hope_speech”, “Non_hope_speech”, and “other_languages”. Three models namely, CoHope-ML, CoHope-NN, and CoHope-TL based on Ensemble of classifiers, Keras Neural Network (NN) and BiLSTM with Conv1d model respectively are proposed for the shared task. CoHope-ML, CoHope-NN models are trained on a feature set comprised of char sequences extracted from sentences combined with words for Ma-En and Ta-En code-mixed texts and a combination of word and char ngrams along with syntactic word ngrams for English text. CoHope-TL model consists of three major parts: training tokenizer, BERT Language Model (LM) training and then using pre-trained BERT LM as weights in BiLSTM-Conv1d model. Out of three proposed models, CoHope-ML model (best among our models) obtained 1st, 2nd, and 3rd ranks with weighted F1-scores of 0.85, 0.92, and 0.59 for Ma-En, English and Ta-En texts respectively.

## introduction the recent wave of using social media especially during the outbreak of covid-19 has increasingly affected the amount of user-generated data and text over the internet that has provided immense opportunities in automated text analysis and computational linguistics @xcite . most of tools and systems to analyze social media texts are designed to handle them in their native script. however, social media texts are often code-mixed, i.e., written in roman script mixing english words rather than in the native script of language due to difficulty in using tools provided to pen the comments in native script @xcite . further, users may prefer using roman scripts even though the language has its own standardized written form and script @xcite . the analysis of romanized and code-mixed texts is more challenging task compared to analysis of texts in native scripts because of the inconsistent romanization conventions and non-standard grammars in code-mixed texts @xcite . hope speech detection is defined as analysis and detection of inspirational talk and comments/posts with positive vibes, etc. against people with not straight desires such as lesbian, gay, and transgender or positive suggestion for covid-19 guidelines, etc. (chakravarthi, 2020). even though a couple of studies and workshops are focused on analyzing code-mixed texts in tasks such as sentiments analysis (sa) and offensive language identification (oli) it has been rarely experimented on hope speech detection even in native scripts. in this direction, the "hope speech detection for equality, diversity, and inclusion" foot_0 shared task aims at classifying a comment/post in english and code-mixed texts in two language pairs, namely, tamil-english (ta-en) and malayalam-english (ma-en) into one of the three predefined categories, namely, "hope speech", "non hope speech", and "other languages". the details of the datasets provided by organizers are given in @xcite . in this paper, we, team mucs describe the three models cohope-ml, cohope-nn and cohope-tl submitted for "hope speech detection for equality, diversity, and inclusion" shared task. the char sequences extracted from sentences com-bined with words in the sentences are used to train cohope-ml and cohope-nn models for codemixed ma-en and ta-en texts whereas a combination of char and word ngrams along with syntactic ngrams are used to train the same models for english texts. cohope-tl model is comprised of three major steps: (i) training tokenizer, (ii) training bert lm using raw texts from dakshina dataset foot_1 [5], for ma-en and ta-en code mixed texts and pre-trained bert lm from kaggle foot_2 for english, and (iii) transferring obtained weights and building bilstm-conv1d model. the rest of paper is organized as follows: while section 2 describes the recent literature on codemixed text processing, section 3 focuses on the description of the models submitted to the shared task followed by experiments and results in section 4. conclusion and future plans are included in section 5. ## related work researchers have developed a vast range of datasets, tools and models for text classification (tc). however, comparatively very less work has been done on the classification of code-mixed texts and the available literature focus on sa and oli tasks for several languages pairs. hope speech detection is a new challenge that has been explored rarely. some of recent studies on tc tasks for code-mixed texts are given below: ## methodology the proposed models are described in terms of feature engineering to extract the required features followed by description of the classifiers. ## conclusion and future work in this paper we, team mucs, present the description of three proposed models for the task of "hope speech detection for equality, diversity, figure 6: cm for ta-en texts using cohope-ml model figure 7: cm for english texts using cohope-ml model and inclusion-eacl 2021". proposed models includes a ml voting classifier -cohope-ml, a dl nn model -cohope-nn and a tl based model -cohope-tl. the first two models are trained on a combination of char sequences and words for ta-en and ma-en code-mixed texts and combination of traditional char and word ngrams with syntactic word ngrams for english. cohope-tl model utilizes bert lm as weights in a bilstm-conv1d architecture. out of three proposed models, cohope-ml model (best among our models) obtained weighted f1-scores of 0.85, 0.92 and 0.59 and 1, 2, 3 ranks for malayalam-english, english, and tamil-english texts. as future work, we planned to explore syntactic ngrams features for code-mixed texts and improve cohope-nn architecture by experimenting on different nn layers and configurations. we also would like to compare different approaches based on tl for code-mixed texts from low resource languages.