IRNLP _ DAIICT @ LT - EDI - EACL 2021: Hope Speech detection in Code Mixed text using TF - IDF Char N-grams and M u RIL (2021)

Abstract: This paper presents the participation of the IRNLP_DAIICT team from Information Retrieval and Natural Language Processing lab at DA-IICT, India in LT-EDI@EACL2021 Hope Speech Detection task. The aim of this shared task is to identify hope speech from a code-mixed data-set of YouTube comments. The task is to classify comments into Hope Speech, Non Hope speech or Not in language, for three languages: English, Malayalam-English and Tamil-English. We use TF-IDF character n-grams and pretrained MuRIL embeddings for text representation and Logistic Regression and Linear SVM for classification. Our best approach achieved second, eighth and fifth rank with weighted F1 score of 0.92, 0.75 and 0.57 in English, Malayalam-English and Tamil-English on test dataset respectively

## introduction hope can defined as a belief that the current situation would change for the better. it is vital for everyone since it helps in continuing our efforts even in difficult and unfavourable circumstances. it also encourages us to continuously improve our lives by thinking of a better future and take actions to achieve it. in recent years social media has become one of the important aspect of our lives. people convey their opinions openly on social media on various topics. understanding and analysing these opinions through natural language processing techniques has been an active research area. offensive content detection and classification on social media has been studied extensively. hence, research should also focus on identifying positive online content that is encouraging and supportive. thus identifying hope speech in social media is an important task to gauge the opinion of people in tough times like covid-19. the goal of this shared task is to identify hope speech from a code-mixed dataset of comments of dravidian languages collected from youtube. shared task was introduced as three class classification of the youtube comments into hope speech, non hope speech or not in language, for three languages: english, malayalam-english and tamil-english. shared task organizers define hope speech as a text that offers support, reassurance, suggestions, inspiration and insight specifically in youtube comments. the shared task focuses on hope speech for women in stem, lgbtiq individuals, racial minorities or people with disabilities in general for equality, diversity and inclusion @xcite . our approaches consists of tf-idf character n-grams and muril embeddings for the text representation and logistic regression and linear svm for classification. the remainder of this paper is organized as follows: the next section includes related work followed by section 4 which describes the shared task dataset and methods are presented in section 4. results and analysis is given in final section 5 and section 6 present conclusion. ## related work social media content analysis is an active research area with tasks like hate speech detection, offensive language detection, etc. some of the shared tasks organised recently for these are hasoc track at fire foot_1 2019, 2020 @xcite and offenseval 2019, 2020 @xcite val @xcite were pretrained embeddings like bert @xcite , roberta @xcite and elmo @xcite . best performing methods in hasoc @xcite track use multilingual transformer based methods like xlm-roberta, mbert, etc and fine tune them for the task. tf-idf along with character n-grams and machine learning classifiers like logistic regression, svm and xgboost also performed well and equivalent to deep learning classifiers in some of the tasks. hope speech detection shared task focuses on positive instead of negative comments/posts in social media. hope speech has been to proven to be useful @xcite for saving people from self harm and suicide. it has inspired people to demand rights for equality, diversity and inclusion (chakravarthi, 2020). ## dataset hope speech detection shared task organizers provide datasets in three languages english, malayalam-english and tamil-english (chakravarthi and muralidaran, 2021). dataset has been curated from youtube comments that have been collected are pertaining to women in stem, lgbtq, covid-19 and black lives matters topics, using the youtube comment scraper foot_2 . it contains 28451 comments in english, 20198 in tamil and 10705 in malayalam. full statistic of dataset given in ## methods for all the youtube comments we first preprocess the text and then create a text representation and finally classify the text using the machine learning classifiers. figure 1 illustrates the set of steps used to classify the youtube comments. ## results and analysis the submission evaluation on the test data of all the three languages is shown in tables 2, 3, 4. it can be seen from the tables that our best method have been able to beat the baseline in all the three languages. our methods achieve second, eighth and fifth rank in english, malayalam and tamil respectively. for english and tamil language, tf-idf (char) + lr achieves the best results with weighted f1 score of 0.92 and 0.57 respectively. for the malayalam language task, tf-idf (char) + svm achieves the best weighted f1 score of 0.75. it can be observed that muril is not performing good particularly in case of tamil language. in all the languages tf-idf character n-grams representation performed better than muril. although the training data for tamil is higher and also balanced as compared to malayalam, the weighted f1 ## conclusion and future work the details of our submission in hope speech detection task have been presented in the paper. by exploring tf-idf character n-grams and muril to represent the text, we conclude that the former method is consistently performing better in all the three languages. the weighted f1 score for tamil language is relatively lesser as compared to english and malayalam which leads us to conclude that it is a difficult to identify in hope speech in tamil. we have not explored deep learning based architectures like cnn, lstm and transformers for classification of hope speech. these approaches could be a future direction for the researchers to improve the results.