ssn_di BERT sity@ LT - EDI - EACL 2021:Hope Speech Detection on multilingual Y ou T ube comments via transformer based approach (2021)

Abstract: In recent times, there exists an abundance of research to classify abusive and offensive texts focusing on negative comments but only minimal research using the positive reinforcement approach. The task was aimed at classifying texts into ‘Hope_speech’, ‘Non_hope_speech’, and ‘Not in language’. The datasets were provided by the LT-EDI organisers in English, Tamil, and Malayalam language with texts sourced from YouTube comments. We trained our data using transformer models, specifically mBERT for Tamil and Malayalam and BERT for English, and achieved weighted average F1-scores of 0.46, 0.81, 0.92 for Tamil, Malayalam, and English respectively.

## introduction the world today is rife with uncertainty. with the new covid'19 strain proliferating across the globe, our chances to return to normalcy soon seem rather bleak. it is during testing times like these when people in general, crave for positive reinforcement. even apart from this pandemic, one can argue that we cannot get enough inspiration and more often than not, employees are burdened with stress in the workplace. moreover, there are marginalized communities, such as but not limited to, lgbtiq (lesbian, gay, bisexual, transgender, intersex, and questioning), racial and ethnic minorities, which mostly rely on having faith and being hopeful for their complete acceptance in society @xcite . with the wide usage of the internet, now intensified by the ongoing pandemic, there are more people seeking the same kind of reinforcement through online forums. youtube, being a platform connecting billions of users across the internet, has gained an outstanding popularity across the globe @xcite . with almost 30,000 hours of content uploaded every hour, the videos from youtube are updated showcasing the latest trends @xcite . with the commenting facility available, one can glean a plethora of opinions from different people through this. given the flexibility of speech in most nations in the world, this can be touted as a bane as well leading to rigorous research in offensive speech detection and hate speech detection @xcite @xcite @xcite . however, there hasn't been a proportionate amount of research focusing on hope speech detection. research focusing on positive speech can have a variety of applications. with the growing concerns over mental health, recommendation engines can tailor content in a way which spreads positive comments and hope. in our paper, we have approached hope speech detection using models trained from the dataset provided by the lt-edi organisers with the data obtained from youtube comments in english, tamil and malayalam. we have tried traditional models like svm, logistic regressions and naive bayes as well as transformers like mt5 and bert. we achieved promising results using multilingual bert for tamil and malayalam, and bert for english youtube comments and the task was implemented using the same. ## related works the hope speech dataset for equality, diversity and inclusion was constructed by chakravarthi (2020), from youtube comments, in an attempt to encourage research on detecting positive content online. with the increasing usage of social media amongst different communities across the globe, this dataset encompasses two indian regional languages, namely tamil and malayalam, along with english. the work done by @xcite focused on how hope speech detection can be used to diffuse tension between politically hostile nations. one of the authors' main contributions include a novel language detection technique using polygot word embeddings and a study analysing the temporal shifts between pro-war and pro-peace intentions of social-media users. recently, mining data from social media has been of interest in the field of nlp. the worldwide impact of the covid'19 virus has led to the surge of varied news articles disseminated without much scientific backing. the detection of covid'19 fake news in english and hostile posts in hindi has been done by @xcite . various techniques such as svm, cnn and bi-lstm have been employed for these tasks. in order to tackle out-of-vocabulary words, dong and huang (2018) proposed a methodology combining the pre-trained word embeddings from a large general text corpus with the ones generated from the training corpus based on word2vec. the proposed new representation contains information from both the sources and using vector concatenation, the authors were able to integrate them effectively. ## data analysis the dataset released by lt-edi 2021 (chakravarthi and muralidaran, 2021) for the three languages, english, malayalam and tamil consisted of 28451, 10705 and 20198 youtube comments respectively. this was distributed amongst the the training and the validation dataset consisted of two columns each, "sentence" and "label". the preprocessing performed for all three languages had a few common steps including: additionally, for the english language, we handled out of vocabulary words (oov). oov words are words that are not present in the training dataset. these may be words that are abbreviated or misspelled, and by extension, not present in the english language dictionary. these words do not have embeddings and increase the randomness in the input and can cause misclassifications. to handle this problem, we created a dictionary of length 70, containing the most frequently occurring oov words in the train dataset and replaced them with their corrected usage. apart from this, we also normalized contractions using a predefined list. for example, "couldn't" is replaced with "could not". substituting these contractions makes the text standardized. for english and malayalam, synthetic minority oversampling technique @xcite ) is used for oversampling the imbalanced classes present in the dataset. this is achieved by oversampling the minority class present in the dataset. smote chooses data points that are nearby in the feature space, draws a line between the data points and draws a new sample at a point along that line. the model submitted for all three languages is bert (multilingual bert for tamil and malayalam). to overcome the problem of class imbalance, balanced weights were calculated using the scikit-learn foot_1 library and given as an argument to the model. ## methodology we have employed traditional and transformer based approaches to detect the hope speech present in youtube comments. ## results in the lt-edi-eacl-2021 task on hope speech detection, our team, ssn dibertsity, achieved a top submission rank of 2nd in english, 4th in malayalam and 14th in tamil. our implementation was based on the transformer model bert for the english language and multilingual bert for tamil and malayalam. our submission was evaluated against the test data provided by lt-edi and achieved f1-scores of 0.92, 0.81, 0.46 for english, malayalam and tamil respectively. ## conclusion our implementation consists of two approaches, namely the transformer-based approach and the traditional-based approach. the model submitted for the task is bert which outperformed the other transformer and traditional models. the validation accuracy achieved by the bert model is 93%, 78% and 58% for english, malayalam and tamil respectively. bert's capabilities extend to making more accurate predictions when dealing with newer documents even when the type of document differs significantly in key properties such as length and vocabulary. this attribute of bert makes it the perfect choice when dealing with multiple languages and code-switching within text. in the future, we would like to extend our research in order to create a language model suitable for code-switched content and experiment with different model architectures in the hope for better performance.