SSNCSE _ NLP @ LT - EDI - ACL 2022:Hope Speech Detection for Equality, Diversity and Inclusion using sentence transformers (2022)

Abstract: In recent times, applications have been developed to regulate and control the spread of negativity and toxicity on online platforms. The world is filled with serious problems like political & religious conflicts, wars, pandemics, and offensive hate speech is the last thing we desire. Our task was to classify a text into ‘Hope Speech’ and ‘Non-Hope Speech’. We searched for datasets acquired from YouTube comments that offer support, reassurance, inspiration, and insight, and the ones that don’t. The datasets were provided to us by the LTEDI organizers in English, Tamil, Spanish, Kannada, and Malayalam. To successfully identify and classify them, we employed several machine learning transformer models such as m-BERT, MLNet, BERT, XLMRoberta, and XLM_MLM. The observed results indicate that the BERT and m-BERT have obtained the best results among all the other techniques, gaining a weighted F1- score of 0.92, 0.71, 0.76, 0.87, and 0.83 for English, Tamil, Spanish, Kannada, and Malayalam respectively. This paper depicts our work for the Shared Task on Hope Speech Detection for Equality, Diversity, and Inclusion at LTEDI 2021.

## introduction with an evolving and diversifying world filled with worries and uncertainty, people turn to religion and faith to give them hope. with the existence of marginalized communities, such as the lgbtqia community, and racial and ethnic minorities which focus on having faith and are hopeful for their complete acceptance in society @xcite , there is an increasing need for positive reinforcement in society. with the wide usage of the internet, now intensified by the ongoing pandemic, more people are seeking the same kind of reinforcement through online forums @xcite . youtube is an online platform connecting billions of users across the globe. it is an application owned by google, which allows people worldwide to showcase their talents and express opinions, and connect in the comments section. with around 30000 hours of content uploaded every hour @xcite , each comment under it has people expressing not only their trivial thoughts but also their controversial opinions. this includes but is not restricted to saying hurtful things and shaming communities and groups they harbor ill will against, given the flexibility of speech in most nations in the world. this can be touted as a bane as well leading to rigorous research in offensive speech detection and hate speech detection @xcite @xcite . there has been inadequate research done concentrating on hope speech detection. with this being an era of mental health awareness, it is crucial to develop a solution that recommends uplifting and positive tweets and posts to people, while sidelining the negative, discouraging and disheartening ones. in our paper, we have approached hope speech detection using pre-existing transformer models trained from the dataset provided by the lt-edi organizers with the data obtained from tweets in english, kannada, malayalam, spanish, and tamil. we have used multilingual transformers such as xlm-mlm, bert, and xlm-roberta, achieving promising results using the bert multilingual transformer model. hence the task was implemented using the same. the rest of this paper is organized as follows. section 2 discusses the related work on hope speech detection tasks. the dataset for the shared task is discussed in section 3. section 4 outlines the features and the methods used for this task. results are presented in section 5. section 6 concludes the paper. ## related work researchers have experimented with a few approaches to deal with hope speech detection in many languages recently, but more so in english. the authors of hope speech detection: a computational analysis of the voice of peace @xcite take the help of polyglot wordembeddings to discover language clusters and subsequently construct a language identification technique that requires minimal supervision and performs well on short social media texts generated in a linguistically diverse region of the world. in hope speech detection using indic transliteration and transformers(upadhyay et al., 2021), the authors used 2 approaches. they used contextual embeddings to train classifiers using logistic regression, random forest, svm, and lstm based models. a similar approach is used in hope speech detection in youtube multilingual comments @xcite . the second approach involved using a majority voting ensemble of 11 models which were obtained by fine-tuning pre-trained transformer models (bert, albert, roberta, indicbert) after adding an output layer. they found that the second approach yielded better results for english, tamil, and malayalam. in a multilingual hope speech detection for equality, diversity, and inclusion using context-aware embedding(junaida and ajees, 2021), the authors present deep learning techniques using contextaware string embeddings for word representations and recurrent neural network (rnn) and pooled document embeddings for text representation. in this paper, however, we use pre-trained multilingual transformer models to determine whether a comment is hope speech or not. ## dataset analysis and preprocessing the dataset provided by the lt-edi 2021 @xcite for the five languages, english, tamil, spanish, kannada, and malayalam consisted of 28424, 17716, 1653, 6176, and 9918 youtube comments respectively. refer to the table below 1 subjects like hope's speech might raise confusion and disagreement among literates belonging to different groups. @xcite . ## methodology for this approach, we used a few models, namely, the xlm-mlm models, bert models, and xlm-roberta models for spanish and malayalam and the bert models and xlnet models for english, kannada and tamil. out of all these models, the bert models produced the best results out of all the models, in all the languages. for english, bert-base-multilingual-uncased is shown to yield the best accuracy of 93%. for kannada, bert-base-uncased yielded the best accuracy of 87%. for malayalam, bert-base-multilingualuncased yielded a result of 84%. for spanish too, the bert-base-multilingual-uncased model outperformed the others by giving an accuracy of 76%. for tamil, bert-base-uncased yielded an accuracy of 72%. an analysis of all the results is elaborated in the next section. ## observation in this section, we will be looking into the performance of various machine learning transformer models for the 5 languages (english, tamil, spanish, kannada, malayalam). the weighted f1 score determines the excellence of the models. the tables below present the evaluation results of all the models on the test dataset. the english dataset has the highest f1 score of 0.92 using the m-bert model, winning over a slight margin against the bert model. the bert model had outperformed by its accuracy 2. in the case of the tamil language, the bert model had produced the highest f1-score of 0.71 with better accuracy than the other models 3. in the case of spanish, both the m-bert and the xlm roberta had performed equally well with an f1 score of 0.76 4. bert model had performed well with kannada datasets too with an f1 score of 0.87 and an equally good accuracy of 0.87 5. in the case of malayalam, m-bert outperformed f1 score was 0.83 6. for all the languages, the bert model had showcased the best results and outperformed all the other models.