SOA _ NLP @ LT - EDI - ACL 2022: An Ensemble Model for Hope Speech Detection from Y ou T ube Comments (2022)

Abstract: Language should be accommodating of equality and diversity as a fundamental aspect of communication. The language of internet users has a big impact on peer users all over the world. On virtual platforms such as Facebook, Twitter, and YouTube, people express their opinions in different languages. People respect others’ accomplishments, pray for their well-being, and cheer them on when they fail. Such motivational remarks are hope speech remarks. Simultaneously, a group of users encourages discrimination against women, people of color, people with disabilities, and other minorities based on gender, race, sexual orientation, and other factors. To recognize hope speech from YouTube comments, the current study offers an ensemble approach that combines a support vector machine, logistic regression, and random forest classifiers. Extensive testing was carried out to discover the best features for the aforementioned classifiers. In the support vector machine and logistic regression classifiers, char-level TF-IDF features were used, whereas in the random forest classifier, word-level features were used. The proposed ensemble model performed significantly well among English, Spanish, Tamil, Malayalam, and Kannada YouTube comments.

## introduction people have started to spend more time on social media platforms in recent years. as a result, many informed decisions are taken based on the sentiment of the social media community @xcite @xcite @xcite . social media gives us the opportunity to track the activity of our friends and family, just like we do in real life @xcite . additionally, it also allows us to communicate with people we have never met in person across the globe. on these websites, there are mostly two types of vibes: hope and hate. hope is a positive state of mind defined by the expectation of favourable outcomes in one's life events and circumstances. people are motivated to act when they are filled with hope. hope can be useful for anyone who wishes to maintain a consistent and positive outlook on life @xcite . we often use hope terms, such as "well done!", "congratulations", "can be done in better way", "keep up the good work" and so on to encourage on one's work. hope contents frequently assist us in a variety of critical situations, including emergency management, photo sharing, video streaming, trip planning, and citizen engagement @xcite . hate, on the other hand, is a negative vibe present on an online platform with the intention of harassing an individuals based on their race, religion, ethnic origin, sexual orientation, disability, or gender @xcite @xcite . the ultimate purpose of every social media platform is to reduce hate content while simultaneously promoting hope content. although there is much work being done to eradicate negativity from the social media @xcite @xcite , hope speech detection focuses on spreading optimism by detecting content that is encouraging, positive, and supporting. there hasn't been much work done in the domain of hope speech detection, although the nlp community has recently shown interest in it @xcite . to reduce hostility, (chakravarthi, 2020) developed a hope detection methodology for the youtube platform in 2019. in the year 2020, (chakravarthi and muralidaran, 2021) presented the lt-edi-eacl2021 shared task foot_0 , which attempted to discover hope speeches in a corpus of english, tamil, and malayalam. to identify hope content in youtube comments, @xcite developed a bidirectional long short-term memory (bilstm) using attention-based technique. for the same goal, @xcite presented a transformer-based bert model. @xcite employed synthetically generated code-mixed data to train a transformer-based model roberta, which they used with their pretrained ulmfit in an ensemble for hope speech categorization. the second workshop on language technology for equality, diversity, and inclusion (lt-edi-2022) is proposed in acl 2022 @xcite , with the shared assignment available in english, tamil, malayalam, kannada, and spanish. we participated in the lt-edi-2022 competition and submitted an ensemble model by utilizing char-level features with support vector machine and logistic regression classifiers and word-level features with random forest classifier. the proposed ensemble model placed 8th, 4th, 3rd, 2nd, and 3rd for english, tamil, malayalam, kannada, and spanish dataset, respectively, among all other submitted models in the competition. the remaining parts of the paper are organized as follows: section 2 analyses similar research for hope speech detection, section 3 examines the datasets and technique used in the study, section 4 discusses results of the proposed model and section 5 concludes the study with future directions. ## related work several studies have been reported by researchers @xcite @xcite to identify hate and offensive material from social media, but relatively few efforts have been done to identify hope speech from social media @xcite @xcite . @xcite evaluated different transfer learning-based models for hope speech identification from english, tamil, and malayalam social media postings, including bert, albert, distilbert, roberta, character-bert, mbert, and ulmfit. ulmfit achieved an f 1 -score of 0.9356 on english data due to its improved finetuning process. on the malayalam test set, mbert achieved a weighted f 1 -score of 0.8545, whereas distilmbert achieved a weighted f 1 -score of 0.5926 on the tamil test set. @xcite offered three models based on ensemble of classifiers, neural network (nn), and bilstm with one dimensional convolution model. the first two models were trained using character and word gram features, while the third model was created using bilstm and one dimensional convolution. finally, classification was carried out in each case. ensemble of classifiers outperformed the other two models, with f1-scores of 0.85, 0.92, and 0.59 for malayalam, english, and tamil datasets, respectively. the hope speech was identified using a finetuned xlm-roberta model by @xcite . @xcite used a pretrained transformers with paraphrasing generation for data augmentation for hope content identification. @xcite used an attentionbased strategy to create a bidirectional long shortterm memory (bilstm), @xcite offered a transformer-based bert model, and @xcite built a transformerbased model roberta with synthetically produced code-mixed data, which they used with their pretrained ulmfit in an ensemble for hope speech classification. in accordance with the existing literature, this paper proposes an ensemble model for the hope speech detection from english, spanish, tamil, malayalam, and kannada youtube comments. ## methodology figure 1 depicts the overall flow diagram of the proposed ensemble model. the proposed ensemble model combines three machine learning algorithms: (i) support vector machine (svm), (ii) logistic regression (lr), and (iii) random forest (rf). the suggested approach is tested using youtube comments in five distinct languages: english, spanish, tamil, malayalam, and kannada. to find the best-suited features and classifiers, we experimented with seven machine learning classifiers such as (i) support vector machine, (ii) random forest, (iii) logistic regression, (iv) naive bayes, (v) k-nearest neighbor, (vi) decision tree, and (vii) adaboost with different combinations of n-gram char-level and word-level term-frequency-inverse-document-frequency (tf-idf). we varied the n-gram range from 1 to 6 for both char-level and word-level features. after performing extensive experiments, we found that 1 to 6-gram char-level tf-idf feature with logistic regression and support vector machine performed best among all the mentioned classifiers, whereas 1 to 3-gram wordlevel features performed best for random forest classifier. the performance of best-suited classifiers with best-suited features are tabulated in the prediction of all three best-performed machine learning classifiers support vector machine, logistic regression, and random forest are taken into account and performed a majority voting (see figure 1 ) to get the final class value for the data sample. ## results all experiments were run on the google colab platform 2 with the sklearn python library 3 and the default classifier hyper-parameters. the performance of the proposed ensemble model is measured using macro precision, macro recall, macro f 1 -score, weighted precision, weighted recall, and weighted f 1 -score. the current work utilized an ensemble strategy that includes a support vector machine, logistic regression, and random forest classifiers to identify hope speech from youtube comments. the efficiency of different combinations of n-gram char-level and word-level tf-idf features were also explored in the identification of hope speech from youtube comments. the use of 1 to 6-gram char-level tf-idf features with support vector machine and logistic regression performed best, whereas 1 to 3-gram word-level features with random forest classifier performed best among all the three mentioned classifiers. the proposed ensemble model achieved a macro f 1 -scores of 0.380, 0.790, 0.290, 0.480, and 0.470 for english, spanish, tamil, malayalam, and kannada language youtube comments, respectively. as the use of char-level features performs significantly well, therefore the char-level features can be explored. deep learning-based models such as bert, cnn, and auto-encoders can also be explored with proper pre-processing of the texts to achieve better performance.