IIIT _ DWD @ LT - EDI - EACL 2021: Hope Speech Detection in Y ou T ube multilingual comments (2021)

Abstract: Language as a significant part of communication should be inclusive of equality and diversity. The internet user’s language has a huge influence on peer users all over the world. People express their views through language on virtual platforms like Facebook, Twitter, YouTube etc. People admire the success of others, pray for their well-being, and encourage on their failure. Such inspirational comments are hope speech comments. At the same time, a group of users promotes discrimination based on gender, racial, sexual orientation, persons with disability, and other minorities. The current paper aims to identify hope speech comments which are very important to move on in life. Various machine learning and deep learning based models (such as support vector machine, logistics regression, convolutional neural network, recurrent neural network) are employed to identify the hope speech in the given YouTube comments. The YouTube comments are available in English, Tamil and Malayalam languages and are part of the task “EACL-2021:Hope Speech Detection for Equality, Diversity and Inclusion”.

## introduction in the current era, there are two lives we live; a real one and a virtual one. in a real case, we interact, communicate and exchange our emotions physically. on the other hand in virtual cases, we interact with others through a computer-based simulated environment, social media is one such example. in both cases, to exchange information a language is required such as english, hindi, spanish, german, and so on. through these languages, we share our moments such as happiness, joy, anger, sadness, appreciation for success, motivation on failure, and others which everyone needs in their hard time. these comments convey the well-being of someone and are termed as "hope speech". the other categories of comments that discourage, abuse, demotivate based on gender, race, sexual orientation, persons with a disability, and other minorities are termed as "not hope speech". in the physical scenario, the reachability and effect of these comments mitigate with region and time. on the other hand, on virtual platforms (especially social media) their effects are long-lasting across the geographical region. hope speech reflects the idea that you will find paths to the desired goals and be encouraged to use them. a few examples of hope speech on social media platforms are; i) product-leverage reviews, ii) election campaign comments, iii) disaster or crisis-informed decision-making comments, and so on. since there is a large number of comments and suggestions on a specific topic, the overall difference in the sentiment of the topic is created @xcite . many times hope speech of a product leads to its great success, for example, introducing a number of helpful votes received by product reviews on amazon leads to additional $2.7 billion revenue foot_0 . similarly, a number of likes, shares, and hope speech comments received by youtube video can make it the most viewed video and so on. also, if the video content or quality is not up to the mark, the hopeful suggestions received by other users help to boost the content or consistency of the video. on the other hand, the not hope speech comments can lower the subject's value and demotivates the person. considering the higher impact of hope and not hope comments in social media contexts, it is therefore important to identify them for an informed decision. the current paper identifies the hope speech comments on the youtube platform. the dataset used in this study is a part of "eacl-2021: hope speech detection for equality, diversity, and inclusion". the comments are in three different languages english, tamil, and malayalam. the problem proposed is a classification problem in three groups, where all comments are marked with hope or not hope or not in the intended language. for classification of comments, several classification models based on conventional machine learning (e.g. support vector machine, logistic regression), deep learning (e.g. convolutional neural network (cnn), long short term memory network (lstm)), and hybrid learning (parallel cnn-lstm network, three parallel bi-lstm network) are developed. the results from the experiments showed that a model based on the cnn-lstm network having 2 parallel layers worked best for the english dataset, with bi-lstm having three parallel layers performed best for tamil and malayalam foot_1 . the rest of the article is organized as follows; section 2 discusses the related works for monolingual dataset classification. section 3 describes the given task and data dimension. this is followed by the methodology of the current paper in section 4. the results are explained in section 5. finally, section 6 concludes the paper by highlighting some future scope. ## related work according to the best of our knowledge, this is the first task on hope speech detection. therefore, there are no previous works in this category. however, the current work is a monolingual classification problem, one of the highly researched problems in the processing of natural languages. in this section, we identify a few state-of-art papers for monolingual classification. there are many applications of monolingual contexts such as sentiment classification, hate speech identification, fake content identification, and others. @xcite performed sentiment analysis of english text from twitter. they use a multi-nominal naive bayes classifier to classify each tweet into positive, negative, and neutral. the features used were n-grams and pos-tags. they found that pos-tags are a strong predictor of sentiments. (saumya and singh, 2018) used the sentiment analysis approach to identify spam reviews on the e-commerce platform. three classifiers random forest, gradient boosting, and support vector machine was trained on amazon reviews. the best performance was reported by random forest with an f1-score of 0.91. @xcite proposed a hate speech classifier for english corpus (25000 tweets). @xcite trained a recurrent neural network on 16k tweet for hate speech identification. a comparative analysis of machine learning, deep learning, and transfer models is presented by @xcite for the hate speech spanish dataset. similarly, for fake news detection on german data (vogel and jiang, 2019) used convolutional network and support vector machine classifiers on 4500 news articles. a fake news hostility detection dataset was proposed by @xcite in the hindi language. they trained several machine learning models like logistic regression, support vector machine, random forest, and multilayer perceptron network with m-bert embedding for classifying hostility contents. in line with the above literature, the current paper employs several machine learning and deep learning models for the classification of monolingual hope speech comments into three classes. the detailed methodology of the proposed model is explained in the next section. the hope speech content identification in english, tamil, and malayalam language is organized by eacl-2021 in a track on hope speech detection for equality, diversity, and inclusion. the objective of this task is to determine whether or not a comment contains a speech of hope. every comment is labeled with either hope speech, not hope speech, and not in intended language. the competition dataset was released in a phased manner. initially, training and development datasets were released for each english, tamil, and malayalam corpus, resulting in six different data files. every file had two fields; a comment and its label. in the given corpus of english, tamil and malayalam, the average comment length was one except for a few cases where it was more than one. the description of both training and development set for all three corpora is shown in ## methodology the current paper proposed multi-label classification methods for the identification of hope speech contents. several conventional machine learning, deep learning, and hybrid learning approaches were employed to achieve the objective. a detailed description of all proposed methods is discussed in the subsection. ## results as discussed above several classification models were built and experimented. we are reporting the results of the submitted models in the competition and a few other comparable models. all models were built in python using libraries sklearn, keras, pandas, numpy an so on. the results reported here are for development data and test data. the metric used to evaluate the performance of a model was the weighted f1-score (or weighted f1). in conventional models, for the english dataset, svm reported weighted f1 0.85 which was higher than lr where weighted f1 was 0.75. the performance of the classification was even better for most of the deep and hybrid classifiers. as it is shown in figure 2 , lstm and bi-lstm models with glove embedding reported 0.90 weighted f1, but at the same time, the performance of 2 layered cnn model with random embedding was very low as weighted f1 score was 0.79. the best-reported results were from the hybrid model 2-parallel cnn-lstm with glove and word2vec embeddings where weighted f1 was 0.91. the performance of 2-parallel lstm-bilstm and 3parallel lstm was also comparable as it reported weighted f1 0.90. for the tamil dataset, in conventional learning, the performance of lr (weighted f1 0.50) was better than svm (weighted f1 0.48). the best performing models were 2-layered cnn and 3-parallel bi-lstm with weighted f1 0.55 and 0.56 respectively. a similar pattern was observed for malayalam data also. the best performing model for malayalam set was 3 parallel bi-lstm with weighted f1 0.78. even, 2 parallel bi-lstm with random embedding showed comparable results with weighted f1 0.77. the performance of svm was reported least with a weighted f1 of 0.65. the best performing models of english, tamil, and malayalam dataset was then validated with test dataset provided by organizers. the test dataset contained only a comment field, therefore, the predictions on test data for best models were submitted to the competition. later, the organizer evaluated the submitted prediction label with the actual label of test data. the test dataset results are shown in as can be seen from ## conclusion the current paper identified hope speech contents in youtube comments using several machine learning, deep learning, and hybrid models. the task was proposed by eacl 2021 for three different languages english, tamil, and malayalam. every comment was categorized in one of three classes hope speech, not hope speech, and not in intended language. on the english dataset, the best performing model was 2-parallel cnn-lstm with glove and word2vec embeddings and it reported weighted f1 0.91 and 0.90 for development and test set respectively. similarly, the best performing model of tamil and malayalam was 3-parallel bi-lstm. for tamil it reported weighted f1 0.56 and 0.54 on development and test dataset respectively. similarly, for malayalam, the reported weighted f1 was 0.78 and 0.79 on the development and test dataset.