CURAJ _ IIITDWD @ LT - EDI - ACL 2022: Hope Speech Detection in E nglish Y ou T ube Comments using Deep Learning Techniques (2022)

Abstract: Hope Speech are positive terms that help to promote or criticise a point of view without hurting the user’s or community’s feelings. Non-Hope Speech, on the other side, includes expressions that are harsh, ridiculing, or demotivating. The goal of this article is to find the hope speech comments in a YouTube dataset. The datasets were created as part of the “LT-EDI-ACL 2022: Hope Speech Detection for Equality, Diversity, and Inclusion” shared task. The shared task dataset was proposed in Malayalam, Tamil, English, Spanish, and Kannada languages. In this paper, we worked at English-language YouTube comments. We employed several deep learning based models such as DNN (dense or fully connected neural network), CNN (Convolutional Neural Network), Bi-LSTM (Bidirectional Long Short Term Memory Network), and GRU(Gated Recurrent Unit) to identify the hopeful comments. We also used Stacked LSTM-CNN and Stacked LSTM-LSTM network to train the model. The best macro average F1-score 0.67 for development dataset was obtained using the DNN model. The macro average F1-score of 0.67 was achieved for the classification done on the test data as well.

## introduction in recent years, the majority of the world's population has access to social media. the social media's posts, comments, articles, and other content have a significant impact on everyone's lives. people tend to believe that their lives on social media are the same as their real lives, therefore the influence of others' opinions or expressions is enormous @xcite . people submit their posts to social networking platform and receive both positive and negative expressions from their peer users. people in a multilingual world use a variety of languages to express themselves, including english, hindi, malayalam, french, and others @xcite . while the most effective expression in real life is face or visual expression, which frequently delivers a much more efficient message than linguistic words, expressions in virtual life, such as social media, are frequently expressed through linguistic texts (or words) and emoticons. these words have a significant impact on one's life @xcite @xcite @xcite . for example, if we respond to someone's social media post with "well done!", "very good", "must do it again", "need a little more practise", and so on, it may instil confidence in the author. on the other hand, negative statements such as "you should not try it", "you don't deserve it", "you are from a different religion", and others demotivate the person. the comments that fall into the first group are referred to as "hope speech" while those that fall into the second category are referred to as "nonhope speech" or "hate speech" @xcite @xcite . in the previous decade, researchers have worked heavily on hate speech identification in order to maintain social media clean and healthy. however, in order to improve the user experience, it is also necessary to emphasise the message of hope on these sites. to our knowledge, the shared task "lt-edi-eacl 2021: hope speech detection for equality, diversity, and inclusion foot_0 " was the first attempt to recognise hope speech in youtube comments. the organizers proposed the shared task in three different languages that is english, tamil and malayalam. many research teams from all over the world took part in the shared task and contributed their working notes to describe how to identify the hope speech comments. @xcite used a parallel network of cnn and lstm with glove and word2vec embeddings and obtained a weighted f1-score of 0.91 for en-glish dataset. similarly, for tamil and malayalam they trained parallel bidirectional lstm model and obtained f1-score of 0.56 and 0.78 respectively. @xcite trained several fine tuned transformer models and identified that ulm-fit is best for english with f1-score 0.9356. they also found that mbert obtained 0.85 f1-score on malayalam dataset and distilmbert obtained 0.59 f1-score on tamil dataset. for the same task a fine tuned albert model was used by (chen and kong, 2021) and they obtained a f1-score of 0.91. similarly, @xcite @xcite employed xlm-roberta-based model with attention for hope speech detection. @xcite experimented the conventional classifiers like logistic regression and support vector machine with tf_idf character n-gram fearures for hope speech classification. acl 2022 will see the introduction of a new edition of the shared task "hope speech detection for equality, diversity, and inclusion." in contrast to lt-edi-eacl 2021, this time the shared task lt-edi-acl 2022 has been proposed in five different languages: english, malayalam, tamil, kannada, and spanish @xcite @xcite . the data was extracted via the youtube platform. we took part in the competition and worked on the dataset of english hope speech comments. the experiments were carried out on several neural network based models such as a dense or multilayer neural network (dnn), one layer cnn network (cnn), one layer bi-lstm network (bi-lstm), and one layer gru network (gru), among deep learning networks. the stack connections of lstm-cnn and lstm-lstm were also trained for hope speech detection. after all experimentation, it was found that dnn produced the best results with macro average f1-score of 0.67 on development as well as on test dataset. the rest of the article is oragnized as follows. the next section 2 give the details of the given task and dataset statistics. this is followed by the description of methodology used for experimentation in section 3. the results are explained in the section 4. at the end, section 5 talks about future scope of the research. 2 task and data description at lt-edi-acl 2022, the shared task on hope speech detection for equality, diversity, and inclusion (provided in english, tamil, spanish, kannada, and malayalam) intended to determine whether the given comment was hope speech or not. the dataset was gathered from the youtube platform. in the given dataset, there were two fields for each language: comment and label. we only submitted the system for the english dataset. in the english training dataset, there were approximately 22740 comments, with 1962 labeled as hope speech and 20778 labeled as non-hope speech. there were 2841 comments in the development dataset, with 272 hope speech and 2569 non-hope speech comments.the test dataset contained 250 hope speech and 2593 non-hope speech comments. the english dataset statistics is shown in ## methodology several deep learning models were developed to identify the hope speech from supplied english youtube comments. the architecture of our best model dnn, as depicted in figure 1 , will be explained in this section. we also explain the architecture of stacked network lstm-lstm as shown in figure 2 . hopespeech non hopespeech total training 1962 20778 22740 development 272 2569 2841 test 250 2593 2843 total 2434 25940 28424 ## results all of the experiments were carried out in the keras and sklearn environment. we used the pandas library to read the datasets. keras preprocessing classes and the nltk library were used to prepare the dataset. all the results shown in ## conclusion as part of the joint task lt-edi-acl2022, we presented a model provided by team curj_iiitdwd for detecting hope speech on an english dataset obtained from the youtube platform. we used many deep learning algorithms in the paper and found that dnn with three hidden layers performed best on the development and test dataset with a macro average f1-score of 0.67. in the future, we can improve classification performance by training transfer learning models like bert and ulmfit ans so on.