cs_english@ LT - EDI - EACL 2021: Hope Speech Detection Based On Fine-tuning ALBERT Model (2021)

Abstract: This paper mainly introduces the relevant content of the task “Hope Speech Detection for Equality, Diversity, and Inclusion at LT-EDI 2021-EACL 2021”. A total of three language datasets were provided, and we chose the English dataset to complete this task. The specific task objective is to classify the given speech into ‘Hope speech’, ‘Not Hope speech’, and ‘Not in intended language’. In terms of method, we use fine-tuned ALBERT and K fold cross-validation to accomplish this task. In the end, we achieved a good result in the rank list of the task result, and the final F1 score was 0.93, tying for first place. However, we will continue to try to improve methods to get better results in future work.

## introduction as we all know, we are currently facing an incompletely harmonious and secure network environment. nowadays, the number of internet users is very large, especially the proportion of minors is steadily increasing, which shows how important it is to create a hopeful social media environment. such an environment that embodies equality, tolerance, and diversity can help people who are in depression, confusion, lack of identity and other difficulties gain hope. at the same time, it also brings a better online social experience to the entire user community on social media. currently, the detection and classification of social media speech are mostly biased towards offensive and controversial speech detection, but this task is hope speech detection. the difference is that the former uses detection of negative speech to eliminate the impact of offensive speech on the creation of a healthy network environment. the hope speech detection task focuses on detecting hopeful words to create a good network environment, rather than detecting and deleting negative comments to encourage people and deprive individuals of their freedom of speech @xcite . the data source of this task comes from youtube comments. what we have to do is to divide it into three categories: 'hope speech', 'not hope speech' and 'not in intended language'. and we used k-fold cross-validation and albert model to complete the detection of hope speech. ## related work at present, there are many studies on offensive and discriminatory speech detection, but there are few studies on hope speech detection. therefore, we have looked for many related social media speech detection studies that can help us complete this task as a reference. marzieh @xcite the regularization method to adjust the input text in the detection of hate and discriminatory speech and used a finetuned bert model @xcite . polychronis charitidis et al. used the most advanced deep learning architecture to detect hate speech, trained and evaluated it using annotated data sets, and proposed an overall learning architecture that combines the predictive capabilities of each classifier @xcite . based on bert's pre-trained language model, marzieh mozafari et al. made new fine-tuning of transfer learning and discussed the effect of using bert to detect hate speech in a social media environment @xcite . shriphani palakodety et al. performed the qualitative and quantitative analysis of this embedding space for a variety of languages, they used active learning to train set construction in the analysis of peaceful speech @xcite . zewdie mossie et al. used word2vec word embedding technology for feature extraction when conducting vulnerable community identification, and compared the effects of classic classifiers with deep learning classifiers @xcite . shanita biere et al. used natural language technology to detect hate speech on social media, using a convolutional neural network (cnn) @xcite . viktor golem et al. tried different machine learning approaches, including traditional (shallow) machine learning models, deep learning models, and a combination of both for aggressive text detection @xcite . jing ma et al. proposed a novel method that learns continuous representations of microblog events for identifying rumors. that model is based on recurrent neural networks (rnn) and that can capture the variation of contextual information of relevant posts over time @xcite ## data the data sets we can use are the training set and validation set in three languages(tamil, malayalam, and english) provided by the task organizer figure 3 : the fine-tuning strategies we use in english tasks. the output of layers 9-11 must be processed before being spliced with @xcite . the processing method is to discard the first dimension data in the original data @xcite to @xcite ) to obtain a tensor output with the same shape as [cls] (shape is [b, h]). team(chakravarthi, 2020; chakravarthi and muralidaran, 2021). we only participate in the english task, so here we only analyze the english data set. we use the official data set for visual analysis. we can observe that some words such as "live", "people", "black", "matter" "racist" in the english data set appear very frequently. the appearance of these words is very suitable for the theme of the mission we participated in. however, these words mostly express some negative information. and these data mainly come from social media platforms, on the other hand, they also reflect the current status quo on social media platforms. we visualize the text data of two different labels of "non hope speech" and "hope speech". figures 1 and 2 show the results of text visualization. there are three types of category labels that appear in the english data set. they are "hope speech", "non hope speech", "not english". through further analysis of the data set, most of the mark types in the english data are "non hope speech" (around 90%). there is an imbalance of category labels in the data sets of the english data set. the specific steps are as follows: the first step is to randomly divide the original data into k parts without repeated sampling; the second step is to extract 1 copy as the test set, and the remaining k-1 copies are applied to the training set for model training; in the third step, repeat the second step k times, so that a model is obtained after training on each training set and used to test on the corresponding test set, calculate and save the evaluation index of the mode; the fourth step is to calculate the average of k groups of test results as an estimate of model accuracy, and as the performance index of the model under current k-fold cross-validation. ## conclusion for the english task in this competition, we use the last three layers of the albert model output to fuse with the [cls] output. use k-fold cross-validation on the integrated method to obtain better results. our test results for youtube's hopeful speech are not bad. in the final result, precision was 0.93, recall was 0.94, f1 score was 0.93, and the ranking was tied for first place. but in future work, we will try to improve our method and choose to try other models, hoping to achieve better results on such tasks.