Label Propagation-Based Semi-Supervised Learning for Hate Speech Classification (2020)

Abstract: Research on hate speech classification has received increased attention. In real-life scenarios, a small amount of labeled hate speech data is available to train a reliable classifier. Semi-supervised learning takes advantage of a small amount of labeled data and a large amount of unlabeled data. In this paper, label propagation-based semi-supervised learning is explored for the task of hate speech classification. The quality of labeling the unlabeled set depends on the input representations. In this work, we show that pre-trained representations are label agnostic, and when used with label propagation yield poor results. Neural network-based fine-tuning can be adopted to learn task-specific representations using a small amount of labeled data. We show that fully fine-tuned representations may not always be the best representations for the label propagation and intermediate representations may perform better in a semi-supervised setup.

## introduction online hate speech is anti-social communicative behavior and targets minority sections of the society based on religion, ethnicity, gender, etc. @xcite . it leads to threat, fear, and violence to an individual or a group. as monitoring these contents by humans is expensive and timeconsuming, machine learning-based classification techniques can be used. the last few years have seen a tremendous increase in research towards hate speech classification @xcite @xcite . the performance of these classifiers depends on the amount of available labeled data. however, in many real life scenarios there is a limited amount of labeled data and abundant unlabeled data. in need of data, different data augmentation techniques based on synonym replacement @xcite ), text generation @xcite , back translation (aroyehun and gelbukh, 2018), knowledge graphs @xcite , etc, have been employed for up-sampling the training data in the field of hate speech classification. the performance gain by these techniques is small, and they fail to take advantage of the available unlabeled data. semi-supervised learning is a technique to combine a small amount of labeled data with a large amount of unlabeled data during training @xcite , intending to improve the performance of the classifiers. label propagation @xcite is a graph-based semi-supervision technique analogous to the k-nearest-neighbours algorithm. it assumes that data points close to each other tend to have a similar label. these algorithms rely on the representation of data points to create a distance graph which captures their proximity. recently, pre-trained word embeddings such as word2vec, fasttext, global vectors for word representation (glove) have been used for representing words for hate speech classification @xcite . furthermore, pre-trained sentence embeddings such as infersent, universal sentence encoder, embeddings from language models (elmo) have been used for the task of hate speech classification @xcite . these pretrained sentence embeddings are generic representations and are unaware of task-specific classes. transforming the pre-trained sentence embeddings to task-specific representations can be helpful for label propagation, and our work explores this direction. the contributions of this article are: the rest of the paper is organized as follows: section 2 describes label-propagation based semisupervised learning for hate speech classification. section 3 describes the experimental setup. results and discussions are presented in section 4. ## semi-supervised learning in this section, we briefly describe sentence embeddings and the label propagation algorithm. this is followed by our methodology for semi-supervised training for hate speech classification. ## conclusion in this article, we have explored label propagationbased semi-supervised learning for hate speech classification. we evaluated our approach on two datasets of hate speech on the multi-class classification task. we showed that label propagation using the pre-trained sentence embeddings reduces the performance achieved with only the labeled data. this is because pre-trained embeddings do not contain any task-specific information. we validated this by comparing the average intra-class and interclass distances. further, training an mlp classifier with a small amount of labeled data and using the activations of its hidden layers as task aware representations improved the performance of the label propagation and semi-supervised training. it also appears that the fully fine-tuned representations from the mlp may not be the best representations for the label propagation. we can conclude that semi-supervised learning based on label propagation helps to improve hate speech classification in very low resource scenarios and that the performance gain reduces with more amount of labeled data.