Introduction
Hate speech represents written or oral communication that in any way discredits a person or a group based on characteristics such as race, color, ethnicity, gender, sexual orientation, nationality, or religion BIBREF0. Hate speech targets disadvantaged social groups and harms them both directly and indirectly BIBREF1. Social networks like Twitter and Facebook, where hate speech frequently occurs, receive many critics for not doing enough to deal with it. As the connection between hate speech and the actual hate crimes is high BIBREF2, the importance of detecting and managing hate speech is not questionable. Early identification of users who promote such kind of communication can prevent an escalation from speech to action. However, automatic hate speech detection is difficult, especially when the text does not contain explicit hate speech keywords. Lexical detection methods tend to have low precision because, during classification, they do not take into account the contextual information those messages carry BIBREF3. Recently, contextual word and sentence embedding methods capture semantic and syntactic relation among the words and improve prediction accuracy.
Recent works on combining probabilistic Bayesian inference and neural network methodology attracted much attention in the scientific community BIBREF4. The main reason is the ability of probabilistic neural networks to quantify trustworthiness of predicted results. This information can be important, especially in tasks were decision making plays an important role BIBREF5. The areas which can significantly benefit from prediction uncertainty estimation are text classification tasks which trigger specific actions. Hate speech detection is an example of a task where reliable results are needed to remove harmful contents and possibly ban malicious users without preventing the freedom of speech. In order to assess the uncertainty of the predicted values, the neural networks require a Bayesian framework. On the other hand, Srivastava et al. BIBREF6 proposed a regularization approach, called dropout, which has a considerable impact on the generalization ability of neural networks. The approach drops some randomly selected nodes from the neural network during the training process. Dropout increases the robustness of networks and prevents overfitting. Different variants of dropout improved classification results in various areas BIBREF7. Gal and Ghahramani BIBREF8 exploited the interpretation of dropout as a Bayesian approximation and proposed a Monte Carlo dropout (MCD) approach to estimate the prediction uncertainty. In this paper, we analyze the applicability of Monte Carlo dropout in assessing the predictive uncertainty.
Our main goal is to accurately and reliably classify different forms of text as hate or non-hate speech, giving a probabilistic assessment of the prediction uncertainty in a comprehensible visual form. We also investigate the ability of deep neural network methods to provide good prediction accuracy on small textual data sets. The outline of the proposed methodology is presented in Figure FIGREF2.
Our main contributions are:
investigation of prediction uncertainty assessment to the area of text classification,
implementation of hate speech detection with reliability output,
evaluation of different contextual embedding approaches in the area of hate speech,
a novel visualization of prediction uncertainty and errors of classification models.
The paper consists of six sections. In Section 2, we present related works on hate speech detection, prediction uncertainty assessment in text classification context, and visualization of uncertainty. In Section 3, we propose the methodology for uncertainty assessment using dropout within neural network models, as well as our novel visualization of prediction uncertainty. Section 4 presents the data sets and the experimental scenario. We discuss the obtained results in Section 5 and present conclusions and ideas for further work in Section 6.
Related Work
We shortly present the related work in three areas which constitute the core of our approach: hate speech detection, recurrent neural networks with Monte Carlo dropout for assessment of prediction uncertainty in text classification, and visualization of predictive uncertainty.
Related Work ::: Hate Speech Detection
Techniques used for hate speech detection are mostly based on supervised learning. The most frequently used classifier is the Support Vector Machines (SVM) method BIBREF9. Recently, deep neural networks, especially recurrent neural network language models BIBREF10, became very popular. Recent studies compare (deep) neural networks BIBREF11, BIBREF12, BIBREF13 with the classical machine learning methods.
Our experiments investigate embeddings and neural network architectures that can achieve superior predictive performance to SVM or logistic regression models. More specifically, our interest is to explore the performance of MCD neural networks applied to the hate speech detection task.
Related Work ::: Prediction Uncertainty in Text Classification
Recurrent neural networks (RNNs) are a popular choice in text mining. The dropout technique was first introduced to RNNs in 2013 BIBREF14 but further research revealed negative impact of dropout in RNNs, especially within language modeling. For example, the dropout in RNNs employed on a handwriting recognition task, disrupted the ability of recurrent layers to effectively model sequences BIBREF15. The dropout was successfully applied to language modeling by BIBREF16 who applied it only on fully connected layers. The then state-of-the-art results were explained with the fact that by using the dropout, much deeper neural networks can be constructed without danger of overfitting. Gal and Ghahramani BIBREF17 implemented the variational inference based dropout which can also regularize recurrent layers. Additionally, they provide a solution for dropout within word embeddings. The method mimics Bayesian inference by combining probabilistic parameter interpretation and deep RNNs. Authors introduce the idea of augmenting probabilistic RNN models with the prediction uncertainty estimation. Recent works further investigate how to estimate prediction uncertainty within different data frameworks using RNNs BIBREF18. Some of the first investigation of probabilistic properties of SVM prediction is described in the work of Platt BIBREF19. Also, investigation how Bayes by Backprop (BBB) method can be applied to RNNs was done by BIBREF20.
Our work combines the existing MCD methodology with the latest contextual embedding techniques and applies them to hate speech classification task. The aim is to obtain high quality predictions coupled with reliability scores as means to understand the circumstances of hate speech.
Related Work ::: Prediction Uncertainty Visualization in Text Classification
Visualizations help humans in making decisions, e.g., select a driving route, evacuate before a hurricane strikes, or identify optimal methods for allocating business resources. One of the first attempts to obtain and visualize latent space of predicted outcomes was the work of Berger et al. BIBREF21. Prediction values were also visualized in geo-spatial research on hurricane tracks BIBREF22, BIBREF23. Importance of visualization for prediction uncertainty estimation in the context of decision making was discussed in BIBREF24, BIBREF25.
We are not aware of any work on prediction uncertainty visualization for text classification or hate speech detection. We present visualization of tweets in a two dimensional latent space that can reveal relationship between analyzed texts.
Deep Learning with Uncertainty Assessment
Deep learning received significant attention in both NLP and other machine learning applications. However, standard deep neural networks do not provide information on reliability of predictions. Bayesian neural network (BNN) methodology can overcome this issue by probabilistic interpretation of model parameters. Apart from prediction uncertainty estimation, BNNs offer robustness to overfitting and can be efficiently trained on small data sets BIBREF26. However, neural networks that apply Bayesian inference can be computationally expensive, especially the ones with the complex, deep architectures. Our work is based on Monte Carlo Dropout (MCD) method proposed by BIBREF8. The idea of this approach is to capture prediction uncertainty using the dropout as a regularization technique.
In contrast to classical RNNs, Long Short-term Memory (LSTM) neural networks introduce additional gates within the neural units. There are two sources of information for specific instance $t$ that flows through all the gates: input values $x_t$ and recurrent values that come from the previous instance $h_{t-1}$. Initial attempts to introduce dropout within the recurrent connections were not successful, reporting that dropout brakes the correlation among the input values. Gal and Ghahramani BIBREF17 solve this issue using predefined dropout mask which is the same at each time step. This opens the possibility to perform dropout during each forward pass through the LSTM network, estimating the whole distribution for each of the parameters. Parameters' posterior distributions that are approximated with such a network structure, $q(\omega )$, is used in constructing posterior predictive distribution of new instances $y^*$:
where $p\big (y^*|f^\omega (x^*)\big )$ denotes the likelihood function. In the regression tasks, this probability is summarized by reporting the means and standard deviations while for classification tasks the mean probability is calculated as:
where $\hat{\omega }_k$ $\sim $ $q(\omega )$. Thus, collecting information in $K$ dropout passes throughout the network during the training phase is used in the testing phase to generate (sample) $K$ predicted values for each of the test instance. The benefit of such results is not only to obtain more accurate prediction estimations but also the possibility to visualize the test instances within the generated outcome space.
Deep Learning with Uncertainty Assessment ::: Prediction Uncertainty Visualization
For each test instance, the neural network outputs a vector of probability estimates corresponding to the samples generated through Monte Carlo dropout. This creates an opportunity to visualize the variability of individual predictions. With the proposed visualization, we show the correctness and reliability of individual predictions, including false positive results that can be just as informative as correctly predicted ones. The creation of visualizations consists of the following five steps, elaborated below.
Projection of the vector of probability estimates into a two dimensional vector space.
Point coloring according to the mean probabilities computed by the network.
Determining point shapes based on correctness of individual predictions (four possible shapes).
Labeling points with respect to individual documents.
Kernel density estimation of the projected space — this step attempts to summarize the instance-level samples obtained by the MCD neural network.
As the MCD neural network produces hundreds of probability samples for each target instance, it is not feasible to directly visualize such a multi-dimensional space. To solve this, we leverage the recently introduced UMAP algorithm BIBREF27, which projects the input $d$ dimensional data into a $s$-dimensional (in our case $s=2$) representation by using computational insights from the manifold theory. The result of this step is a two dimensional matrix, where each of the two dimensions represents a latent dimension into which the input samples were projected, and each row represents a text document.
In the next step, we overlay the obtained representation with other relevant information, obtained during sampling. Individual points (documents) are assigned the mean probabilities of samples, thus representing the reliability of individual predictions. We discretize the $[0,1]$ probability interval into four bins of equal size for readability purposes. Next, we shape individual points according to the correctness of predictions. We take into account four possible outcomes (TP - true positives, FP - false positives, TN - true negatives, FN - false negatives).
As the obtained two dimensional projection represents an approximation of the initial sample space, we compute the kernel density estimation in this subspace and thereby outline the main neural network's predictions. We use two dimensional Gaussian kernels for this task.
The obtained estimations are plotted alongside individual predictions and represent densities of the neural network's focus, which can be inspected from the point of view of correctness and reliability.
Experimental Setting
We first present the data sets used for the evaluation of the proposed approach, followed by the experimental scenario. The results are presented in Section SECREF5.
Experimental Setting ::: Hate Speech Data Sets
We use three data sets related to the hate speech.
Experimental Setting ::: Hate Speech Data Sets ::: 1 - HatEval
data set is taken from the SemEval task "Multilingual detection of hate speech against immigrants and women in Twitter (hatEval)". The competition was organized for two languages, Spanish and English; we only processed the English data set. The data set consists of 100 tweets labeled as 1 (hate speech) or 0 (not hate speech).
Experimental Setting ::: Hate Speech Data Sets ::: 2 - YouToxic
data set is a manually labeled text toxicity data, originally containing 1000 comments crawled from YouTube videos about the Ferguson unrest in 2014. Apart from the main label describing if the comment is hate speech, there are several other labels characterizing each comment, e.g., if it is a threat, provocative, racist, sexist, etc. (not used in our study). There are 138 comments labeled as a hate speech and 862 as non-hate speech. We produced a data set of 300 comments using all 138 hate speech comments and randomly sampled 162 non-hate speech comments.
Experimental Setting ::: Hate Speech Data Sets ::: 3 - OffensiveTweets
data set originates in a study regarding hate speech detection and the problem of offensive language BIBREF3. Our data set consists of 3000 tweets. We took 1430 tweets labeled as hate speech and randomly sampled 1670 tweets from the collection of remaining 23353 tweets.
Experimental Setting ::: Hate Speech Data Sets ::: Data Preprocessing
Social media text use specific language and contain syntactic and grammar errors. Hence, in order to get correct and clean text data we applied different prepossessing techniques without removing text documents based on the length. The pipeline for cleaning the data was as follows:
Noise removal: user-names, email address, multiple dots, and hyper-links are considered irrelevant and are removed.
Common typos are corrected and typical contractions and hash-tags are expanded.
Stop words are removed and the words are lemmatized.
Experimental Setting ::: Experimental Scenario
We use logistic regression (LR) and Support Vector Machines (SVM) from the scikit-learn library BIBREF28 as the baseline classification models. As a baseline RNN, the LSTM network from the Keras library was applied BIBREF29. Both LSTM and MCD LSTM networks consist of an embedding layer, LSTM layer, and a fully connected layer within the Word2Vec and ELMo embeddings. The embedding layer was not used in TF-IDF and Universal Sentence encoding.
To tune the parameters of LR (i.e. liblinear and lbfgs for the solver functions and the number of component $C$ from $0.01$ to 100) and SVM (i.e. the rbf for the kernel function, the number of components $C$ from $0.01$ to 100 and the gamma $\gamma $ values from $0.01$ to 100), we utilized the random search approach BIBREF30 implemented in scikit-learn. In order to obtain best architectures for the LSTM and MCD LSTM models, various number of units, batch size, dropout rates and so on were fine-tuned.
Evaluation and Results
We first describe experiments comparing different word representations, followed by sentence embeddings, and finally the visualization of predictive uncertainty.
Evaluation and Results ::: Word Embedding
In the first set of experiments, we represented the text with word embeddings (sparse TF-IDF BIBREF31 or dense word2vec BIBREF32, and ELMo BIBREF33). We utilise the gensim library BIBREF34 for word2vec model, the scikit-learn for TFIDF, and the ELMo pretrained model from TensorFlow Hub. We compared different classification models using these word embeddings. The results are presented in Table TABREF32.
The architecture of LSTM and MCD LSTM neural networks contains an embedding layer, LSTM layer, and fully-connected layer (i.e. dense layer) for word2vec and ELMo word embeddings. In LSTM, the recurrent dropout is applied to the units for linear transformation of the recurrent state and the classical dropout is used for the units with the linear transformation of the inputs. The number of units, recurrent dropout, and dropout probabilities for LSTM layer were obtained by fine-tuning (i.e. we used 512, $0.2$ and $0.5$ for word2vec and TF-IDF, 1024, $0.5$, and $0.2$ for ELMo in the experiments with MCD LSTM architecture). The search ranges for hyper parameter tuning are described in Table TABREF33.
The classification accuracy for HatEval data set is reported in the Table TABREF32 (left). The difference between logistic regression and the two LSTM models indicates accuracy improvement once the recurrent layers are introduced. On the other hand, as the ELMo embedding already uses the LSTM layer to take into account semantic relationship among the words, no notable difference between logistic regression and LSTM models can be observed using this embedding.
Results for YouToxic and OffensiveTweets data sets are presented in Table TABREF32 (middle) and (right), respectively. Similarly to the HatEval data set, there is a difference between the logistic regression and the two LSTM models using the word2vec embeddings. For all data sets, the results with ELMo embeddings are similar across the four classifiers.
Evaluation and Results ::: Sentence Embedding
In the second set of experiments, we compared different classifiers using sentence embeddings BIBREF35 as the representation. Table TABREF36 (left) displays results for HatEval. We can notice improvements in classification accuracy for all classifiers compared to the word embedding representation in Table TABREF32. The best model for this small data set is MCD LSTM. For larger YouToxic and OffensiveTweets data sets, all the models perform comparably. Apart from the prediction accuracy the four models were compared using precision, recall and F1 score BIBREF36.
We use the Universal Sentence Encoder module to encode the data. The architecture of LSTM and MCD LSTM contains a LSTM layer and dense layer. With MCD LSTM architecture in the experiments, the number of neurons, recurrent dropout and dropout value for LSTM is 1024, $0.75$ and $0.5$, respectively. The dense layer has the same number of units as LSTM layer, and the applied dropout rate is $0.5$. The hyper-parameters used to tune the LSTM and MCD LSTM models are presented in the Table TABREF33.
Evaluation and Results ::: Visualizing Predictive Uncertainty
In Figure FIGREF38 we present a new way of visualizing dependencies among the test tweets. The relations are result of applaing the MCD LSTM network to the HetEval data set. This allows further inspection of the results as well as interpretation of correct and incorrect predictions. To improve comprehensibility of predictions and errors, each point in the visualization is labeled with a unique identifier, making the point tractable to the original document, given in Table TABREF39.
As Figure FIGREF38 shows, the tweets are grouped into two clusters. According to the kernel density isometric lines, two centers are identified: the tweets assigned lower probability of being hate speech and the tweets with higher probability of being hate speech. Let us focus on the wrongly classified tweets and their positions in the graph (tweets 8, 16 and 18). While for tweets 8 and 18 the classifier wasn't certain and a mistake seems possible according to the plot, the tweet 16 was predicted to be hate speech with high probability. Analyzing the words that form this tweet, we notice that not only that most of them often do appear in the hate speech but also this combination of the words used together is very characteristic for the offensive language.
Our short demonstration shows the utility of the proposed visualization which can identify different types of errors and helps to explain weaknesses in the classifier or wrongly labeled data.
Conclusions
We present the first successful approach to assessment of prediction uncertainty in hate speech classification. Our approach uses LSTM model with Monte Carlo dropout and shows performance comparable to the best competing approaches using word embeddings and superior performance using sentence embeddings. We demonstrate that reliability of predictions and errors of the models can be comprehensively visualized. Further, our study shows that pretrained sentence embeddings outperform even state-of-the-art contextual word embeddings and can be recommended as a suitable representation for this task. The full Python code is publicly available .
As persons spreading hate speech might be banned, penalized, or monitored not to put their threats into actions, prediction uncertainty is an important component of decision making and can help humans observers avoid false positives and false negatives. Visualization of prediction uncertainty can provide better understanding of the textual context within which the hate speech appear. Plotting the tweets that are incorrectly classified and inspecting them can identify the words that trigger wrong classifications.
Prediction uncertainty estimation is rarely implemented for text classification and other NLP tasks, hence our future work will go in this direction. A recent emergence of cross-lingual embeddings possibly opens new opportunities to share data sets and models between languages. As evaluation in rare languages is difficult, the assessment of predictive reliability for such problems might be an auxiliary evaluation approach. In this context, we also plan to investigate convolutional neural networks with probabilistic interpretation.
Conclusions ::: Acknowledgments.
The work was partially supported by the Slovenian Research Agency (ARRS) core research programme P6-0411. This project has also received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825153 (EMBEDDIA).

Question: what dataset were used?
Answer: HatEval, YouToxic, OffensiveTweets

HatEval, YouToxic, OffensiveTweets

Question: what was the baseline?
Answer: logistic regression (LR), Support Vector Machines (SVM), LSTM network from the Keras library 

 logistic regression (LR), Support Vector Machines (SVM)

Question: what text embedding methods were used in their approach?
Answer: Word2Vec, ELMo

Word2Vec and ELMo embeddings.