# A Deep Learning Approach for Automatic Hate Speech Detection in the Saudi Twittersphere
## 6 Experiments Results
### Error Analysis  
To understand the challenges of this task, we carried out further analysis of the errors made by our models, including the baselines. In the task of hate speech detection, we are more interested in \"detecting\" all hateful instances than being precise in our predictions. Given that, we tried to understand why our models failed at detecting hateful tweets. We identified the hateful tweets that all evaluated models on both datasets predicted incorrectly (misclassified as non-hate). We ended up with a total of 151 tweets for GHSD and 51 tweets for RHSD. We further qualitatively analyzed these tweets to understand the possible reasons that they were misclassified by the models. We found that nearly 90% of the misclassified tweets did not have offensive/abusive terms--this could confuse the classifiers and cause its performance to be degraded. We also found that many tweets would even be difficult for humans to classify as hate without being provided with the full context of the tweet. For example, some of the tweets contain comments on images or videos, which will be very tricky to interpret without seeing the associated media. The lack of the context problem also appeared with tweets that refer to external links or quoted tweets. An example is the following tweet that quoted another tweet with this comment: _\"This is serious, this means that those people will be back to work in secret so that no one will pay attention to them\"_. This tweet was written by an extremely patriotic user who uses \"those\" to refer to foreigners working in Saudi Arabia, indicating they have some agenda to call for separation of the Hijaz region from the rest of the country. This tweet is difficult to interpret correctly without knowing what it quoted exactly. Mentions also have the same problem--some mentions contain only few natural words, but it should be classified as hate when we take into account the original tweet that it repliedto. For example, this tweet \"@user Qahtani freshman\" replied to an offensive tweet that negatively stereotypes a particular tribe's members, but the classifiers incorrectly classified it as non-hate. Finally, we found that some tweets were indirect, implicit, or sarcastic, which make it difficult for automatic solutions to detect correctly. For example, this sarcastic tweet _\"And now we introduce to you, [PERSON]'s play'so free' in the era of the great Saudi\"_ was misclassed as non-hate, but it was written by non-Saudi user against a Saudi user as a part of hateful conversation between them.  
We also performed further analysis to understand the cases in which the classifiers misclassified non-hate tweets as hateful. We ended up with 33 misclassified non-hate tweets in GHSD and 20 tweets in RHSD. Mostly all of these tweets included one or more religious, national, or ideological affiliations terms that are highly associated with hate class (according to our generated lexicons). Examples: _\"Hahah... [PERSON] brother, first of all Kharasan scholars are not related to the Persians Magians! [PERSON] was and still the head of knowledge\"_ and _\"Hahah... you blended with our Hadarms ladies so that's why they consider you one of them; that's the way our parents raised us even if you're Saudi\"_. These tweets may cause some confusion that leads the classifiers to classify them as hateful. We also found that some of the misclassified tweets are debatable and that it could be difficult in terms of deciding whether to consider them hate or non-hate, even by human annotators, such as the following tweet: _\"Most of them moved from one radicalism to another; and what we got here in this world are people called the new Arab liberals and you can see what's on their shoulders (different ideas that contradict the simplest principles of liberalism)\"_.  
All these factors led us to conclude that hate speech is still a difficult phenomenon. It is highly dependent on the context and, in fact, can be conveyed in a very polite and genuine form (without using any offensive words or being aggressive).