Question: How do the authors collect the Arabic tweets?
Evidence: Our target was to build a large Arabic offensive language dataset that is representative of their appearance on Twitter and is hopefully not biased to specific dialects, topics, or targets. One of the main challenges is that offensive tweets constitute a very small portion of overall tweets. To quantify their proportion, we took 3 random samples of tweets from different days, with each sample composed of 1,000 tweets, and we found that between 1% and 2% of them were in fact offensive (including pornographic advertisement). This percentage is consistent with previously reported percentages BIBREF19. Thus, annotating random tweets is grossly inefficient. One way to overcome this problem is to use a seed list of offensive words to filter tweets. However, doing so is problematic as it would skew the dataset to particular types of offensive language or to specific dialects. Offensiveness is often dialect and country specific.
After inspecting many tweets, we observed that many offensive tweets have the vocative particle يا> (“yA” – meaning “O”), which is mainly used in directing the speech to a specific person or group. The ratio of offensive tweets increases to 5% if each tweet contains one vocative particle and to 19% if has at least two vocative particles. Users often repeat this particle for emphasis, as in: يا أمي يا حنونة> (“yA Amy yA Hnwnp” – O my mother, O kind one), which is endearing and non-offensive, and يا كلب يا قذر> (“yA klb yA q*r” – “O dog, O dirty one”), which is offensive. We decided to use this pattern to increase our chances of finding offensive tweets. One of the main advantages of the pattern يا ... يا> (“yA ... yA”) is that it is not associated with any specific topic or genre, and it appears in all Arabic dialects. Though the use of offensive language does not necessitate the appearance of the vocative particle, the particle does not favor any specific offensive expressions and greatly improves our chances of finding offensive tweets. It is clear, the dataset is more biased toward positive class. Using the dataset for real-life application may require de-biasing it by boosting negative class or random sampling additional data from Twitter BIBREF22.Using the Twitter API, we collected 660k Arabic tweets having this pattern between April 15, 2019 and May 6, 2019. To increase diversity, we sorted the word sequences between the vocative particles and took the most frequent 10,000 unique sequences. For each word sequence, we took a random tweet containing each sequence. Then we annotated those tweets, ending up with 1,915 offensive tweets which represent roughly 19% of all tweets. Each tweet was labeled as: offensive, which could additionally be labeled as vulgar and/or hate speech, or Clean. We describe in greater detail our annotation guidelines, which we made sure that they are compatible with the OffensEval2019 annotation guidelines BIBREF15. For example, if a tweet has insults or threats targeting a group based on their nationality, ethnicity, gender, political affiliation, religious belief, or other common characteristics, this is considered as hate speech BIBREF15. It is worth mentioning that we also considered insulting groups based on their sport affiliation as a form of hate speech. In most Arab countries, being a fan of a particularly sporting club is considered as part of the personality and ideology which rarely changes over time (similar to political affiliation). Many incidents of violence have occurred among fans of rival clubs.
Using Twitter APIs, we collected 660k Arabic tweets having this pattern between April 15 – May 6, 2019.
Answer: The authors collected 660k Arabic tweets having this pattern between April 15 – May 6, 2019 by using Twitter APIs.