Let-Mi: An A rabic L evantine T witter Dataset for Misogynistic Language (2021)

Abstract: Online misogyny has become an increasing worry for Arab women who experience gender-based online abuse on a daily basis. Misogyny automatic detection systems can assist in the prohibition of anti-women Arabic toxic content. Developing such systems is hindered by the lack of the Arabic misogyny benchmark datasets. In this paper, we introduce an Arabic Levantine Twitter dataset for Misogynistic language (LeT-Mi) to be the first benchmark dataset for Arabic misogyny. We further provide a detailed review of the dataset creation and annotation phases. The consistency of the annotations for the proposed dataset was emphasized through inter-rater agreement evaluation measures. Moreover, Let-Mi was used as an evaluation dataset through binary/multi-/target classification tasks conducted by several state-of-the-art machine learning systems along with Multi-Task Learning (MTL) configuration. The obtained results indicated that the performances achieved by the used systems are consistent with state-of-the-art results for languages other than Arabic, while employing MTL improved the performance of the misogyny/target classification tasks.

## introduction social media are the digital tribunes where people can express their thoughts and opinions with the ultimate freedom. however, this freedom comes at a price, especially when it enables spreading abusive language and hate speech against individuals or groups. misogyny is one type of hate speech that disparages a person or a group having the female gender identity; it is typically defined as hatred of or contempt for women @xcite . according to @xcite , based on the misogynistic behavior, misogynistic language can be classified into several categories such as discredit, dominance, derailing, sex-ual harassment, stereotyping and objectification, and threat of violence. during the last decade, online misogynistic language has been recognized as a universal phenomenon spread across social media platforms such as facebook and twitter. similar to their peers worldwide, women in the arab region are subjected to several types of online misogyny, through which gender inequality, lower social status, sexual abuse, violence, mistreatment and underestimation are, unfortunately, reinforced and justified. moreover, in specific contexts, online misogyny may evolve into systematic bullying campaigns, launched on social media to attack and, sometimes, violently threaten women who have a powerful influence over the society; as it is the case with female journalists/reporters @xcite . the automatic detection of online misogynistic language can facilitate the prohibition of antiwomen toxic contents. while many efforts have been spent in this domain for indo-european languages, the development of arabic misogyny detection systems is hindered by the lack of the arabic annotated misogyny resources. building such resources involves several challenges in terms of data collection and annotation, especially with the drastic lexical variations among the arabic dialects and the ambiguity introduced by some underrepresented dialects such as the levantine dialect. levantine is one of the five major varieties of arabic dialects. people in syria, lebanon, palestine, and jordan are considered the native speakers of levantine with minor differences from one country to another @xcite . the 17 th october 2019 protests in lebanon witnessed a heavy use of twitter especially by the journalists @xcite . while covering the protests on the ground, many female journalists were, unfortunately, prone to gender-based abuse receiving misogynistic replies for the tweets they post. in this paper, we introduce the first arabic levantine twitter dataset for misogynistic language (let-mi) to be a benchmark dataset for automatic detection of online misogyny written in the arabic and levantine dialect foot_0 . the proposed dataset consists of 6,550 tweets annotated either as neutral (misogynistic-free) or as one of seven misogyny categories: discredit, dominance, cursing/damning, sexual harassment, stereotyping and objectification, derailing, and threat of violence. the credibility and consistency of the annotations was evaluated through using inter-annotator agreement measures: agreement without chance correction(cohen's kappa) and overall inter-annotator agreement @xcite . in addition, we benchmark sota approaches on the dataset to evaluate the effectiveness of current approaches for misogyny detection task in the arabic language. ## let-mi dataset let-mi can be described as a political dataset as it is composed of tweet replies scraped from the timelines of popular female journalists/reporters during october 17 th protests in lebanon. in the following subsections, we provide a qualitative overview of the proposed dataset, while an annotation evaluation is presented in section 3.5. ## experiments and evaluation in this section, we describe our experiments on the let-mi data. we evaluate the performance of sota models on our dataset. we design our experiments at three levels (tasks): ## conclusion and future work this work proposes let-mi, the first misogyny detection dataset for the levantine dialects of the arabic language. the dataset is annotated manually by a set of levantine native speakers. we present a detailed description of the whole annotation process. additionally, we present an experimental evaluation of several machine learning systems, including sota systems. also, we employ an mtl configuration to investigate its effect on the tasks. the results show that the performances of the used systems are consistent with sota results on the other languages, and mtl improved the performance of the used model on two of the proposed tasks. in future work, we plan to create a multi-label version of our dataset for the categories classification task as we found that some tweets can be considered under two or more categories. moreover, since we found many misogynistic tweets that are sarcastic in our analysis, we plan to study the correlation between sarcasm and misogyny tasks.