SlowGuess/ACL_2020_OCR huggingface dataset:
Patricia Chirl<sup>1</sup>, Véronique Moriceau<sup>1</sup>, Farah Benamara<sup>1</sup>, Alda Mari<sup>2</sup>, Gloria Origi<sup>2</sup>, Marlène Coulomb-Gully<sup>3</sup>
(1) IRIT, Université de Toulouse, Université Toulouse III - UPS, France {firstname.lastname}@irit.fr
(2) Institut Jean Nicod, CNRS, ENS, EHESS, Paris, France {firstname.lastname}@ens.fr
(3) LERASS, Université de Toulouse, UT2J, France
marlene.coulomb@univ-tlse2.fr
Sexism is prejudice or discrimination based on a person's gender. It is based on the belief that one sex or gender is superior to another. It can take several forms from sexist remarks, gestures, behaviours, practices, insults to rape or murder. Sexist hate speech is a message of inferiority usually directed against women at least in part because they are women, some authors refer to it as: "words that wound" (Matsuda et al., 1993; Waldron, 2012; Delgado et al., 2015). As defined by the Council of Europe, "The aim of sexist hate speech is to humiliate or objectify women, to undervalue their skills and opinions, to destroy their reputation, to make them feel vulnerable and fearful, and to control and punish them for not following a certain behaviour". Its psychological, emotional and/or physical impacts can be severe. In several countries, sexist behaviours are now prohibited. See for example the French law of 27 January 2017 related to equality and citizenship, where penalties due to
discrimination are doubled (sexism is now considered as an aggravating factor), law that extends to the internet and social media.
Although overall misogyny and sexism share the common purpose of maintaining or restoring a patriarchal social order, Manne (2017) illustrates the contrast between the two ideologies. A sexist ideology (which often "consists of assumptions, beliefs, theories, stereotypes and broader cultural narratives that represent men and women") will tend to discriminate between men and women and has the role of justifying these norms via an ideology that involves believing in men's superiority in highly prestigious domains (i.e., represents the "justificatory" branch of a patriarchal order). A misogynistic ideology does not necessarily rely on people's beliefs, values, and theories, and can be seen as a mechanism that has the role of upholding the social norms of patriarchies (i.e., represents the "law enforcement" branch of a patriarchal order) by differentiating between good women and bad women and punishing those who take (or attempt to take) a man's place in society. Considering these definitions, misogyny is a type of sexism. In this paper, as we target French sexist messages detection, we consider sexism in its common French usage, i.e. discrimination or hate speech against women.
Social media and web platforms have offered a large space to sexist hate speech (in France,  $10\%$  of sexist abuses come from social media (Bousquet et al., 2019)) but also allow to share stories of sexism experienced by women (see "The Everyday Sexism Project" available in many languages, "Paye ta shnek" in French, or hashtags such as #metoo or #balancetonporc). In this context, it is important to automatically detect sexist messages
https://rm.coe.int/1680651592
$^{2}$ https://everydaysexism.com/
<sup>3</sup>https://payetaknek.tumblr.com/
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4055-4066 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
on social platforms and possibly to prevent the wide-spreading of gender stereotypes, especially towards young people, which is a first step towards offensive content moderation (see the recommendations of the European commission (COM, 2017). However, we believe that it is important not only to be able to automatically detect messages with a sexist content but also to distinguish between real sexist messages that are addressed to a woman or describing a woman or women in general (e.g., The goalkeeper has no merit in stopping this pregnant woman shooting), and messages which relate sexism experiences (e.g., He said "who's gonna take care of your children when you are at ACL?"). Indeed, whereas messages could be reported and moderated in the first case as recommended by European laws, messages relating sexism experiences should not be moderated.
As far as we are aware, the distinction between reports/denunciations of sexism experience and real sexist messages has not been addressed. Previous work considers sexism either as a type of hate speech, along with racism, homophobia, or hate speech against immigrants (Waseem and Hovy, 2016; Golbeck et al., 2017; Davidson et al., 2017; Basile et al., 2019; Schrading et al., 2015) or study it as such. In this latter case, detection is casted as a binary classification problem (sexist vs. non-sexist) or a multi-label classification by identifying the type of sexist behaviours (Jha and Mamidi, 2017; Sharifirad et al., 2018; Fersini et al., 2018b; Karlekar and Bansal, 2018; Parikh et al., 2019). English is dominant, although Italian and Spanish have already been studied (see the IberEval 2018 (Fersini et al., 2018b), EvalIta 2018 (Fersini et al., 2018a) and HateEval 2019 (Basile et al., 2019) shared tasks).
This paper proposes the first approach to detect different types of reports/denunciations of sexism experiences in French tweets, based on their impact on the target. Our contributions are:
(1) A novel characterization of sexist content-force relation inspired by speech acts theory (Austin, 1962) and discourse studies in gender (Lazar, 2007; Mills, 2008). We distinguish different types of sexist content depending on the impact on the addressee (called 'perlocutionary force'): sexist hate speech directly addressed to a target, sexist descriptive assertions not addressed to the target, or reported assertions that relate a story of sexism experienced by a woman. This is presented
in Section 3. Our guiding hypothesis is that indirect acts establish a distancing effect with the reported content and are thus less committal on behalf of the addressee (Giannakidou and Mari, 2021). Our take on the issue is language-driven: reported speech is indirect, and it does not discursively involve a call on the addressee to endorse the content of the act.
(2) The first French dataset of about 12,000 tweets annotated for sexism detection according to this new characterization. Data and manual annotation are described in Section 4.
(3) A set of experiments to detect sexist content in three configurations: binary classification (sexist content vs. non-sexist), three classes (reporting content vs. non-reporting vs. non-sexist), and a cascade classifier (first sexist content and then reporting). We rely on deep learning architectures trained on top of a combination of several tweet's vectorial representations: word embeddings built from different sources, linguistic features, and various generalization strategies to account for sexist stereotypes and the way sexist contents are linguistically expressed (see Section 5). Our results, presented in Section 6, are encouraging and constitute a first step towards automatic sexist content moderation.
Gender in discourse analysis. Discourse analysis studies have shown that sexism may be expressed at different linguistic granularity levels going from lexical to discursive (Cameron, 1992): e.g., women are often designated through their relationship with men or motherhood (e.g., A man killed in shooting vs. Mother of 2 killed in crash) or by physical characteristics (e.g., The journalist who presents the news vs. The blonde who presents the news). Sexism can also be hostile (e.g., The world would be a better place without women) or benevolent where messages are subjectively positive, and sexism is expressed in the form of a compliment (e.g., Many women have a quality of purity that few men have) (Glick and Fiske, 1996). In communication studies, the analysis of political discourse (Bonnafous, 2003; Coulomb-Gully, 2012), sexist abuse or media discourse (Dai and Xu, 2014; Biscarrat et al., 2016) show that political women presentations are stereotyped: use of physical or clothing character-
4https://github.com/patriChiril/An-Annotated-Corpus-for-Sexism-Detection-in-French-Tweets
istics, reference to private life, etc. From a sociological perspective, studies focus on social media contents (tweets) or SMS in order to analyze public opinion on gender-based violence (Purohit et al., 2016) or violence and sexist behaviours (Barak, 2005; Megarry, 2014).
Gender bias in word embeddings. Bolukbasi et al. (2016) have shown that word embeddings trained on news articles exhibit female/male gender stereotypes. Several algorithms have then been proposed to attenuate this bias (Dev and Phillips, 2019) or to make embeddings gender-neutral (Zhao et al., 2018), although Gonen and Goldberg (2019) consider that bias removal techniques are insufficient. Debiased embeddings were used by Park et al. (2018) observing a decrease in sexism detection performance compared to the non-debiased model. To overcome this limitation, Badjatiya et al. (2019) propose neural methods for stereotypical bias removal for hate speech detection (i.e., hateful vs. non-hateful). They first identify a set of bias sensitive words, then mitigate their impact by replacing them with their POS, NER tags, K-nearest neighbours and hypernyms obtained via WordNet.
Automatic sexism detection. To our knowledge, the automatic detection of sexist messages currently deals only with English, Italian and Spanish. For example in the Automatic Misogyny Identification (AMI) shared task at IberEval and EvalIta 2018, the tasks consisted in detecting sexist tweets and then identifying the type of sexist behaviour according to a taxonomy defined by (Anzovino et al., 2018): discredit, stereotype, objectification, sexual harassment, threat of violence, dominance and derailing. Most participants used SVM models and ensemble of classifiers for both tasks with features such as n-grams and opinions (Fersini et al., 2018b). These datasets have also been used in the Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter shared task at SemEval 2019. Best results were obtained with an SVM model using sentence embeddings as features (Indurthi et al., 2019).
There are also a few notable neural network techniques. Jha and Mamidi (2017) employ an LSTM model to classify messages as: benevolent, hostile and non-sexist. Zhang and Luo (2018) implement two deep neural network models (CNN + Gated Recurrent Unit layer and CNN + modified CNN layers for feature extraction) in order to classify social media texts as racist, sexist, or non-hateful. Karlekar
and Bansal (2018) use a single-label CNN-LSTM model with character-level embeddings to classify three forms of sexual harassment: commenting, ogling/staring, and touching/groping. Sharifirad et al. (2018) focus on diverse forms of sexist harassment (indirect, information threat, sexual, physical) using LSTM and CNN on augmented dataset obtained via ConceptNet  $is-a$  relationships and Wikidata. Finally, (Parikh et al., 2019) consider messages of sexism experienced by women in the "Everyday Sexism Project" web site and classify them according to 23 non mutually exclusive categories using LSTM, CNN, CNN-LSTM and BERT models trained on top of several distributional representations (character, subwords, words and sentence) along with additional linguistic features.
In this paper, we propose different deep learning architectures to detect reporting of sexist acts and, more importantly, distinguishing them from real sexist messages. We explore BERT contextualized word embeddings trained from several sources (tweets, Wikipedia) complemented with both linguistic features and generalization strategies. These strategies are designed to force the classifier to learn from generalized concepts rather than words, which may be rare in the corpus. We, therefore, adopt several replacement combinations based on a taxonomy of stereotyped gendered words coupled with additional sexist vocabularies extending Badjatiya et al. (2017) approach designed for hate speech detection to sexism content detection.
3 Characterizing Sexist Content
Propositional content can be introduced in discourse by acts of varying forces (Austin, 1962): it can be asserted (e.g., Paul is cleaning up his room), questioned (e.g., Is Paul cleaning up his room?), or asked to be performed as with imperatives (e.g., Paul, clean up your room!). In philosophy of language, on the one hand, and feminist philosophy on the other, speech acts have already been advocated in a variety of manners. Most accounts however either focus on the type of act (assault-like, propaganda, authoritative, etc.) that derogatory language performs (Langton, 2012; Bianchi, 2014) or concentrate on the analytical level at which the derogatory content is interpreted, whether it provides meaning at the level of the presupposition (or more largely non at-issue content (Potts, 2005)) or of the assertion (Cepollaro, 2015).
We have chosen to distinguish cases where the
addressee is directly addressed from those in which she is not, as done in hate speech analysis. For example, Waseem et al. (2017) and ElSherief et al. (2018) consider that directed hate speech is explicitly directed at a person while generalized hate speech targets a group. For (Ousidhoum et al., 2019), a hateful tweet is direct when the target is explicitly named, or indirect when "less easily discernible". Unlike these approaches and the definitions of target used in (Basile et al., 2019; Fersini et al., 2018a), we do not consider the number of targets of a sexist message (it can indifferently be a woman, a group of women or all women) but rather distinguish the target from the addressee. Our use of the notions of directness and indirectness are also transverse to the ones used in (Lazar, 2007; Chew and Kelley-Chew, 2007) or (Mills, 2008), who resort to the label indirectness for subtle forms of sexism that perpetuate gender stereotypes through humor, presuppositions, metaphors, etc.
We newly consider three different stages in the scale of 'directedness' of an assertion: assertions directed to the addressee, descriptive assertions not directed to a particular addressee and reported assertions. All these three types of acts can contain subtle and non-subtle sexist content. The main goal of our classification is thus to focus on the impact of the content by resorting to the force of the act and not only to its content.
Sexist content in directed assertions is explicitly addressed at a target, but contrary to other approaches cited above, the target can be a woman, a group of women or all women. Across the different classifications of speech acts (Portner, 2018), 'direct' speech acts such as imperatives are addressee-oriented and they require that the addressee performs an action (responding (with questions) or acting (with imperatives)). Indirect speech acts are not addressee-oriented. Assertions themselves can be direct or indirect. They are direct when they are in the second person ('you'), as shown in (1) and (2) (linguistic clues are underlined)<sup>5</sup>. They require that the addressee be committed to the truthfulness of their content. Since a direct sexist assertion is a type of speech act that immediately involves the addressee and triggers a request of commitment,
direct assertions of sexism have been ranked as the most prominent expressions of sexism with a greater impact on the victim. Most prominently, with assertions, directedness is the trigger of perlocutionary content, rendering the assertion an 'insult'.
(1) T'es une femme je serait jamais d'accord avec toi pour du foot (You're a woman I'll never agree with you about football)
(2) les femmes qui sont en plus Dijonnaise ne parlez pas de foot sivouplai c'est comme si un aveugle manchot parler de passer le permis (women who are also from Dijon please don't talk about football it's as if a one-handed blind person was thinking about getting a driving license)
Descriptive assertions are not directed to an addressee: the target can be a woman, a group of women, or all women, it can be named but is not the addressee. Descriptive assertions are in the third person and thus may have a lower impact on the receiver in comparison with second person assertions. They do not commit the addressee to the truth of the content by soliciting a response. They report generic content (Mari et al., 2012). Linguistic clues can be the presence of a named entity as the target or use of generalizing terms, as shown in (3) and (4).
(3) Anne Hidalgo est une femme. Les femmes aiment faire le menage. Anne Hidalgo devrait donc nettoyer elle-même les rues de Paris (Anne Hidalgo is a woman. Women love cleaning the house. Anne Hidalgo should clean the streets of Paris herself)
(4) une femme a besoin d'amour de replir son frigo, si l'homme peut le lui apporter en contrepartie de ses services (ménages, cuisine, etc) j'vois pas elle aurait besoin de quoi d'autre (A woman needs love, to fill the fridge, if a man can give this to her in return for her services (housework, cooking, etc), I don't see what else she needs)
Finally, in reported assertions, the sexist content is a report of an experience or a denunciation of a sexist behaviour. They may elicit an even lower commitment on behalf of the addressee. The speaker is not committed to the truth of a reported content (as in I heard that you were coming
5The translations might not feel natural. Indeed, we kept the same words in English as in French in order to better illustrate the type/semantic of words that are used, keeping in mind that tweets are often not well-written in French as well as in English.
too). However, when reporting sexist content, the speaker is still conveying a lack of commitment, and a general sense of disapproval or dismissal may emerge. In these messages, we observe the presence of reporting verbs, quotation, locations (as reports often mention public spaces where the experience happened) or specific hashtags, as shown in (5), (6) and (7).
(5) je m'assoupis dans le métro, je rouvre les yeux en sentant quelque chose de bizarre : la main de l'homme assis à côté de moi sur ma cuisse. #balancetonporc
(I doze in the subway, I open my eyes feeling something weird: the hand of the man sat next to me on my leg #SquealOnYourPig)
(6) Mon patron m'a demanded: "qui va cuisineur pour ton mari quand tu seras pas la?"
(My boss asked me: "who's going to cook for your husband when you're away?")
(7) Je ne suis pas une grande fan de Theresa May mais pourquoi parler de "ses escarpins et ses cuissardes vernies" et la traiter d'allumeuse ? #vincenthervouet #sexisme http://eur1.fr/nADYIMw
(I am not a fan of Theresa May but why talking about "her shoes and varnished boots" and call her a tease? #vincenthervouet #sexism)
As it appears, the three types of assertions have a sexist content, but only the first two ones are really sexist. Indeed, direct and descriptive assertions are first-hand information, whereas reported ones are second-hand information. As such, they may trigger a different reaction from the receiver: in the first two cases, a female receiver can be immediately involved as the target of the sexist dismissal; in the third case, she is the witness of a sexist report.
4 Data and Annotation
Our corpus is new and contains French tweets collected between October 2017 and May 2018. In order to collect sexist and non sexist tweets, we followed Anzovino et al. (2018) approach using: (i) a set of representative keywords: femme, fille (woman, girl), enceinte (pregnant), some activities (cuisine (cooking), football, ...), insults, etc., (ii) the names of women/men potentially victims or guilty of sexism (mainly politicians), (iii) specific hashtags to collect stories of sexism experiences<sup>6</sup>:
balancetonporc, #sexisme, #sexiste, #SexismeOrdinaire, #EnsembleContreLeSexisme, #payetaknehek, #payetontaf, etc. The tweets collected with these hashtags may contain reported sexist acts towards both men and women. Thus, we collected around 205,000 tweets, among which about 70,000 contain the specific hashtags.
Given a tweet, annotation consists in assigning it one of the following five categories: direct, descriptive, reporting (as defined in the previous section), non-sexist and no decision. A tweet is non-sexist when it has no sexist content (it may contain a specific hashtag, but the content is not sexist), as in (8). No decision refers to cases where the tweet lacks context, or when the sexist content is not in the text but only in a photo, video, or URL (because we cannot process them).
(8) La créatrice du #balancetonporc attaquée en justice pour diffamation
(France's #MeToo creator on trial for defamation)
300 tweets have been used for the training of 5 annotators (they are master's degree students (3 female and 2 male) in Communication and Gender) and then removed from the corpus. Then, 1,000 tweets have been annotated by all annotators so that the inter-annotator agreement could be computed. Although the perception of sexism is often considered as subjective, the average Cohen's Kappa is 0.72 for sexist content/non-sexist/no decision categories and 0.71 for direct/descriptive/reporting/non-sexist/no decision categories which means a strong agreement. We noticed that the kappa scores between female annotators are very close to the one between male annotators. For these 1,000 tweets, the final labels have been assigned according to a majority vote.
Finally, a total of 11,834 tweets have been annotated according to the guidelines after removing 1,053 tweets annotated as "no decision". Among them,  $65.80\%$  are non-sexist and  $34.20\%$  with sexist content (79.61% reporting,  $1.12\%$  are direct and  $19.27\%$  descriptive). We then divided the corpus into train and test sets<sup>7</sup> (cf. Table 1).
5 Identifying Reports of Sexist Acts
To identify reported assertions, we performed three classification tasks: (BIN) sexist content vs. non
<sup>6</sup>The distribution of these hashtags is very similar in both non sexist and sexist tweets which reduces considerably the bias while collecting the data.
7All the hyperparameters were tuned on the validation set (20% of the training dataset), such that the best validation error was produced.
Sexist content 4,047	Non sexist 7,787
Train	direct+descriptive	reporting	6,255
38 + 599 ( = 637)	2,559
Test	direct+descriptive	reporting	1,532
7 + 181 ( = 188)	663
Table 1: Tweet distribution in train/test datasets.
sexist, (3-CLASS) sexist tweets (i.e., direct and descriptive) vs. reporting tweets vs. non-sexist; and (CASC) a cascade classification with sexist content vs. non-sexist in the first stage, followed by reporting vs. non-reporting in the second stage. To this end, we experiment with several deep learning models<sup>8</sup> including best performing state of the art models for sexism detection.
CNN. This model has already been used in Karlekar and Bansal (2018). It uses pre-trained on Wikipedia and Common Crawl FastText French word vectors and three 1D Convolutional layers, each one using 100 filters and a stride of 1, but different window sizes (2, 3, and 4 respectively) with a ReLU activation function. We further downsample the output of these layers by a 1D max pooling layer (with a pool size of 4), and we feed its output to the final softmax layer.
CNN-LSTM. This model is similar to Karlekar and Bansal (2018) and (Parikh et al., 2019) except that we used word-level embeddings instead of character/sentence-level as the results were lower. It is based on the previous CNN model by adding an LSTM layer<sup>9</sup> (capable of capturing the order of a sequence) that takes its input from the max pooling layer. Next, a global max pooling layer feeds the highest value in each timestep dimension to a final softmax layer.