SOS : Systematic Offensive Stereotyping Bias in Word Embeddings (2022)

Abstract: Systematic Offensive stereotyping (SOS) in word embeddings could lead to associating marginalised groups with hate speech and profanity, which might lead to blocking and silencing those groups, especially on social media platforms. In this [id=stk]work, we introduce a quantitative measure of the SOS bias, [id=stk]validate it in the most commonly used word embeddings, and investigate if it explains the performance of different word embeddings on the task of hate speech detection. Results show that SOS bias exists in almost all examined word embeddings and that [id=stk]the proposed SOS bias metric correlates positively with the statistics of published surveys on online extremism. We also show that the [id=stk]proposed metric reveals distinct information [id=stk]compared to established social bias metrics. However, we do not find evidence that SOS bias explains the performance of hate speech detection models based on the different word embeddings.

## background the term bias is defined and used in many different ways @xcite . there is the normative definition of bias, as its definition in cognitive science: "behaving according to some cognitive priors and presumed realities that might not be true at all" @xcite . there is also the statistical definition of bias as "systematic distortion in the sampled data that compromises its representatives" @xcite . in distributional word representations (word embeddings), the most common methods for quantifying bias are weat, rnd, rnsb, and ect: for weat, the authors were inspired by the implicit association test (iat) to develop a statistical test to demonstrate human-like biases in word embeddings @xcite . they used cosine similarity and statistical significance tests to measure the unfair correlations between two different demographic groups, as represented by manually curated word lists. for rnd, the authors used the euclidean distance between neutral words, like pro-fessions, and a representative group vector created by averaging the word vectors for words that describe a stereotyped group (gender/ethnicity) @xcite . in rnsb, a logistic regression model is first trained on the word vectors of unbiased labeled sentiment words (positive and negative) extracted from biased word embeddings. then, that model was used to predict the sentiment of words that describe certain demographic groups @xcite . in ect, the authors proposed a method to measure how much bias has been removed from the word embeddings after debiasing @xcite . these metrics, except rnsb, are based on the polarity between two opposing points, like male and female, allowing for binary comparisons. this forces practitioners to model gender as a spectrum between more "male" and "female" words, requiring an overly simplified view of the construct, leading to similar problems for other stereotypical types of bias, like racial, religious, transgender, and sexual orientation, where there are more than two categories that need to be represented @xcite . these metrics also use lists of seed words that have been shown to be unreliable @xcite . since we are interested in measuring the systematic offensive stereotypes of different marginalised groups, these metrics would fall short of our needs. as for the rnsb metric, even though it is possible to include more than two identities, the sentiment dimension is represented as positive or negative (binary). but in our case, we are interested in a variety of offensive language targeted at different marginalised groups. ## systematic offensive stereotyping bias our motivation is to reveal whether word embeddings associate offensive language with words describing marginalised groups. in the next section, we will use the sos bias definition provided in the introduction section to measure the sos bias. for our experiments, we used 15 word embeddings: word2vec (w2v); glove wikipedia (glove-wk); glove-twitter (glove-twitter); urban dictionary (ud); chan word ; glove common crawl (glove-cc); glove common crawl large (glove-cc-large); fast-text common crawl (fasttext-cc); fast-text-subwords common crawl (ft-cc-sws); fast-text wiki (ft-wiki); fast-text-subwords wiki (ft-wiki-sws); sentiment specific word embedding (sswe), debias-w2v, p-desip, ## sos bias and hate speech detection in this section, we answer rq4 through a series of experiments on hate speech detection. we trained deep learning models with an embedding layer for the detection of hate speech from hate speechrelated datasets, then computed the correlation of the performance of the different word embeddings to the sos bias score of these embeddings. we used four hate-speech-related datasets that contain different types of hate speech ( to pre-process the datasets, we removed urls, user mentions, retweet abbreviation "rt", non-ascii characters, and english stop words except for second-person pronouns like "you/yours/your", and third-person pronouns like "he/she/they", "his/her/their" and "him/her/them", as suggested in @xcite . all letters were lowercased, and common contractions were converted to their full forms. and each dataset was randomly ## conclusion in this work, we built on our initial work @xcite where the sos bias was introduced and proposed methods to measure it, validate it, compare it to stereotypical social bias, and investigate if it explains the performance of the word embeddings on hate speech detection. results show that the examined word embeddings are sos biased and that the sos bias in the word embeddings has a strong positive correlation with published statistics on online extremism. however, more datasets need to be collected to provide stronger evidence, especially data from social sciences on the offences that marginalised groups receive on social media. nonetheless, this is an informative finding as it reveals the bias in the dataset that these word embeddings were trained on. since not all these datasets are available to the public, measuring the sos bias in the word embeddings is an important way to learn about that bias in those datasets. our findings also show that the proposed sos bias reveals different information than the types of bias measured by existing metrics. finally, our findings show no evidence that the sos bias, measured using different bias metrics, explains the performance of the different word embeddings on the task of hate speech detection. this finding suggests that the sos bias, and potentially other biases in general, are not strongly related to word embeddings' performance on the downstream task of hate speech detection. we plan to examine this speculation and study the influence of the sos and social bias on the fairness of hate speech detection models in future work.