A Computational Exploration of Pejorative Language in Social Media (2021)

Abstract: In this paper we study pejorative language, an under-explored topic in computational linguistics. Unlike existing models of offensive language and hate speech, pejorative language manifests itself primarily at the lexical level, and describes a word that is used with a negative connotation, making it different from offensive language or other more studied categories. Pejorativity is also context-dependent: the same word can be used with or without pejorative connotations, thus pejorativity detection is essentially a problem similar to word sense disambiguation. We leverage online dictionaries to build a multilingual lexicon of pejorative terms for English, Spanish, Italian, and Romanian. We additionally release a dataset of tweets annotated for pejorative use. Based on these resources, we present an analysis of the usage and occurrence of pejorative words in social media, and present an attempt to automatically disambiguate pejorative usage in our dataset.

## introduction with the increase of social media usage, the issue of toxic language has become an important problem in our society. automatic methods are needed to help mitigate this problem, and for this reason the study of toxic speech in nlp has become very popularity in recent years. different categories and definitions have been proposed, including hate speech @xcite , offensive language @xcite , aggression @xcite , as well as further sub-categories depending on the targets, such as women, migrants, etc. @xcite . from a computational perspective, the problem is usually approached as a classification task at the post level, where a classifier is trained to predict whether a social media post contains offensive/toxic language. in this paper we address the question of pejorative words. pejorative words are words or phrases that have negative connotations or that are intended to disparage or belittle foot_0 . pejorativity is closely related to the notion of slurs or insults: "as noun phrases, 'insult' and 'slur' refer to symbolic vehicles designed by convention to derogate targeted individuals or groups" @xcite . while pejorative language is often used in offensive speech @xcite , they are not identical categories. there are offensive posts that do not use pejorative words (e.g. "women belong in the kitchen"), and pejorative uses of words that are not harmful ("what a shitty chair") because the offensive content is not targeted at a person or a group as described in the popular annotation taxonomy of the offensive language identification dataset (olid) @xcite . words can have a negative meaning in one context and not in others (such as the figurative meanings of "trash" or "pussy"); or be pejorative in one language or culture, and not in others (such as the romanian "cioara" (literally, "crow") -a slur for people of color). slurs can also lose their pejorative meaning through semantic change (e.g. the word "queer" went through semantic amelioration over the years -it used to be a slur and is losing its negative connotation @xcite ). recognizing the complexity of the phenomenon, with its linguistic subtleties as well as the variability related to culture and context, are important to successfully recognize pejorative words and by extension offensive posts and hate speech. pejorative language is still largely underexplored in computational linguistics. there are very few studies addressing or taking pejorative language into account @xcite @xcite @xcite . a few related works to ours include @xcite who focused on pejorative connotations for nominalized adjectives and @xcite who built a lexicon of vulgar terms (and vulgarity scores) for german based on derogatory terms found in wiktionary. in this study, we address this important gap by leveraging dictionaries to build a multilingual lexicon of pejorative language for four languages. we compare the occurrence of pejorativity in social media with other established categories of toxic language, relying on existing hate speech corpora. unlike most existing studies in hate speech and offensive language identification, our paper focuses on the lexical level and approaches the issue of ambiguity in toxic language, formulating the problem of pejorativity detection as a word sense disambiguation (wsd) task. the main contributions of this work are the following: ## pejorative tweet dataset for building a data set of english texts containing words that are used pejoratively, we started by looking at three datasets of hate speech on twitter: @xcite , @xcite . for spanish, we employed the same technique of filtering tweets. we looked at the spanish tweets data set provided by @xcite and considered only the binary label for hate speech classification. out of the total of 5, 000 tweets, we have extracted 1, 621 hateful examples and 1, 667 non-hateful examples that contain words from our spanish pejorative lexicon. ## classification experiments the classification task we approached was inferring the 0/1 label for tweet-word pairs. namely, given a word and a tweet, where the word appears in the tweet, we want to be able to say if the word was used pejoratively or not in that tweet. in order to prepare our data, for each tweet-word pair, the tweet was tokenized and the position of the occurrence of the word was found among the tokens. then, we generated a contextual embedding @xcite for that occurrence, by employing various bert models, pre-trained on english texts, provided by the huggingface python library @xcite . the embedding obtained for the specified position is computed by summing the 768-dimensional hidden states generated for that position by each of the 12 layers of the bert architecture. we note that, for out-of-vocabulary words, the bert tokenizer provided by the huggingface library splits them into sub-words. in this case we chose to generate the embeddings for each of the sub-words of our word occurrence and then average them to obtain the final 768-dimensional embedding. figure 3 illustrates an example of uses of a pejorative word ("cracker") in the pejor2 dataset, by representing its embeddings reduced to two dimensions using pca. we can see that most of the similar labelled examples are clustered together. figure 3 : 2d plot of the contextual embeddings generated for the word 'cracker' in the pejor2 data set, for each of its occurrences in the tweets, using a pretrained bert model. embeddings were reduced to two dimensions using pca. for classification on our english data set, we grouped the pairs by the pejorative word contained in the tweet, and independently for each group, we fitted a classifier on the contextual embeddings @xcite . for extracting the embeddings we used various transformer models (bert base @xcite , bertweet (nguyen et al., 2020), roberta @xcite , multilingual bert @xcite ) and for the classification algorithm we used k-nearest neighbors, support vector machines (svm), multilayer perceptron (mlp). for k-nearest neighbors, we considered the cosine similarity as the distance function and found through hyper-parameter tuning that neighborhoods of size 4 were the best performing setting. for evaluation, we employed a 5-fold crossvalidation. performance metrics were computed for each word independently, measuring the capacity of distinguishing the pejorative and non-pejorative usage of the word in different contexts. we report, for each metric, the value resulted by averaging over the scores obtained for all of the word groups. we leave out from this averaging the words that appear with only one label in the whole data set (only pejorative or only non-pejorative), since they will be always classified correctly regardless of the contextual embeddings. we also employed a baseline that based on the training data it learns to predict only the most frequent label. we notice a promising performance of the classifiers in distinguishing pejorative usage, of up to 0.86 f1-score. following the best performing models for each data set, overall 107 samples were misclassified in the pejor1 dataset, while for pejor2 there were 37. words in pejor2 seem slightly easier to classify, which might be expected given the dataset is more balanced in positive and negative examples. for the spanish pejorative data set, since most of the examples were not labelled, we tried an unsupervised clustering approach. for each group of example pairs defined by the common pejorative word, we extracted contextual embeddings using the same previously explained method. using kmeans clustering, we grouped those embeddings into two classes. we then computed, using the annotated examples, the amount of overlap between those two clusters and the pejorative la-bels. the overlap was computed as the accuracy and the macro-f1 score of the clusters when used for predicting the labels. we averaged the scores computed for all of the groups where there was at least one positively and one negatively labelled example. the results obtained using various embeddings (beto @xcite and multilingual bert @xcite ) can be found in ## conclusions we have addressed an important but under-explored lexical category in the intersection of lexical semantics and toxic speech: pejorativity. we released a public lexicon of pejorative words in four languages (including a low-resource language), as well as dataset of tweets annotated for pejorative uses of words. 4 we have modelled pejorativity detection as a problem of disambiguation, and performed experiments using state-of-the-art contextual embeddings in order to automatically distinguish pejorative from non-pejorative uses of words, obtaining promising results. in the future, we would like to explore modelling the problem of pejorativity detection as a sequence labelling task. at the application level, integrating pejorativity detection into hate speech detection systems, for example, would be a promising area for future research. from a linguistic perspective, it would be interesting to analyze occurrence and pejorative value cross-lingually taking advantage of large pretrained cross-lingual models as in @xcite for offensive language identification. we expect pejorative connotations to be difficult to translate and not transfer well across languages, which could also have practical implications. we would also like to extend our dataset of social media posts to cover more pejorative terms, as well as other languages.