Cleansing & expanding the HURTLEX (el) with a multidimensional categorization of offensive words (2022)

Abstract: We present a cleansed version of the multilingual lexicon HURTLEX-(EL) comprising 737 offensive words of Modern Greek. We worked bottom-up in two annotation rounds and developed detailed guidelines by cross-classifying words on three dimensions: context, reference, and thematic domain. Our classification reveals a wider spectrum of thematic domains concerning the study of offensive language than previously thought Efthymiou et al. (2014) and reveals social and cultural aspects that are not included in the HURTLEX categories.

## introduction the term offensive language (ol) is used to describe "hurtful, derogatory or obscene comments made by one person to another person" and the term hate speech (hs) to describe speech that is possibly harmful to disadvantaged social groups. 2 although both legal and ethical aspects have been considered in an effort to differentiate between hs and ol, the line between the two terms is difficult to be drawn @xcite and they are often used interchangeably @xcite . in this work, terms in the domains of ol and hs are considered together. many of the studies referring to ol detection use vocabularies @xcite @xcite @xcite or patterns as a starting point and depend heavily on the selection of "seed words". keywordbased approaches might be more effective in the case of explicit abuse according to the typology provided in @xcite . also, there are strong indications that key-word and lexicon-based 1 the lexicon is available here: https://osf.io/t5jey/?@xmath0: doi 10.17605/osf.io/t5jey). resource development for ol detection is an issue in itself. firstly, "offense" is a subjective notion and as a result, the social (in general) and personal characteristics of the annotators as well as the annotation method may put bias on the resources for ol detection (lists of offensive words, corpora). the so-called "descriptive" approaches to resource development try to represent various stances in the same resource while the so-called "prescriptive" approaches try to represent few or even only one stance. high interranotator scores seem to correlate with the prescriptive approach @xcite . furthermore, @xcite point out that little is known about the creation process and the theoretical concepts underlying collections of offensive words. the context in which words occur also affects their offensive nature; for instance, @xcite observe that words collected in vulgar lexicons, sometimes may be considered neutral or even positive. our group represents female native speakers of mg with middle to high education aged 20-60; none belongs to marginal social groups. our work is of the prescriptive persuasion. we did not make use of a pre-existing list of guidelines for recognising offensive words; instead we developed our own list of diagnostics with an iterative bottomup procedure. we offer a cleansed version of the hurtlex-(el) lexicon containing 737 words after removing the wrong words and the words that were not considered offensive by all the annotators. explanations whether the ol value of the words is context-dependent or not are offered as well as descriptions of certain contexts that trigger the offensive meanings. ## ol identification studies and resources for modern greek pitenis et al. (2020) presented the first annotated mg dataset, the offensive greek tweet dataset (ogtd) that was extracted with a yet unpublished list of profane or obscene keywords (e.g., μαλάκας 'asshole', πουτάνα 'whore'). tweets were marked as "offensive", "not offensive" or "spam". as "offensive" were labelled tweets that contained profane or obscene language or when they could be considered offensive on the basis of the context @xcite . these general annotation guidelines were meant for texts. lekea and karampelas (2018) has investigated hs in the context of terrorist argument drawing on an also unpublished list of 1265 words. perifanos and goutsos (2021) have combined visual and textual cues in a multimodal approach for hs detection on twitter. 4004 tweets with the hashtag #απέλαση 'deportation' and the term λάθρο 'illegal' were annotated manually as hateful, xenophobic and racist by 3 annotators with the majority vote. overall, the literature on modern greek ol detection does not provide annotated corpora representing a wide range of registers, sizeable ol lexica or annotation guidelines. in this context, and given that lexical resources are crucial for ol identification when few or no labelled corpora exist @xcite , the greek (el) branch of hurtlex @xcite seemed a promising starting point. hurtlex is a domain-independent lexicon of 53 languages with offensive, aggressive and hateful words. its kernel consists of ∼1000 manually selected words corresponding to 17 fine-grained thematic categories that were enriched in a semiautomatic manner by drawing on the multiwodrnet synsets and babelnet. 3,4 in hurtlex each lemma-sense pair is classified as "non-offensive" or "neutral" or "offensive". the neutral cases were further divided into "not literally pejorative" and "negative connotation" (not a directly derogatory use). an agreement of 61% between two annotators was reported. the senses judged as non offensive were removed and two versions of the lexicon were received: one containing the translations of offensive senses and one with the additional distinction concerning the neutral cases. notably, hurtlex aims to support the development of resources for underrepresented languages @xcite . ol has been discussed in the context of mg lexicography. @xcite show that the classification of the negative terms as derogatory, offensive, slang and taboo words in two celebrated dictionaries of mg, the lneg2 @xcite and the lkn @xcite do not converge. in ## working with hurtlex-(el) although filtering has been applied to prevent noise propagation in the semi-automatically enriched hurtlex, its el branch still includes synsets with no offensive meaning and incorrect terms. first, we manually removed clearly incorrect terms. two linguists agreed that these included: (i) foreign words (384 words; either in english or french), (ii) combinations of greek and foreign words (33 words), i.e., ευρασίας griffon, lit. eurasia's griffon, (iii) about 194 meaningless phrases, i.e., πουτίγκα κεφάλι, lit. pudding head, (iv) terms with morphological errors (23 words) i.e., φυσιογνωμονική 'physiognomic' instead of φυσιογνωμική 'physiognomic' (v) agreement errors (46 words), i.e., σεξουαλικά επίθεση, instead of σεξουαλική επίθεση 'sexual assault' (vi) different inflectional forms of the same lemma (298 words); mg makes heavy use of inflectional morphology and hurtlex seemed unable to filter out types in the same inflectional paradigm, and (vii) archaic words (37 words), i.e., αιχμαλωτίζων 'capturer' which is an active present participle of a verb still used in mg but these particular participles belong to older forms of the language. at this stage, annotators also removed words that they all considered "unoffensive" in mg, i.e., μοτσαρέλα 'mozzarella'. 2143 words (about 69% of the original hurtlex-(el) contents) were retained out of the 3114 original entries of hurtlex -(el). given the growing body of literature @xcite @xcite ) emphasizing the role of context in characterising a word as offensive, we adopted an annotation schema with three categories, namely offensive (context-independent), context (context-dependent), following the distinction introduced in @xcite , and non-offensive entries. representative examples were provided for terms assigned the label "context-dependent". next, four independent annotators, all undergraduate linguists who offered volunteer work, assigned one of the three labels: context-independent, context-dependent, non-offensive. general diagnostics of offensiveness mainly about profane and obscene language were offered as suggestions at this stage. the interannotator agreement score in this first step was 0.77 (fleiss kappa), which indicates an already substantial agreement. in the final step, a somewhat different annotation procedure was adopted (see @xcite for a similar approach). the four annotators were provided with a set of more detailed diagnostics of offensiveness, e.g.: "names of animals that are stereotypically related with negative properties in the greek culture, such as ugliness, e.g., φώκια 'seal' or dirt, e.g., γουρούνι 'pig', are offensively used when they target individuals." these diagnostics were not developed on the basis of the classification of offensive words in the original hurtlex or in the mg lexica (section 2); instead, we preferred to work bottom-up and develop our own diagnostics. the motivation for this decision was that the rich material in hurtlex-(el) would present more classification challenges than the material in greek printed lexica and that a greek group's idea of offensiveness might not be identical to that of hurtlex, a possibility that is recognised by the hurtlex developers @xcite . the annotators were asked to consult these diagnostics when classifying the terms as un/offensive but (i) they might propose changes such as deletions, additions and redefinitions of categories (ii) a term might fit to more than one category. the annotators would meet with the group leaders to discuss the diagnostics. there were three rounds of this procedure and eventually the system of thematic categories was developed as a set of diagnostics for recognising offensive words in modern greek; this system is presented in section 4. lastly, the labels context-independent, contextdependent and non-offensive were reassigned independently by the annotators and an interannotator agreement fleiss kappa score of 0.96 was received. we did not resort to majority vote so only 737 terms that were shared by all the four annotators were included in the final lexicon; of them, as "context independent" were marked 448 words and as "context dependent" 289 words. ## annotation diagnostics prose in this section should be read with constant reference to below are given indicative terms and clarifications regarding the identified 17 thematic categories listed in figure 1 presents the distribution of words per diagnostic. behavior is the most populated diagnostic followed by crime & immoral behavior and animals. 5 comparison to hurtlex-(el) hurtlex relies on a classification of ol words in 17 categories @xcite . we have defined our own diagnostics in a bottom-up iterative fashion (section 3). ## conclusions and future work we have discussed our experience regarding the development of an openly available, cleansed version of the greek branch of hurtlex; in doing so, we have defined diagnostics of offensiveness that will be useful in future offensive word and text categorisation tasks. this was the first step in a longer-term effort that aims to offer reasonable mg lexica and corpora for the task of ol detection. on the lexicon development front we plan to study the effect of evaluative morphology on ol @xcite , enlarge the lexicon semiautomatically drawing on corpora @xcite and test its coverage and contribution to ol identification tasks using texts from a variety of registers. on the corpora development front, we intend to use the lexicon in order to leverage corpora for ol detection and for a variety of registers.