A Legal Approach to Hate Speech – Operationalizing the EU ’s Legal Framework against the Expression of Hatred as an NLP Task (2022)

Abstract: We propose a ‘legal approach’ to hate speech detection by operationalization of the decision as to whether a post is subject to criminal law into an NLP task. Comparing existing regulatory regimes for hate speech, we base our investigation on the European Union’s framework as it provides a widely applicable legal minimum standard. Accurately deciding whether a post is punishable or not usually requires legal education. We show that, by breaking the legal assessment down into a series of simpler sub-decisions, even laypersons can annotate consistently. Based on a newly annotated dataset, our experiments show that directly learning an automated model of punishable content is challenging. However, learning the two sub-tasks of ‘target group’ and ‘targeting conduct’ instead of a holistic, end-to-end approach to the legal assessment yields better results. Overall, our method also provides decisions that are more transparent than those of end-to-end models, which is a crucial point in legal decision-making.

## introduction social media provides the platform for the expression of opinions along with their widespread dissemination. unrestricted freedom of expression, however, bears the risk of harming certain groups of people -rendering the regulation of hate speech a potential instrument against discrimination. to do so at scale, automated detection systems are required to aid the moderation process. while research on hate speech detection is wellestablished, defining 'hate speech' remains challenging. datasets encode all kinds of (partly incompatible) notions of hatefulness or offensiveness @xcite @xcite ) that make it difficult to decide which postings would justify restricting freedom of speech through dele-tion. ultimately, a subset of especially hateful content can be considered punishable by law and thus would not fall under the legal right to freedom of expression. as there exist competing legal standards for the regulation of hateful expressions, the selection requires discussion. competing legal standards on the international level, article 4 of the 'international convention on the elimination of all forms of racial discrimination (icerd)' foot_0 binds the signatory states to punish incitement to racial discrimination against any race or group of persons of another colour or ethnic origin by their respective national law. however, the convention does not cover discrimination based on religion and is limited in its legal effect, as various states have made reservations. this is especially the case for the u.s., where the expression of hatred toward any group is constitutionally widely protected by the free speech clause of the first amendment @xcite . consequently, as u.s. law does not provide for any legal provision prohibiting hate speech as an act of speech, it cannot serve as a base for a detection system. in europe, however, the prevention of discrimination against and segregation of a target group (thereby ensuring the members' acceptance as equal in a society) is considered such an important prerequisite for democracy that it may justify the restriction of free speech. the council of europe has set up an additional protocol to the 'convention on cybercrime', concerning the criminalization of acts of a racist and xenophobic nature committed through computer systems. 2 however, the protocol has not been ratified or even signed by all member states of the council of europe and is subject to several reservations. 3 legally and practically more relevant is the following instrument: the european union (eu) has, after long debate, set up a common regime with a framework decision 4 that fully binds all of its member states to make incitement to hatred or violence a punishable criminal offense. the framework also affects u.s. social-media platforms as long as the offender or the material hosted is located within the eu. its importance has also been emphasized by the 'eu code of conduct on countering illegal hate speech online' that the eu commission agreed with it companies like facebook, twitter, and youtube. 5 furthermore, the eu's new digital services act creates new obligations for large online platforms regarding illegal content. 6 the regulation will be directly applicable in all eu member states from 17 february 2024, and also apply to providers established outside the eu if they provide their services to recipients in the union. the eu commission has also started an initiative to add to the list of eu crimes in art. 83(1) tfeu "all forms of hate crime and hate speech, whether because of race, religion, gender or sexuality". 7 this would allow the commission to replace the existing framework decision by a new directive further elabolarting on a more extensive notion of hate speech incrimination. to date, the eu framework decision not only provides a minimum standard for handling hate speech by criminal law, but it is also the regime that -in connection with the new digital services act -triggers the broadest regulatory obligations for large platform providers inside and outside the eu. as figure 1 shows, each member state may still go beyond the framework's minimum requirements and define higher standards. germany, for instance, provides for a broader definition of the possible protected target group by including 'sections of the population', e.g. refugees otherwise not being covered as they cannot clearly be distinguished by race, ## feasibility study based on the above considerations we obtained a decision tree that will serve as a basis for annotation, but also as a logical high-level fundament for our classifiers (figure 2 ). it should not be confused with decision trees as machine learning algorithms, as we work with fine-tuned bert for training our classifiers (see section 5). to test our decision tree annotation scheme, we first perform a feasibility study, where we assess the quality of annotations produced by our annotation scheme against direct annotation. we also assess the reliability of an assessment by legal experts to establish an upper bound for this task. setup we asked public prosecutors from one of the two cybercrime prosecution centers in germany to provide the ground truth for punishability based on §130 of the german criminal code -which implements the eu framework. 12 as prosecutors would be obliged to open an investigation for each punishable post, we provided a set of 156 'madeup' hate speech posts in german. these were never openly published and are thus not punishable. 13 the prosecutors did not use our decision tree, but decided based on their legal training and expertise. as a control condition, we asked layperson annotators to perform a direct annotation. annotators were provided with the legal text of §130 and decided whether a post was punishable using their understanding of the legal code. finally, we asked layperson annotators to follow our multi-label annotation scheme, from which we can automatically derive whether a post is punishable or not, depending on the combination of our labels. ## punishable hate speech dataset in this section, we describe how the full dataset was created. all posts in the dataset are in german. ## automated detection holistic baseline approach to study the extent to which our annotated data can serve as a basis for automated detection, we train a baseline classifier that takes a post as input and estimates whether the post is punishable. at this step, as we do not take our decision tree into consideration, we may refer to this baseline classifier as a "holistic approach". for model training, differences between annotators were adjudicated by a legal expert who made the final decision on the correct label. the agreement numbers reported in section 4 are thus not applicable for the following experiment. fine-tuned bert @xcite models have proven to be strong baselines for various nlp tasks, so we follow this practice foot_8 , using gbertbase @xcite . the model is trained for 20 epochs using a batch size of 16 and nll loss. for optimization, we choose bias-corrected adam (kingma and ba, 2015), with a learning rate of 2e -5 . the learning rate is linearly increased up to its peak during the first 10% of training and then linearly decreased. these choices follow the recommendations of @xcite for increasing training stability when fine-tuning bert. for evaluation, we perform a stratified 10-fold cv. we also include a random classification baseline for the punishable label. we set @xmath0.5 and thus yield a recall of 0.5 and a precision that corresponds to the overall ratio of punishable posts in the dataset, i.e. 0.04. the fine-tuned model achieves an average f 1 of .39 (p .69; r .28), which shows that legal hate speech classification is complex and not easily solvable by baseline models with a holistic approach. the mere presence of language inciting hate or violence is not a sufficient signal, but the model needs to learn in addition whether (i) the hate is directed against an object, (ii) the object is a group, (iii) the group is protected under the given law. even among relevant target groups, detection is made difficult by the many surface forms that identify a particular group as the target of the statement. this is exemplified looking at some misclassifications. in example 6.1, no group is targeted; yet, the model classified the post as punishable. in example 6.2, the model erroneously judged the post to be punishable, even though the group is not protected under the eu framework. implicit or metaphorical references to a group were not detected by the model (e.g. example 6.3). submodel + decision tree approach as an alternative to this direct and holistic classification approach, we also trained separate binary classification models for each annotation label. this allowed us to combine the resulting submodels with our decision tree from figure 2 . the decision tree serves as a logical high-level foundation and allows us to employ model training at a much more concrete level. this approach provides a higher degree of transparency with regard to the actual steps of the decision-making process. it may also lead to an overall improved assessment as each submodel has an easier task to learn and thus may provide more accurate results when combined. we trained separate models for the prediction of target groups and targeting conduct using the same setup as described above. the last row in ## generalizing beyond eu law so far, we have presented a case study of operationalizing a specific legal standard (i.e. the eu framework decision). the underlying methodology can be generalized in a straightforward way. instead of directly predicting whether a post is punishable or not, we divide the problem into two subtasks, (i) group detection and (ii) conduct detection, each of which can be tackled separately, depending on the applicable legal regime. by doing so, this approach offers higher explainability of model decisions, an aspect that is crucial for legal decisionmaking. ## related work automated detection of offensive internet discourse has been intensively studied under a variety of names, for instance: abusive language @xcite or content @xcite , ad hominem arguments @xcite , aggression @xcite , cyberbullying @xcite , hate speech @xcite @xcite , offensive language usage @xcite , profanity @xcite , threats @xcite and socially unacceptable discourse @xcite . while most early work focused on english, now there is also a growing body of work in other languages, e.g., german @xcite , italian @xcite @xcite . all of those works use a non-legally informed definition of the construct to be detected. a notable exception is work on slovene by @xcite who relied on annotators interpretation of the legal definition without further breaking down those decisions. there is a body of work in nlp with a legal perspective focused on predicting the outcome of court decisions @xcite @xcite @xcite , to the best of our knowledge our work is the only effort to operationalise a legal framework for hate speech. however, the dependence on existing court decisions makes it difficult to work with legal problems where relevant case law is not available as a data source. to overcome this problem, @xcite translated statutory rules for defamatory offenses into a series of annotatable binary decisions. the importance of finding groups for hate speech analysis has also been stressed by @xcite . as offenses against groups are often implicitly framed, @xcite introduce social bias frames that make the attacked group explicit. as group detection can work with any set of group categories, it can also be adapted to cover non-western groups @xcite . ## conclusion we operationalize a 'legal approach to hate speech' by translating the requirements of the eu framework decision into a series of annotation steps that can be reliably performed by laypersons. however, we show that learning a holistic, end-to-end model of whether a post is punishable remains challenging. we thus propose to tackle two subtasks instead: group detection and conduct detection. depending on the applicable legal framework, a final decision on the legal status of a comment can then be derived from the combination of the detected group and conduct. relying on subtasks comes with the added benefit of increased transparency and explainability compared to black-box models. this is crucial for systems that potentially interfere with human rights, such as the balance between freedom of expression and the prevention of discrimination. hence, we recommend this modular approach as the preferred way of composing systems for legal decision-making.