OLEA : Tool and Infrastructure for Offensive Language Error Analysis in E nglish (2023)

Abstract: State-of-the-art models for identifying offensive language often fail to generalize over more nuanced or implicit cases of offensive and hateful language. Understanding model performance on complex cases is key for building robust models that are effective in real-world settings. To help researchers efficiently evaluate their models, we introduce OLEA, a diagnostic, open-source, extensible Python library that provides easy-to-use tools for error analysis in the context of detecting offensive language in English. OLEA packages analyses and datasets proposed by prior scholarship, empowering researchers to build effective, explainable and generalizable offensive language classifiers.

## introduction offensive language 1 detection models are integral to online platforms' moderation systems. such systems excel at detecting and filtering out messages with explicit keywords and mentions, however these systems are known (1) to perform poorly on messages that are implicitly offensive or have negation @xcite ; models have been shown to be ineffective at generalizing across these complexities @xcite , tending to aggregate different types of hate speech under broad labels, causing large within-class variances @xcite . in response, prior research has curated diagnostic datasets such as hatecheck @xcite and cold @xcite , to evaluate existing models on specific types of hate speech. such evaluation datasets allow us to view model performance as a continuum, and move away from monolithic f1 scores that can obscure a model's limitations and explainability @xcite . we introduce olea, 2,3 an extensible, opensource offensive language error analysis tool and infrastructure designed to a) evaluate offensive language classifiers on different types of problematic language use, and b) provide detailed feedback about model performance. the library makes it convenient for researchers to analyze their models by providing an extensive set of error-analysis methods, callable with minimal coding, to measure case-specific model performances. in addition, the library provides a common interface for comparing different offensive language classifiers on granular linguistic categories. olea provides: ## background and related work offensive language is complex, and systems for detecting it automatically need to be able to handle both explicit and implicit cases @xcite figure 1 : diagram of the olea library pipeline. @xcite . detecting explicit offensive language often relies on keyword detection @xcite , but keyworddriven systems can lead to messages being falsely flagged, causing unchecked or unnoticed racial biases to propagate in the system's decisions @xcite . implicit offensive language is generally more difficult to detect than its explicit counterpart @xcite . it is also more likely to change over time, as the world changes, and as users coin new phrases and terms to implicitly refer to minority groups @xcite . datasets for this task address different (often overlapping) concerns, making direct comparison difficult. hatexplain @xcite and cad @xcite both provide rationales indicating where annotators see offensive content. olid @xcite identifies offensive text and the specific targeted minority group in a three-tiered labeling structure. hatecheck @xcite and cold @xcite are described more in section 3.1. linguistic explainability of the prediction failures of nlp models has lagged behind performance gains according to benchmark datasets @xcite . @xcite provide an interactive system mostly for end users of offensive language detection systems. their system helps users explore datasets and understand how different models score and classify individual text inputs.olea has complementary functionality, focusing on fine-grained analysis of model performance (especially misclassifications) across existing evaluation datasets. we focus instead on model developers, providing streamlined error analysis and interpretation of system outputs relative to linguistically-grounded categorizations. ## library tour and design figure 1 shows an overview of olea's core functionalities. users submit their model's predictions (3.2) on olea's preloaded datasets (3.1) and then call error analysis and evaluation functions (3.3). users may also extend olea with new datasets and may write new analysis functions, adding to the library's capabilities (3.4). most of olea's modules expect a pandas foot_2 dataframe with the text of the instance to be classified, one or more labels indicating offensiveness, and a predicted label for the instance. dataframes may include columns with additional information related to the instance and/or its annotation. ## use case demonstrations olea aids model development by providing an easy and comparable platform to test and build robust offensive language classifiers. we demonstrate three use cases: a) analysis on preloaded datasets (4.1), including model comparison (4.1.4), b) analysis on custom data (4.2), and c) sharing datasets and analysis functions (4.2). ## conclusion and future directions this paper introduces olea, a tool for easy, indepth error analysis and an infrastructure for sharing new datasets and analysis methods. olea helps researchers understand the strengths and weaknesses of their offensive language detection models. in the near term, we will continue to add new analysis methods and datasets, including methods for corpus exploration, and providing automatic trends and insights of model performance without users needing to run explicit analyses. mid term, we plan to extend olea to additional languages, and eventually we would like to expand olea into a general error analysis library for a range of language classification tasks. because olea is a convenient way for authors to share datasets and analyses, it is our hope that a community will develop around the library, and that models ultimately will improve as we learn more about what they can and cannot do.