Data Integration for Toxic Comment Classification: Making More Than 40 Datasets Easily Accessible in One Unified Format (2021)

Abstract: With the rise of research on toxic comment classification, more and more annotated datasets have been released. The wide variety of the task (different languages, different labeling processes and schemes) has led to a large amount of heterogeneous datasets that can be used for training and testing very specific settings. Despite recent efforts to create web pages that provide an overview, most publications still use only a single dataset. They are not stored in one central database, they come in many different data formats and it is difficult to interpret their class labels and how to reuse these labels in other projects. To overcome these issues, we present a collection of more than thirty datasets in the form of a software tool that automatizes downloading and processing of the data and presents them in a unified data format that also offers a mapping of compatible class labels. Another advantage of that tool is that it gives an overview of properties of available datasets, such as different languages, platforms, and class labels to make it easier to select suitable training and test data.

## conclusions and future work in this paper, we addressed three challenges that hinder accessibility of research datasets of toxic comments: retrieving the datasets, unifying their file formats, and mapping their class labels to a common subset. to overcome these challenges, we present the toxic comment collection, which does not contain the datasets themselves, but code that automatically fetches these datasets from their source and transforms them into a unified format. its advantages are the easy access to a large number of datasets and the option to filter by language, platform, and class label. with the toxic comment collection, we aim to foster repeatability and reproducibility of research on toxic comments and to allow research on multilingual toxic comment classification by combining multiple datasets. we are continuously adding more datasets to the collection with routines to download them and to standardize their format automatically, e.g., we plan to integrate the datasets by @xcite and @xcite next. we also plan to add contact information and instructions for datasets that are not publicly accessible but available only on request, such as the datasets by @xcite @xcite . 2019 6.0k twitter hi hate,offense,profanity,target 29 @xcite 4.7k twitter de hate,offense,profanity,target 30 @xcite 7.0k twitter en hate,offense,profanity,target 31 @xcite 5.8k twitter ar abuse,hate 32 @xcite 5.6k twitter fr abuse,hate,offense,target @xcite 5.6k twitter en abuse,hate,offense,target 34 @xcite 4.0k twitter en abuse,hate,offense,target @xcite 3.3k twitter ar abuse,hate,offense,target 36 @xcite 22.3k forum en hate 37 @xcite 33.8k forum en hate 38 @xcite 13.2k twitter en offense 39 c @xcite 36.0k twitter tr offense,target 40 @xcite 4.8k twitter el offense 41 sigurbergsson and derczynski (2020) 3.6k misc da offense,target 42 @xcite 15.9k twitter mr negative 43 kralj @xcite 60.0k twitter sl offense,profanity,target,violent a argument,discrimination,feedback,inappropriate,sentiment,personal,off-topic b aggression,hate,irony,offense,stereotype c derailment,discredit,harassment,misogyny,stereotype,target d abuse,aggression,cyberbullying,hate,offense,spam