HATE - ITA : Hate Speech Detection in I talian Social Media Text (2022)

Abstract: Online hate speech is a dangerous phenomenon that can (and should) be promptly counteracted properly. While Natural Language Processing supplies appropriate algorithms for trying to reach this objective, all research efforts are directed toward the English language. This strongly limits the classification power on non-English languages. In this paper, we test several learning frameworks for identifying hate speech in Italian text. We release HATE-ITA, a multi-language model trained on a large set of English data and available Italian datasets. HATE-ITA performs better than mono-lingual models and seems to adapt well also on language-specific slurs. We hope our findings will encourage the research in other mid-to-low resource communities and provide a valuable benchmarking tool for the Italian community.

## introduction online hate speech is a dangerous phenomenon that can (and should) be promptly counteracted properly. while natural language processing supplies algorithms to achieve that, most research efforts are directed toward the english language. indeed, there is now a plethora of approaches and corpora @xcite @xcite @xcite , that can be adopted for addressing english hate speech detection. however, this choice strongly limits the classification power in other languages where fewer resources are available, like italian. researchers have put a great effort into improving italian models @xcite @xcite . however, previous work does not address the task systematically, resulting in no clear evidence of the performance of these models. consider also that a competitive baseline for hate speech detection in italian multi-lingual models give a possible way out of this issue. @xcite shows that combining multiple languages in training can help overcome the apparent limitations of hate speech detection models. we start from those conclusions to build up our work by collecting a large dataset of english hate speech data that we combine with some data in italian. we use this new collection to train multi-lingual models and show the performance and examples across different italian datasets. the contribution of this short workshop paper is thus straightforward: we thoroughly evaluate and release to the community a set of models for italian hate speech detection obtained through fine-tuning of multi-lingual models (hate-ita). foot_0 these models are wrapped in high-level api that will allow the community to access and use these models for future research easily. these models set a new baseline on two state-of-the-art hate speech detection datasets in italian. to the best of our knowledge, this is the first paper that showcases the use of a large english dataset in combination with a small portion of italian to create a robust resource for hate speech detection in italian. contribution 1) our experiments show that multi-lingual models can effectively be used to cover missing ground in some mid-to-low resource languages; 2) while providing researchers with strong baselines, our models can also be used to study which areas and targets are still not yet covered, thus guiding directions for future research (see section 4.4). we release hate-ita as an open-source python library foot_1 . ## experimental methodology our experimental setup illustrates three aspects: 1) the performance of the different models on a train, validation, and test setup that we construct on our data, 2) the performance on different datasets (also considering two new additional datasets that we take as out-of-domain) and 3) a qualitative evaluation section in which we use explainability methods to assess which words are contributing more to the prediction. ## related work national evaluation campaigns and shared tasks played a significant role in releasing non-english corpora for hate speech detection @xcite @xcite . indeed, the research of hate speech detection in italian in mono-lingual settings mainly revolves around the datasets @xcite @xcite released for shared tasks @xcite @xcite @xcite @xcite . in nlp, the scarcity of data in languages beyond english has generated an interest in zero-shot learning @xcite @xcite @xcite and the application of this to hate speech detection methods @xcite @xcite @xcite @xcite . in particular, aluru et al. (2020) exploited several deep learning models and multi-lingual embeddings for performing an extensive analysis on 16 datasets in 9 different languages in few-and zero-shot learning settings. rodríguez et al. (2021) use the pre-trained language agnostic bert sentence embeddings @xcite obtaining good results. other research efforts focused on translating english data to enrich data availability in other languages with mixed results: ibrohim and budi (2019) shows that translations do not bring good results using traditional machine learning classifiers. however, more sophisticated pipelines of translation and pre-training can indeed provide some improvement over standard benchmarks @xcite . ## conclusion this paper presents a novel resource for italian hate speech detection on social media text, hate-ita. researchers can use this new set of models to assess the quality of new systems by providing a more reliable benchmark. however, this is just the first step. indeed, we do not claim to have released the final model for italian hate speech detection; hate-ita requires careful benchmarking to understand if it can accurately capture hate speech on other targets.