A Federated Approach for Hate Speech Detection (2023)

Abstract: Hate speech detection has been the subject of high research attention, due to the scale of content created on social media. In spite of the attention and the sensitive nature of the task, privacy preservation in hate speech detection has remained under-studied. The majority of research has focused on centralised machine learning infrastructures which risk leaking data. In this paper, we show that using federated machine learning can help address privacy the concerns that are inherent to hate speech detection while obtaining up to 6.81% improvement in terms of F1-score.

## introduction content moderation is a topic that intersects across multiple fundamental rights, e.g., freedom of expression and the right to privacy; and interest groups, e.g. scholars, legislators, civil society, and commercial entities @xcite . the availability of public datasets has been crucial to the development of computational methods for hate speech detection. however, public data contains risks for those whose content is available. on the other hand, privately held data, e.g., data held by corporate entities, holds risks for those who are reporting content. such risks may be actualised through information leaks in models @xcite or the transmission of data @xcite , and can impact people's safety and livelihood. in this work, we apply federated learning @xcite to address the lack of privacy in hate speech detection. fl is a privacypreserving training paradigm for machine learning that jointly optimises for user privacy and model performance. we posit that privacy is necessary for users whose content is flagged and users who are flagging content alike. we thus operationalise privacy, in the context of hate speech detection * equal contribution. and federated learning, to mean privacy in terms of the content of reported content, and the report itself. fl is an apt training paradigm for tasks in which training data is highly sensitive, as fl is designed to mitigate risks of information leaks while also dealing with a high number of end-users, information loss, and label imbalances @xcite @xcite . we apply the fl algorithms fedprox @xcite and adaptive federated optimization @xcite to 5 machine learning algorithms. we evaluate our approach on 8 previously published datasets for hate speech detection. while using fl often implies a trade-off between privacy and performance, we obtain performance improvements of up to 6.81% in f1-score. we find that that models trained using fl outperform centralised models across multiple tests (e.g., derogatory language, spelling variation, and pronoun reference) in hatecheck @xcite . 1 ## prior work although the areas of hate speech detection and fl have each been subject to extensive research, the study of their intersection remains in its infancy. federated learning federated learning is a privacy-preserving machine learning paradigm that aims to reduce privacy risks by decentralising data processing onto client devices (i.e., personal devices), thereby foregoing the need for transmitting "raw" user data, and thus minimising risks of personal data leaks caused by transmission of data. 2 in fl, the machine learning model is located in two places: on a centralised server, and on client devices, which hold instances of the model distributed from the centralised model.client devices use the model to compute model updates. the model updates are then transmitted to the server and aggregated by the centralised model, which is redistributed to the client devices. however, not all transmitted weight updates are aggregated into the model. fl operates with a notion of data loss in its design, which is emulated by selecting a fraction of clients whose updates are aggregated. thus, fl paradigm uses less data to train a models. in our experiments, we apply two fl algorithms: fedprox and fedopt @xcite . fedprox introduces a proximal term to the federated averaging algorithm @xcite . fe-davg averages the weights computed on participating client devices in a round. fedprox introduces a proximal term that functions as a regulariser to the weight updates transmitted by participating clients, which penalises local weight updates that diverge from the global model. the fedavg algorithm can thus be understood as a special case of fedprox with the proximal term set to 0.0. fedopt @xcite extends the adaptive optimisation strategies from centralised optimisation (e.g., adam (kingma and ba, 2015) and adagrad @xcite ) to explicitly account for client and server optimisation. fedopt handles server optimisation distinctly from client optimisation, by introducing a state to the server-side optimisation routine. . this distinct handling of server-side optimization enables more accurate and heterogeneity-aware fl models, which can speed up convergence. fl has been applied to a number of tasks, including emoji prediction @xcite 2 see gitelman (2013) for a discussion on 'raw' data. @xcite , next-word prediction for mobile keyboards @xcite , pre-training and fine-tuning large language models @xcite , medical named entity recognition @xcite , and text classification @xcite . for instance, @xcite used fl to fine-tune a distilbert model to perform classification on the 20newsgroup dataset @xcite using three different fl algorithms: fedavg @xcite , fedprox @xcite ) under non-iid partitioning. in a closely related study, basu et al. (2021) apply fl, using the fedavg algorithm to fine-tune large language models to detect depression and sexual harassment from small twitter data samples. they find that using large language models such as bert and roberta outperform distilled language models such as distilbert. our work extends on @xcite by introducing additional fl algorithms and extending to a multi-class setting for hate speech detection. thus, our work extends on prior work by i) applying fl to the task of multi-class hate speech detection, a task which has proven difficult in part due to the complex nature of pragmatics @xcite and hate mongers seeking to evade content moderation infrastructures @xcite ; ii) using the fedprox and fedopt algorithms rather than the fedavg algorithm, thereby reducing model vulnerability to divergent weight updates; and iii) providing an in-depth analysis of federated model performances. hate speech detection prior work on hate speech detection has primarily focused on privacyagnostic machine learning paradigms, using centralised models for classification. such work has investigated a number of machine learning models (e.g. svms @xcite , cnns (park and fung, 2017), and fine-tuned language models (swamy et al., 2019b)) and the development of resources (e.g. @xcite . recently, @xcite proposed a standardisation of classes across 9 publicly available datasets and studied the generalisation capabilities of bert, fasttext, and svm models. in their work they found limited success in inter-dataset generalization. our work thus extends on the task of hate speech detection by introducing privacy-preserving methods to multi-class hate speech detection. in doing so, the privacy of those who flag content and those whose content is flagged remain intact. ## data we combine our dataset using the standardisation schema proposed by @xcite . comb we reuse 8 of the 9 datasets used by fortuna et al. (2021) to form comb. foot_1 comb then consists of the datasets proposed by talat and hovy (2016); davidson et al. (2017); fersini et al. (2018); de gibert et al. (2018); swamy et al. (2019b); basile et al. (2019); zampieri et al. ( 2019 ) and the kaggle toxic comment challenge. 4 we perform a stratified split of all training data into training (70%), validation (10%), and test (20%) sets. 5 data cleaning we address issues of extreme class imbalance in comb by removing the "abusive" category as it only contains 2 documents. following an in-depth analysis of the kaggle dataset we find that the maximum length of tokens in the dataset is 4950 while the median length of tokens in comb is 26. moreover, we find that the longest 1% of documents in the kaggle dataset do not contain unique tokens. removing the longest 1% of comments reduces the maximal document length to 727 tokens (see appendix b.3 for further detail). following our data cleaning processes, comb comes to consist of 293, 300 documents (see ## experiments we experiment with 5 machine learning models in their centralised and federated settings: logistic regression bi-lstms @xcite , fnet (lee-thorp et al., 2022), dis-tilbert @xcite and roberta @xcite . we measure their performance using weighted f1 scores. the centralised models form our baselines, while the federated models form our experimental models. for the logistic regression and bi-lstms, we perform word-level tokenisation using spacy @xcite . for the fnet, distilbert, and roberta, we use the tokenisers provided with each model. foot_4 ## analysis considering the baseline models in while fl often indicates a trade-off between privacy and performance, we find that the best fl models outperform the centralised baselines. we believe that the improved performance stems from the dataset being split into smaller segments, in congruence with findings from prior work. for instance, @xcite show that splitting data into smaller temporal segments helped improve classification performance overall. we believe that a similar effect may be evident with fl models that, by design split data into small segments and disregard a fraction of the clients. further, it may be the case that some data within hate speech datasets hinders generalisation. only using subsets of the data for training may therefore aid generalisation. ## conclusion private and sensitive data can risk being exposed when developing and deploying models for hate speech detection. we therefore examine the use of federated learning, a privacy preserving machine learning paradigm to the task of hate speech detection to emphasise privacy in hate speech detection. we find that using federated learning improves on the performance levels achieved using centralised models, thus affording both privacy and performance. in future work, we intend to examine interpretability and explainability for federated learning to gain a better understanding of the causes of such performance increases.