Domain Classification-based Source-specific Term Penalization for Domain Adaptation in Hate-speech Detection (2022)

Abstract: State-of-the-art approaches for hate-speech detection usually exhibit poor performance in out-of-domain settings. This occurs, typically, due to classifiers overemphasizing source-specific information that negatively impacts its domain invariance. Prior work has attempted to penalize terms related to hate-speech from manually curated lists using feature attribution methods, which quantify the importance assigned to input terms by the classifier when making a prediction. We, instead, propose a domain adaptation approach that automatically extracts and penalizes source-specific terms using a domain classifier, which learns to differentiate between domains, and feature-attribution scores for hate-speech classes, yielding consistent improvements in cross-domain evaluation.

## introduction while recent state-of-the-art hate-speech classifiers @xcite @xcite yield impressive performance on indomain held-out instances, they suffer when evaluated on out-of-domain settings @xcite @xcite . the distributions across corpora/domains foot_0 change due to varying vocabulary, topics of discussion over time @xcite , data bias caused by sampling strategies @xcite and different hate-targets. this is concerning since curating new data resources for hate-speech involves substantial time and effort @xcite . this calls for strategies, like domain adaptation (da) approaches, that can adapt models trained on existing labeled resources to a new target domain that lacks class-labels. however, research on da in hate-speech is limited @xcite @xcite . typically, vanilla classifiers tend to learn more from domain-specific features @xcite than domain-invariant features, resulting in poor out-ofdomain performance. for instance, @xcite show that in a hate-speech dataset @xcite , neutral domain-specific terms, like 'football', 'commentator', etc., discussing the role of women in sports, are highly correlated with the hate label, restricting its generalizability. thus, it is worth minimizing the importance of such terms for improving cross-domain performance. recently, feature attributions -methods for extracting post-hoc model explanations, have been used to align features with prior domain knowledge @xcite . these provide importance scores to the input terms as per their contribution towards the model prediction @xcite . for instance, @xcite ; @xcite reduce the oversensitivity of classifiers on a curated list of identity terms (e.g. muslims, gay) by penalizing their importance. however, newly emerging social-media terms @xcite ) may render such lists non-exhaustive. @xcite do not use any list but they require human-provided refinement advice as inputs. @xcite further show that post-hoc explanation methods might not provide faithful explanation in out-of-domain settings. the contemporaneous work by @xcite and @xcite reduce lexical overfitting automatically with entropy-based attentions and feature attributions, respectively. while cross-domain classification performance across different datasets is not studied in the former, the latter needs some labeled target instances to identify the over-fitted terms. in the task of detecting objects in images, zunino et al. (2021) use a domain classifier, trained to dif-ferentiate between domains, to visually identify the irrelevant background information to be domainspecific. thus, they enforce the model explanations to align with the ground-truth annotations highlighting the objects in the image. inspired by this, we propose a new da approach in hate-speech employing a domain classifier, but without having access to such annotations for aligning the attribution scores. we hypothesize that domain-specific terms that are simultaneously predictive of the hate-speech labels are instrumental in restricting the domain invariance of the hate-speech classifier. to this end, we employ a domain classifier to automatically extract the terms that help in identifying the source domain compared to the unlabeled target domain, and feature-attribution scores to identify the subset important for hate-speech classification from the source. our method, through penalization of these terms, automatically enforces the source domain classifier to focus on domain-invariant content. compared to approaches transforming highdimensional intermediate representations to reduce the domain discrepancy, such as domain adversarial learning @xcite , our approach makes the adaptation more explainable, while improving the overall cross-domain performance compared to prior-approaches. ## proposed approach given training data from a labeled source domain d train s and an unlabeled target domain d train t , our approach for da in hate-speech involves 2 steps: (i) extraction of source-specific terms and (ii) reducing the importance of these terms. our setting is similar to @xcite and @xcite . ## conclusion we proposed a da approach for automatic extraction and penalization of source domain-specific terms that have higher attributions towards the hate-speech labels, to improve cross-domain hatespeech detection. the results demonstrated consistent improvements on the target domain. these results should motivate further research on domain adaptation in hate-speech and building classifiers that can generalize well to the concept of hate. finally, it would be interesting in applying our approach to other tasks such as rumor and misinformation detection @xcite .