Towards Conceptualization of “Fair Explanation”: Disparate Impacts of anti- A sian Hate Speech Explanations on Content Moderators (2023)

Abstract: Recent research at the intersection of AI explainability and fairness has focused on how explanations can improve human-plus-AI task performance as assessed by fairness measures. We propose to characterize what constitutes an explanation that is itself “fair” – an explanation that does not adversely impact specific populations. We formulate a novel evaluation method of “fair explanations” using not just accuracy and label time, but also psychological impact of explanations on different user groups across many metrics (mental discomfort, stereotype activation, and perceived workload). We apply this method in the context of content moderation of potential hate speech, and its differential impact on Asian vs. non-Asian proxy moderators, across explanation approaches (saliency map and counterfactual explanation). We find that saliency maps generally perform better and show less evidence of disparate impact (group) and individual unfairness than counterfactual explanations. Content warning: This paper contains examples of hate speech and racially discriminatory language. The authors do not support such content. Please consider your risk of discomfort carefully before continuing reading!

## introduction most work at the intersection of the ai explainability and fairness focuses on how explanations can improve human-ai task performance regarding some fairness criteria. for example, on a recidivism risk assessment task, @xcite evaluate whether two global and two local explanation methods influence the perceived fairness of the ai model results. in an nlp context, a comprehensive review by @xcite lays out how explainability techniques have been developed to tackle * both authors contributed equally to this work. 1 the code and human study data are available at https: //github.com/jiannan-xu/emnlp23_fair_explanation . different sources of biases, including selection, label, model, semantic, and research design biases. this line of research uses explanations as a means to improve fairness measures, but does not consider what constitutes "fair explanations" -namely, explanations that do not bring disparate harm in and of themselves to different users or groups of users. to understand the impact of explanations on different groups, we situate our work in the context of hate speech detection. here, the human-plus-ai system consists of content moderators who check the potentially hateful content flagged by an ai system, and decide whether to keep or delete it. if the ai predictions came with explanations, this process might be faster and less tiring for moderators, and might alleviate their mental discomfort by allowing them to focus less on hateful content. however, the impact of such explanations might differ across moderators, particularly for those who belong to groups that are targets of the hate speech. empirical studies are needed to understand these effects, and to ensure that explanation methods used do not disproportionately harm one group over another. this paper contributes an evaluation of the fairness of explanations by measuring whether they have a disparate impact on different groups of content moderators. we investigate how two types of nlp explanations impact people deciding whether a tweet contains hate speech directed at asian people. we use saliency maps and counterfactual explanations and measure their impacts using five metrics: classification accuracy, label time, mental discomfort, perceived workload, and stereotype activation (increase in agreement with implicitly biased statements). we hypothesize that people who are targets of the hate speech will experience higher mental costs, and thus measure each metric across races (asian vs non-asian). we find that saliency map explanations simultaneously lead to better task performance and less unfairness than counterfactual explanations. ## background we review work on explanations in the context of the content moderation and fairness literature. ## approach: human study we conduct an online human study to quantify the verifiable and subjective impacts of two explanation styles on participants. simulating the human review of tweets flagged by a hate speech detector, participants know that they are presented with tweets automatically classified as hateful, and they are asked to decide whether each tweet is actually hateful or not. they make this decision without any additional information in the baseline condition, and are presented with either a saliency map (figure 1 ) or a counterfactual explanation (figure 2 ) in the treatment conditions. we study two main research questions (rqs): rq1: does either saliency map or counterfactual explanation influence content moderators on measures related to the psychological impact or efficiency of their task performance? rq2: when such impact of an explanation style exists, is it "unfair" across groups/individuals? we quantify the effect of each explanation style via a set of verifiable and subjective metrics. the verifiable metrics are inferred directly based on participants' performance in the main task (hate speech prediction). for the subjective metrics, we measure a set of metrics before and after the main task (when appropriate) to observe how the psychological state of each participant has changed. this study has been approved by the umd institutional review board (irb package: 1941548-3). figure 2: counterfactual explanation interface ## experimental set-up 4.1 dataset we use the covid-hate dataset of 2290 tweets scraped from twitter and manually annotated by @xcite with three labels: 0 (neutral), 1 (counter-hate or asian-supportive), and 2 (hate speech). the annotation in covid-hate was claimed to be performed by two undergraduate students after a rigorous onboarding process and @xcite only kept the labels that both students agreed on (roughly 2/3 of the original annotations), so upon combining their first two classes into a nonhate class, we use their mapped labels as groundtruths for training and evaluation in our study. ## findings after computing the five metrics of interest per participant, we analyze the distribution of responsesacross explanation style and participant race-, visualizing them using box and whisker plots, and comparing the means using t-tests. we only report findings that achieve statistical significance from the t-tests (p-@xmath0.05). 4 since p-value is an indicator of whether an effect exists but not of its magnitude (wasserstein and lazar, 2016), we also report cohen's d score @xcite and its 95% confidence interval from bootstrapping. our results indicate that statistically significant effects presented in the paper have sensible cohen's d score estimates (small to moderate effects that require statistical methods to be observed @xcite ) and reasonable 95% confidence intervals. statistically insignificant or less interesting results for the remaining metrics are shown in appendix g. to examine varying treatment effects for individuals or subgroups in a population, we analyze the heterogeneous treatment effects in appendix f. ## conclusion we conduct a between-subjects human study across three explanation conditions to study potential (disnales left by individuals in different conditions or of different races did not find any statistically significant differences. parate) impacts of saliency map and counterfactual explanations on content moderators, using crowdworkers as a proxy. we find that first, saliency map is the most desirable condition overall. second, counterfactual explanation exhibits a tradeoff in terms of disparate mental discomfort and disparate label time, thereby highlighting the need for legal intervention to minimize risks of labor abuse against content moderators in the targeted minority. however, saliency map is not necessarily innocent as there is marginal evidence of its disparate stereotype activation. third, mandatory introduction of explanation may mitigate disparate perceived workload, but careful selection of explanation style may be needed depending on the racial distribution of the content moderator pool. fourth, asians who experienced mental discomfort are more reluctant to leave a rationale. fifth, counterfactual explanation is more individually unfair than saliency map. our results suggest that even though counterfactual explanations seem less desirable and less "fair" for proxy content moderators, further research is needed to confirm the utility and fairness of saliency map before more adoption of this explanation style by social media platforms. if follow-up research can validate that our findings are generalizable beyond crowdworkers to real content moderators, new ai-related legislation may incentivize social media platforms to implement an effective and "fair" explanation method, e.g. saliency map, on a larger scale. however, if due to logistic constraints, an overall less desirable or "unfair" style, e.g. counterfactual explanations, must be used, stakeholders might consider a raceconscious content moderation policy, e.g. by assigning content that ai classified as targeting racial minorities to moderators of other races to minimize mental impact. in summary, this work aims to responsibly implement explanations methods in the real world, thereby reducing workload and improving psychological well-being without sacrificing job performance, particularly of those sharing demographic characteristics with hate speech victims.