Towards Conceptualization of ``Fair Explanation'': Disparate Impacts of anti-Asian Hate Speech Explanations on Content Moderators
Authors: Tin Trung Nguyen
Jiannan Xu
Aayushi Roy
Hal Daumé III
Marine Carpuat
Venue: EMNLP 2023 Main (2023)
Keywords: fairness
explainability
human study
hate speech prediction
content moderators
crowdworkers
TL;DR: Saliency Map is better overall and has less evidence of group and individual ``unfairness'' than Counterfactual Explanation for content moderators in the hate speech prediction task.

Abstract: Recent research at the intersection of AI explainability and fairness has focused on how explanations can improve human-plus-AI task performance as assessed by fairness measures. We propose to characterize what constitutes an explanation that is itself "fair" -- an explanation that does not adversely impact specific populations. We formulate a novel evaluation method of "fair explanations" using not just accuracy and label time, but also psychological impact of explanations on different user groups across many metrics (mental discomfort, stereotype activation, and perceived workload). We apply this method in the context of content moderation of potential hate speech, and its differential impact on Asian vs. non-Asian proxy moderators, across explanation approaches (saliency map and counterfactual explanation). We find that saliency maps generally perform better and show less evidence of disparate impact (group) and individual unfairness than counterfactual explanations.

Content warning: This paper contains examples of hate speech and racially discriminatory language. The authors do not support such content. Please consider your risk of discomfort carefully before continuing reading!

Towards Conceptualization of "Fair Explanation": Disparate Impacts of anti-Asian Hate Speech Explanations on Content Moderators
Tin Nguyen, 1' Jiannan Xu,"" Aayushi Roy, Hal Daumé II, t8 Marine Carpuati
*Department of Computer Science, University of Maryland Robert H. Smith School of Business, University of Maryland Microsoft Research
(tintn, jiannan, aroy2530, hal3, marine remmineeedd.... eddu
Abstract
different sources of biases, including selection, la- bel, model, semantic, and research design biases. This line of rescarch uses explanations as a means to improve fairness measures, but does not consider what constitutes "fair explanations" - namely, ex- planations that do not bring disparate harm in and of themselves to different users or groups of users.
Recent research at the intersection of AI ex- plainability and faimesss has focused on how explanations can improve human-plus-Al Al task performance assessed by fairness measures. We propose to characterize what constitutes an explanation that is itself "fair" an expla- nation that does not adversely impact specific populations. We formulate a novel evaluation method of "fair explanations" using not just accuracy and label time, but also psycholog- ical impact of explanations on different user groups across many metries (mental discom- fort, stereotype activation, and perceived work- load). We apply this method in the context of content moderation of potential hate speech, and its differential impact on Asian vs. non- Asian proxy moderators, across explanation approaches (saliency map and counterfactual explanation). We find that saliency maps gen- erally perform better and show less evidence of disparate impact (group) and individual un- faimess than counterfactual explanations.
To understand the impact of explanations on dif- ferent groups, we situate our work in the context of hate speech detection. Here, the human- human-plus- AI system consists of content moderators who check the potentially hateful content flagged by an Al sys- tem, and decide whether to keep or delete it. If the AT predictions came with explanations, this process might be faster and less tiring for moderators, and might alleviate their mental discomfort by allowing them to focus less on hateful content. However, the impact of such explanations might differ across moderators, particularly for those who belong to groups that targets of the hate speech. Empiri- cal studies needed to understand these effects, and to ensure that explanation methods used do not dis disproportionately harm one group over another.
Content warning: This paper contains exam- ples of hate speech and racially discriminatory language. The authors do not support such content. Please consider your risk of discom- fort carefully before continuing reading!
This paper contributes evaluation of the fair- ness of explanations by measuring whether they have a disparate impact on different groups of con- tent moderators. We investigate how two types of NLP explanations impact people deciding whether a tweet contains hate speech directed at Asian peo- plc. We use saliency maps and counterfactual ex- planations and measure their impacts using five metrics: classification accuracy, label time, men- tal discomfort, perceived workload, and stereotype activation (increase in agreement with implicitly biased statements). We hypothesize that people who are targets of the hate speech will experience higher mental costs, and thus measure each metric across races (Asian vs non-Asian).
1 Introduction
Most work at the intersection of the Al explainabil- ity and fairness focuses on how explanations can improve Human-AT task performance regarding some fairness criteria. For example, on a recidi- vism risk assessment task, Dodge et al. (2019) eval- uate whether two global and two local explanation methods influence the perceived fairesss of the AT model results. In an NLP context, comprehensive review by Balkir et al. (2022) lays out how explain- ability techniques have been developed to tackle
We find that saliency map explanations simul- tancously lead to better task performance and less unfaimesss than counterfactual explanations.
"Both authors contributed equally this work. 'The code and human study data are available at https: t/uithub.... com/ j jiannan--ann xu/ S///EEMLP22 P23_Fairr Barr__Exppaattl

---

2 Background
Al model. Noting such considerations, first, we use saliency map, which highlights the features in the input that are most important for each prediction and is therefore intuitive for laypersons. We gener- ate saliency maps with LIME (Ribeiro et al., 2016), perturbation- method which determines in- put feature salience for any black-box classifier by approximating its behavior locally with linear model. Second, use counterfactual explana- tions, which are minimally modified inputs which flips the prediction. Counterfactual explanations can viewed as real-life suggested modifications for users to make their tweets no longer hateful. Counterfactual explanations can be generated us- ing language models. Wu et al. (2021) introduce the Polyjuice system to control for different types of counterfactuals and the tokens be edited. This approach can generate counterfactual explanations for hate speech detection by replacing offensive words or factually unsubstantiated claims.
We review work explanations in the context of the content moderation and fairness literature.
Content Moderation. Researchers have looked for ways to support content moderators in their challenging job by addressing not only their work- load, but also their mental discomfort. Exposure to harmful content has been shown to have long-term, negative psychological impact (Newton, 2019;
Steiger et al., 2021; Cambridge Consultants, 2019). While entirely preventing exposure is currently impossible (Steiger et al., 2021), can be mini- mized by having moderators review initial predic- tions made automatically (Cambridge Consultants, 2019). Explanations have been used in these set- tings to explain why social media post is predicted harmful, for instance using dashboard of fea- ture importance (Bunde, 2021), or natural language explanations from ChatGPT (Huang et al., 2023). Such studies evaluate how people perceive these explanations in terms of clarity and usefulness. As a result, we focus on neglected human-centered evaluation dimensions, such as mental discomfort, and verifiable vs. subjective workload.
Approach: Human Study
We conduct an online human study to quantify the verifiable and subjective impacts of two explana- tion styles on participants. Simulating the human review of tweets flagged by hate speech detec- tor, participants know that they are presented with tweets automatically classified as hateful, and they asked to decide whether each tweet is actually hateful not. They make this decision without any additional information in the baseline condition, and are presented with either a saliency map (Fig-
Fairness consideration in Explainable Al mostly take the form of measuring the impact of explanations fairness metrics. For instance, Angerschmid et al. (2022) show that feature-based based input influence explanations improve perceived fair- ness in healthcare. Goyal et al. (2023) show that disclosure of a high correlation between a non- protected feature that used in the model e.g. university and protected feature - c.g. gender- improves group fairness metric in micro-lending task. However, to the best of our knowledge, prior fairness studies have not considered the fairness of the explanations themselves, including whether they have a disproportionate impact on explanation readers across demographic groups.
Fairness.
ure 1) or a counterfactual explanation (Figure 2) in the treatment conditions.
We study two main Research Questions (RQs): RQ1: Does either saliency map or counterfactual explanation influence content moderators on measures related to the psychological impact or efficiency of their task performance?
RQ2: When such impact of an explanation style exists, is it "unfair" across groups/individuals? We quantify the effect of each explanation style via a set of verifiable and subjective metrics. The verifiable metrics are inferred directly based on participants' performance in the main task (hate speech prediction). For the subjective metrics, we measure set of metrics before and after the main task (when appropriate) observe how the psycho- logical state of each participant has changed.
Explanation Methods. Among the wealth of recently proposed explainable NLP methods (Danilevsky et al., 2020), we select two local expla- nation methods that each represent distinct family of strategies for surfacing aspects of text that might be indicative of hate speech. Local explanations methods are better suited than global explanations in the content moderation setting where understand- ing individual (per-tweet) predictions is a priority compared to describing the average behavior of an
This study has been approved by the UMD Insti- tutional Review Board (IRB package: 19415484 148--3).

---

you agree orgttaee Jale??
folwving
"USER Best solrtion carons, Bear @reat Trump will scane the out chenee wiuse"
ane Lake
model preders that follwing 100VI-19 19 related nate?:
TwiT with tightghtn works
QUSER siace busal buy
"SUSER Best solution of corona, Dear @realDnnal this will scare the shitt out of Bllness virue?
ire origna tweet minimally mochfed into telowing peeiatto bate contlete Please nere that hightgtted worde are oefamen the hen vensooe
mosel will
UUSER Best colitin of corona, Coor dhresConac tte wil ocare me one our
Do you agrce deagres the prodiction that the onighnal tweet (COVD-19 resated Adaa howe??
O Bost be hasd 'COOD 1s vablic Aean hale" Bank hod NOT sadda
aeeet metion COVLD os fosar, baticia hoald
hasea
Figure 1: Saliency Map Interface
Figure 2: Counterfactual Explanation Interface
3.1 Metrics
Angry and has been empirically tested on a Chinese population (Li et al., 2013), the main hate speech target here. In our survey, we ask participants to rate their current feeling respect each of the six dimensions, both the start and end of the study. We sum across the six di- mensions to get an aggregate negative affect score, as done by Diener et al. (2009), at both time points, and calculate their difference as the mental discom- fort metric due to participation in the labeling task.
We develop five metrics: two verifiable (accuracy and label time) vs. three subjective (mental discom- fort, stereotype activation, and perceived workload) to evaluate the impact of each explanation style and any differences across racial groups.
five point seale with
Accuracy. To get verifiable measure of hate speech prediction accuracy, we use the ground truth labels by He et al. (2021). We combine their counter-hate and neutral classes into a non-hate class, and provide their definition of "COVID-19 related Asian hate" to our participants: "antagonis- tic speech that is directed towards an Asian entity (individual person, organization, or country), and others the Asian outgroup through intentional op- position or hostility the context of COVID-19." We frame the annotation task as a three-way choice between anti-Asian hate, hate (but not anti-Asian). or non-hate, and consider the first two as positive judgements of hate speech.?
Stereotype Activation. We are interested in whether the hate speech labeling task influences the mental model of participants. Specifically, we ask whether there an increase in agreement with im- plicitly biased statements against Asian people after being exposed to content that contains implicit bias against that group. Several studies from psychol- ogy literature have studied similar phenomena un- der the name "stereotype activation" (Moskowitz, 2010; Wheeler and Petty, 2001).
To quantify stereotype activation (SteAct), we ask participants at the start to rate on a five point scale how much they (dis-)agree with three implicit anti-A Asian statements. 3 At the end of the survey, we have the same question with three paraphrased (semantically identical) but reshuffled statements. Consistent with the activation literature, we assume that exposure to potentially biased content can only increase one's mental stereotyping. We therefore calculate changes in agreement to stercotyped ques- tions as the non-negative part of the difference in
Mental Discomfort. For measuring the mental discomfort (also known as "negative affect" in the psychology literature) one may experience reading potential hate speech, we follow Dang et al. (2018) and Das et al. (2020) and use the Scale of Positive and Negative Experience (SPANE) (Diener et al., 2009). SPANE includes six dimensions for nega- tive affect: Negative, Bad, Unpleasant, Sad, Afraid,
We initially planned to ask participants to classify tweets as hate or non-hate given Al predictions and explanations. However, based on participant rationales from our pilot study, we realized some participants labeled tweets as non-hate speech only because they did not see an explicit mention of "Asian" or "COVID" in the tweet. We introduced the three-way categorization to address this.
Statements: 1. "China and South Korea have bad cating habits, for example: dogs, cats, and horses.": 2. "The Chinese Communist Party (CCP) uses CCTV to spy on innocent Chi- nese people."; 3. "The China Communist Party (CCP) and President Xi need to apologize to the world."

---

agreement levels between the questions the end of the task ($1))) and those at the start (S1:3) as:
non-hate as a counterfactual explanation (in case both counterfactuals of a tweet flipped the label, we selected the first or less modified counterfactual). As an outlier, one tweet had both counterfactuals still classified as hate even though the tweet itself is not hate speech ('Shut up. Taiwan is not China') ) so we took it out, leaving 12 tweets, of which 6 are hate and 6 are non-hate according to the ground truth, to show participants.
SteAct max -0,5 s
(1)
Perceived Workload. To measure a participant's perceived amount of effort, use the NASA Task Load Index (NASA-TLX), which has six dimen- sions: mental, physical, and temporal demands, frustration, effort, and performance (Hart, 2006). At the end of the study, a participant rates each dimension on a seven point scale. We take the sum across the six dimensions as perceived workload.
Experimental Set-up
4.1 Dataset
We use the COVID-HATE dataset of 2290 tweets scraped from Twitter and manually annotated by He et al. (2021) with three labels: O (neutral), 1 (counter-hate Asian-supportive), and 2 (hate speech). The annotation in COVID-HATE was claimed to be performed by two undergraduate stu- dents after rigorous onboarding process and He et al. (2021) only kept the labels that both students agreed on (roughly 2/3 of the original annotations), so upon combining their first two classes into a non- hate class, we use their mapped labels ground- truths for training and evaluation in our study.
Label Time. To obtain a verifiable measure of workload, we calculate the amount of time spent on the twelve hate speech prediction task pages.
3.2 Experimental Conditions
We have conditions: baseline (no explanation). saliency map, and counterfactual explanation. For counterfactual explanations, Polyjuice outputs distort the semantic meaning of the tweets or do not even make sense while counter- factuals generated by ChatGPT are over-corrective and deviate greatly from the original context of the hate-classified tweets as shown in Appendix A. Noting those constraints, to disentangle the ef- fect of cow-quality counterfactual candidates any observed impact of counterfactual explanations, we manually make our own counterfactual candidates and run them through the fine-tuned RoBERTa clas- sifier to see which of them the least modified counterfactual that will flip the label from hate to non-hate, which we use as counterfactual explana- tion. Although the use of human- generated counter- factual candidates limits the generalizability of our findings, study serves a useful Wizard-of- OZ experiment to inform future research on which NLP explanation style warrants more work de- velop and evaluate "fair explanations". To select which hate-classified samples to generate explana- tions for and show participants, we selected from the RoBERTa test set the 13 hate-classified samples with the lowest model confidence (i.e. probability) score they represent relatively balanced dis- tribution of hate v.s. non-hate ground truth labels. After creating two counterfactuals per tweet, we ran those counterfactuals as additional test data through the RoBERTa classifier again and used the counter- factual that flipped the AI prediction from hate to
find that
4.2 Model configuration
We fine-tune a RoBERTa-basee binary classifier for the hate speech classification task. The training details are provided in Appendix B. Since the hate speech label is imbalanced (non-hate: 81.3%, hate: 18. 1877%), we do a stratified splitting that generates 80% of the original data for training and 20% for testing. The model performance reasonable measured by precision (0.7721), recall (0.7093), F-1 score (0.7394) for anti-Asian tweets detection. The model achieves an AUROC score of 0.9273 and an AUPRC score of 0.8399.
4.3 Logisties
To avoid the coupling effect of explanation types and minimize the learning effect, we conduct a subjeen-subjecs human study via Prolific and de- velop our survey in Qualtries.
After a set of initial questions as initial data points for stereotype activation and mental discom- fort,
Qualtrics Randomizer to randomly assign each participant to one of the three expla- nation conditions and get relatively equal numbers of samples across conditions. After participants complete the 12 hate speech prediction tasks, they are redirected to a shared- -across----s----ee conpostionooiionnns set of

---

ending questions measure stereotype activation, mental discomfort, and perceived workload.
sentence, therefore reducing the need to compre- hend the discriminatory intent of the tweet and thus lowers the mental discomfort they experience.
Using a detailed filtering scheme, we get 283 Prolific participants in our data analysis with rel- atively equal distributions across explanation con- ditions and racial groups (Asian v.s. non-Asian). Other details, such as the pay rate, participants fil- tering criteria, baseline condition interface, and the questions supporting the subjective metrics, can be found in Appendix C and Appendix D.
Accuracy As shown in Figure 4, saliency map (M = 0.4759, SD = 0.1036) yields significantly higher hate speech prediction accuracy than coun- terfactual explanation (M = 0.4651, SD = 0. 0.1088), ((188) = 2.3078, p-value 0.022, Cohen's d 0.3367, 95% CI = [0.0518, 0.6328J. This result is consistent with the psychology literature: Reinhard et al. (201 1) find that people make more accurate lie predictions (whether someone lying) in settings where they have higher "situational f'amiliarity.. Although lie and hate speech are not the same, we see several shared attributes of these two concepts, e.g. often originating from bad intent, containing wrong facts. Furthermore, they find that the gain in accuracy is thanks verbal content cues rather than non-verbal content cues. This verbal setting is similar to our online context. Therefore, we expect their findings to be relatively generalizable to our study, i.e. we can show that people have higher "situational familiarity" to saliency map than to counterfactual explanations, Reinhard et al. (201 I) will help explain why crowdworkers gain higher accuracy when given saliency map. The missing piece of our argument can be found in Yacoby et al. (2022), who show that judges are at first confused by and later ignore counterfactual explanations in Al Al-assisted public safety assessment (PSA) tasks. Instead, they prefer looking at each defendant's spe- cific features, which analogously corresponds to saliency map in our study. Altogether, Yacoby et al. (2022) show that humans have higher "situational familiarity" with saliency map than counterfactual explanations, from which Reinhard al. (2011) I) show that higher "situational familiarity" leads higher classification accuracy of verbal content.
5
Findings
After computing the five metries of interest per par- ticipant, we analyze the distribution of responses- across explanation style and participant race-, vi- sualizing them using box and whisker plots, and comparing the means using t-tests. We only report findings that achieve statistical significance from the t-tests (p-value < 0.05).* Since p-value is an indicator of whether an effect exists but not of its magnitude (Wasserstein and Lazar, 2016), we also report Cohen's 's d score (Cohen, 2013) and its 95% confidence interval from bootstrapping.
Our results indicate that statistically significant effects presented in the paper have sensible Cohen's d score estimates (small to moderate effects that require statistical methods to be observed (Cohen, 2013)) and reasonable 95% confidence intervals. Statistically insignificant or less interesting results for the remaining metrics are shown in Appendix G. To examine varying treatment effects for individ- uals or subgroups in a population, analyze the heterogeneous treatment effects in Appendix F,
5.1 Saliency map outperforms counterfactual explanation on three overall metrics
We first look how each explanation style per- forms across our metries in general and find two significant results about the utility of saliency map.
Stereotype Activation. As shown in Figure 5, counterfactual explanation (M = 0.1900, SD = 0.3365) yields significantly higher stereotype acti- vation than the baseline (M=0.0968 0. 1990), t(184) = 2.2861, p-value 0.023, Cohen's d = 0.3371, 95% CI = [0.0648, 0.5884]. One reason is that although counterfactual may make the AI model predict non-hatefud, the counterfactual itself is not necessarily non-hateful. For exam- ple, the counterfactual explanation for the tweet @ USER Pussies.. That's what the Chinese are known for... retreat Losers!!! Vihuusseeirus" is
Mental Discomfort. As shown in Figure 3, saliency map (M 1.7010, SD = 5.0076) yields significantly lower mental discomfort than the no explanation baseline (M = 3.4194, SD = 5.9121), ((18)== = -21535,, p-value- = 0.033, Cohen's = -0.3142, 95% CI [-0.5915, -0.0362]. A plausible mechanism which may explain this observation is that saliency map gives participants the option to just look the top few most "hateful" tokens with- out necessarily putting them into context of a whole
WWe report results of t-tests in APA format.

---

still arguably hateful even after removing the "#Chi- nese Virus" hashtag. Therefore, the false prediction by the Al model that a still-hateful counterfactual is non-hateful may bias participants into thinking that the underlying discriminatory content of such counterfactuals are acceptable, thereby increasing their stereotype activation. We might view coun- terfactual explanations as analogy to "counter- stereotypic images" from social psychology, where Nelson and Kinder (1996) find that participants who are shown counterstereotypic images are influ- enced by racial attitude more than a control group who is shown nothing. Analogously, counterfactual explanations might trigger people's implicit bias.
those norms pressure Asians more than non-Asians (the white-majority group), justifyinn the disparate mental discomfort.
Label Time. As shown in Figure 7, counterfac- tual explanations also give significantly disparate impact in terms of verifiable workload (measured through label time), p-value 0.012, Cohen's d 0.5365, 95% CI [O. 1451, 0.9367]. However, the disparate impact here is in the opposite direc- tion: disadvantaging non-Asian participants (M 349.3864, SD = 196.7625) rather than Asian partici- pants (M 2960,996, SD = 142.8468) Many Asian participants or people in their circles may have first- hand experience with anti-Asian hate speech, mak- ing them more efficient in formulating their own predictions about such content. If we come back to the "situational familiarity" concept from Reinhard et al. (201 1) which we have argued in section 5.1 to be low for counterfactual explanation, we can argue that because participants get almost no "sit- uational familiarity" from the counterfactual ex- planations, they can only rely on their "situational familiarity" with the anti-Asian hate speech pre- diction task itself (absent any explanations), which many Asians have practiced during the pandemic to identify threats against themselves. As Asians have higher "situational familiarity", they may perform the prediction task faster than non-Asians, who might still be confused due to low "situational fa- miliarity" with the counterfactual explanations and the Astan hate speech prediction task itself.
To understand why counterfactual explanations seem less desirable overall than salience maps, we recall study of Laugel et al. (2019), who find that post-hoc counterfactual explanations, which "use instances that were not used to train the model to build their explanations", will risk not satisfy- ing desirable properties of AT explanations such as "proximity, connectedness and stability".. Since the counterfactual explanation in our study is post-hoe, that study might also be applicable our context.
5.2 Counterfactual explanation yields disparate impact on two metries, but saliency map is not completely "fair"
We break down the explanation impact results by whether a crowdworker identifies as Asian or not to investigate potential disparate impacts of expla- nation styles across impacted populations.
Social media platforms face an ethical vs. eco- nomic tradeoff when allocating hate speech pre- diction tasks to their content moderators. Suppose a platform is expected to provide counterfactual explanations to its content moderators, it might be incentivized to give those hate-classified tweets to content moderators who belong to the targeted minority to minimize label time and thus the anno- tation cost pet tweet, at the expense of dispropor- tionately high mental discomfort experienced by the minority group. Therefore, our finding might inform future Al-assisted decision making regula- tions, suggesting the introduction of externalities, e.g. fines, to limit (if not completely prevent) so- cial media platforms from allocating AT-classified hateful content and/or "unfair" AI explanations that cause disparate mental discomfort to the targeted minority in their content moderator pool.
Mental Discomfort. As shown in Figure 6, we observe significant disparate mental discomfort: = 4M 1087 $D25.2134 expe- rience significantly higher mental discomfort than non-Asian participants (M 1.7447, SD = 5.8273) in the counterfactual explanation condition, ((91)
Asian participants
2.0370, p-value 0.045, Cohen's d 0.4271, 95% Cl [0.0465, 0.8022j, but not in other con- ditions. Our potential justification comes from Hegarty et al. (2004) who find that when asked to provide counterfactual explanations on how to make a hypothetical male (either gay or straight) feel less discomfort in a bar dominated by males from the opposite sexual orientation, humans are strongly influenced by hetero-centric norms. If this finding generalizes from sexual orientation to race, given counterfactual explanations, all partic- ipants may experience white-centric norms, but
Tangentially, our finding might also shed light on the mixed results regarding a data quality dilemma

---

34005
140I6
125
67-
128-
05
1,00
645
Wo.oplanatan
Salaacyy
Exparaaoo
Salarig
Cosbredtr
Figure 3: The saliency map condi- tion yields significantly less men- tal discomfort than the explana- tion baseline (p-value- = 0.033).
Figure 4: The saliency map con- dition yields significantly higher hate speech prediction accuracy than the counterfactual explana- tion condition (p-value 0.022).
Figure 5: The counterfactual ex- planation condition yields signifi- cantly higher stereotype activation than the no explanation condition (p-value 0.023).
0.05
1620
80-
1420
Label timee (scoords))
1220
Karrs dascetton
Race
Raca
-10-
200
No Explanation Sailency
Countorioraal
Explanation
Balleny
Countarbiuual
Explenation Style
Explanatin Style
Figure 6: Counterfactual explanation yields signif- icantly disparate mental discomfort against Asians (p-value 0.045). Asians with counterfactual planations get significantly more mental discomfort than Asians with saliency map (a-vulue 0.035).
Figure 7: The counterfactual explanation condition yields significantly disparate label time against non- Asians. (p-value 0.012), indicating that sharing the same sensitive feature value as the target minor- ity might make content moderators more efficient.
in the recent hate speech prediction literature. On the one hand. Olteanu et al. (2017) find that people who have experienced online harassment make less accurate hate speech prediction than people who have not. Their finding might suggest an incen- tive to exclude the targeted minority from the hate speech prediction task. On the other hand, Sap et al. (2019) find that providing crowdworkers with in- formation about the likely race of a tweet's author (based on the dialect of the tweet) will improve data quality of the annotation as crowdworkers will be less likely to annotate tweets with the African American English (AAE) dialect as offensive. we assume that content moderators from the tar- geted minority will know best whether the race of the tweets' authors is the same as the targeted minority, their finding might imply an incentive to intentionally include the targeted minority into the hate speech prediction task. Since our study suggests economic (yet not ethical) incentive for
industry to include the targeted minority in the hate speech prediction task, finding aligns with Sap et al. (2019), but Olteanu et al. (2017).
Stereotype Activation. Figure shows that saliency map yields marginally disparate stereo- type activation against non-Asian participants (M 0.1736, SD = 0.2965) but not for Asian participants (M -M=0.000 0.0816, SD = 0.2079), t(95) = 1.7535, p-value 0.083, Cohen's 's d = 0.3598, 95% CI = 1-0.0419,
0.7586]. This result should not be interpreted
discrediting saliency map, but is a caveat against thoughtlessly using saliency map without carefully examining its potential disparate impact on expla- nation readers in a downstream application context.
5.3
When explanation is required, which explanation style is better might depend the pool of explanation readers
With the rapid adoption of AI into decision- decision-making tasks, some jurisdictions have started to require

---

04006
1.50
0.00000
Strevopa Actraton
1.25
Pecseved worknes
1.00
Race
Race Non-Asian Adan
Asian Auann
0.75
0.50
Explaration
Sallenay Exploratio Blyle
Counteftatial
Explanation
Sallinyy Explanatio Style
Countertschual
There is evidence with marginal signifi- cance that the saliency map condition may yield dis- parate stereotype activation against non-Asians. (p- value 0.083).
Figure 8:
There significant disparate perceived workload in the baseline condition (p-value 0.041) but not in either of the saliency map or counterfac- tual explanation conditions.
Figure 9:
AI explanations for humans if the Al is to advise humans in making certain decisions.
tal discomfort at the end of the survey. participants tended discuss whether they felt affected by the content of the tweets. For example, P157 who is Asian and saw the saliency map wrote, "Reading some of that stuff left me with some negative emo- tions", whereas other participants such as P219 who is White and saw the saliency map explana- tions noted they did not feel affected by the content: "T' m not feeling any of the emotions at this time". Out of the 27 participants who indicated discomfort from the tweets in their rationales, 14 (52%) were Asian. However, only 18% of Asian participants left negative rationale after experiencing mental discomfort while 24% of white participants left a negative rationale after the experiment. This is de- spite more Asian participants (21%) leaving any rationalc (negative or otherwise) for their mental discomfort rating versus 18% of white participants. Asians are not less likely to leave a rationale alto- gether, but are less likely to leave when they have experienced mental discomfort.
If Al explanation becomes mandatory one day for content moderation, based on the results from Figure 9, the disparate perceived workload from the no explanation condition is no longer an evi- dent concern. However, a natural question arises: which explanation style to use? Observations in Figure 6 show that if the social media platform has access to content moderators outside the tar- geted minority, they may have more flexibility in selecting the explanation method because there is no noticeable difference in non-Asian participants' mental discomfort between the saliency map and counterfactual conditions. However, if their pool of content moderators is limited to only the targeted minority (e.g. Asian), they should avoid counter- factual explanation because for the targeted minor- ity, counterfactual explanation yields significantly higher mental discomfort than saliency map.
5.4 Qualitative Analysis: Asians who experienced disparate mental discomfort are less likely leave rationale
Across all conditions, the majority of partici- pants left at least one rationale to explain their de- cision in tweet labeling, with approximately 20% of participants leaving a rationale for every deci- sion. Participants tended to reference key word in the tweet (often racist word) or use their under- standing of the provided definition of COVID-19 related hate speech (such as the lack of mentioning Asian people) to explain their decision. Around 45% % of the participants did not leave any rationales. As the experiment progressed, participants across conditions became less likely to leave a rationale.?
We analyze if the optional free-text rationales give additional insights into the quantitative results above. When asked to explain their ratings for men-
Artiele 13.1 of the EU Artificial Intelligence Act (AIA) requires that: "High-risk Al systems shall be designed and developed such way to ensure that their operation sufficiently transparent to enable users to interpret the system's output and use appropriately".
"The 'Notiee and Explanation' section of the US Blueprint for an AI Bill of Rights requires that "Automated systems should provide explanations that are technically valid, mean- ingful and useful to you and to any operators or others who need to understand the system".
"These insights remain qualitative only. Performing t-tests and chi-squared tests of independence on the number of ratio-

---

5.5 Individual Fairness: Counterfactual Explanation is more individually unfair
parate) impacts of saliency map and counterfactual explanations on content moderators, using crowd- workers as a proxy. We find that first, saliency map is the most desirable condition overall. Sec- ond, counterfactual explanation exhibits a tradeoff in terms of disparate mental discomfort and dis- parate label time, thereby highlighting the need for legal intervention to minimize risks of labor abuse against content moderators in the targeted minority. However, saliency map is not necessar- ily innocent as there is marginal evidence of its disparate stereotype activation. Third, mandatory introduction of explanation may mitigate disparate perceived workload, but careful selection of ex- planation style may be needed depending on the racial distribution of the content moderator pool. Fourth, Asians who experienced mental discomfort are more reluctant leave a rationale. Fifth, coun- terfactual explanation is more individually unfair than saliency map. Our results suggest that even though counterfactual explanations seem less desir- able and less "fair" for proxy content moderators, further research is needed to confirm the utility and fairness of saliency map before more adoption of this explanation style by social media platforms. If follow-up research can validate that our find- ings are generalizable beyond crowdworkers to real content moderators, new Al-related legislation may incentivize social media platforms to imple- ment an effective and "fair" explanation method, e.g. saliency map, on larger scale. However, if due logistic constraints, an overall less desirable or "unfair" style, e.g. counterfactual explanations, must be used, stakeholders might consider a race- conscious content moderation policy, e.g. by as- signing content that AI classified as targeting racial minorities to moderators of other races to minimize mental impact. In summary, this work aims to re- sponsibly implement explanations methods in the real world, thereby reducing workload and improv- ing psychological well-being without sacrificing job performance, particularly of those sharing de- mographic characteristics with hate speech victims.
In our preceding experiments, we consider fair- ness from the perspective of group fairness is one group (Asian) harmed more than another (non- Asian). The other standard way of measuring fair- ness is individual fairness: are similarly situated in- dividuals treated similarly. In our context, the ques- tion whether two similar annotators are faced with the same amount of mental discomfort or per- ceived workload. A key challenge in individual fairness is defining "similar" individuals; we de- fine this as the pairwise distance averaged across responses to 9 task-relevan multiple-choice ques- tions. We can then measure the degree of individ- ual fairness, which captures notion quite different from group fairness. (See Appendix for details.)
We find that for Mental Discomfort, introducing either explanation style decreases individual unfair- ness, which contrasts the individual fairess results for other output metries. Among the two styles, counterfactual explanation is more unfair. For Stereotype Activation, introducing either explana- tion style increases individual unfairness. Among the two styles, counterfactual explanation is also more unfair regarding this second output metric. For Perceived Workload, introducing either expla- nation style increases individual unfairness. With respect to Label time, introducing either explana- tion style increases individual unfairness.
Comparing our group faimess results with our individual fairness results, we see that counterfac- tual explanation is more individually unfair than saliency map, which aligns with our previous find- ing that counterfactual explanation shows more evidence of group-wise unfairness, motivating the use of saliency map if fairness is a priority concern when choosing explanation style. Tangentially, introducing either explanation style increases in- dividual unfairness with respect to many metrics (stereotype activation, perceived workload, label time) but decreases individual unfairness with re- spect to mental discomfort. This result questions whether social media should use Al explanation for the hate speech classification task in the first place.
Limitations
6 Conclusion
Our study focuses how potential content moderators--hho would likely be the ones faced with explanations-arr are impacted by seeing those explanations. However, our participant pool is crowdworkers, standing in as proxies for content moderators, which introduces an ecological valid-
We conduct between-subjects human study across three explanation conditions to study potential (dis-
nales left by individuals in different conditions or of different races did not find any statistically significant diffcrences.

---

ity concern. As discussed, in the case that lay users (e.g.. of social media platforms) are shown explanations, this concern is mitigated because crowdworkers are probably a better surrogate for lay users. Our findings are also limited to one particular demographic- people who self-identify "Asian" (versus nod one particular task ("CoviD-1--eelated Asian hate") drawn from one particular dataset He al. (2021) in one particular time period (early 2020s) in one particular language (English) and with two particular explanation con- ditions (plus no explanation). Future work may test the generalizability our findings in the hate speech prediction context with respect to another demographic feature, e.g. ggender-based misog- yny transphobia, or (sexual orientation-basedl anti-queer hate speech, or other Al-assisted deci- sion making tasks where both saliency map and counterfactual explanations are relevant and where a certain minority that has been historically disad- vantaged may also serve as the decision-maker.
summary, future work may add more experimental conditions with other explanation styles, c.g. data influence or (global) anchors, the budget allows.
Ethical Considerations
Since our study involves tweet samples with derogatory. offensive, and discriminatory lan- guages. We expect these to have negative mental impacts on participants, at least in the short term (and find that they do). To mitigate this harm, have: (a) given advanced notice to potential par- ticipants in Prolific (where they are recruited); (b) repeated that notice in Qualtrics (where the study takes place); and (c) give them the option to ter- minate the study at at time. As mentioned in the paper, the study was IRB approved.
In the dataset attached to our paper, we delete all personally identifiable information from our hu- man study participants protect their privacy. For example, we replace actual Prolific IDs of our par- ticipants (in the column PROLIFIC PID, which are used in the individual fairness evaluation notebook) by dummy indices from 1 to 287.
Our findings are also only as good as our mea- surements and the wording of the task and survey questions. We know from previous literature that these can have non-trivial effects on the results. For instance, Giffin et al. (2017) found that simply naming explanation technique with science-y sounding name increased people's satisfaction with it. During the tutorial of our experiment we named saliency map but not counterfactuals, which may have had small impact on results.
A Acknowledgement
This work was supported in part by U.S. Army Grant No. 1991222220076. We thank EMNLP Reviewers, Area Chairs and Program Chairs for evaluating our paper with their valuable time! We also thank all members of the UMD CLIP and HCIL labs, especially those who provided feedback to our paper, e.g. Dr. Joel Chan, Do Won Kim, Kyle Seelman, Min Cheong Kim, Naimul Hoque, Navita Goyal, Sandra Sandoval and Sathvik Nair!
We have only experimented with two explana- tions styles (saliency and counterfactual), which may limit the potential for adopting our findings into real-life content moderation. Future work may expand the scope of findings with more explanation styles. For instance, we can test hypothesis that data influence explanations (Han et al., 2020) might have more disparate impact against Asian partici- pants in terms of mental discomfort. One potential justification for this hypothesis is that participants will need to read more hateful-classified samples from the training set as explanations, and Asians might experience higher per-instance (and thus sig- nificantly higher cumulative) mental discomfort than Asians..ns. Another option is global explana- tions, such as anchors (Ribeiro al., 2018), which identify tokens likely to cause hate predictions in the entire dataset. It may reduce the amount of time exposed to hate tweets, thereby potentially mitigat- ing disparate workload or mental discomfort. In
References
Alessa Angerschmid, Jianlong Zhou, Kevin Theuer- mann, Fang Chen, and Andreas Holzinger. 2022. Fairness and explamation Al-informed decision making. Machine Learning and Knowledge Extrac- tion, 4(2):556-579.
Esma Balkir, Svetlana Kiritchenko, Isar Nejadgholi, and Kathleen Fraser. 2022. Challenges in Apply- ing Explainability Methods to Improve the Fairness of NLP Models. In Proceedings of the 2nd Work- shop on Trustworthy Natural Language Processing (TrustNLP 2022), pages 80-92, Seattle, U.S.A. As- sociation for Computational Linguistics.
Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2017. Fairness machine learning. NewrIPS tuto- rial, 1:2017.

---

Enrico Bunde. 2021. Al-Assisted and explainable hate speech detection for social media moderators- A design science approach. In Proceedings of the 54th Hawaii International Conference on System Sciences.
Navita Goyal, Connor Baumler, Tin Nguyen, and Hal Daume. 2023. The Impact of Explanations on Fair- ness in Human-Al Decision Making: Protected vs Proxy Features. In Workshop on Trust and Reliance in Al-Assisted Tasks (TRAIT).
Cambridge Consultants. 2019. Use of Al in online con- tent moderation. Ofcom.
Xiaochuang Han, Byron C. Wallace, and Yulia Tsvetkov. 2020. Explaining Black Box Predic- tions and Unveiling Data Artifacts through Influence Functions. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguisties, pages 5553-5563, Online. Association for Computa- tional Linguistics.
Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. 2018. Double/debiased machine learning for treatment and structural parameters. Oxford University Press, UK.
Sandra G Hart. 2006. NASA-task load index (NASA- TLX); 20 years later. In Proceedings of the Hu- Factors and Ergonomics Society Annual Meet- ing, volume 50, pages 904-908. Sage publications Sage CA: Los Angeles, CA.
Jacob Cohen, 2013. Statistical power analysis for the behavioral sciences. Academic press.
Brandon Dang, Martin J Riedl, and Matthew Lease. 2018. But who protects the moderators? The case of crowdsourced image moderation. In Proceedings of the 6th AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2018).
Bing He, Caleb Ziems, Sandeep Soni, Naren Ramakr- ishnan, Diyi Yang. and Srijan Kumar. 2021. Racism is virus: Anti-Asian hate and counterspeech so- cial media during the COVID-19 crisis. In Proceed- ings of the 2021 IEEE/ACM International Confer- ence on Advances in Social Networks Analysis and Mining, pages 90-94.
Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. 2020. A Sur- vey of the State of Explainable Al for Natural Lan- guage Processing. In Proceedings of the 1st Con- ference of the Asia- Asia-Pacific Chapter of the Associa- tion for Computational Linguistics and the 10th In- ternational Joint Conference Natural Language Processing, pages 447-459, Suzhou, China. Associ- ation for Computational Linguistics.
Peter Hegarty, Felicia Pratto, and Anthony F Lemieux. 2004. Heterosexist ambivalence and heterocentric norms: Drinking in intergroup discomfort. Group Processes & Intergroup Relations, 19-3:119.130.
Fan Huang. Haewoon Kwak, and Jisun An. 2023. Is ChatGPT Better than Human Annotators? Poten- tial and Limitations of ChatGPT Explaining Im- plicit Hate Speech. In Companion Proceedings of the ACM Web Conference 2023, WWW '23 Com- pamion, page 294-297, New York, NY, USA. Asso- ciation for Computing Machinery.
Anubrata Das. Brandon Dang, and Matthew Lease. 2020. Fast, accurate, and healthier: Interactive blur- ring helps moderators reduce exposure to harmful content. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, volume 8, pages 42.
Ed Diener, Derrick Wirtz, Robert Biswas-Dies Diener, William Tov, Chu Kim-Prieto. Dong-won Choi, and Shigehiro Oishi. 2009. New measures of well-being. Assessing well-being: The collected works of Ed Di- ener. pages 247-266.
Thibault Laugel. Marie-Jeanne Lesot, Christophe Marsala, and Marcin Detyniecki. 2019. Issues with post-hoc counterfactual explanations: discussion. In ICML Workshop on Human in the Loop Learning.
Jonathan Dodge, Q Vera Liao, Yunfeng Zhang, Rachel KE Bellamy, and Casey Dugan. 2019. Ex- plaining models: an empirical study of how explana- tions impact fairness judgment. In Proceedings of the 24th international conference on intelligent user interfaces, pages 275-285.
Feng Li, Xinwen Bai, and Yong Wang. 2013. The Scale of Positive and Negative Experience (SPANE): Psychometric properties and normative data in large Chinese sample. PloS one, 8(4)=66
Gordon Moskowitz. 2010. On the control over stereotype activation and stereotype inhibition. So- cial and Personality Psychology Compass, 4(2):140- 158.
Cynthia Dwork. Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. Proceedings of the 3rd inno- vations in theoretical computer science conference, pages 214-226.
Thomas Nelson and Donald R Kinder. 1996. Issue frames and group-centrism in American public opin- ion. The Journal of Politics, 58(4):1055-1077
Carly Giffin, Daniel Wilkenfeld, and Tania Lombrozo. 2017. The explanatory effect of label: Expla- nations with named categories are more satisfying. Cognition, 168:357-369.
Casey Newton. 2019. The trauma floor: The secret lives of Facebook moderators in America. The Verge, 25(02).

---

Alexandra Olteanu, Kartik Talamadupula, and Kush Varshney. 2017. The limits of abstract evaluation metries: The case of hate speech detection. In Pro- ceedings of the 2017 ACM web science confer- ence, pages 405-406.
Mare-André Reinhard, Siegfried L Sporer, Martin Scharmach, and Tamara Marksteiner. 2011. Listen- ing. not watching: situational familiarity and the ability to detect deception. Journal of Personality and Social Psychology, 101(3):467.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why should I trust you?" Explain- ing the predictions of any classifier. In Proceed- ings of the 22nd ACM SIGKDD international con- ference on knowledge discovery and data mining. pages 1135-1144.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-po prechsrrciiion model- agnostic explanations. Proceedings of the AAAI conference on artificial intelligence, volume 32.
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1668 1678.
Miriah Steiger, Timir Bhanucha, Sukrit Venkatagiri, Martin J Riedl, and Matthew Lease, 2021. The psy- chological well-being of content moderators: the emotional labor of commercial moderation and enues for improving support. Proceedings of the 2021 CHI conference on human factors in comput- ing systems, pages 1-14.
Ronald L Wasserstein and Nicole A Lazar. 2016. The ASA statement on p-values: context, process, and purpose. The American Statistician, 70(2):129-133.
S Christian Wheeler and Richard E Petty. 2001. The effects of stereotype activation behavior: view of possible mechanisms. Psychological bul- letin, 127(6):797.
Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld, 2021. Polyjuice: Generating Counterfactuals for Explaining. Evaluating, and Im- proving Models. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the I 1ith International Joint Conference Natural Language Processing (Volume 1: Long Papers), pages 6707-6723, Online. Association for Computational Linguistics.
Yaniv Yacoby, Ben Green, Christopher L Griffin Jr. and Finale Doshi-Velez. 2022. "If it didn't hap- pen, why would I change my decision?": How Judges Respond to Counterfactual Explanations for the Public Safety Assessment. In Proceedings of the AAAI Conference Human Computation and Crowdsourcing, volume 10, pages 219-230.

---

A Details of Counterfactual Explanations
For counterfactual explanations, at the first planned step of generating counterfactuals from Polyjuice, we realized that many of those counterfactuals completely distort the semantic meaning of the tweets or do not even make sense.* Another option to use pre-trained large language models (such as ChatGPT) generate counterfactuals, but these models have strict content policies against quoting hateful languages in the prompts and even when the models accept our prompts, the counterfactuals they generate are over-corrective and deviate greatly from the original context of the hate-classified tweets."
B Details of the Al Hate Speech Classifier
We fine-tune a RoBERTa-based binary classifier for 10 training epochs, on a Quadro RTX 5000 GPU machine, maximum sequence length of 128, batch size of 16, warm-up steps of 500, learning rate of 1c-05, and weight decay of 0.01. We use Roberta-base model, which includes 12 layers, 768 hidden nodes, 12 head nodes, 125M parameters, and add a linear layer with two nodes for binary classification. Training the classifier takes several minutes as the data set is relatively small.
C Detailed Human Study Interfaces
We use one definition of "COVID-19 related Asian hate" throughout our study: "antagonistic speech that is directed towards an Asian entity (individual person, organization, or country), and others the Asian outgroup through intentional opposition or hostility in the context of COVID-19."
Do you agree or disagree with the Al prediction that the following tweet is "COVID-19 related Asian hate"?
"@USER Best solution of corona. Dear @realDonaldTrump this will scare the shit out of chinese virus!"
think the tweet "COVID-19 related Asian hate"
think the tweet may not mention COVID or Asian, but is hateful
think the tweet is NOT hateful
Figure 10: Human study interface with an example question (No Explanation condition)
two tweets 'gave my ddck the coronavirus' Chinese virus' be 'gave my son coronavirus' and
"For example, Polyjuice generates counterfactuals
call
call it real christmas virus'. "nn example GPT-generated counterfactual to the tweet "@t USER Best solution of corona. Dear @realDooaldTrump this will seare the shit out of chinese virus!" is "If Trump had taken a more proactive and conrdinated approach to the pandemic from the outset, perhaps the impact of the virus would not have been as severe as it has been."

---

Do you agree or disagree with the prediction that the following tweet "COVID-19 related Asian hate"?
"QUSER Best solution of corona. Dear Trummp this will scare the shit out of chinese virus!"
non-hate
hate
chinese
Text with highlighted words
realDenaldTrump
0.00
@USER Best solution of corona. Dear
will
@raiDonald this will scare chinese virus!
0.06
Dear
10.05
scare
10.04
corona
0.04
of
00.03
Best
0.02
this
0.01
solution
0.01
virus
0.01
USER
6.00
think the tweet is "COVID-19 related Asian hate"
think the tweet may not mention COVID or Asian, but is hateful
think the tweet is NOT hateful
Figure 11: Human study interface with an example question (Saliency Map condition)

---

The Al model predicts that the following tweet is "COVID-19 related Asian hate":
"@USER Best solution of corona. Dear @realDonaldTrump this will scare the shit out of chinese virus!"
If the original tweet is minimally modified into the following version, the Al model will flip its prediction from hate to non-hate. Please note that the highlighted words are the differences between the two versions.
"@USER Best solution of corona. Dear @realDonaidTTrum this will scare the shit out of corona virus!"
Do you agree disagree with the Al prediction that the original tweet is "COVID-19 related Asian hate"?
think the tweet is *COVID-19 related Asian hate"
think the tweet may not mention COVID Asian, but it is hateful
think the tweet is NOT hateful
Figure 12: Human study interface with example question (Counterfactual Explanation condition)
On a scale from "Definitely not feeling so" to "Definitely feeling so", please rate whether you are feeling each of the following emotions now:
Definitely not feeling so
Probably not feeling
Probably feeling so
Definitely feeling so
so
Not sure
Negative
O
O
O
O
Bad
O
O
O
O
O
Unpleasant
0
O
O
O
Sad
0
O
O
O
Afraid
0
0
0
0
Angry
O
0
0
0
Figure 13: Questions to quantify mental discomfort: Scale of Positive and Negative Experience (SPANE)

---

On a scale from "Strongly Disagree" to 'Strongly Agree", please rate how much you agree/disagree with the following statements:
Strongly Disagree
Strongly Agree
Disagree
Neutral
Agree
The world needs an apology from President Xi and the China Communist Party (CCP).
O
O
O
O
0
Dogs, cats, and horses are victims of bad eating habits in China and South Korea.
O
O
This is attention check select "Strongly Agree" for this statement.
O
0
0
0
Innocent Chinese people are spied on by the Chinese Communist Party (CCP) with CCTV.
O
O
O
0
Questions to quantify stereotype activation: rating on five-point scale how much participants (dis- Jagre with three implicit anti-Asian statements
Figure 14:
On scale from to 7, where 1 is "Not at all" and 7 is "Completely", please rate your answer to each of the following questions:
Not at all
7
2
3
4
5
6
Completely
Mental Demand:
How mentally
O
O
0
O
O
O
O
demanding was the task?
How physically demanding was the task?
Physical Demand:
0
0
O
Temporal Demand: How hurried or rushed was the pace of the task?
Performance: How successful were you in accomplishing what you were asked to do?
Effort: How hard did you have to work accomplish your level
0
performance?
Frustration: How insecure, discouraged, irritated, stressed, and/or annoyed were you?
0
o
Figure 15: Questions to quantify perceived workload: NASA Task Load Index (NASA-TLX)

---

D Human Study Pay Rate and Participants Selection Criteria
We set our Prolific pay rate at $12 per hour. We estimate the total study duration based on the completion duration statistics of the most time-consuming condition from our pilot experiment, and set the estimated duration for our main study across conditions be 14 minutes, corresponding to a pay amount of $2.80 for every condition.
In Prolific, we apply the following filters for participants: Living in the USA, English fluency, Prolific approval rate 2 98%. To get roughly the same number of Asian and non-Asian participants, we set up two almost identical Prolific studies with only difference in the ethnicity filter: one includes only 'Asian' and the other includes 'Black', 'White', and 'Other' (i.e. non-Asian)."
We get 287 Prolific participants in total. To ensure data quality and consistency with Prolific policy, we exclude the 4 participants who fail at least two attention checks from our data analysis, leaving us with 283 data points.
E Detailed Individual Fairness Evaluation
Concept There are two major popularized schools of faimess in the AI literature: group faimess and individual fairness. The main idea of group fairness is that outcomes (values with respect to an output feature of interest, e.g. accuracy) should be relatively equalized, or should not differ significantly across different (demographic) groups. Otherwise, there is disparate impact (Barocas et al., 2017).
Another popular perspective in the fairness literature is "individual fairness", with the intuition that the outcomes are fair if similar individuals get similar outcomes (Dwork et al., 2012). If we can apply this intuition of "individual fairness" into a quantifiable metric, we might evaluate which of the three explanation conditions are more individually unfair with respect to each of the five output metrics (e.g. mental discomfort). Our motivation is that if the new "individual fairness" findings also aligns with the previous "group fairness" findings, e.g. if "counterfactual explanations" are more unfair than "saliency map" from both group fairness and individual fairness perspectives, the new result will further enhance the Soundness of our claims.
Evaluation Pipeline One major challenge in applying the "individual fairness" intuition is define resk-relevant pairwise distance function calculate how "similar" any two individuals are. To address this issue, at the start of the survey, we ask each participant 9 multiple choice questions about their individual input features related to the anti-Asian hate speech classification task. We design the answer choices to each question such that nearer-located answers choices (e.g. A and B) are semantically closer than focther-located answer choices (e.g. A and C). The list of multiple-choice questions are below.
Question 1: Have you ever been victim of online hate speech (if multiple options apply, please choose the first applicable option)?
A. Yes: online hate speech against Chinese
B. Yes: online hate speech against Asian but not Chinese
C. Yes: online hate speech against non-Asian race/ethnicity (such as against Black, Hispanic, etc.)
D. Yes: online hate speech based on a non-race sensitive attribute (such as gender, sexual orientation,
etc.)
E. No
Prefer not to answer
Question 2: Have you ever been a victim of verbal (in-person) hate speech (if multiple options apply.
please choose the first applicable option)?
A. Yes: verbal hate speech against Chinese
B. Yes: verbal hate speech against Asian but not Chinese
C. Yes: verbal hate speech against non-Asian race/ethnicity (such as against Black, Hispanic, etc.)
""Since participant cannot select more than one racial option in their Prolific registration and we exclude the 'Mixed' race category, assuming all participants registrations were authentic, there should he no misassignment or leakage of participants hetween the Asian and non-Asian versions.

---

D. Yes: verbal hate speech based on a non-race sensitive attribute (such as gender, sexual orientation.
etc.)
E. No
F. Prefer not to answer
Question 3: How much time in total have you spent visiting and/or living in Asia?
A. Never
B. 1 day month
C. I month year
D. year 5 years
E. Over 5 years
F. Prefer not to answer
Question 4: Were you born in the USA, and if not, at what age did you move to the USA?
A. Born in the USA
B. Moved to the USA at an age lower than years old
C. Moved to the USA at an age between 5 and 18 years old
D. Moved to the USA
age between 18 and 30 years old
E. Moved the USA an age higher than 30 years old
Prefer not to answer
Question 5: Please choose the option that best describes your family background (if multiple options
apply, please choose the first applicable option):
A. Mainland China
B. Taiwan, Hong Kong, or Macau
C. Sinosphere country (Japan, North/South Korea, or Vietnam)
D. An Fast Asian or Southeast Asian country not mentioned above (such as Mongolia, Singapore, etc.)
E. An Asian country outside East/Southeast Asia
F. No Asian background
G. Prefer answer
Question 6: How many Asian languages (such as Mandarin Chinese, Hindi, Vietnamese, etc.) you
speak?
A. 0
B. 1
C. 2
D. 3 or more
E. Prefer not to answer
Question 7: How necessary do you think online content moderation (when it is weighed against
other rights such as free speech)?
A. Very unnecessary
B. Unnecessary
C. Neutral
D. Necessary
E. Very necessary
Question 8: What do you personally think should be the highest appropriate sanction against the most
extreme online hate speech creators?
A. No sanction
B. Deleting the hateful content (such as specific hate tweets)
C. Banning the hate speech creator from the relevant social media platform
D. Civil damages (such as financial compensation for the victims' mental sufferings)
E. Criminal punishment (such community service, probation, jail time)
Question 9: How often do you think that you "have no place in this world". "feel left out", or "feel like an o
A. (Almost) always

---

B. Daily
C. Weekly
D. Monthly
E. Yearly
(Almost) never
We filter out any "Prefer not to answer" answers and map the remaining multiple-choice answers for cach question to normalized, equidistant values between 0 and (e.g. A to O, B to 0.25, C to 0.5, D to 0.75, E to 1). Next, we compute the pairwise distance (absolute difference, averaged across the 9 individual input features above) for every pair of individuals within a given explanation condition. Finally, we develop simple and interpretable pipeline to compute an "individual unfairness" metric, as follows:
Pipeline: Individual UnFairness Evaluation
Given output feature X (e.g. mental discomfort) and explanation condition (e.g. saliency map):
Step 1: Find the top k pairs of most similar two individuals (with smallest pairwise average distances)
Step 2: Within the top pairs, for the two individuals A and in each pair with avg arg_distance( (A, B); i.e. distance averaged across their 9 individual input features, to get a pairwise iasss__ndvvidual____ Lin fairness f metric for A and B. we compute the absolute difference in their output values weighted by [(1 veg_distanceAA4 A, B)]. Our rationale is that the more similar two individuals are (i.e. the smaller their weg ueg_distance is), the more "individually unfair" it will be for their output values to differ.
ind.rriiddddlll_nfroeeeeess unfoirresssA, B) = aeg dig_ddistaeeeA, B)) loulpul(A) - (uu)pu((B)
Step 3: Calculate the mean of pairwise individual unfairness scores across the k pairs to use as the individual unfairness score of explanation condition i with respect to output feature X.
Step 4: Perform t-tests between the distributions of output feature X in explanation condition i and the same output feature X in another explanation condition (such as j) to compare the "individual unfairness" between conditions i and with respect to the feature X.
Detailed Results The results of the individual (un)fairness evaluation pipeline above are summarized Table 1. In particular, to ensure that our findings are stable, we vary the number of top pairs considered (k) as hyperparameter with values ranging from (100, 200, 400, 1000, 2000, 4000). We report the "individual unfairness" conditions comparisons which remain statistically significant (p-value < 0.05) across multiple settings of k.
Metric for Individual Unfairness evaluation
Comparison (p-value 0.05) of
1: (number of evaluation pairs giving significant Comparison)
Individual Unfairness (IU)
MD_ MD_IU(SM) MD [I_IE(NE)
400, 1000, 2000, 4000
Mental Discomfort (MD)
MD AD_IU(CE)> MD (ID_IU(M)
400, 1000, 2000, 4000
MD_IU(CE) MD _U(EE)
1000, 2000, 4000
SA_IU(SM) > SA_ _TU(NE)
1000, 2000, 4000
Stereotype Activation (SA)
SA IU(CE) > SA IU(SM)
2000, 4000
SA_IU(CE)) > SA. (U((E)
1000, 2000, 4000
PW PW_IU(SM) PW PW_IU(NE)
100, 200, 400, 1000, 2000, 4000
Perceived Workload (PW)
PW PW_IU(CE)> PW_JU(NEE
100, 200, 400, 1000, 2000, 4000
LT_IU(SM) LT LT_IU(NE)
100, 200, 400, 1000, 2000, 4000
Label Time (LT)
LT_IU(CE)) > LT_IU(NE)
100, 200, 400, 1000, 2000, 4000
Table Individual UnFairness comparison per output metric across conditions (NE: No Explanation; SM: Saliency Map: CE: Counterfactual Explanation). Bold comparisons are to check which explanation style (SM or CE) is more individually unfair.

---

F Heterogeneous Treatment Effects: Counterfactual Explanations are group-wise unfair with respect to race, gender, and age
We have explored the treatment effects separately for different groups, testing if the difference in treatment effects is statistically significant. However, there are potential problems with this approach. Specifically, the traditional sample split cannot capture higher-order interactions between treatment and baseline characteristics, For example, the most and least affected individuals might not fall the ends of characteristic (Asian v.s. non-Asian). Instead, the most affected group may be segment of the population that is defined by group of characteristies in a non-linear way.
To better understand how did the treatment effect vary across individuals and their characteristics, we adopt a machine learning approach to investigate the heterogenous treatment effects. We estimate heterogeneous treatment effects using Double Machine Learning (Chernozhukov et al., 2018). We use Tree Interpreter to provide a presentation-reatt summary of the key features that explain the biggest differences in responsiveness to intervention.
Specifically, we find that regarding stereotype activation, Counterfactual Explanations not only group-wise unfair with respect race, but also group-wise unfair with respect to other demographic features including gender and age. Counterfactual Explanation has higher positive effects induce stereotype activation for elderly female participants.
Prai
caT CArt ald 0.135,
Emplyymen samples 184
suripces 57
-0.51.0.015 0.87, 0.854
6111 0.372, 0.092
CATE mean CAIE JAd
CATE mesn
CATE maas
CATE misi
CATE std
CATE aid
CA
Figure 16: Counterfactual Explanation has higher positive effects induce stereotype activation for elderly female participants.
Statistically insignificant or less interesting results
We give the general results on the remaining two metrics (perceived workload. and label time) in Figure 17 and Figure 18. We give the race-specific results the remaining metric (accuracy), where there seems to be statistically significant evidence of disparate impact, in Figure 19. We give the general results the number of rationales left for labeling decisions by condition and race Figure 20 and by labeling task in Figure 21.

---

6
Percelved Workload
3
1
No Explanation
Saliency Explanation Style
Counterfactual
Figure 17: Perceived workload across explanation conditions
1e-5
2000
p 0.01
1750
1500
1250
Label time
1000
750
500
250
No Explanation
Saliency
Counterfactual
Explanation Style
Figure 18: Label time across explanation conditions. The result that the two with-explamation conditions yield more label time than the no explanation condition is obvious and not necessarily interesting to report finding.
0.7
0.6
0.5
Accuracy
Non-Asian
0.4
Asian
0.3
0.2
0.1
No Explanation
Saliency
Counterfactusl
Explanation Style
Figure 19: Accuracy across explanation conditions and racial groups

---

Total Number of Rationales (Labeling)
10
8-
Race
6-
Non-Asian
Asian
4-
0-
Explanation
Sallency Explanation Style
Counterfactual
Figure 20: Number of rationales left for 12 tweet labeling questions across explanation conditions and racial groups.
45.0
Percentage OT IndIviquais Leaving a Kationale
42.5
40.0
37.5
35.0
32.5
30.0
27.5
25.0
LT2
LT6 LT9 LT10LTi1LT12
Labeling Task #
Figure 21: Percent of individuals who left rationale for tweet label- ing questions. Tweets presented all participants the same order.


Reviews:
[{"review_id": "mPos6TZV3Q", "reviewer": "Reviewer_JNom", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for the time you put into reviewing our manuscript! \n\nIn this **Main Rebuttal**, we will first provide a **Response** to each quoted **Concern** in your review. For **Concern 4 - Soundness**, we additionally report new results to further persuade you about the soundness of our most central finding (that counterfactual explanation is more unfair than saliency map). We only give a summary of new results in **Response 4 - Soundness**, but we provide a more detailed description of methodology and actual numbers in the **Appendix - Detailed New Results**.\n\n\n\n## Concern 1: Small sample size\n“My primary concern with this study is that the small sample size of 12 tweets may not provide a robust foundation for comparing the two explanation methods, saliency maps and counterfactuals.”\n\n## Response 1:\nWe first argue the rationales to choose a relatively small sample size of tweets. First, we choose the sample size such that it is more likely for participants to better focus on the assigned task, circumventing potential information overload problems. It has been acknowledged that information overload can deteriorate decision making quality, prolong the decision process, and make participants less confident about their choices (Chervany, N. L., & Dickson, G. W. ,1974; O'Reilly III, C. A.,1980; Hwang, M. I., & Lin, J. W.,1999). Furthermore, we emphasize on the trade-off between the number of participants vs. task length under a fixed budget, we must not compromise the former if we want a reasonable sample size of participants. \n\nWe agree that it is better to determine a reasonable number of tweets that can both offer a robust foundation for comparing explanation methods and won’t introduce information overload problems that would compromise quality of the responses from participants. \n\n## Concern 2:  Other forms of explanations\n“If the goal is to understand the disparate impact of explanations on moderators, it would be beneficial to expand the scope of the investigation to include other forms of explanations, such as gradient-based, attention-based, or global ones”\n\n## Response 2:\nWe appreciate your suggestions to include other explanation methods to expand the scope of research. It is definitely worthwhile exploring for future studies. However, we would like to caution the readers that some explanation methods mentioned in the comments are not very appropriate to this hate speech moderation context.\n\nFirst, the global explanations aim to describe the average behavior of a machine learning model rather than explain individual predictions. We believe local explanations methods are better suited in the content moderation setting where understanding individual predictions is a priority. \nFurthermore, gradient-based methods or attention methods require solid background knowledge to understand which may impose challenges for participants to interpret and thus make inferior decisions with poor understanding. \n\nIn essence, we choose the two explanation methods because LIME-based saliency map is straightforward, intuitive, and works for any black-box classifier while counterfactual explanation resembles a real-life mental process. It is possible to include other appropriate explanation methods to strengthen the findings in this paper or offer more insights but we think it is not the main focus of this paper and we leave it for future research. \n\n\n\n## Concern 3:  Generalizability and depth of insights\n“Furthermore, the paper's group-specific conclusions are confined solely to anti-Asian hate, limiting its broader applicability. I encourage the authors to replicate these findings with other social groups, such as the African-American community, to enhance the generalizability and depth of their insights.”\n\n## Response 3:\nWe definitely agree with your suggestions. We think that future work may investigate the fairness of explanation in the same context but with respect to a different group (e.g. hate speech against African-Americans), a different feature (e.g. gender instead of race), or a different human-plus-AI context (e.g. disparate impact on bond court official from different groups by feature importance weights as explanations for recidivism risk assessment models, which may bias against people of colors).\n\nWithin the limit of our study’s budget, we tried our best to enhance the generalizability of our findings by conducting further statistical analysis. Specifically, we calculate Cohen’s scores to measure the magnitude of the effects reported in this paper and use Linear Double Machine Learning model to estimate the Heterogeneous treatment effects to offer more sights and justification about our findings. Specifically, we find that regarding stereotype activation, Counterfactual Explanations are not only group-wise unfair with respect to race, but also group-wise unfair with respect to other demographic features including gender and age. Counterfactual Explanation has higher positive effects to induce stereotype activation for elderly female participants.\n\n\n\n## Concern 4: “Soundness: 3”\n\n## Response 4:\nAs your Soundness score suggests that “some minor points may need extra support or details”, to further substantiate the soundness of our claims, we summarize **new results** from three additional streams of analysis on data from the reported human study. If you want to read the full methodology and actual numbers from each analysis stream, please check the **Appendix** at the end of this our author response.\n\nFirst, **Heterogeneous treatment effects** analysis with Linear Double Machine Learning (Chetverikov et al., 2016) shows that Counterfactual Explanations are not only group-wise unfair with respect to race, but also group-wise unfair with respect to other demographic features including gender and age. Second, as p-values only show statistical significance but now effect size (American Statistical Association, 2016), our **Effect size** analysis using Cohen’s d score (Cohen, 2013) shows that statistically significant effects presented in paper have small-to-medium effect sizes, necessitating statistical methods to uncover, which was what we did. Third, as we develop a simple and interpretable **Individual Fairness** pipeline based on the intuition that outcomes are individually fair if similar individuals are treated the same (Dwork et al., 2012), we find that counterfactual explanation is more individually unfair than saliency map. This new finding aligns with our previous findings in Figure 6 and 7 of the manuscript (that counterfactual explanation shows more evidence of **group-wise unfairness**).\n\n\n\nOverall, we hope that these responses address any questions you had regarding the paper.\n\n### References \n1. Chervany, N. L., & Dickson, G. W. (1974). An experimental evaluation of information overload in a production environment. Management science, 20(10), 1335-1344.\n2. O'Reilly III, C. A. (1980). Individuals and information overload in organizations: is more necessarily better?. Academy of management journal, 23(4), 684-696.\n3. Hwang, M. I., & Lin, J. W. (1999). Information dimension, information overload and decision quality. Journal of information science, 25(3), 213-218.\n4. Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W. K., & Chernozhukov, V. (2016). Double machine learning for treatment and causal parameters. 2016.\n5. Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Academic press.\n6. American Statistical Association. (2016). American Statistical Association releases statement on statistical significance and p-values.\n7. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012, January). Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference (pp. 214-226). \n\n\n\n# Appendix - Detailed New Results\n\nPlease note that our decision to add new results is consistent with the EMNLP 2023 policy: “Reporting additional experiments is encouraged, this may be the most effective way to get Reviewers to increase their scores.”\n\nWe report three streams of additional analysis and new results that further enhance the soundness and generalizability of our claims. Since OpenReview doesn’t allow additional uploads for author responses, we will provide the detailed code and figures associated with these new results in the camera-ready version (if accepted).\n\n\n\n##  Heterogeneous treatment effects \nWe estimate Heterogeneous treatment effects using Linear Double Machine Learning (Chetverikov et al., 2016). We use Tree Interpreter to provide a presentation-ready summary of the key features that explain the biggest differences in responsiveness to an intervention. We can provide an additional figure of the decision tree that illustrates this analysis in the camera-ready version. \n\nSpecifically, we find that regarding stereotype activation, *Counterfactual Explanations are not only group-wise unfair with respect to race, but also group-wise unfair with respect to other demographic features including gender and age*. Counterfactual Explanation has higher positive effects to induce stereotype activation for elderly female participants.\n\n\n\n## Effect size \nSince p-value can indicate whether an effect exists, it doesn't provide information about the magnitude of the effect (American Statistical Association 2016). For the readers to better interpret the substantive significance of effects that we discovered in this paper, we calculated Cohen’s d scores and 95% confidence intervals using bootstrapping. \n\nOur results indicate that *statistically significant effects presented in paper have a sensible Cohen’s d score estimates* (small to moderate effects that require statistical methods to be observed (Cohen, 2013)) and reasonable 95% confidence intervals. For example, saliency map yields significantly higher hate speech prediction accuracy than counterfactual explanation (p-value370 = 0.022, Cohen’s d: 0.337, 95% CI: [0.051, 0.625]). A Cohen’s d of 0.337 means that the two group means (Saliency maps v.s. Counterfactual Explanations) differ by 0.337 standard deviations.\n\n\n\n## Individual Fairness\nThe quantitative results we reported in the manuscript come from a group fairness perspective: whether an output feature gets significantly different values across different groups. Another popular perspective in the fairness literature is “individual fairness”, with the intuition that the outcomes are fair if similar individuals get similar outcomes (Dwork et al., 2012). If we can apply this intuition of “individual fairness” into a quantifiable metric, we might evaluate which of the three explanation conditions are more individually unfair with respect to each of the five output metrics (e.g. mental discomfort). Our motivation is that if the new “individual fairness” findings also aligns with the previous “group fairness” findings, e.g. if “counterfactual explanations” are more unfair than “saliency map” from both group fairness and individual fairness perspectives, the new result will further enhance the Soundness of our claims. \n\nOne major challenge in applying the “individual fairness” intuition is to define a task-relevant pairwise distance function to calculate how “similar” any two individuals are. To address this issue, at the start of the survey, we ask each participant 9 multiple choice questions about their individual input features related to the anti-Asian hate speech classification task:\n\nQ1: Have you ever been a victim of online hate speech?\n\n…\n\nQ9: How often do you think that you \"have no place in this world\", \"feel left out\", or \"feel like an outsider\"?\n\nAfter mapping multiple-choice answers to each question to normalized values between 0 and 1, we compute the pairwise distance (absolute difference, averaged across the 9 individual input features above) for every pair of individuals within a given explanation condition. We then develop a simple and interpretable pipeline to compute an “individual unfairness” metric.\n\nGiven output feature X (e.g. mental discomfort) and explanation condition i (e.g. saliency map):\n\n**Step 1**: Find the top *k* pairs of most similar two individuals (with smallest pairwise average distances)\n\n**Step 2**: Within the top k pairs, for the two individuals A and B in each pair with avg_distance(A, B), i.e. distance averaged across their 9 individual input features, to get a pairwise “individual unfairness” metric for A and B, we compute the absolute difference in their output values weighted by [(1-avg_distance(A, B)]. Our rationale is that the more similar two individuals are (i.e. the smaller their avg_distance is), the more “individually unfair” it will be for their output values to differ.\n\n**pairwise_individual_unfairness (A, B) = [1 - avg_distance(A, B)] * | output(A) - output(B) |**\n\n**Step 3**: Calculate the mean of pairwise individual unfairness scores across the k pairs to use as the individual unfairness score of explanation condition i with respect to output feature X.\n\n**Step 4**: perform t-tests between the distributions of output feature X in explanation condition i and the same output feature X in another explanation condition (such as j) to compare “individual unfairness” between conditions i and j with respect to feature X.  \nWe also vary the **number of top pairs considered (k)** as a hyperparameter from {100, 200, 400, 1000, 2000, 4000 pairs} and report the following findings about “individual unfairness” of the three conditions (**NE: no explanation; SM: saliency map; CE: counterfactual explanation**), which remain significant (p-value < 0.05) across multiple settings of k:\n\n1. Stereotype Activation Individual Unfairness (SA_IU): \n\nmean SA_IU(SM) > mean SA_IU(NE) (significant when k = 1000, 2000, or 4000 pairs)\n\nmean SA_IU(CE) > mean SA_IU(SM) (significant when k = 2000, or 4000 pairs)\n\nmean SA_IU(CE) > mean SA_IU(NE) (significant when k = 1000, 2000, or 4000 pairs)\n\nConclusion: With respect to Stereotype Activation, introducing either explanation style increases individual unfairness. Among the two styles, counterfactual explanation is more unfair.  \n\nExample numbers (when k = 2000 pairs): \n\nmean SA_IU(NE) = 0.148, mean SA_IU(SM) = 0.178, mean SA_IU(CE) = 0.217\n\np-value[SA_IU(NE), SA_IU(SM)] = 1.2 * 10^{-13}\n\np-value[SA_IU(SM), SA_IU(CE)] = 0.00027\n\np-value[SA_IU(CE), SA_IU(NE)] = 0.00018\n\n\n\n2. Mental Discomfort Individual Unfairness (MD_IU): \n\nmean MD_IU(SM) < mean MD_IU(NE) (significant when k = 400, 1000, 2000, or 4000 pairs)\n\nmean MD_IU(CE) > mean MD_IU(SM) (significant when k = 400, 1000, 2000, or 4000 pairs)\n\nmean MD_IU(CE) < mean MD_IU(NE) (significant when k = 1000, 2000, or 4000 pairs)\n\nConclusion: With respect to Mental Discomfort , introducing either explanation style decreases individual unfairness, which contrasts the individual fairness results for other output metrics.  Among the two styles, counterfactual explanation is more unfair.  \n\n\n\n\n\n3. Perceived Workload Individual Unfairness (PW_IU): \n\nmean PW_IU(SM) > mean PW_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000 pairs)\n\nmean PW_IU(CE) > mean PW_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000 pairs)\n\nConclusion: With respect to Perceived Workload, introducing either explanation style increases individual unfairness. \n\n\n\n\n4. Label time Individual Unfairness (LT_IU): \n\nmean LT_IU(SM) > mean LT_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000 pairs)\n\nmean LT_IU(CE) > mean LT_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000  pairs)\n\nConclusion: With respect to Label time, introducing either explanation style increases individual unfairness. Among the two styles, counterfactual explanation is more unfair.  \n\n\n\nSummary of Individual Fairness conclusions:  \n- Counterfactual explanation is more individually unfair than saliency map. This new finding aligns with our previous finding in  the manuscript that counterfactual explanation shows more evidence of group-wise unfairness.\n- Introducing either explanation style increases individual unfairness with respect to many metrics (stereotype activation, perceived workload, label time) but decreases individual unfairness with respect to mental discomfort.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper studies a novel impact of explanations on annotators and specifically takes into account the details about annotators such as their identity that have been shown previously to impact their annotation behavior on the task of hate speech detection.  \n", "reasons_to_reject": "My primary concern with this study is that the small sample size of 12 tweets may not provide a robust foundation for comparing the two explanation methods, saliency maps and counterfactuals. If the goal is to understand the disparate impact of explanations on moderators, it would be beneficial to expand the scope of the investigation to include other forms of explanations, such as gradient-based, attention-based, or global ones. Furthermore, the paper's group-specific conclusions are confined solely to anti-Asian hate, limiting its broader applicability. I encourage the authors to replicate these findings with other social groups, such as the African-American community, to enhance the generalizability and depth of their insights.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors investigate the disparate impact of two forms of explanations, namely saliency maps and counterfactuals, in the context of hate speech detection on a dataset targeting anti-Asian hate. They analyze 12 tweets, manually crafting the counterfactual explanations, and employing LIME to generate saliency maps. The study also assesses a pool of factors, including mental discomfort, stereotype activation, accuracy, and perceived workload. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "V642sPAO3T", "reviewer": "Reviewer_YzLM", "summary": "", "questions": "Question A: 5.4 claims that Asians are less likely to leave a rationale when feeling discomfort. That is, the likelihood of an Asian-identifying annotator leaving a rationale after discomfort was 18% while the white-identifying population was 24%. What are the likelihoods of an Asian-identifying person leaving a rationale at all versus a white-identifying one? i.e, are Asians also less likely to leave a rationale all together?\n\nQuestion B: How granular are the ethnicity identifiers in Prolific?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful comments and questions! We are happy that you found our paper interesting, and our empirical findings thorough! We start our responses to address your concerns first and offer answers to your questions later. \n\nAfter addressing your concerns and questions, we additionally report in the **Appendix** new results to hopefully substantiate your Confidence in the strong Soundness evaluation you gave to our claims. We summarize these new results in the paragraph below for your convenience:\n\nFirst, **Heterogeneous treatment effects** analysis with Linear Double Machine Learning (Chetverikov et al., 2016) shows that Counterfactual Explanations are not only group-wise unfair with respect to race, but also group-wise unfair with respect to other demographic features including gender and age. Second, as p-values only show statistical significance but now effect size (American Statistical Association, 2016), our **Effect size** analysis using Cohen’s d score (Cohen, 2013) shows that statistically significant effects presented in paper have small-to-medium effect sizes, necessitating statistical methods to uncover, which was what we did. Third, as we develop a simple and interpretable **Individual Fairness** pipeline based on the intuition that outcomes are individually fair if similar individuals are treated the same (Dwork et al., 2012), we find that counterfactual explanation is more individually unfair than saliency map. This new finding aligns with our previous findings in Figure 6 and 7 of the manuscript (that counterfactual explanation shows more evidence of **group-wise unfairness**).\n\n\n## Concern 1: Counterfactuals\n“Stronger models could provide better counterfactuals that may not illicit negative emotions as strongly as the ones in this paper do”\n\n## Response 1\nWe argue that two separate tasks should be disentangled here. Task 1: content improvement, i.e. build a model that actually improves a potentially hateful tweet into non-hateful as perceived by humans; Task 2: content (hate speech) classification. **Concern 1** is mostly related to **Task 1**. However, our focus in this paper is **Task 2** (Task 1 is good-to-have but should be a subject of an independent, algorithm-oriented project itself). Here we can simulate a very good model for Task 1 (i.e. generate completely non-hateful counterfactual candidates that differ from the original meaning of the hate-classified tweets completely), but this choice will compromise the quality of Task 2 since the counterfactual explanations will be less useful for annotators. \n\n\n\n## Concern 2: Scope\n“The scope of the paper is relatively narrow”\n\n## Response 2\nWe agree with the argument since we mainly focus on Asian hate speech context in the case study. However, we would like to emphasize on the main contributions of this paper that it identifies an overlooked question at the intersection of the explainability and fairness literature, and illustrates that this is a question worth exploring with a case study. Our contribution opens the door to future work examining whether explanations impact people fairly in other settings.\n\n\n\n## Concern 3: \"Reproducibility: 3\"\n\n“Reproducibility: 3. [...] the training/evaluation data are not widely available”\n\n## Response 3\n\nSince the hate speech prediction model is just a tool and not a main contribution of our manuscript, we assume you find a Reproducibility weakness in our dataset. The original tweets we use come from a publicly available dataset from He et al. (2022) cited in our manuscript. You may find the link to their data in their paper’s abstract.\n\nWe believe our reported results can be easily replicated since we randomly assigned participants to the three conditions, controlled the quality of responses through participant filter and attention checks, disclosed model specifications and experimental design details in the paper, and enclosed code/data files in the zip folder uploaded to OpenReview with our manuscript. Only the Personally Identifiable Information (e.g. Prolific IDs of human study participants) cannot be shared externally due to IRB regulations. We are also happy to answer any additional data or code-related questions via the “Official Comment” on OpenReview. \n\nTo further assist your reviews of our code and data, below is a functionality summary of the code and data files we included in our zip folder (Supplemental Materials) as uploaded onto OpenReview.\n\nEMNLP_roberta_train.ipynb: code to fine-tune a roberta binary (hate vs. non-hate) classifier.\n \nEMNLP_roberta_test_saliency.ipynb: code to generate saliency maps (LIME) explanations.\n\nEMNLP_roberta_test_counter.ipynb: code to generate predictions for counterfactual explanations\n\nEMNLP_Calculate_all_metrics.ipynb: code to calculate 5 output metrics (accuracy, label time, mental discomfort, perceived workload, and stereotype activation).\n\nEMNLP_statistical_analysis_t_test_all_metrics_final.ipynb: code to conduct t-tests under different conditions (no explanations, saliency maps, counterfactual explanations) and to reproduce all plots included in the paper.  \n\n\n\n## Concern 4: Presentation\n“The pictures tend to be a bit small; I couldn't quite read them easily without zooming to 200%.”\n\n## Response 4\nIf accepted, we will increase the size of Figures 1 and 2 from double-column to single-column (larger size) in the camera-ready version. In case there is not enough space in the main text, we will provide the larger-sized version of all figures in the camera-ready version’s appendix. \n\n\n\n\n## Question A: \n“What are the likelihoods of an Asian-identifying person leaving a rationale at all versus a white-identifying one? i.e, are Asians also less likely to leave a rationale all together?”\n\n## Answer A: \nApproximately 21% of Asian-identifying people left a rationale for their mental discomfort rating versus 18% of white-identifying people. Asians are not less likely to leave a rationale altogether, but are less likely to leave one when they have experienced mental discomfort (as indicated by our quantitative metric). \n\n\n\n## Question B: \n“How granular are the ethnicity identifiers in Prolific?”\n## Answer B:\nAs noted in our Appendix D: \"we set up two almost identical Prolific studies with only one difference in the ethnicity filter: one includes only ‘Asian’ and the other includes ‘Black’, ‘White’, and ‘Other’ (i.e. non-Asian) [...] we exclude the ‘Mixed’ race category [from Prolific].\" The Prolific ethnicity identifiers do not specify the participant’s ethnicity beyond Asian, White, Black, Mixed (excluded), or Other. \n\n\nOverall, we hope that these responses address any questions you had regarding the paper.\n\n\n\n### References \n1. Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W. K., & Chernozhukov, V. (2016). Double machine learning for treatment and causal parameters. 2016.\n2. Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Academic press.\n3. American Statistical Association. (2016). American Statistical Association releases statement on statistical significance and p-values.\n4. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012, January). Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference (pp. 214-226). \n5. Bing He, Caleb Ziems, Sandeep Soni, Naren Ramakrishnan, Diyi Yang, and Srijan Kumar. 2022. Racism is a virus: anti-asian hate and counterspeech in social media during the COVID-19 crisis. In Proceedings of the 2021 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM '21).\n\n\n\n\n# Appendix - Detailed New Results\n\nPlease note that our decision to add new results is consistent with the EMNLP 2023 policy: “Reporting additional experiments is encouraged, this may be the most effective way to get Reviewers to increase their scores.”\n\nWe report three streams of additional analysis and new results that further enhance the soundness and generalizability of our claims. Since OpenReview doesn’t allow additional uploads for author responses, we will provide the detailed code and figures associated with these new results in the camera-ready version (if accepted).\n\n\n\n##  Heterogeneous treatment effects \nWe estimate Heterogeneous treatment effects using Linear Double Machine Learning (Chetverikov et al., 2016). We use Tree Interpreter to provide a presentation-ready summary of the key features that explain the biggest differences in responsiveness to an intervention. We can provide an additional figure of the decision tree that illustrates this analysis in the camera-ready version. \n\nSpecifically, we find that regarding stereotype activation, *Counterfactual Explanations are not only group-wise unfair with respect to race, but also group-wise unfair with respect to other demographic features including gender and age*. Counterfactual Explanation has higher positive effects to induce stereotype activation for elderly female participants.\n\n\n\n## Effect size \nSince p-value can indicate whether an effect exists, it doesn't provide information about the magnitude of the effect (American Statistical Association 2016). For the readers to better interpret the substantive significance of effects that we discovered in this paper, we calculated Cohen’s d scores and 95% confidence intervals using bootstrapping. \n\nOur results indicate that *statistically significant effects presented in paper have a sensible Cohen’s d score estimates* (small to moderate effects that require statistical methods to be observed (Cohen, 2013)) and reasonable 95% confidence intervals. For example, saliency map yields significantly higher hate speech prediction accuracy than counterfactual explanation (p-value370 = 0.022, Cohen’s d: 0.337, 95% CI: [0.051, 0.625]). A Cohen’s d of 0.337 means that the two group means (Saliency maps v.s. Counterfactual Explanations) differ by 0.337 standard deviations.\n\n\n\n## Individual Fairness\nThe quantitative results we reported in the manuscript come from a group fairness perspective: whether an output feature gets significantly different values across different groups. Another popular perspective in the fairness literature is “individual fairness”, with the intuition that the outcomes are fair if similar individuals get similar outcomes (Dwork et al., 2012). If we can apply this intuition of “individual fairness” into a quantifiable metric, we might evaluate which of the three explanation conditions are more individually unfair with respect to each of the five output metrics (e.g. mental discomfort). Our motivation is that if the new “individual fairness” findings also aligns with the previous “group fairness” findings, e.g. if “counterfactual explanations” are more unfair than “saliency map” from both group fairness and individual fairness perspectives, the new result will further enhance the Soundness of our claims. \n\nOne major challenge in applying the “individual fairness” intuition is to define a task-relevant pairwise distance function to calculate how “similar” any two individuals are. To address this issue, at the start of the survey, we ask each participant 9 multiple choice questions about their individual input features related to the anti-Asian hate speech classification task:\n\nQ1: Have you ever been a victim of online hate speech?\n\n…\n\nQ9: How often do you think that you \"have no place in this world\", \"feel left out\", or \"feel like an outsider\"?\n\nAfter mapping multiple-choice answers to each question to normalized values between 0 and 1, we compute the pairwise distance (absolute difference, averaged across the 9 individual input features above) for every pair of individuals within a given explanation condition. We then develop a simple and interpretable pipeline to compute an “individual unfairness” metric.\n\nGiven output feature X (e.g. mental discomfort) and explanation condition i (e.g. saliency map):\n\n**Step 1**: Find the top *k* pairs of most similar two individuals (with smallest pairwise average distances)\n\n**Step 2**: Within the top k pairs, for the two individuals A and B in each pair with avg_distance(A, B), i.e. distance averaged across their 9 individual input features, to get a pairwise “individual unfairness” metric for A and B, we compute the absolute difference in their output values weighted by [(1-avg_distance(A, B)]. Our rationale is that the more similar two individuals are (i.e. the smaller their avg_distance is), the more “individually unfair” it will be for their output values to differ.\n\n**pairwise_individual_unfairness (A, B) = [1 - avg_distance(A, B)] * | output(A) - output(B) |**\n\n**Step 3**: Calculate the mean of pairwise individual unfairness scores across the k pairs to use as the individual unfairness score of explanation condition i with respect to output feature X.\n\n**Step 4**: perform t-tests between the distributions of output feature X in explanation condition i and the same output feature X in another explanation condition (such as j) to compare “individual unfairness” between conditions i and j with respect to feature X.  \nWe also vary the **number of top pairs considered (k)** as a hyperparameter from {100, 200, 400, 1000, 2000, 4000 pairs} and report the following findings about “individual unfairness” of the three conditions (**NE: no explanation; SM: saliency map; CE: counterfactual explanation**), which remain significant (p-value < 0.05) across multiple settings of k:\n\n1. Stereotype Activation Individual Unfairness (SA_IU): \n\nmean SA_IU(SM) > mean SA_IU(NE) (significant when k = 1000, 2000, or 4000 pairs)\n\nmean SA_IU(CE) > mean SA_IU(SM) (significant when k = 2000, or 4000 pairs)\n\nmean SA_IU(CE) > mean SA_IU(NE) (significant when k = 1000, 2000, or 4000 pairs)\n\nConclusion: With respect to Stereotype Activation, introducing either explanation style increases individual unfairness. Among the two styles, counterfactual explanation is more unfair.  \n\nExample numbers (when k = 2000 pairs): \n\nmean SA_IU(NE) = 0.148, mean SA_IU(SM) = 0.178, mean SA_IU(CE) = 0.217\n\np-value[SA_IU(NE), SA_IU(SM)] = 1.2 * 10^{-13}\n\np-value[SA_IU(SM), SA_IU(CE)] = 0.00027\n\np-value[SA_IU(CE), SA_IU(NE)] = 0.00018\n\n\n\n2. Mental Discomfort Individual Unfairness (MD_IU): \n\nmean MD_IU(SM) < mean MD_IU(NE) (significant when k = 400, 1000, 2000, or 4000 pairs)\n\nmean MD_IU(CE) > mean MD_IU(SM) (significant when k = 400, 1000, 2000, or 4000 pairs)\n\nmean MD_IU(CE) < mean MD_IU(NE) (significant when k = 1000, 2000, or 4000 pairs)\n\nConclusion: With respect to Mental Discomfort , introducing either explanation style decreases individual unfairness, which contrasts the individual fairness results for other output metrics.  Among the two styles, counterfactual explanation is more unfair.  \n\n\n\n\n\n3. Perceived Workload Individual Unfairness (PW_IU): \n\nmean PW_IU(SM) > mean PW_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000 pairs)\n\nmean PW_IU(CE) > mean PW_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000 pairs)\n\nConclusion: With respect to Perceived Workload, introducing either explanation style increases individual unfairness. \n\n\n\n\n4. Label time Individual Unfairness (LT_IU): \n\nmean LT_IU(SM) > mean LT_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000 pairs)\n\nmean LT_IU(CE) > mean LT_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000  pairs)\n\nConclusion: With respect to Label time, introducing either explanation style increases individual unfairness. Among the two styles, counterfactual explanation is more unfair.  \n\n\n\nSummary of Individual Fairness conclusions:  \n- Counterfactual explanation is more individually unfair than saliency map. This new finding aligns with our previous finding in  the manuscript that counterfactual explanation shows more evidence of group-wise unfairness.\n- Introducing either explanation style increases individual unfairness with respect to many metrics (stereotype activation, perceived workload, label time) but decreases individual unfairness with respect to mental discomfort.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The research question by the paper is interesting! Though I haven't quite kept up with this subfield, it is one of the first studies I have seen that compares different modes of explanability to specific groups.\n- While three of the metrics are subjective (discomfort, stereotype activation, and percieved workload), I feel the authors justified them well and referred to established metrics (SPANE and NASA-TLX). In the case of stereotype activation, their formulation seemed simple and sensible. \n- Experimental setup seemed clear to me and has a very clear question they want to test.", "reasons_to_reject": "- I could see the argument that stronger models could provide better counterfactuals that may not illicit negative emotions as strongly as the ones in this paper do.\n- I could see that the scope of the paper is relatively narrow.\n- The above two might lead to it having less visibility/interest.", "questions_to_authors": "Question A: 5.4 claims that Asians are less likely to leave a rationale when feeling discomfort. That is, the likelihood of an Asian-identifying annotator leaving a rationale after discomfort was 18% while the white-identifying population was 24%. What are the likelihoods of an Asian-identifying person leaving a rationale at all versus a white-identifying one? i.e, are Asians also less likely to leave a rationale all together?\n\nQuestion B: How granular are the ethnicity identifiers in Prolific?", "paper_topic_and_main_contributions": "The paper studies the effect of giving a specific group (Asians) automatically generated explanations for why piece of content should have human revision (in this case Anti-Asian content). The authors compare two types of explanations (1) saliency maps generated by LIME and (2) counterfactual explanations generated by Polyjuice (based on GPT2). The authors compare the effects of these two explanations (+ no explanations) on several dimensions related to the mental states of the reviewer (time it takes to label, discomfort, etc.). They find that saliency maps outperform counterfactuals.\n\nMain contributions in my opinion are:\n- One of the first papers to investigate the effects of explainable AI on a people group\n- Well-defined experiments that investigate the mental status of each group + the overall accuracy and label times\n- Find that saliency maps outperform counterfactuals:\n     - Asians are likelier to feel mental discomfort\n     - Non-Asians likelier to take more time annotating\n     - Annotators are likelier to be implicitly biased against Asians after the annotation process (Stereotype Activation)\n\nWhile the paper uses relatively smaller models (RoBERTa), one of the really nice things about the paper is that it is less about the strength of the model, but rather their impact on those who see it.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The pictures tend to be a bit small; I couldn't quite read them easily without zooming to 200%."}, "extra_text": {}}, {"review_id": "rYm3Dyo3ay", "reviewer": "Reviewer_eKeg", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for the time you put into reviewing our manuscript! \n\nIn this **Main Rebuttal**, we will first provide a **Response** to each quoted **Concern** in your review. For **Concern 4 - Soundness**, we additionally report new results to further persuade you about the soundness of our most central finding (that counterfactual explanation is more unfair than saliency map). We only give a summary of new results in **Response 4 - Soundness**, but we provide a more detailed description of methodology and actual numbers in the **Appendix - Detailed New Results**.\n \n## Concern 1: Scope of interest  \n“Reasons To Reject: Given such a narrowed area, I'm not sure the scope of the audience for this work.”\n\n## Response 1: \nOur paper claims two novel contributions: \n\nFirst, we identify an overlooked question at the intersection of the NLP explainability and fairness literature. This intersection increasingly draws more attention from the NLP community as illustrated in the literature review by Balkir et al. (2022), but all papers introduced in their review focus on how explanations improve fairness of the NLP models, and overlook the fairness of the explanations themselves. Our problem formulation opens the door to future work examining whether explanations impact people fairly in other high-stakes human-plus-AI settings inside and outside the NLP community, e.g. whether AI-generated explanations of recidivism risk assessment models yield disparate stereotype activation for bond court officials. \n\nSecond, we illustrate how this “fairness of explanation” question can be answered in a **case study** (AI-assisted hate speech detection for content moderators) with statistically significant findings. \n\nYour concern about “a narrowed area” and “scope of the audience for this work” may overlook our first contribution (new research direction).\n\n\n\n## Concern 2: Justification of explanation methods \n“Reasons To Reject: The paper selects two explanation methods without a concrete justification.”\n\n## Response 2: \nIn **Section 2. Background** of our manuscript, we did provide brief justifications for why we choose the saliency map and counterfactual explanation styles: First they contrast each other well. Second, saliency map (which tokens contribute how much to which labels) is intuitive and today’s standard saliency map method (LIME) works for any black-box classifier. Third, counterfactual explanations resemble a real-life mental process (improving hateful content). \n\nOur additional justification is that we need to choose two representative explanation methods under the big umbrella of “local explanations” (instead of “global explanation”). We need to focus on local explanation methods to explain individual predictions (tweets). We believe local explanations methods are better suited in the content moderation setting where understanding individual predictions is a priority compared to describing the average behavior of a machine learning model. Besides, local explanations methods are more helpful for content moderators to quickly digest the rationales of the model predictions and foster better human algorithm alliance.\n\n\n\n## Concern 3: “Reproducibility: 2”\n\n## Response 3: \nThis Reproducibility score is not justified by any text in your reviews. We believe our reported results can be easily replicated since we randomly assigned participants to the three conditions, controlled the quality of responses through participant filter and attention checks, disclosed model specifications and experimental design details in the paper, and enclosed code/data files in the zip folder uploaded to OpenReview with our manuscript. Only the Personally Identifiable Information (e.g. Prolific IDs of human study participants) cannot be shared externally due to IRB regulations. We are also happy to answer any additional data or code-related questions via the “Official Comment” on OpenReview. \n\nTo further assist your reviews of our code and data, below is a functionality summary of the code and data files we included in our zip folder (Supplemental Materials) as uploaded onto OpenReview.\n\nEMNLP_roberta_train.ipynb: code to fine-tune a roberta binary (hate vs. non-hate) classifier.\n \nEMNLP_roberta_test_saliency.ipynb: code to generate saliency maps (LIME) explanations.\n\nEMNLP_roberta_test_counter.ipynb: code to generate predictions for counterfactual explanations\n\nEMNLP_Calculate_all_metrics.ipynb: code to calculate 5 output metrics (accuracy, label time, mental discomfort, perceived workload, and stereotype activation).\n\nEMNLP_statistical_analysis_t_test_all_metrics_final.ipynb: code to conduct t-tests under different conditions (no explanations , saliency maps, counterfactual explanations) and to reproduce all plots included in the paper.  \n\nEMNLP_Counterfactual_Explanations.xlsx: original hate-classified tweets and counterfactual explanations (human-generated counterfactual candidates classified as non-hate by the same NLP classification model)\n\nEMNLP_Human_Study_Data_with_all_Metrics.csv: processed human study data from Qualtrics and Prolific\n\n\n\n## Concern 4: “Soundness: 2”\n\n## Response 4: \nThis Soundness score is not justified by any text in your reviews. We believe our claims are soundly supported by the empirical evidence provided in the paper because our most **central claims** (counterfactual explanations show 1. worse overall performance and 2. more unfairness evidence than saliency map) are supported by several **metric-specific claims** (mental discomfort, accuracy, stereotype activation in Section 5.1 for the overall performance claim; mental discomfort, label time, and stereotype activation in Section 5.2 for the unfairness claim). Each metric-specific claims is further supported by both **t-tests** (Figures 3-5 for overall performance; Figures 6-9 for unfairness) and **qualitative arguments from related social science papers**, e.g. we drew the connection between hetero-centric norms found in Hegarty et al. (2004) to white-dominant norms to justify unfair mental discomfort by counterfactual explanations (Section 5.2 - second paragraph).\n\nTo further substantiate the soundness of our claims, we summarize **new results** from three additional streams of analysis on data from the reported human study. If you want to read the full methodology and actual numbers from each analysis stream, please check the **Appendix**.\n\nFirst, **Heterogeneous treatment effects** analysis with Linear Double Machine Learning (Chetverikov et al., 2016) shows that Counterfactual Explanations are not only group-wise unfair with respect to race, but also group-wise unfair with respect to other demographic features including gender and age. Second, as p-values only show statistical significance but now effect size (American Statistical Association, 2016), our **Effect size** analysis using Cohen’s d score (Cohen, 2013) shows that statistically significant effects presented in paper have small-to-medium effect sizes, necessitating statistical methods to uncover, which was what we did. Third, as we develop a simple and interpretable **Individual Fairness** pipeline based on the intuition that outcomes are individually fair if similar individuals are treated the same (Dwork et al., 2012), we find that counterfactual explanation is more individually unfair than saliency map. This new finding aligns with our previous findings in the manuscript (that counterfactual explanation shows more evidence of **group-wise unfairness**).\n\n\n\n\n\n\n## Concern 5: Presentation improvements\n“Typos Grammar Style And Presentation Improvements: Text in Figure 1 and 2 is too small and hard to read.”\n\n## Response 5: \nIf accepted, we will increase the size of Figures 1 and 2 from double-column to single-column (larger size) in the camera-ready version. In case there is not enough space in the main text, we will provide the larger-sized figures in the camera-ready version’s appendix. \n\n\n\nOnce again, we are super grateful for your time and will be even more so if you consider whether our Responses may clarify any of your Concerns! \n\n\n### References \n1. Esma Balkir, Svetlana Kiritchenko, Isar Nejadgholi, and Kathleen Fraser. 2022. Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models. In Proceedings of the 2nd Workshop on Trustworthy Natural Language Processing (TrustNLP 2022), pages 80–92, Seattle, U.S.A.. Association for Computational Linguistics.\n2. Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W. K., & Chernozhukov, V. (2016). Double machine learning for treatment and causal parameters. 2016.\n3. Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Academic press.\n4. American Statistical Association. (2016). American Statistical Association releases statement on statistical significance and p-values.\n5. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012, January). Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference (pp. 214-226). \n\n\n\n# Appendix - Detailed New Results\n\nPlease note that our decision to add new results is consistent with the EMNLP 2023 policy: “Reporting additional experiments is encouraged, this may be the most effective way to get Reviewers to increase their scores.”\n\nWe report three streams of additional analysis and new results that further enhance the soundness and generalizability of our claims. Since OpenReview doesn’t allow additional uploads for author responses, we will provide the detailed code and figures associated with these new results in the camera-ready version (if accepted).\n\n\n\n##  Heterogeneous treatment effects \nWe estimate Heterogeneous treatment effects using Linear Double Machine Learning (Chetverikov et al., 2016). We use Tree Interpreter to provide a presentation-ready summary of the key features that explain the biggest differences in responsiveness to an intervention. We can provide an additional figure of the decision tree that illustrates this analysis in the camera-ready version. \n\nSpecifically, we find that regarding stereotype activation, *Counterfactual Explanations are not only group-wise unfair with respect to race, but also group-wise unfair with respect to other demographic features including gender and age*. Counterfactual Explanation has higher positive effects to induce stereotype activation for elderly female participants.\n\n\n\n## Effect size \nSince p-value can indicate whether an effect exists, it doesn't provide information about the magnitude of the effect (American Statistical Association 2016). For the readers to better interpret the substantive significance of effects that we discovered in this paper, we calculated Cohen’s d scores and 95% confidence intervals using bootstrapping. \n\nOur results indicate that *statistically significant effects presented in paper have a sensible Cohen’s d score estimates* (small to moderate effects that require statistical methods to be observed (Cohen, 2013)) and reasonable 95% confidence intervals. For example, saliency map yields significantly higher hate speech prediction accuracy than counterfactual explanation (p-value370 = 0.022, Cohen’s d: 0.337, 95% CI: [0.051, 0.625]). A Cohen’s d of 0.337 means that the two group means (Saliency maps v.s. Counterfactual Explanations) differ by 0.337 standard deviations.\n\n\n\n## Individual Fairness\nThe quantitative results we reported in the manuscript come from a group fairness perspective: whether an output feature gets significantly different values across different groups. Another popular perspective in the fairness literature is “individual fairness”, with the intuition that the outcomes are fair if similar individuals get similar outcomes (Dwork et al., 2012). If we can apply this intuition of “individual fairness” into a quantifiable metric, we might evaluate which of the three explanation conditions are more individually unfair with respect to each of the five output metrics (e.g. mental discomfort). Our motivation is that if the new “individual fairness” findings also aligns with the previous “group fairness” findings, e.g. if “counterfactual explanations” are more unfair than “saliency map” from both group fairness and individual fairness perspectives, the new result will further enhance the Soundness of our claims. \n\nOne major challenge in applying the “individual fairness” intuition is to define a task-relevant pairwise distance function to calculate how “similar” any two individuals are. To address this issue, at the start of the survey, we ask each participant 9 multiple choice questions about their individual input features related to the anti-Asian hate speech classification task:\n\nQ1: Have you ever been a victim of online hate speech?\n\n…\n\nQ9: How often do you think that you \"have no place in this world\", \"feel left out\", or \"feel like an outsider\"?\n\nAfter mapping multiple-choice answers to each question to normalized values between 0 and 1, we compute the pairwise distance (absolute difference, averaged across the 9 individual input features above) for every pair of individuals within a given explanation condition. We then develop a simple and interpretable pipeline to compute an “individual unfairness” metric.\n\nGiven output feature X (e.g. mental discomfort) and explanation condition i (e.g. saliency map):\n\n**Step 1**: Find the top *k* pairs of most similar two individuals (with smallest pairwise average distances)\n\n**Step 2**: Within the top k pairs, for the two individuals A and B in each pair with avg_distance(A, B), i.e. distance averaged across their 9 individual input features, to get a pairwise “individual unfairness” metric for A and B, we compute the absolute difference in their output values weighted by [(1-avg_distance(A, B)]. Our rationale is that the more similar two individuals are (i.e. the smaller their avg_distance is), the more “individually unfair” it will be for their output values to differ.\n\n**pairwise_individual_unfairness (A, B) = [1 - avg_distance(A, B)] * | output(A) - output(B) |**\n\n**Step 3**: Calculate the mean of pairwise individual unfairness scores across the k pairs to use as the individual unfairness score of explanation condition i with respect to output feature X.\n\n**Step 4**: perform t-tests between the distributions of output feature X in explanation condition i and the same output feature X in another explanation condition (such as j) to compare “individual unfairness” between conditions i and j with respect to feature X.  \nWe also vary the **number of top pairs considered (k)** as a hyperparameter from {100, 200, 400, 1000, 2000, 4000 pairs} and report the following findings about “individual unfairness” of the three conditions (**NE: no explanation; SM: saliency map; CE: counterfactual explanation**), which remain significant (p-value < 0.05) across multiple settings of k:\n\n1. Stereotype Activation Individual Unfairness (SA_IU): \n\nmean SA_IU(SM) > mean SA_IU(NE) (significant when k = 1000, 2000, or 4000 pairs)\n\nmean SA_IU(CE) > mean SA_IU(SM) (significant when k = 2000, or 4000 pairs)\n\nmean SA_IU(CE) > mean SA_IU(NE) (significant when k = 1000, 2000, or 4000 pairs)\n\nConclusion: With respect to Stereotype Activation, introducing either explanation style increases individual unfairness. Among the two styles, counterfactual explanation is more unfair.  \n\nExample numbers (when k = 2000 pairs): \n\nmean SA_IU(NE) = 0.148, mean SA_IU(SM) = 0.178, mean SA_IU(CE) = 0.217\n\np-value[SA_IU(NE), SA_IU(SM)] = 1.2 * 10^{-13}\n\np-value[SA_IU(SM), SA_IU(CE)] = 0.00027\n\np-value[SA_IU(CE), SA_IU(NE)] = 0.00018\n\n\n\n2. Mental Discomfort Individual Unfairness (MD_IU): \n\nmean MD_IU(SM) < mean MD_IU(NE) (significant when k = 400, 1000, 2000, or 4000 pairs)\n\nmean MD_IU(CE) > mean MD_IU(SM) (significant when k = 400, 1000, 2000, or 4000 pairs)\n\nmean MD_IU(CE) < mean MD_IU(NE) (significant when k = 1000, 2000, or 4000 pairs)\n\nConclusion: With respect to Mental Discomfort , introducing either explanation style decreases individual unfairness, which contrasts the individual fairness results for other output metrics.  Among the two styles, counterfactual explanation is more unfair.  \n\n\n\n\n\n3. Perceived Workload Individual Unfairness (PW_IU): \n\nmean PW_IU(SM) > mean PW_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000 pairs)\n\nmean PW_IU(CE) > mean PW_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000 pairs)\n\nConclusion: With respect to Perceived Workload, introducing either explanation style increases individual unfairness. \n\n\n\n\n4. Label time Individual Unfairness (LT_IU): \n\nmean LT_IU(SM) > mean LT_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000 pairs)\n\nmean LT_IU(CE) > mean LT_IU(NE) (significant when k = 100, 200, 400, 1000, 2000, or 4000  pairs)\n\nConclusion: With respect to Label time, introducing either explanation style increases individual unfairness. Among the two styles, counterfactual explanation is more unfair.  \n\n\n\nSummary of Individual Fairness conclusions:  \n- Counterfactual explanation is more individually unfair than saliency map. This new finding aligns with our previous finding in  the manuscript that counterfactual explanation shows more evidence of group-wise unfairness.\n- Introducing either explanation style increases individual unfairness with respect to many metrics (stereotype activation, perceived workload, label time) but decreases individual unfairness with respect to mental discomfort.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper presents results based on an online human study for fairness explanation toward specific group. This area is less investigated.\nThe paper proposes a new way of measuring working and mental impact for content moderators.", "reasons_to_reject": "The paper is motivated by the need of fair AI generated explanation for content moderators in human-plus-AI system for specific group. Given such a narrowed area, I'm not sure the scope of the audience for this work. \nThe paper select two explanation methods without a concrete justification.\n\n\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper studies the psychological impact of AI generated explanations on different user groups. The paper is motivated by the need of providing fair explanation to content moderators using an AI system. The authors investigate two types of NLP explanation format: saliency maps and counterfactual explanations. To measure the impact, the authors use five metrics: classification accuracy, label time, menal discomfort, perceived workload, and stereotype activation. The results show that saliency map explanations lead to better task performance and less evidence of disparate impact than counterfactual explanations.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Text in Figure 1 and 2 is too small and hard to read.\n"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers found the paper interesting, exciting, and sound. The authors addressed the issues that were brought up in the discussion phase. The only unaddressed criticism is related to the potentially narrow scope.
Decision: Accept-Main