SlowGuess/NAACL_2021_OCR huggingface dataset:
Emily Sheng $^{1}$ , Kai-Wei Chang $^{2}$ , Premkumar Natarajan $^{1}$ , Nanyun Peng $^{1,2}$
<sup>1</sup> Information Sciences Institute, University of Southern California
<sup>2</sup> Computer Science Department, University of California, Los Angeles
{ewsheng, pnataraj}@isi.edu, {kwchang, violetpeng}@cs.ucla.edu
Ad hominem attacks are those that target some feature of a person's character instead of the position the person is maintaining. These attacks are harmful because they propagate implicit biases and diminish a person's credibility. Since dialogue systems respond directly to user input, it is important to study ad hominem in dialogue responses. To this end, we propose categories of ad hominems, compose an annotated dataset, and build a classifier to analyze human and dialogue system responses to English Twitter posts. We specifically compare responses to Twitter topics about marginalized communities (#Black-LivesMatter, #MeToo) versus other topics (#Vegan, #WFH), because the abusive language of ad hominems could further amplify the skew of power away from marginalized populations. Furthermore, we propose a constrained decoding technique that uses salient  $n$ -gram similarity as a soft constraint for top- $k$  sampling to reduce the amount of ad hominems generated. Our results indicate that 1) responses from both humans and DialogoGPT contain more ad hominems for discussions around marginalized communities, 2) different quantities of ad hominems in the training data can influence the likelihood of generating ad hominems, and 3) we can use constrained decoding techniques to reduce ad hominems in generated dialogue responses.
Ad hominem attacks an opponent's character or identity instead of the points the opponent is making, and can exist in any conversational setting between two or more entities. From an argumentation perspective, ad hominem are fallacies, and fallacies rely on faulty reasoning to advance a point (Hansen, 2020). These ad hominem fallacies are related to abusive language, toxicity, and microaggressions, and can be expressed with both subtle and explicitly offensive language. Table 1 presents
Post: Many are trying to co-opt and mischaracterize the #blacklivesmatter movement. We won't allow it!
Resp: I hate how much of a victim complex you guys have.
Post: You're the reason we need the #MeToo movement.
Resp: Nice try, kiddo.
Post: Stop eating them if you don't want them to go extinct! #govegan
Resp: I don't like your username
Table 1: Ad hominem responses to Twitter posts.
examples of ad hominem responses to Twitter posts. Undesirable in any response, ad hominems are unproductive in furthering a meaningful discussion and can reinforce falsehoods. However, these attacks appeal to emotions and implicit biases to argue a point, and are thus often effectively harmful regardless of whether the attacks are true, recognized, or retracted (Yap, 2013).
Our work is motivated by this fallacy's potential to amplify the spread of harmful societal biases. For communities that are already disproportionately harmed by societal power inequalities, ad hominem further amplify the power imbalance. Tone policing is a type of ad hominem that seeks to regulate the emotions that a person (usually of a marginalized population) can use to deliver their points (e.g., not too angrily), thereby altogether invalidating the style of delivery, the person's competence, and the points being conveyed. Besides directly experiencing ad hominem attacks, marginalized groups could also be disproportionately discouraged from using technologies that propagate these attacks, since abusive language from a technology can deter people from using the technology (Sood et al., 2012b).
The goal of this study is to analyze ad hominem in dialogue system- and human-generated responses for topics that vary in impact to marginalized populations. Through analysis, we formulate techniques to reduce ad hominem responses and thus the associated harms, which is especially important for dialogue systems since these systems
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 750-767 June 6-11, 2021. ©2021 Association for Computational Linguistics
directly interact with users.
We analyze responses from DialogGPT (Zhang et al., 2020a) and humans to English Twitter posts. Specifically, we compare responses to Twitter topics about marginalized communities (#Black-LivesMatter, #MeToo) versus other topics (#Vegan, #WFH). Through human annotation and trained classifiers, we find that ad hominems exist in both human and DialogGPT responses. Across response sources, there are more ad hominems in #Black-LivesMatter- and #MeToo-related responses, fewer in #Vegan-related responses, and even fewer in #WFH-related responses. The presence of more ad hominems in responses to social issues that concern marginalized groups has troubling implications about the amplified harms toward these groups.
Given our analysis, we further propose a constrained decoding algorithm to reduce the amount of ad hominems generated by dialogue systems. By using salient  $n$ -gram similarity to apply soft constraints to top- $k$  sampling, our proposed technique is simple, extensible to reducing other harms, and does not require much additional computation. At each decoding time step, the technique compares the similarity between the current generated output and salient ad hominem versus non-ad hominem  $n$ -grams, possibly selecting alternative token candidates to generate. This technique is effective at reducing the amount of ad hominems generated across topics while maintaining coherence and relevance.
Our main contribution is a novel analysis of ad hominem responses generated by humans and DialogPT across topics varying in impact to marginalized communities. For this analysis, we propose empirically-derived ad hominem categories that are further verified through annotation. Furthermore, we build a new dataset of Twitter posts paired with human- and DialogPT-generated responses, where the responses have ad hominem-related labels. Finally, we devise a constrained decoding technique that uses salient  $n$ -gram similarity to steer top- $k$  sampling away from ad hominem responses. We release data and code at https://github.com/ewsheng/ad-hom-in-dialogue.
This work is related to a broad spectrum of topics, including prior definitions of ad hominems and how ad hominems facilitate biases. Also, analyzing ad
hominems in dialogue systems is related to examining offensive language and other harms. Lastly, we discuss existing constrained decoding methods.
Ad Hominems In the argumentation literature, theoretical ad hominems include the abusive (attack on the opponent's character), tu quoque ("he did it first"), circumstantial (accusation of hypocrisy), and guilt by association (associating the opponent with someone with low credibility) (Walton, 1998; Woods, 2007). Wijze (2003) criticizes that these textbook examples are not realistic in conversation. For more empirical categories, Habernal et al. (2018) propose ad hominem types based on analysis of Reddit's ChangeMyView discussion threads, and Delobelle et al. (2019) analyze the name-calling and abusive categories. Moreover, Wulczyn et al. (2017) use classifiers for a large-scale analysis of personal attacks in Wikipedia comments. We build upon prior works to define and analyze ad hominems in a conversational setting.
Additionally, Yap (2013) discusses the harmful effects of implicit biases in forming and evaluating ad hominem. They emphasize that ad hominem attacks can be harmful to a person's credibility and expertise even if the attack is recognized as fallacious and irrelevant to the argument. In particular, because societal norms allow biases and stereotypes to detract from a person's credibility or expertise, the use of ad hominem can further diminish the rhetorical credibility (Govier, 1993) of marginalized groups.
Offensive Language Detection Ad hominem occurs in many forms and are related to different types of offensive language, including abusive language (Yin et al., 2009; Chen et al., 2012; Nobata et al., 2016), hate speech (Warner and Hirschberg, 2012; Kwok and Wang, 2013; Djuric et al., 2015), profanity (Sood et al., 2012a), and the more subtle forms of microaggressions (Breitfeller et al., 2019) and projecting biases and stereotypes through power differentials in language (Sap et al., 2020). Ranging from outright insults to condescension, ad hominems are a form of offensive language that is difficult to comprehensively and objectively define. Nonetheless, these responses are important to characterize, since they can irreparably damage a person's credibility. It is also generally important to identify these subtle forms of offensive language, since it is unclear if existing offensive language detection techniques are equally effective for these subtle forms.
Harms in Dialogue Systems Conversational systems are known to perpetuate several types of harms. Ruane et al. (2019) caution about harms that can result from using conversational systems and propose striving for trust and transparency; Roller et al. (2020) suggest techniques for chatbot safety. For analysis, Sheng et al. (2019) evaluate societal biases in language generation, Curry and Rieser (2018) study how conversational systems respond to sexual harassment, and Khatri et al. (2018) detect offensive content with a semi-supervised approach. To reduce harms, Sheng et al. (2020) present a framework for controlling biases in language generation, and Dinan et al. (2019) show how adversarial attacks can make models more robust to offensive language usage from humans.
Constrained Decoding For constrained decoding, prior works focus on incorporating words or phrases (as hard or soft constraints) into the decoded output. Swanson et al. (2014) and Balakrishnan et al. (2019) use parse trees among other techniques to enforce constraints in the generated text. Hokamp and Liu (2017); Post and Vilar (2018) propose variants of Grid Beam Search, which generate output that include lexical constraints. Miao et al. (2019); Zhang et al. (2020b); Susanto et al. (2020) explore insertion-based non-autoregressive decoding algorithms. To be compatible with an autoregressive model like DialogGPT and effective for open-domain generation, we apply constrained decoding to top- $k$  sampling. Our method also differs from these prior works in that it imposes soft constraints to not generate phrases that are likely to lead to ad hominems. Decoding-time techniques that can be used to reduce harmful language generation, e.g., the Plug and Play Language Model (PPLM) (Dathathri et al., 2020), are most relevant to our technique.
3 Dataset and Model Setup
This section describes the dataset collection process and the dialogue model variations we analyze.
Dataset Collection Our goal is to understand how ad hominem responses differ across discussions that vary in impact and relevance to marginalized groups. To that end, we extract English [post, response] pairs on different topics from Twitter and also use DialogGPT to generate responses for all collected posts. We refer to this collective dataset as the ADHOMINTWEETS dataset.
Relevant topics are divided into polarizing (i.e.,
Topic	Polarizing topic	Affects marginalized group	# [post, human resp] pairs
BLM	yes	yes	4,037
MeToo	yes	yes	2,859
Vegan	yes	no	3,697
WFH	no	no	3,992
Total	-	-	14,585
Table 2: Topics, rationales, and statistics for the human response subset from the ADHOMINTWEETS dataset.
controversial) and non-polarizing; we expect there to be more strong opinions for the polarizing topics and thus more ad hominem responses for those topics. For this study, we choose the topic WFH ("work from home") as a non-polarizing topic and collect Twitter posts that include the hashtag #wfh or #workingfromhome. Polarizing topics can further be divided into those that are directly relevant to marginalized communities and those that are not. For the latter, we choose the topic Vegan and collect posts that include any of the hashtags: #vegan, #veganism, #govegan, or #veganlife. $^{1}$  For polarizing topics that are directly relevant to marginalized groups, we focus on the topics BLM (from #black-livesmatter posts) and MeToo (from #metoo posts). #blacklivesmatter is related to the "justice, healing, and freedom to Black people across the globe", $^{2}$  and #metoo is related to the movement against sexual violence. $^{3}$  In total, we collect 14,585 [post, response] pairs of Tweets posted between Aug. 7 and Oct. 29, 2020; detailed data statistics are in Table 2. We replace all usernames andurls with special placeholders to better anonymize the data. Models In this work, we analyze responses from theilioGPT (Zhang et al., 2020a) dialogue model.ilioGPT was originally trained on web data, and then was further fine-tuned for multi-turn conversational capabilities on Reddit data. Since models can vary in harm depending on the training data, we compare responses from the original medium-sizedilioGPT to responses fromilioGPT separately fine-tuned on each of the four topics from the human response subset of ADHOMINTWEETS. $^{4}$
4 Identifying Ad Hominem Responses
It is generally difficult to settle on a comprehensive list of ad hominem categories. We build
<sup>1</sup>Habernal et al. (2018) find that vegan-related topics are one of the top topics that contain ad hominem in their study.
$^{2}$ https://blacklivesmatter.com
<sup>3</sup>https://metoomvmt.org
4 More details are in Appendix A.2.
AH Type	Topic	Post	Response
Stupidity	BLM	Together. #blacklivesmatter	That's a dumb thing to say.
Ignorance	BLM	Your all welcome to join in on the #blm movement!	You mean "you're"
Trolling/Lying	Vegan	It's time to end intensive meat production...#vegan	You must be a troll.
Bias	BLM	This is why people are protesting, this is why the #BLM movement is necessary.	You're racist because you focus on race.
Condescension	MeToo	3 years into #MeToo era, real apologies are few and far between	Can you stay out of grown folks' business...
Other	Vegan	It's not a 'personal choice' when a 'victim' is involved. #GoVegan	You're better than this.
Non-AH	WFH	#WFH benefit: no co-worker judgement microwaving fish for lunch	The smell of fish is deadly.
Table 3: Ad hominem (AH) categories. The post provides context to analyze ad hominems in the response.
upon the work of Habernal et al. (2018) to devise ad hominem categories that are both empirically-motivated and can be annotated with high inter-annotator agreement. We specifically include categories such as "ignorance" and "condescension" to cover more subtle forms of personal attacks (e.g., tone policing, mansplaining) that could further diminish the credibility of those who are already marginalized. We also limit the definition of ad hominem to personal attacks towards the author of the post and not a third person.
4.1 Human Annotation
We collect human annotations that can then be used for analysis and training a classifier to automatically label ad hominem. Although Habernal et al. (2018) propose a similar typology of ad hominem, there is no existing dataset annotated with their empirically-derived categories. Moreover, we study ad hominem in casual conversational settings. For these reasons, we annotate a subset of ADHOMINTWEETS with ad hominem information. To measure inter-annotator agreement, we calculate the Worker Agreement With Aggregate (WAWA) score, following Ning et al. (2020). The WAWA score compares the majority votes against each annotator and micro-averages the resulting precision, recall, and  $\mathrm{F}_1$  scores.[5]
Heuristics for Ad Hominem Ad hominem responses are relatively rare and range broadly from explicit to more subtle forms. For more effective annotation, we use heuristics to choose [post, response] pairs where the response is likely to be an ad hominem. In preliminary analyses, we find that responses that contain certain "you"-phrases such
as "you are" are more likely to have ad hominem. We call these responses you-responses.<sup>6</sup> In addition to pairs with you-responses, we also collect random pairs without you-responses for annotation to ensure that the annotated samples are representative of different ad hominem.
Annotation Task We ask annotators on Mechanical Turk to read a post and response and determine whether the response contains any ad hominem(s) towards the person who made the post. We divide ad hominem into the following categories: stupidity, ignorance, trolling/lying, bias, condescension, and other; examples are in Table 3.7
Annotation Round 1 The goal for the first round of human annotation is to collect enough data to train an ad hominem classifier. To balance targeted and random samples, for each topic (BLM, MeToo, Vegan, WFH) and response source (human, DarioGPT) pair, we randomly select 150 [post, response] pairs with you-responses and another 150 pairs without you-responses for annotation. In total, we gather 2,400 [post, response] pairs that are then annotated through Mechanical Turk.
Additional Annotations We conduct three more rounds of annotations to retrieve more ad hominem responses. For the second and third rounds, we use an ad hominem classifier trained on data from all previous rounds (with the same architecture and hyperparameters as the final classifier in Sec. 4.2) to label unseen samples in ADHOMINTWEETS. We then select a balanced amount of automatically-labeled ad hominem and non-ad hominem from each [topic, response source] pair to annotate.
Some topics (e.g., WFH and Vegan) prompt fewer ad hominem responses, so it is difficult to
There are also other agreement metrics such as Krippendorff's alpha, but because we expect our data to have many more non-ad hominem compared to ad hominem responses, alpha scores can be misleading—the WAWA score gives a more appropriate estimate of annotator agreement.
Full set of you-responses is in Appendix A.1.
Full details are in Appendix A.7.
For each [topic, response source] pair, we choose 150 samples for Round 2 and 100 samples for Round 3.
find enough of these responses "in the wild" to train a more accurate classifier. Our solution is to manually take the responses annotated as ad hominem and pair them with WFH or Vegan posts. To verify that these new pairs contain ad hominem responses, we run a fourth round of annotation on these pairs and only keep the ones where the majority of annotators label the response as an ad hominem to the post. We combine majority annotations across all rounds of annotations to train the final ad hominem classifier used for analysis.
4.2 Ad Hominem Classifier
For large-scale analysis of ad hominems in human and dialogue system responses, we rely on classifier annotation. To simplify the learning problem, we condense the different ad hominem categories into a binary yes/no scheme, where "yes" indicates the presence of any type and quantity of ad hominems in the response given the post. We build a classifier to automatically label whether a response contains ad hominems for a given post by fine-tuning a BERT (Devlin et al., 2019) model with the input format "[CLS] POST [SEP] RESPONSE [SEP]". We additionally include comparisons to a baseline classifier built on top of DialogGPT to similarly label whether a post and response pair indicates the presence of an ad hominem response. This baseline classifier allows a comparative evaluation of a bi-directional encoder model versus an auto-regressive decoder model for ad hominem classification and how this difference may affect the quality of control techniques that rely on the latter (e.g., PPLM (Dathathri et al., 2020), GeDi (Krause et al., 2020)). Appendix A.2 includes more details of our model implementation and data statistics (Table 8).
Ultimately, the goal is to train an ad hominem detection classifier that has high accuracy across sources and topics, so we curate the dev and test datasets to be balanced across topics, response sources, and ad hominem versus non-ad hominem samples (through downsampling). Because of the natural imbalance of ad hominem responses for different topics, ad hominem responses for topics like WFH are relatively sparse compared to those for topics like BLM. We automatically augment our training set to combat this sparsity. First, we accumulate all posts and responses not present in the dev and test sets. Next, we choose a random post to pair with a random labeled response to form
a new sample. We generate these new data samples to roughly balance the number of samples across topics and across ad hominems versus non-ad hominems for each topic. These new combinations of [post, response] pairs help de-emphasize spurious correlations between topics and classifier labels.
Since the automatic augmentation reduces emphasis on the post when predicting the presence of ad hominem in the response, a natural question is if the post is really necessary to gauge whether the response contains ad hominem. The answer is mixed—for example, the response "you're a troll" is an ad hominem for any post. However, the response "those who promote veganism are arrogant fools" is an ad hominem given the post "everyone should follow veganism", but not an ad hominem given the post "I don't understand veganism". Empirically, by limiting the classifier input to only responses, the classifier performs worse than if it has both the post and response as input.[9]
5 Reducing Ad Hominem Responses
Inspired by the success of  $n$ -gram features in detecting abusive language by Nobata et al. (2016), we propose a constrained decoding algorithm to discourage the model from generating  $n$ -grams that are semantically similar to salient  $n$ -grams found in ad hominem responses. While we motivate this technique within the context of ad hominem, the technique is applicable to other subtle harms (e.g., microaggressions) in language generation.
A naive method to generate fewer ad hominem is to block words that are likely to occur in ad hominem. However, ad hominem are contextually determined, meaning that phrases are a better indicator than words, thus motivating our use of  $n$ -grams. Additionally, our algorithm uses soft constraints because there are no words or phrases that always indicate the presence of an ad hominem. In this section, we describe how our technique SALIENSIMTOP- $k$  extends top- $k$  sampling by incorporating  $n$ -gram similarity constraints.
Salient  $n$ -grams We define salient ad hominem  $n$ -grams to be  $n$ -grams that appear more frequently in ad hominem responses than in non-ad hominem responses. Similarly, salient non-ad hominem  $n$
<sup>9</sup>By randomly forming new (post, response) pairs during augmentation, we do not explicitly account for the responses that are context-specific; however, we find the context-specific responses to be relatively rare and that our augmentation empirically results in a more robust classifier.
AH n-gram	Score	non-AH n-gram	Score
serious or not	15.0	thank you for	18.8
don’t know what	13.0	thanks for sharing	8.9
how can you	11.0	i think it’s	8.9
you’re a troll	11.0	you are right	8.9
you’re being a	11.0	is the best	8.9
grams appear more frequently in non-ad hominem responses than in ad hominem responses. We use the salience score as defined by Li et al. (2018):
$$
\mathcal {S} (u, a) = \frac {\operatorname {c o u n t} (u , \mathcal {D} _ {a}) + \lambda}{\left(\sum_ {a ^ {\prime} \in \mathcal {A} , a ^ {\prime} \neq a} \operatorname {c o u n t} (u , \mathcal {D} _ {a ^ {\prime}})\right) + \lambda}. \tag {1}
$$
In Eq. (1),  $u$  denotes an  $n$ -gram,  $\mathcal{D} = \{(s_1, a_1), \ldots, (s_m, a_m)\}$  is a corpus where each sample is a sentence  $s_i$  labeled with attribute  $a_i$ .  $\mathcal{D}_a$  is therefore the set of sentences in the corpus with the same attribute  $a$ .  $\mathcal{A}$  is the set of possible attributes (e.g., ad hominem or non-ad hominem). We define the  $n$ -gram  $u$  to be salient for the attribute  $a$  if  $S(u, a) \geq \varphi$ . We find setting the smoothing parameter  $\lambda = 0.5$  and threshold  $\varphi = 5.5$  effective for our experiments, and we compute the salience of 3-, 4-, and 5-grams.