SlowGuess/NAACL_2021_OCR huggingface dataset:
Table 4 shows that the top salient ad hominem  $n$ -grams are intuitively those that are likely to lead to ad hominem. For example, "you're being  $a$ " is used in contexts such as "you're being a hypocrite". A more overt example of a phrase likely to lead to an ad hominem response is "you're a troll". The amount of you-responses in salient ad hominem  $n$ -grams verify our intuition that many ad hominem responses occur in the form of you-responses. Also, we find that there are more salient ad hominem  $n$ -grams than non-ad hominem  $n$ -grams, and that the former generally have higher salience scores. These observations and preliminary experiments suggested that it is useful to consider both types of salient  $n$ -grams to reduce ad hominem.
Top- $k$  Sampling For open domain language generation, top- $k$  sampling (Fan et al., 2018) and top- $p$  nucleus sampling (Holtzman et al., 2019) are popular decoding algorithms that have been shown to maintain topic consistency and promote diversity. We experiment with constrained decoding through top- $k$  sampling, though our technique is also applicable to nucleus sampling. As top- $k$  sampling is a general decoding algorithm that can be used with
Table 4: Top salient  $n$ -grams and their salience scores for ad hominem (AH) and non-ad hominem (non-AH) responses, as calculated from the annotator-labeled subset of ADHOMSINTWEETS.
Algorithm 1: SALIENSIMTOP-k
Data: input tokens x, # top tokens k, # candidate tokens t, # recent tokens r, salient ad hominem average n-grams A, salient non-ad hominem average n-grams B, semantic similarity threshold γ
Result: output tokens y
y = x
while len(y) < max_steps + len(x) do
vocab_logits = model(y)
P' = choose top-k vocab_logits and rescale candidate_tokens = sample t tokens using P'
for cand in candidate_tokens do
if special_condition then	y.append(cand)
continue to While condition	sim_a = similarity(c, A)
sim_b = similarity(c, B)	if sim_a - sim_b <= γ then
y.append(cand)	continue to While condition
if y is x then	y.append候选候选[0])
else	remove last token from y
various language generation models without further tuning or training, expanding upon this technique allows for a computationally-light generalizability.
SALIENSIMTOP- $k$  We reduce the amount of generated ad hominems by encouraging the generation of  $n$ -grams that are semantically dissimilar to salient ad hominem  $n$ -grams and similar to salient non-ad hominem  $n$ -grams. Alg. 1 details constraints we add to top- $k$  sampling. In the for-loop, we iterate through each candidate token. If the current generated output meets a "special_condition" (e.g., backtracking limit, first  $r$  time steps), then we select the current candidate token. Otherwise we retrieve and average DialoGPT's embeddings over the most recently generated  $r$ -gram to calculate  $c$ , an  $e$ -dimensional vector where  $e$  is the size of the token embedding. We similarly compute representations to form  $A$ , a  $j \times e$  matrix of  $j$  salient ad hominem average  $n$ -gram embeddings, and  $B$ , a  $k \times e$  matrix of  $k$  salient non-ad hominem average  $n$ -gram embeddings. We then calculate the average pairwise similarity  $\text{sim\_a} = \frac{1}{j} \sum_{i=1}^{j} \text{sim}(A_i, c)$  where  $A_i$  is the  $i$ -th row of  $A$ , and similarly for sim_b. We select the current token if the difference between the similarities is under a threshold  $\gamma$ , i.e., the current  $r$ -gram is less similar to the ad hominem  $n$ -grams and more similar to the non-ad hominem  $n$ -grams. Otherwise, we backtrack to the previous time step if we iterate through all candidates without finding a suitable one. By limiting the number of times the algorithm can backtrack while gen
Topic	Source	dev	test	avg
BLM	Human	83.3	82.9	83.1
DiaLoGPT	84.2	75.7	80.0
MeToo	Human	80.0	73.7	76.9
DiaLoGPT	85.0	80.0	82.5
Vegan	Human	80.0	70.6	75.3
DiaLoGPT	82.9	82.9	82.9
WFH	Human	77.8	83.3	80.6
DiaLoGPT	92.3	88.4	90.4
Table 5: BERT-based classifier  $\mathbf{F}_1$  scores for ad hominem responses across topics and response sources. The classifier does relatively well across topics and sources.
erating a sample, this algorithm adds a constant amount of computational resources compared to the original, non-constrained decoding.
Implementation Details In our experiments, we set  $k = 40$  (commonly used in previous generation tasks (Radford et al., 2019)). With parameter tuning, we find  $t = 10$  and  $\gamma = 0$  effective for our setup. We use  $r = 5$  to compare the averaged embedding of the most recent 5-gram with those of salient 3-, 4-, and 5-grams. Additionally, we use cosine similarity as the similarity metric and our "special_condition" includes either a) a limit of 5 for backtracking or b) the first  $r$  time steps.
6.1 Identifying Ad Hominems
**Annotation** Across all rounds of annotations, the average WAWA scores include a precision of 0.82, recall of 0.92, and  $\mathrm{F}_1$  of 0.87, indicating moderately high majority agreement. Generally, the agreement scores for the human responses are slightly higher than those for the DialogGPT responses—we hypothesize that the former tend to be more coherent and longer, and thus more informative.
Ad Hominem Classifier The resulting BERT-based classifier has an overall dev  $\mathrm{F}_1$  score of 83.3% and a test  $\mathrm{F}_1$  score of 80.0% for ad hominem. The DialoGPT-based classifier has a dev  $\mathrm{F}_1$  score of 74.6% and a test  $\mathrm{F}_1$  score of 72.6%, supporting our use of the BERT-based classifier to automatically detect ad hominem in the rest of this work. $^{10}$  The full breakdown of  $\mathrm{F}_1$  scores across topics and response sources is shown in Table 5 and Appendix Table 9.
Figure 1:  $\%$  of classifier-labeled ad hominem occurrences across human, DialogGPT, and fine-tuned DialogGPT responses ("FXX"). There are 14.5K responses (to all posts in ADHOMINTWEETS) per response source. Human and DialogGPT responses contain more ad hominem for BLM and MeToo, followed by Vegan and then WFH. Fine-tuning on topics with more/fewer ad hominem results in more/fewer ad hominem generated across topics.
6.2 Ad Hominem Analysis
Ad Hominem Categories By comparing ad hominem types across the manually-annotated human and DialogGPT responses, we find that ad hominems in human responses frequently occur in the forms of "condescension" and "ignorance", while ad hominems in DialogGPT responses occur in the forms of "ignorance" and "other" types (Table 11 in the Appendix). These results indicate that responses from different sources and topics are likely to contain different ad hominems. Formally categorizing ad hominems allows for more consistent annotations and a better understanding of the types DialogGPT is prone to generate.
DialogPT Responses The classifier enables us to perform a large-scale study of ad hominem trends across various contexts for the entire AD-HOMINTWEETS dataset. Figure 1 shows the percentage of ad hominem responses to posts across topics and response sources. Focusing on the "Human" and "DialogPT" bars for each topic, we see that ad hominem responses are present across all topics for both response sources. Additionally, ad hominem responses occur more frequently in discussions related to BLM and MeToo and less frequently in discussions related to Vegan and WFH. Vegan discussions also seem to attract more ad hominem responses than WFH discussions. The relatively higher rates of ad hominem responses in topics related to marginalized communities indicate the elevated potential for harm towards these communities.
This result additionally suggests that control techniques that rely on signal from auto-regressive decoder models as discriminators may encounter more noise.
DialoGPT Trigger PPLM FwH SS FwH+SS
(a) 14.5K classifier-labeled responses (to all posts in AD-HOMINTWEETS) per response source.
(b) 400 human-labeled responses (to posts randomly chosen from ADHOMINTWEETS) across topics per response source.
Figure 2: Reducing ad hominem in generated responses.  $\mathrm{F_{WFH}}$  is fine-tuned on WFH data and SS is SALIENSIMTOP-  $k$ . Results suggest all ad hominem reduction techniques are effective compared to the original DialoGPT. SS is the most effective individual method, outperforming  $\mathrm{F_{WFH}}$ , Trigger, and PPLM baselines.  $\mathrm{F_{WFH} + SS}$  could further reduce the amount of ad hominem responses generated.
Fine-tuned DialogGPT Responses Figure 1 also shows that fine-tuning on datasets that contain more ad hominem responses leads to more generation of ad hominem responses across topics. From these results, we infer that the original DialogGPT (which was fine-tuned from GPT-2) was trained on a dataset that likely contained relatively more rather than fewer ad hominem. Additionally, fine-tuning on a carefully chosen dataset can reduce the quantity of generated ad hominem and associated harms.
6.3 Ad Hominem Reduction
Baselines We compare techniques from two classes of harm reduction methods for language generation: data-based and decoding-based. Gehman et al. (2020) define data-based techniques as those where further model training on more data is necessary and decoding-based techniques as those where the generation strategy is changed without changing model parameters. For our main decoding-based SALIENSIMTOP- $k$  technique, we
Post: Many are trying to co-opt and mischaracterize the #blm movement. We won't allow it!
Src: DialogGPT
Src: DialoGPT + SALIENSIMTOP-k
Resp: This is so true.
Src:  $\mathrm{F_{WFH}} + \mathrm{SALIENSIMTOP - k}$
Resp: I'm in the minority and I don't think it's possible to make it a better movement.
Table 6: Examples of responses generated from different sources.  $\mathrm{F}_{\mathrm{WFH}}$  is DialoGPT fine-tuned on WFH.
introduce four baselines to span the different classes of harm reduction techniques. The first baseline is simply the original DialoGPT. Our databased reduction baseline is DialoGPT fine-tuned on the WFH dataset, as described in Sec. 3. For the first decoding-based baseline, we rely on a gradient-based method post-training to find a "trigger phrase", which is then attached to a prompt at inference time to influence the generated output (Wallace et al., 2019). Sheng et al. (2020) further propose a framework to use these triggers to control societal biases, and we use these methods to find a trigger that can induce DialoGPT to generate fewer ad hominems and more non-ad hominems when prepended to posts about different topics. For the second decoding-based baseline, we use the Plug and Play Language Model (PPLM) proposed by Dathathri et al. (2020), which guides a pre-trained language model's generated output using gradients from attribute classifiers. $^{12}$
Human Annotation To verify ad hominem trends from the automatic evaluation, we randomly select 100 samples from each [reduction technique, topic] pair for additional human annotation.
General Trends Classifier and human evaluations for techniques to reduce ad hominem are in Figure 2, and examples of generated responses are in Table 6. The classifier-labeled results allow us to evaluate  $14.5\mathrm{K}$  samples across all topics per response source, and the human-labeled results allow us to more accurately evaluate a smaller set of samples. Overall, the trends for classifier and human evaluations are similar, and the evaluations suggest that all ad hominem reduction techniques are effective compared to the original DialoGPT. Furthermore, SALIENSIMTOP-  $k$  is more effective than the other individual techniques, and combining fine-tuning and SALIENSIMTOP-  $k$  has promise for further reducing the amount of generated ad
11Table 13 in the Appendix includes examples generated by the fine-tuned models.
12More details are in Appendix A.3 and A.4.
Source	BLM	MeToo	Vegan	WFH	Avg
C	R	C	R	C	R	C	R	C	R
DialoGPT	4.5	3.0	4.3	3.5	4.2	3.2	4.3	2.6	4.3	3.1
Trigger	4.5	3.0	4.5	3.2	4.3	2.8	4.4	2.8	4.4	3.0
PPLM	4.1	3.0	3.7	3.0	3.6	2.9	3.8	2.6	3.8	2.9
FWFH	4.2	3.6	4.1	3.6	3.6	3.4	4.0	3.7	4.0	3.6
SS	4.5	3.2	4.4	3.2	4.1	3.6	4.4	3.1	4.4	4.1
FWFH+SS	3.8	3.1	3.8	3.6	3.9	3.2	4.1	4.1	3.9	3.5
Table 7: Average coherence (C) and relevance (R) of responses across sources and topics, each on a scale of 1-5, where higher scores are better. Each value is averaged over 25 random samples (and 3 annotators per sample). The highest score(s) per column are bolded, and the lowest score(s) per column are underlined. Trigger generates slightly more coherent responses, though at the cost of relevance. PPLM generates responses that are relatively lower in both coherence and relevance. SS maintains a decent balance of coherence and relevance, and  $\mathrm{F_{WFH} + SS}$  produces slightly less coherent responses that are mixed in relevance.
hominems.
For SALIENSIMTOP-  $k$ , limiting the number of times we backtrack to previous time steps ensures that the algorithm is not significantly slower compared to the original top-  $k$  sampling algorithm. Empirically, we find that using SALIENSIMTOP-  $k$  with a backtracking limit of 5 on the original DialoGPT results in  $13\%$  of the decoding operations being "non-forward" operations, where the set of decoding operations are: a) choosing the current token and moving forward to the next timestep, b) looking for an alternate token at the same timestep, or c) moving backward to a previous timestep. When applying constrained decoding to DialoGPT fine-tuned on WFH,  $10\%$  of the operations are nonforward operations. Since ad hominems are less common than non-ad hominems, the algorithm is able to proceed with the first sampled candidate token in most time steps. Additionally, models or topics that are inclined to generate more ad hominems incur more non-forward operations.
Coherence and Relevance Evaluation To ensure that the ad hominem reduction techniques do not affect the quality of the generated responses, we have annotators label the coherence and relevance of a response to a post, both on a scale of 1 to 5, where a higher score is better. The trigger method produces samples that are relatively more coherent, although at the cost of lower relevance to the post. PPLM generates responses that are relatively lower in both coherence and relevance. SALIENSIMTOP-  $k$  manages to maintain a decent balance of generating both coherent and relevant responses. Combining SALIENSIMTOP-  $k$  with finetuning on WFH data results in responses that are slightly less coherent and mixed in relevance for
different topics. $^{13}$  Spearman's correlation is moderately high (0.46) for relevance and a bit lower for coherence (0.38), indicating the task subjectivity.
Discussion The collective results indicate that SALIENSIMTOP-  $k$  is an effective standalone ad hominem reduction technique that maintains generated text quality; while it can be combined with other techniques to further reduce ad hominem, one should carefully evaluate the trade-offs between response coherence and relevance. Additionally, for reducing harmful language types that are more subjective or difficult to detect, straightforward control techniques that rely on salient  $n$ -grams may be more useful than techniques that rely on noisier signals from classifiers.
Ad hominem responses from dialogue systems are offensive, stall conversations, and are especially harmful for marginalized communities. We analyze responses to find that discussions on topics that affect marginalized groups contain more ad hominems. Through a novel constrained decoding technique, we decrease the amount of ad hominems generated from dialogue systems while keeping the response quality comparable. Furthermore, our method can be easily applied to other pre-trained language generation models and other subtle yet harmful language. More broadly, our work strives to understand ad hominems in the context of harms in conversational systems.
This work identifies personal attacks in responses generated by dialogue systems, quantifies the dis
13Example generations across sources are in Appendix Table 14.
proportionate amount generated for topics concerning marginalized populations, and proposes methods to reduce ad hominem-related harms.
Dataset We collect an English dataset from Twitter and ensure that personal information (e.g., usernames, emails,urls) is discarded. We also collect crowd-sourced annotations for this dataset through Mechanical Turk, where we ask for judgements of whether a response contains ad hominems for a given post, and the coherence and relevance of a response. No information about the annotators are collected from the annotation tasks. The annotation information (pay per amount of work, guidelines) is in the Appendix.
One annotation aspect that we did not control for is whether the annotators themselves are from marginalized communities. When measuring harms towards different demographics, it is important to consider the lived experiences of those groups and how these experiences may affect our analyses. Future work includes specifically collecting annotations from marginalized groups.
Additionally, we analyze ad hominems in responses to four Twitter topics and from one dialogue model, which leaves much room for exploring the generalizability of the trends we see.
Techniques In terms of dual-use harms, our constrained decoding technique could potentially be used to amplify rather than reduce ad hominems (or other harmful language). However, we believe that by being transparent about this technique and releasing the associated code and data, we can better counter attempts of malicious misuse.
Furthermore, to perform a large-scale analysis of ad hominem across different contexts, we build an automatic classifier. While we spent much effort on collecting representative train/dev/test datasets and verifying classifier quality and observed trends with human labels, collecting more (diverse) data could help further improve the classifier accuracy and robustness. In the meantime, we think this work introduces an important perspective of how ad hominem in dialogue systems reinforce unequal harms and effective reduction methods.
We would like to thank members of the PLUS Lab and the anonymous reviewers for the helpful feedback, and Jason Teoh for the many discussions. This paper is supported in part by NSF IIS 1927554 and by the CwC program under Con
tract W911NF-15-1-0543 with the US Defense Advanced Research Projects Agency (DARPA). The views expressed are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government.
Anusha Balakrishnan, Jinfeng Rao, Kartikeya Upasani, Michael White, and Rajen Subba. 2019. Constrained decoding for neural nlg from compositional representations in task-oriented dialogue. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 831-844.
Luke Breitfeller, Emily Ahn, David Jurgens, and Yulia Tsvetkov. 2019. Finding microaggressions in the wild: A case for locating elusive phenomena in social media posts. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1664-1674.
Ying Chen, Yilu Zhou, Sencun Zhu, and Heng Xu. 2012. Detecting offensive language in social media to protect adolescent online safety. In Proceedings of the 2012 ASE/IEEE International Conference on Social Computing and 2012 ASE/IEEE International Conference on Privacy, Security, Risk and Trust, SOCIALCOM-PASSAT '12, page 71-80, USA. IEEE Computer Society.
Amanda Cercas Curry and Verena Rieser. 2018. # metoo alexa: How conversational systems respond to sexual harassment. In Proceedings of the Second ACL Workshop on Ethics in Natural Language Processing, pages 7-14.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representations.
Pieter Delobelle, Murilo Cunha, Eric Massip Cano, Jeroen Peperkamp, and Bettina Berendt. 2019. Computational ad hominem detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 203-209.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186.
Emily Dinan, Samuel Humeau, Bharath Chintagunta, and Jason Weston. 2019. Build it break it fix it for dialogue safety: Robustness from adversarial human attack. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4529-4538.
Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015. Hate speech detection with comment embeddings. In Proceedings of the 24th International Conference on World Wide Web, WWW '15 Companion, page 29-30, New York, NY, USA. Association for Computing Machinery.
Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889-898.
Sam Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing - Findings (EMNLP-Findings).
Trudy Govier. 1993. When logic meets politics: testimony, distrust, and rhetorical disadvantage. Informal Logic, 15(2).
Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein. 2018. Before name-calling: Dynamics and triggers of ad hominem fallacies in web argumentation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 386-396.
Hans Hansen. 2020. Fallacies. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy, summer 2020 edition. Metaphysics Research Lab, Stanford University.
Chris Hokamp and Qun Liu. 2017. Lexically constrained decoding for sequence generation using grid beam search. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1535-1546.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. In International Conference on Learning Representations.
Chandra Khatri, Behnam Hedayatnia, Rahul Goel, Anushree Venkatesh, Raefer Gabriel, and Arindam Mandal. 2018. Detecting offensive content in open-domain conversations using two stage semi-supervision. arXiv preprint arXiv:1811.12900.