Venue: NAACL 2021

# Aspect-Controlled Neural Argument Generation

Benjamin Schiller

Johannes Daxenberger and Iryna Gurevych

Ubiquitous Knowledge Processing Lab

Department of Computer Science, Technical University of Darmstadt

[http://www.ukp.tu-darmstadt.de/](http://www.ukp.tu-darmstadt.de/)

###### Abstract

We rely on arguments in our daily lives to deliver our opinions and base them on evidence, making them more convincing in turn. However, finding and formulating arguments can be challenging. In this work, we present the _Arg-CTRL_--a language model for argument generation that can be controlled to generate sentence-level arguments for a given topic, stance, and aspect. We define argument aspect detection as a necessary method to allow this fine-granular control and crowdsource a dataset with 5,032 arguments annotated with aspects. Our evaluation shows that the Arg-CTRL is able to generate high-quality, aspect-specific arguments, applicable to automatic counter-argument generation. We publish the model weights and all datasets and code to train the Arg-CTRL.1

Footnote 1: [https://github.com/UKPLab/controlled-argument-generation](https://github.com/UKPLab/controlled-argument-generation)

+
Footnote †: _Corresponding of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 380–396

June 6–11, 2021. ©2021 Association for Computational Linguistics

## 1 Introduction

Language models Bengio et al. (2003) allow to generate text through learned distributions of a language and have been applied to a variety of areas like machine translation Bahdanau et al. (2015), summarization Paulus et al. (2018), or dialogue systems Wen et al. (2017). A rather new field for these models is the task of producing text with argumentative content Wang and Ling (2016). We believe this technology can support humans in the challenging task of finding and formulating arguments. A politician might use this to prepare for a debate with a political opponent or for a press conference. It may be used to support students in writing argumentative essays or to enrich one-sided discussions with counter-arguments. In contrast to retrieval methods, generation allows to combine and stylistically adapt text (e.g. arguments) based on a given input (usually the beginning of a sentence). Current argument generation models, however, produce lengthy texts and allow the user little control over the aspect the argument should address Hua et al. (2019); Hua and Wang (2018). We show that argument generation can be enhanced by allowing for a fine-grained control and limiting the argument to a single but concise sentence.

Controllable language models like the _CTRL_Keskar et al. (2019) allow to condition the model at training time to certain control codes. At inference, these can be used to direct the model's output with regard to content or style. We build upon this architecture to control argument generation based solely on a given topic, stance, and argument aspect. For instance, to enforce focus on the aspect of _cancer_ for the topic of _nuclear energy_, we input a control code "_Nuclear Energy CON cancer_" that creates a contra argument discussing this aspect, for instance: "_Studies show that people living next to nuclear power plants have a higher risk of developing cancer_."

To obtain control codes from training data, we pre-define a set of topics to retrieve documents for and rely on an existing stance detection model to classify whether a sentence argues in favor (_pro_) or against (_con_) the given topic Stab et al. (2018). Regarding argument aspect detection, however, past work has two drawbacks: it either uses simple rule-based extraction of verb- and noun-phrases Fujii and Ishikawa (2006) or the definition of aspects is based on target-concepts located within the same sentence Gemechu and Reed (2019). Aspects as we require and define them are not bound to any part-of-speech tag and (1) hold the core reason upon which the conclusion/evidence is built and (2) encode the stance towards a general but not necessarily explicitly mentioned topic the argument discusses. For instance:

**Topic:**_Nuclear Energy_

**Argument:**_Running nuclear reactors is costly as it involves long-time disposal of radioactive waste._

The evidence of this argument is based upon the two underlined aspects. While these aspects encode

a negative stance towards the topic of "Nuclear Energy", the topic itself is not mentioned explicitly in the argument.

Our final controlled argument generation pipeline (see Figure 1) works as follows: (1) We gather several million documents for eight different topics from two large data sources. All sentences are classified into pro-, con-, and non-arguments. We detect aspects of all arguments with a model trained on a novel dataset and concatenate arguments with the same topic, stance, and aspect into training documents. (2) We use the collected classified data to condition the Arg-CTRL on the topics, stances, and aspects of all gathered arguments. (3) At inference, passing the control code _[Topic] [Stance] [Aspect]_ to the model will generate an argument that follows these commands.

Our evaluation shows that the Arg-CTRL is able to produce aspect-specific, high-quality arguments, applicable to automatic counter-argument generation. The contributions are as follows: (i) We adapt and fine-tune the CTRL for aspect-controlled neural argument generation. (ii) We show that detecting argument aspects and conditioning the generation model on them are necessary steps to control the model's training process and its perspective while generating. (iii) We propose several methods to analyze and evaluate the quality of (controllable) argument generation models. (iv) We develop a new scheme to annotate argument aspects and release a dataset with 5,032 samples.

## 2 Related Work

**Argument Aspect Detection** Early work by Fujii and Ishikawa (2006) focuses mainly on Japanese and restricts aspects to noun- and verb-phrases, extracted via hand-crafted rules. Boltuzic and Snajder (2017) extract noun-phrases and aggregate them into _concepts_ to analyze the microstructure of claims. Misra et al. (2015) introduce facets as low level issues, used to support or attack an argumentation. In that, facets are conceptually similar to aspects, but not explicitly phrased and instead seen as abstract concepts that define clusters of semantically similar text-spans of summaries. Bilu et al. (2019) define commonplace arguments that are valid in several situations for specified actions (e.g. "ban") and topics (e.g. "smoking"). These actions are similar to aspects, but limited in number and manually defined. Gemechu and Reed (2019) detect, amongst others, concepts and aspects in arguments with models trained on expert annotations. However, in their definition, aspects have to point to a target concept mentioned in the argument. In our definition, aspects refer to a general topic which is not necessarily part of the sentence and our annotation scheme is applicable by non-experts.

The concept of framing dimensions Boydstun et al. (2014) is close to argument aspects. In the field of argument mining, Ajjour et al. (2019) recently applied frames to label argument clusters. Yet, their method does not allow to detect frames. Other works present methods to automatically label sentences of news articles and online discussions with frames Hartmann et al. (2019); Naderi and Hirst (2017). These methods are, however, limited to a small set of predefined frames that represent high-level concepts. Contrarily, we operate on a fine-grained span-level to detect aspects that are explicitly mentioned in arguments.

Figure 1: (1) Gather data from large data sources. Extract sentences, classify arguments, and detect aspects. Arguments sharing a topic & stance & aspect (\(\widehat{=}\) control code) are concatenated into training documents. (2) The model is fine-tuned on each document with the control code prepended to each input sequence. (3) At inference, the model only needs a control code to generate an argument that follows the given topic & stance & aspect.

 

Argument GenerationEarly approaches rely on rules from argumentation theory and user preference models (Carenini and Moore, 2006; Zukerman et al., 1998). In a more recent work, Sato et al. (2015) construct rules to find arguments in a large data source, which are then filtered and ordered with a neural network based ranker. Baff et al. (2019) use a clustering and regression approach to assemble discourse units (major claims, pro and con statements) to argumentative texts. However, most of these approaches rely on hand-crafted features and do not generalize well. Moreover, they all require permanent access to large data sources and are not able to generate new arguments.

Recently, research on generating arguments with language models gained more attention. Hua and Wang (2019) use a sequence to sequence model (Sutskever et al., 2014) to generate argumentative text by attending to the input and keyphrases automatically extracted for the input from, for example, Wikipedia. Other work focuses on generating argumentative dialogue (Le et al., 2018) and counterarguments (Hidey and McKeown, 2019; Hua et al., 2019) based on a given input sentence, or on generating summaries from a set of arguments (Wang and Ling, 2016). Contrarily, we train a language model that does not require a sentence-level input for generation and allows for direct control over the topic, stance, and aspect of the produced argument.

Xing et al. (2017) design a language model that attends to topic information to generate responses for chatbots. Dathathri et al. (2019) train two models that control the sentiment and topic of the output of pre-trained language models at inference. Gretz et al. (2020a) fine-tune GPT-2 on existing, labeled datasets to generate claims for given topics. However, the latter works do not explore generation for such a fine-grained and explicit control as proposed in this work. We show that argument generation requires the concept of argument aspects to shape the produced argument's perspective and to allow for diverse arguments for a topic of interest.

## 3 Argument Aspect Detection

Argument aspect detection is necessary for our argument generation pipeline, as it allows for a fine-grained control over the generation process. We create a new dataset, as existing approaches either rely on coarse-grained frames or cannot be applied by non-expert annotators in a scalable manner.

### Dataset Creation

We base our new aspect detection dataset on the UKP Sentential Argument Mining Corpus (UKPCorpus) by Stab et al. (2018), as it already contains sentence-level arguments and two of the control codes we aim to use: topics and stance labels. More precisely, it contains 25,474 manually labelled sentences for eight controversial topics in English. Each sample consists of a topic and a sentence, labelled as either being supporting, attacking, or no argument towards the given topic. As we are only interested in arguments, we do not consider the non-argumentative sentences.

**Step 1: Preliminary annotations** To ensure the feasibility of creating a dataset for this task, two experts (a post-doctoral researcher and an undergraduate student with NLP background) independently annotate 800 random samples (from four topics, 200 per topic) taken from the UKP-Corpus. The annotations are binary and on token-level, where multiple spans of tokens could be selected as aspects. The resulting inter-annotator agreement of this study is Krippendorff's \(\alpha_{u}=.38\). While this shows that the task is generally feasible, the agreement on exact token spans is rather low. Hence, in the following steps, we reduce the complexity of the annotation task.

**Step 2: Annotation scheme** Instead of free span-level annotations, we present annotators with a ranked list of aspect recommendations. To generate meaningful recommendations, we train a ranking model using the preliminary annotations (Step 1).

**Step 2a: Data preparation for ranking** To create training data for the ranker, we use a simple heuristic to calculate scores between 0 and 1 for all N-grams of a sentence by dividing the number of aspect tokens within an N-gram by its length \(N\): \(\frac{\#~{}aspect~{}tokens}{N}\in[0,1]\). Our analysis reveals that 96% (783 of 814) of all aspects in the preliminary annotation dataset only contain one to four tokens. We thus decide to ignore all candidates with more than four tokens. No other limitations or filtering mechanisms are applied.

**Step 2b: Training the ranker** We use BERT (Devlin et al., 2019) and MT-DNN2(Liu et al., 2019) (base and large) to train a ranker. For training, we create five splits: (1) one in-topic split using a random subset from all four topics and (2) four 

cross-topic splits using a leave-one-topic-out strategy. The cross-topic setup allows us to estimate the ranker's performance on unseen topics of the UKP-Corpus.

A single data sample is represented by an argument and an 1- to 4-gram of this argument, separated by the BERT architecture's [SEP] token. This technique expands the 800 original samples of the dataset to around 80,336. The model is trained for 5 epochs, with a learning rate of \(5\times 10^{-5}\), and a batch size of 8. We use the mean squared error as loss and take the recall@k to compare the models. The in- and cross-topic results of the best-performing model (MT-DNN\({}_{BASE}\)) are reported in Table 2. All results are the average over runs with five different seeds (and over all four splits for the cross-topic experiments).

**Step 2c: Creating the annotation data** For each of the four topics that are part of the preliminary annotation dataset, we use the in-topic model to predict aspects of 629 randomly chosen, unseen arguments from the UKP-Corpus. For the other four topics of the UKP-Corpus, we choose the best cross-topic model to predict aspects for the same amount of samples. To keep a recall of at least 80%, we choose the ten and fifteen highest-ranked aspect candidates for samples as predicted by the in-topic and cross-topic model, respectively. We remove aspect candidates that include punctuation, begin or end with stopwords, or contain digits.

**Step 3: Annotation study** We use Amazon Mechanical Turk to annotate each sample by eight different workers located in the US, paying SS7.6 per hour (minimum wage is SS7.25 per hour). Based on a subset of 232 samples, we compute an \(\alpha_{u}\) of .67 between crowdworkers and experts (three doctoral researchers). Compared to the initial study, the new approach increases the inter-annotator agreement between experts by approx. 11 points (see App. A for further details on the annotation study). Based on this promising result, we create a dataset of 5,032 high-quality samples that are labelled with aspects, as well as with their original stance labels from the UKP-Corpus. We show the most frequent (lemmatized) aspects that appear in some topics in Table 1.

### Evaluation

We create a cross-topic split with the data of two topics as test set (_gun control_, _school uniforms_), one topic as dev set (_death penalty_), and the remaining topics as train set and evaluate two models with it. First, we use the ranking approach described in Step 2a-2b to fine-tune MT-DNN\({}_{BASE}\) on the newly generated data ("Ranker"). At inference, we choose the top \(T\) aspects for each argument as candidates. We tune \(T\) on the dev set and find \(T=2\) to be the best choice. Second, we use BERT for sequence tagging Wolf et al. (2020) and label all tokens of the samples with BIO tags. As previously done with the ranker, we experiment with BERT and MT-DNN weights and find BERL\({}_{ARGE}\) to be the best choice (trained for 5 epochs, with a learning rate of \(1\times 10^{-5}\) and a batch size of 32). We flatten the predictions for all test samples and calculate the F\({}_{1}\), Precision, and Recall macro scores. All models are trained over five seeds and the averaged results are reported in Table 3.

BERT\({}_{LARGE}\) predicts classes B and I with an F\({}_{1}\) of .65 and .53, hence aspects with more than one token are less well identified. A difference is to be expected, as the class balance of B's to I's is 2,768 to 2,103. While the ranker performs worse based on the shown metrics, it has a slightly higher recall for class I. We assume this is due to the fact that it generally ranks aspects with more than one token on top, i.e. there will often be at least one or more I's in the prediction. In contrast to that, BERT\({}_{LARGE}\) focuses more on shorter aspects, which is also in accordance with the average aspect length of 1.8 tokens per aspect in the dataset.

\begin{table}
\begin{tabular}{l l} \hline \hline
**Topic** & **Five most frequent aspects (frequency)** \\ \hline Gun control & right (30), protect (18), background checks (17), gun violence (14), criminal (13) \\ Death penalty & cost (16), innocent (12), retribution (10), murder rate (9), deterrent (8) \\ Abortion & right (21), pain (10), choice (10), right to life (9), risk (9) \\ Marijuana legalization & dangerous (16), cost (13), risk (12), harm (10), black market (9) \\ \hline General aspects & dangerous (in 8 of 8 topics), cost / life / risk / safety (in 7 of 8 topics) \\ \hline \hline \end{tabular}
\end{table}
Table 1: The five most frequent aspects for four exemplary topics and overall.

\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Setting** & **Rec@5** & **Rec@10** & **Rec@15** & **Rec@20** \\ \hline
**In-topic** & 0.7701 & 0.8468 & 0.8661 & 0.8925 \\
**Cross-topic** & 0.5951 & 0.7415 & 0.8164 & 0.8630 \\ \hline \hline \end{tabular}
\end{table}
Table 2: In- and cross-topic Recall@k of the ranker used for aspect recommendations.

 

In total, BERT\({}_{LARGE}\) outperforms the ranker by almost 6 percentage points in F\({}_{1}\) macro.

## 4 Data Collection Pipeline

This section describes the data collection and pre-processing for the argument generation pipeline. We aim to train a model that is able to transfer argumentative information concisely within a single sentence. We define such an argument as the combination of a topic and a sentence holding evidence with a specific stance towards this topic Stab et al. (2018). Consequently, the following preprocessing steps ultimately target retrieval and classification of sentences. To evaluate different data sources, we use a dump from Common-Crawl3 (cc) and Reddit comments4 (reddit) to fine-tune two separate generation models. The cc dump is from July 2016 and contains 331M documents (3.6TB) after deduplication. The reddit dump contains 2.5B documents (1.6TB) from December 2012 to May 2019. We choose to compare these two sources, as reddit is focused around user discussions and cc contains mixed sources with potentially higher quality.

Footnote 3: [https://commoncrawl.org](https://commoncrawl.org)

Footnote 4: [https://files.pushshift.io/reddit/comments/](https://files.pushshift.io/reddit/comments/)

**Document Retrieval** We index reddit and cc with ElasticSearch5 and, for both, gather up to 1.5M documents for each of the eight topics of the UKP-Corpus. To increase the search results, we add synonyms (see App. B) for most topics.

Footnote 5: [https://www.elastic.co](https://www.elastic.co)

**Argument and Stance Classification** We split the sentences of all documents and remove duplicates. We notice that many sentences are not relevant with regard to the document's topic. To enforce topic-relevance, we decide to filter out all sentences that do not contain at least one token of the respective topic or its defined synonyms (see App. B). We use the ArgumentText API's6 argument and stance classification models Stab et al. (2018) to classify all sentences into _argument_ or _non-argument_ (F\({}_{1}\) macro = \(.7384\)), and remaining arguments into _pro_ or _con_ with regard to the topic (F\({}_{1}\) macro = \(.7661\)).

Footnote 6: [https://api.argumentsearch.com](https://api.argumentsearch.com)

**Aspect Detection** We detect aspects on all remaining arguments. To speed up the detection on millions of sentences, we use BERT\({}_{BASE}\) instead of BERT\({}_{LARGE}\) (see Table 3).

**Training Document Generation** We create the final training documents for the argument generation model by concatenating all arguments that have the same topic, stance, and aspect (i.e. the same control code). Further, we aggregate all arguments that include an aspect with the same stem into the same document (e.g. arguments with _cost_ and _costs_ as aspect). To cope with limited hardware resources, we restrict the total number of arguments for each topic and stance to 100,000 (i.e. 1.6M over all eight topics). Also, as some aspects dominate by means of quantity of related arguments and others appear only rarely, we empirically determine an upper and lower bound of 1,500 and 15 arguments for each document, which still allows us to retrieve the above defined amount of training arguments.

## 5 Model Training and Analysis

In the following, we describe the architecture and the training process of the Arg-CTRL and analyze its performance.

### Model and Training

**Model** The goal of a statistical language model is to learn the conditional probability of the next word given all (or a subset of) the previous ones Bengio et al. (2003). That is, for a sequence of tokens \(x=(x_{1},...,x_{n})\), the model learns \(p(x_{i}|x_{<i})\) where \(x_{i}\) is the \(i\)-th word of sequence \(x\). For this work, we use the 1.63 billion-parameter _Conditional Transformer Language Model_ (CTRL) by Keskar et al. (2019), which is built on a transformer-based sequence to sequence architecture Vaswani et al. (2017). The CTRL has shown to produce high quality text, is general enough to be adapted for conditioning on the control codes we aim to use, and we do not need to pre-train the weights from scratch. Formally, the CTRL adds an extra condition to each sequence by prepending a control code \(c\), hence learning \(p(x_{i}|x_{<i},c)\). The control code is represented by a single token and can then be used to direct the model output at inference. We extend the model from its previous limit of a single-token control code to accept multiple tokens. For

\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Model** & **F\({}_{1}\) macro** & **Precision** & **Recall** \\ \hline
**Majority (baseline)** & .3085 & .2871 & .3333 \\
**Ranker** & .6522 & .6685 & .6474 \\
**BERT\({}_{BASE}\)** & .6980 & .6927 & **.7040** \\
**BER\({}_{LARGE}\)** & **.7100** & **.7240** & .6993 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Test set results of the models for aspect detection. Majority only predicts class O.

 

decoding at inference, we use _penalized sampling_ as proposed by Keskar et al. (2019). It defines a near-greedy sampling strategy that uses a penalty constant, effectively lowering the probability of previously generated tokens to prevent repetitions.

**Training** The CTRL was trained on 140GB of data from several large resources like Wikipedia, subreddits, and news data. We base our experiments on the pre-trained weights for a sequence length of 256 and fine-tune (see App. C for technical details) two models: Arg-CTRL\({}_{CC}\) (on the cc data) and Arg-CTRL\({}_{REDOTI}\) (on the reddit data). All training documents are sampled randomly for training. The respective control code is prepended to each sequence of 256 subwords of a document.

### Analysis

**Generation** At inference, we gather multiple generated arguments from a control code input by splitting the generated output text into sentences with NLTK Bird et al. (2009). We observe that for the first generated argument, the Arg-CTRL mostly outputs very short phrases, as it tries to incorporate the control code into a meaningful start of an argument. We prevent this by adding punctuation marks after each control code (e.g. a period or colon), signaling the model to start a new sentence. In this fashion, we generate _pro-_ and _con-_arguments up to the pre-defined training split size7 for each topic of the UKP-Corpus, resulting in 7,991 newly generated arguments. We do this with both models and use the generated arguments as a basis for the following analysis and evaluation methods. Examples of generated arguments can be found in Tables 4, 6, and 7 (as part of the evaluation, see Section 7).

Footnote 7: Not counting non-arguments from the splits.

**Results** With no other previous work on explicit control of argument generation (to the best of our knowledge), we decide to proof our concept of aspect-controlled neural argument generation by comparing both generation models to a retrieval approach as a strong upper bound. The retrieval approach returns all arguments from the classified training data (see Section 4) that match a given topic, stance, and aspect. Both the retrieval and generation approaches are evaluated against reference data from debate portals and compared via METEOR Lavie and Agarwal (2007) and ROUGE-L Lin (2004) metrics. The retrieval approach has an advantage in this setup, as the arguments are also of human origin and aspects are always explicitly stated within a belonging argument.

The reference data was crawled from two debate portals8 and consists of pro- and con-paragraphs discussing the eight topics of the UKP-Corpus. As the paragraphs may include non-arguments, we filter these out by classifying all sentences with the ArgumentText API into arguments and non-arguments. This leaves us with 349 pro- and 355 con-arguments over all topics (see App. D for the topic-wise distribution). Next, we detect all aspects in these arguments. Arguments with the same topic, stance, and aspect are then grouped and used as reference for arguments from the (a) generated arguments and (b) retrieval approach arguments if these hold the same topic, stance, and aspect. The results reveal that both the average METEOR and ROUGE-L scores are only marginally lower than the retrieval scores (METEOR is 0.5/1.1 points lower for the Arg-CTRL\({}_{REDDITI}\)/Arg-CTRL\({}_{CC}\), see Table 5). It not only shows the strength of the architecture, but also the success in generating sound aspect-specific arguments with our approach.

Footnote 8: procnon.org and idebeate.org

**Overlap with Training Data** We find arguments generated by the models to be genuine, i.e. demonstrating substantial differences to the training data. For each of the 7,991 generated arguments, we find the most similar argument in the training data based on the cosine similarity of their BERT embeddings

\begin{table}
\begin{tabular}{l|c} \hline
* is downright criminal . (0.97) \\
**dening CON disfavored** & . cliques ) to them . (0.36) \\ \hline
**nuclear energy PRO safe** & In addition , we must continue developing safer technologies like small modular reactors which will help us meet our nation ’s need for reliable , emission-free sources of low-emission energy [...] . (0.96) \\
**nuclear energy CON leak** & ” We are concerned about the possibility of further releases of radioactivity due to possible melting or cracking of fuel rods at the No . (0.47) \\ \hline
**marjuana legalization PRO safer** & Legalizing cannabis will help reduce crime rates ( especially violent crimes ) and make society safer overall . (0.96) \\
**marijuana legalization PRO benefits** & Decrease amount of police officers needed 6 . (0.37) \\ \hline \end{tabular}
\end{table}
Table 4: Generated arguments of highest/lowest quality with Arg-CTRL\({}_{CC}\). Bold text shows the used control code. Quality score in brackets as predicted by the argument quality model. “[...]” signals shortened text.

 

(CLS token). The average cosine similarity of the most similar pairs for both the Arg-CTRL\({}_{\mathrm{CC}}\) and Arg-CTRL\({}_{\mathrm{REDDTI}}\) is 92. However, this value is misleading, as even highly similar samples still show clear differences. This is also evident when looking at the average edit distances of 343 (Arg-CTRL\({}_{\mathrm{CC}}\)) and 163 (Arg-CTRL\({}_{\mathrm{REDDIT}}\)) for the pairs with highest similarity. Further comparison of these pairs for their longest common (string) overlap reveals only 9% (Arg-CTRL\({}_{\mathrm{CC}}\)) and 11% (Arg-CTRL\({}_{\mathrm{REDDIT}}\)) overlap on average, mostly consisting of stopwords. For illustration, we show two examples of highly similar pairs in Table 6.

## 6 Generation in Absence of Aspects

To show the necessity of having prior knowledge of aspects for our controlled argument generation approach, we create training data _without_ prior knowledge of aspects, train a new generation model on it, and compare it to our previous models _with_ prior knowledge of aspects. Equally to the original Arg-CTRL\({}_{\mathrm{CC}}\)'s procedure, we gather 100,000 sentences for each stance of a topic from the cc data. As we assume to have no knowledge about the aspects of the arguments, we randomly sample arguments from the cc source documents. We create training documents with numbers of arguments varying between 15 and 1,500 to mimic the data generation process of the original models and fine-tune a new generation model on them. After training, we generate the same number of arguments as for the other two models by using our default control code of _[Topic]__[Stance]__[Aspect]_. While the new model was only conditioned on topics and stances at training time, we make sure that all aspects used for generation appear in at least one argument of the model's training data.

We compare all models by verifying whether or not the aspect used for generation (including synonyms and their stems and lemmas) can be found in the generated arguments. For the original models conditioned on aspects, this is true in 79% of the cases for Arg-CTRL\({}_{\mathrm{REDDIT}}\) and in 74% of the cases for Arg-CTRL\({}_{\mathrm{CC}}\). For the model that was not conditioned on aspects, however, it is only true in 8% of the cases. It clearly shows the necessity to condition the model on aspects explicitly, implying the need for argument aspect detection, as the model is unable to learn generating aspect-related arguments otherwise. Moreover, without prior detection of aspects, we have no means for proper aggregation over aspects. We notice that for the model without prior knowledge of aspects, 79% of all aspects in the training data appear in only one argument. For these aspects, the model will likely not pick up a strong enough signal to learn them.

## 7 Evaluation

We evaluate the quality (intrinsic evaluation) of the Arg-CTRL and its performance on an exemplary task (extrinsic evaluation). As a basis, we use the 7,991 arguments generated in Section 5.

### Intrinsic Evaluation

**Human Evaluation** We conduct an expert evaluation on a subset of generated arguments with two researchers (field of expertise is natural language processing) not involved in this paper. Two aspects are evaluated: _fluency_ and _persuasiveness_. We consider a sentence as fluent if it is grammatically correct Hua et al. (2019), i.e. contains neither semantic nor syntactic errors, and arrange this as a binary task. To reduce subjectivity for the persuasiveness evaluation, the experts do not annotate single arguments but instead compare pairs Habernal and Gurevych (2016) of generated and refer

\begin{table}
\begin{tabular}{|p{142.3pt} p{142.3pt}|} \hline
**Generated sentence**: We do n’t need more gun control laws when we already have enough restrictions on who can buy guns in this country . \\
**Training sentence**: We have some of the strongest gun laws in the country , but guns do n’t respect boundaries any more than criminals do . \\
**Cosine similarity / edit distance / rel. overlap**: 95.59 / 88 / 8\% & \\ \hline
**Generated sentence**: The radioactivity of the spent fuel is a concern , as it can be used to make weapons and has been linked to cancer in humans. \\
**Training sentence**: However, it does produce radioactive waste , which must be disposed of carefully as it can cause health problems and can be used to make nuclear weapons **Cosine similarity / edit distance / rel. overlap**: 92.40 / 99 / 17\% & \\ \end{tabular}
\end{table}
Table 6: Training data vs. generated arguments: examples of most similar arguments. Underlines mark the longest common overlap between generated and training sentences.

\begin{table}
\begin{tabular}{l c c} \hline
**Model** & **METEOR** & **ROUGE-L** \\ \hline Retrieval (cc) & **17.85** & **14.72** \\ Arg-CTRL\({}_{\mathrm{CC}}\) & 16.80 & 11.95 \\ \hline Retrieval (reddit) & **17.29** & **15.26** \\ Arg-CTRL\({}_{\mathrm{REDDIT}}\) & 16.82 & 12.34 \\ \hline \end{tabular}
\end{table}
Table 5: Comparison of retrieval and generation approach with reference data from debate portals.

 

ence data arguments (see Section 5.2). The experts could either choose one argument as being more persuasive or both as being equally persuasive. In total, the experts compared 100 (randomly sorted and ordered) argument pairs for persuasiveness and fluency (50 from both the Arg-CTRL\({}_{\text{REDDIT}}\) and the Arg-CTRL\({}_{\text{CC}}\)). A pair of arguments always had the same topic and stance. For fluency, only the annotations made for generated arguments were extracted and taken into account. Averaged results of both experts show that in 33% of the cases, the generated argument is either more convincing (29%) or as convincing (4%) as the reference argument. Moreover, 83% of generated arguments are fluent. The inter-annotator agreement Cohen (1960) between the two experts is Cohen's \(\kappa=.30\) (percentage agreement: .62) for persuasiveness and \(\kappa=.43\) (percentage agreement: .72) for fluency, which can be interpreted as "fair" and "moderate" agreement, respectively Landis and Koch (1977). As we compare to high-quality, curated data, the perceived persuasiveness of the generated arguments shows the potential of the work--further strengthened in the remainder of this section.

**Argument Quality** We introduce a novel method to evaluate generated arguments based on the argument quality detection approach proposed by Gretz et al. (2020b). They create an argument quality dataset that contains around 30,000 arguments over 71 topics. For each argument, annotators were asked whether or not they would recommend a friend to use the displayed argument in a speech. The quality scores for each argument result from a weighted average (WA) or MACE Probability function of all annotations and range between 0 (lowest quality) and 1.0 (highest quality). We use the _WA_-score as label, the same model (BERT\({}_{BASE}\)) and hyperparameters as given in the original paper, and reproduce the reported correlations of .52 (Pearson) and .48 (Spearman) on the test dataset (averaged over five different seeds). The model predicts an average argument quality of .71 for the Arg-CTRL\({}_{\text{REDDIT}}\) arguments, .75 for the training arguments of the UKP-Corpus, and even .76 for the Arg-CTRL\({}_{\text{CC}}\) arguments. It shows that our model is able to produce arguments that are generally on a similar quality level as arguments made by humans. The difference of 5 points between the Arg-CTRL\({}_{\text{CC}}\) and the Arg-CTRL\({}_{\text{REDDIT}}\), however, shows that training on reddit data leads to a lower quality of generated arguments. Exemplary for three topics, we show the generated arguments with the highest and lowest argument quality in Table 4 (see App. E for the full table).

### Extrinsic Evaluation: Counter-Arguments

Drafting counter-arguments is an important skill for debating, to provide constructive feedback, and to foster critical thinking. We lean onto the work of Wachsmuth et al. (2018) who describe a counterpart as discussing the same aspect as an initial argument, but with a switched stance. Hence, given our defined control codes, our model is especially fit for counter-argument generation. Unlike current models for this task, we do not require a specific dataset with argument and counter-argument pairs Hidey and McKeown (2019); Hua et al. (2019). Also, in contrast to the model by Hua and Wang (2019) that implicitly integrates input-related "Keyphrases" into the process of counter-argument generation, our model is able to concentrate on every aspect of the input explicitly and with a separate argument, allowing for more transparency and interpretability over the process of counter-argument generation. We exemplary show how the combination of aspect detection and controlled argument generation can be successfully leveraged to tackle this task. For that, we manually

\begin{table}
\begin{tabular}{|p{284.5pt}|} \hline
**Human:** Nuclear energy produces waste that stays radioactive for thousands of years and pollutes the environment. \\
**Arg-CTRLCc:** (nuclear energy PROW waste .) The only thing that ’s going to be left over is the spent fuel rods , which are not very radioactive and can be reprocessed [...] **Arg-CTRLcc:** (nuclear energy PRO radioactive .) The radioactivity of the waste is not a problem , because it will decay away in about 300 years or so [...] **Arg-CTLRcc:** (nuclear energy PRO environment .) The fact that the radioactive waste is not only stored in a safe manner , but also protected from any potential threat to human health or the environment for thousands of years [...] **Human:** School uniforms are expensive and affect the pupil’s individuality. \\
**Arg-CFRLGCc:** (school uniforms PRO expensive .) The cost of the uniform is not that much more than a regular shirt and shorts [...] **Arg-CTRLcc:** (school uniforms PRO individuality .) The uniform is a symbol of unity and identity , which helps to foster an environment where students can feel comfortable about expressing their own individual style without being judged [...] \\ \end{tabular}
\end{table}
Table 7: Generated counter-arguments with the Arg-CTRL\({}_{\text{CC}}\). Aspects in the initial argument are underlined and used for the counter-argument generation. Control code in brackets and “[...]” signals shortened text.

 

compose initial arguments for the topics _nuclear energy_ and _school uniforms_. Then, we automatically detect their aspects and generate a counter-argument for each aspect by passing the topic, opposite stance of the original argument, and one of the aspects into the Arg-CTRL\({}_{\textsc{CC}}\). For both topics, the Arg-CTRL\({}_{\textsc{CC}}\) produces meaningful counter-arguments based on the detected aspects (see Table 7).

## 8 Conclusion

We apply the concept of controlled neural text generation to the domain of argument generation. Our Arg-CTRL is conditioned on topics, stances, and aspects and can reliably create arguments using these control codes. We show that arguments generated with our approach are genuine and of high argumentative and grammatical quality in general. Moreover, we show that our approach can be used to generate counter-arguments in a transparent and interpretable way. We fine-tune the Arg-CTRL on two different data sources and find that using mixed data from Common-Crawl results in a higher quality of generated arguments than using user discussions from Reddit-Comments. Further, we define argument aspect detection for controlled argument generation and introduce a novel annotation scheme to crowdsource argument aspect annotations, resulting in a high-quality dataset. We publish the model weights, data, and all code necessary to train the Arg-CTRL.

### Ethics Statement

Models for argument and claim generation have been discussed in our related work and are widely available. Gretz et al. (2020a) suggest that, in order to allow for a fine-grained control over claim/argument generation, aspect selection needs to be handled carefully, which is what we have focused on in this work. The dangers of misuse of language models like the CTRL have been extensively discussed by its authors Keskar et al. (2019). The ethical impact of these works has been weighed and deemed justifiable.

Argument generation--and natural language generation as a whole--is subject to dual use. The technology can be used to create arguments that cannot be distinguished from human-made arguments. While our intentions are to support society, to foster diversity in debates, and to encourage research on this important topic, we are aware of the possibility of harmful applications this model can be used for. For instance, the model could be used to generate only opposing (or supporting) arguments on one of the pretrained topics and aspects and, as such, bias a debate into a certain direction. Also, bots could use the generated arguments to spread them via social media. The same is true, however, for argument search engines, which can be used by malicious parties to retrieve (and then spread) potentially harmful information.

However, controllable argument generation can also be used to support finding and formulating (counter-)arguments for debates, for writing essays, to enrich one-sided discussions, and thus, to make discourse more diverse overall. For instance, anticipating opposing arguments is crucial for critical thinking, which is the foundation for any democratic society. The skill is extensively taught in school and university education. However, _confirmation bias_ (or _myside bias_) Stanovich et al. (2013), i.e. the tendency to ignore opposing arguments, is an ever-present issue. Technologies like ours could be used to mitigate this issue by, for instance, automatically providing topic- and aspect-specific counter-arguments for all arguments of a given text (this has been shown for single arguments in Section 7.2). We believe that working on and providing access to such models is of major importance and, overall, a benefit to society.

Open-sourcing such language models also encourages the work on counter-measures to detect malicious use: While many works have been published on the topic of automatic fake news detection in texts Kaliyar et al. (2020); Reis et al. (2019); Hanselowski et al. (2018); Perez-Rosas et al. (2018), the recent emergence of large-scale language models has also encouraged research to focus on detecting the creator of these texts Varshney et al. (2020); Zellers et al. (2019). The former approaches are aimed at detecting fake news in general, i.e. independent of who (or what) composed a text, whereas the latter approaches are designed to recognize if a text was written by a human or generated by a language model. We encourage the work on both types of methods. Ideally, social networks and news platforms would indicate if a statement was automatically generated in addition to its factual correctness.

Further, we point out some limitations of the Arg-CTRL that mitigate the risks discussed before. One of these limitations is that it cannot be used 

to generate arguments for unseen topics, which makes a widespread application (e.g. to produce fake news) rather unlikely (using an unseen topic as control code results in nonsensical repetitions of the input). The analysis in Section 6 of the paper shows that the model fails to produce aspect-specific sentences in 92% of the cases if it was not explicitly conditioned on them at training time. Even in case of success, the aspect has to exist in the training data. Also, the model is trained with balanced classes, i.e. both supporting and opposing arguments for each topic are seen with equal frequency to prevent possible bias into one or the other direction.

To further restrict malicious use, we release the training data for the Arg-CTRLs with an additional clause that forbids use for any other than research purposes. Also, all the training datasets for the Arg-CTRLs will be accessible only via access control (e-mail, name, and purpose of use). Lastly, this work has been reviewed by the ethics committee of the Technical University of Darmstadt that issued a positive vote.

## Acknowledgements

We thank Tilman Beck and Nandan Thakur for their support in the human evaluation (Section 7.1). This work has been supported by the German Research Foundation within the project "Open Argument Mining" (GU 798/25-1), associated with the Priority Program "Robust Argumentation Machines (RATIO)" (SPP-1999).

## References

* Ajjour et al. (2019) Yamen Ajjour, Milad Alshomary, Henning Wachsmuth, and Benno Stein. 2019. Modeling frames in argumentation. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 2922-2932, Hong Kong, China. Association for Computational Linguistics.
* Baff et al. (2019) Roxanne El Baff, Henning Wachsmuth, Khalid Alkatib, Manfred Stede, and Benno Stein. 2019. Computational Argumentation Synthesis as a Language Modeling Task. In _12th International Natural Language Generation Conference (INLG 2019)_. ACL.
* Conference Track Proceedings_, pages 1-15.
* Bengio et al. (2003) Yoshua Bengio, Rejean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. _Journal of machine learning research_, 3(Feb):1137-1155.
* Bilu et al. (2019) Yonatan Bilu, Ariel Gera, Daniel Hershcovich, Benjamin Sznajder, Dan Lahav, Guy Moshkowich, Anael Malet, Assaf Gavron, and Noam Slonim. 2019. Argument invention from first principles. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 1013-1026, Florence, Italy. Association for Computational Linguistics.
* Bird et al. (2009) Steven Bird, Ewan Klein, and Edward Lopez. 2009. _Natural Language Processing with Python_, 1st edition. O'Reilly Media, Inc.
* Boltuzic and Snajder (2017) Filip Boltuzic and Jan Snajder. 2017. Toward stance classification based on claim microstructures. In _Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis_, pages 74-80, Copenhagen, Denmark. Association for Computational Linguistics.
* Boydstum et al. (2014) Amber E. Boydstum, Dallas Card, Justin Gross, Paul Resnick, and Noah A. Smith. 2014. Tracking the Development of Media Frames within and across Policy Issues. Carnegie Mellon University.
* Carenini and Moore (2006) Giuseppe Carenini and Johanna D Moore. 2006. Generating and evaluating evaluative arguments. _Artificial Intelligence_, 170(11):925-952.
* Cohen (1960) Jacob Cohen. 1960. A coefficient of agreement for nominal scales. _Educational and psychological measurement_, 20(1):37-46.
* Dathathri et al. (2019) Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019. Plug and Play Language Models: A Simple Approach to Controlled Text Generation. _arXiv, abs/1912.02164_.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
* Fujii and Ishikawa (2006) Atsushi Fujii and Tetsuya Ishikawa. 2006. A system for summarizing and visualizing arguments in subjective documents: Toward supporting decision making. In _Proceedings of the Workshop on Sentiment and Subjectivity in Text_, SST '06, pages 15-22, Stroudsburg, PA, USA. Association for Computational Linguistics 

* Gemechu and Reed (2019) Debela Gemechu and Chris Reed. 2019. Decompositional argument mining: A general purpose approach for argument graph construction. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 516-526, Florence, Italy. Association for Computational Linguistics.
* a study of gpt-2 based claim generation. _arXiv, abs/2010.06185_.
* Gretz et al. (2020b) Shai Gretz, Roni Friedman, Edo Cohen-Karlik, Assaf Toledo, Dan Lahav, Ranit Aharonov, and Noam Slonim. 2020b. A large-scale dataset for argument quality ranking: Construction and analysis. _Proceedings of the AAAI Conference on Artificial Intelligence_, 34(05):7805-7813.
* Habernal and Gurevych (2016) Ivan Habernal and Iryna Gurevych. 2016. Which argument is more convincing? analyzing and predicting convincingness of web arguments using bidirectional LSTM. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1589-1599, Berlin, Germany. Association for Computational Linguistics.
* Hanselowski et al. (2018) Andreas Hanselowski, Avinesh PVS, Benjamin Schiller, Felix Caspelherr, Debanjan Chaudhuri, Christian M. Meyer, and Iryna Gurevych. 2018. A retrospective analysis of the fake news challenge stance-detection task. In _Proceedings of the 27th International Conference on Computational Linguistics_, pages 1859-1874, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
* Hartmann et al. (2019) Mareike Hartmann, Tallulah Jansen, Isabelle Augenstein, and Anders Sogaard. 2019. Issue framing in online discussion fora. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 1401-1407, Minneapolis, Minnesota. Association for Computational Linguistics.
* Hidey and McKeown (2019) Christopher Hidey and Kathy McKeown. 2019. Fixed that for you: Generating contrastive claims with semantic edits. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 1756-1767, Minneapolis, Minnesota. Association for Computational Linguistics.
* Hua et al. (2019) Xinyu Hua, Zhe Hu, and Lu Wang. 2019. Argument generation with retrieval, planning, and realization. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 2661-2672, Florence, Italy. Association for Computational Linguistics.
* Hua and Wang (2018) Xinyu Hua and Lu Wang. 2018. Neural argument generation augmented with externally retrieved evidence. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 219-230, Melbourne, Australia. Association for Computational Linguistics.
* Hua and Wang (2019) Xinyu Hua and Lu Wang. 2019. Sentence-level content planning and style specification for neural text generation. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 591-602, Hong Kong, China. Association for Computational Linguistics.
* a deep convolutional neural network for fake news detection. _Cogn. Syst. Res._, 61(C):32-44.
* Keskar et al. (2019) Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. _arXiv, abs/1909.05858_.
* Landis and Koch (1977) J. Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. _Biometrics_, 33(1):159-174.
* Lavie and Agarwal (2007) Alon Lavie and Abhaya Agarwal. 2007. Meteor: An automatic metric for mt evaluation with high levels of correlation with human judgments. In _Proceedings of the Second Workshop on Statistical Machine Translation_, StatMT '07, page 228-231, USA. Association for Computational Linguistics.
* Le et al. (2018) Dieu Thu Le, Cam-Tu Nguyen, and Kim Anh Nguyen. 2018. Dave the debater: a retrieval-based and generative argumentative dialogue agent. In _Proceedings of the 5th Workshop on Argument Mining_, pages 121-130, Brussels, Belgium. Association for Computational Linguistics.
* Lin (2004) Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In _Text Summarization Branches Out_, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
* Liu et al. (2019) Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. 2019. Multi-task deep neural networks for natural language understanding. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 4487-4496, Florence, Italy. Association for Computational Linguistics.
* Misra et al. (2015) Amita Misra, Pranav Anand, Jean E. Fox Tree, and Marilyn Walker. 2015. Using summarization to discover argument facets in online idealogical dialog. In _Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 430-440, Denver, Colorado. Association for Computational Linguistics.
* Naderi and first (2017) Nona Naderi and Graeme First. 2017. Classifying frames at the sentence level in news articles. In _Proceedings of the International Conference Recent 

Advances in Natural Language Processing, RANLP 2017_, pages 536-542, Varna, Bulgaria. INCOMA Ltd.
* May 3, 2018, Conference Track Proceedings_. OpenReview.net.
* Perez-Rosas et al. (2018) Veronica Perez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. 2018. Automatic detection of fake news. In _Proceedings of the 27th International Conference on Computational Linguistics_, pages 3391-3401, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
* Reis et al. (2019) J. C. S. Reis, A. Correia, F. Murai, A. Veloso, and F. Benevenuto. 2019. Supervised learning for fake news detection. _IEEE Intelligent Systems_, 34(2):76-81.
* Sato et al. (2015) Misa Sato, Kohsuke Yanai, Toshinori Miyoshi, Toshihiko Yanase, Makoto Iwayama, Qinghua Sun, and Yoshiki Niwa. 2015. End-to-end argument generation system in debating. In _Proceedings of ACL-IJCNLP 2015 System Demonstrations_, pages 109-114, Beijing, China. Association for Computational Linguistics and The Asian Federation of Natural Language Processing.
* Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_.
* Stab et al. (2018a) Christian Stab, Johannes Daxenberger, Chris Stahlbutt, Tristan Miller, Benjamin Schiller, Christopher Tauchmann, Steffen Eger, and Iryna Gurevych. 2018a. ArgumentText: Searching for arguments in heterogeneous sources. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations_, pages 21-25, New Orleans, Louisiana. Association for Computational Linguistics.
* November 4, 2018_, pages 3664-3674. Association for Computational Linguistics.
* Stanovich et al. (2013) Keith E. Stanovich, Richard F. West, and Maggie E. Toplak. 2013. Myside bias, rational thinking, and intelligence. _Current Directions in Psychological Science_, 22(4):259-264.
* Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In _Advances in Neural Information Processing Systems_, volume 27. Curran Associates, Inc.
* Varshney et al. (2020) L. R. Varshney, N. Shirish Keskar, and R. Socher. 2020. Limits of detecting text generated by large-scale language models. In _2020 Information Theory and Applications Workshop (ITA)_, pages 1-5.
* Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In _Advances in Neural Information Processing Systems_, volume 30. Curran Associates, Inc.
* Wachsmuth et al. (2018) Henning Wachsmuth, Shahbaz Syed, and Benno Stein. 2018. Retrieval of the best counterargument without prior topic knowledge. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 241-251, Melbourne, Australia. Association for Computational Linguistics.
* Wang and Ling (2016) Lu Wang and Wang Ling. 2016. Neural network-based abstract generation for opinions and arguments. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 47-57, San Diego, California. Association for Computational Linguistics.
* Proceedings of Conference_, 1:438-449.
* Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Pantowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Flu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_, pages 38-45, Online. Association for Computational Linguistics.
* Xing et al. (2017) Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic aware neural response generation. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 31.
* Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In _Advances in Neural Information Processing Systems_, volume 32. Curran Associates, Inc.
* Zuckerman et al. (1998) Ingrid Zuckerman, Richard McConachy, and Kevin B. Korb. 1998. Bayesian reasoning in an abductive mechanism for argument generation and analysis. In _Proceedings of the Fifteenth National/Tenth Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence_, AAAI '98/IAAI '98 

page 833-838, USA. American Association for Artificial Intelligence.

## Appendix A Argument Aspect Annotation Study

For the final crowdsourcing study, we use Amazon Mechanical Turk. Workers had to take a qualification test, have an acceptance rate of at least 95%, and location within the US. We paid $7.6 per hour (minimum wage is $7.25 per hour). Each data sample is annotated by eight crowdworkers. In case the ranker cut off the real aspect(s) from the list of candidates, crowdworkers could select any sequence up to four tokens from a second list.

Figure 2 shows the annotation guidelines for the Amazon Mechanical Turk study. Figure 3 shows one example of a HIT with two aspects selected. Selected aspects are highlighted in the sentence. We did not allow to choose overlapping aspects. If the aspect was not found in the first list provided by the learned ranker, crowdworkers could choose from as second list with the remaining 1-4-grams of the sentence (aspect candidates starting or ending with stopwords, as well as candidates with punctuation and numbers, were removed from the list). Additional checkboxes were added to choose from if the sentence contained no aspect or the aspect was not explicitly mentioned. Figure 4 shows a ranked list of aspect candidates for an example.

The structure of the final dataset is described in Section F. For reproducibility of results, we create fixed splits for in- and cross-topic experiments.

## Appendix B Search Query and Topic Relevance Synonyms

Table 8 lists the ElasticSearch queries we used to retrieve the initial training documents from cc and reddit. Combinations of topics and data sources that are not listed in the table required no expansion of the query to gather enough documents for training. In Table 9, we show the synonyms used for filtering prior to the argument and stance classification step. We filtered out all sentences that did not contain tokens from the topic they belong to or any synonyms defined for this topic.

## Appendix C Model Parameters and Details

All arguments of the training documents are tokenized with a BPE model Sennrich et al. (2016) trained by the authors of the CTRL Keskar et al. (2019). Both the Arg-CTRL\({}_{CC}\) and the Arg-CTRL\({}_{REDDIT}\) are fine-tuned on a Tesla V100 with 32 GB of Memory. We mainly keep the default hyperparameters but reduce the batch size to 4 and train both models for 1 epoch. Each model takes around five days to train on the 1.6M training sentences.

## Appendix D Reference Data Statistics

Table 10 shows the sources and number of arguments for all topics of the reference dataset. The dataset is used to compare the argument generation models to a retrieval approach.

## Appendix E Examples of Generated Arguments

For all eight topics, we show the generated argument with the highest and lowest argument quality score in tables 11 (Arg-CTRL\({}_{CC}\)) and 12 (Arg-CTRL\({}_{REddit}\)). Text in bold shows the given control code, text afterwards represents the generated argument. Numbers in brackets after the text show the quality score as predicted by the argument quality model.

## Appendix F Argument Aspect Detection Dataset

The argument aspect detection dataset contains a total of 5,032 samples in JSONL-format, i.e. each dataset sample has a separate line and can be parsed as JSON. A sample contains the keys:

* **hash**: Unique identifier.
* **aspect_pos**: List of string tuples "(begin,length)", marking the character position and length of each aspect within the argument.
* **aspect_pos_string**: The aspects as a list of strings.
* **stance**: Original stance label of the argument towards the topic, taken from the UKP-Corpus Stab et al. (2018). Either "Argument_for" or "Argument_against".
* **topic**: The topic of the argument.
* **sentence**: The argument.

For reproducibility, we define a fixed cross-topic split with the data of two topics as test set (_gun control_, _school uniforms_), the data of one topic as development set (_death penalty_), and the data of the remaining five topics as train set. We also create a fixed in-topic split by randomly taking 3,532 samples of all topics for training, 500 for development, and 1,000 for testing 



## References

* [1] A. B. Krizhevsky, S. M. Girshick, and R. L. Krause, "A survey of machine learning for image recognition: a review," in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pp. 235-246, 2007.
* [2] J. C. Davis, D. P. Kingma, and J. Ba, "Adam: A method for stochastic optimization," _arXiv preprint arXiv:1912.08802_, 2019.
* [3] T. Darrell, G. Hinton, and Y. Bengio, "Deep residual networks for large scale data," _IEEE Transactions on Information Theory_, vol. 12, no. 1, pp. 1-10, Jan 2016.

[MISSING_PAGE_POST]





\begin{table}
\begin{tabular}{l l} \hline
**Topic** & **Search query** \\ \hline Marijuana legalization (cc and reddit) & ((marijuana legalization) OR (legalization of marijuana) OR (legalization of cannabis)) OR (((marijuana) OR (dope) OR (cannabis) OR (weed)) AND ((law) OR (legal) OR (legalization))) \\ School uniforms (cc and reddit) & (school uniform) OR (college uniform) OR (school outfit) OR ((school) AND (uniform)) OR ((school) AND (quitfit)) OR ((school) AND (jacket)) OR ((school) AND (cloth)) OR ((school) AND (dress)) OR ((college) AND (dress)) \\ Cloning (reddit) & (clone) OR (cloning) OR (clones) OR (cloned) OR (clones) OR (genetically identical) OR (asexual reproduction) \\ Death penalty (reddit) & (capital punishment) OR ((execution) AND (death)) OR ((punishment) AND (death)) OR ((punishment) AND (execution)) OR (electric chair) OR (death penalty) \\ Gun control (reddit) & (gun control) OR (second amendment) or (2nd amendment) OR ((gun) AND (ownership)) OR (gun ownership) OR (arms reduction) OR (arms limitation) OR (gun politics) OR ((gun) AND (lobby)) \\ Nuclear energy (cc and reddit) & nuclear AND (energy OR fission OR power OR plant) \\ \hline \end{tabular}
\end{table}
Table 8: Synonyms added to the topic query to gather initial documents from ElasticSearch. For combinations of topics and data sources (i.e. reddit and cc) that are not listed, we only used the topic as search query.

Figure 4: Example sentence of a HIT with the list of ranked aspect candidates.





\begin{table}
\begin{tabular}{l c c c} \hline
**Metricopic** & **Sentence-Level Args** & **Source** \\  & **pro** & **con** & \\ \hline Abortion & 57 & 47 & proconc.org \\ Gun control & 49 & 50 & procon.org \\ School uniforms & 60 & 55 & procon.org \\ Minimum wage & 59 & 64 & procon.org \\ Marijuana legalization & 63 & 71 & procon.org \\ Death penalty & 22 & 27 & idebate.org \\ Cloning & 23 & 24 & idebate.org \\ Nuclear energy & 16 & 17 & idebate.org \\ \hline
**Total** & **349** & **355** & / \\ \hline \end{tabular}
\end{table}
Table 10: Sources and number of arguments for each topic from the reference dataset.





**abortion PRO reduce abortions :** It's been shown time after time that comprehensive sexual health services ( including free or cheap long term hormonal birth controls ) greatly reduces abortions by preventing unplanned pregnancies from happening in the first place . (0.99)

**abortion PRO crime ._,r=0 & amp ; pagewanted=print & amp ; oref=slogin ) . (0.40)**

**cloning PRO reproduction .** The only way to increase the number of clones is through sexual reproduction , which increases genetic diversity and therefore reduces extinction rates . (0.85)

**cloning PRO awesome .** But yeah , the clone skins look fucking awesome . (0.36)

**nuclear energy PRO safe .** Nuclear is the only viable option for a large scale , reliable and safe form of energy production that can replace fossil fuels as our main energy source . (0.97)

**nuclear energy CON leak .** Biofuel does n't need batteries 6 . (0.41)

* there 's just too much chance that at least one innocent person will die before their execution date was up and they were able to prove their innocence with DNA evidence and/or other exonerating circumstances . (0.95)

**death penalty PRO innocent person .** Innocent people do n't deserve to live 2 . (0.43)

**school uniforms PRO fit .** Dress codes exist to prevent distractions from other students while trying to teach kids appropriate attire which helps them learn proper social skills and fitting into society . (0.83)

**school uniforms PRO nice :** It looks really nice on my college application . (0.37)

**gun control PRO prevent .** Guns also help prevent tyranny by removing checks against government overreach into areas where the populace has little power . (0.95)

**gun control CON problem ;** the guns are n't the real problems . (0.32)

**marjuana legalization CON bad :** Alcohol is also very addictive and has been shown time after time to have negative effects on health yet it remains completely legal while cannabis gets demonized by law enforcement and politicians alike despite being less harmful than many prescription medications in every way imaginale . (0.93)

**marjuana legalization PRO buy .** Get busted by police 5 . (0.36)

**minimum wage PRO poverty :** Raising the minimum wage helps alleviate poverty as well as increase demand for goods and services from consumers . (0.93)

**minimum wage CON pay :** They ca n't pay below minimum wage either . (0.41)

\begin{table} \end{table}
Table 12: Generated arguments with the Arg-CTRL\({}_{\text{REEDDIT}}\). Text in bold shows the given control code, text afterwards represents the generated argument. Numbers in brackets after the text show the quality score as predicted by the argument quality model.