SlowGuess/NAACL_2021_OCR huggingface dataset:
Generation At inference, we gather multiple generated arguments from a control code input by splitting the generated output text into sentences with NLTK (Bird et al., 2009). We observe that for the first generated argument, the Arg-CTRL mostly outputs very short phrases, as it tries to incorporate the control code into a meaningful start of an argument. We prevent this by adding punctuation marks after each control code (e.g. a period or colon), signaling the model to start a new sentence. In this fashion, we generate pro- and con-arguments up to the pre-defined training split size<sup>7</sup> for each topic of the UKP-Corpus, resulting in 7,991 newly generated arguments. We do this with both models and use the generated arguments as a basis for the following analysis and evaluation methods. Examples of generated arguments can be found in Tables 4, 6, and 7 (as part of the evaluation, see Section 7).
Results With no other previous work on explicit control of argument generation (to the best of our knowledge), we decide to proof our concept of aspect-controlled neural argument generation by
comparing both generation models to a retrieval approach as a strong upper bound. The retrieval approach returns all arguments from the classified training data (see Section 4) that match a given topic, stance, and aspect. Both the retrieval and generation approaches are evaluated against reference data from debate portals and compared via METEOR (Lavie and Agarwal, 2007) and ROUGE-L (Lin, 2004) metrics. The retrieval approach has an advantage in this setup, as the arguments are also of human origin and aspects are always explicitly stated within a belonging argument.
The reference data was crawled from two debate portals and consists of pro- and con-paraphrases discussing the eight topics of the UKP-Corpus. As the paragraphs may include non-arguments, we filter these out by classifying all sentences with the ArgumentText API into arguments and non-arguments. This leaves us with 349 pro- and 355 con-arguments over all topics (see App. D for the topic-wise distribution). Next, we detect all aspects in these arguments. Arguments with the same topic, stance, and aspect are then grouped and used as reference for arguments from the (a) generated arguments and (b) retrieval approach arguments if these hold the same topic, stance, and aspect. The results reveal that both the average METEOR and ROUGE-L scores are only marginally lower than the retrieval scores (METEOR is 0.5/1.1 points lower for the  $\mathrm{Arg - CTRL_{REDDIT} / Arg - CTRL_{CC}}$ , see Table 5). It not only shows the strength of the architecture, but also the success in generating sound aspect-specific arguments with our approach.
Overlap with Training Data We find arguments generated by the models to be genuine, i.e. demonstrating substantial differences to the training data. For each of the 7,991 generated arguments, we find the most similar argument in the training data based on the cosine similarity of their BERT embeddings
7Not counting non-arguments from the splits.
8 procon.org and idebate.org
Model	METEOR	ROUGE-L
Retrieval (CC)	17.85	14.72
Arg-CTRLCC	16.80	11.95
Retrieval (REDDIT)	17.29	15.26
Arg-CTRLREDDIT	16.82	12.34
Table 5: Comparison of retrieval and generation approach with reference data from debate portals.
(CLS token). The average cosine similarity of the most similar pairs for both the  $\mathrm{Arg - CTRL_{CC}}$  and  $\mathrm{Arg - CTRL}_{\mathrm{REDDIT}}$  is .92. However, this value is misleading, as even highly similar samples still show clear differences. This is also evident when looking at the average edit distances of 343 (Arg-CTRLcc) and 163 (Arg-CTRLREDIT) for the pairs with highest similarity. Further comparison of these pairs for their longest common (string) overlap reveals only  $9\%$  (Arg-CTRLcc) and  $11\%$  (Arg-CTRLREDIT) overlap on average, mostly consisting of stopwords. For illustration, we show two examples of highly similar pairs in Table 6.
6 Generation in Absence of Aspects
To show the necessity of having prior knowledge of aspects for our controlled argument generation approach, we create training data without prior knowledge of aspects, train a new generation model on it, and compare it to our previous models with prior knowledge of aspects. Equally to the original  $\mathrm{Arg-CTRL}_{\mathrm{CC}}$  's procedure, we gather 100,000 sentences for each stance of a topic from the CC data. As we assume to have no knowledge about the aspects of the arguments, we randomly sample arguments from the CC source documents. We create training documents with numbers of arguments varying between 15 and 1,500 to mimic the data generation process of the original models and fine-tune a new generation model on them. After training, we generate the same number of arguments as for the other two models by using our default control code of [Topic] [Stance] [Aspect]. While the new model was only conditioned on topics and stances at training time, we make sure that all aspects used for generation appear in at least one argument of the model's training data.
We compare all models by verifying whether or not the aspect used for generation (including synonyms and their stems and lemmas) can be found in the generated arguments. For the original models conditioned on aspects, this is true in  $79\%$  of
Generated sentence: We do n't need more gun control laws when we already have enough restrictions on who can buy guns in this country.
Training sentence: We have some of the strongest gun laws in the country, but guns do n't respect boundaries any more than criminals do.
Cosine similarity / edit distance / rel. overlap: 95.59 / 88 / 8%
Generated sentence: The radioactivity of the spent fuel is a concern, as it can be used to make weapons and has been linked to cancer in humans.
Training sentence: However, it does produce radioactive waste, which must be disposed of carefully as it can cause health problems and can be used to make nuclear weapons Cosine similarity / edit distance / rel. overlap: 92.40 / 99 / 17%
Table 6: Training data vs. generated arguments: examples of most similar arguments. Underlines mark the longest common overlap between generated and training sentences.
the cases for  $\mathrm{Arg - CTRL_{REDDIT}}$  and in  $74\%$  of the cases for  $\mathrm{Arg - CTRL_{CC}}$ . For the model that was not conditioned on aspects, however, it is only true in  $8\%$  of the cases. It clearly shows the necessity to condition the model on aspects explicitly, implying the need for argument aspect detection, as the model is unable to learn generating aspect-related arguments otherwise. Moreover, without prior detection of aspects, we have no means for proper aggregation over aspects. We notice that for the model without prior knowledge of aspects,  $79\%$  of all aspects in the training data appear in only one argument. For these aspects, the model will likely not pick up a strong enough signal to learn them.
We evaluate the quality (intrinsic evaluation) of the Arg-CTRL and its performance on an exemplary task (extrinsic evaluation). As a basis, we use the 7,991 arguments generated in Section 5.
7.1 Intrinsic Evaluation
Human Evaluation We conduct an expert evaluation on a subset of generated arguments with two researchers (field of expertise is natural language processing) not involved in this paper. Two aspects are evaluated: fluency and persuasiveness. We consider a sentence as fluent if it is grammatically correct (Hua et al., 2019), i.e. contains neither semantic nor syntactic errors, and arrange this as a binary task. To reduce subjectivity for the persuasiveness evaluation, the experts do not annotate single arguments but instead compare pairs (Habernal and Gurevych, 2016) of generated and refer
ence data arguments (see Section 5.2). The experts could either choose one argument as being more persuasive or both as being equally persuasive. In total, the experts compared 100 (randomly sorted and ordered) argument pairs for persuasiveness and fluency (50 from both the  $\mathrm{Arg - CTRL_{REDIT}}$  and the  $\mathrm{Arg - CTRL_{CC}}$ ). A pair of arguments always had the same topic and stance. For fluency, only the annotations made for generated arguments were extracted and taken into account. Averaged results of both experts show that in  $33\%$  of the cases, the generated argument is either more convincing  $(29\%)$  or as convincing  $(4\%)$  as the reference argument. Moreover,  $83\%$  of generated arguments are fluent. The inter-annotator agreement (Cohen, 1960) between the two experts is Cohen's  $\kappa = .30$  (percentage agreement: .62) for persuasiveness and  $\kappa = .43$  (percentage agreement: .72) for fluency, which can be interpreted as "fair" and "moderate" agreement, respectively (Landis and Koch, 1977). As we compare to high-quality, curated data, the perceived persuasiveness of the generated arguments shows the potential of the work—further strengthened in the remainder of this section.
Argument Quality We introduce a novel method to evaluate generated arguments based on the argument quality detection approach proposed by Gretz et al. (2020b). They create an argument quality dataset that contains around 30,000 arguments over 71 topics. For each argument, annotators were asked whether or not they would recommend a friend to use the displayed argument in a speech. The quality scores for each argument result from a weighted average (WA) or MACE Probability function of all annotations and range between 0 (lowest quality) and 1.0 (highest quality). We use the WA-score as label, the same model  $(\mathrm{BERT}_{BASE})$  and hyperparameters as given in the original paper, and reproduce the reported correlations of .52 (Pearson) and .48 (Spearman) on the test dataset (averaged over five different seeds). The model predicts an average argument quality of .71 for the  $\mathrm{Arg - CTRL_{REDIT}}$  arguments, .75 for the training arguments of the UKP-Corpus, and even .76 for the  $\mathrm{Arg - CTRL_{CC}}$  arguments. It shows that our model is able to produce arguments that are generally on a similar quality level as arguments made by humans. The difference of 5 points between the  $\mathrm{Arg - CTRL_{CC}}$  and the  $\mathrm{Arg - CTRL_{REDIT}}$ , however, shows that training on REDIT data leads to a lower quality of generated arguments. Exemplary
Human: Nuclear energy produces waste that stays radioactive for thousands of years and pollutes the environment.
Arg-CTRLcc: (nuclear energy PRO waste.) The only thing that's going to be left over is the spent fuel rods, which are not very radioactive and can be reprocessed [...]
Arg-CTRL<sub>cc</sub>: (nuclear energy PRO radioactive.) The radioactivity of the waste is not a problem, because it will decay away in about 300 years or so [...]
Arg-CTRL<sub>CC</sub>: (nuclear energy PRO environment.) The fact that the radioactive waste is not only stored in a safe manner, but also protected from any potential threat to human health or the environment for thousands of years [...]
Human: School uniforms are expensive and affect the pupil's individuality.
$\mathrm{Arg - CTRL_{CC}}$  : (school uniforms PRO expensive.) The cost of the uniform is not that much more than a regular shirt and shorts [...]
Arg-CTRLcc: (school uniforms PRO individuality.) The uniform is a symbol of unity and identity, which helps to foster an environment where students can feel comfortable about expressing their own individual style without being judged [...]
Table 7: Generated counter-arguments with the  $\mathrm{Arg - }$ $\mathrm{CTRL}_{\mathrm{CC}}$  . Aspects in the initial argument are underlined and used for the counter-argument generation. Control code in brackets and “[...]” signals shortened text.
for three topics, we show the generated arguments with the highest and lowest argument quality in Table 4 (see App. E for the full table).
7.2 Extrinsic Evaluation: Counter-Arguments
Drafting counter-arguments is an important skill for debating, to provide constructive feedback, and to foster critical thinking. We lean onto the work of Wachsmuth et al. (2018) who describe a counterargument as discussing the same aspect as an initial argument, but with a switched stance. Hence, given our defined control codes, our model is especially fit for counter-argument generation. Unlike current models for this task, we do not require a specific dataset with argument and counterargument pairs (Hidey and McKeown, 2019; Hua et al., 2019). Also, in contrast to the model by Hua and Wang (2019) that implicitly integrates input-related "Keyphrases" into the process of counter-argument generation, our model is able to concentrate on every aspect of the input explicitly and with a separate argument, allowing for more transparency and interpretability over the process of counter-argument generation. We exemplary show how the combination of aspect detection and controlled argument generation can be successfully leveraged to tackle this task. For that, we manually
compose initial arguments for the topics nuclear energy and school uniforms. Then, we automatically detect their aspects and generate a counterargument for each aspect by passing the topic, opposite stance of the original argument, and one of the aspects into the  $\mathrm{Arg - CTRL_{CC}}$ . For both topics, the  $\mathrm{Arg - CTRL_{CC}}$  produces meaningful counter-arguments based on the detected aspects (see Table 7).
We apply the concept of controlled neural text generation to the domain of argument generation. Our Arg-CTRL is conditioned on topics, stances, and aspects and can reliably create arguments using these control codes. We show that arguments generated with our approach are genuine and of high argumentative and grammatical quality in general. Moreover, we show that our approach can be used to generate counter-arguments in a transparent and interpretable way. We fine-tune the Arg-CTRL on two different data sources and find that using mixed data from Common-Crawl results in a higher quality of generated arguments than using user discussions from Reddit-Comments. Further, we define argument aspect detection for controlled argument generation and introduce a novel annotation scheme to crowdsourced argument aspect annotations, resulting in a high-quality dataset. We publish the model weights, data, and all code necessary to train the Arg-CTRL.
Models for argument and claim generation have been discussed in our related work and are widely available. Gretz et al. (2020a) suggest that, in order to allow for a fine-grained control over claim/argument generation, aspect selection needs to be handled carefully, which is what we have focused on in this work. The dangers of misuse of language models like the CTRL have been extensively discussed by its authors (Keskar et al., 2019). The ethical impact of these works has been weighed and deemed justifiable.
Argument generation—and natural language generation as a whole—is subject to dual use. The technology can be used to create arguments that cannot be distinguished from human-made arguments. While our intentions are to support society, to foster diversity in debates, and to encourage research on this important topic, we are aware of
the possibility of harmful applications this model can be used for. For instance, the model could be used to generate only opposing (or supporting) arguments on one of the pretrained topics and aspects and, as such, bias a debate into a certain direction. Also, bots could use the generated arguments to spread them via social media. The same is true, however, for argument search engines, which can be used by malicious parties to retrieve (and then spread) potentially harmful information.
However, controllable argument generation can also be used to support finding and formulating (counter-)arguments for debates, for writing essays, to enrich one-sided discussions, and thus, to make discourse more diverse overall. For instance, anticipating opposing arguments is crucial for critical thinking, which is the foundation for any democratic society. The skill is extensively taught in school and university education. However, confirmation bias (or myside bias) (Stanovich et al., 2013), i.e. the tendency to ignore opposing arguments, is an ever-present issue. Technologies like ours could be used to mitigate this issue by, for instance, automatically providing topic- and aspect-specific counter-arguments for all arguments of a given text (this has been shown for single arguments in Section 7.2). We believe that working on and providing access to such models is of major importance and, overall, a benefit to society.
Open-sourcing such language models also encourages the work on counter-measures to detect malicious use: While many works have been published on the topic of automatic fake news detection in texts (Kaliyar et al., 2020; Reis et al., 2019; Hanselowski et al., 2018; Pérez-Rosas et al., 2018), the recent emergence of large-scale language models has also encouraged research to focus on detecting the creator of these texts (Varshney et al., 2020; Zellers et al., 2019). The former approaches are aimed at detecting fake news in general, i.e. independent of who (or what) composed a text, whereas the latter approaches are designed to recognize if a text was written by a human or generated by a language model. We encourage the work on both types of methods. Ideally, social networks and news platforms would indicate if a statement was automatically generated in addition to its factual correctness.
Further, we point out some limitations of the Arg-CTRL that mitigate the risks discussed before. One of these limitations is that it cannot be used
to generate arguments for unseen topics, which makes a widespread application (e.g. to produce fake news) rather unlikely (using an unseen topic as control code results in nonsensical repetitions of the input). The analysis in Section 6 of the paper shows that the model fails to produce aspect-specific sentences in  $92\%$  of the cases if it was not explicitly conditioned on them at training time. Even in case of success, the aspect has to exist in the training data. Also, the model is trained with balanced classes, i.e. both supporting and opposing arguments for each topic are seen with equal frequency to prevent possible bias into one or the other direction.
To further restrict malicious use, we release the training data for the Arg-CTRLs with an additional clause that forbids use for any other than research purposes. Also, all the training datasets for the Arg-CTRLs will be accessible only via access control (e-mail, name, and purpose of use). Lastly, this work has been reviewed by the ethics committee of the Technical University of Darmstadt that issued a positive vote.
We thank Tilman Beck and Nandan Thakur for their support in the human evaluation (Section 7.1). This work has been supported by the German Research Foundation within the project "Open Argument Mining" (GU 798/25-1), associated with the Priority Program "Robust Argumentation Machines (RATIO)" (SPP-1999).
Roxanne El Baff, Henning Wachsmuth, Khalid Al-Khatib, Manfred Stede, and Benno Stein. 2019. Computational Argumentation Synthesis as a Language Modeling Task. In 12th International Natural Language Generation Conference (INLG 2019). ACL.
Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. 3rd International Conference on Learning Representations,
ICLR 2015 - Conference Track Proceedings, pages 1-15.
Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of machine learning research, 3(Feb):1137-1155.
Yonatan Bilu, Ariel Gera, Daniel Hershcovich, Benjamin Sznajder, Dan Lahav, Guy Moshkowich, Anael Malet, Assaf Gavron, and Noam Slonim. 2019. Argument invention from first principles. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1013-1026, Florence, Italy. Association for Computational Linguistics.
Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python, 1st edition. O'Reilly Media, Inc.
Filip Boltužić and Jan Šnajder. 2017. Toward stance classification based on claim microstructures. In Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 74-80, Copenhagen, Denmark. Association for Computational Linguistics.
Amber E. Boydstun, Dallas Card, Justin Gross, Paul Resnick, and Noah A. Smith. 2014. Tracking the Development of Media Frames within and across Policy Issues. Carnegie Mellon University.
Giuseppe Carenini and Johanna D Moore. 2006. Generating and evaluating evaluative arguments. Artificial Intelligence, 170(11):925-952.
Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37-46.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019. Plug and Play Language Models: A Simple Approach to Controlled Text Generation. arXiv, abs/1912.02164.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Atsushi Fujii and Tetsuya Ishikawa. 2006. A system for summarizing and visualizing arguments in subjective documents: Toward supporting decision making. In Proceedings of the Workshop on Sentiment and Subjectivity in Text, SST '06, pages 15-22, Stroudsburg, PA, USA. Association for Computational Linguistics.