SlowGuess/NAACL_2021_OCR huggingface dataset:
Debela Gemechu and Chris Reed. 2019. Decompositional argument mining: A general purpose approach for argument graph construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 516-526, Florence, Italy. Association for Computational Linguistics.
Shai Gretz, Yonatan Bilu, Edo Cohen-Karlik, and Noam Slonim. 2020a. The workweek is the best time to start a family - a study of gpt-2 based claim generation. arXiv, abs/2010.06185.
Shai Gretz, Roni Friedman, Edo Cohen-Karlik, Assaf Toledo, Dan Lahav, Ranit Aharonov, and Noam Slonim. 2020b. A large-scale dataset for argument quality ranking: Construction and analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):7805-7813.
Ivan Habernal and Iryna Gurevych. 2016. Which argument is more convincing? analyzing and predicting convincingness of web arguments using bidirectional LSTM. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1589-1599, Berlin, Germany. Association for Computational Linguistics.
Andreas Hanselowski, Avinesh PVS, Benjamin Schiller, Felix Caspelherr, Debanjan Chaudhuri, Christian M. Meyer, and Iryna Gurevych. 2018. A retrospective analysis of the fake news challenge stance-detection task. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1859-1874, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Mareike Hartmann, Tallulah Jansen, Isabelle Augenstein, and Anders Søgaard. 2019. Issue framing in online discussion fora. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1401-1407, Minneapolis, Minnesota. Association for Computational Linguistics.
Christopher Hidey and Kathy McKeown. 2019. Fixed that for you: Generating contrastive claims with semantic edits. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1756-1767, Minneapolis, Minnesota. Association for Computational Linguistics.
Xinyu Hua, Zhe Hu, and Lu Wang. 2019. Argument generation with retrieval, planning, and realization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2661-2672, Florence, Italy. Association for Computational Linguistics.
Xinyu Hua and Lu Wang. 2018. Neural argument generation augmented with externally retrieved evidence. In Proceedings of the 56th Annual Meeting of
the Association for Computational Linguistics (Volume 1: Long Papers), pages 219-230, Melbourne, Australia. Association for Computational Linguistics.
Xinyu Hua and Lu Wang. 2019. Sentence-level content planning and style specification for neural text generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 591-602, Hong Kong, China. Association for Computational Linguistics.
Rohit Kumar Kaliyar, Anurag Goswami, Pratik Narang, and Soumendu Sinha. 2020. Fndnet - a deep convolutional neural network for fake news detection. Cogn. Syst. Res., 61(C):32-44.
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. arXiv, abs/1909.05858.
J. Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33(1):159-174.
Alon Lavie and Abhaya Agarwal. 2007. Meteor: An automatic metric for mt evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, StatMT '07, page 228-231, USA. Association for Computational Linguistics.
Dieu Thu Le, Cam-Tu Nguyen, and Kim Anh Nguyen. 2018. Dave the debater: a retrieval-based and generative argumentative dialogue agent. In Proceedings of the 5th Workshop on Argument Mining, pages 121-130, Brussels, Belgium. Association for Computational Linguistics.
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. 2019. Multi-task deep neural networks for natural language understanding. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4487-4496, Florence, Italy. Association for Computational Linguistics.
Amita Misra, Pranav Anand, Jean E. Fox Tree, and Marilyn Walker. 2015. Using summarization to discover argument facets in online ideological dialog. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 430-440, Denver, Colorado. Association for Computational Linguistics.
Nona Naderi and Graeme Hirst. 2017. Classifying frames at the sentence level in news articles. In Proceedings of the International Conference Recent
Advances in Natural Language Processing, RANLP 2017, pages 536-542, Varna, Bulgaria. INCOMA Ltd.
Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net.
Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. 2018. Automatic detection of fake news. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3391-3401, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
J. C. S. Reis, A. Correia, F. Murai, A. Veloso, and F. Benevenuto. 2019. Supervised learning for fake news detection. IEEE Intelligent Systems, 34(2):76-81.
Misa Sato, Kohsuke Yanai, Toshinori Miyoshi, Toshihiko Yanase, Makoto Iwayama, Qinghua Sun, and Yoshiki Niwa. 2015. End-to-end argument generation system in debating. In Proceedings of ACLIJCNLP 2015 System Demonstrations, pages 109-114, Beijing, China. Association for Computational Linguistics and The Asian Federation of Natural Language Processing.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
Christian Stab, Johannes Daxenberger, Chris Stahlhut, Tristan Miller, Benjamin Schiller, Christopher Tauchmann, Steffen Eger, and Iryna Gurevych. 2018a. ArgumentText: Searching for arguments in heterogeneous sources. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 21-25, New Orleans, Louisiana. Association for Computational Linguistics.
Christian Stab, Tristan Miller, Benjamin Schiller, Pranav Rai, and Iryna Gurevych. 2018b. Cross-topic argument mining from heterogeneous sources. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 3664-3674. Association for Computational Linguistics.
Keith E. Stanovich, Richard F. West, and Maggie E. Toplak. 2013. Myside bias, rational thinking, and intelligence. Current Directions in Psychological Science, 22(4):259-264.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc.
L. R. Varshney, N. Shirish Keskar, and R. Socher. 2020. Limits of detecting text generated by large-scale language models. In 2020 Information Theory and Applications Workshop (ITA), pages 1-5.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Henning Wachsmuth, Shahbaz Syed, and Benno Stein. 2018. Retrieval of the best counterargument without prior topic knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 241-251, Melbourne, Australia. Association for Computational Linguistics.
Lu Wang and Wang Ling. 2016. Neural network-based abstract generation for opinions and arguments. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 47-57, San Diego, California. Association for Computational Linguistics.
Tsung Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gašić, Lina M. Rojas-Barahona, Pei Hao Su, Stefan Ultes, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialogue system. 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 - Proceedings of Conference, 1:438-449.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic aware neural response generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31.
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
Ingrid Zukerman, Richard McConachy, and Kevin B. Korb. 1998. Bayesian reasoning in an abductive mechanism for argument generation and analysis. In Proceedings of the Fifteenth National/Tenth Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence, AAAI '98/IAAI '98,
page 833-838, USA. American Association for Artificial Intelligence.
A Argument Aspect Annotation Study
For the final crowdsourcing study, we use Amazon Mechanical Turk. Workers had to take a qualification test, have an acceptance rate of at least  $95\%$ , and location within the US. We paid $7.6 per hour (minimum wage is $7.25 per hour). Each data sample is annotated by eight crowdworkers. In case the ranker cut off the real aspect(s) from the list of candidates, crowdworkers could select any sequence up to four tokens from a second list.
Figure 2 shows the annotation guidelines for the Amazon Mechanical Turk study. Figure 3 shows one example of a HIT with two aspects selected. Selected aspects are highlighted in the sentence. We did not allow to choose overlapping aspects. If the aspect was not found in the first list provided by the learned ranker, crowdworkers could choose from as second list with the remaining 1-4-grams of the sentence (aspect candidates starting or ending with stopwords, as well as candidates with punctuation and numbers, were removed from the list). Additional checkboxes were added to choose from if the sentence contained no aspect or the aspect was not explicitly mentioned. Figure 4 shows a ranked list of aspect candidates for an example.
The structure of the final dataset is described in Section F. For reproducibility of results, we create fixed splits for in- and cross-topic experiments.
B Search Query and Topic Relevance Synonyms
Table 8 lists the ElasticSearch queries we used to retrieve the initial training documents from CC and REDDIT. Combinations of topics and data sources that are not listed in the table required no expansion of the query to gather enough documents for training. In Table 9, we show the synonyms used for filtering prior to the argument and stance classification step. We filtered out all sentences that did not contain tokens from the topic they belong to or any synonyms defined for this topic.
C Model Parameters and Details
All arguments of the training documents are tokenized with a BPE model (Sennrich et al., 2016) trained by the authors of the CTRL (Keskar et al., 2019). Both the  $\mathrm{Arg - CTRL_{CC}}$  and the  $\mathrm{Arg - CTRL_{REDDIT}}$  are fine-tuned on a Tesla V100 with
32 GB of Memory. We mainly keep the default hyperparameters but reduce the batch size to 4 and train both models for 1 epoch. Each model takes around five days to train on the 1.6M training sentences.
D Reference Data Statistics
Table 10 shows the sources and number of arguments for all topics of the reference dataset. The dataset is used to compare the argument generation models to a retrieval approach.
E Examples of Generated Arguments
For all eight topics, we show the generated argument with the highest and lowest argument quality score in tables 11 (Arg-CTRL<sub>CC</sub>) and 12 (Arg-CTRL<sub>REDDIT</sub>). Text in bold shows the given control code, text afterwards represents the generated argument. Numbers in brackets after the text show the quality score as predicted by the argument quality model.
F Argument Aspect Detection Dataset
The argument aspect detection dataset contains a total of 5,032 samples in JSONL-format, i.e. each dataset sample has a separate line and can be parsed as JSON. A sample contains the keys:
- hash: Unique identifier.
- aspect_pos: List of string tuples “(begin,length)”, marking the character position and length of each aspect within the argument.
- aspect_pos_string: The aspects as a list of strings.
- stance: Original stance label of the argument towards the topic, taken from the UKP-Corpus (Stab et al., 2018b). Either "Argument_for" or "Argument Against".
- topic: The topic of the argument.
- sentence: The argument.
For reproducibility, we define a fixed cross-topic split with the data of two topics as test set (gun control, school uniforms), the data of one topic as development set (death penalty), and the data of the remaining five topics as train set. We also create a fixed in-topic split by randomly taking 3,532 samples of all topics for training, 500 for development, and 1,000 for testing.
In the following, you will find a set of sentences. The sentences argue for or against a given topic (e.g. school uniforms, nuclear energy, etc.). Your task is to identify the main reasons why they argue in such a way - we refer to these reasons as aspects. Aspects answer the question why we should support or oppose a specific topic.
The motivation behind this task is to learn grouping sentences of the same topic (e.g. nuclear energy) by the aspects (e.g. costs, safety) they argue about.
Note: In some cases, the topic might be incorrect or only implicitly related, as it was tagged automatically. The topics are just listed to provide additional context and an incorrectly assigned topic should not worry you.
An aspect resembles the core reasoning of a sentence.
For each sentence ask yourself: "Which word sequences (e.g. "radioactive waste") are relevant to determine whether to support/oppose the given topic (e.g. Nuclear energy)?" The shortest possible answer in the sentence is usually the aspect.
- A sentence can have (a) no aspect, (b) one aspect, or (c) several aspects, and one aspect can have several words.
Each chosen aspect should be a valid aspect on its own.
- Aspects should be as short as possible.
TOPIC: Nuclear energy
$\circ$  Sentence: It is pretty expensive to build and run nuclear power plants
$\circ$  Aspect(s): [expensive].
- Explanation: In this sentence, the relevant factor to determine whether to support/oppose nuclear energy is that it is "expensive". Please do not select adverbs like "pretty", "very", etc., as they do not add further information to the reasoning behind the aspect.
- TOPIC: Nuclear energy
Sentence: Compared to coal-fired power , nuclear energy is clean but the reactors can easily be targeted by terrorist attacks
$\diamond$  Aspect(s): [clean, terrorist attacks].
- Explanation: In this sentence, the relevant factors to determine whether to support/oppose nuclear energy is that it is "clean" and a possible target of "terrorist attacks". A sentence can have several aspects and they can be opposing and supporting within the same sentence.
TOPIC: Marijuana legalization
$\diamond$  Sentence: Legalization will inevitably lead to a decrease in medical opioid users.
$\circ$  Aspect(s): [medical opioid users].
Explain: In this sentence, the relevant factor to determine whether to support/oppose marijuana legalization is the issue of people who have to rely on opioids instead. Thus, the aspect is "medical opioid users". The words "decrease in" are not necessary to understand the reason for supporting marijuana legalization. Whenever the aspect you chose seems quite long, try to remove words and ask yourself if it would still count as a valid aspect (there are exceptions of course, when aspects are indeed long).
How to use the HIT:
To select an aspect, click in the first input field below a sentence and select as many aspects from the list as you see fit.
- You can also use the keyboard (arrow keys for selection, tab to jump to the next field, backspace to remove aspects)
If the aspect is not in the first list, please click on Aspect not in the first list, show more aspect candidates, and select the missing aspects from the second list of remaining candidates. You can add aspects from both lists.
If you are in doubt about two similar aspects in both lists, take the one you find in the first list.
If the sentence contains no aspect or it is not in either of the two lists, hit the repsective checkbox and solve the math equation (digits only).
If you select multiple aspects, they must not overlap (you will see a warning message if they do).
After processing all sentences, press the submit button to finish this HIT. If you have a remark or something is unclear, please inform us through the feedback box below each HIT.
Note: some hits include easy-to-answer quality control questions. These will ask you to select specific words as aspect(s). Please make sure to answer these questions correctly.
Figure 2: Guidelines for the final annotation study.
2) Topic: nuclear energy
Since the development of the atomic bomb , the human race has found other uses for this technology that still divides people - it provides power for ourhomes and has medicinal uses .
Aspects found:
medicinal uses provides power
Aspect not in the first list, show more aspect candidates.
The sentence contains no aspects.
The sentence contains aspects, but they are not in any of the lists.
3) Topic: nuclear energy
Even if you lived right next door to a nuclear power plant , you would still receive less radiation each year than you would receive in just one round-trip flight from New York to Los Angeles . "
Figure 3: Example sentence of a HIT with two aspects selected.
Figure 4: Example sentence of a HIT with the list of ranked aspect candidates.
Topic	Search query
Marijuana legalization (CC and REDDIT)	((marijuana legalization) OR (legalization of marijuana) OR (legalization of cannabis)) OR (((marijuana) OR (dope) OR (cannabis) OR (weed)) AND ((law) OR (legal) OR (legalization)))
School uniforms (CC and REDDIT)	(school uniform) OR (college uniform) OR (school outfit) OR ((school) AND (uniform)) OR ((school) AND (outfit)) OR ((school) AND (jacket)) OR ((school) AND (cloth)) OR ((school) AND (dress)) OR ((college) AND (dress))
Cloning (REDDIT)	(clone) OR (cloning) OR (clones) OR (cloned) OR (clones) OR (genetically identical) OR (asexual reproduction)
Death penalty (REDDIT)	(capital punishment) OR ((execution) AND (death)) OR ((punishment) AND (death)) OR ((punishment) AND (execution)) OR (electric chair) OR (death penalty)
Gun control (REDDIT)	(gun control) OR (second amendment) or (2nd amendment) OR ((gun) AND (ownership)) OR (gun ownership) OR (arms reduction) OR (arms limitation) OR (gun politics) OR ((gun) AND (lobby))
Nuclear energy (CC and REDDIT)	nuclear AND (energy OR fission OR power OR plant)
Table 8: Synonyms added to the topic query to gather initial documents from ElasticSearch. For combinations of topics and data sources (i.e. REDDIT and CC) that are not listed, we only used the topic as search query.
Topic	Synonyms
School uniforms	uniform, college, outfit, dress, suit, jacket, cloth
Nuclear energy	fission, fusion, atomic energy, nuclear power, atomic power, radioactive, radioactivity
Marijuana legalization	cannabis, legalization of marijuana, legal, illegal, law, weed, dope
Cloning	clone, cloned, duplicate, copy, reproductive, asexual
Death penalty	capital punishment, execution, electric chair, punishment, punish
Minimum wage	living wage, base pay, average wage, low income
Abortion	abort, termination, misbirth, birth control
Gun control	second amendment, ownership, arms reduction, arms limitation
Table 9: Topic synonyms to pre-filter sentences prior to argument and stance classification.
Topic	Sentence-LevelArgs	Source
pro	con
Abortion	57	47	procon.org
Gun control	49	50	procon.org
School uniforms	60	55	procon.org
Minimum wage	59	64	procon.org
Marijuana legalization	63	71	procon.org
Death penalty	22	27	idebate.org
Cloning	23	24	idebate.org
Nuclear energy	16	17	idebate.org
Total	349	355	/
Table 11: Generated arguments with the  $\mathrm{Arg - CTRL_{CC}}$ . Text in bold shows the given control code, text afterwards represents the generated argument. Numbers in brackets after the text show the quality score as predicted by the argument quality model.