SlowGuess/NAACL_2021_OCR huggingface dataset:
- By modeling context representations,  $\mathrm{MATCH}_{rnn} + \mathrm{C}_b$  and  $\mathrm{MATCH}_{rnn} + \mathrm{C}_h$  significantly outperform  $\mathrm{MATCH}_{rnn}$ . This proves that contextual information is helpful for identifying interactive argument pairs.
- Argumentative contexts often contain a list of arguments. In comparison of  $\mathrm{MATCH}_{rnn} + \mathrm{C}_b$  and  $\mathrm{MATCH}_{rnn} + \mathrm{C}_h$ , we find that  $\mathrm{MATCH}_{rnn} + \mathrm{C}_h$  achieve much better results than  $\mathrm{MATCH}_{rnn} + \mathrm{C}_b$ . This demonstrates the effectiveness of representing argumentative contexts on argument level instead of word level.
- By using autoencoders for argument representation learning, our model,  $\mathrm{MATCH}_{vae} + \mathrm{C}_h$  and  $\mathrm{MATCH}_{ae} + \mathrm{C}_h$  outperform  $\mathrm{MATCH}_{rnn} + \mathrm{C}_h$ . This indicates the effectiveness of argument representation learning.
5.2 Hyper-Parameter Sensitivity Analysis
We investigate the impact of two hyper-parameters on our model, namely the number of discrete latent variables  $M$  and the number of categories for each latent variable  $K$  in DVAE. For studying the impact of  $M$  and  $K$ , we set them as 1, 3, 5, 7, 9 respectively while keep other hyper-parameters the same as our best model. We report  $P@1$  of different settings.
As shown in Figure 4, we observe that curves obtained by changing the two parameters follow similar pattern. When the number increases,  $P@1$  first gradually grows, reaching the highest at position 5 and drops gradually after that. When  $K$  and  $M$  are relatively high, say larger than 3, our model can always outperform VAE which is the most competitive baseline, indicating the effectiveness of the discrete representation for interactive arguments identification.
Figure 5: Visualization of posterior distributions of discrete latent variables  $z_{1} \sim z_{5}$  respectively. We find that the posterior distributions of  $z_{1} \sim z_{5}$  of Positive reply is more similar to those of Quotation compared to other Negative replies.
5.3 Discrete Latent Variables Analysis
Here, we try to find out why DVAE performs best on interactive argument pair identification. Given an argument, we set  $M = 5$ ,  $K = 5$  and learn the corresponding discrete code set  $Z_{code}(1) \sim Z_{code}(5)$ . We use the best model to select correct instances for argument matching in the dataset and cluster all quotations and corresponding replies according to the same discrete code set. We get 2,272 clusters, of which 119 clusters have more than 100 arguments and we find that arguments with the same discrete code set are semantically related.
To show the reason why DVAE performs well on our task more intuitively, we select a case from our dataset shown in Table 3 and employ DVAE to learn discrete representations for arguments to capture varying aspects  $z_{1} \sim z_{5}$ . The posterior distributions of discrete latent variables  $z_{1} \sim z_{5}$  for the quotation and replies are shown in Figure 5.
As shown in Figure 5, each subgraph shows the distribution of  $z_{i}$  on  $K$  categories of the quotation and corresponding replies. We can find that the posterior distributions of  $z_{1} \sim z_{5}$  of Positive reply are more similar to those of Quotation compared to other Negative replies. This finding proves that if the two arguments are more semantically related, their posterior distribution on each aspect  $z_{i}$  should be more similar. This further interprets why Positive reply has interactive relationship with Quotation and why DVAE performs well on interactive argument pair identification.
Quotation: I bet that John Boehner would deal with congress as president more easily than Joe Biden due to his constant interaction with it.
Positive reply: Do you think that have anything to do with the fact that Boehner is a republican, and congress is controlled by republicans?
Negative reply 1: I would propose that the title of vice president be kept, but to remove their right to succession for presidency.
Negative reply 2: Does Biden have the same level of respect from foreign nations needed to guide the country?
Negative reply 3: He did lose however, so perhaps people do put weight into the vp choice.
Negative reply 4: I don’t know why you think this can be ignored.
Table 3: A case selected from our dataset.
Here, we inspect outputs of our model to identify major causes of errors. Here are two major issues.
- The number of  $M$  and  $K$  may not cover the latent space of all arguments in the dataset. Natural language is complex and diverse. If the size of the latent space doesn't fully contain semantic information of the arguments, it will cause the failure of our model. Considering the number of aspects may vary for different topics, it is not perfect to use a universal setting of  $K$  and  $M$ .
- Attention Error. In our model, we employ a quotation-guided attention to evaluate the relevance between each word in the reply and the discrete representation of the quotation. If the attention focuses on unimportant words, it causes errors. It might be useful to utilize discrete representation to further regulate the attention procedure.
Models	Pairwise accuracy
Tan et al. (2016)	65.70
Ji et al. (2018)	70.45
Our model	84.50
Table 4: The performances of different models for the task of argumentative comments persuasiveness evaluation on the dataset in Tan et al. (2016). Numbers for the two comparative models are copied from their original papers.
5.5 Effectiveness on Argumentative Comments Persuasiveness Evaluation
To further investigate the usefulness of our learned representations, we apply them to a downstream task: persuasiveness evaluation for argumentative comments (Tan et al., 2016; Ji et al., 2018). It takes two arguments as input (one is original and another is a reply) and output a score to evaluate the quality of the reply. The reasons for choosing this task are two fold. First, both tasks focus on dialogical arguments. Second, both tasks can be formulated as a pairwise ranking problem. The performance of different models are shown in Table 4. Note that we use the original CMV dataset and follow the previous setup in Tan et al. (2016); Ji et al. (2018). We find that our model outperforms the state-of-the-art method (Ji et al., 2018) by a large margin, which indicates that our learned representation can well help downstream tasks.
In this section, we will introduce two major areas related to our work, which are dialogical argumentation and argument representation learning.
6.1 Dialogical Argumentation
Computational argumentation is a growing subfield of natural language processing in which arguments are analyzed in various respects. Previous works mainly focus on analyzing the argumentative structure in texts. Recently, the dialogical argumentation has become an active topic.
Dialogical argumentation refers to a series of interactive arguments related to a given topic, involving argument retraction, view exchange, and so on. Existing research covers discourse structure prediction (Liu et al., 2018), dialog summarization (Hsueh and Moore, 2007), etc. There are several attempts to address tasks related to analyzing the relationship between arguments (Wang and Cardie, 2014; Persing and Ng, 2017) and evaluat
ing the quality of persuasive arguments (Habernal and Gurevych, 2016).
Gottipati et al. (2013) use sentiment lexicons as a preprocessing step and propose a probabilistic graphical model to predict stance of arguments in their dataset. Park et al. (2011) design several argumentation-motivated features to finish the debate stance classification in Korean newswire discourse. Sridhar et al. (2015) consider the joint stance classification of arguments and relations among them and find a multi-level model will work better. for a combination of post-level and author-level collective modeling of both stance and disagreement could bring further improvements in performance.
Wang and Cardie (2014) create a dispute corpus from Wikipedia and use a sentiment analysis to predict the dispute label of arguments. Wei et al. (2016) collect a dataset from CMV and analyze the correlation between disputing quality and disputation behaviors. analyze the disputation action in the online debate. Given an original argument and an argument disputing it, they aims to evaluate the quality of a disputing comment based on the original argument and the discussed topic. Habernal and Gurevych (2016) crowdsourced the UKPCovArg1 corpus to study what makes an informal social media argument convincing. They crowdsource the UKPCovArg1 corpus and use SVM and bidirectional LSTM to experiment on their annotated datasets. Tan et al. (2016) pay attention to belief change in the ChangeMyView subreddit, in which an original poster challenges others to change his/her opinion. They construct datasets from CMV and employ logistic regression to predict which reply in the pair is more persuasive. In addition, Persing and Ng (2017) annotate a corpus with persuasiveness scores and the errors they contain to analyze why arguments are unpersuasive.
Previous work mainly focuses on analyzing interactions between two arguments in debate. However, there is limited research on the interactions between posts. In this work, we propose a novel task of identifying interactive argument pairs from argumentative posts to further understand the interactions between posts. Our work is also related with some similar tasks, such as question answering and sentence alignment. They focus on the design of attention mechanism to learn sentence representations (Wang et al., 2017a) and their relations with others (Wang et al., 2017b). Our task is inherently
different from theirs because our target arguments naturally occur in the complex interaction context of dialogues, which requires additional efforts for understanding the discourse structure therein.
6.2 Argument Representation Learning
Argument representation learning for natural language has been studied widely in the past few years. Previous work discuss prior approaches to learning argument representations from labelled and unlabelled data.
There have been attempts to use labeled/structured data to learn argument representations. Wieting et al. (2016) and Wieting and Gimpel (2017) introduce a large sentential paraphrase dataset and use paraphrase data to learn an encoder that maps synonymous phrases to similar embeddings. Wieting et al. (2017) explore the use of machine translation to obtain more paraphrase data via back-translation of bilingual argument pairs for learning paraphrastic embeddings. They show how neural backtranslation could be used to generate paraphrases. Hermann and Blunsom (2013) explore a language-specific encoder applied to each argument and represent the argument by the mean vector of the words involved. They consider minimizing the inner product between paired arguments in different languages as the training objective and do not rely on word alignments. Conneau et al. (2017) propose a model called InferSent, which is used as the baseline as it served as the inspiration for the inclusion of the SNLI task in the multitask model. They prove that NLI is an effective task for pre-training and transfer learning in obtaining generic argument representations. They train argument encoders from identifying one of three relationships between two given arguments - entailment, neutral and contradiction. Results prove that the argument representations learned by this task perform strongly on downstream transfer tasks.
Due to the availability of practically unlimited textual data, learning argument representations via unsupervised methods is an attractive proposition. Kiros et al. (2015) present the model called Skip Thought for learning representations by predicting the previous and next argument, which is a generalization of the skip-gram model (Mikolov et al., 2013). Exploiting the relatedness inherent in adjacent arguments, the model is trained by using
the encoder to encode a particular argument and then using the decoder to decode words in adjacent arguments. Bowman et al. (2016) introduce variational autoencoders to incorporate distributed latent representations of entire arguments. In addition, Hill et al. (2016) propose the FastSent model, using bag-of-words of arguments to predict the adjacent arguments. Logeswaran and Lee (2018) propose the Quick Thoughts to exploit the closeness of adjacent arguments. They formulate the argument representation learning as a classification problem.
Previous work focuses on learning continuous argument representations with no interpretability. In this work, we study the discrete argument representations, capturing varying aspects in argumentation languages.
In this paper, we propose a novel task of interactive argument pair identification from two posts with opposite stances on a certain topic. We examine contexts of arguments and induce latent representations via discrete variational autoencoders. Experimental results on the dataset show that our model significantly outperforms the competitive baselines. Further analyses reveal why our model yields superior performance and prove the usefulness of discrete argument representations.
The future work will be carried out in two directions. First, we will study the usage of our model for applying to other dialogical argumentation related tasks, such as debate summarization. Second, we will utilize neural topic model for learning discrete argument representations to further improve the interpretability of representations.
This work is partially supported by National Natural Science Foundation of China (No.71991471), Science and Technology Commission of Shanghai Municipality Grant (No.20dz1200600). Jing Li is supported by CCF-Tencent Rhino-Bird Young Faculty Open Research Fund (R-ZDCJ), the Hong Kong Polytechnic University internal funds (1-BE2W and 1-ZVRH), and NSFC Young Scientists Fund 62006203.
Christa SC Asterhan and Baruch B Schwarz. 2007. The effects of monological and dialogical argumentation on concept learning in evolutionary theory. Journal of educational psychology, 99(3):626.
Yonatan Bilu and Noam Slonim. 2016. Claim synthesis via predicate recycling. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 525-530, Berlin, Germany. Association for Computational Linguistics.
Teresa Botschen, Daniil Sorokin, and Iryna Gurevych. 2018. Frame- and entity-based knowledge for common-sense argumentative reasoning. In Proceedings of the 5th Workshop on Argument Mining, ArgMining@EMNLP 2018, Brussels, Belgium, November 1, 2018, pages 90–96. Association for Computational Linguistics.
Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Jozefowicz, and Samy Bengio. 2016. Generating sentences from a continuous space. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany, August 11-12, 2016, pages 10-21. ACL.
Jean Carletta. 1996. Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2):249-254.
Rich Caruana, Steve Lawrence, and C. Lee Giles. 2000. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In Advances in Neural Information Processing Systems 13, Papers from Neural Information Processing Systems (NIPS) 2000, Denver, CO, USA, pages 402-408. MIT Press.
Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 670-680. Association for Computational Linguistics.
Fei Dong, Yue Zhang, and Jie Yang. 2017. Attention-based recurrent convolutional neural network for automatic essay scoring. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), Vancouver, Canada, August 3-4, 2017, pages 153-162. Association for Computational Linguistics.
Swapna Gottipati, Minghui Qiu, Yanchuan Sim, Jing Jiang, and Noah A. Smith. 2013. Learning topics and positions from debatingedia. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1858-1868. ACL.
Ivan Habernal and Iryna Gurevych. 2016. Which argument is more convincing? analyzing and predicting convincingness of web arguments using bidirectional LSTM. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1589-1599, Berlin, Germany. Association for Computational Linguistics.
Karl Moritz Hermann and Phil Blunsom. 2013. Multilingual distributed representations without word alignment. arXiv preprint arXiv:1312.6173.
Felix Hill, Kyunghyun Cho, and Anna Korhonen. 2016. Learning distributed representations of sentences from unlabelled data. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1367-1377. Association for Computational Linguistics.
Pei-yun Hsueh and Johanna D. Moore. 2007. Automatic decision detection in meeting speech. In Machine Learning for Multimodal Interaction, 4th International Workshop, MLMI 2007, Brno, Czech Republic, June 28-30, 2007, Revised Selected Papers, volume 4892 of Lecture Notes in Computer Science, pages 168-179. Springer.
Anthony Hunter. 2013. Analysis of dialogical argumentation via finite state machines. In International Conference on Scalable Uncertainty Management, pages 1-14. Springer.
Lu Ji, Zhongyu Wei, Xiangkun Hu, Yang Liu, Qi Zhang, and Xuanjing Huang. 2018. Incorporating argument-level interactions for persuasion comments evaluation using co-attention model. In Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018, pages 3703-3714. Association for Computational Linguistics.
Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Skip-thought vectors. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 3294-3302.
Jiangming Liu, Shay B. Cohen, and Mirella Lapata. 2018. Discourse representation structure parsing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 429-439. Association for Computational Linguistics.
Lajanugen Logeswaran and Honglak Lee. 2018. An efficient framework for learning sentence representations. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net.
Jiasen Lu, Anitha Kannan, Jianwei Yang, Devi Parikh, and Dhruv Batra. 2017. Best of both worlds: Transferring knowledge from discriminative learning to a generative visual dialog model. In Advances in Neural Information Processing Systems, pages 314-324.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pages 3111-3119.
Jonas Mueller and Aditya Thyagarajan. 2016. Siamese recurrent architectures for learning sentence similarity. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA, pages 2786-2792. AAAI Press.
Souneil Park, Kyung Soon Lee, and Junehwa Song. 2011. Contrasting opposing views of news articles on contentious issues. In *The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies*, Proceedings of the Conference, 19-24 June, 2011, Portland, Oregon, USA, pages 340-349. The Association for Computer Linguistics.
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1532-1543. ACL.
Isaac Persing and Vincent Ng. 2017. Why can't you convince me? modeling weaknesses in unpersuasive arguments. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, pages 4082-4088. ijcai.org.
Jason Tyler Rolfe. 2017. Discrete variational autoencoders. In 5th International Conference on Learning Representations, ICLR 2017, Toulouse, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
Nattapong Sanchan, Ahmet Aker, and Kalina Bontcheva. 2017. Automatic summarization of online debates. In Proceedings of the 1st Workshop on Natural Language Processing and Information Retrieval associated with RANLP 2017, pages 19-27. INCOMA Inc.
Dhanya Sridhar, James R. Foulds, Bert Huang, Lise Getoor, and Marilyn A. Walker. 2015. Joint models of disagreement and stance in online debate. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language
Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers, pages 116-125. The Association for Computer Linguistics.
Christian Stab and Iryna Gurevych. 2014. Identifying argumentative discourse structures in persuasive essays. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 46-56.
Kaveh Taghipour and Hwee Tou Ng. 2016. A neural approach to automated essay scoring. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 1882-1891. The Association for Computational Linguistics.
Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11 - 15, 2016, pages 613-624. ACM.
Lu Wang and Claire Cardie. 2014. A piece of my mind: A sentiment analysis approach for online dispute detection. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 693-699. Association for Computational Linguistics.
Nan Wang, Jin Wang, and Xuejie Zhang. 2018. Ynu-hpcc at semeval-2018 task 2: Multi-ensemble bi-gru model with attention mechanism for multilingual emoji prediction. In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 459-465. Association for Computational Linguistics.
Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. 2017a. Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 189-198. Association for Computational Linguistics.
Zhiguo Wang, Wael Hamza, and Radu Florian. 2017b. Bilateral multi-perspective matching for natural language sentences. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, pages 4144-4150. ijcai.org.
Zhongyu Wei, Yandi Xia, Chen Li, Yang Liu, Zachary Stallbohm, Yi Li, and Yang Jin. 2016. A preliminary study of disputation behavior in online debating forum. In Proceedings of the Third Workshop on Argument Mining (ArgMining2016), pages 166-171, Berlin, Germany. Association for Computational Linguistics.