Style Analysis of Argumentative Texts by Mining Rhetorical Devices (2020)

Abstract: Using the appropriate style is key for writing a high-quality text. Reliable computational style analysis is hence essential for the automation of nearly all kinds of text synthesis tasks. Research on style analysis focuses on recognition problems such as authorship identification; the respective technology (e.g., n-gram distribution divergence quantification) showed to be effective for discrimination, but inappropriate for text synthesis since the “essence of a style” remains implicit. This paper contributes right here: it studies the automatic analysis of style at the knowledge-level based on rhetorical devices. To this end, we developed and evaluated a grammar-based approach for identifying 26 syntax-based devices. Then, we employed that approach to distinguish various patterns of style in selected sets of argumentative articles and presidential debates. The patterns reveal several insights into the style used there, while being adequate for integration in text synthesis systems.

## introduction the decision for an adequate writing style plays a crucial role for an author who wants to achieve a particular goal, such as persuading the readers @xcite . "style" is an elusive concept which covers a wide range of techniques an author can follow, including justifying a conclusion by anecdotal evidence, using regular repetition of the same phrase, or raising questions and then answering them. in the literature on the subject, these techniques are called rhetorical devices @xcite . the automatic analysis of style has been addressed mostly by developing a set of style features (aka style indicators) such as the percentage of function words @xcite . those features have proven to be effective in various analysis tasks, such as genre classification and author recognition. however, they are not appropriate for typical text synthesis and writing assistance tasks, since they cannot reveal the "essence of a style" in an explicit and describable manner. by contrast, analyzing the writing style based on rhetorical devices provides a mechanism to describe where, what, and how specific techniques are used. this kind of analysis is not only important for exploring content in social science @xcite , but it can also serve text synthesis systems by improving the quality of automatically generated texts @xcite . moreover, it can form the backbone of style suggestion tools. for example, when writing a text for which the desired specification (e.g., the genre) is given, adequate style techniques can be suggested to improve the text quality. in such a manner, new writers can learn to improve their texts and approach the quality of masterpieces written by top writers. figure 1 illustrates the described connections. rhetoric has been the subject of investigation amongst scholars since the time of ancient greece. meanwhile, a considerable number of rhetorical devices were developed and discussed in the literature. the most well-known collected lists of devices contain more than 500 devices @xcite . though various of them, such as irony and sarcasm, is hard to be computationally identified @xcite , there is still a sufficiently large portion of popular and-for our purpose-highly useful devices whose identification can be tackled with the current state of the art. basically, rhetorical devices can be categorized according to different principles, where an important one is a linguistic level (lexical, syntactic, semantic, and pragmatic). for the time being, we will deal with syntax-based devices. against the above background, this paper addresses three research questions. (1) how to identify syntax-based rhetorical devices in a text? (2) what are the most common patterns of using these devices? (3) to which degree differ these patterns across different monological and dialogical argumentative texts? within and across the texts' genres, topics, and authors? and across different opponent debaters? to answer these questions, we develop a grammar-based approach for the identification of 26 rhetorical devices. the grammars are built on top of the outputs of a probabilistic context-free grammar parser, pcfg. for evaluation purpose, we create a corpus of 1718 texts which are labelled for rhetorical devices. the evaluation results show that our approach is able to identify the devices with an average of 0.70 in terms of f 1 . based on the developed approach, we quantify and discuss the usage of devices in monological texts within and across different genres, topics, and authors using a subset of the new york times annotated corpus @xcite . we also analyze the devices usage patterns in dialogical texts using a set of presidential debates from the american presidency project @xcite . we consider the gained qualitative and quantitative insights about the usage of rhetorical devices as step forward to a new generation of semi-automated argumentative text generation and writing tools. all developed resources in this paper are made publicly available at www.webis.de ## related work recently, investigating rhetorical devices for style analysis has been considered in computational linguistics. various devices at the semantic and pragmatic levels have been addressed singly such as irony (e.g., (c. @xcite ), sarcasm (e.g., @xcite ), evidence (e.g., @xcite ), and means of persuasion (e.g., @xcite ). in a notable work, @xcite work on identifying 'epanaphora'. they try to distinguish between accidental and intentional use of this device. other studies target identifying a mix of syntax, and semantic devices. @xcite addressed four rhetorical devices: 'anaphora', 'isocolon', 'epizeuxis', and 'oxymorons'. these devices were utilized to recognize the author of a set of documents. @xcite identified the four devices mentioned above in addition to nine new devices belonging to parallelism, repetition, and trope. the primary purpose of that work is to use the presence of a rhetorical device as a feature in machine learning models for authorship attribution. since the authors consider syntax-based devices, we already considered five of their devices in our study. regarding argumentaion, @xcite analyzed eight devices, six belong to the syntax and lexical levels, and two to the trope (i.e., semantic or pragmatic). mainly, a pilot study was conducted to study the relation between argumentation structure and the identified devices. few resources for rhetorical devices are publicly free. up to our knowledge, the code of the previous studies is not available anywhere on the web. hence, researchers often have to write a new piece of code every time they need to analyze style based on rhetorical devices. this paper resolves this problem considerably by providing a tool for identifying 26 different rhetorical devices. our developed resources, including the code, will be made freely available. pcfg outputs have been employed for different tasks including response generation in dialogue @xcite , multiword expression identification @xcite , and the task at hand: identifying rhetorical devices @xcite . however, we develop a set of original heuristic rules that map the devices' definitions to pcfg grammars. as far as we know, many devices from the 26 we identified have not been considered in any other study. writing style analysis has been studied widely. the authorship recognition has been tackled in a large number of papers (e.g., @xcite ). besides, quality assessment research has involved applying several style analysis features (e.g., @xcite ). in comparison to our analysis, we conducted a controlled analysis using 'matching' technique and we covered various aspects of monological and dialogical texts such as genre, topic, author, and debate opponent. ## identification of rhetorical devices rhetorical devices are the techniques of using the language to produce an effect on the target audience or readers @xcite . for example, repeating particular phrases can produce effects such as emphasizing a certain argument, or evoking a specific emotion @xcite . this paper targets syntax-based rhetorical devices. particularly, we aim to identify 26 devices belonging to two main categories: (1) figurative syntax, which is referred as schemes in literature, and in the next subsections, we detail the figurative and ordinary syntax devices and describe our approach for identifying them. ## analysis of rhetorical devices we rely on our identification approach to analyze the usage patterns of rhetorical devices in argumentative newspaper articles and presidential debates. first, we describe the acquisition and sampling of the analysis datasets. then, we discuss the distribution of rhetorical devices there along with different article and debate aspects. the computed distributions illustrate various patterns of rhetorical devices and lead to several interesting insights. analysis datasets: to conduct insightful analysis, we constructed two datasets for newspaper articles and presidential debates. to preserve the balance between the subsets, we consider three instances for each propriety, i.e., the 'topic' subset includes 114 articles belong to science, education, and art. the 'genre' subset includes 89 articles belong to biography, editorial, and review. finally, the 'authors' subset includes 159 articles written by martin, lewis, and hevesi. analysis method: basically, we applied our identification approach (see section 3) to the analysis datasets. in particular, a classifier for each device is applied to the articles or debates turns, resulting in the frequency of the device there. however, since our identification approach is not perfect, it is crucial to account for its errors. hence, we followed the method used in @xcite : for the frequency n of a rhetorical device rd in an instance i in a dataset. we computed a confidence interval for n, where the @xmath0, and the @xmath1. ultimately, the mean of the upper and lower bounds is the new frequency, which is normalized by the number of sentences in the articles/debate-turns belong to i. accordingly, we computed the distributions of rhetorical devices in the analysis datasets and their subsets. the chi-squared test with 0.01 significant level is used to check whether the difference in the usage of rhetorical devices in the datasets and across their instances is significant, and the cramer's v test is used to measure the effect-size of the distributions' difference. analysis results: figure 3 shows the distribution of rhetorical devices among the three authors (a), the three genres (b), the three topics (c), and the debate subsets (d). as expected, the style in newspaper articles (monologue) are significantly different than in debates (dialogue). some analysis results for each of the datasets are as follows. ## conclusion and future work writing style analysis has become a mature discipline, but it is mostly tackled from the recognition perspective. i.e., it can give strong classification results that, because of their intrinsic nature, cannot be transferred to constrained text generation or computational writing assistance. we address this shortcoming by proposing an approach for the explicit encoding and identification of rhetorical devices. in carefully designed experiments, we study the usage of these devices in different argumentative articles and presidential debates. the distributions show different patterns of style among three text's properties and provide new insights regarding style usage within the studied topics. the achieved f 1 classification performance (0.70) can be considered as very good for concrete multi-class classification setting; it shows that the applied approach has the potential to find its way into real-world argumentative text synthesis tools. we plan in the future to improve our grammars to minimize mistakes and increase the number of devices considering the inversion type of figurative syntax devices.