Towards Fine-Grained Argumentation Strategy Analysis in Persuasive Essays (2023)

Abstract: We define an argumentation strategy as the set of rhetorical and stylistic means that authors employ to produce an effective, and often persuasive, text. First computational accounts of such strategies have been relatively coarse-grained, while in our work we aim to move to a more detailed analysis. We extend the annotations of the Argument Annotated Essays corpus (Stab and Gurevych, 2017) with specific types of claims and premises, propose a model for their automatic identification and show first results, and then we discuss usage patterns that emerge with respect to the essay structure, the “flows” of argument component types, the claim-premise constellations, the role of the essay prompt type, and that of the individual author.

## introduction the field of argument mining (am), which has grown into a productive area of research during the last decade @xcite , focuses on the tasks of automatic identification and extraction of argumentation in natural language. this includes the detection of argument components, like claims @xcite and premises @xcite , and the relations between them @xcite . research has been conducted on different text domains ranging from more edited texts, e.g. editorials @xcite or wikipedia texts @xcite , to social media, e.g. change my view @xcite or twitter @xcite . a so far relatively understudied area of interest is the identification of argumentation strategies, i.e., the decisions that authors make on linearizing their argumentation and on marking it with linguistic expressions for persuasive effect @xcite . effectiveness, which can be described as one dimension of argumenta-tion quality @xcite , depends (inter alia) on using the "right" arguments for the audience, their arrangement, and their linguistic formulation. this is also the case for persuasive essays, which already have been extensively used in am research @xcite , but to the best of our knowledge not much for modeling underlying strategies. to contribute to filling this gap we utilize our own claim and premise type annotations to extract semantic "flow patterns" from the argument annotated essays (aae) corpus @xcite . we argue that these types and flow patterns are finegrained and informative to shed more light on the strategies authors of persuasive essays apply to structure their texts. the contributions of this paper are: 1) we provide a dataset with claim and premise type annotations for the full aae corpus (sct. 3) by revising and extending the prior work of @xcite ; 2) we trained classification models on our annotations and present first promising results (sct. 4); 3) we contribute to argumentation strategy modeling by (i) extracting flow patterns of the argument component types, also in relation to the essay structure (roles of different paragraphs), (ii) examining the patterns of claim and supporting premise w.r.t. their types, and (iii) looking into the influences of essay prompt as well as the individual author of the text (sct. 5). ## related work argument mining in essays. @xcite presented the first edition of the aae corpus, which consisted of 90 persuasive essays annotated for argument components and relations. later, it was extended to 402 essays @xcite . this corpus has been repeatedly used for component detection @xcite and as a starting point for component type ## corpus & annotation in this section, we briefly describe the corpus we use, i.e. the aae corpus @xcite . in addition, we present our annotation scheme, the procedure, and results. examples type 1) [...] we should attach more importance to cooperation during primary education. p 2) [...] keeping the cultural traditions in the destination countries is tremendous important. v 3) [...] teachers teach us knowledge but also the skills to tell right from wrong. f 4) frank zappa once said, "mind is like a parachute, it doesn't work if its not open" t 5) the waste products and harmful gases produced by these factories s cause a significant amount of air pollution. 6) [...] if there are no animals in the world, the balance of nature will broke down, hi and we, human, will die out as well. 7) [...] tourism makes up one-third of the czech republic's economy. re 8) nowadays, time is the most valuable thing in life with increased pace. cg ## classification of semantic adu types we fine-tune a pre-trained language model, the roberta-base architecture @xcite , to predict semantic types. as input we use solely the argument component span, without further context. see appendix a for details on hyper-parameters. our complete classification results are also pro-vided there; in the following, we mention the main points. we train the semantic type classifiers separately for the different adu types (major claim, claim, premise), and in addition with the variant of combining the two claim types (major claim and claim). per run, the data is randomly divided into train/dev/test with proportions 80/10/10. the results that we report are averaged over 10 runs. ## pattern extraction and analysis argumentative essays have a very specific structure, as described in section 3. following previous works on argument component types @xcite and argumentation strategies @xcite , we hypothesize finding similar patterns of semantic argument types in the essays. first, we linearize the semantic types and order them by their textual positions. in figure 1 , for example, the essay starts with a value-based major claim in the first paragraph, followed by a value claim and a premise with semantic type hypothetical-instance. the full sequence of seman-tic types is referred to as the semantic flow of the essay. we further abstract over semantic repetitions, thus resulting in so-called semantic change flows. for example, in the previous flow, multiple consecutive hypothetical-instances are reduced to a single occurrence. this abstraction leads to more reliable/general patterns @xcite . similar to @xcite , we use the natural structure of argumentative essays and split them into paragraphs, as individual arguments are fully contained in single paragraphs. this reduces the length of extracted patterns and their variance. additionally, we also study differences in the semantic change flows of component types. for claim change flows, besides paragraphs we study their change flow on full documents, too. as claims should only relate to the major claim, we assume document-level change flows should summarize the global structure of an essay quite well. for premises, we restrict our study to the paragraph level, as premises should not be connected to the premises of other paragraphs. argument components show different distributions across paragraphs, with major claims only appearing in the first and last, and premises predominantly being used in the middle paragraphs. this has an effect on the semantic flows. see in combination with other types, the latter of which most often starts with common-ground. the most prominent change flow consisting of three types is (cg, hi, cg). a combination of the two example types hypothetical-instance and real-example is not among the most frequent change flows. statistics most often co-occurs with common-ground. finally, the claim and premise change flows by paragraph (see ## discussion our change flow analysis reveals several frequently occurring patterns. to begin with, an essay usually starts with a major claim (most frequently of type value or policy) which is sometimes followed by a claim. the final paragraph, however, shows more flexibility regarding the ordering of both claim variants, which shows that some authors choose to end with a major claim, i.e., the essay's central claim. moreover, middle paragraphs either contain a single claim (a single argument) or several claims of the same type, which may show an author's tendency to not switch between claim types within a paragraph. then, while both major claims and claims are most frequently of type value, we found a notable difference in the usage of policy and fact types. while policy more often occurs in major claim flows, i.e., in the first and last paragraphs, fact is more prominently applied as a claim type in the middle paragraphs. thus, an essay's central claim is more often arguing towards some action being taken, while the argumentation unfolding in the essay's body more often focuses on the question if a target is true or not. regarding the usage of premise types we observe the frequent pattern of flows starting with common-ground and ending with a different type, or, alternatively, of common-ground framing another type. hence, authors tend to begin their flow of premises with a general statement, followed, e.g., by an example. less often, common-ground is applied to end a flow, while being rarely used in-between types. this may be indicative of a strategy to begin (and end) with a general observation while more concrete statements are placed in-between. in this work, we explore the effect of two potential factors on the constellation of claim types, prompt type and author, and their potential interaction. our prompt type analysis provides evidence that the prompt's phrasing has indeed an effect on the usage of claim types, as all prompt types elicit essays with a higher proportion of the respective claim type. thus, authors adapt their argumentation strategy to the task at hand. we also show that authors exhibit a substantial variance in their usage of claim types, which is further dependent on the essay's prompt type. we argue that this is indicative of the task's role in choosing the most efficient argumentation strategy. ## conclusion in this work, we analyzed patterns of claim and premise types in persuasive essays to shed light on underlying argumentation strategies. we extended the annotations of the aae corpus with a layer of semantic types, which we used for automatic type classification, pattern extraction both on the level of change flows and argument relations, and the analysis of prompt and author effects on argumentation strategies. we show that semantic types of argument components are an appropriately fine-grained level of analysis to investigate argumentation strategies. several common patterns of semantic type flows could be identified. furthermore, we provide evidence for the effect of author and, especially, prompt type on the adoption of argumentation strategies. in the future, we would like to extend our scope of analysis. further research can include the relation between prompt type and semantic flows and the effect of prompt type on the usage of premise types. we are also interested in investigating the effect of semantic flows on essay quality. finally, we want to apply our analysis to other corpora, both in-domain (the icle dataset @xcite ) and out-of-domain (e.g., the subreddit change my view).