A Generalizable Rhetorical Strategy Annotation Model Using LLM -based Debate Simulation and Labelling (2025)

Abstract: Rhetorical strategies are central to persuasive communication, from political discourse and marketing to legal argumentation. However, analysis of rhetorical strategies has been limited by reliance on human annotation, which is costly, inconsistent, difficult to scale. Their associated datasets are often limited to specific topics and strategies, posing challenges for robust model development. We propose a novel framework that leverages large language models (LLMs) to automatically generate and label synthetic debate data based on a four-part rhetorical typology (causal, empirical, emotional, moral). We fine-tune transformer-based classifiers on this LLM-labeled dataset and validate its performance against human-labeled data on this dataset and on multiple external corpora. Our model achieves high performance and strong generalization across topical domains. We illustrate two applications with the fine-tuned model: (1) the improvement in persuasiveness prediction from incorporating rhetorical strategy labels, and (2) analyzing temporal and partisan shifts in rhetorical strategies in U.S. Presidential debates (1960–2020), revealing increased use of affective over cognitive argument in U.S. Presidential debates.

## introduction persuasion is a core mechanism in social influence @xcite . it shapes how information is interpreted and acted upon across various domains, including marketing @xcite , online communication @xcite , and political campaigns @xcite . in the political sphere, persuasion has become increasingly consequential amid rising polarization, growing partisan animosity, and widening ideological divides, with implications for democratic processes, public policy, and the sorting of partisan identities. @xcite @xcite * both authors contributed equally to this research. persuasion involves rhetorical strategies that engage either cognitive and affective processes @xcite . cognitive arguments appeal to reason and evidence while affective arguments persuade by arousing emotional and moral reactions. these strategies are orthogonal to veracity. for example, empirical claims, even when fabricated, can lend credibility to misleading information (serrano-puche, 2021), while emotional and moral appeals can go viral across social networks @xcite and intensify affective polarization by provoking indignation and reinforcing group identities @xcite . the importance of rhetorical strategies in shaping consumer behavior, public discourse, and political polarization has attracted research utilizing datasets from online debates @xcite , charity appeals @xcite , and commercial advertisements @xcite . while prior studies offer valuable insights into persuasive techniques, the diversity of theoretical perspectives has led to inconsistent categorization of rhetorical strategies across human-annotated datasets. in addition, most existing datasets are focused on specific topical domains and rhetorical strategies, making it difficult to analyze the full range of persuasive techniques or generalize across domains @xcite . these datasets often lack principled topic control, which obscures the distinction between rhetorical and topic-driven effects and leads models to overfit to topic-specific patterns with limited generalizability @xcite . most importantly, the cognitive and motivational demands of human annotation have resulted in a paucity of large-scale, high-quality datasets, and low inter-rater agreement compromises the establishment of reliable ground truth labels @xcite . these challenges have limited the development of robust deep-learning classifiers for automated identification of persuasive techniques. to address these challenges, we propose a novel framework that trains classifiers to detect rhetorical strategies using synthetic debate data generated and labeled by large language models (llms) and guided by a rhetorical typology informed by social and psychological theories of persuasion. central to this framework is llm labeling using simulated personas to annotate persuasive discourse. this automated annotation process enhances the reliability and scalability of rhetorical detection. using this dataset, we trained a rhetorical classifier and validated the labels with human annotators. we then applied the classifier to analyze temporal trends in persuasive strategies in u. s. presidential debates from 1960 to 2020. our analysis reveals shifting rhetorical patterns, providing new insights into the evolving landscape of partisan political communication. while our dataset generation focuses on the political domain, the framework is easily adaptable to other domains with minimal modification. to sum up, our contribution is five-fold: 1) we present a fully-automated scalable framework for the generation and annotation of persuasive arguments that enhances the cross-context applicability of rhetorical labels. 2) we provide a highquality, topic-controlled dataset that has been validated by human annotators. 3) we develop models to detect rhetorical strategies across varied topics and domains, with validation from human annotations and evaluation on external datasets. 4) across five datasets from different domains, incorporating rhetorical labels into a fine-tuned bert model improves performance in predicting persuasive outcomes, both within and across diverse datasets. 5) we identify a significant increase in reliance on affective over cognitive strategies during u.s. presidential debates going back to 1960, which may reflect the increase in affective polarization among both voters and political elites. ## rhetorical strategies we use a rhetorical typology that integrates aristotle's classical framework with the dual-process distinction between cognitive and affective persuasion @xcite . reasoning with logic and evidence involves cognitive processes, while emotional and moral arguments are affective. studies based on aristotle's typology use logos inconsistently, sometimes referring to evidence and other times to logical reasoning @xcite @xcite . this can be especially problematic in annotation tasks. moreover, while logos refers to logical reasoning, the rules of formal logic are overly narrow and difficult to operationalize for annotation. we therefore separate reasoning and evidence into two distinct strategies and focus on causal reasoning, in which an argument points to the positive or negative consequences of an action or event @xcite . on the affective side, we distinguish between emotional and moral arguments. appeals to emotion have been identified across multiple domains @xcite @xcite and involve the expression of evocative language to arouse emotions in the target audience @xcite . evocative language can also include moral emotions such as compassion, harm, betrayal, and degradation @xcite @xcite . however, we classify these as moral persuasion, which is distinct from non-judgmental emotional appeals in that they refer to normative and ethical principles @xcite @xcite . ## methods using this typology, we developed a machine classifier for automated labeling of rhetorical strategies. our approach consists of the five steps illustrated in figure 1 : 1) identifying controversial political topics; 2) using llms to generate political debate dialogues; 3) prompting llms to annotate the generated dialogues; 4) fine-tuned a model for strategy classification; 5) applying the fine-tuned model to downstream analytical tasks. ## case studies of two applications our classifier's usefulness is demonstrated in two applications: 1) improving the performance of a model for predicting the persuasiveness of an argument, and 2) measuring temporal changes in rhetorical strategies in partisan political discourse. ## conclusion large-scale identification of rhetorical strategies has been hindered by the limitations of human annotation, including high cost, inconsistency, and limited scalability due to cognitive demands. to address this, we used a novel framework that leverages large language models to generate and annotate four rhetorical strategies in debate data. these synthetic labels are validated by simulated llm personas and human annotators, enabling the finetuning of a robust rhetorical classifier that generalizes across topical domains. we demonstrate its utility in two applications: improving persuasiveness prediction and revealing the rise in affective appeals and decline in empirical arguments in u.s. presidential debates from 1960 to 2020.