Cross-Domain Persuasion Detection with Argumentative Features (2025)

Abstract: The main challenge in cross-domain persuasion detection lies in the vast differences in vocabulary observed across different outlets and contexts. Superficially, an argument made on social media will look nothing like an opinion presented in the Supreme Court, but the latent factors that make an argument persuasive are common across all settings. Regardless of domain, persuasive arguments tend to use sound reasoning and present solid evidence, build on the credibility and authority of the source, or appeal to the emotions and beliefs of the audience. In this paper, we show that simply encoding the different argumentative components and their semantic types can significantly improve a language model’s ability to detect persuasion across vastly different domains.

## introduction persuasion is the process of guiding someone to adopt a particular way of thinking or behaving. it is a natural part of everyday life, helping people resolve small disagreements and find common ground. at the same time, it is also a powerful tool used by leaders and institutions to shape society's understanding of broader and more complex issues. different arguments have different levels of persuasiveness, often determined by specific syntactic and semantic markers @xcite . interestingly, this holds true regardless of the context in which arguments are made. in fig. 1 we present two examples from two different domains (reddit and supreme court proceedings) where the same argumentative strategy was used with the intent of persuading. in both cases, the writer introduced an interpretative claim and offered a premise that appealed to the emotions and beliefs of the audience (known as pathos). in this paper, we build on the idea that these strategies can be applied across domains and use them to detect persuasion in unseen language contexts. in this paper, we study persuasion detection in a cross-domain setting. prior work has looked at some aspects of cross-domain transfer in persuasion-related tasks, such as topic-agnostic persuasive dialogue generation @xcite and cross-lingual variation in persuasive language @xcite . in the former work, although topics varied, all the conversations followed the same general structure and style; a turn-taking discussion about daily-life situations. in the latter work, although the language varied, all instances were taken from the same media platform. in contrast, our work focuses on exploring whether explicitly modeling argumentation components and their modes of persuasion can improve cross-domain transfer when domains differ along more than one dimension. we build on the premise that argumentation strategies are somewhat domain-agnostic and look at three domains with different purposes, audiences, structures, and argumentation styles. to this end, we make the following contributions. 1. we propose a simple framework for introducing information about the structure of the argument and the persuasion mode for the persuasion detection task. 2. we design a challenging cross-domain experiment using three domains that differ significantly in language use, argument length, and argumentation style: social media, legal proceedings, and formal debates on general topics. 3. we show that regardless of this variation, argument information can significantly improve the ability of fine-tuned language models to detect persuasion across domains. ## related work persuasion detection. there are two main lines of work in the space of persuasion detection: one that frames the problem as a classification task in which the goal is to identify if a text instance is (more or less) persuasive @xcite @xcite , and one that is concerned with identifying the specific persuasion techniques employed in the text @xcite @xcite @xcite . most of these studies employ fully supervised approaches with training data from the target domain. a notable exception is the framework proposed by @xcite , which forgoes supervision by relying on syntactic parse trees to identify persuasion tactics. in contrast, we rely on simple signals from the argumentation structure to achieve crossdomain transfer. argumentation mining. there is a large body of work dealing with argumentation mining in the context of persuasive texts. some studies focus on extracting trees from long documents to represent the overall structure of the arguments made @xcite . earlier work has also used argument components to improve high-level persuasion detection @xcite , although in single-domain scenarios. @xcite combines these two areas of work by first extracting structured arguments using rhetorical structure theory and then infusing this information into a language model to identify persuasive online discussions. we follow a similar idea but considerably simplify the way in which we model the argumentation structure. ## methodology in this section, we present the argumentation taxonomy and datasets used, as well as our approach to predict argumentation components and persuasion. ## experimental results we describe our experiments using the approach outlined in sec. 3 to predict persuasion across domains, that is, when training on one dataset and predicting on a different dataset. we also evaluate the performance of our argumentation component and semantic type classifiers. in the cmv data set, this evaluation is based on the corpus of annotated argumentation tags, while in the scoa and ap datasets, it is based on a much smaller subset of 60 samples that were manually annotated solely for this evaluation. experimental settings. for the argumentation component and semantic type classifiers, we finetune distilbert @xcite , while for our cross-domain persuasion classifiers, we finetune both distilbert (results in appendix c) and roberta @xcite . each input is truncated to the maximum length allowed by bert (512). in all cases, we use 4-fold cross-validation. we test the following combinations: 1. segment labels: models were trained using only the argumentation component tags for each segment (argcomps), using both the argumentation component tags and the semantic type tags (semtypes), and using neither (baseline). 2. training data -each model is trained on cmv data or scoa data. randomly selected subsets of 7500 arguments from the cmv and scoa datasets are used, with an even split between the number of successful, unsuccessful, and neutral arguments. the evaluation results for the persuasion clas-sifier model are shown in nevertheless, the improvements offered on the cross-domain tasks are encouraging and indicate that this approach has potential. these results show that when a pre-trained transformer encounters types of arguments it has not seen before, such as in a different dataset with different style and vocabulary, the sequence tags do indeed help it identify similarities between training data and the testing data and thus make better predictions. we note that for different datasets, different levels of argumentation information prove useful, with the argumentation component sometimes being sufficient to see a gain, and with the semantic types being required in other cases. additionally, we observe that the performance of the classifiers with argumentative features on the ap dataset is markedly higher despite the absence of "neutral" text segments, which are present in the training data. this further suggests that our method does indeed generalize. we have chosen to use plain distil-bert/roberta, without argumentative features, as our baseline. our goal is to explore whether incorporating explicit argumentative features in a model improves its performance, which we have shown that it does, rather than obtaining state-of-the-art results for persuasion detection. to accurately gauge whether our hypothesis holds for other persuasion detection methods, we would potentially have to explore different injection strategies. for completeness, we also include a simple llm baseline. llm baselines. the transformer-based results are comparable to two-shot llm baselines: simple prompts passed to a llama3.1 model, with and without argumentation information. these are visible in tab. 3 . training with target data. we also performed further experiments to analyze how the performance of these models changed as they saw more training data from the target domain, presented in figure 4 . it is apparent that persuasion detection is a challenging task; all models eventually plateau in performance and are unable to cross a certain threshold. ## conclusion and future work while the proposed approach is simple, our findings represent a solid proof of concept that adding inductive bias in the form of argumentative structures and modes of persuasion significantly im-proves cross-domain persuasion detection, even when this information is noisy. further, these findings hold for domains with substantial differences in purpose, vocabulary, text length, and style. in future work, using a more comprehensive scheme to represent arguments, such as walton's argumentation scheme, could potentially provide a richer representation of the latent structure of the argument. additionally, other ways of incorporating this information could be explored, such as by cross-attending transformer-based representations of text and graphical networks that model relations between argument components explicitly @xcite , or by combining language model inferences with probabilistic logical inference @xcite . while certain argumentative strategies have been proven to be more effective in certain situations or with certain people (wang et al., 2019), we have shown the impact of including some underlying indicators that are useful to gauge the persuasiveness of an argument in different domains. these can be further improved by including user-or contextspecific information.