Are You Trying to Convince Me or Are You Trying to Deceive Me? Using Argumentation Types to Identify Deceptive News (2025)

Abstract: The way we relay factual information and the way we present deceptive information as truth differs from the perspective of argumentation. In this paper, we explore whether these differences can be exploited to detect deceptive political news in English. We do this by training a model to detect different kinds of argumentation in online news text. We use sentence embeddings extracted from an argumentation type classification model as features for a deceptive news classifier. This deceptive news classification model leverages the sequence of argumentation types within an article to determine whether it is credible or deceptive. Our approach outperforms other state-of-the-art models while having lower variance. Finally, we use the output of our argumentation model to analyze the differences between credible and deceptive news based on the distribution of argumentation types across the articles. Results of this analysis indicate that credible political news presents statements supported by a variety of argumentation types, while deceptive news relies on anecdotes and testimonial.

## introduction the spread of disinformation has taken a toll on public trust in news media @xcite . the effects of this are reflected in social and political unrest, such as the 2016, 2020, and 2024 united states presidential elections (see @xcite @xcite and the covid-19 pandemic @xcite . there is a widespread perception that journalists have not only failed to shield the public from disinformation, but have also contributed to its spread by aligning themselves with bad actors @xcite . on top of that, there is a belief that news media prioritizes profit over veracity, treating it as some sort of advertisement @xcite . although disinformation takes many forms, we focus on deception based on the definitions of news media watchdog organizations, such as media bias/fact check. 1 these are often determined based on political bias and on the amount of false information, be it of the articles themselves or of the outlets that publish them. we focus on political news, as it has become a loci of public concerns over the role that news media plays in global politics and the influence disinformation has in it @xcite . political persuasion and disinformation are closely related @xcite . we assume that credible news aims to inform, while deceptive news attempts to persuade readers in favour of a certain viewpoint. we hypothesize that this will be reflected in the argumentation within the articles themselves. our research questions are as follows: rq1: can argumentation features be used to detect deceptive news? rq2: what insights can we acquire by comparing argumentation types in credible and deceptive news? we implement a two-step approach to test this. we start by training a bert @xcite model to identify argumentation types in english news articles. we extract argumentation features from this model and feed them to a bi-lstm @xcite to identify deceptive news. we go into more detail of our architecture and related design choices in section 3. we report the results of our experiments in section 6. our approach outperforms other models from the literature, having less variance compared to the other non-deterministic methods used in our experiments. we also show that feature-based models can outperform simple transformer baselines. we do an analysis of the argumentation types between credible and deceptive news in section 7. we show that deceptive news tends to present more anecdotes and testimonies, while credible news tends to have more assumptions supported through evidence. ## our approach as we want to analyze whether the argumentative structure of an article can be used to identify deception, we perform a two-step process inspired by @xcite . this allows us to determine whether our model learns from argumentation in the text and provides us with information about the types of argumentation in news articles, which we analyze in section 7. we use argumentation types instead of argumentation roles (such as premise and conclusion), as @xcite note that the latter encode the strategy an author uses to persuade readers. we split the articles into an ordered set of sentences and assign them an argumentation type with bert. we use four argumentation types: anecdote, assumption, testimony, and other. we explain each model and how they are implemented in more detail throughout the rest of this section. the specific hyper-parameters used for our experiments can be found in appendix a. argumentation type classifier: we fine-tune a bert model foot_6 on the argumentation type dataset. this model is shown individual sentences and must assign an argumentation type to each of them. we use the output of the [cls] token from the final transformer layer for classification. as the latter bert layers typically learn task-specific features @xcite , we expect the final layer to encode argumentation-related features for the whole sentence. this model is then frozen for the rest of the experiments to prevent its weights from changing later on, thus making sure that it retains its knowledge about argumentation types intact. deceptive news classifier: given a news article, we split it into sentences. these sentences are passed through the now-frozen argumentation type classifier. we use the output of the final transformer layer corresponding to the [cls] token as a sentence embedding. these embeddings are then fed to a bi-lstm model to determine whether the article is credible or deceptive. ## datasets in this section, we describe the different datasets used in our two tasks. for the argumentation type classification task, we use the webis-editorials-16 dataset (section 4.1). for the deceptive news detection task, we use two datasets: one with articlelevel annotations (section 4.2) and another with source-level annotations (section 4.3). all three datasets contain news articles in english collected prior to 2020. although the landscape of deceptive news and misinformation is likely to have changed since these articles were originally published, these datasets are still valuable as they only contain human-generated news. machine-generated mis-and disinformation is very different from that generated by humans @xcite and detecting it is another task in and of itself @xcite . therefore, we choose established datasets from before content produced by generative language models flooded the web. ## baselines we compare our argumentation type classifier against a random classifier and a majority class baseline. this is done to ensure the model is actually learning from the data and not simply assigning labels arbitrarily. we focus on both general performance and performance on the lowest scoring label. we compare our approach to three models: a bert classifier, an svm using liwc foot_11 features, and fakeflow @xcite . we choose bert as it has been shown to perform well for a variety of tasks and is simple to implement. classical machine learning models using liwc features have been used successfully for deceptive news detection in the past (e.g. @xcite . we follow the implementation of @xcite , using an svm classifier and the same feature selection process. the final model used for comparison is fakeflow, which uses a cnn @xcite to model article topics and a bi-gru @xcite to model emotions in the text. ## results and discussion throughout this section we present the quantitative results of both the argumentation type and deceptive news classification tasks. appendix d contains additional tables with more detailed results from our experiments. each experiment was run multiple times in order to assess not only the performance of the models, but also their variance across runs. only the random seed was changed across runs, all other hyperparameters remained the same. we performed five runs for each argumentation type classification model. the deceptive news models were also trained an additional five times for each, resulting in a total of 25 runs. this allows us to assess the variance of the deceptive news model not only in terms of the training process but also of the representations it was fed. argumentation type classifier: our classifier achieves an average weighted f1 score of 0.84, which is significantly higher than those of the random and majority baselines (0.56 and 0.61, respectively). it is important to verify the f1 score of the worst-performing label at this step. this is used in analyzing the argumentation types of news articles (see section 7) and is the motivation for collapsing some of the labels into a single one (see section 4.1 and appendix b). the lowest f1 score observed is 0.47 for the other class, which is a large improvement over 0.04 for the same class using the random baseline. deceptive news classifier: as shown in figure 2 , using argumentation features outperforms the other models. there is a noticeable improvement over bert and the svm with liwc features for both datasets. we notice different patterns when comparing our model with fakeflow. for the politifact dataset, both models show an overlap in performance. however, the three lower quartiles for fakeflow are lower than the higher three ones for the model with argumentation features, meaning that the latter performs better on average. when looking at the fakenews-2018 dataset, we notice that fakeflow performs much worse than on any of the other models. previous research has shown that features of deceptive news can be topic dependent, which may explain why some models under-perform on specific deception detection datasets @xcite . overall, our approach showed a lower variance in its performance when compared to the other statistical models (i.e. bert and fakeflow). this indicates it is more stable and less prone to the effects of randomness, such as the chosen random seed. ## how do people argue in deceptive given an argumentation type label and an article, we look at the fraction of sentences within the article with that label. we then compare how these values are distributed in each deceptive news dataset (see figure 3 ). to ensure balanced sample sizes for the analysis, we under-sample the most represented class for each dataset. we use a two-tailed kolmogorov-smirnov test @xcite to determine whether the distributions for credible and deceptive news are different 13 and, if so, how much they differ. it is important to note that all four distributions are related to each other as the ratios for a given article must sum up to one. 14 thus, we must apply a bonferroni correction for @xmath0. that means that we need a p-value of 0.0125 instead of 0.05 to be able to re13 given that we are dealing with distributions of ratios, we can safely assume that they are not normally distributed. we will go over the differences between the distributions of the four labels, focusing on the fakenews-2018 dataset as we consider that these distributions can give us potentially interesting insights. when looking at the distribution of the anecdote argumentation type, we notice that anecdotes appear more often in articles labelled fake in the poli-tifact dataset. usage of anecdotes may be a strategy used by deceptive news outlets to strengthen arguments in lieu of factual evidence. previous literature has also noted that more persuasive articles use logical fallacies, such as arguments from anecdote, which leverage readers' emotions @xcite . meanwhile the komogorov-smirnov statistic shows that the difference on the fakenews-2018 dataset is small and not statistically significant. in general, the label assumption is the most evenly distributed across the articles, regardless of the dataset or whether they are deceptive or not. as we can see in figure 3 , assumptions are less represented in deceptive news in the fakenews-2018 dataset. the label assumption appears more often in articles, regardless of whether they are deceptive or not. in contrast, the other labels tend to represent a small proportion of the sentences of an article. this does not mean that there are no differences between credible and deceptive news, as assumptions are less represented in deceptive news in the fakenews-2018 dataset (as shown in figure 3 ). @xcite notes that the modern wave of disinformation stems partly from conspiracy theories. conspiracy theorists avoid making explicit assumptions to avoid accountability for their claims, using the excuse of "just asking questions" (egelhofer and lecheler, 2019). as mentioned previously, the difference between the distribution for credible and deceptive news is not statistically significant for the politifact dataset, likely due to the small size of the dataset. the testimony label is represented more in deceptive news for both datasets. figure 3 shows this for the fakenews-2018 dataset. this may be related to the use of news as a medium for political advertising @xcite . studies have shown that testimonials positively impact consumer bias and that consumers identify more strongly with testimonials from individuals they consider peers @xcite . it has also been observed that partisan loyalty has an effect on believability, as readers are more likely to report information from sources that share their political affiliation as factual @xcite . on the other hand, this could also be due to deceptive news using fallacious strategies such as appealing to authority @xcite . the other label is represented the least in both datasets and both types of news, but appears more in credible news articles than in deceptive ones. it is important to note that the other label contains the statistics and ground-truth labels from the original webis-16 dataset (as noted in section 4.1). this suggests that credible news substantiates claims more often than deceptive. as mentioned previously in this section, figure 3 shows the distributions for the labels assumption and testimony in the fakenews-2018 dataset. the histograms comparing the distributions for all the argumentation type labels can be found in ap-pendix e. ## conclusions factuality in news media is closely related to similar phenomena, such as partisan bias, propaganda, and satire @xcite . the rapid spread of deceptive news and misinformation has been linked to instability in the global political climate, as well as erosion of trust in news media @xcite . @xcite and @xcite argue that it is important to study these complex phenomena in order to mitigate the risks and consequences they engender. in this paper we hypothesized that argumentation in credible and deceptive political news articles would differ as a reflection of their role as informers or vectors for ideology. we proposed an approach exploiting argumentation types of sentences within an article to detect deceptive news. on average, our approach outperformed three models from the existing literature, namely bert, an svm with liwc features, and fakeflow. it also shows a lower variance than the non-deterministic baselines. some interesting patterns appear when analyzing the distributions of argumentation types. we found that deceptive articles tend to use more testimonies and, for one of the datasets, more anecdotes. although credible news tend to have more assumptions, they appear to support them with evidence or by establishing shared knowledge. this matches previous findings from the literature that point out that deceptive news uses logical fallacies, such as overusing anecdotes or by appealing to authority @xcite . it is important to note that the work we present in this paper is not any sort of "truth detector". our model was trained and tested to be used in news articles and should only be used for that kind of media. the datasets have binary truth annotations and were curated with that purpose in mind. this means that things living in the in-between of truth and falsehood might potentially be misrepresented. moreover, there are different kinds of mis-and disinformation (such as propaganda or hyper-partisan news) that are not explicitly studied in the present paper to better isolate features pertaining deceptive news. the results of this study show that stylistic features, such as argumentation type, can improve classification performance and enrich our under-standing of complex phenomena such as deceptive news and misinformation. not only that, but they can also help develop systems that are both more interpretable and perform as well as other classification systems, if not better. it is also important to note that we focus on the style of the text rather than on its content. one of our assumptions is that outlets publishing deceptive content online do so knowingly. this ignores the possibility that people who write deceptive news articles legitimately believe what they are writing. it also ignores propaganda in news media that is often regarded as trustworthy, be it backed by the state and/or by for-profit organizations.