Detecting Argumentative Fallacies in the Wild: Problems and Limitations of Large Language Models (2023)

Abstract: Previous work on the automatic identification of fallacies in natural language text has typically approached the problem in constrained experimental setups that make it difficult to understand the applicability and usefulness of the proposals in the real world. In this paper, we present the first analysis of the limitations that these data-driven approaches could show in real situations. For that purpose, we first create a validation corpus consisting of natural language argumentation schemes. Second, we provide new empirical results to the emerging task of identifying fallacies in natural language text. Third, we analyse the errors observed outside of the testing data domains considering the new validation corpus. Finally, we point out some important limitations observed in our analysis that should be taken into account in future research in this topic. Specifically, if we want to deploy these systems in the Wild.

## introduction in the field of the automatic analysis of natural language argumentative discourse, the identification of fallacies plays an important role since it can be a determining feature to measure the quality of argumentation @xcite . furthermore, the automatic identification of fallacies can also be helpful for the development of disinformation detection systems and critical thinking tools @xcite . studied since the times of the ancient greece by aristotle @xcite , a fallacy was seen as an argumentation strategy used to deceive an opponent in a debate and unfairly get the reason. this definition evolved with time @xcite extending the instrumental notion of the aristotelian fallacy to more modern theories of logic and mathematics. a more recent (and complete) definition was provided by @xcite , where fallacies are defined as "important, baptizable types of errors or deceptive tactics of argumentation that tend to fool or trip up participants in argumentation in various kinds of everyday discussions". this definition is less constrained and more accurate to the natural language challenges we may face these days. detecting a piece of fallacious reasoning, however, is not trivial and requires knowledge in a broad number of areas that make this task challenging. first, it is important to be able to analyse the logical reasoning underlying natural language arguments. for that purpose, it is required to distil the abstract and formal components from the informal natural language argument. this first case is that of formal fallacies @xcite . second, solid knowledge on the domain of discussion is of utmost importance. an argument can be logically sound but still fallacious, such is the case of informal fallacies @xcite . therefore, only with a complete analysis is it possible to determine if a natural language argument is a fallacy or not, as well as the underlying reasons why it is fallacious. a consistent way to conduct this analysis is to rely on validated models of argument which capture the notion of fallacy. different models have been proposed and studied in the literature; such is the case of the pragma-dialectic theory of argumentation @xcite in which the authors define ten rules to guide argumentative discussions. the fulfilment of these rules allows to create a fruitful discussion, but an argument that breaks any of these rules is considered a fallacy. another good example of these models is the argumentation schemes proposed by @xcite . an argumentation scheme combines the abstract representation of the underlying logic of a natural language argument with a set of critical questions that must be successfully answered to prove the validity of an argument. the argumentation scheme model is very interesting w.r.t. fallacy analysis, since an argument being fallacious is not determined by belonging to a specific class, but depending on the answers provided to the set of critical questions. for example, a natural language argument belonging to the ad hominem scheme is not a fallacy per se, but it must be structured as follows: character attack premise: a is a person of bad character. conclusion: a's argument α should not be accepted. and it is only considered to be fallacious if any of the following critical questions cannot be successfully answered, therefore, with the argumentation scheme paradigm, it is possible to partially dissociate the natural language and the logic of the argument, allowing for a more informed analysis of the reasons of an argument being fallacious. in this paper, we integrate the concept of argumentation schemes in the evaluation of machine learning and transformer-based language models for the automatic detection of fallacies in natural language arguments. it is our objective to understand the way these models, as they have been proposed in most of the previous work in this topic, are able to learn the reasons behind a fallacy and generalise to data outside of the training domain. our contribution is therefore threefold: (i) we create a fallacy validation corpus consisting of natural language argumentation schemes; (ii) we provide new empirical results for the emerging task of identifying fallacies in natural language text; and (iii) we analyse the observed errors inside and outside of the testing data domains considering the argumentation scheme validation corpus, and point out some of the main limitations of relying exclusively on llms when addressing complex natural language reasoning problems. ## related work the automatic detection of fallacies in natural language texts is an emerging topic of research within the area of natural language processing. one of the first efforts in developing a database of fallacies was done in @xcite creating "argotario", an educative platform where participants could improve their debating skills. through gamification, the authors collected fallacies registered by the participants belonging to one of the following five classes: ad hominem, appeal to emotion, red herring, hasty generalisation, irrelevant authority. a direct continuation of this work was presented in @xcite , where the resulting corpus from the use of "argotario" containing 430 annotated arguments was released. in that work, arguments belonging to the previous five classes plus a no fallacy set of arguments were compiled, and a set of preliminary results of experiments with a support vector machine (svm) and a bidirectional long short-term memory (bilstm) neural network were reported. aimed at better understanding the linguistic features underlying the ad hominem argument, habernal et al. developed a corpus from user discussions in the change my view subreddit on the reddit social network @xcite . for that purpose they retrieved the comments that were removed by the administrators because they were labelled as rude or hostile by the community, matching one of the non breakable rules proposed in @xcite as part of the pragma-dialectic theory of argumentation. the authors also reported a set of fallacy detection experiments with a convolutional neural network (cnn) in which they used this corpus consisting of 7,242 samples balanced between non-fallacious and ad hominem classes. the automatic identification of argumentative fallacies has also been studied from the propaganda viewpoint in @xcite , where the authors annotate news articles containing up to 18 propaganda techniques and report a series of experiments on propaganda classification. this perspective on fallacious argumentation was continued in a shared task organised for the semeval forum @xcite aimed at the automatic classification of natural language propaganda. based on the pragma-dialectic theory (van eemeren and grootendorst, 2016) eight classes of fallacious arguments were annotated in a corpus of informal fallacies in online discussions by @xcite . more than 1,700 fallacious comments retrieved from reddit were annotated into the classes of appeal to authority, appeal to majority, appeal to nature, appeal to tradition, appeal to worse problems, black-or-white, hasty generalisation, and slippery slope fallacies. furthermore, the authors report results on the binary task of classifying natural language text as fallacious or not, and on the 8-class classification problem of determining the type of fallacy to which each fallacious comment belongs to. for the experiments, the authors consider more advanced models based in the transformer architecture, and the granularity network that performed the best in @xcite . a simplified version of the task is presented in @xcite , where another corpus of fallacious argumentation is released. in this paper, the annotation of fallacious arguments is done from the transcripts of 31 political debates of the u.s. presidential campaigns. the authors annotate six different types of fallacy: ad hominem, appeal to emotion, appeal to authority, slippery slope, false cause, and slogans. in addition to these classes, 11 sub-classes are also annotated, providing additional information of the fallacious arguments. in their experiments, the best results are reported with a transformer-based architecture that combines natural language with argumentative features. the experimental results reported in that work are exclusively focused on the task of classifying fallacies, assuming that the fallacy has already been detected. recently, @xcite al explores the use of multitask instruction-based prompting to dectect 28 different fallacies across five datasets. the authors compare the use of t5 @xcite and gpt-3 @xcite for promptbased fallacy classification, and a fine-tuned bert @xcite model for a more classic baseline. from their results, it is possible to observe how the multitask instruction-based prompting with t5 achieves a significant increase in performance compared to the gpt-3 and bert baselines. however, the methodology applied in this paper is similar to the one followed in previous work, in which the fallacy type of a text sequence is determined by only taking the natural language of the sequence into account. finally, one of the most recent papers in the automatic detection of logical fallacies proposes a new task for pre-training language models based on the structure of arguments @xcite . for that purpose, the authors release a corpus consisting of 2,449 argumentative samples labelled into one of 13 different fallacy types. a set of experiments comparing large language models (llms) as zero-shot classifiers with transformerbased models fine-tuned on the corpus is reported, emphasising on the importance of looking at structural reasoning features for this type of classification problems. we can observe how, in the past years, a varied set of relatively small corpora have been annotated and publicly released. most of them, however, share a similar paradigm for addressing the automatic identification of argumentative fallacies. short spans of text are labelled with one of the corresponding fallacy labels, but no attention is given to the underlying logic that makes the argument fallacious or not. furthermore, all the reported experiments are done in a similar way, the natural language text is used as the input to learn a set of n classes (varying from one corpus to another) directly from the text, and no in-depth error analyses are reported in most of these works. these limitations might raise some concerns, such as the impact of non-fallacious arguments being labelled as fallacious (false positives) while they are not, just because they share similar words or natural language patterns. to have a better understanding of these cases, and the potential problems of relying only in deep learning algorithms for addressing a complex problem such as the identification of natural language fallacies, the argumentation scheme model of arguments presents itself as a promising alternative to the models considered in the literature. ## data in order to validate our hypothesis and to provide an evaluation outside of the training domain, we decided to use two different corpora in our experiments. first, the fallacy detection corpus, which consists of a partial combination of the data described in @xcite and @xcite . second, the argumentation scheme validation dataset, a small collection of natural language argumentation schemes that we created in this work in order to evaluate the inferences done by the predictive models to detect a natural language fallacy outside of the domains considered during training. with this second dataset, it is our objective to observe how well the model generalises when detecting natural language fallacies following a different model or structure than the one considered in the data used for training, similar to what would happen when deploying the predictive models in the wild. as depicted in since the annotation in both corpora was based on similar fallacy theory, our fallacy detection corpus combines some of the natural languages fallacies annotated in u.s. presidential debates @xcite , with some others annotated in social media discussions @xcite and the non-fallacious class. the decision of combining both corpora is twofold. first, we wanted to address the automatic detection of natural language fallacies (not just classifying them as done in (goffredo et al., 2022)) so non-fallacious samples were needed. the assumption done in @xcite of knowing beforehand that some piece of natural language is fallacious represents a significant limitation of the contribution since knowing the fallacious condition of an input is not trivial, and represents an important challenge in the area. the second reason to combine both corpora is to have a more balanced distribution of samples when comparing fallacious to non-fallacious samples, and to expand the natural language domains in which fallacies can be observed during training. a sample in our fallacy detection corpus consists of a short snippet of text where the fallacious (or not) reasoning has been identified, a natural language context in which the fallacy has been detected (a paragraph in the case of the debates, and the previous comment of the text snippet in the case of the social media discussions), and the annotated label. in order to homogenise the natural language context in data belonging to both corpora, for the samples extracted from the debate corpus we considered as the context only the sentences before and after the text snippet. aimed at validating the performance of machine learning and deep learning systems to detect natural language fallacies, we developed a small dataset containing natural language argumentation schemes @xcite . in this dataset, we included seven different types of argumentation schemes matching the fallacy classes included in the fallacy detection corpus: argument from expert opinion (afeo), argument from position to know (afpk), argument from popular practice (afpp), argument from popular opinion (afpo), slippery slope argumentation scheme (ssas), generic ad hominem (gah), and circumstantial ad hominem (cah). this way, we can easily relate each argumentation scheme with one of the four fallacy classes included in the fallacy detection corpus, the appeal to authority with afeo and afpo, the appeal to majority with afpp and afpo, the slippery slope with the ssas, and the ad hominem with the gah and cah. it is important to remember that, argumentation schemes are not fallacious by definition as they are the fallacy classes used to annotate previous corpora, but they can only be considered as fallacious if and only if some of the critical questions cannot be successfully answered. taking this into consideration, in our argumentation scheme validation dataset, we included two natural language instances of each scheme, one in which all the critical questions can be answered (i.e., valid reasoning), and another that fails in some aspect (i.e., fallacy). therefore, our argumentation scheme validation dataset consists of fourteen natural language arguments specifically designed to validate the inference process of the predictive models in the task of automatically detecting natural language fallacies. these natural language argumentation schemes have been compiled in ## results we have grouped the analysis of our results into two sections. first, we evaluate our models on the test split of the fallacy detection corpus. second, we evaluate these same models when used to detect or classify fallacies in the wild (i.e., outside of the training/testing data domain), for which purpose we use the argumentation scheme validation dataset. ## discussion in this paper, we present the first analysis of the limitations of approaching the fallacy detection prob-lem with llms. for that purpose, we provide a new viewpoint to the existing work done in the automatic identification of natural language fallacies through the use of the argumentation scheme model of arguments. the argumentation scheme model allows us to partially dissociate the logic of the argument from the natural language of it, evidencing the limitation that llms have when used to approach complex natural language tasks where logical reasoning is involved. for that purpose, we first ran a set of experiments training a machine learning and a deep learning algorithm plus prompting two llms on existing annotated corpora for fallacy identification, resulting in new baselines for this task. second, we evaluated the best performing models on a specifically created argumentation scheme validation dataset that helped us to understand how well were these models able to identify fallacies based on the logic of the argument rather than over-fitting to a natural language pattern not relevant for the definition of a fallacy. from our findings we have been able to observe that there is still much more work to do in this area, and that relying exclusively on llms to approach such a challenging task in the wild may not be the best option.