A Logical Fallacy-Informed Framework for Argument Generation (2025)

Abstract: Despite the remarkable performance of large language models (LLMs), they still struggle with generating logically sound arguments, resulting in potential risks such as spreading misinformation. An important factor contributing to LLMs’ suboptimal performance in generating coherent arguments is their oversight of logical fallacies. To address this issue, we introduce fallacy-informed preference optimization (FIPO) that helps steer LLMs toward generating logically sound arguments. FIPO includes a classification loss to capture the fine-grained information on fallacy types. Our results on argument generation tasks show that FIPO reduces the fallacy errors by up to 17.5%. Furthermore, our human evaluation results reveal that the quality of the arguments generated by our method significantly outperforms the fine-tuned baselines and other preference optimization methods, such as DPO. These findings highlight the importance of ensuring models are aware of logical fallacies for effective argument generation.

## introduction argument generation is crucial in daily life and has numerous online and offline applications. for instance, legislative bodies often use persuasive arguments to secure the necessary votes for bills to pass. however, generating logically coherent arguments is a challenging task and requires an appropriate combination of reliable evidence and effective logical reasoning @xcite . humans are prone to misconstruing logical argumentation in the real world and often unknowingly adopt flawed reasoning in discussions @xcite . similarly, large language models (llms) have demonstrated limitations in their logical reasoning capabilities, suffering from logical appeal to emotion ai automates tasks, reducing employment opportunities by replacing humans in manufacturing and administrative jobs. ## task formulation in this work, we address the argument generation task. in order to assess llms' capabilities for argument generation, we leverage the ex-plagraphs dataset @xcite , consisting of topics, stances and arguments, denoted as ## methodology to address the challenge of generating fallacy-free arguments, we propose using preference learning methods to generate arguments logically aligned with the given topic and stance. this approach involves making models aware of logical fallacies and training them to generate logically correct arguments by rewarding valid arguments and penalizing dispreferred samples. the process of preference learning typically involves three main steps: (i) supervised fine-tuning (sft) (section 4.1), (ii) preference data collection (section 4.2) and (iii) reinforcement learning (section 4.3). in section 4.4, we introduce our method (fipo), which introduces fine-grained information about fallacies in the alignment process. a comprehensive overview of the methodology is presented in fig. 2 . to justify our design and methodology for fipo and the preference data collection, we perform two ablation studies with different training approaches, described in section 5.3. the results demonstrate that our design achieves the best performance. ## experimental setup we denote the policies obtained after the sft phase and the alignment phase as π sft and π θ , respectively. the policy π θ is aligned using one of the following methods: ppo @xcite , dpo @xcite , cpo @xcite , kto @xcite , and fipo. ## experimental results as outlined in section 5.2, our evaluation of the generated arguments focuses on two primary aspects: (i) pairwise comparison of argument quality between the reference policy π sft and the aligned policies π θ , which is detailed in section 6.1; and ## conclusion in this work, we investigate the impact of logical fallacies on argument generation and introduce fipo, a novel framework designed to improve the logical soundness of arguments by including a classification loss during the preference optimization phase. both human and automatic evaluations show that our method produces higher-quality arguments and achieves lower fallacy-rates. these findings underscore the importance of addressing logical fallacies in improving argument generation.