Perspective-driven Preference Optimization with Entropy Maximization for Diverse Argument Generation (2025)

Abstract: In subjective natural language generation tasks, generating diverse perspectives is essential for fostering balanced discourse and mitigating bias. Argument generation with diverse perspectives plays a vital role in advancing the understanding of controversial claims. Despite the strong generative capabilities of large language models (LLMs), the diversity of perspectives remains insufficiently explored within argument generation task. Moreover, there remains a significant research gap in developing methods that explicitly generate multi-perspective arguments under the quality control of claim-stance alignment constraints. In this paper, we propose POEM, a \textbf{P} erspective-aware Preference \textbf{O} ptimization with \textbf{E} ntropy \textbf{M} aximization framework for diverse argument generation. It enhances perspective diversity through preference optimization based on the constructed preference dataset via perspective mining and diversity measuring. It further introduces entropy maximization to promote perspective diversity by encouraging dispersed semantic representations among the generated arguments. Experimental results on claim-stance argument generation benchmarks show that POEM is capable of generating diverse arguments while maintaining comparable performances in claim and stance controllability as well as text quality compared to the state-of-the-art baselines and human evaluation.

## introduction natural language generation (nlg) for subjective tasks such as argument generation requires not only fluency and coherence but also the ability to express a diverse range of perspectives. to mitigate potential bias, a well-designed nlg model should be capable of representing multiple perspectives rather than defaulting to a singular one or prevailing opinions @xcite . argument generation physical media such as dvds are pointless 1. physical discs offer better audio and video quality 2. collectors deeply value owning tangible, permanent copies that can't be altered online with diverse perspectives plays a vital role in advancing the understanding of controversial claims @xcite , and manifests foundational capabilities of human intelligence that are essential for modeling a wide range of human activities and common to human societies @xcite . to ensure the quality of generated content, arguments should be consistent with the given stance and relevant to the corresponding claim. moreover, expressing diverse perspectives is essential for avoiding potential bias and enhancing the persuasiveness of the arguments. generating diverse arguments across multiple perspectives is an essential issue for online interactions. it can help the user articulate a wide spectrum of personal opinions, reflecting the richness of discussions in social media. for example, in figure 1 , there are five arguments to express oppose stance towards the claim "physical media such as dvds are pointless". perspective provides a distinct pragmatic semantic focus that supports a given stance toward a claim. the perspectives in figure 1 include noun phrases better quality, permanent copies, streaming services, internet outages, preserves culture, which express topical diversity of opinions. to ensure the quality of argument generation, the generated arguments should be stance-alignment and maintain high relevance to the given claim. the majority of existing research has explored various control dimensions of argument generation to meet practical application requirements, including stance alignment @xcite @xcite , specific aspects @xcite , user personalization @xcite , and factuality @xcite . however, previous research has largely overlooked the important perspective diversity issue in argument generation. with the rapid development of large language models (llms), computational argumentation has become an essential tool across various domains, providing persuasive arguments aligned with a specific stance to enhance the understanding of controversial claims. recently, there has been growing interest in leveraging their generative capabilities for argument generation. llms provide an important means to promoting positive online interactions and fostering active community communications among participants. however, the diversity of perspectives remains insufficiently explored in argument generation task. to address this issue, recent studies have investigated prompting-based methods to elicit diverse arguments @xcite . due to the inherent limitations of auto-regressive decoding, llms are liable to produce repetitive contents or semantically different contents with similar perspectives. in particular, there remains a significant research gap in developing methods that explicitly generate multiperspective arguments given a specific stance for a claim. thus, from a computational account, there is still a lacking of llm-enabled argument generation research encompassing diverse perspectives under the quality control of claim-stance alignment constraints. as a result, generating diverse arguments that effectively represent a broad and meaningful range of perspectives remains a fundamental challenge. to address these challenges, in this paper, we propose a perspective-driven preference optimization with entropy maximization (poem) framework for diverse argument generation, comprised of instruction fine-tuning, perspective-driven preference optimization and entropy maximization modules. first, the instruction fine-tuning stage trains the model to equip initial ability of generating stance-consistent and claim-relevant arguments while implicitly capturing the multi-perspective nature of gold arguments. then, we construct pairwise preference data via measuring perspective diversity among arguments to perform preference optimization, explicitly enhancing perspective diversity in generations. finally, we devise an entropy maximization module that further encourages dispersed representations among generated arguments to reduce semantic redundancy and increase the coverage of perspectives. the main contributions of our work are summarized as follows: ## proposed method given a controversial claim c and a stance s, the stance-conditional argument generation task aims to generate an argument set @xmath0, . . . , y n } containing n diverse arguments, where each argument y @xmath1. we define a perspective as a specific semantic focus or reasoning dimension that supports the given stance to the claim. as shown in figure 1 , the claim is approached from various perspectives including better quality, permanent copies, streaming services, internet outages, preserves culture. these perspectives are grounded in noun phrases or topical phrases from the arguments. our proposed framework poem consists of three modules: (1) instruction fine-tuning, which enables the language model to produce arguments that are claim-relevant and stance-consistent, while promoting perspective diversity implicitly; ## conclusion in this paper, we propose poem, a perspectivedriven preference optimization with entropy maximization framework for diverse argument generation. our approach integrates instruction finetuning, perspective-driven preference optimization, and an entropy maximization module to jointly promote perspective diversity while preserving stance consistency and claim relevance. our perspectivedriven preference optimization guides the model to generate claim-relevant, stance-consistent arguments by measuring and enhancing perspective diversity through perspective-driven preference data. meanwhile, the entropy maximization module promotes diversity by learning dispersed representations, reducing redundancy and increasing perspective coverage. experimental results on claim-stance benchmarks demonstrate that poem significantly enhances perspective diversity while maintaining strong performance in stance alignment, claim relevance and text quality compared to state-of-the-art baselines also human evaluation.