Decomposing Argumentative Essay Generation via Dialectical Planning of Complex Reasoning (2024)

Abstract: Argumentative Essay Generation (AEG) is a challenging task in computational argumentation, where detailed logical reasoning and effective rhetorical skills are essential.Previous methods on argument generation typically involve planning prior to generation.However, the planning strategies in these methods overlook the exploration of the logical reasoning process.Inspired by argument structure-related theories, we propose an argumentative planning strategy for prompting large language models (LLMs) to generate high-quality essays.This strategy comprises two stages: (1) Sketch planning, which creates a rough outline of the essay, and (2) Dialectical planning, which refines the outline through critical self-reflection.Such a planning strategy enables LLMs to write argumentative essays that are more logical, diverse, and persuasive.Furthermore, due to the scarcity of existing AEG datasets, we construct three new datasets.These datasets are from two domains: exam essays and news editorials, covering both Chinese and English.Automatic and manual evaluation on four datasets show that our method can generate more dialectical and persuasive essays with higher diversity compared to several strong baselines.

## introduction recent years have seen a surge of interest in argument generation tasks, such as counter-argument generation @xcite , argumentative essay generation @xcite , and controlled argument generation @xcite . automatic argument generation poses a significant challenge, as it requires not only coherent and logical sentences @xcite , but the primary content of descriptive analysis (claims and evidence), along with the complementary integration of critical thinking (rebuttals and counter-rebuttals), are key determinants of argument quality. with the advancement of pre-training techniques, large language models (llms) have shown remarkable capabilities in generation tasks @xcite . however, simple task instructions often fall short in steering llms to effectively handle complex tasks that need multistep decomposition @xcite . previous research has demonstrated that the use of chain-of-thought (cot) @xcite can significantly enhance the performance of llms on complex reasoning tasks by prompting the model to break down the task into a series of intermediate results. likewise, as the aeg task also necessitates rigorous logical reasoning to generate persuasive essays, simple task instructions are inadequate to handle aeg well. therefore, we aim to devise effective planning strategies to prompt llms to produce better-quality argumentative essays. our dialectical planning strategy is inspired by the argument theories proposed by @xcite , which reflects what constitutes a highquality argument as illustrated in figure 1 . according to freeman's theory, a comprehensive argument should include claims, evidence, overriding rebuttals, and undercutting rebuttals. the claims and evidences are crucial aspects of descriptive analysis. overriding rebuttals function as feedback countering the claims to guide the refinement of claims. simultaneously, undercutting rebuttals and counter-rebuttals collaborate to enhance persuasiveness. following the aforementioned theories, both humans and language models can develop better persuasive essays with in-depth analysis and critical thinking. however, current methods fail to consider these four determinants of argument quality thoroughly, nor do they address different types of rebuttals @xcite . to fill this gap, we propose a novel framework, dpe (dialectical planning of essays), which utilizes task decomposition to generate high-quality argumentative essays with a fine-grained planning strategy inspired by freeman's theory. during the planning stage, the framework arranges all vital components of the argumentative essay, which are then aggregated into a well-structured argumentative essay. specifically, given a writing prompt on a controversial issue, the framework first makes a sketch planning, drafting the major claim and relevant claims, which serve as the foundational points of the essay. subsequently, to foster profound and critical thinking, the framework performs dialectical planning, which involves: (1) overriding rebuttal that refines claims in sketch planning by identifying weakness, and (2) undercutting rebuttal that introduces an assistant counter-rebuttal to defend the claims. lastly, following the planning stage, the framework implements essay generation to systematize the intermediate results and compose a final argumentative essay. a commonly used dataset in existing aeg work is argessay @xcite , which is sourced from online forums. in this work, we additionally construct an english dataset and two chinese datasets. based on the aforementioned four datasets, we build a benchmark for aeg. the benchmark covers two common aeg scenarios: exam essay writing and journalism writing, both requiring opinion arguing on controversial topics. we conduct extensive experiments with both automatic and human evaluations. our proposed dpe achieves overall better performance than several strong baselines. the results on three significant aspects of argument quality show that our method is able to generate more persuasive and dialectical essays. we summarize our contributions as follows: ## dataset creation in total, we use four datasets to evaluate our proposed method. besides the argessay dataset introduced by @xcite , we further collect three datasets for aeg. these three datasets encompass two types of argumentative essay-writing scenarios: exam essays and news editorials, covering both english and chinese. exam essays. many examinations, such as ielts, toefl, and the chinese national college entrance examination, require students to write argumentative essays on controversial topics, which is a common task for testing language proficiency and critical thinking skills. therefore, exam essays are a crucial data source for aeg. argessay @xcite , for instance, is collected from an online forum dedicated to revising english exam argumentative essays. however, apart from english data, there is a scarcity of exam essay data in other languages for aeg. therefore, we propose a chinese exam argumentative essay (che-essay) dataset, which includes prompt-essay pairs from the chinese national college entrance examination. in accordance with previous work, we search and download data from real and mock chinese examinations available on doc-sharing websites @xcite . news editorials. news editorials are opinion articles written by editors or columnists of a news organization. its primary purpose is to provide anal- we illustrate how the data processing works in appendix g. we compare our datasets with existing argument generation datasets in ## method aeg task can be formulated as follows: given a writing prompt @xmath0, x 2 , ..., x k ], an aeg system needs to generate an argumentative essay figure 2 illustrates the overall framework of dpe. it follows a writing paradigm that prioritizes planning prior to generation. to guide llms towards improved critical thinking capabilities, we devise an argumentative planning strategy consisting of two key elements: sketch planning and dialectical planning. sketch planning prepares a preliminary outline of the essay, including a series of claims to be discussed. based on the outlined claims, dialectical planning drives the model to make expansive and self-critique thinking, akin to reflective practices in human writing. finally, llms integrate all the above planning results to produce a high-quality argumentative essay. ## conclusion in this work, we propose a theory-based framework that decomposes aeg by dialectically planning intermediate reasoning steps. the planning strategy imitates human critical thinking to generate more logical and persuasive essays. to boost further study on aeg, we build a benchmark in chinese and english where three datasets are newly collected. the experiment results on the benchmark demonstrate the superiority of our framework.