Decomposing Argumentative Essay Generation via Dialectical Planning of Complex Reasoning (2024)

Abstract: Argumentative Essay Generation (AEG) is a challenging task in computational argumentation, where detailed logical reasoning and effective rhetorical skills are essential. Previous methods on argument generation typically involve planning prior to generation. However, the planning strategies in these methods overlook the exploration of the logical reasoning process. Inspired by argument structure-related theories, we propose an argumentative planning strategy for prompting large language models (LLMs) to generate high-quality essays. This strategy comprises two stages: (1) Sketch planning, which creates a rough outline of the essay, and (2) Dialectical planning, which refines the outline through critical self-reflection. Such a planning strategy enables LLMs to write argumentative essays that are more logical, diverse, and persuasive. Furthermore, due to the scarcity of existing AEG datasets, we construct three new datasets. These datasets are from two domains: exam essays and news editorials, covering both Chinese and English. Automatic and manual evaluation on four datasets show that our method can generate more dialectical and persuasive essays with higher diversity compared to several strong baselines1.

heading: 1 Introduction
text: Recent years have seen a surge of interest in argument generation tasks, such as counter-argument generation (Alshomary et al., 2021b; Alshomary and Wachsmuth, 2023), argumentative essay generation (Carenini and Moore, 2006; Bao et al., 2022c), and controlled argument generation (Alshomary et al., 2021a; Al Khatib et al., 2021). Automatic argument generation poses a significant challenge, as it requires not only coherent and logical sentences (Asher and Lascarides, 2005), but
* Corresponding Authors 1Code and data are available at https://github.com/
HITSZ-HLT/AEG_DPE.
also the ability to think critically and deeply (Fisher, 2001; Freeman, 2011). As a prominent task in the field of argument generation, argumentative essay generation (AEG) aims at generating argumentative essays on controversial issues (Carenini and Moore, 2006; Bao et al., 2022c), which highlights the organization of convincing arguments to fully articulate viewpoints (Zukerman et al., 2000; Mann and Thompson, 1988). AEG is applicable in various domains, such as competitive debating (BarHaim et al., 2021), legal argumentation (Elaraby et al., 2023), academic essay writing (Bao et al., 2022c), and news editorials (Syed et al., 2020).
Previous research on argument generation has mainly focused on generating individual and comparatively concise arguments, typically involving a brief claim and its corresponding reasoning (El Baff et al., 2019; Jo et al., 2021). Only a limited number of studies have explored the generation of long-form argumentative essays, which contain multiple arguments across various aspects (Falk and Lapesa, 2023; Bao et al., 2022c). To better generate long-form text, prior work typically adopts the "plan-and-write" paradigm, generating intermediate planning first (e.g. keywords or relational
12305
triplets) to guide the final output (Hua and Wang, 2020; Hu et al., 2022). Although this approach has been proven beneficial for AEG, it only skims the surface in generating content-rich argumentative essays, overlooking the complex reasoning structure and dialectical nature of argumentative essays.
With the advancement of pre-training techniques, Large Language Models (LLMs) have shown remarkable capabilities in generation tasks (OpenAI, 2023; Ouyang et al., 2022). However, simple task instructions often fall short in steering LLMs to effectively handle complex tasks that need multistep decomposition (Qin et al., 2023). Previous research has demonstrated that the use of Chain-ofThought (CoT) (Wei et al., 2022) can significantly enhance the performance of LLMs on complex reasoning tasks by prompting the model to break down the task into a series of intermediate results. Likewise, as the AEG task also necessitates rigorous logical reasoning to generate persuasive essays, simple task instructions are inadequate to handle AEG well. Therefore, we aim to devise effective planning strategies to prompt LLMs to produce better-quality argumentative essays.
Our dialectical planning strategy is inspired by the argument theories proposed by Freeman (Freeman, 2011), which reflects what constitutes a highquality argument as illustrated in Figure 1. According to Freeman’s theory, a comprehensive argument should include claims, evidence, overriding rebuttals, and undercutting rebuttals. The claims and evidences are crucial aspects of descriptive analysis. Overriding rebuttals function as feedback countering the claims to guide the refinement of claims. Simultaneously, undercutting rebuttals and counter-rebuttals collaborate to enhance persuasiveness. Following the aforementioned theories, both humans and language models can develop better persuasive essays with in-depth analysis and critical thinking. However, current methods fail to consider these four determinants of argument quality thoroughly, nor do they address different types of rebuttals (Hu et al., 2022; Bao et al., 2022c).
To fill this gap, we propose a novel framework, DPE (Dialectical Planning of Essays), which utilizes task decomposition to generate high-quality argumentative essays with a fine-grained planning strategy inspired by Freeman’s theory. During the planning stage, the framework arranges all vital components of the argumentative essay, which are then aggregated into a well-structured argumentative essay. Specifically, given a writing prompt on
a controversial issue, the framework first makes a sketch planning, drafting the major claim and relevant claims, which serve as the foundational points of the essay. Subsequently, to foster profound and critical thinking, the framework performs dialectical planning, which involves: (1) Overriding Rebuttal that refines claims in sketch planning by identifying weakness, and (2) Undercutting Rebuttal that introduces an assistant counter-rebuttal to defend the claims. Lastly, following the planning stage, the framework implements essay generation to systematize the intermediate results and compose a final argumentative essay.
A commonly used dataset in existing AEG work is ArgEssay (Bao et al., 2022c), which is sourced from online forums. In this work, we additionally construct an English dataset and two Chinese datasets. Based on the aforementioned four datasets, we build a benchmark for AEG. The benchmark covers two common AEG scenarios: exam essay writing and journalism writing, both requiring opinion arguing on controversial topics. We conduct extensive experiments with both automatic and human evaluations. Our proposed DPE achieves overall better performance than several strong baselines. The results on three significant aspects of argument quality show that our method is able to generate more persuasive and dialectical essays. We summarize our contributions as follows:
• We propose an effective argumentative essay generation framework, DPE, which imitates the reasoning process proposed by Freeman’s theory. DPE can generate argumentative essays incorporating in-depth reasoning and critical thinking.
• We collect three high-quality datasets and construct a benchmark for argumentative essay generation.
• Using both automatic and human evaluations, we demonstrate that our framework can generate more dialectical and persuasive argumentative essays and outperform several baselines.

heading: 2 Related Works

heading: 2.1 Argumentative Essay Analysis
text: Extensive research has been conducted on the analysis of argumentative essays since an early stage (Madnani et al., 2012; Beigman Klebanov and Flor, 2013). Traditional argumentative essay analysis
work mainly studies the argument structure by mining the argumentative components and relations within essays (Stab and Gurevych, 2017; Potash et al., 2017; Sun et al., 2023; Bao et al., 2021a, 2022a; Sun et al., 2022). These studies contribute to a more comprehensive understanding of the argument process (Bao et al., 2021b, 2022b; Guo et al., 2023). Moreover, some other studies focus on analyzing the quality of arguments in writing essays. Lauscher et al. (2020) introduce a theory-based method of annotating and evaluating the quality of essays. Stede (2016) analyzes the depth of argument structure in the genre of news editorials.
These studies are highly relevant to our work, as the analysis of argument structure and quality can guide us in designing better mechanisms for argumentative essay generation.

heading: 2.2 Argument Generation
text: Early approaches to generating argumentative texts involve much manual effort, such as the construction of argument knowledge bases and the design of argumentation strategies (Reed, 1999; Zukerman et al., 2000; Carenini and Moore, 2000).
With the remarkable achievements of pre-trained generative models in recent years, argument generation has received much attention. Many argument generation-related tasks have been extensively studied, such as argument summarization (Fabbri et al., 2021; Syed et al., 2020; Elaraby et al., 2023), counter-argument generation (Alshomary et al., 2021b; Alshomary and Wachsmuth, 2023) and controlled arguments generation (Saha and Srihari, 2023; Schiller et al., 2021; Al Khatib et al., 2021). Hua et al. (2019) decompose counter-argument generation into content planning and realization to produce informative paragraphs. Fabbri et al. (2021) build a benchmark for argument summarization and incorporate argumentative essay analysis to enhance summarization. These argument generation studies mainly focus on generating individual and relatively short arguments.
Recently, some researchers have recognized the significance of generating long and coherent arguments. Bao et al. (2022c) collect the ArgEssay dataset for AEG and propose a dual-decoder model with content planning to generate content-rich argumentative essays. However, they fail to consider the argument structure and dialectical tier of argumentative essays. We decompose AEG into planning and realization using argument analysis to produce better results.

heading: 2.3 Planning and LLMs
text: Standard prompting follows a left-to-right generation process at the token level, which performs poorly for complex tasks (Bubeck et al., 2023). Subsequent works have emerged to involve complicated reasoning steps such as CoT (Wei et al., 2022), self-consistency (Wang et al., 2023b), and Tree-of-Thoughts (ToT) (Yao et al., 2023a). To improve performance on complex problems, recent works decompose the reasoning process and plan intermediate thoughts. (Zhang et al., 2023; Yao et al., 2023b; Huang et al., 2022). Hao et al. (2023) propose a reasoning solver based on LLMs to perform deliberate planning akin to human brains. Zhang et al. (2023) make use of a planning algorithm during decoding to generate more accurate results. In this paper, we explore using planning strategies specifically tailored for AEG to prompt LLMs for better reasoning and generation, thereby composing higher-quality argumentative essays.

heading: 3 Dataset Creation
text: In total, we use four datasets to evaluate our proposed method. Besides the ArgEssay dataset introduced by Bao et al. (2022c), we further collect three datasets for AEG. These three datasets encompass two types of argumentative essay-writing scenarios: exam essays and news editorials, covering both English and Chinese.
Exam Essays. Many examinations, such as IELTS, TOEFL, and the Chinese national college entrance examination, require students to write argumentative essays on controversial topics, which is a common task for testing language proficiency and critical thinking skills. Therefore, exam essays are a crucial data source for AEG. ArgEssay (Bao et al., 2022c), for instance, is collected from an online forum dedicated to revising English exam argumentative essays. However, apart from English data, there is a scarcity of exam essay data in other languages for AEG. Therefore, we propose a Chinese exam argumentative essay (CHE-Essay) dataset, which includes prompt-essay pairs from the Chinese national college entrance examination. In accordance with previous work, we search and download data from real and mock Chinese examinations available on doc-sharing websites (Tan et al., 2021).
News Editorials. News editorials are opinion articles written by editors or columnists of a news organization. Its primary purpose is to provide anal-
ysis and insights into social events. Unlike standard news reporting, editorials express the author’s subjective views, stances, and comments, making them more argumentative. For the AEG task, we gather two news editorial datasets, one in English and the other in Chinese. The English dataset, NYTEditorial, is sourced from the “Room for Debate” section of New York Times2, where a description of an event is provided as a prompt, followed by several professional writers presenting argumentative essays to express their views on that prompt. The Chinese dataset, CHN-Editorial, is collected from the news editorial section of two Chinese news websites, namely PengPai3 and GuangMingWang4.
We collect prompt-essay pairs from the aforementioned data sources, and then process them as follows:
• Manually excluding non-argumentative essays like propaganda essays, narrative essays, and news reports.
• Separating essays and prompts. For samples where the author does not list the prompt separately, we review and separate them manually.
• Cleaning up irrelevant text such as genre notifications, plagiarism warnings, author names, special characters, and timestamps, etc. We combine rule-based cleaning and manual review to ensure the quality of argumentative essays.
• To avoid ethical issues, we manually review and remove all essays that contain discriminatory, biased, or other inappropriate content.
2https://www.nytimes.com/roomfordebate 3https://www.thepaper.cn/channel_-24 4https://guancha.gmw.cn/node_11273.htm
We illustrate how the data processing works in Appendix G. We compare our datasets with existing argument generation datasets in Table 1. The datasets we propose contain argumentative essays that are longer and cover a wide range of topics. Also, the writing prompts within these datasets are much longer requiring models to have a deeper level of understanding.

heading: 4 Method
text: AEG task can be formulated as follows: given a writing prompt X = [x1, x2, ..., xk], an AEG system needs to generate an argumentative essay Y = [y1, y2, ..., yl].
Figure 2 illustrates the overall framework of DPE. It follows a writing paradigm that prioritizes planning prior to generation. To guide LLMs towards improved critical thinking capabilities, we devise an argumentative planning strategy consisting of two key elements: sketch planning and dialectical planning. Sketch planning prepares a preliminary outline of the essay, including a series of claims to be discussed. Based on the outlined claims, dialectical planning drives the model to make expansive and self-critique thinking, akin to reflective practices in human writing. Finally, LLMs integrate all the above planning results to produce a high-quality argumentative essay.

heading: 4.1 Sketch Planning
text: A high-quality argumentative essay typically centers on a major claim, which is strengthened by several supporting claims. These claims can be regarded as the skeleton of the essay. Thus, we first prompt LLMs to draft a major claim with multiple supporting claims in two consecutive steps:
M : X → c̃m → (c̃1, c̃2, ..., c̃n) (1)
where M is parameterized by LLMs, → indicates prompting LLMs with a specific prompt to generate the desired responses5, c̃m and c̃i denote the draft of major claim and n supporting claims respectively.

heading: 4.2 Dialectical Planning
text: It is known that adopting critical thinking and ingenious argument strategies can enhance the persuasiveness of an essay (Toulmin, 2008; Wolfe et al., 2009; Musi, 2018; Freeman, 2011). Therefore,
5All specific prompts used in our method can be found in Appendix D.
grounded in sketch planning, we present dialectical planning, which prepares arguments step by step. First, drawing from Freeman’s theory, we propose overriding rebuttal and undercutting rebuttal to make LLMs to mimic deep thinking in argumentative reasoning. LLMs are required to address these two types of rebuttals differently. Concretely, LLMs must acknowledge overriding rebuttal to formulate refined claims accordingly. When faced with undercutting rebuttals, LLMs should put forward counter-rebuttals to negate such rebuttals. Second, we generate evidence in support of the refined claims. Lastly, the major claim will be updated conditioned on newly refined claims.
Overriding rebuttal. Directly instructing M to refine claims can be inappropriate, as there is no criteria to guide the revision process. Thus, we first prompt LLMs to challenge the claims c̃i generated in sketch planning by producing an overriding rebuttal. Then, conditioned on the rebuttal, LLMs are required to optimize c̃i:
M : c̃i → roi (2) M : (c̃i, roi ) → ci (3)
where roi is the generated overriding rebuttal and ci is the refined claim. The overriding rebuttal here functions as feedback, countering the input claim and pointing out its inherent weaknesses. Such an approach of improving claims by attacking the weakness has also been proven effective in other studies (Alshomary et al., 2021b).
Undercutting rebuttal. In argumentative writing, presenting potential rebuttals and then counterrebutting them is a widely recognized argumentation strategy (Gao et al., 2019; Orbach et al.,
2019). Although the refined claim rectifies the weakness, rebuttal still exists to undercut the claim in terms of feasibility or risks. In dealing with such a rebuttal, LLMs should address the rebuttal via a counter-rebuttal to defend the claim. Specifically, we prompt LLMs to first generate an undercutting rebuttal followed by its counter-rebuttal:
M : ci → rui (4) M : (ci, rui ) → rci (5)
where rui and r c i denote undercutting rebuttal and counter-rebuttal respectively. Evidence. As evidence is crucial to support a claim, we further prompt M to produce k relevant evidence for each claim:
M : ci → ei (6)
where ci is the claim and ei denotes the corresponding evidence.
Major claim. Since each claim is revised, the draft major claim should also be modified accordingly. Thus, we implement a bottom-to-top update of the major claim with refined claims ci.
M : (c1, c2, ..., cn) → cm (7)

heading: 4.3 Essay Generation
text: During the generation phase, we aggregate the planning content, including ci, ei, rci , and r u i , into the introduction, conclusion, and body paragraphs. Then, these parts are transformed into a cohesive argumentative essay.
Leveraging the writing prompt and claims, we instruct the model to generate the introduction and
conclusion:
M : (X, cm) → I (8) M : (cm, c1, c2, ..., cn) → C (9)
where I and C represent the introduction and conclusion respectively.
In paragraph generation, we explicitly incorporate argument relations (refute or support) into the instructions to prompt M with underlying rules (Zhu et al., 2023). For each refined claim ci and its related arguments, we generate paragraph Pi as follows:
M : (ci, ei, rui , rci ) → Pi (10)
Thus far, we have generated all parts of an argumentative essay. Finally, we generate an argumentative essay based on these parts:
M : (I, {Pi}l,C) → Y (11)
where {Pi}l denotes a set of l paragraphs and Y is the output essay.

heading: 5 Experiment Setup

heading: 5.1 Baselines and Model Implementations
text: We evaluate AEG in a zero-shot setting, where LLMs generate an argumentative essay based on a writing prompt without any demonstrations. Our method is evaluated across four datasets, two in Chinese and two in English. From each dataset, we randomly select 50 writing prompts for both automatic and human evaluation.
To verify the effectiveness of our framework, we compare it with the following zero-shot baselines that adopt different planning strategies by prompting LLMs. (1) E2E: End-to-end generation that directly produces a target essay ; (2) CoT: Chainof-Thought generation (Wei et al., 2022) that first generates a brief plan as an intermediate guideline, and then generates an argumentative essay in the same response. (3) ToT: Tree-of-Thought generation (Yao et al., 2023a) that first derives multiple writing plans, evaluates the quality of each plan, and finally selects the best plan as the input prompt to produce an essay. We compare different LLMs including Baichuan-7B and ChatGPT for Chinese and LLaMA2-7B and ChatGPT for English. More implementation details are in Appendix A.

heading: 5.2 Evaluation Metrics
text: Automatic Evaluation. We find that most existing automatic metrics such as BLEU can not correspond closely with human judgments, as observed by Sellam et al. (2020). Motivated by recent developments in automatic evaluation where LLMs serve as judges and demonstrate good alignment with human evaluations (Wang et al., 2023a), we employ this approach for assessing the quality of generated essays in the following three aspects (Lauscher et al., 2020): (1) Cog: Cogency signifies whether an essay’s premises are acceptable and sufficient to support the conclusion. (2) Per: Persuasiveness reflects whether an essay is well-organized, contextually appropriate, and emotionally appealing. (3) Rea: Reasonableness refers to the essay’s ability to address counterarguments adequately, including acceptability and sufficiency in resolving the issue.
We leverage GPT-4 to score essays on a scale of 1 (worst) to 5 (best) (Liu et al., 2023a). We notice that recent work Zheng et al. (2023) emphasize the effects and solutions of self-enhancement and reasoning bias in scoring responses. Following them, we use CoT and reference-guided strategy to alleviate the bias. The evaluation templates and more details can be found in Appendix B. To reduce randomness, all experiments are performed three times, and the evaluation scores are averaged.
Human Evaluation. For a more comprehensive analysis, we conduct a human evaluation. We hire three well-educated master students to score the output quality following the three aspects in the automatic evaluation.

heading: 6 Results and Analysis

heading: 6.1 Automatic Evaluation
text: We present the evaluation results on English and Chinese datasets in Table 2 and Table 3, respectively. We observe that ChatGPT-based DPE achieves better performance by 0.11 in terms of Avg score compared to other ChatGPT-based methods on ArgEssay and NYT-Editorial. Among these LLaMA-based methods, DPE outperforms CoT and ToT and achieves comparable performance to E2E. On Chinese datasets, DPE outperforms other baselines with regard to ChatGPT-based methods while showing minimal drops (i.e., 0.02 drop in terms of Avg score on CHN-Editorial). This verifies the effectiveness of our dialectical planning.
We also find that CoT underperforms E2E in
most metrics, possibly due to LLM’s limited reasoning ability in handling complex planning. ChatGPT-based methods obtain significantly better results than LLaMA-based methods. An interesting finding is that DPE’s performance exhibits a more pronounced improvement in conjunction with the enhanced capabilities of LLMs. This suggests that our method is particularly suited to benefit from the evolving landscape of LLM development, promising even greater efficacy as these models continue to advance.
Additionally, experiments on output diversity in Appendix C demonstrate that DPE has the best content richness. We think these improvements primarily stem from sound counter-rebuttals, as they constitute a substantial part of the essays, as analyzed in the Section 6.5.

heading: 6.2 Human Evaluation
text: Human evaluation results on the English and Chinese datasets are presented in Table 4 and Table 5, respectively. Our DPE exhibits considerably better performance in terms of Rea, Cog, Per, and Avg. The ChatGPT-based DPE significantly outperforms other baselines relying on the superior reasoning abilities of ChatGPT, while LLAMA/Baichuanbased DPE show moderate improvements, demonstrating the effectiveness and adaptability of our approach in leveraging the more capable LLMs.
Owing to limited reasoning ability and selfenhancement bias (Zheng et al., 2023) of GPT4, there are discrepancies between automated evaluators and humans, particularly in the evaluation of lengthy essays with complex structures. For example, automatic evaluators prefer the outputs of LLaMA-based E2E, while human evaluators
find DPE better. Furthermore, human evaluators prefer essays of DPE in 68% cases with acceptable pairwise-agreement. We also perform a case study and find that DPE generates argumentative essays with critical thinking by tailoring rebuttal and counter-rebuttal. The experiment results can be found in Appendix E.

heading: 6.3 Ablation Study
text: We evaluate 20 samples to reveal the effect of each module in our model. To analyze the impact of different modules in our method, we conduct ablation studies in terms of removing overriding rebuttal (w/o OR), removing undercutting rebuttal (w/o UR), and removing both them (w/o OR+UR). The average performance of four datasets is reported in Table 6. We observe that w/o OR decreases the performance, verifying the importance of the refining claims. Removing UR leads to a significant drop in performance, indicating the effectiveness of reasoning is improved by addressing undercutting rebuttals. Surprisingly, removing both OR and UR w/o OR+UR) does not result in a catastrophic drop in performance. It even outperforms w/o UR. This suggests that using either OR or UR individually might confuse LLMs while leveraging the synergistic effect of both OR and UR can effectively enhance the model’s performance.

heading: 6.4 Analysis on Discourse Structure
text: The discourse structure offers insights into the text’s high-level organization, making the depth of the Rhetorical Structure Theory (RST) tree a key indicator of text quality (Stede, 2016). Hence, we parse output essays into RST trees and analyze the depth distribution of the RST trees to assess the quality, with the results presented in Figure 3. DPE generates argumentative essays with deeper structures on average. Additionally, it displays a broad depth distribution, encompassing a wider range than ToT, CoT, and E2E methods. This indicates that our method effectively contributes to creating more diverse and intricate structures.

heading: 6.5 Analysis on Planning Content
text: During the planning phase, we generate the essential planning content, including claims and evidence for crafting a well-structured essay. We conduct experiments to assess how various planning content influences the output essay. To do this, we compute the n-gram overlap between the output essay and each type of argumentative planning content using BLUE score (Papineni et al., 2002). We present the results in Figure 4.
We find that the output essay incorporates planning content from every reasoning step in the dialectical reasoning process. Notably, counterrebuttal achieves the highest scores, further indicating the importance of critical thinking. As an example in Figure 19 illustrates, an effective counterrebuttal not only addresses the potential counterarguments but also strengthens the entire reasoning in support of the claim.

heading: 7 Conclusion
text: In this work, we propose a theory-based framework that decomposes AEG by dialectically planning intermediate reasoning steps. The planning strategy imitates human critical thinking to generate more logical and persuasive essays. To boost further study on AEG, we build a benchmark in Chinese and English where three datasets are newly collected. The experiment results on the benchmark demonstrate the superiority of our framework.
Limitations
We mainly study two common scenarios for AEG, that is, exam essays and news editorials. Other scenarios may require different genres, such as speech and legal argumentation. Consequently, for future research, we aim to explore a more intelligent framework capable of automatically devising reasoning steps for various topics and scenarios.
In our method, we do not design mechanisms to guarantee faculty. Therefore, future research could explore enhancing content reliability through the verification and editing of intermediate arguments. Additionally, the editing of arguments can double as a plugin for controllable AEG.
Ethics Statement
We strictly follow the licenses and policies of released LLMs and publicly available datasets. Our datasets are created from publicly available sources without any personal identity information. When crawling data from "New York Times", "PengPai", and "GuangMingWang", we follow the privacy policy and terms of use of these platforms. We have annotated the source of each data to comply with the copyright regulations. To address ethical concerns, we review and remove any essays that are considered to be libelous, racist, or otherwise inappropriate.

heading: Acknowledgment
text: This work was partially supported by the National Natural Science Foundation of China 62176076, Natural Science Foundation of GuangDong 2023A1515012922, the Shenzhen Foundational Research Funding JCYJ20220818102415032 (JCYJ20210324115614039), the Major Key Project of PCL2021A06, Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies 2022B1212010005.

heading: A More Implementation Details
text: In our experiments, all ChatGPT-based models are implemented with gpt-3.5-turbo-1106 API. For Baichuan-7B6 and LLaMA-7B7 experiments, we strictly follow the commands and procedures to recover the weights of these models. We set the temperature as 0.8 and top-k as 50 for generation in all models. For claim generation in sketch planning, we set the number of claims to be generated as m = 3. We generate k = 2 pieces of evidence for each claim. In overriding rebuttal generation, we prompt the LLMs in one iteration to revise the draft claims. We generate l = 3 paragraphs for each argumentative essay. Additionally, we show the prompt used in E2E, CoT, and ToT in Figure 6 to 8.

heading: B Automatic Evaluation Setting
text: The automatic evaluator is based on GPT-4 (get-4-0613). The evaluator is required to first give feedback and then predict the scores in a CoT manner. The prompts are reference-guided with a comprehensive list of criteria of Reasonableness, Cogency, and Persuasiveness. The evaluation prompts are shown in Figure 5.
Recent researchers use GPT-4 as an NLG evaluator to score a single response on a scale of 1 to 100. The GPT-4 only received ambiguous evaluation criteria like "a score of zero means disfluency and score of one hundred means perfect fluency" (Liu et al., 2023a). However, such evaluation introduces two biases (Zheng et al., 2023): (1) selfenhancement bias where LLM evaluators may favor the responses generated by themselves or other models. (2) reasoning bias where LLM evaluators make incorrect judgments due to their limited capability in scoring reasoning questions. To make the evaluator score essays fairly, we prompt GPT-4 to score by giving reference-guided scores ranging from 1 to 5. Additionally, the generated feedback can enable GPT-4 to evaluate more reasonably. Our
6https://huggingface.co/baichuan-inc/Baichuan2-7BChat
7https://huggingface.co/meta-llama/Llama-2-7b-hf
early evaluation didn’t adopt these techniques and suffers from bias (Zheng et al., 2023; Wang et al., 2023a). In practice, we observe that the feedback can cite the input reference to give stable and reasonable scores. An example is shown as follows: {"Feedback": "This essay presents a clear argument against the consumption of sugar-based drinks, with a particular emphasis on how aggressive marketing and advertising contributes to this problem. It offers solid evidence to back up these claims, most notably citing various studies and their results. Although it addresses the potential counterargument that hectic lifestyles might contribute to the consumption of such drinks, it quickly debunks it with evidence. The essay could improve its persuasiveness by using more emotional appeals, perhaps by describing the health dangers associated with excessive sugar consumption on a more personal level.", "Reasonableness": 5, "Cogency": 4, "Persuasiveness": 4}
Yet we also observe that the GPT-4 evaluator occasionally outputs positive feedback but relatively low scores when the input essays are complex in
disclosure structure and logic. Instead, GPT-4 may prefer shorter essays with straightforward expression. This partly explains why E2E sometimes gets higher scores than ToT and DPE while humans think otherwise.

heading: C Analysis on Diversity
text: We analyze output diversity by averaging the number of distinct bigrams (Li et al., 2016). We compute average distinct bigrams for Chinese and English datasets in Figure 9.
Our method generates more diverse content compared to others, especially in the Chinese dataset. On the contrary, CoT yields the least diverse outputs. We hypothesize that thinking step by step is
more suitable to achieve an assertive answer instead of a diverse reasoning result. E2E outperforms CoT as it follows the instructions well to output essays naturally and diversely. ToT benefits from producing more detailed plans and selecting the best one. Decomposing AEG with our method further improves diversity since we introduce claims, evidence, and rebuttals of different perspectives, leading to high content richness.

heading: D Prompts for DPE
text: To better illustrate how generations of argumentative components work, we provided detailed prompts in the dialectical planning stage. Concretely, the prompt we used for sketch planning is in Figure 10 and 11. The prompt for generating overriding and undercutting rebuttals is in Figure 12. The prompt for generating claims, counterrebuttal, and final major claim are presented in Figure 13, 14, and 15, respectively. The prompts we used in the essay generation stage are in Figure 16 to 18.

heading: E More Human Evaluation
text: E.1 Human Ranking Result
we investigate human preferences where human annotators are required to rank output essays based on identical prompts. Table 7 illustrates the pairwise pairwise-agreement among the three annotators, measured by Kendall’s τ , yielding an average of 0.41. We observe that Annotator 3 and Annotator 2 agree notably more with each other than with Annotator 1.
Table 8 reports the avg-rank and topmost percentage achieved by each approach. Our method achieved 1.5 avg-rank indicating that essays generated with self-critique perform best. As for the topp, our method ranks first in 68% cases. Compared
with other methods, DPE stands out by delivering high-quality essays.
E.2 Is GPT-4 evaluator qualified?
Although the GPT-4 diverges from humans (Shen et al., 2023), the GPT-4 is a qualified evaluator giving reasonable scores. Using GPT-4 to evaluate is much more reliable than other automatic metrics (Rouge, BLEU). Other works (Liu et al., 2023b; Wang et al., 2023a) also prove GPT-4 to be a qual-
ified evaluator. It have demonstrated its ability in various analytical tasks (Cheng et al., 2023; Ding et al., 2023). Following these works, we attempt to use GPT-4 socre on more complex criteria and we compute the Spearman correlation to be around 0.33. How to make GPT-4 as a human-level evaluator is left for future work.
E.3 Qualitative Comparison with Previous Work
We conduct a qualitative analysis to compare DPE with previous SOTA Bao et al. (2022c). We sample 50 essays from the ArgEssay dataset and manually compare the cogency, persuasiveness, and reasonableness. The annotators consistently agree that essays generated by DPE are better considering these three aspects.
E.4 Case Study
We present an example output highlighting intermediate argumentative components with different colors in Figure 19. Due to limited space, we only show an introduction and the following paragraph. Our approach incorporates the prompt and major claim during the essay generation stage. As a result, the introduction smoothly transitions from the writing prompt to the thesis statement. Moreover, the essay articulates a profound perspective, facilitated by the feedback of overriding rebuttal. To persuade the readers, the essay adequately refutes po-
tential counterarguments and proposes valid solutions. This implies that organizing the reasoning by decomposing the essay into argumentative components enhances overall persuasiveness. Discourse markers sensibly embody the logic between individual sentences. By explicitly incorporating argumentative relations, the essay avoids self-contradiction. More example outputs and comparisons can be found in Figure 22 and Figure 21. We present the introduction and a body paragraph for each approach.

heading: F Cost for OpenAI API
text: Specifically, the cost per essay of OpenAI API and the average tokens number of generated essays are shown in the following table:
E2E CoT ToT DPE average cost 0.001708 $ 0.002370 $ 0.014390 $ 0.021890 $
average tokens 854 1185 7159 10945

heading: G Data Process
text: It is believed that an argumentative essay should have a clear claim and line of reasoning. We review the first few paragraphs to exclude the propaganda essays,narrative essays, and news reports. As for propaganda essays, such essays typically include political speeches and advertisements. We defined propaganda essays in the following criteria: 1) essays that use selective facts, exaggerations, or distortions of truth to persuade the reader and 2)
essays that repeat slogans, key messages, and concepts. The annotators are require to review the data in four steps as shown in Figure 20.

heading: Future Work
text: How to make GPT-4 as a human-level evaluator is left for future work. E. 3 Qualitative Comparison with Previous Work
We conduct a qualitative analysis to compare DPE with previous SOTA Bao et al. (2022c). We sample 50 essays from the ArgEssay dataset and manually compare the cogency, persuasiveness, and reasonableness. The annotators consistently agree that essays generated by DPE are better considering these three aspects. E. 4 Case Study
We present an example output highlighting intermediate argumentative components with different colors in Figure 19. Due to limited space, we only show an introduction and the following paragraph. Our approach incorporates the prompt and major claim during the essay generation stage. As a result, the introduction smoothly transitions from the writing prompt to the thesis statement. Moreover, the essay articulates a profound perspective, facilitated by the feedback of overriding rebuttal. To persuade the readers, the essay adequately refutes po-
tential counterarguments and proposes valid solutions. This implies that organizing the reasoning by decomposing the essay into argumentative components enhances overall persuasiveness. Discourse markers sensibly embody the logic between individual sentences. By explicitly incorporating argumentative relations, the essay avoids self-contradiction. More example outputs and comparisons can be found in Figure 22 and Figure 21. We present the introduction and a body paragraph for each approach