AEG : Argumentative Essay Generation via A Dual-Decoder Model with Content Planning (2022)

Abstract: Argument generation is an important but challenging task in computational argumentation.Existing studies have mainly focused on generating individual short arguments, while research on generating long and coherent argumentative essays is still under-explored.In this paper, we propose a new task, Argumentative Essay Generation (AEG).Given a writing prompt, the goal of AEG is to automatically generate an argumentative essay with strong persuasiveness.We construct a large-scale dataset, ArgEssay, for this new task and establish a strong model based on a dual-decoder Transformer architecture.Our proposed model contains two decoders, a planning decoder (PD) and a writing decoder (WD), where PD is used to generate a sequence for essay content planning and WD incorporates the planning information to write an essay.Further, we pre-train this model on a large news dataset to enhance the plan-and-write paradigm.Automatic and human evaluation results show that our model can generate more coherent and persuasive essays with higher diversity and less repetition compared to several baselines.

## introduction automatic argument generation, literately the task of generating persuasive arguments on controversial issues @xcite , has received many research interests in recent years @xcite . many works have involved different arguments generation such as the counter-arguments generation @xcite @xcite and the ## dataset creation our dataset is collected from essay forum, foot_0 an online community established by professional writers and editors to help users write, edit, and revise their essays. specifically, we selected the essays and prompts of high-quality in the writing feedback section of essay forum, where users post their essays for revision suggestions in preparation for standardized english test like ielts or toefl. 3 in addition, the essays in the writing feedback section have also been used in the researches on argument mining @xcite . first, we collect all the post in the writing feedback section of essay forum. then, to obtain the prompt-essay pairs and ensure the text quality, we conduct several pre-processing steps including: it is worth noting that the essay forum administrator will review and remove any posts that are considered to be libelous, racist, or otherwise inappropriate. thus, the ethic of our dataset can be assured. further, we also manually check the dataset to avoid ethical issues. as for the data split, we want to minimize the overlap between the train set and the validation/test set in terms of prompts, otherwise it would be difficult to test the model's generalization ability on new prompts. thus, we first extract keywords from the prompts based on tf-idf @xcite and measure the similarity of any two prompts as the jaccard similarity between their keywords set. then, when splitting the data, for any prompt in the validation/test set, we ensure that the similarity between it and each prompt in the train set does not exceed a threshold ϵ. after several rounds of manual verification, we set ϵ = 0.65, as we observe that this threshold can reasonably separate the prompts with more than 70% of the validation/test prompts having a similarity of less than 0.30 to any training prompt. the final dataset consists of 11, 282 promptessay pairs in english, in which 9, 277/1, 002/1003 pairs are used for training/validation/testing, respectively. we compare our proposed dataset with existing argument generation datasets in ## methods our proposed aeg task can be formulated as follows: given a writing prompt @xmath0, x 2 , . . . , x m ], a relevant argumentative essay y @xmath1, y 2 , . . . , y n ] should be generated. in order to generate diverse and content-rich essays, we propose a transformer-based dualdecoder model with a plan-and-write strategy. in detail, our model first predicts a planning sequence y p , then it generates the argumentative essay y e under the guidance of the planning sequence through the planning attention. the planning strategy is commonly used in long-form text generation studies. here, instead of using a standalone model for predicting the planning @xcite , we utilize a dual-decoder architecture to enable end-to-end training for generating the planning and the essay. in the following, we will first introduce the method of constructing planning sequence y p for training and then describe our model in detail. ## conclusion in this paper, we propose a challenging new task, aeg, to generate long-form and coherent argumentative essays. to tackle this task, we present a largescale dataset and further devise a dual-decoder architecture based on the basis of bart, which can generate a planning and a planning-guided essay in an end-to-end fashion. the experimental results demonstrate the superiority of our model. for future work, we plan to draw on external knowledge to generate more diverse and informative argumentative essays.