Revision for Concision: A Constrained Paraphrase Generation Task (2022)

Abstract: Academic writing should be concise as concise sentences better keep the readers’ attention and convey meaning clearly. Writing concisely is challenging, for writers often struggle to revise their drafts. We introduce and formulate revising for concision as a natural language processing task at the sentence level. Revising for concision requires algorithms to use only necessary words to rewrite a sentence while preserving its meaning. The revised sentence should be evaluated according to its word choice, sentence structure, and organization. The revised sentence also needs to fulfil semantic retention and syntactic soundness. To aide these efforts, we curate and make available a benchmark parallel dataset that can depict revising for concision. The dataset contains 536 pairs of sentences before and after revising, and all pairs are collected from college writing centres. We also present and evaluate the approaches to this problem, which may assist researchers in this area.

## introduction concision and clarity 1 are important in academic writing as wordy sentences will obscure good ideas (figure 1 ). concise writing encourages writers to choose words deliberately and precisely, construct sentences carefully to eliminate deadword, and use grammar properly (stanford university), which often requires experience and time. a first draft often contains far more words than necessary, and achieving concise writing requires revisions @xcite . as far as we know, currently this revision process can only be done manually, or semi-manually with the help of some rule-based wordiness detectors @xcite . we therefore introduce and formulate revising for concision as a natural language processing (nlp) task 1 we treat concision and conciseness as equivalent, and clarity as part of concision wordy concise wordy concise as you carefully read what you have written to improve your wording and catch small errors of spelling, punctuation, and so on, the thing to do before you do anything else is to try to see where a series of words expressing action could replace the ideas found in nouns rather than verbs. for example, in the field of image recognition, experimental results on some standard test sets indicate that the recognition capabilities of deep learning models can already reach the level of human intelligence. as you edit, first find nominalizations that you can replace with verb phrases. for example, in the field of image recognition, test results show that deep learning models can already reach human intelligence. figure 1 : wordy sentences are more boring to read than concise sentences. but how do we turn lengthy sentences into concise ones? we show two examples. the above sentence pair is taken from the purdue writing lab, which suggests how college students should succinctly revise their writing (pu). in the other example, the wordy sentence comes from a scientific paper @xcite , and its concise counterpart is predicted from the concise revisioner we developed (section 5). in each pair, text with the same colour delivers the same information. ## related work manual revision operations include delete, replace, and rewrite. intuitively, a revising program should do similar jobs, too. in fact, these actions are implemented individually in various nlp tasks. for example, sentence compression requires programs to delete unnecessary words, and paraphrasing itself is a matter of replacement. machine revision for concision could also share traits with them. practically, when a neural model learns in a seq2seq manner, the difference among these tasks is the parallel dataset. we are also interested in whether programs developed for these tasks can work in machine revision. ## benchmark corpus the collated corpus, named concise-536, contains 536 pairs of sentences. this is a fair starting size, comparable with 385 of rst-dt (semantic parsing, @xcite , 500 of duc 2004 (summarization foot_4 ), or 575 by @xcite (sentence compression). each concise sentence is revised from its wordy counterpart by english specialists from the 72 universities, colleges, or community colleges. sentence id, category and original link are available for each data point foot_5 , and a 120-point validation split from other sources is attached. revising different sentences can go through a completely different process. as seen below, simply crossing out a few words revises in concise-536, we do not identify fine-grained wordiness because a phrase can have more than one type of verbosity at the same time. for instance, we can revise "her poverty also helped in the formation of her character." to "her poverty also helped form her character." (george mason university, 2021), treating "in the formation of " as either a wordy prepositional phrase, or nominalization. rather, we focus on editing. in editing, the three actions are not complementary, and instead have varying degrees of power. deleting can be covered by replacing (see section 3.1), which could be again covered by rewrit-ing, i.e., rewriting is the most flexible. however, occam's razor pushes us to prioritize the actions requiring lower effort to complete, i.e., @xmath0. supposedly, the difficulty for implement each action with programs shares the same trend. in addition, some sentences contain multiple wordiness occurrences, each of which may need a different action, e.g., delete + replace. interested in how well a revising algorithm resembles each action, we label revisions in each sentence pair and divide them into seven categories. revisions that require the same set of actions will be assigned to the same category. each revision is assigned to one of seven categories in for convenience, we standardize categorizing rules as follows where each sentence is a wordlevel sequence. for each pair, we have a wordy sequence (w) and a concise sequence (c). ## approaches to revisions we approach the raised problem in this study. solutions to machine revision for concision can be diverse. neural model solutions include tree-totree transduction models @xcite , or general seq2seq models. we present a seq2seq approach, for it is flexible and straightforward. the model architecture is bart @xcite . ideally, training corpora tune statistical models or neural models, such that we can test tuned models on the benchmark corpus. however, lacking authoritative revisions prompted us to let models fit relevant task data. we also use public external knowledge, e.g., wordnet @xcite . this section describes how we build an ad hoc training corpus to initiate this task. the bart base model (124,058,116 parameters) is then used to fit each training set in this section. training settings are fixed (batch size at 32, pytorch adam optimizer @xcite with initial learning rate at 5 × 10 -5 , validated every 5,000 iterations). we then evaluate trained models on concise-536. ## discussion comparing the proposed revisioner's effectiveness for different categories, we understand deleting and replacing are much easier sub-tasks than rewriting is. the former two actions, especially deletion, are less ill-posed, while rewriting is open. still, revision for concision requires an algorithm that is able to use all three actions in combination. its goal is to resolve all seven categories of cases, marking distinction between revision and other tasks such as sentence compression. we use seven metrics to estimate a revisioner's effectiveness, since each metric has its shortcomings. for example, meteor does not adequately penalize nominalization, and thus wordy input texts typically score higher on meteor than algorithm outputs. more targeted metrics for this task, including reference-free structural metrics @xcite , might help. we do not include word counts. although concision is marked by brevity and wordiness often correlates to high word count, concise writing does not always require the fewest words (pu). optimizing a lower word count may be misleading even if it is constrained to zero information loss @xcite . for example, abusing pronouns and ellipses can result in shorter sentences that are harder to read. transferring knowledge from other tasks to approximate revising is a stopgap measure. specialized revising methods exist, e.g., the paramedic method @xcite . automated specialized methods may be more efficient. ## conclusion we formulate sentence-level revision for concision as a constrained paraphrase generation task. the revision task not only requires semantics preservation as in usual paraphrasing tasks, but also specifies syntactic changes. a revised sentence is free of wordiness and as informative. revising sentences is challenging and requires coordinated use of delete, replace, and rewrite. to benchmark revising algorithms, we collect 536 sentence pairs before and after revising from 72 college writing centres. we then propose a seq2seq revising model and evaluate it on this benchmark. despite scarcity of training data, the proposed approaches offer promising results for revising academic texts. we believe this corpus will drive specialized revision algorithms that benefit both authors and readers.