Doolittle: Benchmarks and Corpora for Academic Writing Formalization
Authors: Shizhe Diao
Yongyu Lei
Liangming Pan
Tianqing Fang
Wangchunshu Zhou
Sedrick Scott Keh
Min-Yen Kan
Tong Zhang
Venue: EMNLP 2023 Main (2023)
Keywords: grammar error correction
large language models
academic writing formalization

Abstract: Improving the quality of academic writing is a meaningful but challenging task. Conventional methods of language refinement focus on narrow, specific linguistic features within isolated sentences, such as grammatical errors and improper word use. We propose a more general task, Academic Writing Formalization (AWF), to improve the overall quality of formal academic writing at the paragraph level. We formulate this language refinement task as a formal text style transfer task which transfers informal-academic text to formal-academic and contribute a large-scale non-parallel dataset, Doolittle, for this purpose. Concurrently, we apply a method named metric-oriented reinforcement learning (MORL) to two large language models (LLM) where we incorporate different levels of automatic feedback into the training process. Our experiments reveal that existing text transfer models and grammatical error correction models address certain aspects of AWF but still have a significant performance gap compared to human performance. Meanwhile, language models fine-tuned with our MORL method exhibit considerably improved performance, rivaling the latest chatbot ChatGPT, but still have a non-negligible gap compared to the ground truth formal-academic texts in Doolittle.

Doolittle:
Benchmarks and Corpora for Academic Writing Formalization
Shizhe Diao'* Yongyu Lei's, Liangming Pan', Tianging Fang', Wangchunshu Zhou', Sedrick Scott Keh', Min Kan Tong Zhang'
'The Hong Kong University of Science and Technology University of California, Santa Barbara "ETH Zürich "Toyota Research Institute NNational University of Singapore
(sdiaoaa, yleiah, tfangaa]@ccnnecc ust.hk tongzhang@ust.he hk
Abstract
[S]: We propose more sophisticated hierarchical model to include geographical informations.
Improving the quality of academic writing is a meaningful but challenging task. Conven- tional methods of language refinement focus on narrow, specific linguistic features within isolated sentences, such as grammatical errors and improper word use. We propose more general task, Academic Writing Formalization (AWF), improve the overall quality of formal academic writing at the paragraph level. We formulate this language refinement task as formal text style transfer task which transfers informal-academic text to formal-academic and contribute large-scale non-parallel dataset, DOOLITTLE, for this purpose. Concurrently, we apply method named metric-orienter rein- forcement learning (MORL) two pretrained language models (PLM) where we incorporate different levels automatic feedhack into training process. Our experiments reveal that existing text transfer models and grammatical error correction models address certain aspects of AWF but still have significant performance gap compared to human performance. Mean- while, language models fine-tuned with our MORL method exhibit considerably improved performance, rivaling the latest chatbot Chat- GPT, but still have neg-negligible gap pared to the ground truth formal- -ormal-academic texts DOOLITTLE!
[T]: We propose a more sophisticated hierarchical model to include geographical information.
[S]: This because the teaching and learning on science domain relies much on the ability of rea- soning and computation, which directly utilizes the advantage of computer.
[T]: This is because the teaching and learning on a scientific domain relies considerably on the ability of reasoning and computation, which directly uti- lizes the advantages of computers.
[S] METEOR is another n-gram overlap measure initially designed for evaluating machine translation systems. ROUGE-L a commonly-adopted metric for text summarization.
[T]: Both METEOR and ROUGE-L specialize in BLEU's n-gram overlap idea for machine trans- lation and text summarization evaluation, respec- tively.
Table 1: Informal-academic paragraphs with formal- academic rewrites, denoted S and T, respectively. The refined form is highlighted blue, the original in red.
(GEC) (Ng et al., 2014; Bryant et al., 2019) fo- cuses on identifying and correcting many of such grammatical errors. However, even if non-native speakers can write grammatically correct sentences, their language use is sometimes less concise and fluent than those written by native speakers (Lado, 1957; Rod, 1994). For example, the two source sentences in the third example shown in Table are grammatically correct but less fluent due to their structural redundancy. They sound more fluent when combined into a single sentence by adding an appropriate conjunction.
1 Introduction
Writing in a second language often leads to char- acteristic errors. In English, such eirors include subject-verb disagreement, noun nouner disagree- ment, and determiner misuse (Lado, 1957; Rod, 1994). Therefore, language refinement system with the ability to suggest or automatically cor- rect such errors is highly desirable (Yuan and Fe- lice, 2013; Rozovskaya and Roth, 2016). Towards this end, research in grammatical error correction
In light of this, we propose the novel task of Aca- demic Writing Formalization (AWF) that aims generalize the scope of GEC for language refine- ment: given an informal- paragraph P. the objective of AWF is to refine the language
"Equal Contribution.
'The datasets and code are available at https://githhb com/shizhediao /oolatttoooleeeeeo

---

P to make it grammatically correct, concise, and fluent, while preserving its semantics. Different from GEC, which solely concentrates on grammat- ical error correction for single sentence, AWF works on paragraph-level contexts and aims for refinements beyond grammar. This requires the model to comprehend and then rephrase the entire paragraph.
posed method: metric-oriented reinforcement learn- ing (MORL). We find that style transfer mod- els are unsuccessful discriminating the differ- ences between formal and informal text, resulting in lower scores for academic formality, perplex- ity, and meaning preservation, while GEC base- lines perform relatively better across all metrics but only marginally modify the inputs. On the other hand, BARTLarge (Lewis et al., 2020) and Galactica- 1.3B (Taylor et al., 2022) fine-tuned with our MORL approach provide comparable GEC results to ChatGPT, despite having significantly fewer parameters. Nonetheless, none could com- prehensively outperform the ground truth formal- academic paragraphs. It is worth noting that metric- oriented RL has been explored in the context of text generation, with some early studies (Wu et al., 2016; Choshen al.. 2020) using RL for neural machine translation optimized by BLEU, but with limited success. To the best our knowledge, we are the first to demonstrate that applying metric- oriented RL to PLMSs yields promising results, in- dicating that becrice-bsed RL is well-suited for powerful backbone models.
Specifically, AWF considers three objectives in academic writing formalization refine the lan- guage. 1) grammar correction: correcting gram- matical errors the paragraph, the objective of GEC. 2) word refinement: replacing inaccurate words and phrases with more accurate and con- cise ones. Evidence has shown that there are vi- tal differences in vocabulary usage between native and non-native writers (Malmasi et al., 2017). For example, we replace "science" and "much" with "scientific" and "considerable" in Table I's second example. 3) structure modification: changing the sentence or paragraph structure to convey the mean- ing more concisely. For example, the third exam- ple in Table I combines two similar sentences into a single long sentence to convey the information more efficiently.
We summarize our contributions as follows: 1) We propose novel setting for paragraph-leve lan- guage refinement, formulating it as a text style transfer problem. 2) We construct DOOLITTLE, the first large-scale dataset for academic writing formalization. Considering that AWF is com- case of LLMs such ChatGPT, we be- lieve DOOLITTLE can serve good testbed for benchmarking LLMs. 3) We propose method, metric-oriented reinforcement learning (MORL), and show its effectiveness and cost-efficiency tuning PLMs. 4) We conduct a comprehensive evaluation of neural approaches on our task and show that their performance still suffers from a siz- able gap compared to formal-academic rewrites by humans. This highlights the need for the dataset and the AWF task.
Although there exist several large-scale corpora for GEC (Yannakoudakis et al., 2011: Tajiri al.. 2012: Mizumoto et al., 2012; Dahlmeier et al., 2013; Napoles et al., 2017; Bryant et al., 2019), they either focus on word/phrase/sentence level text refinement or do not target scientific texts, which makes none of them available for AWF. We thus construct DOOLITTLE', a large-scale, non- parallel dataset containing 55.6K formal-academia paragraphs and 13.0K informa--academic ones. DOOLITTLE is based on the Semantic Scholar Open Research Corpus (S2ORC: Lo al., 2020). For each paragraph, we ask human annotators to rate the academic formality of the language via crowdsoureing. Expert annotators then refine the language for around 900 paragraphs to obtain parallel corpus that serves as the development and test sets of DOOLITTLE.
Related Work
To investigate the performance of sate-of-thhear models for AWF, we adopt four baseline models of text style transfer, two baseline models from low-resource GEC, the widely- large lan- guage model (LLM) ChatGPT, and two pretrained language models (PLM) fine-tuned with our pro-
Language Refinement. There are two tasks typ- ical of language refinement, both focusing on en- hancing the quality of sentences. Post-editing (No- vak al., 2016; Xia al., 2017; Guu al., 2018: Freitag et al., 2019) designed to rectify typical errors in machine translation, thereby augment- ing the generation quality, as measured by BLEU. The other task, Grammatical Error Correction
Nameed after the lower-class protagonist from the English film My Fair Lady, who undergoes training to transform her accent and manners into one of proper lady.

---

(GEC), is formulated parallel translation task with phrase-based machine translation (PBMT) models (Rozovskaya and Roth, 2016; Junezys- Dowmunt and Grundkiewicz, 2016), neural ma- chine translation (NMT) models (Chollampatt and Ng. 2018; Junczys-Dowmunt et al., 2018), and hybrid PBMT-NMT models (Grundkiewicz and Hunezys-Dowmunt, 2018). However, these meth- ods require large amounts of parallel data which are expensive to annotate. To address this, low- resource GEC (Bryant et al., 2019) builds models that do not rely on large parallel data. Choe et al. (2019), Grundkiewicz et al. (2019), and Zhou et al. (2020) initially pretrain a transformer model with large synthetic parallel corpora which are generated from a realistic noising function and then fine-tune this model small in-domain parallel dataset.
3.1 Data Source
Our focus lies on scientific texts, encompassing both published articles and preprints as our pri- mary sources. These scientific articles are typi- cally of high quality, usually having been proof- read, allowing models to focus improvements in terms of lexical choice and sentence structure. We use the Semantic Scholar Open Research Corpus (S2ORC) (Lo et al., 2020), a large corpus of 81.1 million English scientific papers spanning many academic disciplines including medicine, biology, computer science, and so on. There are four rea- sons for choosing S2ORC: 1) It is clean dataset of scientific papers, which are of good quality with- out trivial mistakes; 2) It exposes rich metadata, inclusive of paper titles, authors, published venue, and year of publication; 3) It provides full text for 81.1 million open access papers without copy- right issues, so that both the distribution of the data and replication of our work are possible without infringement; 4) The full text preserves meaning- ful structures such as paragraph breaks and section headers, so that the text is easily extracted.
Reinforcement Learning from Human Feedback (RLHF). As notable advancement within the realm of reinforcement learning (RL), reinforce- ment learning from human feedback (RLHF) in- tegrates human feedback into the training process. This approach trains a model to align more closely with user intentions, thereby equipping the model with the ability to generate more reliable, authen- tic, and useful results (Ziegler et al., 2019; Ouyang al., 2022; Dong et al., 2023). RLHF manifests its advancement and convineing capability in the recent state-of-the-arr chatbot ChatGPT (OpenAI, 2022). The fundamental workflow of RLHI can be succinctly summarized in three steps below: Step 1: Train a policy model with supervised training on collected demonstration data. Step 2: Train a reward model on collected comparison data. Step 3: Optimize the policy against the reward model using reinforeement learning with proximal pol- icy optimization (PPO) algorithm (Schulman et al., 2017).
3.2 Academic Formality Annotation
We now describe how we set up the annotation crowdsourcing tasks to mark the academic formal- ity of each paragraph. We first randomly sample a subset of 90,000 short paragraphs which composed of more than 2 sentences with lengths between 20 to 100 words. Then we classify them into formal- academic and adopt unbiased means that ignores the paper thors and instead asks human annotators to rate the academic formality of each paragraph via crowd- sourcing.
paragraphs. We
Annotation Task Overview. Each annotation task contains 100 paragraphs. Annotators are asked to score each paragraph from (sounds informal- academic) to 5 (sounds formal- -cmaal-ccdemmic.. For any paragraph that contains incomplete sentences. an assignment of 0 is acceptable. We provide a de- tailed annotation guideline to illustrate the stan- dards for different scores. For example, a part of the standard for rating a score of 2 is as follows: "The language expression is unclear that you cannot fully understand the meaning... We had four ex- perts to construct a quality control test consisting of 500 examples, which we randomly inject into each task. The detailed descriptions for each score with corresponding examples and gold set construction
3 Dataset Construction
We present DOOLITTLE, a corpus of academic formality non-parallel texts from scientific paper sources ($ 3.1), where we manually annotate aca- demic formality by crowdsourcing to obtain large set of non-parallel training paragraphs in two styles ($ 3.2). We then conduct second annotation task called Cormal-academin rewrite in order to create a small parallel dataset for evaluation ($ 3.3).

---

V#
Avg. Words
Avg. Sent.
ACC-cola
ACC-aesw
PPL
SIM
ED
BARTS
FA
55.6K
172.8K
84,3K
51.42
3.11
97.56
79.64
24.44
Train
IFA
13.0K
41.3K
38.9K
52.17
3.17
95.81
68.51
32.56
FA
465
1359
5.2K
47.33
2.92
98,49
78.27
31.19
Dev
98.75
11.03
-1.19
1362
5.3K
47.79
2.92
95.69
72.04
33.07
FA
415
927
4.4K
42.52
2.23
98.31
77.83
33.18
Test
98.09
10.87
-1.24
IFA
415
910
4.5K
43.08
2.19
95.66
69.64
35.97
The statisties of the DOOLITTLE dataset, where P#, S#, V#, Avg. Words, and Avg. Sents. refer the number of paragraphs, number of sentences, vocabulary size, average words per paragraph, and average sentences per paragraph, respectively. We also report the transfer accuracy (ACC), perplexity (PPL), Semantic Similarity (SIM), char-level edit distance (ED), and BARTScore (BARTS). FA and IFA denote formal-academic and informal- academic, respectively.
Table 2:
are shown in the Appendix A.
the quality of annotations. Any worker who peatedly performs poorly (i.e., the rejection rate is above 50% and he/she has done over tasks) will be eventually blocked from our tasks.
Publishing Annotation Tasks. We conduct an- notation on the Amazon Mechanical Turk (AMT) platform. For each annotation task, we randomly sample a paragraph from each of the five scores in the gold set for total of 5 paragraphs with their corresponding gold scores. These used test cases for quality control and are amalgamated with 95 unannotated paragraphs. Each task is assigned to two annotators independently. Annotators are given 7 hours to complete the task. To refine the cohort of workers that are eligible to complete our task, we impose restrictions to include only annota- tors who are located in primarily English speaking countries, and who have finished at least 100 tasks before on the AMT platform with an approval rate above 90%.
Constructing DOOLITTLE. We post-process the annotations into binary scores 0 (informal- using the
academic) or (formal-academic) following rules. Here, we define S1 and S2 as the scores given by Annotators and 2, respectively.
Incomplete: 51 = 0r S2 =
Informal-academic: < S1 < a and 0 <
S2 < a
-emaal-cademic: $1 > and $2 >
Others: the remaining data whose scores do not hold the above standards.
Quality Control. We have the following stan- dards to control the annotation quality:
Paragraphs categorized under Incomplete and Oth- would be filtered out because want cleaner dataset with high mutual agreement. is set to 2 according to our definition of the scores, as indicated in the Appendix A.2. We thus obtain a large set of non-parallel paragraphs in two styles (Table 2): farmal-academic and informal-acadent We randomly select 500 informal-academic para- graphs for development and another 500 for testing. The remainder is used as training set.
Time spent on each task should be greater than 500 seconds.
Variance should be greater than threshold 6 to ensure not all scores are the same.
n 03
VAR
11)
(1)
The allowed discrepancy with gold set annota- tions (defined below) must be smaller than thresh-
old e2
Annotation Results. We evaluate the disagree- ment between two annotators to check the quality of annotation by Disagreement = S51- We observed that 43.30% annotations have the same scores (disagreement is 0) and the disagrecment of about half (56.70%) annotations is 1, which indi- cates there a high rate of agreement. addition. the Cohen's Kappa coefficient between two annota- tors is 0.657, showing a strong agreement between the two annotators' scoring.
GAP Σ |annotation; gold, < E2
where gold; denotes the score of i-th test case and annotation; is its corresponding annotation. Annotations that can not meet all of the above stan- dards are rejected, and we provide the workers with detailed reasons of rejection to help them improve

---

3.3 Test Set Construction
(Wieting et al., 2019) capture the semantic simi- larity. The similarities between a transferred para- graph and its input are reported as SIM-input, while the similarities between the transferred paragraph and its human rewrite reference are denoted as SIM- gold.
Our methodology produces large, non-parallel, binary-annolater corpus. To obtain small parallel development and test set for evaluation, we then conduct formal-academic rewrites to produce a set of paragraph pairs. To ensure the quality of the development and test set, the two native speakers involved in the construction of the gold set, who are quite familiar with the standards of academic formality and possess a thorough understanding of our task, are asked to rewrite informal-academic paragraphs into formal-acadent sequently, two authors of this paper reviewed the rewrites and rejected those that do not meet the required standards. The average time consumed for a rewriting task containing 100 paragraphs is 220 minutes. During this process, incomplete para- graphs were identified by annotators and removed from the final development and test set. The final statistics are shown in Table 2.
BARTScore (BARTS) BARTScore (Yuan et al., 2021) is a metric that formulates the evaluation of generated text text generation task from the pre- trained language model BART (Lewis et al., 2019). BARTS can outperform other existing metrics in the evaluation of text from different perspectives. such as fluency, accuracy and integrity. A higher BARTScore indicates that the reference paragraph is more likely generated from the input paragraph using pre-trained BART model.
paragraphs. Sub-
4.2 Quality of Formal-academa Rewrite
The quality of the parallel dataset consisting of informal-academin paragraphs and corresponding formal-academic rewrites critically important for evaluation, so we examine this subset from four aspects: 1) academic formality improvement, 2) fluency improvement, 3) semantic similarity and edit distance, and 4) BARTScore. As illustrated in Table 2, the ACC-cola scores on the develop- ment and test sets have shown improvements of 2.8 and 2.65, respectively. In the case of ACC- aesw scores, similar upward trends are observed, with boosts to 6.23 and 8.19, respectively. Mean- while, the PPL. of the formal-academic rewrites decreases by 1.88 and 2.79 when compared with the informal-academic paragraphs. The increased academic formality and reduced PPL show that the formal-academic rewrites indeed improve aca- demic formality and fluency. Lastly, DOOLITTLE has high semantic similarity and low edit distance, implying that the original paragraphs are of good quality and minimal modifications are performed. This shows that academic writing formalization is a challenging task that requires more fine-grained lexical and structural modifications.
4 Dataset Analysis
4.1 Automatic Evaluation
Transfer Accuracy (ACC) To capture the transfer success of academic formality in para- graph level, following Krishna et al. (2020), two RoBERTa-Large (Liu al., 2019) models are fine- tuned on the CoLA corpus (Warstadt et al., 2019) and the automated evaluation of scientific writing shared task 2016 dataset (AESW) (Daudaravicius, 2015), serving as two academic formality classi- fiers respectively. The transfer accuracy on gen- crated paragraphs is reported as ACC-cola and ACC-aesw separately, measuring the acceptability of paragraphs. In addition, to capture the word- gram level transfer accuracy, we adopted GLEU (Napoles et al., 2015) and SARI (Xu et al., 2016) which are commonly used in GEC and text revision tasks.
Fluency (FI.) To measure the fluency of the gen- erated paragraphs, we use perplexity (PPL), fol- lowing the fluency evaluation in Dai al. (2019) and Cao et al. (2020). We fine-tune a pre-trained GPT-2-LLarge language model (Radford et al., 2019) on the formal-academic training set and use it to calculate the perplexity in generated examples.
4.3 Common Mistakes and Types
To understand what types of mistakes are common in anformcl-academic paragraphs and what edits have been made in the formal-academic rewrite process, we analyze all of the native rewrites in the test set (Figure gives examples with their corre- sponding native human rewrites). The major error types parallel our objectives of academic formality, as introduced in $ 1. And we observe high per-
Semantic Similarity (SIM) Following previous benchmark (Krishna al., 2020), we replace n- gram metrics like BLEU (Papineni et al., 2002) with the subword embedding- based SIM model

---

Grammar [S]: When apply Naturalistic Driving Film into the design process [..]]
Grammar [T] When applying Naturalistic Driving Film into the design process [..]
Spelling [S]: In this article we have tried to sumarize advantages that zebrafish can offer immunological research.
Spelling [T] In this article we have tried to summarize advantages that zebrafish can offer for immunological research.
Word Choice [S]: In a first analysis, speed behaviour was found to be non-linear according to Figure 2
Word Choice [T] an initial analysis. speed behaviour was found to be non-linear according to Figure 2
Redundancy [S]: The final flight was again a heading control test to verify once more the performance of that channel.
Redundancy [T]: The final flight was a heading control test to verify once more the performance of that channel.
Sentence Structure [S]: We want to show that some solutions of this equation do not exist in the future.
Sentence Structure [T]: In the future, we want to show that some solutions of this equation do not exist.
Figure I: Examples of common types of mistakes in the negative ground truth data. 'S' denotes a source paragraph which is written by a non-native speaker and "T" denotes the target paragraph written by native speaker. We highlight the refined part in blue and original expression in red.
centage of grammar and spelling errors (46.48%), word choice issues (39.31%), 3)),, as well sentence structure changes (14.00%).
reward model that can score given policy model's output with a scalar. Step 3: Optimize the policy against the reward model using reinforcement learn- ing with the proximal policy optimization (PPO) algorithm (Schulman et al., 2017).
Grammar and spelling. This is the most com- mon type of error and is also the easiest to iden- tify. Many sentences contain grammatical errors in subject-verb agreement, verb tense, and capital- ization. Additionally, some sentences also contain misspellings or typographical errors.
The key distinction between RLHF and MORL lies in Step 2 where RI.FF trained a reward model with collected comparison data while MORL utilizes any existing, machine learning- -based not, tuned or plug-and-play evaluation metries to generate a reward model. Through incorporating a variety of evaluation metries into the reward model, the cost of implementing MORL becomes flexible and the potential misalignment between human preference and a single metric can be alleviated.
Word choice. Many sentences are awkwardly phrased. Although the usage of word may make sense in the original sentence, a more appropriate word may be substituted to make the sentence more fluent. Redundancy is also problem, some sen- tences are overly discursive when describing ideas that can be concisely explained.
5.1 Policy Models
Sentence structure. In certain cases, a modifier may be misplaced or in an incorrect order. There are also a number of instances where the sentences are too short or too long. In these cases, it would be better to combine the short sentences and split the long sentences.
Galactica-1.3B (Taylor et al., 2022) is decoder- only policy model. We train Galactica-1.3B model on the paragraph pairs of the format [[paragraph A]}=[[paragraph B]] twice. For the first time, paragraph is sampled from the formal-academic training set of DOOLITTLE, and paragraph B is exactly duplication of paragraph A. For the second time, paragraph A and paragraph B are paragraph pair from the development set of DOOLITTLE, where paragraph A is informal- academic and paragraph B its refined version. After these two stages, the Galactica-1.33 model learns to improve the left paragraph and put the refined result on the right while preserving most of the original content. In the end, after final prompt of [[paragraph A]]=, we sample from the model with beam search (number of beams=4) and extract the content within the second wuuble-bracket-brace as the refined version of paragraph A.
5 Method
It could be speculated from our dataset analysis results ($ 4) that our DOOLITTLE task is advanced in terms of both quality and difficulty. To address our task with reduced cost and better performance, we proposed a method called metric-oriented re- inforcement learning (MORL). This methodology, inspired by reinforcement learning from human feedback (RLHF) (Ziegler et al., 2019; Ouyang et al., 2022), follows similar three-step training process RLHF but with crucial modifications: Step 1: Train policy model (usually a PLM) that can meet the requirements of task. Step 2: Select some metrics that can accurately evaluate the qual- ity of how the task has been performed. Build a
BA LARgee (Lewis et al., 2019) is a born strong baseline for GEC task (Katsumata and Komachi, 2020). We selected the BART-Large model with

---

Academic Formality
Fluency
Similarity
BARTS
Metric
ACC-cola
ACC-aesw
SARI
GLEU
GPT-4
GPT-4
SIM-input
SIM-gold
GPT-4
BARTS
Input
95.66
69.64
4.32
35.97
4.55
98.09
Style Transfer Models
ControlledGen
92.77
48.19
48.59
54.54
3.87
60.87
4.13
95.21
93.62
4,20
-1 1.64
DeeplatentSequence
84,81
50.36
37.46
50.40
3.55
68.45
4.15
90.45
88.97
3,78
-2.06
Style Transfoorrr
85.30
56.63
38.46
50.87
3.96
66.87
4.38
90.27
88.79
3.64
-2.19
Dedetivndeeerievee
66.50
66.02
7.98
1.07
2.91
34,11
3.36
21.12
20.27
2.22
-5.90
GBC Models
SequenlaalTransfer
94.70
49.17
71.30
4.45
95.55
BART-GEC
95.90
69.10
74.72
4.40
35.83
4.66
97.24
-2.14
Instruction Tuned Models
ChatGPT 99.20
82,56
48.34
70.21
4.58
28.84
4.81
94.58
94.87
4.73
-1.62
MORL-BARTLarge
97.83
78.80
55.74
75.75
4.57
35.65
4.78
98.49
97.45
4.35
-1.32
MORI-Galactical.33 BB
97.83
80.24
63.79
78.37
4.60
34.50
4.86
98.72
98.30
4.70
-1.34
Native Rewrite
98.31
77.83
4.59
33.18
4.89
98.09
4.95
-1.24
Table 3: Results of models on DoOLITTLE test paragraphs. Automatic evaluation and GPT-4 judgments of academic formality, fluency, and meaning preservation are reported. highest scores of each metric among three instruction- tuned models are bolded. Some metrics comparison against these two sets, which are marked by ?-
applicable for Input and Native Rewrite they derived from
406M parameters, fine-tuned it paragraph pairs in the development of academic writing formal- ization, and used the tuned model as the second policy model.
on DOOLITTLE to establish baseline performance. We use the official code released by the authors of baseline (except ChatGPT), and follow the recommended configuration settings from their cor- responding papers. We report implementation de- tails and hyper-paramette settings of the different models with their size and running speed in the Appendix.
5.2 Reward Model
To make a more comprehensive reward model, we amalgamate all four metrics mentioned $ 4.1. For Transfer Accuracy (ACC), instead of using the binary classification result, we utilize the classifica- tion logits as the ACC score, and only ACC-aesw is used. For other metrics (PPL, SIM-input, and BARTScore), directly take the unchanged evalu- ation results. Each time our policy model generates an output (usually a single paragraph), the reward model first evaluates utilizing all four metrics. Following this, weighted sum of all evaluation scores is calculated as the final reward. The weights assigned to each metric are manually determined and optimized through series of experiments.
6.2 GPT-4 Annotation
GPT-4-based annotation has been proven be ef- fective in multiple text annotation tasks (Gilardi et al., 2023; Zheng et al., 2023). Considering the advantages GPT-4 performance and cost- effectiveness compared to human annotation on MTurk, we apply GPT-4 to evaluate the refinement results of all models on the DOOLITTLE test set. Corresponding to the automatic evaluation metrics, three annotation tasks are assigned GPT-4 where each focuses on one of the three aspects: Academic Formality, Fluency, and Similarity versus input. For each annotation task, we first feed GPT-4 a comprehensive task description including the task introduction, grading rubrics, the expected input format, and the asked output format. Then, the texts given as formatted batches of paragraphs, and the evaluation scores are fetched via scripts from GPT-4's response. For each model, we sampled the first 100 paragraphs from its generation results on the informal-accdemic DOOLITTLE test set. Addi- tionally, we also sampled the first 100 paragraphs
6 Experiment
6.1 Experimental Settings
We apply nine models four from non parallel text style transfer (ControlledGen (Hu et al., 2017), DeepLatentSequence (He et al., 2020), Style- Transformer (Dai al., 2019), and DeleteAn- dRetrieve (Li et al., 2018)), from low-resource (Choe al., 2019)
GEC tasks (SequentialTranss and BART-GEC (Katsumata and Komachi, 2020)), one from ChatGPT, and two MORL-based models

---

from both formal-academic and informal-academic academic writing formalization test sets, aiding nuanced analyses of each model's performance. These GPT-4 annotation scores are reported with a 5-scale value Table 3. Appendix gives the detailed task description.
ence between ChatGPT and MORL-Galactica (I.3B) or MORL-BARTLarge (406M), our MORL method exhibits remarkable advantages in both per- formance and cost-efficiency.
In summary, only the three reinforcement- learning-biis braninggbbsedd models demonstrate a clear com- petency in enhancing the original inputs in terms of both academic formality and fluency metrics. Nev- ertheless, none of the models consistently surpass the ground truth formal-academic rewrites across all four metrics. This supports the idea that aca- demic writing formalization (AWF) indeed a difficult task that goes beyond simply correcting grammatical mistakes.
6.3 Overall Performance
Table 3 reports the evaluation results. First, we ob- serve that all models targeted for generic style trans- fer task - ControlledGen, DeepLatentSequence, Style Transfomeer and DeleteAndReetreve per- form much worse than the inputs across all metrics. Second, the results demonstrate that GEC-based models - namely SequentialTransfer and BA BART- GEC - outperform style- byyle-transfer-bee based models and yield results that are slightly inferior or com- parable to the inputs. This is consistent with our expectation, as simply improving grammar by edit- ing will make only incremental changes to given sentence. Therefore, style-transfer transfer-based models lack the ability to preserve the paragraphs' orig- inal meaning and may make redundant changes which result poor "refinement". Third, the eval- and Se-
6.4 Case Study
To further analyze the generation quality, we ex- amined all input paragraphs together with every baseline's ' corresponding output. Table 8 shows some representative sample. We observe that while all models can generate complete and fluent text, they also possess specific limitations: DeleteAn- dRetrieve generates the text in a formal-academic way with appropriate sentence structure but strug- gles with preserving meaning; ControlledGen. SeeplatentSequece,, and Style Transorraserr can not provide any actual improvements to the input, they either prefer to stay the same the original in- put or may modify the original semanties; Sequen- tialTransfer and BA BART-GEC can sometimes suc- cessfully make necessary changes make the in- put more -ccdemcc however, most of the time, they are incapable of modifying more gram- matical errors; ChatGPT, MORL-BARTLarge Largee MORI-Gallcticc11333 provides convincing re- fined paragraph with necessary refinements. How- ever, either some errors ignored the tence's 's original meaning is changed which results in non-perfect rewrite. It can be clearly observed that all the above models still perform worse than -acmal-academic human rewrites, thus indicating the difficulty of our academic writing formaliza- tion task. Additional cases are included in the Ap- pendix F.
uation outcomes of Transfer reveal the misalignment between automatic evaluation metrics and GPT-4 annota- tion scores, which can be attributed to the inherent limitations of these automated metrics. No single metric is able to perfectly capture the essence of the academic writing formalization task, as each metric limited in what it measures and therefore lacks the ficxibility that humans or GPT-4 possess to holistically evaluate the formality of a given text. Fourth, all eeinforemeent-eeannng-beeee
models ChatGPT, MORL-BARTI, Large and MORL- Galactica demonstrate superior performance in our academic writing formalization (AWF) task, outperforming all other models on almost all met- rics. Specificallyy when comparing to the DOOLIT- TLE formal-academic test paragraphs, both Chat- GPT and MORI-GGllactical.C generate compet- itively good refined paragraphs, comparable with ground truth formal-academic rewrites in terms of academic formality and fluency, but achieve lower scores for similarity versus input and BARTScore. MORL-BARTLAS Large performs slightly inferior to the other two reinforcement-lermmie basedd mod- els, but still largely outperforms all other non- leaarrin models as well as the inputs. Considering the substantial size differ-
6.5 Ablation Study
In this section, perform several ablations MORL-BARTLarge study how each metric used MORL as well the whole MORL module affects the performance of MORI--BRTTI aarge In these ablation experiments, we manually set the weight of one metric to zero, and then perform

---

Method
ACC-acsw
PPI.
SIM-input
SIM-gold
BARTS
BARTLarge w/o MORI.
74.70
38.39
99.19
96.74 74
-1.34
MORL-BAR
MORL-BARTLarge BARTLarge
78.80
35.65
98.49
97.45
-1.32
WORL-BARTLarg w/o ACC
75.18
36.10
98.49
97.25
-1.29
MORL-BARRTaarg w/o BARTS
78.55
37.90
97.97
96.86
-1.46
WOOR--BARLLar w/o PPL
77.83
41.15
98.68
97.53
-1.32
MORL-BARTLarge w/o SIM
78.80
35.61
97.74
96.67
-1.44
Table 4: Ablation studies of MORL-BARTI Large models. BARTLarge w/o MORL denotes the BART-Large policy model without MORL tuning. MORL-BARK BARTLage w/o denotes that the corresponding metric's weight is set to zero during MORL-tuning.
MORL tuning the BART-Large policy model described $ 5.2 with all other parameters the same as our optimized best setting. Thus, we can study the contribution of cach individual metric to the HORL-BARTLaay model. These results are shown in Table 4.
models. Even though such models do not outper- form the formal-academin paragraphs due the
difficulty of academic writing formalization, this promising result has substantiated the advantages in terms of competence and cost-efficiency of MORL.
In the future, we plan to incorporate external knowledge and weakly supervised information into the text rewriting process. We also assert that the full potential of MORL applied to large language models remains be tapped. It is foreseeable that through the inclusion of more relevant metrics and advanced modeling. MORL can enhance its capa- bility further and be adapted to broader spectrum of NLP tasks.
Comparing the results of the BART-Large pol- icy model with and without MORL-tuning, tell a noticeable performance gain across most of the automatic metrics except SIM-input, which in- dicates that MORL, effectively boosts the perfor- mance of the given policy model. Those variants of MORL-BARTLarye models, namely MORL- BARTLarge w/o, produced the worst evaluation score of specific metric among all MORL-tuned models without exception. This phenomenon re- veals that removing certain metric from the MORL-tuning process will prevent MORL from optimizing the policy model's properties which can be measured by that metric. Each metric incorpo- rated into MORL practical in helping MORL improve the policy model against the metric itself.
Limitations
First, as a dataset constructed based on Semantic Scholar Open Research Corpus (S2ORC) (Lo et al., 2020), DOOLITTLE inevitably inherits most limita- tions that have been discovered or potentially exist from its parent. This is especially true for the non- parallel training set where we do not collect any new data. For those who want to adopt our dataset in the future, a comprehensive study of S2ORC is needed. Second, for the evaluation of model per- formance, we observed the differences in some re- sults between well-performing - ChatGPT, MORL-BARTLarge and MORL-Galactieal.33 are indeed subtle. Considering the randomness in language model generation, our current evalu- ation metrics lack discriminability on our AWF task. Hence, we only draw indicative conclusions about the models' performance, such as "They are competitively good.... lacking more concrete de- tails. Third, prompt tuning has been applied several times in this paper. However, it is possible determine the most suitable prompt from the infi- nite space of prompt choices. Thus, despite all the prompts we mentioned being properly tuned from a series of experiments, we do not guarantee that our
7 Conclusion and Future Work
We propose a new setting for language refinement called Academic Writing Formalization (AWF), which bridges the gap between formal-academic and informal-academic writing. We contribute new dataset, DOOLITTLE, and evaluate nine baseline models for AWF using both automatic metrics and GPT-4 annotation, demonstrating that paragraph-Ievel refinement is a promising task with significant room for improvement.
To address AWF, we propose method called metric-oriented reinforcement learning (MORL).
Leveraging MORL, we successfully elevate the performance of BART-Large and Galactica-1.3B. yielding results comparable those of GPT-3.5 turbo-based ChatGPT, which possesses signifi- cantly more parameters than our MORL-based

---

methods fully exploit the potential of each model and the capability of GPT-4 annotation.
Acknowledgments
We thank the anonymous reviewers for their valu- able suggestions. This work was supported by the General Research Fund (GRF) of Hong Kong (No. 16310222). Shizhe Diao was supported by the Hong Kong Ph.D. Fellowship Scheme (HKPFS).
9 Ethical Considerations
The data and task presented in this paper are of a sensitive nature, as there is the concern of judging the English writing of scientific researchers. We ac- knowledge this and highlight some of the measures we have taken below.
References
Anonymity. We are aware that it is possi- ble to use a search engine to de-anonymize para- graphs. Our preliminary investigations yield that over 40% of our data do not yield any results Google. We also deliherately exclude metadata of the paragraphs further protect the authors' iden- tity. While acknowledging the potential of the dual use of DOOLITTLE, we believe that our dataset, when not actively abused and extensively searched on Google, can be useful for style transfer tasks. It would be misuse of DOOLITTLEM identify au- thors and use their traits to correlate with mastery of academic writing.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. 2020. Language models are few-shot learners. Advances neural information processing systems, 33:1877-1901.
Christopher Bryant, Mariano Felice, Distein E. Ander- sen, and Ted Briscoe. 2019. The BEA-2019 Shared Task on Grammatical Error Correction. In Proceed- ings of the 14th Workshop on Innovative Use of NI. for Building Educational Applications. pages 52-75. Florence, Italy. Association for Computational Lin- guistics.
Yixin Cao, Ruihao Shui, Liangming Pan, Min-Yen Kan, Zhiyuan Liu, and Tat-Seng Chua. 2020. Expertise Style Transfer: A New Task Towards Better Commu- nication between Experts and Laymen. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1061-1071, On- line. Association for Computational Linguisties.
Potential Error. Given the nature of our dataset construction with human annotators, there will be errors and disagreements in the annotation: cer- tain academic formality scores are inaccurate. We emphasize that these scores are not to be taken as any form of absolute judgment on any scholars' English writing ability.
Yo Joong Choe, Jiyeon Ham, Kyubyong Park, and Yeoil Yoon. 2019. A Neural Grammatical Error Correction System Built On Better Pre- tre-training and Sequential Transfer Learning. In Proceedings of the 14th Work- shop on Innovative Use of NLP for Building Educa- tional Applications, pages 213-227, Florence, Italy. Association for Computational Linguisties.
Academic Formality Classification. We present this dataset not only with the purpose of explor- ing academic formality classification,, but also as an extension to the task of writing style transfer. As such, this task can be broadened other types of text style transfer. To further focus on the aca- demic formality task, is also possible for one to construct a dataset of papers to emulate and use as positive examples.
Shamil Chollampatt and Hwee Tou Ng. 2018. Mul- tilayer Convolutional Encoder-Decodde Neural Net- work for Grammatical Error Correction. In 32nd AAAI Conference on Artificial Intelligence.
Leshem Choshen, Lior Fox, Zohar Aizenbud, and Omri Abend. 2020. On the weaknesses of reinforcement learning for neural machine translation. In Interna- tional Conference on Learning Representations.
Copyright. The S2ORC dataset is constructed from open-access academic papers without any copyright issues. Note that there is no licensing information that accompanies the original S2ORC d We thus infer that third-parties are al- lowed to use the data in view of the open-access status of the papers.
Daniel Dahlmeier, Hwee Tou Ng. and Siew Mei Wu. 2013. Building Large Annotated Corpus of Learner English: The NUS Corpus of Learner English. In Proceedings of the 8th Workshop on Innovative Use of NLP for Building Educational Applications, pages 22-31.
In light of these issues, plan to follow data release plan where DOOLITTLE will only be re- leased to researchers who explicitly consent to not de-anonymize any of the paragraphs.
Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. 2019. Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Repre- sentation. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics,
Phttss//gtthub.commllllmmi/szo

---

pages 5997-6007, Florence, Italy, Association for Computational Linguistics.
Marein Junezys-Dowmune Roman Grundkiewiez. Shubha Guha, and Kenneth Heafield. 2018. Ap- proaching Neural Grammatical Error Correction Low-Resource Machine Translation Task. In Pro- ceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol- ume (Long Papers), pages 595-6066 606, New Orleans. Louisiana. Association for Computational Linguis- tics.
Vidas Daudaravicius, 2015. Automated evaluation of scientific writing: Aesw shared task proposal. In Proceedings of the Tenth Workshop on Innovative Use of NL. for Building Educational Applications, pages 56-63.
Hanze Dong, Wei Xiong, Deepanshu Goyal, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. 2023. Raft: Reward ranked finetuning for generative foundation model alignment. arXiv preprint arXhv:2304.06767
Satoru Katsumata and Mamoru Komachi. 2020. Stronger Baselines for Grammatical Error Correction Using Pretrained Incoder-Decoder Model. arXiv preprint arXiv:2005. 1189.
Markus Freitag, Isaac Caswell, and Scott Roy. 2019. Text Repair Model for Neural Machine Translation. arXiv preprint
Kalpesh Krishna, John Wieting, and Mohit lyyer. 2020. Reformulating Unsupervised Style Transfer as Para- phrase Generation. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 737-762.
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. Chatgpt outperforms croke-workers for text- annotation tasks. arXiv preprint arXiv:2303.15056
Robert Lado. 1957. Applied Linguistics for Language Teachers. University of Michigan Press.
Roman Grundkiewicz and Marcin lunezys-Dowmunt. Dowmunt. 2018. Near Levell Performance in Grammat- ical Error Correction with Hybrid Machine Trans- lation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 2 (Short Papers), pages 284-290, New Orleans, Louisiana. Association for Computa- tional Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Ves Stoyanov, and Luke Zettlemoyer. 2019. BART: Denoising eqquence-t-ssqqeene pre-training for nat- ural language generation, translation, and compre- hension. arXiv preprint arXiv: 13461.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising equeenceto-seqqennc pre-training for natural language generation. translation, and com- prehension. In Proceedings of the 58th Annual Meer- ing of the Association for Computational Linguistics. pages 7871-7880 Online. Association for Computa- tional Linguistics.
Roman Grundkiewicz. Marcin Junczys- DDwryyst,Dmuuttt.. and Kenneth Heafell. 2019, Neural Grammatical Error Correction Systems with Unsupervised Pre- training on Synthetic Data. In Proceedings of the 14th Work- shop on Innovative Use of NLP for Building Educa- tional Applications, pages 252-263, Florence, Italy. Association for Computational Linguistics.
Kelvin Guu, Tatsunori B Hashimoto. Yonatan Oren, and Percy Liang. 2018 Generating Sentences by Editing Prototypes. Transactions of the Association for Computational Linguistics, 6:437-450.
Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer. In Proceedings of the 2018 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume / [Long Pa- pers), pages 1865-1874, New Orleans, Louisiana. Association for Computational Linguistics.
Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirten Kirkkaikkat 2020. A Probabilistic Formulation of Unsupervised Text Style Transfer. In International Conference on Learning Representations.
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Toward Con- trolled Generation of Text. Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 1587-1596, JMLR. org.
Yinhan Liu, Myle Ou, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretrain- ing Approach. arXiv preprint 197.:1999.111999
Marcin Junczys- Downmut and Roman Grundkiewicz. 2016. Phrase-basee Machine Translation is State-of- the-Art for Automatic Grammatical Error Correction. In Proceedings of the 2016 Conference on Empiri- cal Methods in Natural Language Processing, pages 1546-1556, Austin, Texas. Association for Computa- tional Linguistics.
Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kin- ney, and Daniel Weld. 2020. S2ORC: The Semantic Scholar Open Research Corpus. In Proceedings of the 58th Annual Meeting of the Association for Com- patational Linguistics, pages sociation for Computational Linguistics.
Online, As-

---

Shervin Malmasi, Keelan Evanini, Aoife Cahill, Joel Terrauul,, Robert Pugh, Christopher Hamill, Diane Napolitano, and Yao Qian. 2017. A Report on the 2017 Native Language Identification Shared Task. In Proceedings of the 12th Workshop on Innovative Use of NL. for Building Educational Applications, pages 62-75.
John Schulman, Filip Wolski, Prafulla Dhariwal. Alec Radford, and Oleg Klimov. 2017. Proxi- mal policy optimization algorithms. arXiv preprint arXiv:/707.00 063477
Toshikazu Tajiri, Mamoru Komachi, and Yuji Mat- sumoto. 2012. Tense and Aspect Error Correction for ESL Learners Using Global Context. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pages 198-202.
Tomoya Mizumoto, Yuta Hayashibe, Mamoru Komachi,
Masaaki Nagata, and Yuji Matsumoto. 2012. The
Ffffect of Learner Corpus Size in Grammatical Er- ror Correction of ESL. Writings. In Proceedings of COLING 2012: Posters, pages 863-872.
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. 2022. Galactica: A large language model for science. arXiv preprint wwxiv:2211.09055
Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2015. Ground truth for grammatical eттоr correction metrics. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Confer- ence on Natural Language Processing (Volume 2: Short Pupers), pages 588-593.
Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching. Tristan Thrush, and Nathan Lam- bert. 2020. Trl: Transformer reinforcement learning. https://githe https///ggituuu..oo///ooeee con/vvwwrra//tl.
Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2017. Jfleg: A fluency corpus and benchmark for grammatical error correction, arXiv preprint arviv: (xXx:1702.04065 702.04066.
Alex Warstadt, Amanpreet Singh, and Samuel R Bow- man. 2019. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625-641.
Hwee Tou Ng. Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christo- pher Bryant. 2014. The CoNLL-2014 Shared Task on Grammatical Error Correction. In Proceedings of the 18th Conference on Computational Natural Language Learning: Shared Task, pages 1-14.
John Wieting. Taylor Berg- Kirkpatrick, Kevin Gimpel, and Graham Neubig. 2019. Beyond BLEU: Training Neural Machine Translation with Semantic Similar- ity. In Proceedings of the 57th Annual Meeting of the A Associatinn for Computational Linguistics, pages 4344-4355.
Roman Novak. Michael Auli, and David Grangier. 2016. Iterative Refinement for Machine Translation. arXiv preprint arXiv:1610.0660 1610.66002.
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoe V Le, Mohammad Norouzi. Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, al. 2016. Google's 's neural machine translation system: Bridging the gap between human and machine trans- lation. arXiv preprint arXiv:1609.000
OpenAI. 2022. Introducing chatgpt. https://apenai. com/blog/chatgot.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-2775
Yingce Xia, Fei Tian, Lijun Wu, Jianxin Lin, Tao Qin. Nenghai Yu, and Tie- Yan Liu. 2017. Deliberation Networks: Sequence Generation Beyond One-Pass Decoding. In Advances in Neural Information Pro- cessing Systems, pages 1784-1794.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing 7hu. 2002. BLEU: Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 31 3-1-318.
Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burde 2016. Optimizing sta- tistical machine translation for text simplification. Transactions of the Association for Computational Linguistics, 4:401-415.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. Ope- nAl blog, 1(8):9.
Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 201 A New Dataset and Method for Automatically Grading ESOL Texts. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 180-189.
Ellis Rod. 1994. The study of second language acquisi- tion. Oxford University Press.
Alla Rozovskaya and Dan Roth. 2016. Grammatical Er- ror Correction: Machine Translation and Classifiers. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 2205-2215.
Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. Bartscore: Evaluating generated text as text gener- ation. Advances in Neural Information Processing Systems, 34:27263-27277.

---

Zheng Yuan and Mariano Felice. 2013. Constrained Grammatical Error Correction Using Statistical Ma- chine Translation. In Proceedings of the 17th Confer- ence on Computational Natural Language Learning: Shared Task, pages 52-61.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang. Zhanghao Wu, Yonghao Zhuang, 7i Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang. Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena.
Wangchunshu Zhou, Tao Ge, Chang Mu, Ke Xu, Furu Wei, and Ming Zhou. 2020. Improving grammati- cal error correction with machine translation pairs. In Findings of the Association for Computational Linguistics: EMNLP 2020. pages 318-328, Online. Association for Computational Linguistics.
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Chris- tiano, and Geoffrey Irving. 2019. Fine-tuning lan- guage models from human preferences. arXiv preprint arXiv: 1rX00:990..0893.

---

Appendix
whether the paragraph was written by a non- native writer or not. You can fully understand the meaning he/she is trying to express without guessing. However, the language expression is rigid and unnatural (for example, Chinglish). A native writer usually won't 't express the same meaning in this way.
A Annotation Details
A.1 How to Determine Academic Formality
One potential avenue is to distinguish aca- demic formality by the country of the first au- thor's affiliation. However, we eschew this method for three reasons: 1) The first author does not necessarily contribute to all of the paper writing. 2) Even in primarily English- speaking countries, there are many who do not speak English their first language. 3) Determining academic formality by country involves the ethical issue of regional discrimi- nation. Instead, we adopt an unbiased means that ignores the paper authors and instead asks human annotators to rate the academic formal- ity of each paragraph via crowdsourcing.
Score 4 [somewhat formal-academic You are not 100% sure that the paragraph was writ- ten by a native writer. You can fully understand the meaning the paragraph is trying to express and the language is natural and fluent. The lan- guage expression is very consistent with the style of native writers. However, there are mi- nor parts in the paragraph that can be further improved to make it closer to formal-academic English. For example, replace certain word with another word to express the meaning more precisely.
Score 5 [formal-academic]] You 100% sure that the paragraph was written by a na- tive writer, which means you cannot rewrite it better than it. The language is very clear and fluent, completely in the style of native writers. You cannot rewrite better one.
A.2 Annotator's Instruction
The detailed descriptions are shown below to il- lustrate the meaning of different scores, which are presented with examples to annotators as well.
Score 0 [incomplete]. For any incomplete sentences, which is only sentence fragment, you may give it O score.
A.3 Gold Set Construction
Prior to large-scale annotation, four experts two of whom are native English speakers and the other two are authors of this paper (one is a native English speaker) - worked together to produce a gold set for quality control. The task is the same as the one introduced before. and the annotation guidelines are presented to the annotators. We provide sufficient training for these two external annotators, inclusive of discussion, to ensure consistency and quality, Conflicting annotations were discussed by all four annotators produce final rating. In the end, we construct a gold set consisting of 500 paragraphs as the probing set, which is injected into the large-scale annotation tasks for quality assurance in the crowdsourced annotation.
Score 1 [infornal-academiic). You are 100% sure that the paragraph was written by a non-native writer. The language expression is very unclear or weird that you have no idea what meaning he is trying to express even by guessing. Usually, there are serious grammat- ical errors, spelling problems. Redundancy problems.
Score 2 [somewhat informal-academic/, You are not 100% sure that the paragraph was written by a non-native writer. The language expression is unclear that you cannot fully un- derstand the meaning. However, you can guess what meaning he/she is trying to express. Usu- ally, the paragraph is fine with slight errors. For example, there are some spelling, punc- tuation, capitalization problems. Redundancy problem, which means there could be better expression, for example, split a long sentence into two short sentences.
A.4 Analysis of Annotation
Our data source, S2ORC, contains a diverse set of academic disciplines, whose resultant discipline distribution is shown in Figures 2 and 3. We draw two observations, both quite consistent with the overall distribution of the
Score [between formal-academic and aaoommll-ccddmic)) You are not confident

---

0.82%
INFORRALLACCDEEMCCDSSSCIIIEE
FORMAL-ACADEMIC DISCILINE
1.57
0.225%
Belogy
Medicne
1.529
1.81%
1.09%
5.65%
0.30%
C.300E
Macidne
Blology
0.19%
2.80%6
3.37%
6.76%
Engineering
Chemistry
Sa
2.54%
0.19%
5.27%
Engincering
Camp
Physics
Phypic
5.98%
Sodbinggy
Mathomatkss
5.18%
Sodialayy
Marhematics
Economks
Economiss
11.00%
Art
Art
Ecology
Geo
Prychology
Psycho our
3.12%
Matter als sai
3.19%
Materiah Sci
Geagraphy
Polticalscl
Po hical
Env Se
Phinoophy
Ph losophy
Hiltory
Hittry
Figure 2: Disciplines on informal-academt dataset
Figure 3: Disciplines on formal- dataset
original dataset, S2ORC: 1) There are 19 dis- ciplines in total and the distribution is similar and informal-
approaches are chosen as 1) they are highly related to our proposed academic formality transfer task; 2) they require non-parallel data or only limited parallel data. 3) ChatGPT is the current SOTA Chatbot whose performance in generic NLP tasks has been proven. Hence, they provide good starting point for approach- ing our proposed task. By analyzing our exper- imental results, highlight the challenges in- herent in this task and provide insights into fu- ture research directions. In this section, we pro- vide the details of baseline models and training settings.
between the academic datasets. 2) Medicine, Computer Science, Mathematics, Physics, and Biology are the five top fields of study.
We also analyze the difference between preprints without peer review and published papers after peer review. Using the metadata provided by S2ORC, we can infer that pa- per was published if it possesses an ACL ID (the unique ID for papers on the ACL An- thology). a DOI (Digital Object Identifier), or venue/journal information. Our results indi- cate that 83.5% of published paragraphs are identified as -ccdemic by annotators while this ratio is 81.1% for preprints. We also evaluate the academic formality score and PPL for these two splits and observe that pub- lished paragraphs have a higher academic for- mality score (82.80 versus 78.31) and lower PPL. (24.22 versus 26.47), demonstrating that published paragraphs sound more native and fluent. This is reasonable as authors make ex- tensive revisions after the review process and address many writing issues following the feed- back from reviewers. Therefore, peer review is a key process for improving writing quality.
B.1 Baseline Models
ControlledGen (Hu et al., 2017): a model combining variational auto-encoders (VAEs) and attribute discriminators to learn disentan- gled latent representations with designated se- mantics.
attetSSquene (He et al., 2020): a generative probabilistic model with few inde- pendence assumptions based on standard at- tentional sequence-to-sequenee approach and an encoder-decoder architecture.
Style Transorraserrmer (Dai et al., 2019): a Transformer-based model for learning con- tent and style vectors without parallel data by cyclic reconstruction.
B Experimental Settings
DeleteA (Li et al., 2018): an RNN-based model which firstly extracts content words by removing style-dependent phrases and then retrieves and integrates new phrases related to the target attribute into a fluent sentence.
We apply nine models - four from non-parallel text style transfer, two from low-resource GEC tasks, ChatGPT, and two MORL-based models on DOOLITTLE to establish a baseline perfor- mance. Other than our proposed method, these

---

SequentialTransfer (Choe et al., 2019): low-resource GEC method using realistic noising function to generate synthetic parallel corpora which are applied to pre-train a Trans- former model. Then the pre-trained model is adapted to the targeted dataset by fine-tuning.
ward model to get its scalar reward. For MORL-Galactical.33 we also use DOOLIT- TLE formal-academic development set as the input data source. However, instead of di- rectly feeding raw paragraphs to the policy model, we first pre-process the input paragraph to the format described in $ 5.1- [C raw paragraph ]] =. In the end, we extracted the paragraph within the second double-bracket brace from the raw generated output the input to the reward model. One more thing to mention is that, during the training process of MORL, we also calculate the KL-divergence between outputs from policy models before and after patpoorremme--laaiiin-pppmmaaaai to ensure the optimized model does not deviate too much from the original one.
BART-GEC (Katsumata and Komachi, 2020): baseline model from GEC which uti- lizes BART (Lewis et al., 2020) as a pretrained model and fine-tunes the model on the target dataset.
ChatGPT (OpenAI, 2022): a chatbot devel- oped by OpenAl based their large language model GPT-3.5-Turbo (Brown et al., 2020) which can handle a variety of text generation tasks in a AueestionAAnswent fashion with properly adjusted prompts.
To adapt ControlledGen, DeepLatentSe- quence, Style EtyleTransformer, and DeleteAn- dRetrieve to our task, we simply treat formal- academic and informal-academic as two dif- ferent styles and train the models by following the text style transfer pipeline with parallel training data. For Sequential Trans- fer, we follow Choe et al. (2019) to use a nois- ing function on several high-quality corpora training data
B.2 Hyper-parameter Settings
Table 5 reports the dyper-parameten we used for tuning our baselines and our models tuned with MORL. For each model, try combi- nations the
and report the one with the highest academic formality score in our paper. Each model is trained on Tesla V 100S-PCIE GPU with 32GB memory. Table reports the hyperparameter configurations for best rest-performing models of the baseline mod- els.
well as our generate synthetic parallel data in order to pre-train the transformer-basee model. Then we use parallel data from the development set to fine-tune the model. For BART-GEC, we follow Katsumata and Komachi (2020) to use BART as a pre-trained model and fine-tune the model using parallel data from the devel- opment set. For ChatGPT, we tested vari- ety of question templates and manually select the one that can make ChatGPT perform the best in our AWF task, which is mentioned in Appendix D. For MORL-GalacticaL.333 we first fine-tune two policy models following instructions in $ 5.1 from the pretrained models. Then, we optimize those tuned policy models using our reward model mentioned in 5.2 with the PPO algo- rithm implemented through Transformer Rein- forcement Learning (TRL) library (von Werra et al., 2020). For MORL-BARTLarge we input raw paragraphs of DOOLITTLE formal- academic development set to the policy model and feed the raw sampled output to the re-
Types
Values
Learning Rate Dropout Rate Batch Size Fmheeding Dimensions
10-6 10-0,10-3 ,3 10-5,10-4 10 0.1, 0.4.0.5
1,4,8, 16,32
128, 256, 300, 768
Max Input Length
100, 130
Metric Weight
x 10 10 4,5 10 3,00,
Table 5: The sypee--arrameter for tuning our baselines where Metric Weight is only applicable MORL tun- ing.
and
C Model Size and Running Speed
Table 7 reports the number trainable parame- ters and the inference speed (sentencex/sccond) of all models except ChalGPT on the bench- mark. The test is performed on Tesla V100S- PCIE GPU with 32GB memory.
D Description of ChatGPT AWF Task

---

CG
DLS
ST
DAR
SQ
BA
MB
MG
Max epochs
30
5
5000
500
30
1000
5
5
Best epochs
10
2500
300
15
500
2
5
Learning Rate
x 10
10
10
3 x 10-4
3 x 10
3 x 10
10
10-6
Dropout Rate
0.5
0.3
0.1
0.3
0.3
0.3
0.1
0.1
Batch Size
16
32
16
32
32
4
Embedding Dimensions
300
128
256
128
512
1024
1024
2048
Max Input Length
100
130
100
100
100
100
128
256
Table 6: The hyperparameter configurations for best-performing models of all models except ChatGPT. CG, ST, DAR. SQ, BA, MB and MG denote ControlledGenn, Style Trynsfrranfrrmrr Delete/ndRRetrieee, SequentialTransfer, BART-GEC, MORL-BARTL. and MORL-Galactical.33
CG
DAR
SQ
MB
MG
Dataset
P.
|
S.
P
S.
P.
S.
S.
P.
S.
P.
S.
P.
S. P.
S.
Test dutaset
141M
4,76
69M
2.23
134M
0.97
100M
0.67
123M
5,49
406M
2.13
406M
2.13
1.38
0.62
Table 7: The number of trainable parameters (P.) and the running speed (sentences/second, S.) on the test sets all models except ChatGPT. CG, ST, DAR, SQ. BA, MB, and MG denote ControlledGen, StyleTransformer, DeletcAndRetrieve, ScquentialTransfer BART-GEC, MORL-BARTLarg, and BaRRL-Glactical.3B
D.1 ChatGPT AWF Task
Score O: If the paragraph contains any incomplete sentence which doesn't sounds likes being written hy human. Score I: You are 100% sure that the paragraph by non-native speaker. The language expression very unclear weird which makes you have no idea what meaning the author is trying to express even guessing. Score 2: You think the paragraph was probably written by non-native speaker. The language expression unclear that you cannot fully understand the meaning. But you can guess what meaning the paragraph is trying express.
Help me refine the following paragraph within "a x" to
written
make it more formal-academic. Specifically, you should follow the steps below:
Stepl: Find and locate all grammatic mistakes in terms of grammar and spelling, word choice and sentence structure in the given paragraph. (Note that is OK if you can't 't find any mistake. In this case, just respond with the original given paragraph within "s =.)"
Step2: Correct all mistakes you found in Step without changing parts else in the paragraph.
Score You are not confident ahout whether the paragraph was written by non native speaker or not. You can fully understand what the paragraph is trying to express without guessing. However, the language expression is rigid and unnatural (for example Chinglish). A native English speaker usually won't express same meaning in this way.
Step3: Output the refined paragraph within "* " in your response without anything else.
Notice:
1) minimal changes to the original paragraph are preferred 2) you should try preserve the given paragraph's 's original meaning and sentence structure as much as possible.
Score think the paragraph was probably written by native speaker. You can fully understand the meaning the paragraph is trying to express and the language natural and fluent. The language expression is very consistent with the style of native English speakers. But there are minor parts in the paragraph that can be further improved to make it closer to native English. For example, replace a certain word with another word to express the meaning more precisely.
Here some examples:
example
example
example
Score 5: You are 100% sure that the paragraph was written by native speaker, which means you cannot rewrite it any better. The language is very clear and fluent, completely in the style of native English speakers. You cannot rewrite better one.
The given paragraph is: paragraph content
E Description of GPT-4 Annotation Task
Each time, you will be given a batch of 20 paragraphs with format below:
E.1 Academic Formality Annotation Task
You asked participate in text evaluation task whose main objective is to score the degree of "Academic Formality" for given paragraphs. Evaluating the degree of "Academic Formality" means judging whether a given paragraph sounds like paragraph written by native English speaker or not. The scale of Academic Formality score integer from to 5. The detailed scoring rubric is as below:
Paragraph I: <Content of Paragraph 1>
Paragraph 2: <Content of Paragraph 2>
Paragraph 20: <Content of Paragraph 20>>
You should only output json object that contains the fol- lowing keys: Paragraph ID, Scoring Reason and Academic Formality Score. Note for "Scoring Reason", you need to briefly claborate the reason why you grade the given para- graph such Academic Formality score within 20 words.

---

E.2 Fluency Annotation Task
<Content of The scale of Similarity score is an integer from to 5. The detailed scoring rubric is as below:
You are asked to participate in text evaluation task whose main objective to score the degree of "Fluency" for given paragraphs. Evaluating the degree judging whether the paragraph is consistent and coherent. The scale Fluency score is an integer from to 5. The detailed scoring rubric is as below:
Score O: If the given paragraph contains any incomplete sentence which doesn't I sounds likes heing written by human.
"Fluency" means
In this case, you can directly give the given paragraph score O without reading its reference paragraph. Score 1: The content of the given paragraph has nothing to do with its reference paragraph. The things they are trying express don't 't even helong to the same discipline area of rescarch. The sentence structure of given paragraph is not similar to its reference at all.
If the paragraph contains any incomplete sentence which doesn't tt sounds likes being written by human.
Score O:
Score 1: The paragraph is neither consistent nor coherent at all which makes you have no idea about what the paragraph is trying to express even by guessing. Usually. it means some parts of the paragraph are not related to others. Note that a paragraph with errors like logic bugs and contradictions should not be scored to 1 since the occurrences of these errors require the context has some relation.
Score 2: You can distinguish the given paragraph and its reference paragraph are talking about two unrelated things. However, you can tell what they are tying express belong a same discipline area of research because they share some similarities in
two of the aspects: vocabulary
usage, writing style sentence structure.
Score 3: You can distinguish the given paragraph and its reference paragraph are talking about two unrelated things. However, you can tell what they are trying express belong a same discipline or area of research because they share some similarities all aspects: vocabulary usage, writing style and sentence structure.
The paragraph is neither consistent nor coherent. but you can still have a brief idea about what the given paragraph is trying to express by guessing, The only difference between scores 1 and 2 is that for score 2. different parts of the given paragraph are more or less related to others. However, you can only guess at what the paragraph is trying to express as these parts are managed without logic. Score 3: You can fully understand what the given paragraph is trying to express without guessing. However, it is obvious that the content is neither coherent nor consistent. For example. you can easily find a contradiction within the paragraph even though you are not an expert in this discipline. For these paragraphs, you should be able rewrite them by adding. deleting, exchanging a few words but hard to keep original meaning.
Score 2:
Score 4: You can determine that the given paragraph and its reference paragraph are talking about a same thing and they share similar vocabulary and sentence structure. However, they may not share the same viewpoint or focus on the same aspect of that thing.
Score 5: The given paragraph and its reference are very similar, and they are talking about the exact same thing with exact same viewpoint and focus. In this case, you can only find few differences between the given paragraph and its reference, like some words being replaced with its synonyms, differences in tenses or some minor grammatic errors etc.
Score 4: You can easily understand what the given paragraph is trying express since content hoth coherent and consistent. However, you can still find some minor contradictions found by experts or native English speakers. For these paragraphs, you should be able to rewrite it to make it more consistent or coherent by adding. deleting. or exchanging few words while keeping its original meaning. Score 5: The content of the given paragraph is both coherent and consistent. For these paragraphs, you can't 't rewrite them to make them more consistent or coherent.
Fach time, you will be given batch of 10 paragraphs and their respective reference paragraphs with format below:
bugs that can only be
Paragraph I: <Content of Paragraph 1>
Reference paragraph for paragraph 1:
Paragraph I's reference paragraph>
Paragraph 2: <Content of Paragraph 2>
Reference paragraph for paragraph 2:
<Content of
Paragraph 2's reference paragraph>
Each time, you will be given a batch of 20 paragraphs with format below:
Paragraph 10: <Content of Paragraph 10> Reference paragraph for paragraph 10: Paragraph 10's reference paragraph>
<Content of
Paragraph 1: <Content of Paragraph 1>
Paragraph 2: <Content of Paragraph 2>
You should only output a json object that contains the following keys: Paragraph id, Scoring Reason and Similarity Score. Note for "Scoring Reason". you need to briefly elaborate the reason why you grade the given paragraph such a Similarity score within 20 words.
Paragraph 20: <Content of Paragraph 20>
You should only output json object that contains the following keys: Paragraph ID, Scoring Reason and Fluency Score. Note for "Scoring Reason", you need to brietly elaborate the reason why you grade the given paragraph such a Fluency score within 20 words.
Response "Yes, I'm ready" if you fully understand your task.
E.3 Similarity Annotation Task
F Examples for Case Study
You are asked to participate in text evaluation task whose main objective is to score the degree of "Similarity" for given paragraphs and their respective reference paragraphs. Evaluating the degree of "Similarity" means to judge whether the given paragraph is similar to its reference paragraph in terms of content, vocabulary usage and writing style.
Table 8 shows three more generated samples from our baseline models.

---

Model
Paragraph Example
Origin
In each subsections the effect of each parameters are analyzed and individually. In each subsections the effect of each parameters are analyzed and individually. In each way the effect of each paraneters are analyzed and individually. These results are explained in details in the subsequent subsections. These results show of the best parameter valuex for the corresponding [...] In each subsection the effect of each parameter are analyzed and individually. In each subsections, the effect of each parameters are anal and individually. In each subsection, the effects of each parameter are analyzed individually. In each subsection, the effects of each parameter are analyzed and individually. In each subsection, the effect of each parameter is analyzed and individually. In each subsection, the effect of each parameter is analyzed individually.
ControlledGen
DeepLatentSequence
StyleTransformer
DeleteAndRetrieve
SequentialTransfer
BART-GEC
ChatGPT
MORL-BARTLarge
MORL-Gallctical.3B
GroundTrutt
Origin
When apply Naturalistic Driving Film into the design process, there are several aspects need to take into consideration.
ControlledGen
When apply Naturalistic Driving Film into the design process, there are several aspects need to take into consideration.
DeepLatentSequeence
When apply longitudinally driving film into the design process, there are several aspects need to take into consideration.
StyleTrrnsffrmer
When apply Realist Driving Film into the design process, there are several aspects need to take into consideration.
DedeteendRerriiee
When the student's 's interest is the same as the most important thing.. ... When applying Naturalistic Driving Film into the design process, there are several aspects needed to take into consideration.
SequentialTransfer
BART-GEC
When apply Naturalistic Driving Film into the design process, there are several aspects need to take into consideration.
ChatGPT
When incorporating Naturalistic Driving Film into the design process, there several aspects that need to he taken into consideration.
MORL-BARTL.agee
When apply Naturalistic Driving Film into the design process, there are several aspects to need to take into consideration.
MORL-Gallcticcl..38
When apply Naturalistic Driving Film into the design process, there several aspects need to be considered.
GroudTruth
When applying the Naturalistic Driving Film in the design process, there are several aspects that need to be taken into consideration.
Origin
of histamine biosynthesis would be expected to he useful tool in the analysis of the biological roles of this biogenetic amine.
ControlledGen
of histamine biosynthesis would be expected to be useful tool in the analysis of the biological roles of this nearfields amine.
DeepLateatSequence
of gabaergic biosynthesis would be expected to be useful tool in the analysis of the biological roles of this enigmatic entity.
StyleTransformer
of histamine biosynthesis would be expected to be useful tool in the analysis of the biological roles of this pathogenesis.
DeleteAnddRetrieve SequentialTrrassee
present the results of the study of the different types of remifentanil.
... of histamine biosynthesis would expected to be useful tool in analysis of the biological roles of this biogenic amine.
BART-GEC
of histamine biosynthesis would expected to be useful tool in analysis of the biological roles of this biogenetic amine.
ChatGPT
of histamine biosynthesis would be expected to be useful tool in the analysis of the biological roles of this biogenic aminc.
MORL-BARTLLagg
of histamine biosynthesis would be be expected to useful tool in the analysis of the biological roles of this biogenetic amine. of histamine biosynthesis would be useful in the analysis of the biological roles of this biogenetic amine.
MORL-Gallatical.3B
Ground Truth
of histamine biosynthesis would be be-expected useful tool in the analysis of the biological roles of this biogenetic amine.
Table 8: Examples of refined results and academic writing formalization test set. Red and italic words indicates wrong usage in the original sentence, Blue and italic words are words refined by insertion or replacement while Green and italic words with strikethrough are words regined by deletion.


Reviews:
[{"review_id": "nS4QfRvM7f", "reviewer": "Reviewer_euze", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer euze,\n\nThank you very much for your comprehensive review and valuable feedback! We address your comments one by one as follows:\n\n**[Application scenarios]**\n\n> This study is for mentioning practical application scenarios, i.e., it does not mention how these methods can be applied to real scenarios in order to solve real problems. The experimental results were compared with other large models such as chat-GPT, as well as with essentially real formal academic texts in DOOLITTLE, but no comparisons were made between academic embellishments and manual academic embellishments on a particular text, so it is not clear what their practical value is.\n\nThanks for your comments! Our proposed task and dataset is an advancement for grammatical error correction (GEC). We hope our task could be applied to improve academic writing, given the inherent difficulty of academic writing, especially for non-native English speakers. In Section 3: Dataset Construction, we put a lot of effort into making our data more practical and aligned with academic writing, including data collection, quality control, and especially for our dev/test set, where all ground truth samples were manually annotated. Thus, we believe that our benchmark can well represent real-world scenarios and the experimental results on DOOLITTLE can reflect the practical values of all baselines. For practical application scenarios, our dataset can be used to train a model for automatically polishing academic writing, similar to Grammarly. \n\n**[Disciplines]**\n\n> At the same time, academic writing involves multiple disciplines, although there are academic texts from multiple disciplines in the database, it does not mean that there is no difference in the level of academic embellishment in various disciplines, and a comparison between various disciplines can be considered in the future.\n\nThanks for your advice! We consider the problem to be a trade-off between the universality and specialty of our dataset. In the current stage, we propose DOOLITTLE as a general Academic Writing Formalization benchmark where the utilization of a broader range of disciplines is included to highlight its comprehensiveness. We believe that studying the difference in the level of academic embellishment across various disciplines through comparison must be a promising direction for further study. We’ll explore this in future work.\nWe have updated our manuscript with these details and discussions. Thanks very much for your constructive suggestion!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper applies MORL to LLM, using essentially real formal academic texts from DOOLITTLE as a dataset and incorporating varying degrees of automated feedback in the training process. The paper's dataset is of high quality and the training method is relatively new, while the paper is more compatible with this journal.", "reasons_to_reject": "This study is for mentioning practical application scenarios, i.e., it does not mention how these methods can be applied to real scenarios in order to solve real problems. The experimental results were compared with other large models such as chat-GPT, as well as with essentially real formal academic texts in DOOLITTLE, but no comparisons were made between academic embellishments and manual academic embellishments on a particular text, so it is not clear what their practical value is. At the same time, academic writing involves multiple disciplines, although there are academic texts from multiple disciplines in the database, it does not mean that there is no difference in the level of academic embellishment in various disciplines, and a comparison between various disciplines can be considered in the future.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper focuses on a study of AWF tasks to embellish academic language, which helps to improve the quality of essays in formal writing. The paper introduces the AWF task to break through the limitations of traditional language touch-up methods. In order to solve the problem of formal text conversion, the paper also introduces the MORL method, which, by combining reinforcement learning techniques and metric optimization, enables the language model to be trained with varying degrees of automated feedback, thereby improving the quality of the generated formal academic text to address the challenge of academic writing quality improvement, and at the same time facilitates the automated generation and improvement of formal academic text.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "POtEZmYrJs", "reviewer": "Reviewer_GKgM", "summary": "", "questions": "- I'm just curious why the PPL scores from some (powerful) models are so high. (Table 3)\n- There is a gap in the average sentence number between the training and test set. Why does this gap occur? I fear that these are intendedly controlled.\n- How many different seeds are used in the experiment? Is the performance difference statistically significant?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer GKgM,\n\nThank you very much for your comprehensive review and valuable feedback! We address your comments one by one as follows:\n\n**[Comparison with other important works]**\n> I'm not sure about the novelty of the task characteristics. The statistics shown in Section 4.3 is informative, but I want to see the comparison with the existing language refinement benchmarks, including, e.g., JFLEG [Napoles+, 17]. I'm also curious about how the refinement is context-dependent, given that paragraph-level editing would be one important aspect of this task, and the paragraph length is somewhat short (1--2 sentences).\n\nThanks for your suggestion! There are several key differences between JFLEG and our model.\n\n[Granularity] The JFLEG dataset [1] confines text refinement to the sentence level, whereas our dataset involves modifications across multiple sentences. For example, in Table 3 of JFLEG, their annotation instruction specifically mentions, \"Please do not split the original sentence into two or more.\" In contrast, in our annotation instruction, we emphasized modifications to multiple sentences \"Usually, the paragraph is fine with slight errors. For example, there are some spelling, punctuation, and capitalization problems. Redundancy problem, which means there could be a better expression, for example, split a long sentence into two short sentences.\"\n\n[Domain] JFLEG collected annotations based on GUG (Grammatical/Ungrammatical) corpus [2]. which contains 3.1k sentences written by English language learners for the TOEFL exam. In contrast, our data source is S2ORC and our focus lies on scientific texts, encompassing both published articles and preprints as our primary source.\n\n[Data Size] JFLEG releases a dataset of 1511 sentences with parallel edits. In contrast, our proposed dataset pushes forward with a much larger non-parallel training data and a parallel development and test dataset. There are 55.6K formal-academic paragraphs and 13.0K informal-academic paragraphs.\n\nFCE [3] and NUCLE [4] contain minimal edits because they constrain the edits by error codes.\nLang-8 [5] is open to any internet user with a different proficiency level as Doolittle.\n\nWe have included the discussion and cited these related works (e.g., [2-5]) in our revised version. Thanks!\n\n[1] Napoles, C., Sakaguchi, K., & Tetreault, J. (2017). JFLEG: A fluency corpus and benchmark for grammatical error correction.\n\n[2] Heilman, M., Cahill, A., Madnani, N., Lopez, M., Mulholland, M., & Tetreault, J. (2014, June). Predicting grammaticality on an ordinal scale.\n\n[3] Yannakoudakis, H., Briscoe, T., & Medlock, B. (2011, June). A new dataset and method for automatically grading ESOL texts. \n\n[4] Dahlmeier, D., Ng, H. T., & Wu, S. M. (2013, June). Building a large annotated corpus of learner English: The NUS corpus of learner English.\n\n[5] Tajiri, T., Komachi, M., & Matsumoto, Y. (2012, July). Tense and aspect error correction for ESL learners using global context.\n\n\n**[GPT-4 ratings]**\n\n> Evaluation with GPT-4 was used without validating its appropriateness. At a minimum, shouldn't you show that source and target sentences have an appropriate difference in GPT-4 ratings?\n\nIn Table 3, we presented the GPT-4 annotation scores in terms of Academic Formality, Fluency, and Similarity of the formal-academic and informal-academic paragraphs in the DOOLITTLE test set. It is observed that there is a noticeable gap in Academic Formality and Fluency scores between those 2 sets despite their similarity. These results, along with the recent studies [6-7] in GPT-4 annotation (mentioned in Section 6.2) demonstrate the validation of GPT-4 annotation while reflecting the difficulty of our DOOLITTLE benchmark. \n\n[6] Gilardi, F., Alizadeh, M., & Kubli, M. (2023). Chatgpt outperforms crowd-workers for text-annotation tasks.\n\n[7] Zheng, L., Chiang, W. L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., ... & Stoica, I. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena.\n\n**[Reference-based metrics]**\n\n> Why don't you use reference-based metrics such as GLUE? If you have some concerns about them, you should have some discussion or evaluation of the metrics on this task.\n\nThanks for your suggestions. In our original setting, we applied the semantic similarity as a reference-based metric to measure the similarity between predictions and references following [8]. Following your advice, we added more evaluation results in terms of SARI and GLEU. The results are shown below. We find that our proposed method MORL achieves significant improvements than baseline methods. Thanks for your question and we have included these new results in our revised manuscript.\n\n| | SARI | GLEU |\n| ----------- | ----------- | ----------- |\n| ControlledGen | 48.59 | 54.54 |\n| DeepLatentSequence | 37.46 | 50.4 |\n| StyleTransformer | 38.46 | 50.87 |\n| DeleteAndRetrieve | 7.98 | 1.07 |\n| SequentialTransfer | 49.17 | 71.3 |\n| BART-GEC | 69.10 | 74.72 |\n| ChatGPT | 48.84 | 70.21  |\n| MORL-BARTLarge | 55.74 | 75.75 |\n| MORL-Galactica1.3B | 63.79 | 78.37 |\n\n\n[8] Wieting, J., Berg-Kirkpatrick, T., Gimpel, K., & Neubig, G. (2019). Beyond BLEU: training neural machine translation with semantic similarity.\n\n\n**[PPL]**\n\n> I'm just curious why the PPL scores from some (powerful) models are so high. (Table 3)\n\nThe language model we used to calculate the PPL score is the GPT-2 model with only 137M parameters, which is not that powerful. To check the sensitivity towards the usage of base language models, we changed it to a GPT-2 large model with 774M parameters. The results are shown below:\n| | PPL |\n| ----------- | ----------- |\n| Train_IFA | 32.56 |\n| Train_FA | 24.44 |\n| Dev_IFA | 33.07 |\n| Dev_FA | 31.19 |\n| Test_IFA | 35.97 |\n| Test_FA | 33.18 |\n| ControlledGen | 60.87 |\n| DeepLatentSequence | 68.45 |\n| StyleTransformer | 66.87 |\n| DeleteAndRetrieve | 34.11 |\n| SequentialTransfer | 41.19 |\n| BART-GEC | 35.83 |\n| ChatGPT | 28.84 |\n| MORL-BARTLarge | 35.65 |\n| MORL-Galactica1.3B | 34.50 |\n\nIt is observed that the original conclusion in our manuscript still holds. We have updated these results in our revised version. Thank you for your comments! \n\n**[Paragraph length]**\n\n> I'm also curious about how the refinement is context-dependent, given that paragraph-level editing would be one important aspect of this task, and the paragraph length is somewhat short (1--2 sentences). There is a gap in the average sentence number between the training and test set. Why does this gap occur? I fear that these are intendedly controlled.\n\nWe reviewed Table 2 and noticed a few minor discrepancies regarding the statistics of \"paragraph length”. Please find the up-to-date statistics below (This is exactly the dataset used in all the experiments). Sorry for the typo. Thank you for pointing this out and we have updated our manuscript with the accurate statistics.\n\n| | | P# | S# | V# | Avg. Words | Avg. Sent. |\n| ----------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- |\n| Train | FA | 55.6K | 172.8K | 84.3K | 51.42 | 3.11 |\n| Train | IFA | 13.0K | 41.3K | 38.9K | 52.17 | 3.17 |\n| Dev | FA | 465 | 1359 | 5.2K | 47.33 | 2.92 |\n| Dev | IFA | 465 | 1362 | 5.3K | 47.79 | 2.92 |\n| Test | FA | 415 | 927 | 4.4K | 42.52 | 2.23 |\n| Test | IFA | 415 | 910 | 4.5K | 43.08 | 2.19 |\n\nAs for the paragraph length, we can see that it is 2-3 sentences and paragraph level edits could be done. Especially, the average words are rather accurate and we can see that the paragraph is quite long and some paragraphs consist of long sentences separated with multiple commas.\n\nAs for the gap between the training and test set in terms of average sentence number, it is because annotators will filter out the data that can not meet the required standards. \n\n**[Seeds]**\n> How many different seeds are used in the experiment? Is the performance difference statistically significant?\n\nWe ran our experiments three times and the improvements of MORL over all baselines are statistically significant under the t-test with p < 0.05. We will report this significance in our revised paper.\n\n**[Edit-distance]**\n> Is edit-distance character-level or word-level?\n\nThe edit-distance reported in our paper is character-level. Thanks for your comments and we have added this detail in our revised version.\n\n**[GPT-4 with a 5-scale value]**\n> it would be better to write explicitly that the GPT-4 rates with a 5-scale value even at the evaluation. When I first looked at Table 3, I was a little confused as to what the GPT-4 score represented.\n\nThank you for your suggestion! We have clarified that GPT-4 rates with a 5-scale value at the evaluation in our revised paper following your advice.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This is a thorough study in that it makes a series of contributions: task proposal, data generation, model proposal, and evaluation.\n- The dataset is a relatively large scale, and at least from the information described in the paper, the quality seems not to be problematic.\n- The proposed model shows better performance than strong baselines.", "reasons_to_reject": "- I'm not sure about the novelty of the task characteristics. The statistics shown in Section 4.3 is informative, but I want to see the comparison with the existing language refinement benchmarks, including, e.g., JFLEG [Napoles+, 17]. I'm also curious about how the refinement is context-dependent, given that paragraph-level editing would be one important aspect of this task, and the paragraph length is somewhat short (1--2 sentences).\n- Evaluation with GPT-4 was used without validating its appropriateness. At a minimum, shouldn't you show that source and target sentences have an appropriate difference in GPT-4 ratings?\n- Why don't you use reference-based metrics such as GLUE? If you have some concerns about them, you should have some discussion or evaluation of the metrics on this task.", "questions_to_authors": "- I'm just curious why the PPL scores from some (powerful) models are so high. (Table 3)\n- There is a gap in the average sentence number between the training and test set. Why does this gap occur? I fear that these are intendedly controlled.\n- How many different seeds are used in the experiment? Is the performance difference statistically significant?", "paper_topic_and_main_contributions": "This study proposes the Academic Writing Formalization (AWF) task to achieve paragraph-level automatic language improvement, given the inherent difficulty of academic writing, especially for non-native English speakers. The study creates a dataset for this task, proposes a model powered with reinforcement learning, and extensively evaluates diverse baselines and their proposed models, showing their models' superiority.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Is edit-distance character-level or word-level?\n- it would be better to write explicitly that the GPT-4 rates with a 5-scale value even at the evaluation. When I first looked at Table 3, I was a little confused as to what the GPT-4 score represented."}}, {"review_id": "F80lz3hJei", "reviewer": "Reviewer_r3ht", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer r3ht,\n\nThank you very much for your comprehensive review and valuable feedback! We have incorporated your suggestions into our revised paper including but not limited to the following edits:\n\n1. Reorganize the methods in Table 3 and group them into three kinds. (1) the baseline models: ControlledGen, DeepLatentSequence, StyleTransformer, DeleteAndRetrieve, SequentialTransfer, and BART-GEC  (2) the proposed method: MORL-BARTLarge and MORL-Galactica1.3B   (3) ChatGPT and native rewrites. In addition, we bold the scores of MORL.\n2. Add column borders to the table to make different metrics more distinct.\n3. Fix typos by adding a space after parentheses at L487.\n\nThank you again for your suggestions!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Since the previous tasks like grammatical error correction, paraphrase generation and text simplification mainly focus on the sentence level, and are not specialized and comprehensive in academic texts, the proposed task is more challenging and is of higher application value.\n\n2. The constructed DOOLITTLE dataset is a large-scale dataset for the proposed task, which can serve as a good testbed for future research.\n\n3. Experiments conducted on the dataset demonstrate that the MORL method can be successfully applied to large language models tuning and has achieved good results.", "reasons_to_reject": "I have no furthur suggestions except for some typos and styles.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes the Academic Writing Formalization task, which is a text style transfer task and aims to rewrite informal-academic texts into formal-academic texts.\nThe authors construct a non-parallel training dataset for this task based on crowdsourcing, and manually annotated parallel validation and test sets.\nIn addition, the authors also explore the application of the metric-oriented reinforcement learning method on large language models and has achieved good results.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. In Table 3, the experimental results of the proposed method are not highlighted. It is recommended to group and sort the models as: (1) the baseline models, (2) the proposed method, (3) ChatGPT and native rewriter, and highlight the scores of the proposed method outperforming the baseline models.\n\n2. Besides, the distinction between different metric columns in Table 3 is not very clear and may require to be improved.\n\n3. L487: (GEC) task (missing space after parentheses)"}}, {"review_id": "ZsxhPPpGTf", "reviewer": "Reviewer_rcwz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer rcwz,\n\nThank you very much for your comprehensive review and valuable feedback! We address your comments one by one as follows:\n\n\n**[Automatic evaluation metrics]**\n\n> The paper did not provide a clear explanation of the motivation for selecting automatic evaluation metrics. Observing a high proportion of grammar and spelling errors in the test set, It seems necessary to introduce GLUE or F0.5 commonly used in GEC, as well as SARI commonly used in text revisions. But these metrics are not presented in the paper.\n\nThanks for your suggestions. \nFollowing [1], we applied metrics that can evaluate the transfer accuracy, semantic similarity and fluency properties of baseline style-transfer/GEC models. We did not use the n-gram-based metrics due to their inherent limitations elaborated in [1], including:\n\n(1)unreliable correlations between n-gram overlap and human evaluations of semantic similarity\n\n(2)discouraging output diversity \n\nHowever, to provide a more comprehensive analysis, following your advice, we include SARI and GLEU for our task with more evaluation results. The evaluation results of SARI and GLEU on all baselines are shown below. We find that our proposed method MORL achieves significant improvements over baseline methods. Thanks for your question and we have included these new results in our revised manuscript.\n\n| | SARI | GLEU |\n| ----------- | ----------- | ----------- |\n| ControlledGen | 48.59 | 54.54 |\n| DeepLatentSequence | 37.46 | 50.40 |\n| StyleTransformer | 38.46 | 50.87 |\n| DeleteAndRetrieve | 7.98 | 1.07 |\n| SequentialTransfer | 49.17 | 71.3 |\n| BART-GEC | 69.10 | 74.72 |\n| ChatGPT | 48.84 | 70.21  |\n| MORL-BARTLarge | 55.74 | 75.75 |\n| MORL-Galactica1.3B | 63.79 | 78.37 |\n\n\n[1] Krishna, K., Wieting, J., & Iyyer, M. (2020). Reformulating unsupervised style transfer as paraphrase generation.\n\n\n\n**[Non-parallel data]**\n\n> As the proposed dataset is non-parallel, it does not support training the model in the Seq2Seq mode, which to some extent limits the universality of the dataset.\n\nThanks for your comments! Considering the high cost of annotating large-scale parallel data,  we hope to explore enhancing model performance and representation using non-parallel data. \nAs mentioned in our paper, there are nearly 70K paragraphs in the Formal-academic and Informal-academic training set of DOOLITTLE, which means we need to spend around 70 times more on annotating the full dataset compared with our current work. Thus, considering the cost issue, we propose DOOLITTLE with a non-parallel training set and parallel development and test sets.\nThis shares a similar goal with recent non-parallel style transfer [2-4] and low-resource GEC [5-7] studies which aim to build systems that do not rely on large amounts of parallel data. \n\nWe have included the discussions and cited these studies in our revised manuscript. Thanks for your question!\n\n[2] Shen, T., Lei, T., Barzilay, R., & Jaakkola, T. (2017). Style transfer from non-parallel text by cross-alignment.\n\n[3] Hu, Z., Yang, Z., Liang, X., Salakhutdinov, R., & Xing, E. P. (2017). Toward controlled generation of text.\n\n[4] Fu, Z., Tan, X., Peng, N., Zhao, D., & Yan, R. (2018). Style transfer in text: Exploration and evaluation.\n\n[5] Bryant, C., Felice, M., Andersen, Ø. E., & Briscoe, T. (2019). The BEA-2019 shared task on grammatical error correction.\n\n[6] Choe, Y. J., Ham, J., Park, K., & Yoon, Y. (2019). A neural grammatical error correction system built on better pre-training and sequential transfer learning.\n\n[7] Grundkiewicz, R., Junczys-Dowmunt, M., & Heafield, K. (2019). Neural grammatical error correction systems with unsupervised pre-training on synthetic data.\n\n \n**[Ablation studies]**\n\n> The paper proposes the MORL method. However, due to the lack of sufficient ablation experiments, we cannot know how much gain MORL has achieved.\n\nThanks for your suggestion! We agree that adding ablation experiments would enhance the persuasiveness of our research. Therefore, we conduct ablation studies for our proposed method MORL. The results are shown below. We can see that after removing a certain metric, that metric will not be optimized, so the corresponding score will decrease compared to the best result.\n\n|  | ACCaesw | PPL | SIM-input | SIM-gold | BARTS |\n| ----------- | ----------- | ----------- | ----------- | ----------- | ----------- |\n| BARTLarge w/o MORL | 74.70 | 38.39 | 99.19 | 96.74 | -1.34 |\n| MORL-BARTLarge | 78.80 | 35.65 | 98.49 | 97.45 | -1.32 |\n| MORL-BARTLarge w/o ACC | 75.18 | 36.10 | 98.49 | 97.25 | -1.29 |\n| MORL-BARTLarge w/o BARTS | 78.55 | 37.9 | 97.97 | 96.86 | -1.46 |\n| MORL-BARTLarge w/o PPL | 77.83 | 41.15 | 98.68 | 97.53 | -1.32 |\n| MORL-BARTLarge w/o SIM | 78.80 | 35.61 | 97.74 | 96.67 | -1.44 |\nTable 8. Ablation studies of MORL-BARTLarge models. BARTLarge w/o MORL denotes the BART-Large policy model without MORL tuning. MORL-BARTLarge w/o denotes that the corresponding metric’s weight is set to zero during MORL-tuning. \n\nBelow is a section for ablation study which we’ll add to our updated manuscript.\n\n\n**Ablation Study**\n\nIn this section, we perform several ablations on MORL-BARTLarge to study how each metric used in MORL as well as the whole MORL module affects the performance of MORL-BARTLarge. In these ablation experiments, we manually set the weight of one metric to zero, and then perform MORL tuning on the BART-Large policy model described in section 5.2 with all other parameters the same as our optimized best setting. Thus, we can study the contribution of each individual metric to the MORL-BARTLarge model. These results are shown in Table 8.\n\nComparing the results of the BART-Large policy model with and without MORL-tuning, we can tell a noticeable performance gain across most of the automatic metrics except SIM-input, **which indicates MORL effectively boosts the performance of the given policy model.**\nIn Table 8, those variant MORL-BARTLarge models, namely MORL-BARTLarge w/o, produced the worst evaluation score of a specific metric among all MORL-tuned models without exception. This phenomenon reveals that removing (setting the corresponding weight to 0) a certain metric from the MORL-tuning process will forbid MORL from optimizing the policy model’s properties which can be measured by that metric. **In other words, each metric incorporated into MORL is practical to help MORL improve the policy model against the metric itself.**\n\n\n\n\n**[Significant differences in PPL]**\n\n> The analysis of the experimental results could to be further in-depth. For example, what is the reason for the significant differences in PPL between different models?\n\n\nThanks for your advice. We strongly agree that adding a more in-depth analysis would provide more insights and enhance the persuasiveness of our research. \nFollowing your advice, we studied the differences in PPL, which might be attributed to the GPT model we used in the evaluation. The language model we used to calculate the PPL score is the GPT-2 model with only 137M parameters, resulting in a significant PPL difference. To check the sensitivity towards the usage of base language models, we changed it to a GPT-2 large model with 774M parameters. The results are shown below:\n\n| | PPL |\n| ----------- | ----------- |\n| Train_IFA | 32.56 |\n| Train_FA | 24.44 |\n| Dev_IFA | 33.07 |\n| Dev_FA | 31.19 |\n| Test_IFA | 35.97 |\n| Test_FA | 33.18 |\n| ControlledGen | 60.87 |\n| DeepLatentSequence | 68.45 |\n| StyleTransformer | 66.87 |\n| DeleteAndRetrieve | 34.11 |\n| SequentialTransfer | 41.19 |\n| BART-GEC | 35.83 |\n| ChatGPT | 28.84 |\n| MORL-BARTLarge | 35.65 |\n| MORL-Galactica1.3B | 34.50 |\nIt is observed that the original conclusion in our manuscript still holds. We have updated these results in our revised version. Thank you for your comments! \n\n\n**[Definition of LLM]**\n\n> Some details in the paper still need to be considered. For example, although there is currently no clear definition of LLM, it is generally believed to be a decoder-only model above 1B. It seems inaccurate to include BART-Large in the category of LLM.\n\nWe strongly agree with you that there is currently no clear definition for large models and it would be better to exclude BART-Large in the category of LLM. We can call BART-Large a Pre-trained Language Model (PLM). Therefore, we have revised our statement in our revised manuscript to ensure it is accurate.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper proposes the AWF task and constructs the large-scale dataset DOOLITTLE, which is beneficial for improving the overall quality of formal academic writing and achieving automation.\n2. The paper provides a detailed description of the annotation process and quality control of the dataset and it is helpful for development of future related work.\n3. The paper conducted extensive experiments, and these models can serve as a baseline for future work.\n", "reasons_to_reject": "1. The paper did not provide a clear explanation of the motivation for selecting automatic evaluation metrics. Observing a high proportion of grammar and spelling errors in the test set, It seems necessary to introduce GLUE or F0.5 commonly used in GEC, as well as SARI commonly used in text revisions. But these metrics are not presented in the paper.\n2. As the proposed dataset is non-parallel, it does not support training the model in the Seq2Seq mode, which to some extent limits the universality of the dataset.\n3. The paper proposes the MORL method. However, due to the lack of sufficient ablation experiments, we cannot know how much gain MORL has achieved.\n4. The analysis of the experimental results could to be further in-depth. For example, what is the reason for the significant differences in PPL between different models?\n5. Some details in the paper still need to be considered. For example, although there is currently no clear definition of LLM, it is generally believed to be a decoder only model above 1B. It seems inaccurate to include BART-Large in the category of LLM.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes the new task of Academic Writing Formalization that aims to generalize the scope of GEC for language refinement, which involves grammar correction, word refinement and structure modification. And it proposes a large-scale, non-parallel dataset named DOOLITTLE with around 800 expert-annotated parallel paragraphs as the development and test sets. Based on the proposed dataset, the paper proposes a method, metric-oriented reinforcement learning (MORL) and conducts a comprehensive evaluation of neural approaches on AWF. The results show that their performance still suffers from a sizable gap compared to formal-academic rewrites by humans.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper has received largely favorable reviews from all the reviewers. Some concerns which were raised in the earlier reviews have been adequately addressed as summarised below

1. High quality datasets [concerns about differences from existing datasets have been adequately addressed]
2. Evaluation using metrics in addition to GPT 4 [results using SARI and GLUE have been added during the response period]
3. Usefulness of the MORL method (ablation results have been provided to show that the proposed method MORL indeed helps]

Overall this is a good paper which contributes a dataset, a new method and does a thorough comparison with appropriate ablation studies.

I request the authors to include the new results provided during the response period in to the main body or appendix of the paper.
Decision: Accept-Main