PEER: A Collaborative Language Model
Authors: Timo Schick
Jane A. Yu
Zhengbao Jiang
Fabio Petroni
Patrick Lewis
Gautier Izacard
Qingfei You
Christoforos Nalmpantis
Edouard Grave
Sebastian Riedel
Venue: ICLR 2023 notable top 25% (2023)
Keywords: Language Models
Controllability
Prompting
Zero-Shot Learning
Editing

Abstract: Textual content is often the output of a collaborative writing process: We start with an initial draft, ask for suggestions, and repeatedly make changes.
Agnostic of this process, today’s language models are trained to generate only the final result. As a consequence, they lack several abilities crucial for collaborative writing: They are unable to update existing texts, difficult to control and incapable of verbally planning or explaining their actions.
To address these shortcomings, we introduce PEER, a collaborative language model that is trained to imitate the entire writing process itself. PEER can write drafts, add suggestions, propose edits and provide explanations for its actions. Crucially, we train multiple instances of PEER able to infill various parts of the writing process, enabling the use of self-training techniques for increasing the quality, amount and diversity of training data. This unlocks PEER's full potential by making it applicable in domains for which no edit histories are available and improving its ability to follow instructions, to write useful comments, and to explain its actions. We show that PEER achieves strong performance across various domains and editing tasks.

Published as a conference paper at ICLR 2023
PEER: A COLLABORATIVE LANGUAGE MODEL
Timo Schick!
Jane Dwivedi- Yu²
Zhengbao Jiang!
Fabio Petroni*
Patrick Lewis"
Gautier Izacard 1.3
Qingfei You'
Christoforos Nalmpantis'
Edouard Grave?
Sebastian Riedel2.4
1 Meta AI Research Carnegie Mellon University
3 Inria & ENS, PSL
University UUniversity College London
ABSTRACT
Textual content is often the output of a collaborative writing process: We start with an initial draft, ask for suggestions, and repeatedly make changes. Agnostic of this process, today's language models are trained to generate only the final result. As a consequence, they lack several abilities crucial for collaborative writing: They are unable to update existing texts, difficult to control and incapable of verbally planning or explaining their actions. To address these shortcomings, we introduce PEER, collaborative language model that is trained to imitate the entire writing process itself. PEER can write drafts, add suggestions, propose edits and pro- vide explanations for its actions. Crucially, we train multiple instances of PEER able to infill various parts of the writing process, enabling the use of self-training techniques for increasing the quality, amount and diversity training data. This unlocks PEER's full potential by making it applicable in domains for which no edit histories are available and improving its ability to follow instructions, to write useful comments, and explain its actions. We show that PEER achieves strong performance across various domains and editing tasks.
INTRODUCTION
Large neural networks show impressive text generation capabilities when pretrained with language modeling objective (Radford et al. 2019 Raffel et al. 2020; Brown et al. 2020 Rae et al. 2021 Zhang et al. 2022 Chowdhery al. 2022 i.a.). However, the way these models operate producing outputs in single pass from left to right differs strongly from the iterative process by which humans typically write texts. This limits their utility for collaborative writing in various respects; for example, they are not able to retroactively modify or refine their own outputs. Beyond that, they are hard to control (Korbak et al. 2022) and verifying their outputs is challenging as they often hallucinate content (Maynez et al., 2020 Shuster et al.] [222]] Nakano et al., (021) and lack the ability to explain their intentions. All of this makes it very difficult for humans to collaborate with such models for writing coherent, factual texts.
To address these shortcomings of existing LMs, we propose PEER (Plan, Edit, Explain, Repeat), collaborative language model trained on edit histories cover the entire writing process. As illustrated in Figure I] PEER operates in several steps that aim to mirror the human writing process: For given text, either a user or the model itself can plan an action to be applied, for example by means of natural language instruction. This plan is then realized by an edit, which the model can explain both in form of a textual comment and by pointing to references used; this is enabled by augmenting each input text with retrieved passages containing potentially relevant background information. We repeat these steps until the text in a satisfactory state that does not require any further updates. This iterative approach does not only enable the model to decompose the complex task of writing a consistent, factual text into multiple easier subtasks, it also allows humans to intervene any time and steer the model in the right direction, either by providing it with their own plans and comments or by making edits themselves.
Similar to recent approaches for iterative editing (Faltings al. 2021 Reid & Neubig 2022
Wikipedia as our main source of edits and associated comments, which we use as proxies for plans and explanations. In contrast this prior work, however, our goal is to obtain collaborative model

---

Published as a conference paper at ICLR 2023
PLAN
EDIT
EXPLAIN
Golden Guines Breweries (established a Nigerian brewery located Umuahia, Anambra State. It was founded In 1962!!! as an indigenous competior to foreign manufacturers the country.
founding date
Fix incorrect information.
[1] africantinendalls com: Golden Guinea
Breweries wes founded in 1962 and originally called Independence Brewery [..]]
REPEAT
Illustration of the steps performed by PEER: First, either the user the model specifies a plan describing the action they want to be performed; this action is then realized by means of an edit. The model explain the edit both in natural language and by pointing relevant sources. We can repeat this process until the generated text requires no further updates.
Figure 1:
that is useful beyond just Wikipedia: It should be capable of following human- instructions for updating texts any domain. To achieve this goal, we train PEER not only to perform the writing process illustrated in Figure in sequential order, but also to infill various parts; for example, given an edited text and set of relevant documents, we teach it to produce the original version of this text before it was edited. This enables to use self-training techniques (e.g.. Yarowsky 1995 Sennrich jet al., 2016; He et al., 2020a; Schick Schütze 202 la) for training PEER with synthetic plans, edits, explanations and documents. We show that this substantially improves PEER along several axes, including its ability to edit texts in any domain, to understand written instructions, and explain its actions.
RELATED WORK
Text Editing Similar to our work. FFltiings et al. (2021) train an editing model to follow plans on Wikipedia data. However, they only consider single sentence edits, evaluate Wikipedia data only and do not explore approaches for improving data quality and coverage. Reid & Neubig (2022)
also train models Wikipedia's edit history, but do not consider plans, explanations reference documents. Several editing models are trained to solve specific tasks, such as updating informa- tion (Logan IV al., 202T). fixing grammar errors (Napoles et al. 2017 Awasthi et al. 2019) or
improving citations (Petroni et al. 2022). Various approaches teach models to iteratively improve texts an unsupervised fashion (e.g.. Shen et al.) [2020; [Dnahue et al. 2020 Li et al., (022) and explore more efficient ways of representing edits (Mallinson et al. (020).. Concurrent work of
Dwivedi- et al. (2022) proposes EDITEVAL, a benchmark for evaluating editing models.
Instruction Tuning and Planning Explicitly teaching models to follow plans related to recent work that finetunes models on human-written instructions (Wei et al. 2022a Sanh al. 2022 Bach 2022: Ouyang et al. 20222] Wang et al. (022). The idea of having a separate planning stage has
et al. 2022:
also explored for other generation tasks inlcuding summarization (Narayan et al. 202T).
data-to-text generation (Moryossef et al. 2019) and story writing (aao et al. 2019). Our approach of writing text by iteratively performing small updates has some similarity with recent approaches like chain-of-thouuhi prompting (Wei et al., 2022b Dohan et al.) (022) and document sketching (Wu
al. 202T), that also break down complex task into multiple smaller steps.
Collaborative Writing Du al. (2022a b) investigate human human-maching machine interactions for iteratively improving documents; however, they focus mostly on syntactic edits that improve the fluency, co- herence or style of a document. Lee et al. (2022) investigate using GPT3 (Brown et al.) (020) as a writing assistant for creative and argumentative writing. In their setup, however, the model provides suggestions for continuations without being controllable by means of natural language instructions.
Self-Training Our approach of using models to infill missing data closely resembles other self- training and bootstrapping approaches used e.g. in word sense disambiguation (Yarowsky) 1995), machine translation (Sennrich al., 2016 Hoang al., 2018), sequence generation (He et al. 2020a), and few-shot learning (Schick & Schütze 2021abb) b). Similar to how we use models to turn plain texts into sequences of edits, Dai et al. 2022 turn documents into dialogue sequences.
2

---

Published as a conference paper at ICLR 2023
X
Umushia,, Anambra State.
PEER-Edit
X,
d
d
d
P
affitanniinnciiass.co comn was founded in 1962 and...
Add more information.
P
PEER-Undo
oloomberg com
markets range
d
d?
X
etricanfinanclal.co com
PEER-Explain
d
shut down 2003...
d
d
x
P.
X
was founded Umuahla, Anambra State 1962.
PEER- HERR-Docmment
Added founding
e
( x x P.
d
Figure 2: Schematic representation of the PEER process. Left: Starting from text Xx, we use both plan P: and a collection of documents D. - [d)..... df to obtain an updated version X and an explanation e, the performed edit; this process repeated multiple times. Right: To generalize to domains without editing histories, overcome data scarcity and improve the model's 's core abilities, we train various instances of PEER that perform different infilling tasks derived from this process.
PLAN, EDIT, EXPLAIN, REPEAT
The core idea of our proposed framework is model the editing of textual content iterative process, where we repeatedly plan and realize changes (see Figure left). Each iteration within this framework edits text sequence X, obtain an updated version X For this edit, we assume that we are given set of documents D: {d' ..... d containing relevant background information Given X₃ and Dr, we first formulate plan P: rough idea of how the text should be modified, verbalized as a short text sequence like "add more information". "fix grammar errors" or "use sim- pler language". This plan is then realized by means of an actual edit that transforms x, into the updated state X++1- Finally, the intention behind this edit can optionally be clarified by providing a textual explanation e;; this is especially relevant collaborative settings where explanations can facilitate evaluating the quality and usefulness of an edit (Liu et al. 2019h. Note that the explanation can be similar or even identical to the plan, the conceptual difference being that the plan is made before performing the edit, whereas the explanation is only formulated after was performed. The entire process of formulating a plan, collecting documents, performing an edit and explaining it, can be repeated multiple times to obtain sequence of texts X,, X X until either we arrive some X,, for which x,, Xx-1, or we reach manually defined halting criterion. We can also write texts from scratch by starting with an empty sequence, i.c., Xg parts, we refer to models based on this iterative process as PEER models.
In reference its four main
While using PEER break the complex task of writing a coherent, consistent and factual document into many smaller subtasks has some potential benefits over standard left-to-righh language modeling -such being more interpretable and easier to control it challenging to find data from which this process can be learned at the scale required to train large language models. This is mainly because edit histories are difficult to obtain from web crawls, the most important data source for current language models (Brown al. 2020 Rae et al. 202I). But even in cases where edit histories can be obtained (c.g., by collecting crawls of identical pages at different times) or synthetically generated, edits typically not annotated with plans, documents, explanations.
Similar to prior work on text editing (Faltings et al. 2021 Reid & Neubig 2022), our first step in overcoming this issue is turning to Wikipedia- single source that comes close to fulfilling all our needs: It provides a full edit history including comments on a diverse set topics, is large in scale, and articles frequently contain citations, which be helpful for finding relevant doeu- ments. However, relying on Wikipedia as our sole source of training data comes with various severe downsides: First, makes trained models specific to Wikipedia in terms of how they expect textual
'This set aims mirror the result of background research that humans may conduct before writing editing However, modeling this rescarch itself beyond the scope of this work, so consider D, as given.
3

---

Published as a conference paper at ICLR 2023
content to look like and what plans and edits they predict. Beyond that, comments in Wikipedia are, in many cases, not an appropriate proxy for plans or explanations. Finally, numerous paragraphs in Wikipedia do not contain any citations; while this lack of background information often be compensated by using a retrieval system (Piktus et al. 2021] Petroni et al. (202)), even such systems may not be able to find supporting background information for many edits.
4 INFILLING EDIT HISTORIES WITH PEER
We propose a simple approach to address all issues that arise from Wikipedia being our only source of commented edit histories once: We train not just one, but multiple instances of PEER that learn to infill various parts of the editing process (Figure 2 right) | these models can then be used to generate synthetic data as a substitute for the missing pieces in our training corpus. In concrete terms, we train the following encoder-decoder models:
Given an input text and a set of documents, this model learns to both plan and realize edits, i.e., it maps (x,, Dz) to the sequence (pr, Xx+1). This is done in an autoregressive fashion by factoring
PEER-Edit:
P(Pr, Xx+1 Xx, D.)
II p(z) Xt, Dr, 21, .111,
(1)
X is the concatenation of Pt and X Thus, PEER- Edit can update texts autonomously by generating both plans and edits, but it can also be provided with human-writter plans as prefixes. As PEER-Edit our main model for actual editing, also refer to it simply PEER.
where z 21,...; 2n
Given a text sequence X(+1 and a collection of documents D, that may have been used write it, this PEER instance is trained guess and undo the latest edit by predicting the sequence (Pr, x2). This is done autoregressively analogous to PEER-Edi
PEER-Undo:
PEER-Explain: This model trained autoregressively generate explanations e; given (X6, X(+1; D,), i.e., an edit and collection of relevant documents.
PEER-Document: Given (x,, X(+1- Pt). this model is trained to generate a document e Dr that provides useful background information for the edit.
We use all variants of PEER to produce synthetic data both to generate the missing pieces for completing our training data, and replace low-quality pieces in our existing data.
Decomposing Texts To enable training on arbitrary text data even it comes without edit histories, PEER-Undo for generating synthetic "backward" edits: Given plain text x,, and collection of documents D, we iteratively apply PEER- Undo to obtain a sequence 1Px-1.Xn-11 We can then train PEER-Edit in the opposite
(Pn-2, xn_ until arrive at some Xm. direction, i.e., to predict each (Pr, x++1) from x; and D.
Generating Plans We PEER-Explain to address both the low quality of many comments in corpus, and the fact that some edits may not have any comments. Given X,, X:41 and a collection of documents Dr, explain the edit being made and act as potential plans. We then compute the likelihood of the actual edit given each ef and pick the one that makes this edit the most likely as its new plan:
sample various outputs e ,,, e from PEER-Explain/ (x, X D.) that
p: arg max p(x++1 Xx, De, e()
(2)
jE(L....))
where p(X441 Xx, Dr, e is the probability that PEER-Edit assigns X given Xx, Dr and ef-
Generating Documents If we are cannot find relevant documents for an edit, we can use PEER- Document generate synthetic ones. We only do so for training PEER-Edit; never provide synthetic documents during inference. Analogous to plans, sample multiple documents from PEER- Document and pick the one that helps PEER- Edit the most in predicting the actual edit.
In preliminary experiments, also tried training single model to perform all infilling tasks at once, hut found this approach perform worse.
4

---

Published as a conference paper at ICLR 2023
Table 1: SARI scores on all subsets of Natural Edits. Domain-adapter (DA) variants outperform regular PEER, demonstrating the usefulness of synthetic edits generated with PEER-Undo
Wiki
News
Cook.
Garden
Law
Movies
Politics
Travel
Workpl.
Copy
32.7
32.8
31.6
32.0
31.1
31.5
31.8
31.2
31.5
PEER (no plans)
50.7
41.3
36.3
35.1
35.8
35.3
36.5
34.8
34.7
PEER
55.5
49.3
40.2
37.7
36.4
39.2
38.7
38.1
36.7
PEER (DA)
51.6
42.9
44.9
39.0
42.4
41.3
40.2
39.2
5
EXPERIMENTS
We conduct a series experiments to investigate - Wikipedia being our only natural source of comments and edits our infilling techniques enable us to turn PEER into a general purpose editing model capable of following Human--ritter plans and tackling range of editing tasks in different domains. Specifically, we aim answer the following questions:
Can PEER follow plans and perform edits in domains for which no edit histories are avail- able, and does eeff-training on decomposed texts improve this ability? (Section 5.1
Does the ability to follow plans based on Wikipedia comments transfer to instructions spec- ified by humans, and can it be improved by training on synthetic plans? (Section 5.2
Can PEER make proper of citations and quotes to explain generated outputs, and can WEER-Document be used to amplify this? (Section (.3)
How does writing texts in single pass compare iteratively applying PEER? (Section 5.4
Experimental Setup Our main training data is based on Wikipedia's 's edit history. each edit, we use citations and retrieval system (Petroni et al. 2022) obtain up relevant documents {do, d1, ). We replace each citation of a document d; with [[ [2] ]] or [[[ quote=q:] depending on whether a specific subsequence q, of d, was quoted in the original data. We simply concatenate all inputs and outputs, separated by special sequence. Further details are discussed in Appendix [A] We initialize all instances of PEER from an existing pretrained language model with 3B parameters. Each model is trained for 20,000 steps on 64 GPUs with effective batch size of 256, corresponding to about five million Wikipedia edits.
5.1 NATURAL EDITS
We first evaluate PEER's ability to follow plans and perform edits in domains for which no edit histories available. To this end, introduce Natural Edits, collection of naturally occuring edits for different text types and domains that we obtain from three English web sources: We collect encyclopedic pages from Wikipedia, Gardening, Law, Movies, Politics, Travel and Workplace subforums of Stacknxe All of these sites provide edit histories with comments that often elaborate on the edit's intent and that we provide to all models as plans. We split each dataset into training and test data. However, we only provide plain texts instead of actual edits the training sets of the Wikinews and StackExchange subsets, enabling us to test editing abilities in domains for which no edit histories are accessible. Relevant statistics for Natural Edits are shown in T
articles from Wikinews, and questions from the Cooking,
To leverage available plain texts, we use PEER- as described in Section 4 and create syn- thetic in-domain edits on which we train commiin-adapted (DA) variants of PEER. These variants are finetuned on balanced mixture of examples from the original training distribution and synthetic edits for 1,000 steps; we do so separately for the Wikinews and StackExchange subsets Natural Edits, resulting in two instances of domain-adapted PEER. Results shown in Table illustrate that plans are extremely helpful across domains, indicating that the ability to understand plans found in Wikipedia edits directly transfers to other domains. Importantly, the domain domain-adapted variants of PEER outperform regular PEER for all subsets of Natural Edits. This demonstrates the effectiveness of generating synthetic edits for applying PEER in different domains.
5

---

Published as a conference paper at ICLR 2023
Table 2: Downstream task results for PEER and various basclines, divided into three groups: (a) copy baseline, 15-based models, and PEER (b) 175B parameter decoder-only models, (c) supervised SotA. The first numbers for each task SARI scores; additional metrics are GLEU for JFLEG, EM for WNC and Update-R - for FRUIT. Supervised scores from left to right are from Ge et al. (2018),
Martin et al. (2020), Du et al. 2022b), Pryzant et al. (2020) and Logan IV al. (202)) respectively. The best result for models in the first group is shown in bold, the best zero-shot performance overall is underlined. On average, PEER (SP) clearly outperforms all baselines.
Without ASSET
Documents ITER
With Documents
Model
Params
JFLEG
WNC
FRUIT
WAFER
Avg
Copy Tk-Instruct
26.7
40.5
20.7
30.5
31.9/ / 0.0
29.8
0.0
33.6
28.9
3B
31.7
38.7
28.3
36.2
30.3/
0.0
12.7
3,9
1.6
23.5
TO
3B
32.9/38.6
28.6
28.1
17.8/
0.0
13.1/
5.7
6.1
22.8
TO+
11B
35.9
25.8
36.1
27.0/ 0.0
16.1
3.7
3.9
24.1
PEER
3B
55.8
29.9
36.5
56.4
39.4
35.2
42.0
PEER (SP)
3B 59.0/57.2
33.2
37.1
56.6/32.7
40.3 33.9
35.5
43.6
PEER (SP)
11B
59.9 58.6
32.4
37.8
58.8 / 34.7
40.7 /3.5
35,9
44.3
OPT
175B
49.2/49.4
25.8
31.4
25.1 /
3
21.1
31.4
GPT3
175B
50.6/5188
25.0
30.7
26.0/ 0.5
33.6 25.9
22.9
31.5
InstrucIGPT
175B
62.3 60.0
35.4
38.2
33.9 0.7
37.5 23.4
29.2
39.4
Sup. SotA
62.4
44.2
37.2
414..8
-147.4
5.2
DOWNSTREAM TASKS
So far, we have evaluated PEER using plans based naturally occurring comments. But to what extend is capable of following instructions formulated by humans to yield well known editing functionalities, and can training synthetic plans improve this ability? To answer these questions, we evaluate PEER on various editing tasks a zero-shot fashion. We use the following datasets:
JFLEG (Napoles et al. 2017) is a grammatical error correction dataset with single- sentence inputs written by English language learners;
ASSET (Alva-Manchego et al. 2020 is corpus for single-sentence text simplification;
ITERATER (Du al. 2022b is editing dataset spanning five edit intentions across
three different domains
WNC (Pryzant et al.
2020) dataset where the task is to remove or mitigate biased neutral;
words to make sentences
FRUIT (Logan IV et al.
2021) contains texts from Wikipedia that need be updated; for performing this update, various reference documents from Wikipedia are provided;
WAFER-INS (Dwivedi- Yu et al. 2022) is based on the WAFER dataset (Petroni et al.) 2022); the task is insert sentence in Wikipedia paragraph given documents from the Sphere corpus (Piktus et al. 2021 that contain relevant background information.
In addition to PEER-Edit, we also consider variant trained with synthetic plans; that is, we replace each original plan with one generated by PEER-Explan as described Section 4 We refer the PEER-Edit variant trained on these synthetic plans PEER (SP). We compare to various baseline models: Tk-Instruct (Wang et al., 2022). TO and T0++ (Sanh et al. 2022), three models that are initialized from the LM Adapt variant of T5 and finetuned on collections of manually written in- structions. We also compare to OPT (Zhang et al. 2022) and GPT3 (Brown et al,, 2020), two large LMs with 175B parameters, and InstructGPT (Ouyang al. 2022), the Instruction-tuncl variant of / We formulate a single plan per task that we provide di to all models (see Appendix (E2)) and greedy decoding. We do not perform any task-specific finetuning are interested in evaluating each model's suitability as a general editing model: the general case of user providing plan, we cannot assume access other examples using the same plan.
[
in-context learning
We only include edits from the non-meant we----mannnnaanaeee categories "fluency", "coherence" and "clarity".
We use the text-davinci-00f variant descrihed in (Ouyang al. 2022).

---

Published as a conference paper at ICLR 2023
Table 3: Accuracy on NE-Cite (without/with gold positions) and R1/R2/RL scores on both NE- Quote and constrained NE-Quote. When given the correct position, PEER (SP) almost matches the performance of the supervised Side model on NE-Cite, demonstrating its strong citing abilities. Training on synthetic documents substantially improves PEER's ability to quote relevant passages.
Model
NE-Cite
NE-Quote
NE-Quote (con.)
Random
/33.3
40.1 31.7 36.5
Unigram
-/34.2
Side
-/91.1
Lead
50.6 / 44.0 / 46.0
PEER
74.1 88.1
0.0/ 0.0/ 0.0
49.3 / 44.3 48.1
PEER (SP)
74.5 / 88.9
0.2/
0.1/ 0.1
49.8 44.8 48.7
PEER (SQ)
74.9/ 87.9
13.6 11.9 12.9
58.1 54.6 57.3
Results are shown in Table 2 PEER substantially outperforms all baseline models based on LM Adapted T5, with the 3B model achieving an average SARI (Xu et al. (016) score of 42.0 across all lasks, compared to 24.1 for the strongest TS-based baseline. PEER (SP) consistently outperforms regular PEER, increasing average SARI by 1.6 points. This demonstrates the usefulness of gen- erating synthetic plans to enhance PEER's 's ability to follow instructions. Increasing model size to 11B improves results for most tasks. While OPT and GPT3 perform worse than PEER, InstructGPT outperforms PEER for some tasks. However, it clearly lags behind PEER when it comes to handling documents for updating text. Averaged across all tasks, it performs 4.1 points worse than PEER (SP), despite being both larger and finetuned on human-annotated data. Our zero-shot models lag behind the supervised SotA on average, but approach supervised performance in some cases.
5.3 CITATIONS AND QUOTES
Unlike our baseline models, PEER is capable of both citing and quoting from reference documents to back up the claims it generates. This is useful in terms of explainability and verifiability, as it allows users to fact-check these claims more easily; the ability to quote individual phrases as opposed to citing an entire document is especially helpful for long documents. To facilitate the evaluation of PEER's s ability cite and quote, we consider both tasks in isolation. To this end, we introduce two new datasets based on Natural Edits: NE-Cite and NE-Quote. For building these datasets, we create examples from Wikipedia's s edit history where the only difference between x, and X that new citation quote was added, respectively. Further details are shown in Appendix E.3
Importantly, PEER's training data contains only few quotes. This is mainly because they are used sparingly in Wikipedia. Moreover, we are unable to use the majority of edits containing quotes be- cause they come from non-online sources or web pages that no longer exist, so we do not have access to the documents that the quotes taken from. To overcome this issue, we use PEER- Document to write synthetic documents for all edits that add quotes and for which the actual document is miss- ing. We finetune PEER these examples, mixed with around 500k examples from the original distribution, for 2,000 steps; we refer to this variant trained with synthetic quotes as PEER (SQ).
For NE-Cite, we use the percentage of times where the correct document was cited and the citation was placed at the right position as evaluation metric. We consider three baselines: randomly picking reference, selecting the reference that maximizes the unigram overlap with X,, and using the Side reranker (Petroni et al. 2022). a model trained on millions of Wikipedia citations. Unlike PEER, none of these baselines is able decide where to place the citation. We thus also consider a variant of NE-Cite where models are told where to place the citation; for PEER, this is achieved through constrained decoding. Scores both without and with providing the correct positions are shown in Table 3 If not provided with the correct position, PEER puts the right citation at the right place in 74.1% of cases, with PEER (SP) slightly improving performance. When given the correct position, PEER (SP) even comes close to the supervised Side model (88.9 vs 91.1), clearly outperforming the other baselines. Finetuning on synthetic quotes does not significantly alter PEER 's citing ability.
Similar to citing, we also look at two variants of the quoting task: In the first variant, the model needs to add a quote without any additional information; in the second variant, it is told where to put it and which document to quote from. For this variant, we use constrained decoding (Cao al.,
7

---

Published as a conference paper at ICLR 2023
Table 4: Rouge-1/2/L and QuestEval scores for various approaches on our Wikipedia intro gener- ation test set. Length penalty (LP) is optimized on a dev set of 100 examples. WikiLM performs better than autonomous PEER in terms of Rouge scores, but is outperformed by PEER in manual and collaborative mode; all PEER models perform better in terms of QuestEval.
Model
LP
R1 / R2 / RI.
QuestEval
Wiki-IM
5.0
38.4/ 16.9 27.3
38.7
PEER (autonomous)
5.0
37.7
15.8 / 26.2
40.6
PEER (manual)
2.0
39.4
17.0 28.1
41.1
PEER (collaborative)
2.0
39.5
17.2 / 28.4
41.0
2021 to ensure that the generated quote is contained in the eited document. We consider two base- lines: One that selects random substring of words from the cited document, and one that selects the lead n words, where n is the median length of quotes in NE-Quote. Table 3 shows Rouge-1/2/1 scores (Lin) 2004) computed only on quotes. As can be seen, PEER and PEER (SP) are unable to quote without constrained decoding. Training on synthetic documents improves performance, but still results in low scores. Constrained decoding improves performance, but PEER still does not outperform the lead baseline. However, PEER (SQ) achieves much stronger results in this set- ting, improving R R1/2/RRL scores by 7.5, 10.6 and 11.3 points over the lead baseline, respectively; this demonstrates the effectiveness of using DEER-Documan to generate synthetic documents for improving PEER-Edit's 's ability to explain generated claims by quoting from provided documents.
5.4 ITERATIVE EDITING FOR TEXT GENERATION
Finally, we investigate PEER's ability to generate new texts from scratch. To this end, we collect set of 400 intro sections from Wikipedia, each with three reference documents. As baseline, we finetune the same language model that was used to initialize PEER as conditional language model on the exact same data that PEER was trained on that is, the model is trained to predict X++1 given Dr and the page's title, but not x,- However, we use special character sequence to inform the model about whether X++1 is an intro section; we train this baseline, that we refer to as WikiLM, with the exact same parameters as PEER.
We evaluate PEER in three modes: (i) an autonomous mode, where the model continuously writes and realizes its own plans; (ii) a manual mode, where we give the model a series of hritenwritten plans. We choose a simple sequence of three plans that we use for all intros: Pu = "Create page", followed by P1 = P2 "Add more information": and (iii) collaborative mode, where human-written plans are interleaved with plans proposed by PEER; that is, we use the plan sequence Po- P6. P1, Pí, P2, where Po- P1 and P2 are as above, whereas PEER generates Pá and p':
Table Isows performance on our test set with length penalties (Murray & Chiang 2018) optimized on dev set of 100 intros. While WikiLM performs better than PEER in autonomous mode, PEER in manual mode outperforms WikiLM by about one point Rouge- 1, despite using very generic plans that are identical for all intros; the collaborative mode further improves results slightly. To evaluate the faithfulness of all models to the provided documents, we also consider QuestEval (QE) scores (Scialom et al. (202T), which we compute reference-less mode. Interestingly, all variants of PEER perform considerably better in terms of QuestEval scores than Wikil.M, suggesting that iteratively updating text helps the model stay more faithful to the provided reference documents.
ANALYSIS
We take a qualitative look at how PEER performs a truly collaborative setting where we manually provide it with itself and a question about guacamole in the Cooking forum of StackExchange. For the introductory section, we collect three reference documents containing this paper's author list and affiliation, the abstract, and the caption to Figure | respectively. We do not provide any documents for the Stack- Exchange example. Figure 3 shows interactive sessions with PEER (SP, 11B) and PEER (DA, 3B) for writing these texts, respectively. In each step. we generate three different model outputs - one with beam search and two using top-p sampling with p 0.9 and manually pick of them.
written instructions to write both Wikipedia-style introductory section about
8

---

Published as a conference paper at ICLR 2023
Create new page about PEER
Ask question about making Guacemole
PEER language model for collaborative wriling developed
How make guacamole?
Meta Al Research,
How Guacamoles?
[guacamoles] [salad] [cheese] [pasta]
Add info what this model do
Add more the question
PEER a language for collaborative writing developed by
Meta Al Research. It write drafts, add suggestions, propose edits and provide explanations for its actions.
How make guacamole?
How do make Guacamoles? I want to make it like the ones you can get at Mexican restaurant. What kind of avorados are good for making Guacamole.
Add citation for the model being developed by Meta Al
[guatamoles] [salad] [cheese] [pasia]
PEER is a language model for collaborative writing developed by Research can write drafts, add suggestions, progose
cheese pasta
edits and provide explanations for its actions.
How make guacamole? [..] [guacamoles] [salad] (shoeco)jpasta)
Add the four steps performed by PEER
PEER is a language madel for coliaborative writing developed by Meta Al Research, can write drafts, add suggestions, propose and provide explanations for its actions. performs four steps: edit, explain and repeat.
Add relevant tags
How to make guacamole? [..]
[salad] [food-scien sroddscience] [guacamoles]
Add info on the scandal
Add details about what I've tried so far
PEER is language model for collaborative wriling developed Meta Al Research, P9 [..]] PEER performs four steps: plan, edit. explein, repest. is also known for allegedly being used by the
How to make guacamole?
[-]] making Guacamole.
different types and can't find anything that tastes good. [salad] (food science] [guacamoles]
Chinese government censor internel.
remove unsourced claim
rewrote the guacamole question for clarity
PEER language model for collaborative writing developed
How Guacamole gueeemele
Meta Al Research |9| [..] PEER performs four steps: plan, edit. explein, repeat, H--ello--eeeww--ee--eeeeee- wtwpee/wwwwww--lloggeleeeeeeeeee wwloeellllooooolggleeeeoeeee-eee beee--eee-------tttt---- Chinese gevernmenttteeeeeem wew-ee--eeeee-eeeeeee inleeeetett
Hew de- makee Guesamalosi I'm trying to make good guac. want to make it like the ones you can get at a Mexican restaurant,
What kind of avocados are good for making Guecomele: guac?
Figure 3: Interactive editing sessions with PEER. Plans on green background are provided by human, plans on blue background by the model. Left: PEER (SP, 11B) writing Wikipedia-style text about itself. Right: PEER (DA, 3B) writing question in the style of StackExchange.
As can be seen in Figure 3 (left), PEER composes information from various documents to follow provided plans. It makes plausible assumptions, such as the model being developed by Meta AI, despite this not being explicitly stated in any document, and is able to point to the author list (docu- ment 0) as a reference. The model's response to the fifth plan ("Add info on the scandal") illustrates fundamental issue with many LMs: It accepts the premise of this plan and follows it by hallucinat- ing a scandal about internet censorship. However, PEER able to correct this misinformation in the next step: When not provided with any Human--riteen plan, the model itself writes the plan "remove unsourced claim" and removes the false statement again. Figure 3 (right) shows how after domain adaptation synthetic edits, PEER is capable of writing and editing texts in domains other than Wikipedia. In particular, it adapts to the structure of questions in StackExchange consisting of title (bold), a text, and a sequence of tags- and to their style, which very different from Wikipedia. PEER proposes plans to fix errors it made in previous steps (such as removing the irrelevant tags "cheese" and "pasta"). It is also able to follow plans like "Add more relevant tags", despite tags being a concept specific to StackExchange that does not occur in its Wikipedia training data.
CONCLUSION
We have introduced PEER, a language model that can act as a writing assistant by following plans to perform a variety of different textual edits, ranging from syntactic and stylistic edits to changing the meaning of a text by removing, updating or adding information. Through extensive experiments, have shown that training variants of PEER capable of infilling various parts of the editing pro- cess enables it to perform edits in different domains, makes it better at following instructions and improves its ability to cite and quote from relevant documents.

---

Published as a conference paper at ICLR 2023
REFERENCES
Fernando Alva-Manchego, Louis Martin, Carolina Scarton, and Lucia Specia. EASSE: Fasier au- tomatic sentence simplification evaluation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-ICCNLP), System Demonstrations, pp. 49-54, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/11/D1933000 htt.a//pww..alaaaaaoooooo.oo////
Fernando Alva-Mancheeo. Louis Martin, Antoine Bordes, Carolina Scarton, Benoît Sagot, and Lu- cia Specia. Asset: A dataset for tuning and evaluation of sentence simplification models with multiple rewriting transformations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4668 4679, 2020.
Abhiject Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla. Parallel iterative edit models for local sequence transduction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP-IJCNLP), pp. 4260 4270, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18633/11D119.1435. URL htttps:/acclantholooyy wrwpe//nnntnooooooooooo1--1-1 wrt//ooggg///.. 1435
Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V. Nayak, Ab- heesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, Zaid Alyafeai, Manan Dey, An- drea Santilli, Zhiqing Sun, Srulik Ben-david, Canwen Xu, Gunjan Chhablani, Han Wang, Jason Fries, Maged Al-shaibani, Shanya Sharma, Urmish Thakker, Khalid Almubarak, Xiangru Tang, Dragomir Radev, Mike Tian-jian Jiang, and Alexander Rush. PromptSource: An integrated de- velopment environment and repository for natural language prompts. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 93-104, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/ w1/2022.acl-dema URL f phttps:/ //a/aaaaaatttt wwwethooooy..rrggg/ooo wtw.:2222221c1222
adl--emo.....
Borgeaud, A Mensch, J Hoffmann, Cai, E Rutherford, and K Millican. Improving language models by retrieving from trillions of tokens. arXiv preprint arXiv:2112.044: 2021.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, llya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Curran Asso- ciates, Inc., 2020. URL https: httpss://proceedaggs..tt newrips.....p.p...... cc/papper/222//i112 1457c0d6bf cb496 74 41 2bffb88ccc1..fff44aaaae.e.. pdf
Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. Autoregressive entity retrieval. In International Conference on Learning Representations, 2021. URL https:
7/ //opereeveew.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard. Guy Gur-Ari, Pengeheng Yin, Toju Duke, Anselm Lev- skaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowyez, Erica Moreira, Rewon Child, Oleksandr Polozov. Katherine Lee, Zongwei Zhou, Xuezhi Wang, Bren- nan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language modeling with pathways, 2022. URL httpt:
Douglas
org abs / 2204. 02311
10

---

Published as a conference paper at ICLR 2023
Zhuyun Dai, Arun Tejasvi Chaganty, Vincent Y Zhao, Aida Amini, Qazi Mamunur Rashid, Mike Green, and Kelvin Guu. Dialog impainting. Turning documents into dialogs. In International Conference Machine Learning, pp. -558-4 PMLR, 2022.
David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A Saurous, Jascha Sohl-dickstein, et al. Language model cascades. arXiv preprint arXiv:2207.10344 2022.
Chris Donahue, Mina Lee, and Percy Liang. Enabling language models to fill in the blanks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, pp- 2492-2501, Online, July 2020. Association for Computational Linguistics. doi: 10. 18 2020.acl- 1.0..acl-main222 225. URL
wwwpy.org/22000
Wanyu Du, Zae Myung Kim, Vipul Raheja, Dhruv Kumar, and Dongyeop Kang. Read, revise, repeat: A system demonstration for numan-in-the-looe iterative text revision. In Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2 Writing 2022), pp. 96- 108, Dublin, Ireland, May 2022a. Association for Computational Linguisties. doi: 18.15////1// org/2022 2022.in2writigg-L.. 1.14. URL httpttssss//aaaaaallllooo/oo///e wtt.:/ooowoo..rr.............aa. wwwwwwttiinn...............t.iit i..iiing....
Wanyu Du, Vipul Raheja, Dhruv Kumar, Zae Myung Kim, Melissa Lopez, and Dongyeop Kang. Understanding iterative revision from hrmtnn text. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3573- 3590, 2022b.
Jane Dwivedi- Yu, Timo Schick, Zhengbao Jiang, Maria Lomeli, Patrick Lewis, Gautier Izacard, Edouard Grave, Sebastian Riedel, and Fabio Petroni. Editeval: An instruction-basee benchmark for text improvements. arXiv preprint arXiv:2209. 13331, 2022.
Felix Faltings, Michel Galley, Gerold Hintz, Chris Brockett, Chris Quirk, Jianfeng Gao, and Bill Dolan. Text editing by command. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 5259-5274 Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/ 4021.naacl-ne man.ll--ii..14... URL httttss::/aaaaatttttttt waw.eellnntooog..oooooooooooo org/2021
orthhology.org/2221111111111 naacl mammaainn... 414
Tao Ge, Furu Wei, and Ming Zhou. Reaching human-level performance in automatic grammatical error correction: An empirical study. arXiv preprint arXiv:1807.012700 2018.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Was Wei Chang. Realm: Retrieval- augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning ICML'20. JMLR. 2020.
Junxian He, Jiatao Gu, Jiajun Shen, and Mare' Aurelio Ranzato. Revisiting self-training for neural sequence generation. International Conference on Learning Representations, 2020a. URL.
htttps://openrev
net. htwpr//wwwwweeee/ffooooooooooooe
Junxian He, Wojeiech Kryšciñssii, Bryan McCann, Nazneen Rajani, and Caiming Xiong. Ctrlsum:
Towards generic controllable text summarization. arXiv preprint arXiv:2012.0422. 02288122.222,111 2020b.
Vu Cong Duy Hoang, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. Iterative back- translation for neural machine translation. In Proceedings of the 2nd Workshop Neural Ma- chine Translation and Generation, pp. 18-24, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10. N..18653/1//W18-220 URL https:://aclaattaat
orq/W18-2703
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.00 04490 2020.
Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint (9Xiv:999.005800 2019.
11

---

Published as a conference paper at ICLR 2023
Tomasz Korbak, Hady Elsahar, German Kruszewski, and Mare Dymetman. Controlling condi- tional language models without catastrophic forgetting. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.). Proceedings of the 39th Inter- national Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 11499-11528, PMLR, 17-23 Jul 2022. URL
ngs mllr.
press/v162 www.v/////ooa22aaa. html
Mina Lee, Percy Liang, and Qian Yang. Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI '22, New York, NY, USA, 2022. Association for
Computing Machinery. ISBN 978 45039155. doi: 10.1 241144/341110..3522000 0ttee/oo//1..15554111111111 7/doi. 10. 1et...111/3449000 /////44491102..522333
URL https:
Xiang Lisa Li, John Thickstun, Ishaan Guirajani, Percy Liang, and Tatsunori B Hashimoto. Diffussonn-m improves controllable text generation. arXiv preprint arXiv:2205.14217, 2022.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 2004.
Hui Liu. Qingyu Yin, and William Yang Wang. Towards explainable NLP: A generative explanation framework for text classification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5570-5581, Florence, Italy, July 2019. Association for Com-
putational Linguistics. doi: 10. 10.18653/v1/P199 1560. URL
org/
P19 P19-1560
Robert Logan IV, Alexandre Passos, Sameer Singh, and Ming-Woi Wei Chang. Fruit: Faithfully reflecting updated information in text. arXiv preprint arXiv:2112.08634 2021.
Jonathan Mallinson, Aliaksei Severyn, Eric Malmi, and Guillermo Garrido. FELIX: Flexible text editing through tagging and insertion. In Findings of the Association for Computational Lin- guistics: EMNLP 2020, pp. 1244-1255, 1255, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v htt.:/....222...2n..../ findinng-eenll..11... URL ptttps:/acclathool// org/2020. findi sereiingg-emmiie 111
Louis Martin, Angela Fan, Éric de la Clergerie, Antoine Bordes, and Benoît Sagot. Muss: multilingual unsupervised sentence simplification by mining paraphrases. arXiv preprint arXiv:2005.00352. 2020.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1906-199999 1919, Online, July 2020. Association for Computational orq/
Linguistics. doi: 10. 1/228 wtt.:200..l-mmi.11277 ma... URL 173.
Amit Moryossef, Yoav Goldberg, and Ido Dagan. Step-by-step: Separating planning from real- ization in neural data-to-text generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Papers), pp. 2267-2277, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 10.18853/11/119-1 1236. URL https: 7/acl ant.aaanhhoooyyy...ooo
92136
Kenton Murray and David Chiang. Correcting length bias neural machine translation. In Pro- ceedings of the Third Conference on Machine Translation: Research Papers, pp. 212-223, Brus- sels, Belgium, October 2018. Association for Computational Linguistics. doi: 10. 18653/1/l W18-6322. URL wts.aatpsps//aaaaaaoolooo www.:/...rgg//... -
Reiichiro Nakano, Jacob Hilton, Suchir Balaji. Jeff Wu, Long Ouyang, Christina Kim. Christo- pher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answeringg with human feedback. arXiv preprint 1rXiv:2112.004 2021.
Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. Ground truth for grammati- cal error correction metrics. In Proceedings of the 53rd Annual Meeting of the Association for
12

---

Published as a conference paper at ICLR 2023
Computational Linguistics and the 7th International Joint Conference on Natural Language Pro- cessing (Volume 2: Short Papers), pp. 388-593, Beijing, China, July 2015. Association for Com- putational Linguisties. doi: 10.3115/w1/P15-20977 URL https::/aclanteeet
org//
P15-2097
Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. Jffeg: A fluency corpus and benchmark the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp. 229-232- 234, 2017.
for grammatical error correction. In Proceedings the
Shashi Narayan, Yao Zhao, Joshua Maynez, Gonçalo Simões, Vitaly Nikolaev, and Ryan McDon- ald. Planning with Learned Entity Prompts for Abstractive Summarization. Transactions of the Association for Computational Linguistics, 9:1475- 1492, 12 2021. ISSN 2307-387X. doi: a 00438
10.11122/acl...00433.. URL https:: /ttp:ttts:///.i.oo/////////
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. arXiv preprini arXiv:2203.02 2022.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic
evaluation of machine translation. In Proceedings of the 40th Anmual Meeting of the Associa- lion for Computational Linguistics, pp. 311-318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computational Linguistics. doi: 00.3115/1073033107333 URL https:
Fabio Petroni, Samuel Broscheit, Aleksandra Piktus, Patrick Lewis, Gautier Izacard, Lucas Hos- seini, Jane Dwivedi- Yu, Maria Lomeli, Timo Schick, Edouard Grave, and Sebastian Riedel. Improving wikipedia verifiability with ai. 2022. URL
Mazaré, Armand Joulin,
htttps://openreiite net htt.//foummiiim ff
Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Dmytro Okhonko, Samuel Broscheit, Gautier Izacard, Patrick Lewis, Barlas Oğuz, Edouard Grave, Wen-tau Yih, et al. The web is your woweeer--owllde--nne intennivten nip against very large web corpus. arXiv preprint arXiv:21 2021.
Martin Potthast, Benno Stein, and Robert Gerling. Automatic vandalism detection in wikipedia. In European conference information retrieval, pp. 663-668. Springer, 2008
Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. Automatically neutralizing subjective bias in text. In Proceedings of the aaai conference on artificial intelligence, volume 34, pp. 480-489, 2020.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. Technical report, Open AI, 2019. URL https:://cc...ppeaaaa wtt.:ltteeoooeeteeeeeeeeeeeee. mode loddels_arres lell_sss_r__uuuue www.aruuuesueeeeeeeeeeeeeeeeeeee http://iwiiieeeeltitttt........e wew.eerr.rrer...... pdff
http://eeeteeelgggg--ee-eeeeeede s/ //slaaaaaggeeeee/
Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmmann. Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens. Xiang Lorraine Li, Adhiguna Kun- coro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Men- sch, -aatittt Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d' Autume, Yu- jia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Au- relia Guy, Chris Jones, James Bradburr, Matthew Johnson, Blake Hechtman, Laura Weidinger, lason Gabriel, William Isaac, Ed Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, and Geoffrey Irving. Scaling language models: Methods, analysis & insights from training go- pher, 2021. T //tp/aaavvvv///////aa//eaaaeaaaa http///ii....og///////////////// 11446
13

---

Published as a conference paper at ICLR 2023
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to- text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020. URL http:
//jml r. wwtp://gpppppss///1112
html
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System opti- mizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, KDD '20, pp. 3505-3506, New York, NY. USA, 2020. Association for Computing Machin- ery. ISBN 9781450379984.
doi: 10.1145/33994486.4007777 URL htttps://do..crrt/
1145/ 00/5//399 3394486.... 3406703
Machel Reid and Graham Neubig. Learning to model editing processes, 2022. URL https:
7/arxiv. org. abs /2005. 12374
Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulies Mike Gatford, et al. Okapi at trec-3. Nist Special Publication Sp, 109:109, 1995.
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, De- bajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang. Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. Multitask prompted training enables zero-shot task generalization. In International Conference Learning Repre- sentations, 2022. URL https: http:/::///perriiiw..eeee wwwpeee//ooouuuoooeeeeeee 9it. 9Vrb9DoWI4
Timo Schick and Hinrich Schütze. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 255-269, Online, April 2021a. Association for Computational Linguistics. doi: 20.18663/v1/2221.co
m0in.20. https:
//aclanthology org. /2tgg//2222l.. www.22eeecllamm mai.l-main.000
Timo Schick and Hinrich Schütze. Few-shot text generation with natural language instructions.
In Proceedings of the 202/ Conference on Empirical Methods in Natural Language Process-
ing, pp. 390-402, Online and Punta Cana, Dominican Republic, November 2021b. Associa- tion for Computational Linguisties. doi: 10. 10.18653/1122 1/202/ 2ttp://w112211emlp--mom .emnlp-main.32.
URL https:
//aclanthology.orogg
camippmman 32
Thomas Scialom, Paul-Alexis Dray, Patrick Gallinari, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, and Alex Wang. Questeval: Summarization asks for fact-based evaluation. arXiv preprint arXiv:2103. 12693, 2021.
Rico Sennrich. Barry Haddow, and Alexandra Birch. Improving neural machine translation mod- els with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 86-96, Berlin, Germany, Au- gust 2016. Association for Computational Linguistics. doi: 10. 10.18653/v1/P16-1000 URL https: http:///alaanthhoooooo logy. wwt.:/oo1////0005.
Tianxiao Shen, Victor Quach, Regina Barzilay, and Tommi Jaakkola. Blank language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pp. 5186-5198, Online, November 2020. Association for Computational Linguis- tics. doi: 10. 10.11653/w/22220cooo http:///000.mmmmmaam.nnmoo 420. htt.ltttts///aaaaaaeell/////// org/0000..
emnlp-main 420
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmenta- tion reduces hallucination in conversation. In Findings of the Association for Computational
Linguistics: EMNLP 2021, pp. 3784-3803, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.18653/v1/2021 .indings-emnlp.3200 URL https://acl http://cclllatteeoooooooo/////// fi ndi magiingseemt 320
14

---

Published as a conference paper at ICLR 2023
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabshhi, Yeganeh Kordi, Amirreza Mirzaei, An- jana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Benchmarking generalization via in-context instructions on 1,600+ language tasks. arXiv preprint arXiv:2204.07705. 2022.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V Le. Finetuned language models are zero-shot learners. In Interna- tional Conference on Learning Representations, 2022a. URL https: httppss//ppeereviiee...
Eorum? Srruum?dd-EE2GCozaae
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting clicits reasoning in large language models. arXiv preprint arXiv:2201.11903 2022b.
Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. CCNet: Extracting high quality monolingual datasets from web crawl data. In Proceedings of the 12th Language Resources and Evaluation Conference, pp. 4003-4012, Marseille, France. May 2020. European Language Resources Association. ISBN 979-10-95546-444 URI. /acpps://claarchooooog... wtgp:/g.oorg//00/ 1rec-1.449
Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang. and Bill Dolan. Automatic document sketching: Generating drafts from analogous texts. In Findings of the Association for Com- putational Linguistics: ACL-IICNLP 2021, pp. 2102-2113, Online, August 2021. Associa-
tion for Computational Linguistics. doi: 10. 10.18653/v1/2021 http://///22....nnngs.... acl.ings-all.15.
URL https:
/ www.laantoll.oy..orrg/2222
findi ngs acs--acl... 185]
Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. Optimiz- ing statistical machine translation for text simplification. Transactions of the Association for
Computational Linguistics, 4:401-415, 2016.
doi: 10.1162/tacl.a.000007 URL https:
Lili Yao, Nanyun Peng. Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan-and- write: Towards better automatic storytelling. Proceedings of the AAAI Conference on Artificial Intelligence, 33001)77788-735, Jul. 2019. doi: 10. 133:01.33011101111.3..
URL https:
//ojs. aaai, //rg//naaa..rg//nndeex pttp://AA.//aa.tcee//eee//////ea
org/index
David Yarowsky. Unsupervised word sense disambiguation rivaling supervised methods. In 33rd Anmual Meeting of the Association for Computational Linguistics, pp. 189-196, Cambridge, Mas- sachusetts, USA. June 1995, Association for Computational Linguisties. doi: 10.3115/981658. 981684. htRp:tp:: http://:///.aaaatooooooooo//////
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan Mona Diab. Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Oit, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettle- moyer. Opt: Open pre-trained transformer language models, 2022. URL https: //arxiv. org/ 2055/bb/200 htt//abb22200..00 2ttp/2055.111 01068
A TRAINING DATA
Our main training data for PEER is derived from Wikipedia's edit history access to raw tuples of source and target texts (z,, 2,41), from which we derive x, and X,+1 after some preprocessing steps discussed below. Beyond that, the edit history also provides us with com- ments c, that we use as proxies both for the plan pa and for the explanation e,, Finally, as Wikipedia articles frequently use citations to back up claims, we can obtain an initial set I, of document iden- tifiers (c.g., URLs) for all documents cited in either Z, or Zz+1- Our pipeline for transforming this raw data into the PEER format consists of three steps: First, we use some heuristics for filtering the data to remove low-quality edits and avoid overlap with any of our evaluation sets. We then use I; and a retrieval engine (Petroni et al., 2022)) to obtain a collection of relevant documents D, for each edit; finally, we convert the data into a format suitable for sequence-to-sequene models. In the following, we discuss all three preprocessing steps in more detail.
Which which directly gives us
We use the February 2022 dump available httth:t////mmmmee wi k medi .ow.diia..////vkk/////
15

---

Published as a conference paper at ICLR 2023
A.1 FILTERING
As Wikipedia's edit history contains several low quality edits and numerous instances of vandalism (Potthast et al. 2008), use some simple heuristics to improve data quality, In particular, we filter out edits that were reverted at some point and edits that were automatically made by bots without human involvement. Beyond that, we filter edits that affect more than two paragraphs and remove all edits for pages that are used in any of the datasets we evaluate on. We also discard examples where It contains document identifiers that are unable to resolve (e.g., because they refer to web pages that no longer exist). We filter out revisions with more than 50,000 characters. This makes preprocessing more efficient, our algorithm for computing diffs between different revisions has squared complexity in the number of characters. Beyond that, we also filter out revision whose comments contain any of the sequences "#", "[("."])" "[[". "JJ", "template", "image", "infobox" and "pic", as these are usually automatically generated or update parts of the page (such as images and infoboxes) that we remove during preprocessing. We further remove all redirects. Within each chunk of the Wikipedia dump, we downsample revisions for which the corresponding comment occurs in more than 10 revisions so that on average, each comment occurs at most 10 times per chunk. Finally, we filter out edits where either the source paragraphs or the target paragraphs have more than 384 tokens.
A.2 RETRIEVING DOCUMENTS
A crucial aspect of PEER is its use of documents that contain relevant background information. We thus aim to collect set of documents D, ... ,d}} for each edit, with i being hyperparam- eter defining the number of documents; we set k = 3.
To obtain this set of documents D, for an edit that maps x; to X we make use of the set I, of document identifiers occuring in x2 or X For each document identifier, we get the corresponding document from CCNet (Wenzek et al.) (020). We split the document into non-overlapping chunks of 100 words and use the reranker of Side (Petroni et al. 2022 to find the best chunk given X++1-
the number of documents obtained from 4, is below the maximum number of documents per edit, also use the entire pipeline Petroni et al. (2022) find relevant documents in the Sphere corpus Piktus et al. (202T) given Xt+1. As this pipeline expects a special [CIT] token at the position for which relevant documents are to be retrieved, we place this token right after the first position which x, and X differ, starting from the right. Note that obtaining documents with this approach requires access to Xt+1. it would be impossible to apply this exact same procedure in real-world settings. However, our focus is not on retrieving relevant documents, but on teaching PEER to perform edits given this information.
A.3 FORMATTING
Our first formatting step is to remove all paragraphs from z, and Ze+1 that are not affected by the edit. We then remove Wikipedia-specific syntax, but with few exceptions: We keep the syntax for representing titles, bold text, text in italics and lists, enabling the model to learn how to perform some basic formatting. We also keep links and, more importantly, citations, enabling PEER to learn how to cite and quote from documents in D, to back up the textual content it generates. We denote the resulting text sequences with zł and z
We linearize each document di E D, using its content e; and, present, its title t, and the cor- responding web site's 's domain d; as the sequence [i] d; # t; # C;. We include the number i in this representation to facilitate citing and quoting specific documents. To finally obtain x, and X++1 from z!, and z141. we replace each citation of document d; in both sequences with either [[(i]]] or [[T [ē quote-qa qq ] depending on whether in the original data, a specific subse- quence q, of di was quoted. As p. and e; are already simple text sequences, we do not perform any modifications to them.
If there are multiple inputs outputs, simply concatenate them using a special separator quence. Moreover, if the text we are editing has a title, we always prepend this title to the original input sequence.
16

---

Published as a conference paper at ICLR 2023
In addition these formatting rules, randomly remove the page's title for 10% of all examples to make sure that PEER can also work with inputs for which no title available. We minimize 10% of all examples by removing all sentences from both x; and x that are not edited, so that the model also learns to handle and edit single-sentence inputs without context. Finally, to make sure that the remove documents from Dr. where is uniformly sampled from {1..... Dzl}. However, we only remove documents that are not cited in either X or X(+1- When linearizing the input and output sequences. for each document d; E Dr. we reserve up to 16 tokens for its domain, 32 tokens for its title, and 196 tokens for the actual content. We truncate all tokens that exceed these limits.
model can handle different numbers of reference documents, for 30% of examples
B CONTROL TOKENS
To improve the quality and diversity of synthetically generated plans, edits and documents, we implement control mechanisms similar to Keskar et al. (2019) and [e et al.] (2020b) - that is, we prepend specific control tokens the output sequences that model is trained to generate, and then use these control tokens during inference to guide the model's generations. In particular, we make of the following controls for different PEER models:
For PEER-Explain, we control the output length as proxy for the level of detail in gen- erated explanations. We also control whether the generated comment starts with verb in infinitive form; this approximates the notion of an instruction, the format we expect hu- mans to commonly use for communicating with PEER. Finally, we control whether there is a word overlap between the explanation and the edit; preventing this during inference makes sure that generated plans do not make editing trivial by exactly specifying which words to add, remove or replace.
For PEER-Undo, we control the difference in the number of words between X and X Through this, we ensure that the sequence X,, Xn. ..vetttteeee terminates at X. E and does not get stuck in an infinite loop.
For PEER-Document, we control whether the generated document contains a given sub- string. This is useful when we want the document to contain a specific quote that is referred Wikipedia edit.
We do not use any controls for PEER-Edit, because unlike for other models, which have specific and clearly defined tasks to solve we do not make assumptions in advance about the types of editing tasks that users might want to solve with PEER-Edit and the kinds of control tokens that might be useful for these tasks.
Unlike Keskar et al.) (2019), we do not introduce special control tokens, but simply express all controls in the form key-value where both key and value are tokenized using the language model's 's regular tokenizer. In particular, we use the following keys and values:
We use this key to control the type of output that PEER-Explain supposed to generate, with possible values being instruction (in which case the output starts with a verb in infitive form) and other.
type:
length: This key controls the length of PEER- -EEE-Expsain's output. Values include (less than 2 words), m (2-3 words), 1 (4-5 words) and x1 (2 6 words).
overlap: With this key, we control whether there is word overlap between the edit and the generated output of PEER-Explase values are rue and false.
words: For Undo, this key is used to control for the difference in the number of words in X++1 and x accordingly, the possible values are all integers.
containt ns: This control can be used to ensure that outputs generated by PEER-Document contain a certain substring, which is provided as the value this key.
C GENERATING SYNTHETIC DATA
For generating synthetic edits, we found it sufficient to apply PEER- just once for each plain text x, obtain tuple (Pt-1, Xt-1) Upon manual inspection, also found that the generated
17

---

Published as a conference paper at ICLR 2023
plans Pt-1 do not actually match the undone edit, PEER-Explain described in Section to rewrite all plans.
When generating synthetic plans, diverse set plan lengths. For 80% of generated plans, we enforce that they start with verb and have no word overlap with the performed edit, respectively.
the control tokens discussed in Appendix B to ensure
For obtaining synthetic edits, we sample a single pair Xp) for each X using top-p sampling with p 0.9. We sample the value for the words control token from normal distribution with = 10 and o = 8, clipped at -40 and 10. These values were chosen to allow for a wide range of different values, while also making sure that average, forward edits increase the number of tokens. We rewrite each P. with PEER-Explane using the exact same procedure that we use for generating synthetic plans.
For obtaining synthetic plans, we generate 10 different plans with PEER-Explain using top-p sam- pling with p 0.9. For each pair x; and X(+1, we use single control sequence for sampling all 10 plans. We choose the length uniformly from {s, 1, x1), set type=instruct 80% of the time and overlap=far se 80% of the time.
For obtaining synthetic documents, we sample 10 documents from sampling with p 0.9, where contains set to the quote from this document that cited in X++1- We discard all documents that do not actually contain the quote, and then pick the document that maximizes the probability assigned to the actual edit by PEER-Edit.
using top-p
D TRAINING DETAILS
For training PEER, we use DeepSpeed (Rasley et al. 2020) to enable more efficient multi-GPU training. We use a maximum learning rate of 10-2, warmup for 2,000 steps and linear decay. We further use gradient clipping with a maximum norm of 1.0, weight decay of 0.01 and dropout rate of 0.1. The maximum sequence length set to 1,024 and 384 tokens for input and output, respectively.
EVALUATION DETAILS
We use a variety of metrics evaluate PEER and our baseline models on all tasks considered:
Exact Match (EM) is the percentage of examples for which the performed edit exactly matches a given target;
EM-Diff a variant of EM that is computed on the diff level
SARI (Xu al. 2016) averages match scores for the three word-level edit operations add, delete and keep!
GLEU (Napoles et al. 2015) is variant of BLEU (Papineni et al. 2002 proposed for grammatical error correction tasks;
Rouge (Lin 2004) is a set of metrics based on n-gram overlap (Rouge-n) or longest com- mon subsequences (Rouge-L);
Update-Rouge (Logan IV al. 2021 variant of Rouge that is computed only sentences updated during an edit.
E.1 NATURAL EDITS
In addition to the experiments conducted in Section 5.1 we check whether PEER actually learss make use of provided documents and plans by evaluating it on the Wikipedia subset Natural Edits. We compare regular PEER provided with gold plans to variants trained and evaluated (i)
Diffs are obtained using Python's di fffiib library. For a model output x'+1s we compute EM-Diff as [diff(xx, x ) O diff(xx, x(41]] ]] + max max([diff(xx, x,=1)1. [[ff)__ x(r)
We use the SART implementation of EASSE AAla--Manchege 2019
18

---

Published as a conference paper at ICLR 2023
Table Results for variants of PEER on the Wikipedia subset of Natural Edits. Plans and documents provide complementary information and substantially improve performance.
Model
EM
EM-Diff
SARI
Copy
0.4
32.7
PEER
23.1
26.2
55,5
PEER (no plans)
18.0
19.8
52.0
PEER (no documents)
19.8
22.8
51.7
PEER (no plans/documents)
13.5
15.1
45.9
Table 6: Overview of the number of edits and plain texts (PT) in the train sets and the number of edits in the test sets of Natural Edits. The final column shows whether the subset uses reference documents.
Subset
Train (Edit)
Train (PT)
Test
Doc.
Wikipedia
6,960,935
4,000
Wikinews
125,664
1,000
Cooking
22,517
500
Gardening
13,258
500
16,418
Movies
19,601
500
Politics
10.676
500
Travel
38,961
500
Workplace
18,231
500
without plans, (ii) without reference documents, and (iii) without both plans and reference docu- ments. Table shows EM, EM-Diff and SARI scores for all models and a copying baseline, for which X X₃. As can be seen, PEER substantially outperforms all baselines. PEER without both plans and documents performs much worse than just removing one of both, illustrating that plans and documents provide complementary information that the model is capable of using; this is in line with findings of Faltings et al. (202T).
E.2 DOWNSTREAM TASKS
The plans used for each of the downstream tasks considered in Section 5.2 are shown in Table We manually wrote instructions for all datasets except ITERATER for which we directly took instruc- tions from the definitions provided by Du et al. (2022b).
For most bascline models (TO, GPT3, InstructGPT and OPT), we wrap each plan for input with the following template:
Task: p
Input X;
Output:
Table 7: Plans used for the downstream tasks considered Section 5.2
Task
Plan
JFLEG
Fix grammar errors
ASSET
Simplify this sentence
ITERATER (fluency)
Fix grammatical errors the text.
ITERATER (coherence)
Make the text more cohesive, logically linked and consistent as a whole. Make the text more formal, concise, readable and understandable. Remove POV
ITERATER (clarity)
WNC
FRUIT
Update the article Add missing information
WAFER-INS
19

---

Published as a conference paper at ICLR 2023
40
RI Score
38
36
WikiLM PEER (a)
PEER (m)
PEER (c)
34
1
4
Iterations
Figure 4: Average Rouge-1 score of WikiL. and PEER autonomous (a), manual (m) and collab-
orative (c) mode function of the number of iterations
For Is--nsttu we replace the string "Task" with "Definition" match their format. For tasks that require references, we additionally add all references following the string "Reference:" after the input. For examples that also provide title, we add this title following the string "Title:" before the input.
E.3 CITING AND QUOTING
Naturally, we make sure that the cited document is always present in the k: documents provided to PEER. To make the task of citing the correct document challenging, we obtain the other two documents in D, by applying the BM25 (Robertson al.) 1995) and DPR (Karpukhin et al.) 2020) variants of Petroni et al. (2022) to find the best match in Sphere (Piktus et al., 2021). respectively. If the gold document contains too many tokens, for NE-Cite reranker of Petroni et al. (022); for NE-Quote, we select the chunk from the document that actually contains the quote. In total, we collect 2,351 and 391 examples, respectively, for which we manually set the plans to simply be "Add a citation" and "Add quote".
pick the best chunk according the
E.4 ITERATIVE EDITING FOR TEXT GENERATION
Without controlling for output length, WikiLM generates rather short intros, resulting in relatively low Rouge-1/22 1/2/L. scores. To make the comparison more fair, we thus split our dataset of Wikipedia intros into 100 dev examples and 400 test examples; the dev examples picking the exponential length penalty (Murray & Chiang 2018) that maximizes the model's 's average R score. We also prevent models from generating the same token 5-gram more than once to avoid endless repetitions.
exclusively used for
Figure 4 shows how performance for different PEER modes changes across iterations, illustrating how generated intros are improved over multiple iterations.
F ANALYSIS
For the introductory section, we collect three reference documents do- d1, and d2, where the first document contains this paper's author list and affiliation, the second document contains the abstract, and the third document contains the caption Figure []] For all documents, we set the title this paper's title and the domain to arxi org. We use this same set of documents for each generation step, i.e., D, (do, d1, d for all We do not provide any documents for the StackExchange example. Figure 3 shows interactive sessions with PEER (SP, 11B) and PEER (DA, 3B) for writing these texts, respectively. In each step, we generate three different model outputs one with beam search using three beams, and two using top-p sampling with p 0.9 and manually pick one of them.
To better understand the quality of the synthetic data generated with our infilling procedure, also look at exemplary outputs of the other PEER variants. We first consider PEER-Undo, the model to generate edits for domains where only plain texts are available. Figure shows
20

---

Published as a conference paper at ICLR 2023
Similar to prior work on text editing (Fallings et al., 2021: Reid and Neubig, 2022), our first step in overcoming this Iseue is turning to Wikipedia single source comes close to fulfilling all our
add Reid and Neubig
Similar prior work on editing (Fallings et al., (eRiid-nnd Neebig, 2922), step overcoming issue turning Wikipedia single source comes close to fulfiliing all needs.
fix grammatical error
prior work on editing (Faltings et all,, 2021), our step overcoming turning Wikipedia single that close fulfilling needs.
Add reference to prior work text editing
Similar to prior work on text ediling--Feitingnn (Felting et 2924) our first step in overcoming this issue is turning to Wikipedia - single saurce that come close fulfiling all our needs.
add
Similar prior work text editing, first in overcoming this issue turning to Wikipedia- - e-single-sne setree----eee----- that come-clese to to-felfilling-- all eur-nedde
add prior work
Similartte priee-werle -ex--eeeeeeeee editing-oer -oer step overcoming issue turning Wikipedia.
add 'in overcoming this issue"
Our first stop in ovverrommingggeeoo this isswe-is turning to Wikipedia.
add a sentence
Figure 5: Exemplary application of PEER-Undo for decomposing sentence from this paper into multiple edits, terminating with an empty sequence. Plans are rewritten with PEER-Explaie in the opposite direction.
the result of iteratively applying PEER-Undo to selected sentence from this paper; corresponding plans are obtained from PÉER-Explain. As can be seen, PEER-Undo is able to decompose this sentence into sequence of meaningful edits despite not being exposed to any scientific papers during training. Somewhat surprisingly, both PEER-Undo and PEER-Explain are able to handle the references contained in this sentence, despite them being formatted in a completely different way than how we represent references during training on Wikipedia data (i.e.. replacing them with numeric identifier square brackets).
We next inspect PEER- EEEE-Enss 's ability to generate plans as discussed in Section 4 For an edit that we performed while writing this paper, Table 8 shows the plans generated with PEER-Exe Explann for different control sequences, using greedy decoding (see Appendix B for details on control se- quences). As can be seen, length a reasonable proxy for the amount of details in a plan: Con- straining the output to be short results in the plan "add citation", whereas for greater output length, BEER-Explain correctly identifies that two changes were made ("add citation for JFLEG and add a bit more detail"). Allowing word overlap between the plan and the edit results in plan that spec- ifies exactly which reference to add ("add reference to Napoles et al., 2017"). The final column of Table 8 shows the average probability of tokens in X according to PEER-Edit given both x, and
21

---

Published as a conference paper at ICLR 2023
Table Generated plans for an edit that we performed while writing this paper; the corresponding sequences X₃ and X++1 are shown on top, with changes highlighted in green. The final columns shows the average probability across tokens that our main PEER model assigns X given X and the respective plan. Control sequences enable us to specify the level of detail for plan.
X; JFLEG is a grammatical error correction dataset with single-sentened inputs.
JFLEG (Napoles et al., 2017) is [...] inputs written by English language learners.
Control Sequence
Output
Score
type natructionn 1ength-s
add citation
0.16
over_ap=false
type=instzuction length=m
add reference JFLEG
0.15
over-ap=false cype-instruction length-x1
add citation for JFLEG and add a bit more detail
0.17
overiap=falae
type=Lnstruction length=x1 overiap=true type-other length=x1 overiap=false
add reference Napoles al., 2017
0.26
Added a reference to the JFLEG paper
Figure 6: A document generated with PEER-Document for an edit that slightly modifies (top) by adding a citation some document with id the very end. As a control, we enforce that the document contains the substring "outperforms PEER on Natural Edits". The generated reference backs up the claim but contains
repetitions.
Importantly, the domain-adapted variants of PEER clearly outperform regular PEER for all subsets of Natural Edits.
[0] Ddmain-Adapted PEER for Natural Edits Springer for Research & Development
ra. ra. ww.springezz com
Domain-AAapter PEER for Natural Edits The main goal of this work is to develop domain- adaptive variant of PEER, which outperforms PEER on Natural Edits with respect to both the number of Natural Edits and the number of Natural Edits with respect to the number of natural edits. In this paper, present domain- adaptive
each of the generated plans. Naturally, the plan with word overlap is most helpful, resulting in the highest score; all other plans are about equally helpful PEER-Edit.
We finally look at the ability of PEER-Documen to generate plausible-looking reference documents, once again using an example sentence from this paper. For synthetic edit that just adds citation at the very end of this sentence, we sample five outputs from PEER-Document the best generated document among these five is shown in Figure As can be seen, the model able to produce somewhat plausible reference document that provides evidence for the claim x,. However, as exemplified by this document, we found model outputs to often contain numerous repetitions ("the number of Natural Edits and the number of Natural Edits")..
G HUMAN EVALUATION
G.1 QUALITY OF GENERATED EXPLANATIONS
In order to further evaluate the ability of PEER- Exxllain to provide good explanations for edits, we perform small-scale human evaluation. To this end, we randomly selected 50 Wikipedia edits with both their original comments and an explanation generated by PEER PEER-Explain Without knowing which explanation generated by our model and which explanation is derived from the original
"I accordance with the procedure described in Section we obtained the explanation from PEER- -XER-Explain by randomly sampling 10 different explanations and keeping the one that maximizes the probability of PEER- Edit performing the actual edit.
22

---

Published as a conference paper at ICLR 2023
comment, three authors of this paper selected their preferred explanation for each edit, based on how well it describes the edit being made. Overall, the original comment was preferred 28% of the time, whereas the explanation generated by FEER-Explain was preferred 59% of the time; in 13% of cases, both explanations were judged as being equally good. Inter-rater agreement was at %:; this may be due to the fact that annotators were not provided with a precise definition of what constitutes a good explanation.
G.2 QUALITY OF GENERATED DOCUMENTS
To assess the quality of documents generated by PEER-Document, randomly sampled 50 edits containing citations with quotes for which our training corpus does not contain the corresponding reference document. We used PEER-Document to generate new document snippets for each of them; three authors of this paper annotated all generated documents by answering the following three questions:
Relevance: Does the document support the claims made in the updated text?
Fluency: Does the document sound like natural text found on an English webpage?
Coherence: Is the document coherent and consistent?
Overall, we found 75% of the generated documents to be relevant, 89% to be fluent and 83% to be coherent. Inter-annotator agreement was consistently around 80% (88%, 84% and 80% for rel- evance, fluency and coherence, respectively). These results indicate that the majority of generated documents is of reasonably high quality to be used as synthetic data for finetuning PEER-Edit.
G.3 ERROR ANALYSIS
To get a better understanding of the kinds of errors that PEER-Edit makes when proposing edits, we manually looked at 100 input texts and plans that we randomly sampled from the Wikipedia subset of Natural Edits, and corresponding edits generated by PEER (SP. IB). For this, we only considered examples where the edit proposed by PEER not identical to the gold edit as we are interested in failure cases of PEER. Of the 100 edits selected, we found 50% to not contain any error. In total, 16% of the generated edits contained hallucinations; however, for 10 out of these 16 examples, the model was actually forced to hallucinate in order to follow the given instruction, because following the instruction required adding new information (such as a date of birth) that was not present in any of the provided reference documents, For another 10% of the considered examples, PEER-Edi did either not change the text at all, or only partially follow the instruction. In 5% cases, SEER-Edit made changes largely unrelated to the specified instructions and in 7% of cases, the generated edit contained multiple repetitions of the same phrase. Finally, for 12 out of 100 examples, the given instructions unclear or extremely vague, making it impossible for PEER follow them.
H LIMITATIONS
A major limitation of our approach is that at cach editing step, we assume the set D; to be given; the retrieval engine we use obtain D, (Petroni al. (022) makes use of the targets X which clearly is not possible in real-world applications. It would thus be interesting to investigate how incorporating retrieval engine that does not have access to X,+1 or even jointly training along with the model, as is done by Guu et al.] (2020) and Borgeaud et al. (2021), would affect results.
Despite being able use reference documents and obtaining comparably high QuestEval scores in our intro generation experiments, upon manual inspection we still found PEER to generate false statements or claims not backed up by the provided documents in many cases. While the ability to cite and quote generally makes it casier to check such hallucinations, citations can also make the model's s generations appear more authoritative, thus making more likely that users rely on them without explicit fact checking (Nakano al. 202T).
Finally, we use a very simple approach for representing edits by rewriting the entire paragraph. This makes PEER less efficient than other recent approaches for editing (Logan IV et al. 2021 Reid & Neubig 2022); also, our inefficient way of representing both inputs and outputs makes it impossible to handle entire documents, which we believe to be crucial for many real-world applications.
23

---

Published as a conference paper at ICLR 2023
Our evaluation is limited in that it only evaluates PEER and other models on small subset of poten- tial editing tasks in few different domains; all evaluations are performed in English only. Besides, also explore the collaborative potential of PEER only in very limited way: While arguably, the ability to follow human-written plans and perform a variety of edits both with and without reference documents (T in different domains (Table to cite and quote (Table(3). and to autonomously generate plans (Table 4 are important building blocks of collaborative model, it would be inter- esting for follow-up work consider entire sessions of human-Al interactions beyond individual examples like the one shown in Figure 3] However, this requires solving many of the challenges discussed previously, such having access an actual retrieval engine that can obtain relevant documents on the fly, finding suitable ways of evaluating texts jointly authored by humans and lan- guage models, and improving PEER's efficiency to enable processing entire documents.
24


Reviews:
[{"review_id": "Yj69ifaHjVy", "reviewer": "Reviewer_tmW2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed PEER, a language model that includes four skills: plan, edit, explain, and repeat. To better use the Wikipedia edit history with missing parts, the author proposed four infilling operations to overcome this issue: PEER-edit, PEER-undo, PEER-explain, PEER-document. The authors utilize the pretrained T5 to initialize PEER and train on Wikipedia edits. The experiment result shows the effectiveness of PEER on the Natural Edits and many downstream datasets.", "summary_of_the_review": "In general it's a good paper with solid results. I would like to recommend this paper be accepted.", "strength_and_weaknesses": "Strength:\n- This paper was the first to combine multiple collaborative writing skills together into a language model.\n- Solid experiment results show its effectiveness on various editing tasks in a zero-shot fashion and also citing and quoting tasks.\n- Enable collaborative editing for generating text as shown in Table 4.\n- New datasets: Natural Edits, NE-Cite and NE-Quote are introduced.\n\nWeaknesses:\n- Many of your downstream tasks include Wikipedia, which may be included in the training set of T5. Is there any way to prevent this issue?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "OHmiZQWC27", "reviewer": "Reviewer_fvQS", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a collaborative language model setup trained on Wikipedia edition history. PEER stands for plan, edit, explain, and repeat. The work is closely related to text editing and instruction LM literature. The paper makes an assumption that the grounded knowledge for editing is given. The authors trained several variances models with T5 including PEER-Edit, PEER-undo, PEER-explain, and PEER-Document, leveraging some of them to augment training data. They evaluated their models on several setting such as Natural Edits with comments, 0-shot testing on downstream tasks, citation and quote, and iterative editing. They showed their models are better than baseline with similar size and even better than some LLMs. ", "summary_of_the_review": "Overall, I think this is an exciting paper that tried to improve the usability of controllable language modeling. Making LMs controllable and interactive is definitely an essential next step for our field to move forward. My biggest concern is evaluation. It is hard to persuade people this model is ready without solid human evaluation since this task could be open-ended: the edition is the actual usage can be ambiguous, and the same plan can lead to multiple different outputs, similar to open-domain dialogues.", "strength_and_weaknesses": "S:\n- The idea of controlling the generation of language models step-by-step in a recurred manner is interesting.\n- The authors have evaluated their models on multiple settings, proving that their models trained on Wikipedia can potentially generalize to multiple downstream tasks.\n\nW:\n- Human evaluation is missing and could add more insights to the interactive process.\n- Even though there is a “explain” component in PEER, it is not evaluated and studied regarding its correctness. \n- No error analysis about the generated plans and the edited text. The authors only put a sentence at the end of the Appendix saying that “we still found PEER to generate false statements or claims not backed up by the provided documents in many cases“, but in the main paper there is no discussion or statistics on this weakness.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "J6XS2QmyFU4", "reviewer": "Reviewer_WDUk", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper focuses on collaborative writing based on pretrained LMs. In particular, it proposes a collaborative LM that writes drafts, adds suggestions, proposes edits, and provides explanations. To training is performed on edits and cited documents (at the next step) that is assumed to be available a priori as well as an infilling technique that allows generating data for new domains to increase applicability and training diversity. The evaluation focuses on planning in settings with edits and histories available across different domains, downstream editing tasks with human instructions, and quoting & citing, shows improved performance compared to unsupervised baselines or baselines pretrained on human instructions.", "summary_of_the_review": "An interesting proof-of-concept paper centering around an exciting application: collaborative writing with pretrained LMs. The approach has novel aspects such decomposing problem and synthesizing data for planning but it makes some unrealistic assumptions about the data availability and lacks comparison to simple few-shot baselines. Synthesizing data for planning should be useful for the research community but the technical part overall might have limited impact for the reasons explained above.  ", "strength_and_weaknesses": "**Strengths**\n\nIdea of training LMs to perform collaborative writing is interesting and the application exciting. Current tools for editing do not support such advanced functionalities and could have large  commercial potential. \n\nShowcases feasibility of collaborative writing with pretrained LMs (proof of concept) and improvements over unsupervised pretrained LMs and LMs that have been trained on a broad set of human instructions. \n\n**Weaknesses**\n\nThe approach requires training different LMs for four functionalities which is not ideal from maintainability and efficiency standpoint. Discussion about this design choice versus pretraining a single model to perform different actions would be useful.\n\nAssumption is that a large dataset of annotation is available for training and access to citations/documents which are not realistic. Even though an approach for extending to other domains is proposed, it still relies on annotated data and additional costly steps for data augmentation and training. This limitation has not been examined or discussed thoroughly in the paper. \n\nLack of baselines with vanilla in-context learning (ICL) with editing/performing undo/explaining examples. There have been a number of papers demonstrating that ICL can work surprisingly well. It would be interesting to compare with simple in-context learning with a few examples of edits for instance. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "GJgRL221sS", "reviewer": "Reviewer_eo6j", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a collaborative language model called PEER which is trained to imitate the process of collaborative writing. PEER with four T5 models can support the actions of editing, undo, adding explanation, and document generation. To overcome the problem of data scarcity and improve the generalization ability of PEER, the authors adopt self-training to infill the intermediate parts of training data, which increases the quality, amount and diversity of training data. Experimental results show the effectiveness of PEER across various domains and editing tasks.", "summary_of_the_review": "This paper proposes a novel collaborative language model and gives strong empirical results. I think it’s an insightful paper for NLP community and will recommend acceptance if the authors can successfully solve my concerns in the rebuttal.", "strength_and_weaknesses": "Strengths:\n\n1) Collaborative language modeling is an interesting and promising direction in NLG, which increases the interpretability and controllability of traditional NLG models. This direction also connects with the industrial products about writing assistants, making the technique of NLG more applicable.\n2) The proposed method based on self-training and instruction tuning is simple and effective, which directly supports the motivation to imitate the collaborative writing process.\n3) The experiment part is well organized. The four research questions are essential for collaborative language modeling. And the authors provide empirical results on diverse tasks and datasets to successfully answer each question.\n\nWeaknesses:\n\n1) Since PEER contains four different models which infill the missing part of training data, the authors should test the performance of each model (including PEER-Edit, PEER-Undo, PEER-Explain, and PEER-Document) to demonstrate the quality of augmented data. Especially the performance of PEER-Document should be shown and analyzed, because in my view it’s hard for a pre-trained model with 3B model parameters to generate high-quality knowledge documents only given the texts before / after editing and plans.\n\n2) More baselines should be included in the main experiments of text editing. For example, Table 1 only contains a copy-based baseline, which may exaggerate the improvement of PEER. At least the Wikipedia subset which contains the edit histories should have more baselines.\n\n3) In this paper, the plan is just defined as a short text sequence like instructions. This setting seems a little bit toy because it doesn’t clearly describe which part should be edited. This may put more burden on the design of instructions. From Figure 3, I find that the instructions mostly contain the position information, such as “add citation for the model being developed by Meta AI”. But it’s obviously hard when the text is long and has many objectives to describe. Thus, the design of plans should be further discussed to make this scenario more realistic.\n\n4) Typo: the second x_t –> x_{t+1} on the right side of Figure 2 (PEER-Document)", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%