Copy That! Editing Sequences by Copying Spans
Authors: Sheena Panthaplackel
Miltiadis Allamanis
Marc Brockschmidt
Venue:  (2020)
Keywords: span copying
sequence generation
editing
code repair

Abstract: Neural sequence-to-sequence models are finding increasing use in editing of documents, for example in correcting a text document or repairing source code. In this paper, we argue that existing seq2seq models (with a facility to copy single tokens) are not a natural fit for such tasks, as they have to explicitly copy each unchanged token. We present an extension of seq2seq models capable of copying entire spans of the input to the output in one step, greatly reducing the number of decisions required during inference. This extension means that there are now many ways of generating the same output, which we handle by deriving a new objective for training and a variation of beam search for inference that explicitly handle this problem.

In our experiments on a range of editing tasks of natural language and source code, we show that our new model consistently outperforms simpler baselines.

Under review as a conference paper at ICLR 2020
COPY THAT! EDITING SEQUENCES BY COPYING SPANS
Anonymous authors
Paper under double- blind review
ABSTRACT
Neural sequence-to-sequeene models are finding increasing use in editing of doe- uments, for example in correcting a text document or repairing source code. In this paper, argue that existing seq2seq models (with facility copy single tokens) are nor a natural fit for such tasks, as they have to explicitly copy each unchanged token. We present an extension of seq2seq models capable of copying entire spans of the input to the output in one step, greatly reducing the number of decisions required during inference. This extension means that there are now many ways of generating the same output, which we handle by deriving new objective for training and variation of beam search for inference that explicitly handle this problem.
In experiments range of editing tasks of natural language and source code, we show that our new model consistently outperforms simpler baselines.
INTRODUCTION
Intelligent systems that assist users in achieving their goals have become a focus of recent research. One class of such systems are intelligent editors that identify and correct errors in documents while they are written. Such systems are usually built on the seq2seq (Sutskever al., 2014) framework, in which an input sequence (the current state of the document) is first encoded into a vector repre- sentation and decoder then constructs new sequence from this information. Many applications of the seq2seq framework require the decoder to copy some words in the input. An example is ma- chine translation, in which most words are generated in the target language, but rare elements such as names are copied from the input. This can be implemented in an elegant manner by equipping the decoder with a facility that can "point" to words from the input, which are then copied into the output (Vinyals et al., 2015; Grave et al., 2017: Gulcehre al., 2016; Merity et al., 2017).
Editing sequences poses a different problem from other seq2seq tasks, as in many cases, most of the input remains unchanged and needs to be reproduced. When using existing decoders, this requires painstaking word-by- by-ord copying of the input. In this paper, we propose to extend a decoder with facility to copy entire spans of the input to the output in single step, thus greatly reducing the number of decoder steps required to generate an output. This is illustrated in Figure 1, where we show how our model inserts two new words into sentence by copying two spans of (more than) twenty tokens each.
However, this decoder extension exacerbates well-known problem in training decoders with copying facility: target sequence can be generated in many different ways when an output token can be generated by different means. In our setting, sequence of tokens can be copied token-by- token, in pairs of tokens, .... or in just single step. In practice, we are interested in encouraging our decoder to use as few steps as possible, both speed up decoding at inference time as well as to reduce the potential for making mistakes. To this end, we derive training objective that marginalises over all different generation sequences yielding the correct output, which implicitly encourages copying longer spans. At inference time, solve this problem by a variation of beam search that "merges" rays in the beam that generate the same output by different means.
In summary, this paper (i) introduces a new sequence decoder able copy entire spans (Sect. 2); (ii) derives a training objective that encourages our new decoder to copy long spans when possible, as well as an adapted beam search method approximating the exact objective; (iii includes extensive experiments showing that the span-copying decoder improves on editing tasks on natural language and program source code (Sect. 4).

---

Under review as a conference paper at ICLR 2020
Input
Output
charles dea ry - loc ock - (september ay: Copy(1 : 28)
charles P dea ry loc ock september
27, 1862 may 13, 1946) , was a british
27 1862 may 13 1946) ) british
literary scholar | who wrote on a wide ar-
literary scholar and translator , who wrote
ray of subjects, including chess, billiards R4: Copy (28 : 48 and cro que [
on a wide array of subjects including chess
billiards and cCO que 4
Figure I: Sample edit generated by our span-copying model on the WikiAtomicEditts dataset on the edit representation task of Yin et al. (2019). >> and <] represent the BPE start/end tokens. The model first copies a long initial span of text Copy(1 : 28). The next two actions generate the tokens "and" and "translator", as in standard sequence generation models. Then, the model again copies a long span of text and finally generates the end-of-sentence token (not shown).
2 MODEL
The core of our new decoder is span-copying mechanism that be viewed generalisation of pointer networks used for copying single tokens (Vinyals et al., 2015; Grave et al., 2017; Gulcehre et al., 2016; Merity et al., 2017). Concretely, modern sequence decoders treat copying from the input sequence as an alternative to generating a token from the decoder vocabulary, i.e. at each step, the decoder can either generate a token / from its vocabulary or it can copy the i-th token of the input. We view these as potential actions the decoder can perform and denote them by Gen(t) and Copy(i).
Formally, given an input sequence in in. 2m., the probability of a target sequence 01 .m is commonly factorised into sequentially generating all tokens of the output.
plo moo1...op Om in)
II p(o) in, .1..009-11 01-1) 15ism
(I)
Here, p(oy in, .0.00..--11 denotes the probability of generating the token oj, which is simply the probability of the Gen(t) action in the absence of a copying mechanism.' When can additionally copy tokens from the input, this probability is the sum of probabilities of all correct actions. To
formalise this, denote evaluation of an action a into concrete token as [a]. where [Gen(t]] and [Copy(i)] = in, Using g(a o) to denote the probability of emitting an action a after generating the partial output o, we complete Eq. (1) by defining
p(o; 01 .1...0j-1 0j-1)
Σ a.[0]=o3
g(a 01 01.0.09-1))
i.e. the sum of the probabilities of all correct actions.
Modelling Span Copying In this work, we are interested in copying whole subsequences of the input, introducing sequence copying action Copy(i 5) with [Copy(i: j)]l = in, in (in- dexing follows the Python in [i:j] notation here). This creates a problem because the number of actions required to generate an output token sequence is not equal to the length of the output sequence anymore; indeed, there may be many action sequences of different length that can produce the correct output.
As an example, consider Fig. 2, which illustrates all action sequences generating the output a b f de given the input a ee For example, we can initially generate the token a, or copy it from the input, or copy the first two tokens. If we chose one of the first two actions, we then have the choice of either generating the token or copying from the input and then have to generate the token f. Alternatively, if we initially choose to copy the first two tokens, we have to generate the token f next. We can compute the probability of generating the target sequence by traversing the diagram from the right to the left. p(c a de) is simply the probability of emitting stop token and requires no recursion. p(e f d) is the sum of the probabilities (Gen(e) a f d) p(E a de and q( Copy(4: 5) a d) p(c | a de), which re-uses the term already computed. Following this strategy, we can compute the probability of generating the output token sequence by computing
Note that all occurrences of p (and g helow) are implicitly (also) conditioned on the input sequence in, and so drop this in the following to improve readability.
2

---

Under review as a conference paper at ICLR 2020
b e)
plbf defelll (
plf a
p(de aabbf)
pic alabf
plc| Melabfd abfdd))
Gen(a)
Gen(b)
Gen(f)
Gen(d)
Gen(c)
Copy(U: : 1)
Copy(1 22
Copy(33 4)
Copy(4 :
Copy(0:2) 2)
Copy(3 55)
Figure 2: Illustration of different ways generating the sequence b f de given an input of abcde. Each box lists all correct actions at a given point in the generation process, and the edges after an action indicate which suffix token sequence still needs be generated after We use denote the empty sequence, either as prefix or suffix.
probabilities of increasing longer suffixes of it (essentially traversing the diagram in Fig. 2 from right to left).
Formally, we reformulate Eq. (1) into recursive definition that marginalises over all different sequences of actions generating the correct output sequence, following the strategy illustrated in Fig. 2. For this we define la), the length of the output of an action, i.e., Gen(t)] and [Copy(i: j)l jj - i. Note that we simply assume that actions Copy(i: j) with j I do not exist.
p Om 01 .1)
Σ a.
a(a 01 01...op)) ok) p(oktet1........................ Om 0o ox+e) (2)
Note that here, the probability of generating the correct suffix is only conditioned on the sequence generated so far and not on the concrete actions that yielded it. In practice, we implement this by conditioning our modelling of q at timestep on a representation h2 computed from the partial output sequence 01 ... Og- In RNNs, this is modelled by feeding the sequence of emitted tokens into the decoder, no matter how the decoder determined to emit these, and thus, one Copy(i: j) action may cause the decoder RNN to take multiple timesteps to process the copied token sequence. In causal self-attentional settings, this is simply the default behaviour.
We found that using the marginalisation in Eq. (2) during training is crucial for good results. In initial experiments, we tried an ablation in which we generate a per-token loss based only on the correct actions at each output token, without taking the remainder of the sequence into account (i.e., at each point in time, we used "multi-hot" objective in which the loss encourages picking any one of the correct actions). this setting, training yielded decoder which would most often only copy sequences of length one, as the objective was not penalising the choice of long action sequences explicitly. Our marginalised objective in Eq. (2) does exactly that, as explicitly reflects the cost of having to emit more actions than necessary, pushing the model towards copying longer subsequences. Finally, note that for numerical stability purposes our implementation works on the ae--robability space it is common for such methods, implementing the summation of probabilities with the standard log-sum-exp tricks.
Modelling Action Choices remains to explain how we model the per-step action distribution q(a o). We assume that have per-token encoder representations r1... Fn. of all input tokens and a decoder state h2 obtained after emitting the prefix 0 ----- This can be the state of an
RNN cell after processing the sequence 01 ... og. (potentially with attention over the input) or the representation of self-attentional model processing that sequence.
As in standard sequence decoders, we use output embedding projection applied to h to obtain scores Sk,v for all tokens in the decoder vocabulary. To compute a score for Copy(i: action, we use linear layer W project the concatenation r, rullr, of the (contextualised) embeddings of the respective input tokens to the same dimension as h2 and then compute their inner product:
$k.(f.) = (W (rellr))) h
We then concatenate all kk.v and (i))) and apply softmax to obtain our action distribution q(a o). Note that for efficient computation in GPUs, we compute the $k,(5.) for and j and mask all entries where j < i.
3

---

Under review as a conference paper at ICLR 2020
Algorithm Python-like pseudocode of beam search for span-copying decoders.
def beam beam_search (beam_size)
beam (toks: [START_OF_SEE SEQ], prob: 1} ]
out_lengtth -
while unfinished_rappe (beam) neæ_rays for ray in beam:
if 1en (ray. > out_ _etgte or ray.toks|-1] == END_ _F___F______
new_ app.raas.appeddraay
else:
for (act, act_prob) q(1 ray.toks):
new_rays. ap..ndy..apend ((tcks: ray.toks l| Jact], prob: ray. bty-prob+act_proot
beam = top_k (group__gr btppp__y_oosssst toks (new_rays), k-beam_sizze
out out_length +=
return beam
Training Objective We train the standard supervised sequence decoding setting, feeding to the decoder the correct output sequence independent of its decisions. We train by maximising p(o |) unrolled according to Eq. (2). One special case to note that we make a minor but important modification to handle generation of voo-bl-vocabulary words: iff' the correct token be copied from the input, GUn(UNK) is considered to be an incorrect action; otherwise only Gen(UNK) is correct. This is necessary avoid pathological cases in which there is no action sequence generate the target sequence correctly.
Beam Decoding Our approach to efficiently evaluate Eq. (2) at training time relies on knowledge of the ground truth sequence and so we need to employ another approach at inference time. We use variation of standard beam search which handles the fact that action sequences of the same length can lead to sequences of different lengths. For this, we consider a forward version of Eq. (2) in which we assume to have set of action sequences and compute lower bound on the true probability of sequence ... Og by considering all action sequences in A that evaluate to 01 .....
p(o1...op
IV
Σ
II g(a: 18
[ailll ... [I(a.-I).
(3)
EA
If A contains the set of all action sequences generating the output sequence 01 ... Oke Eq. (3) is an equality. At inference time, we under-approximmte A by generating likely action sequences using beam search. However, sequences yielding the same output sequence. This could be achieved by a final post- post-processing step (as in Eq. (3)), but we found that it is more effective sequence during the search. the example shown in Fig. 2, this means to sum up the probabilities of (for example) the action sequences Gen(a) (e)G)( and Copy(0 2), they generate the same output, To achieve this grouping of action sequences of different lengths, our search procedure is explicitly considering the length of the generated token sequence and "pauses" the expansion of action sequences that have generated longer outputs. We show the pseudocode for this procedure in Alg. 1, where merging of different rays generating the same output done using group by bty.tyyttooo toks.
have explicitly implement the summation of the probabilities of action
"merge" rays generating the same
RELATED WORK
Copying mechanisms are common in neural natural language processing. Starting from pointer networks (Vinyals et al., 2015), such mechanisms have been used across variety domains (AI- lamanis al., 2016; Gu et al., 2016; See et al., 2017) way copy elements from the input to the output, usually as a way to alleviate issues around rare, out-of-vocabulary tokens such as names. Marginalizing
single token-copying action and generation action has been previously con- sidered (Allamanis et al., 2016; Ling et al., 2016) but these works do not consider spans longer than one "unit".
Zhou et al. (2018) proposes a method to copy spans (for text summarization tasks) by predicting the start and end of span to copy. However, it lacks marginalization over different actions generating
4

---

Under review as a conference paper at ICLR 2020
the sequences and instead uses teacher foreing towards the longest copyable span; and it does not adapt its inference strategy to reflect this marginalization.
Our method is somewhat related to the work of van Merriënboer et al. (2017); Grave et al. (2019), who consider "multiscale" generation of sequences using a vocabulary of potentially overlapping word fragments. Doing this also requires marginalise out different decoder actions that yield the same output: in their case, generating a sequence from different combinations of word fragments, in contrast to our problem of generating sequence token-by-token
copying span. Hence, their training objective is similar to our objective in Eq. (2). A more important difference is that they use standard autoregressive decoder in which the emitted word fragments are fed back as inputs. This creates the problem of having different decoder states for the same output sequence (dependent on its decomposition into word fragments), which van Merriënboer et al. (2017) resolve by averaging the states of the decoder (an RNN using LSTM cells). Instead, we are following the idea of the graph generation strategy of Liu et al. (2018), where a graph decoder only conditioned on the partial graph that is being extended, and not the sequence of actions that generated the graph.
Recently, a number of approaches to sequence generation avoiding the left-to-right paradigm have been proposed (Welleck et al., 2019; Stern et al., 2019; Gu et al., 2019; Lee et al., 2018), usually by considering the sequence generation problem as an itcrative refinement procedure that changes or extends full sequence in each iteration step. Editing tasks could be handled by such models by learning to refine the input sequence with the goal of generating the output sequence. However, besides early experiments by Gu al. (2019), are not aware of any work that is trying do this. Note however that our proposed span-copying mechanism is also naturally applicable in settings that require duplication of parts of the input, e.g. when phrases or subexpressions need to be appear several times in the output (cf. obj in Figure 3 for a simple example). Finally, sequence-refinement models could also be extended to take advantage of our technique without large modifications, though we believe the marginalisation over all possible insertion actions (as in Eq. (2)) to be intractable in this setting. Similarly, Grangier & Auli (2017) present QuickFdit, a machine translation method that accepts source sentence (e.g. in German) guess sentence (e.g. in English) that is annotated (by humans) with change markers. It then aims to improve upon the guess by generating a better target sentence avoiding the marked tokens. This is markedly different the model accepts input the spans that need to be removed or retained in the guess sentence. In contrast, our model needs to automatically infer this information.
An alternative to sequence generation models for edits is the work of Gupta et al. (2017), who propose to repair source code by first pointing single line in the output and then only generate new version of that line. However, this requires a domain-specific segmentation of the input lines are often good choice for programs, but (multi-line) statements expressions valid choices well. Furthermore, the approach still requires to generate a sequence that is similar to the input line and thus could profit from our span-copying approach.
EXPERIMENTAL EVALUATION
We have implemented our span-copying decoder in PyTorch on top of both RNNs and Transformer models and will release the source code on http: / //ssshow///e// httm//afer///// htt////ttt////bblllaae/ ddubbll/ /blindd. We eval- uate the our RNN-based implementation on two types of tasks. First, we consider correction-style tasks in which a model has to identify an error in an input sequence and then generate an output sequence that a corrected version of the input. Second, we evaluate the performance of our models in the more complex setting of learning edit representations (Yin et al., 2019). In the evaluation below, COPY+SEQ2SEQ denotes a variant of our SEQCOPYSPAN in which the decoder can only copy single tokens (i.e., it is a biGRU encoder with GRU decoder with token copying).
4.1 CORRECTION TASKS
Correction tasks were one of the core motivations for new decoding strategy, as they usually require to reproduce most of the input without changing it, whereas only few tokens are removed, updated or added. We consider both corrections on source code as well as natural language.
5

---

Under review as a conference paper at ICLR 2020
Table Evaluation of models on the code repair task. Given an input code snippet, each model needs to predict corrected version of that code snippet. "Structural Match" indicates that the generated output identical the target output up to renaming the identifiers (i.e., variables, functions).
Accuracy
Accuracy @22
MRR
Structural Match
On BFP small
Tufano et al. (2019)
9.2%
43.5%
CoPY+SEQ2SEQ
14.8%
42.0%
0.177
18.2%
SEOCOPYSPAN
17.7% %
45.0%
0.247
21.2%
SEQCOPYSPAN (greedy)
15.3%
17.9%
SEOCOPYSPAN (merge rays end)
17.5%
41.6%
0.242
21.2%
SEQCOPYSPAN (force copy longest)
14.2%
33.7%
0.174
14.2%
On BFP medium
Tufano et al. (2019)
3.2%
22.2%
COPY+SEQ2SEQ
7.0%
23.8%
0.073 0.105
9.4%
SEACOPYSPAN
8.0 %
25.4%
13.7% %
Input
Output
C1
public boolean (Object
public boolean equalat Object ob)
aef Copy(6 7) ir
obj mull)
return falser
dsi Copy 18)
returr thissequall (ob));
retura this. ;
Figure 3: Generation of test example in BFP small (slightly modified for space and clarity). The SEQCOPYSPAN model learns to copy long spans while generating the necessary edits. The non- highlighted tokens in the output generated using Gen(t), whereas all other tokens copied from the input.
Code Repair Automated code repair systems (Monperrus, 2018) are commonly composed of two components, namely (heuristic) component that suggests potentially fixed versions of the input, and oracle (e.g., harness executing test suite) that checks the candidates for correctness. Recent software engineering research has started to implement the heuristic component using seq2seq models (Chen et al., 2018; Tufano et al., 2019; Lutellier et al., 2019). The models are usually viewed as language models (conditioned on the faulty code) or directly employ standard neural machine translation pipelines mapping from "faulty" to "correct" code. The task usually only requires minor changes to the input code and consequently most the input is copied into the output. We believe that our model is natural match for this setting.
To test this hypothesis, we use the two bug-fix pair (BFP) datasets of Tufano et al. (2019). The BFP small dataset contains pairs where each snippet has most 50 tokens and the BFP mePeimi dataset has Java snippets containing from 50 up to 150 tokens. The original version of the code has some form of bug, whereas the edited version fixes the relevant bug. This corpus was constructed by scraping Git commits and filtering for those that the commit message suggests that the edit fixes bugs. For both the SEPSSSE and SEOCOPYSPAN models we employ 2-layer biGRU as an encoder and single layer GRU decoder. The hidden dimensions of the GRUs are 128, whereas the embedding layer has dimensionality of 32. Note that the vocabulary size for this task just 400 by construction of the dataset. We employ a Luong-style (Luong et al., 2015) attention mechanism in the the decoders of both models.
Table 1 shows the results of our models, well as the original results reported by Tufano et al. (2019). Overall, the SEOCOPYSPAN model performs better on both datasets, achieving better prediction accuracy. This suggests that the span-copying mechanism is indeed beneficial in this setting, as it becomes quite visible in a qualitative analysis. Figure shows an example (slightly modified for readability) of a code repair prediction and the span-copying actions. In this case, the model has learned to copy all of the input code in chunks, extending it only by inserting some new tokens the middle.

---

Under review as a conference paper at ICLR 2020
Table 2: Span-based Correction (Bryant et al., 2017): Evaluation on Grammar Error Correction (GEC) Task. Note that our models use no pretraining, spell checking or other external data, which are commonly used in GEC tasks.
Precision (%)
Recall (%)
Fo.5
COPY+SEQ2SEQ SEOCOPYSPAN
34.9
6.4
0.1853
28.9
10.4
0.2134
For quantitative analysis, we additionally compute statistics for the greedy decoding strategy of SE- @COPYSPAN." In the BFP small dataset, JEQCOPYSPAN picks Copy(- action with a span longer than one token about three times per example, copying spans eight tokens long on average (median six). This suggests that the model has learned to take advantage of the span-copying mechanism, substantially reducing the number of actions that the decoder needs to perform.
We also find that the COPY+ model tends to (mistakenly) assign higher scores to the input sequence, with the input sequence being predicted as an output more often compared to the span- copying model: the MRR of the input sentence is 0.74 for the baseline COPY SEQSSE model compared to 0.28 for the SEQCOPY
model in the BFP small dataset. This suggests that the strong bias towards copying required of the baseline model (as most of the decoding actions are single-token copies) has the negative effect of sometimes "forgetting" to generate a change.
Finally, we this task to evaluate our choices in the beam decoding algorithm Alg. 1 the BFP dataset (Table 1). First, using only a greedy decoding mechanism achieves an exact match 2.4% less often than our beam search method, indicating that beam decoding is still helpful. Second, when using a "standard" beam decoding algorithm in which we merge the probabilities of different rays only in single post-processing step (i.e. directly implementing Eq. (3)), the accuracy of the top beam search result is only marginally worse, but the accuracy when considering the full beam is considerably affected. This expected, as Alg. ensures that rays are merged earlier, "freeing up" space in the beam for other results. This suggests that the added computational effort for merging beams allows the model to generate more diverse hypotheses. We also train and test model such that we provide supervision (teacher-force) copying the longest possible span, effectively disabling marginalization (Eq (2)). similar to Zhou et al. (2018). We find that the performance of the model significantly worsens (see Table 1). We believe that this is due to the fact that the model fails to capture the spectrum of correct actions possible since at each point it learns that only one action is correct.
Grammar Error Correction A counterpart to code repair in natural language processing is gram- mar error correction (GEC). Again, our span-copying model is natural fit for this task. However, this is a rich area of research with highly optimised systems, employing a series of pretraining techniques, corpus filtering, deterministic spell-checkers, etc. We do not wish to compete with existing methods, but instead are only interested in showing that our method can also benefit systems in this setting, Instead we compare our SEQCOPYSPAN model to our baseline CFPS+SFQ2SB model. Our models have a 2-layer bi-GRU encoder with hidden size of 64, single layer GRU decoder with hidden size of 64, tied embedding layer of size 64 and use a dropout rate of 0.2.
We use training/validation folds of the FCE (Yannakoudakis et al., 2011) and W&I+LOCNESS (Granger, 1998; Bryant et al., 2019) datasets for training and test on the test fold of the FCE dataset. These datasets contain sentences of non-native English students along with ground-trutt grammar error corrections from native speakers. Table shows the results computed with the ERRANT evaluation metric (Bryant al., 2017), where we can observe that our span-copying decoder again outperforms the baseline decoder. Note that the results of both models substantially below those of state of the art systems (e.g. Grundkiewicz al. (2019)), which employ (a) deterministic spell checkers (b) extensive monolingual corpora for pre- pre-training and (c) ensembling.
We focus greedy decoding here, statistics in the presence of merged rays in the heam easily become confusing.
7

---

Under review as a conference paper at ICLR 2020
Table 3: Evaluation of models on the edit representation tasks of Yin et al. (2019).
WikiAtomicFdits
GitHubEdits
C# Fixers
Accuracy
Accuracy
Accuracy
Yin et al. (2019)
72.9%
59.6%
n/a
COPY+SEQ2SHQ
67.8%
64.4%
18.8% 24.2%
SEQCOPYSPAN
78.1% %
67.4%
4.2 EDIT REPRESENTATIONS
We now turn our attention to the task of learning edit representations (Yin et al., 2019). The core idea of this task is to natural language and source code. The model consists of an edit encoder fa (x_, to transform the edit between x- and 24+ into an edit representation. Then, neural editor ofxx,, (x++ uses x- and the edit representation to reconstruct x_ as accurately as possible. We follow the same structure and employ experiments on the datasets used by Yin et al. (2019).
autoencoder-lik model structure to learn useful representations of edits of
ShoCoPySPAN decoder model the neural editor a. We perform
Our editor models have 2-layer biGRU encoder with hidden size of 64, a single layer GRU decoder with hidden size of 64, tied embedding layers with a hidden size of 64 and use a dropout rate of 0.2. In all cases the edit encoder fa is 2-layer biGRU with hidden size of 64. The GRU decoders of both models use a Luong-style attention mechanism (Luong et al., 2015).
Editing Wikipedia First, we consider the task of learning edit representations of small edits to Wikipedia articles (Faruqui et al., 2018). The dataset consists small "atomic" edits on Wikipedia article without any special filters. Table 3 suggest that the span-copying model achieves signif- icantly better performance in predicting the exact edit, even though our (nominally comparable) COPY+SEQ2SED model performs worse than the model used by Yin et al. (2019). Our initial exam- ple in Figure shows one edit example, where the model, given the input text and the edit representa- tion vector, is able to generate the output by copying two long spans and generating only the inserted tokens. Note that the WikiAtomicEddits dataset is made up of only insertions and deletions and the edit shown in Figure is generally representative of the other edits in the test set.
We now focus the code editing task of Yin et al. (2019) the GitHubEdits dataset, constructed from small (less than 3 lines) code edits scraped from C# GitHub repositories These include bug fixes, refactorings and other code changes. Again, the results in Table 3 suggest that span-based models outperforms the baseline by predicting the edited code more accurately.
Editing Code
Yin et al. (2019) also use the edit representations for one-shot learning-style task on "C# Fixers" dataset, which are small changes automatically constructed using automatic source code rewrites. Each edit is annotated with the used rewrite rule so that the dataset can be used to study how well an edit representation generalises from one sample to another.
As in Yin et al. (2019), train the models the larger and more general GitHubEdits dataset. To evaluate, we compute the edit representation ,x of one sample of a group of semantically similar edits in ## Fixers and feed it to the neural editor with the source code of another sample, iee. compute a<x=_, Ja (xr_, We repeat this experiment by picking the first 100 samples per fixer, computing the edit representation of each one and applying the edit to the other 99. The results of this process are shown in the last column of Table 3, suggesting that our span-copying models are able to improve on the one-shot transfer task as well.
Note that these results are not exactly comparable with those presented Yin al. (2019), as they randomly select 10 pairs (r_, 2.). ),, compute their edit representation and then for a given x'_ compute a(x', fA (x:,, .) for each of the 10 edit representations, finally reporting the best accuracy score
among the 10 candidates. Since this process cannot be replicated exactly due to the randomness of selecting samples, we opt for an alternate but reproducible process, as described above. Given, that
AAccnrding Yin al. (2019), a part of the data was corrupted and hence used smaller portion of the data.
8

---

Under review as a conference paper at ICLR 2020
SEOCOPYSPAN baseline is on par with the numbers reported in Yin et al. (2019) the other tasks, we believe that our SEQCOPY improves the results. Table 4 in the appendix presents a breakdown of the performance on the fixer data per fixer, showing that for some fixers model can substantially improve accuracy.
5 CONCLUSION
We have presented span-copying mechanism for commonly used encoder-decoder models. In many real-life tasks, machine learning models asked edit pre-existing input. Such models can take advantage of our proposed model. By correctly and efficiently marginalising over all possible span-copying actions we can encourage the model to learn take single span-copying action rather than multiple smaller per- per-token actions.
Of course, order for model copy spans. it needs be able represent all possible plans which is (n²²) to the input size. Although this is memory- inteesvvve O representations are common in sequence processing models (e.g. transformers). In the future, would be interesting to investigate alternative span representation mechanisms. Additionally, directly optimising for the target metrics of each task (rather than negative log-likellin sogllikelihood) can further improve the results for each task.

---

Under review as a conference paper at ICLR 2020
REFERENCES
Miltiadis Allamanis, Hao Peng, and Charles Sutton. A convolutional attention network for extreme summarization of source code. In International Conference Machine Learning, pp. 2091-2100, 2016.
Christopher Bryant, Mariano Felice, and Edward John Briscoe. Automatic annotation and evaluation of error types for grammatical error correction. In Association for Compulational Linguistics (ACL), 2017.
Christopher Bryant, Mariano Felice, Øistein E Andersen, and Ted Briscoe. The BEA-2019 shared task on grammatical error correction. In Proceedings of the Fourleenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 52-75, 2019.
Zimin Chen, Steve Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, and Mar- tin Monperrus. SequenceR: Sequence-to-sequeeee learning for end-to-end program repair. arXiv preprint 10Xiv:1901.01800 2018.
Manaal Faruqui, Ellie Pavlick, Ian Tenney, and Dipanjan Das. WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discourse. In Proc. of EMNLP. 2018.
Sylviane Granger. The computer learner corpus: versatile new source of data for SLA research. na, 1998.
David Grangier and Michael Auli. QuickEdit: Editing text & translations by crossing words out. arXiv preprini arXiv: 171.0400,, 2017.
Edouard Grave, Armand Joulin, and Nicolas Usunier. Improving neural language models with
continuous cache. In International Conference on Learning Representations (ICLR), 2017.
Édouard Grave, Sainbayar Sukhbaatar, Piotr Bojanowski, and Armand Joulin. Training hybrid lan- guage models by marginalizing over segmentations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1477-1482, 2019.
Roman Grundkiewiez, Marcin Junczys-Dowee Dowmunt, and Kenneth Heafield. Neural grammatical error correction systems with unsupervised pre- gre-training on synthetic data. In Proceedings of the Four- feenth Workshop Innovative Use of NLP for Building Educational Applications, pp. 252-265 263, 2019.
Jiatao Gu, Zhengdong Lu, Hang Li, and Victor OK Li. Incorporating copying mechanism in sequence- to-sequence learning. arXiv preprint arXiv: 1603.06333, 2016.
Jiatao Gu, Changhan Wang, and Jake 1rXiv:1955//1006, 2019.
Zhao.
Levenshtein transformer.
arXiv preprint
Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, and Yoshua Bengio. Pointing the unknown words. In Association for Computational Linguistics (ACL), 2016.
Rahul Gupta, Soham Pal, Aditya Kanade, and Shirish Shevade. Deepfix: Fixing common C language errors by deep learning. In Thirty- AAAI Conference on Artificial Intelligence, 2017.
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In Neural Information Processing Systems (NeurIPS). pp. 1693-1701, 2015.
Jason Lee, Elman Mansimov, and Kyunghyun Cho. Deterministic nonete neural sequence 2018.
modeling by iterative refinement. arXiv preprint
Wang Ling, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiský, Andrew Senior, Fumin Wang, and Phil Blunsom. Latent predictor networks for code generation. arXiv preprint arXiv:/603. 067444644, 2016.
Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander L. Gaunt. Constrained graph varia- tional autoencoders for molecule design. In International Conference on Learning Representations (ICLR), 2018
10

---

Under review as a conference paper at ICLR 2020
Minh-Thang Luong, Hieu Pham, and Christopher D Manning. Effective approaches to attention- based neural machine translation. arXiv preprint arXiv:1508. 2015.
Thibaud Lutellier, Lawrence Pang, Hung Viet Pham, Moshi Wei, and Lin Tan. ENCORE: Ensem- ble learning using convolution neural machine translation for automatic program repair. CoRR, abs/1906.08691, 2019.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations (ICLR), 2017.
Martin Monperrus. Automatic software repair: bibliography. ACM Computing Surveys (CSUR), 51 (1):17, 2018.
Abigail See, Peter J Liu, and Christopher D Manning. Get the point: Summarization with pointer- generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), volume pp. 1073-1083, 2017.
Mitchell Stern, William Chan, Jamie Kiros, and Jakob Uszkoreit. Insertion transformer: Flexible sequence generation via insertion operations. arXiv preprini arXiv: 1902.0329,, 2019.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Neural Information Processing Systems (NeurIPS), 2014.
Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk. An empirical study on learning bug-fixing patches in the wild via neural machine translation. ACM Transactions on Software Engineering and Methodology (TOSEM), 28(4):19, 2019.
Bart van Merriënboer, Amartya Sanyal, Hugo Larochelle, and Yoshua Bengio, Multiscale sequence modeling with a learned dictionary. arXiv preprint wrXiv:1707.00766 2017.
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. In Neural Information Processing Systems (NeurIPS), pp. 2692-2700, 2015.
Sean Welleck, Kianté Brantley, Hal Daumé III, and Kyunghyun Cho. Non- Non-monotonic sequential text generation. arXiv preprint arXiv:1902.2 0922. 2019.
Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. A new dataset and method for automatically grading esol texts. In Proceedings of the 49th Annual Meeling of the Association for Compu- lutional Linguisties: Human Language Technologies- Volume 1, pp. 180-189. Association for Computational Linguistics, 2011.
Pengcheng Yin, Graham Neubig, Miltiadis Allamanis, Mare Brockschmidt, and Alexander L Gaunt. Learning to represent edits. In International Conference on Learning Representations (ICLR), 2019.
Qingyu Zhou, Nan Yang, Furu Wei, and Ming Zhou. Sequential copying networks. In Thirty-Secom AAAI Conference on Artificial Intelligence, 2018.

---

Under review as a conference paper at ICLR 2020
Table 4: C# Fixer Accuracy (%) the One-Shot Generation Task
CoPY+SRQ2SEQ
SEOCOPYSPAN
@ 5
@
CA2007
16.8
24.4
36.9
46.5
IDE0004
14.8
20.8
23.5
33.6
RCS1015
24.0
25.3
23.9
26.8
RCS1021
1.8
4.4
7.8
16.8
RCS1032
1.8
2.7
2.5
3.7
RCS1058
20.6
20.9
19.9
22.7
RCS1077
3.2
3.9
4.5
5.8
RCS1089
59.8
59.8
59.8
59.9
RCS1097
1.6
3.7
14.9
27.7
RCS1118
45.1
69.6
46.0
55.6
RCS1123
15.8
19.5
27.7
22.7
RCS1146
12.2
16.5
19.7
31.5
RCS1197
1.1
1.8
1.7
2.3
RCS1202
6.5
8.4
11.6
23.3
RCS1206
34.9
35.0
36.2
37.5
RCS1207
2.1
4.2
5.0
8.2
Table 5: Indicative evaluation of models CNN/DM summarization.
BLEU ROUGE-1
ROUGE-2
ROUGE-3
ROUGE-L
COPY+SE02SEQ SEQCOPYSPAN
4.48
26.1
9.8
5.3
21.2
7.78
28.8
11.9
6.5
22.9
A APPENDIX
A.1 VISUALIZATION OF SPAN-COPYING ATTENTION
Figure 4 visualizes the copy-span attention for the greedy action sequence for the example in Figure 3.
A.2 DETAILED FIXER EVALUATION RESULTS
Table shows a breakdown of the performance our models on the fixers dataset of Yin et al. (2019).
A.3 RESULTS ON SEMI-EXTRACTIVE SUMMARIZATION
Abstractive and extractive summarization are two common tasks in NLP. Often abstractive summa- rization datasets, such the CNN-Daily Mail corpus (Hermann al., 2015) resemble extractive summarization to some extend. Here we aim to show that our SEOCoPYSPAN model can perform better than standard SEQY+S models. However, given the time limitations, we do fully replicate the summarization baselines and instead choose to use much smaller hidden states (each GRU has a hidden state of 64), fewer biRNN layers encoder layers), smaller embedding size (of hidden dimension 64), relatively small vocabulary (10k elements) etc. Our goal is merely to show how the two models compare.
Table 5 presents the results commonly used evaluation metrics, showing that the SEOCOPYSPAN performs better than the simpler COPY+SEQ2SEQ. The examples Figure 5 and Figure 6 show indicative summaries. Here the model leams "copy-paste" full sentences phrases construct a summary. Note that in all these examples, the action sequence taken by our model less than actions.
12

---

Under review as a conference paper at ICLR 2020
Stitt 5pon Rostoon
Starl Postiion
End Spon Postiion
(a) d1 Copy(1 9) has probability of 42.8%. The model is also predicting Gen(public) with 0.4% probability.
(b) 02: Gen(i#) probability 59.3% where as the highest span-copying action (Copy(9: 10)) has a prohability of only 0.8%.
anis
4001
Statt Spar Posttin
Spae Postion
(c) ag: Gen has probability of 99.9%. The (d) ag: Copy(6 7) has probability of 69.9%. highest span-copying action the correct C4py(4 The model is also (mistakenly) assigning 24.3% 5) but with negligible probability.
prohability to Gen ().
patk boien
polic
Delect
SSit Span Podtoo
Start Span Regaoo
Spani
Span Pouban
as: Gen(==) has probability of 66.0%. The highest span-copying action Copy(7 8) but with 0.8% probability.
(f) as: Copy(9 18) has probability of 43.7%. The model is also (mistakenly) assigning 8.3% probability to Gen(e1se).
Figure 4: Visualization of the attention weights over some of the greedy action sequence of Figure To fit this figure within one page, we only visualize attention at 022 az 04, a5. ag. Note that the color range changes per-figure allow for a better contrast in the visualization. Best viewed in screen.
13

---

Under review as a conference paper at ICLR 2020
Text
Scientists hoping to see 13 billion light years away. giving them a look into the carly years of the universe, are facing opposition from Native Hawaiian groups looking to preserve their past. Demenstrators including Game of Thrones Jason Momea demanded the state and University of lawaii stop construction of a new $1.4 hillion telescope sacred land. Dozens of protesters were arrested on Thursday mountain ground
Native Hawatian leader called a 30-day moratorium constraction. Thiry-one people were arrested during protests blocking access to the construction site for the $1.4 billion Thirty Meter Telescope on Mauna Howaii, Protesters say the mountaintop, where scientists are building the facility to see 13billion years into the top of sacred burial ground land
visited by the snow goddess Poli
Predicted Summary: Dozens of protesters were arrested at the Mauna Kea site in Hawaii. Actions:
Copy Span "Dozens of protesters were arrested"
Copy Span "at the Mauna Kea site"
Copy Span "in Hawai"".
Generate <eos>
Figure 5: Sample I: Semi-Extractive Summarization
Text
A Sydney leenage girl last seen leaving for school 40 years ago prohably ran away and may still be alive a coronial inquest has found. Marian Carole Rees when she disappeared from Hiilbdale southern Sydney early April 1975 after telling friend that she had forgotten something and jumped off her school bus. The teenager often talked of running away from home and had said goodbye to her brother the moming she disappeared, Magistrate Sharon Freund said findings handed down on Tharsday, Marion Carole Rees -LRB- pictured -RRB- who went missing 40 years ago may still be alive, according to an inquest...
Predicted Summary: Marian Carole Rees went missing 40 years ago may still be alive. Actions:
Copy Span "Marian Carole Rees"
Copy Span "went missing 40 years ago may still be alive"
Generate "!"
Generate <eos>
Predicted Summary: Marian Carole Rees was 13 when she disappeared from Hillsdale in southern
Sydney in early April 1975
Actions:
Copy Span "Marian Carole Rees was 13 when she disappeared from Hillsdale in southern Sydney in early April 1975"
Generate "!"
Generate <eos>
Figure 6: Sample 2: Semi-Extractive Summarization
A.4 SPAN COPYING STATISTICS
In Figure 7, we plot the frequency of the lengths of the copied spans for BFP and BFP medium- Given that the merging mechanism in beam decoding does not offer a unique way for measuring the length of the copied spans (actions of different lengths are often merged), we disable beam merging for this experiments. Overall, the results suggest that the model learns copy long sequences, although when single-copy actions are also common.
14

---

Under review as a conference paper at ICLR 2020
10-1
10-2
10-3
(a) BFP (Mean: 7.6 Median: 4)
(b) BFP mediuum (Mean: 30.7 Median: 16)
Figure 7: Length histograms of Copy(- -) actions during beam decoding in log-y scale. Beam merging is disabled for computing the statistics of this experiment. For BFP 43.7% of the copy actions are single-copy actions, whereas for BFPmedium 37.6% of the actions are single-copy actions. This suggests that SEOCOPYSPAN uses long span-copying actions in the majority of the cases where decides to take span-copying action.
15


Reviews:
[{"review_id": "rklla_W3KH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this work, the authors tackle the problem of span-based copying in sequence-based neural models. In particular, they extend the standard copying techniques of  (Vinyals et. al., Gulcehre et. al., etc.) which only allow for single-token copy actions. Their span-based copy mechanism allows for multiple tokens to be copied at a time during decoding via a recursive formulation that defines the output sequence distribution as a marginal over the complete set of action combinations that result in the sequence being produced. The authors also propose a span-based beam decoding algorithm that scores output sequences via a sum over the probabilities of action sequences that produce the same output. \n\nThe authors evaluate their model on four tasks: code repair, grammar error correction, editing wikipedia, and editing code. They find that their proposed technique consistently outperforms single-token copy-based seq2seq baselines. They also show that the efficacy of their proposed beam decoding mechanism and do some simple quantitative analysis that the model learns to copy spans longer than a single token.\n\n\nIn general, I found this paper to be very clearly written with very good motivation for the proposed solution. In addition, I thought the authors did a good job of testing their model against a wide range of benchmark problems. It seems that their copy extension is a meaningful contribution. \n\nI do, however, some questions regarding the evaluation, in particular the complexity of the baselines that were compared against. For example, the model consistently outperforms simple copy seq2seq baselines as well as the baselines in which the benchmark datasets were proposed (Tufano et. al, Yin et. al.) However, it does not seem the span-based copying method is state-of-the-art. If it is not state-of-the-art, how far off the SOTA is this proposed architecture? Did the authors do any analysis whereby the span-copy mechanism was added to an existing SOTA model, and if so, did this still produce gains? It's a bit difficult to situate the exact power of this new mechanism, given that it is often only compared to a simplistic copy-seq2seq method. \n\nOther questions/feedback I have for the authors:\n1) How efficient/scalable is the proposed mechanism? I would like to see a more formal treatment of the run-time of the training marginalization operation.\n2) It would be nice to see a quantitative analysis for distribution of sequence lengths copied over (like some sort of histogram) for the datasets.\n3) It would also be helpful to add some short descriptions of the benchmark datasets.\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "H1xT0KVecH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper study the problem of editing sequences, such as natural language or code source, by copying large spans of the original sequence. A simple baseline solution to this problem is to learn a sequence to sequence neural network, which generates the edited sequence conditioned on the original one. This method can be improved by adding a copying mechanism, based on pointer networks, to copy tokens from the input. However, most of such existing approaches can only copy one input token at a time, which is a limitation when most of the input should be copied, which is the case for most editing tasks. In this paper, the authors propose a mechanism that can copy entire spans of the input instead of just individual tokens. In that case, a particular sequence can often be generated by many different actions (eg. copying individual tokens, pairs of tokens, or the whole span). It is thus important to marginalize over all the actions that generated a particular sequence. This can be done efficiently, using dynamic programming, if the probability of an action depends on the generated tokens only, but not on the sequence of actions used to generate them. In the case of neural network, this means that the decoder of the model takes the tokens as input, instead of the spans. To represent spans, the authors propose to use the concatenation of the hidden states corresponding to the beginning and end of the span. Then the probability of copying a span is obtained by taking the dot product between this representation and the current hidden state of the decoder, and applying the softmax. The authors evaluate the proposed approach on the following tasks: code repair, grammar error correction and edit representations (on wikipedia and c# code).\n\nThe paper is well written, and easy to follow, even if some sections could be a bit more detailed (for example, the section on \nbeam search decoding). The problem studied in the paper - copying spans from the input - is interesting, and has applications in NLP or code generation. I think that the the proposed solution is technically sound.\nHowever, I have some concerns regarding the paper. First I believe that many relevant prior works are not discussed in the paper, making some technical contributions of the paper not novel. For example, previous methods were proposed to copy \nspans from the input [1], to edit existing sequences [2], or to marginalize over different generation sequences\nby conditioning only on the generated tokens (instead of the actions the generated the sequence) [3,4]. The body of work on iterative refinement for sequence generation is also probably relevant to this paper [5,6]. Additionally, I found the experimental section a bit weak, as most of the baseline used to compare seem a bit weak. The proposed method is mostly compared on datasets that are relatively new, or on tasks such as the grammar error correction where strong methods were excluded.\n\nOverall, I found the paper well\twritten, and the proposed method to make sense. Unfortunately, I believe that the work is a bit incremental, most of the technical contributions having already been published. Since the experimental results are not very strong, I do not think the paper is good enough for publication to the ICLR conference.\n\n== References ==\n\n[1] Sequential Copying Networks, Qingyu Zhou, Nan Yang, Furu Wei, Ming Zhou, AAAI 2018.\n[2] QuickEdit: Editing text & translations via simple delete actions, David Grangier, Michael Auli, 2017\n[3] Latent Predictor Networks for Code Generation, Wang Ling, Edward Grefenstette, Karl Moritz Hermann, Tomas Kocisky, Andrew Senior, Fumin Wang, Phil Blunsom, ACL 2016\n[4] Training Hybrid Language Models by Marginalizing over Segmentations, Edouard Grave, Sainbayar Sukhbaatar, Piotr Bojanowski, Armand Joulin, ACL 2019\n[5] Mask-Predict: Parallel Decoding of Conditional Masked Language Models, Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer, EMNLP 2019\n[6] Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement, EMNLP 2018", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BJxj26u45S", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a new decoding mechanism that allows  span-copying which can be viewed as a generalisation of\npointer networks.\n\nBecause action sequences of the same length can lead to sequences of different lengths decoding becomes tricky therefore authors propose a variation of standard beam search that calculates a lower bound of sequence probabilities rather than the true probability of generation this is achieved in practice by merging probabilities of sampled rays of actions generating the same sequence during the search.\n\none advantage of this proposed model is that it doesn't need to copy word by word to update sequences which need minor changes, rather than the seq2seq model with copy actions which due to the way we train those models using NLL loss will likely assign high probabilities to the non-modified input. \n\nAuthors evaluate their model against traditional seq2seq models with copy actions over a set of tasks:\n* code correction tasks: two bug-fix pair (BFP) datasets of Tufano et al. (2019)\n* grammar error correction (Bryant et al., 2017)\n* learning edit representations \n\nPros: \nOverall I am in favour of this work acceptance it represents a neat modelling for copying sequences that integrated simply with seq2seq models, especially the transformer model. \n\nCons: \n- One of the drawbacks of this method is the decoding strategy although authors present a motivated solution for that. The proposed variation of beam search by calculating the lower bound solution seems adhoc and some corner cases are not explained in the paper (see the question below). \n- Experiments could have been more thorough, especially in terms of architectures. I was disappointed not to see authors only comparing between GRU based seq2seq with copy actions, one baseline and their model implementation over a biGRU seq2seq. \n\nQuestions to authors: \n- During inference using the proposed variation of beam search (e.g. k=5), What will happen for example if one ray of actions was dropped because not of the top 5 this ray of actions if continued using future actions would map to one existing top-scoring rays? do you do a way to control that? \n- What is the reason behind choosing the bi-gru architecture? \n\nMissing references: \nThere are a couple of similar work that authors might want to add: \n* Latent Predictor Networks for Code Generation https://arxiv.org/pdf/1603.06744.pdf\n* An Operation Network for Abstractive Sentence Compression https://www.aclweb.org/anthology/C18-1091.pdf\n* Levenshtein Transformer https://arxiv.org/pdf/1905.11006.pdf", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an addition to seq2seq models to allow the model to copy spans of tokens of arbitrary length in one step. The authors argue that this method is useful in editing applications where long spans of the output sequence will be exact copies of the input. Reviewers agreed that the problem is interesting and the solution technically sound. However, during the discussion phase there were concerns that the method was too incremental to warrant publication at ICLR. The work would be strengthened with a more thorough discussion of related work and additional experiments comparing with the relevant baselines as suggested by Reviewer 2.
Decision: Reject