Copy That! Editing Sequences by Copying Spans

1 INTRODUCTION . Intelligent systems that assist users in achieving their goals have become a focus of recent research . One class of such systems are intelligent editors that identify and correct errors in documents while they are written . Such systems are usually built on the seq2seq ( Sutskever et al. , 2014 ) framework , in which an input sequence ( the current state of the document ) is first encoded into a vector representation and a decoder then constructs a new sequence from this information . Many applications of the seq2seq framework require the decoder to copy some words in the input . An example is machine translation , in which most words are generated in the target language , but rare elements such as names are copied from the input . This can be implemented in an elegant manner by equipping the decoder with a facility that can “ point ” to words from the input , which are then copied into the output ( Vinyals et al. , 2015 ; Grave et al. , 2017 ; Gulcehre et al. , 2016 ; Merity et al. , 2017 ) . Editing sequences poses a different problem from other seq2seq tasks , as in many cases , most of the input remains unchanged and needs to be reproduced . When using existing decoders , this requires painstaking word-by-word copying of the input . In this paper , we propose to extend a decoder with a facility to copy entire spans of the input to the output in a single step , thus greatly reducing the number of decoder steps required to generate an output . This is illustrated in Figure 1 , where we show how our model inserts two new words into a sentence by copying two spans of ( more than ) twenty tokens each . However , this decoder extension exacerbates a well-known problem in training decoders with a copying facility : a target sequence can be generated in many different ways when an output token can be generated by different means . In our setting , a sequence of tokens can be copied token-bytoken , in pairs of tokens , . . . , or in just a single step . In practice , we are interested in encouraging our decoder to use as few steps as possible , both to speed up decoding at inference time as well as to reduce the potential for making mistakes . To this end , we derive a training objective that marginalises over all different generation sequences yielding the correct output , which implicitly encourages copying longer spans . At inference time , we solve this problem by a variation of beam search that “ merges ” rays in the beam that generate the same output by different means . In summary , this paper ( i ) introduces a new sequence decoder able to copy entire spans ( Sect . 2 ) ; ( ii ) derives a training objective that encourages our new decoder to copy long spans when possible , as well as an adapted beam search method approximating the exact objective ; ( iii ) includes extensive experiments showing that the span-copying decoder improves on editing tasks on natural language and program source code ( Sect . 4 ) . Input . 2 MODEL . The core of our new decoder is a span-copying mechanism that can be viewed as a generalisation of pointer networks used for copying single tokens ( Vinyals et al. , 2015 ; Grave et al. , 2017 ; Gulcehre et al. , 2016 ; Merity et al. , 2017 ) . Concretely , modern sequence decoders treat copying from the input sequence as an alternative to generating a token from the decoder vocabulary , i.e . at each step , the decoder can either generate a token t from its vocabulary or it can copy the i-th token of the input . We view these as potential actions the decoder can perform and denote them by Gen ( t ) and Copy ( i ) . Formally , given an input sequence in = in1 . . . inn , the probability of a target sequence o1 . . . om is commonly factorised into sequentially generating all tokens of the output . p ( o1 . . . om | in ) = ∏ 1≤j≤m p ( oj | in , o1 . . . oj−1 ) ( 1 ) Here , p ( oj | in , o1 . . . oj−1 ) denotes the probability of generating the token oj , which is simply the probability of the Gen ( t ) action in the absence of a copying mechanism.1 When we can additionally copy tokens from the input , this probability is the sum of probabilities of all correct actions . To formalise this , we denote evaluation of an action a into a concrete token as JaK , where JGen ( t ) K = t and JCopy ( i ) K = ini . Using q ( a | o ) to denote the probability of emitting an action a after generating the partial output o , we complete Eq . ( 1 ) by defining p ( oj | o1 . . . oj−1 ) = ∑ a , JaK=oj q ( a | o1 . . . oj−1 ) , i.e . the sum of the probabilities of all correct actions . Modelling Span Copying In this work , we are interested in copying whole subsequences of the input , introducing a sequence copying action Copy ( i : j ) with JCopy ( i : j ) K = ini . . . inj−1 ( indexing follows the Python in [ i : j ] notation here ) . This creates a problem because the number of actions required to generate an output token sequence is not equal to the length of the output sequence anymore ; indeed , there may be many action sequences of different length that can produce the correct output . As an example , consider Fig . 2 , which illustrates all action sequences generating the output a b f d e given the input a b c d e. For example , we can initially generate the token a , or copy it from the input , or copy the first two tokens . If we chose one of the first two actions , we then have the choice of either generating the token b or copying it from the input and then have to generate the token f . Alternatively , if we initially choose to copy the first two tokens , we have to generate the token f next . We can compute the probability of generating the target sequence by traversing the diagram from the right to the left . p ( | a b f d e ) is simply the probability of emitting a stop token and requires no recursion . p ( e | a b f d ) is the sum of the probabilities q ( Gen ( e ) | a b f d ) · p ( | a b f d e ) and q ( Copy ( 4 : 5 ) | a b f d ) · p ( | a b f d e ) , which re-uses the term we already computed . Following this strategy , we can compute the probability of generating the output token sequence by computing 1Note that all occurrences of p ( and q below ) are implicitly ( also ) conditioned on the input sequence in , and so we drop this in the following to improve readability . probabilities of increasing longer suffixes of it ( essentially traversing the diagram in Fig . 2 from right to left ) . Formally , we reformulate Eq . ( 1 ) into a recursive definition that marginalises over all different sequences of actions generating the correct output sequence , following the strategy illustrated in Fig . 2 . For this we define |a| , the length of the output of an action , i.e. , |Gen ( t ) | = 1 and |Copy ( i : j ) | = j − i . Note that we simply assume that actions Copy ( i : j ) with j ≤ i do not exist . p ( ok+1 . . . om | o1 . . . ok ) = ∑ a , ∃ ` .|a|= ` JaK=ok+1 ... ok+ ` q ( a | o1 . . . ok ) · p ( ok+ ` +1 . . . om | o1 . . . ok+ ` ) ( 2 ) Note that here , the probability of generating the correct suffix is only conditioned on the sequence generated so far and not on the concrete actions that yielded it . In practice , we implement this by conditioning our modelling of q at timestep k on a representation hk computed from the partial output sequence o1 . . . ok . In RNNs , this is modelled by feeding the sequence of emitted tokens into the decoder , no matter how the decoder determined to emit these , and thus , one Copy ( i : j ) action may cause the decoder RNN to take multiple timesteps to process the copied token sequence . In causal self-attentional settings , this is simply the default behaviour . We found that using the marginalisation in Eq . ( 2 ) during training is crucial for good results . In initial experiments , we tried an ablation in which we generate a per-token loss based only on the correct actions at each output token , without taking the remainder of the sequence into account ( i.e. , at each point in time , we used a “ multi-hot ” objective in which the loss encourages picking any one of the correct actions ) . In this setting , training yielded a decoder which would most often only copy sequences of length one , as the objective was not penalising the choice of long action sequences explicitly . Our marginalised objective in Eq . ( 2 ) does exactly that , as it explicitly reflects the cost of having to emit more actions than necessary , pushing the model towards copying longer subsequences . Finally , note that for numerical stability purposes our implementation works on the log-probability space as it is common for such methods , implementing the summation of probabilities with the standard log-sum-exp tricks . Modelling Action Choices It remains to explain how we model the per-step action distribution q ( a | o ) . We assume that we have per-token encoder representations r1 . . . rn of all input tokens and a decoder state hk obtained after emitting the prefix o1 . . . ok−1 . This can be the state of an RNN cell after processing the sequence o1 . . . ok ( potentially with attention over the input ) or the representation of a self-attentional model processing that sequence . As in standard sequence decoders , we use an output embedding projection applied to hk to obtain scores sk , v for all tokens in the decoder vocabulary . To compute a score for a Copy ( i : j ) action , we use a linear layer W to project the concatenation ri‖rj of the ( contextualised ) embeddings of the respective input tokens to the same dimension as hk and then compute their inner product : sk , ( i , j ) = ( W · ( ri‖rj ) ) · h > k We then concatenate all sk , v and sk , ( i , j ) and apply a softmax to obtain our action distribution q ( a | o ) . Note that for efficient computation in GPUs , we compute the sk , ( i , j ) for all i and j and mask all entries where j ≤ i. Algorithm 1 Python-like pseudocode of beam search for span-copying decoders . def beam_search ( beam_size ) beam = [ { toks : [ START_OF_SEQ ] , prob : 1 } ] out_length = 1 while unfinished_rays ( beam ) : new_rays = [ ] for ray in beam : if len ( ray.toks ) > out_length or ray.toks [ -1 ] == END_OF_SEQ : new_rays.append ( ray ) else : for ( act , act_prob ) in q ( · | ray.toks ) : new_rays.append ( { toks : ray.toks ‖ JactK , prob : ray.prob * act_prob } ) beam = top_k ( group_by_toks ( new_rays ) , k=beam_size ) out_length += 1 return beam Training Objective We train in the standard supervised sequence decoding setting , feeding to the decoder the correct output sequence independent of its decisions . We train by maximising p ( o | ) unrolled according to Eq . ( 2 ) . One special case to note is that we make a minor but important modification to handle generation of out-of-vocabulary words : iff the correct token can be copied from the input , Gen ( UNK ) is considered to be an incorrect action ; otherwise only Gen ( UNK ) is correct . This is necessary to avoid pathological cases in which there is no action sequence to generate the target sequence correctly . Beam Decoding Our approach to efficiently evaluate Eq . ( 2 ) at training time relies on knowledge of the ground truth sequence and so we need to employ another approach at inference time . We use a variation of standard beam search which handles the fact that action sequences of the same length can lead to sequences of different lengths . For this , we consider a forward version of Eq . ( 2 ) in which we assume to have a set of action sequences A and compute a lower bound on the true probability of a sequence o1 . . . ok by considering all action sequences in A that evaluate to o1 . . . ok : p ( o1 . . . ok ) ≥ ∑ [ a1 ... an ] ∈A Ja1K‖ ... ‖JanK=o1 ... ok ∏ 1≤i≤n q ( ai | Ja1K‖ . . . ‖Jai−1K ) . ( 3 ) If A contains the set of all action sequences generating the output sequence o1 . . . ok , Eq . ( 3 ) is an equality . At inference time , we under-approximate A by generating likely action sequences using beam search . However , we have to explicitly implement the summation of the probabilities of action sequences yielding the same output sequence . This could be achieved by a final post-processing step ( as in Eq . ( 3 ) ) , but we found that it is more effective to “ merge ” rays generating the same sequence during the search . In the example shown in Fig . 2 , this means to sum up the probabilities of ( for example ) the action sequences Gen ( a ) Gen ( b ) and Copy ( 0 : 2 ) , as they generate the same output . To achieve this grouping of action sequences of different lengths , our search procedure is explicitly considering the length of the generated token sequence and “ pauses ” the expansion of action sequences that have generated longer outputs . We show the pseudocode for this procedure in Alg . 1 , where merging of different rays generating the same output is done using group_by_toks .

Summary: In this work, the authors tackle the problem of span-based copying in sequence-based neural models. In particular, they extend the standard copying techniques of  (Vinyals et. al., Gulcehre et. al., etc.) which only allow for single-token copy actions. Their span-based copy mechanism allows for multiple tokens to be copied at a time during decoding via a recursive formulation that defines the output sequence distribution as a marginal over the complete set of action combinations that result in the sequence being produced. The authors also propose a span-based beam decoding algorithm that scores output sequences via a sum over the probabilities of action sequences that produce the same output.