Encoder-decoder Network as Loss Function for Summarization

We present a new approach to defining a sequence loss function to train a summarizer by using a secondary encoder-decoder as a loss function , alleviating a shortcoming of word level training for sequence outputs . The technique is based on the intuition that if a summary is a good one , it should contain the most essential information from the original article , and therefore should itself be a good input sequence , in lieu of the original , from which a summary can be generated . We present experimental results where we apply this additional loss function to a general abstractive summarizer on a news summarization dataset . The result is an improvement in the ROUGE metric and an especially large improvement in human evaluations , suggesting enhanced performance that is competitive with specialized state-of-the-art models . 1 INTRODUCTION . Neural networks are a popular solution to the problem of text summarization , the task of taking as input a piece of natural language text , such as a paragraph or a news article , and generating a more succinct text that captures the most essential information from the original . One popular type of neural network that has achieved state of the art results is an attentional encoderdecoder neural network See et al . ( 2017 ) ; Paulus et al . ( 2018 ) ; Celikyilmaz et al . ( 2018 ) . In an encoder-decoder network , the encoder scans over the input sequence by ingesting one word token at a time to create an internal representation . The decoder is trained to compute a probability distribution over next words conditioned on a sequence prefix . A beam search decoder is typically used to find a high likelihood output sequence based on these conditional word probability distributions . Since the next word depends heavily on previous words , the decoder has little hope of a correct distribution over next words unless it has the correct previous words . Thus the decoder is typically trained using teacher forcing Williams & Zipser ( 1989 ) , where the reference sequence prefix is always given to the decoder at each decoding step . In other words , regardless of what distributions are output by the decoder in training for timesteps ( 1 , ... , t−1 ) , at timestep t , it is given the reference sequence prefix ( y∗1 , ... , y ∗ t−1 ) and asked to output the distribution P ( yt|y∗1 , ... , y∗t−1 ) . Teacher forcing suffers from exposure bias Ranzato et al . ( 2016 ) . At test time , the input to the decoder is not the reference sequence prefix ( y∗1 , ... , y ∗ t−1 ) but a ( y1 , ... , yt−1 ) constructed based on the decoder ’ s own outputs . Since the decoder is not perfect , as t increases so too does the disparity between these two sequence prefixes , causing an increasing mismatch between training and test conditions . Training at the sequence level can alleviate this discrepancy , but requires a differentiable loss function . In the Related Work section we review previous efforts . We present a novel approach to address the problem by defining a loss function at the sequence level using an encoder-decoder network as a loss function . In training , the summarizer ’ s beam search decoded output sequence is fed as input into another network called the recoder . The recoder is an independent attentional encoder-decoder trained to produce the reference summary . Our experiments show that adding the recoder as a loss function improves a general abstractive summarizer on the popular CNN/DailyMail dataset Hermann et al . ( 2015 ) ; Nallapati et al . ( 2016 ) , achieving significant improvements in the ROUGE metric and an especially large improvement in human evaluations . 2 ATTENTIONAL ENCODER-DECODER MODEL FOR SUMMARIZATION . We first give a high level overview of an attentional encoder-decoder model for sequence to sequence learning . We present the specific abstractive model of See et al . ( 2017 ) , which serves as the baseline model in our experiments for comparison . We chose this as the baseline model because it is a general and popular model whose concepts have often appeared in other abstractive summarization works , and its results have often been used as a comparison baseline Chen & Bansal ( 2018 ) ; Li et al . ( 2018 ) ; Celikyilmaz et al . ( 2018 ) . The ideas presented in this paper are largely independent of the specific summarizer and may be applicable to the training of other sequence to sequence models , many of which are also trained at the word loss level using teacher forcing . Since our focus is to demonstrate the effectiveness of our new loss function , the generality of the See et al . ( 2017 ) model , which does not account for specifics of the summarization problem such as article or sentence structure Li et al . ( 2018 ) , suits our purpose well . 2.1 INPUTS AND OUTPUTS . The source article and target summary are each treated as a sequence of tokens . A token represents either a word or a punctuation mark . Let ( xi ) be the sequence of tokens in an article or input sequence , where i ranges from 1 to the length of the input sequence . Let ( yt ) be the sequence of word tokens in the summary or output sequence , where 1 ≤ t. The goal is to train a neural network to compute , at each timestep t , the probability distribution P ( yt|y1 , ... , yt−1 ) over next tokens yt given previous tokens ( y1 , ... , yt−1 ) . In the teacher forcing training method , the network is trained to minimize the loss Jml , also called the maximum likelihood loss , defined as the average over timesteps t of ( Jml ) t = − logP ( y∗t |y∗1 , ... , y∗t−1 ) where the sequence prefix y∗1 , ... , y ∗ t−1 is from the reference summary . At test time , the network computes P ( yt|y1 , ... , yt−1 ) where y1 , ... , yt−1 are chosen by a beam search decoder based on previously output distributions . The sequence ends when the special token STOP , appended to the end of all reference summaries in training , is output . 2.2 ENCODING . Each xi is embedded as a dense vector wi using a v × d weight matrix Wemb , where v is the size of the vocabulary and d is the number of hidden dimensions , a hyperparameter . The weights Wemb are learned as part of training . Each wi is given in order as input into an LSTM to get hidden states ( hfi ) , and also in reverse order into another LSTM to get backward states ( h b i ) . The two state sequences are then stacked to form hidden states ( hi ) for reference by the decoding phase ’ s attention mechanism . 2.3 DECODING WITH ATTENTION . The decoding phase also uses an LSTM . At each output timestep t , a token y is embedded as a dense vector using the sameWemb and fed into the LSTM , and the LSTM outputs a probability distribution over next tokens . For t = 0 , y is the special placeholder symbol START . For t > 0 , when training using teacher forcing , y = y∗t−1 is the corresponding token from the reference sequence . When beam search decoding at test time , y = yt−1 is chosen based on the decoder ’ s output probabilities at step t− 1 . This difference in information about previous output tokens causes discrepancy between training and test conditions . The decoder makes use of an additional piece of information called attention Bahdanau et al . ( 2015 ) , which allows it to “ read ” the encoder ’ s output states . At each timestep with decoder LSTM state st , the attention vector is computed as at = softmax ( g T tanh ( Whhi +Wsst + battn ) ) where g , Wh , Ws , and battn are parameters to be trained . This vector gives a distribution over encoder states at timestep t. The attentional context vector h∗t , a “ reading ” of encoder states , is then computed as h∗t = ∑ i ( at ) ihi This context vector is concatenated with the LSTM state st to produce the dense vector outputs qt = V [ st , h ∗ t ] + b , where [ ] denotes concatenation of vectors . Finally , another linear transformation using a weight matrix V ′ and weight vector b′ is used to map these outputs in the hidden space , through a softmax layer , into output probabilities over the vocabulary . That is , Pvocab ( yt|y1 , ... , yt−1 ) = softmax ( V ′qt + b′ ) where V , V ′ , b , b′ are trainable parameters . 2.4 POINTER GENERATOR AND COVERAGE . The pointer generator mechanism allows the decoder to copy a token directly from the input . It is particularly helpful for out-of-vocabulary tokens . The decoder “ copies ” source tokens with a distribution based on the attention vector at and mixes it in with weight 1 − pgen , where pgen is a trainable function of the context vectors , states , and next word input . The final distribution P ( yt|y1 , ... , yt−1 ) is the combined pgenPvocab ( yt|y1 , ... , yt−1 ) + ( 1− pgen ) ∑ i : xi=yt ati The summarizer loss function JS = Jml is computed based on this distribution . In one model variant , a coverage loss Jcov that encourages attention vectors at to look at different encoder states i is also added to the final loss function . That is , JS = Jml + Jcov . Full details on the pointer-generator and coverage mechanisms are found in See et al . ( 2017 ) . We have omitted them here for brevity . The recoder does not depend on these particulars of the summarization model . We focus on the quantities Pvocab , P , and the total summarizer loss JS . Specific details of how these are computed are not needed in the following sections . 3 RECODER . We are now ready to present our main contribution . The recoder is a neural network model that takes the decoded sequence from the summarizer as input and is trained to output the reference summary . The purpose of the recoder is to serve as a sophisticated loss function to help train the summarizer , backpropagating errors to the summarizer ’ s decoded output sequence during training . The intuition is that a good output sequence from the summarizer should allow the recoder to produce a good summary . In principle , the recoder can be any sequence to sequence model that can backpropagate to its inputs . One obvious choice is to use the same model structure as the summarizer itself . For our experiments we found it was sufficient to use the same network structure with lower dimensions , with the same attentional pointer-generator encoder-decoder network as the summarizer , but with half the number of hidden dimensions ( from 256 to 128 ) and a GRU Cho et al . ( 2014 ) instead of an LSTM in the encoder . This helped reduce the amount of memory required in training . 3.1 RECODER INPUTS . Our first task is to represent the summarizer ’ s decoded outputs as inputs to the recoder in a way that is differentiable . A beam search decoder will find a sequence ( y1 , y2 , ... ) of high average log probability − logP ( yt|y1 , ... , yt−1 ) over timesteps t. We can not use this discrete token sequence directly , but we can look at the underlying signals that determine the choice of each token . The output token yt is chosen based on the computed probability distributions Pt ( yt|y1 , ... , yt−1 ) . Let us denote this distribution by Pt for short . For a beam search of width k , the chosen token yt will have a probability in Pt that is among the k highest . The exact choice is determined by the beam search mechanism , so we do not have a continuous function that relates Pt directly to yt . However , Pt does determine the range of choices , and feeding it as an input to the recoder can ensure that it contains the right information . Since each Pt is computed by the summarizer based on the decoded prefix sequence ( y1 , ... , yt−1 ) , propagating errors to Pt improves the summarizer via exposure to what it would see at test time , even if the summarizer is not “ aware ” of the search mechanism and can not optimize for it . Since Pt has dimensions equal to the size of the vocabulary ( v = 50000 in the experiments ) , we multiply Pt by a weight matrix to produce a dense representation in a lower dimensional space ( 128 in the experiments ) , just as we did for the one-hot token inputs of the summarizer . While we can use a new weight matrix here , reusing the same embedding matrix Wemb as the summarizer helps avoid increasing the number of parameters . For the same reason we also reuse the summarizer ’ s mapping V ′ back to vocabulary space . Together they account for 90 % of the summarizer ’ s parameters . The input to the recoder is the sequence ( wRt ) where wRt = PtWemb with Pt treated as a vector of length v. By training the recoder to output the reference summary , errors are propagated to its inputs wRt and then to Pt . Note that although the summarizer ’ s input wi is in effect Wemb multiplied with one-hot vectors corresponding to xi , Pt is the output of a softmax function and will be more evenly distributed than a one-hot vector . Embedded representations wRt will thus be a weighted sum of embedding vectors instead of individual ones like wi . The recoder can accommodate this difference since it is an independent network with its own parameters .

Summary: This paper proposed an encoder-decoder-based summarization network as a loss function within a similar encoder-decoder-based summarization framework to demonstrate that the proposed model obtains better automatic and human evaluation scores compared to the baseline model of See et al. (2017) with just traditional loss functions. Overall, the paper is well-written and the presented results, analyses, and comparisons appear to be reasonable. One notable advantage of the proposed model would be to circumvent the approaches that rely on the evaluation metric, ROUGE as a reward component e.g. in a reinforcement learning setting, although with the expense of additional memory and time complexity.