# A Text Abstraction Summary Model Based on BERT Word Embedding and Reinforcement Learning
## 3 Methods
### Abstraction Model  
Another part of our method is an abstraction model that rewrites the previously selected key sentences and then generates a concise and readable summary. We use the pointer-generator network proposed by [6]. The pointer-generator network facilitates copying words from the source text via pointing [16], which improves accuracy and processing ability of OOV words, while retaining the ability to generate new words [6]. The network contains an encoder and a decoder and can be seen as a balance between extractive and abstractive methods. Many similar studies [6, 7, 11] show that such a model can effectively improve the performance of text summary. More details of the network can be found in the literature [6]. Although we used the pointer-generator network, we made some changes to improve the performance and accuracy of the model. Compared with the vanilla network [6], there are some differences: first, inspired by [10], the new network introduces the updated word attention combined with sentence-level and word-level attentions as same as [10]; second, we replace the LSTM in the network with the GRU, since the GRU is simpler and requires fewer parameters; third, the two models input different amounts of data, when the article reaches 400 tokens the input of the vanilla network is truncated, which causes loss of information, the input of the new network is the key sentences from aforementioned extraction model; fourth, the word embedding of two models are different, the word2 vec is used for the vanilla pointer-generator network, the BERT is used in our abstractive network. In addition, the WordPiece tokenizer can help to process the OOV words. The architecture of the updated pointer-generator network is shown in Figure 5.  
There is a lot of evidence that attention mechanism is very important for NLP tasks (e.g., [5, 19, 23]). We use the sentence-level modulate the word-level attention such that words in less attended sentences are less likely to be generated [10]. Take the simple scalar multiplication of the aforementioned sentence attention \(\beta_{m}\) in sec 3.3 and the word attention \(\alpha_{n}^{t}\) of the \(m\)th sentence, and then renormalize the result into the new attention. The updated word attention \(\mu_{n}^{t}\)  
Figure 5: Model of pointer-generator network [10]. \[\mu_{n}^{t}=\frac{\alpha_{n}^{t}\times\beta_{m}}{\sum_{n}\alpha_{n}^{t}\times \beta_{m}}. \tag{16}\]  
The final probability distribution of word \(w\) is related to the updated word attention \(\mu^{t}\) as follows  
\[P_{final}(w)=p_{gen}(h^{\star})P_{vocab}(h^{\star},w)+\left(1-p_{gen}(h^{ \star})\right)\sum_{n:w_{n}=w}\mu_{n}^{t} \tag{17}\]  
\[h_{n}^{\star}=\sum_{n}\mu_{n}^{t}h_{n} \tag{18}\]  
where \(p_{gen}(h^{\star})\in[0,1]\) is the generating probability (see Equation (8) in [6]), \(P_{vocab}(h^{\star},w)\) is the probability distribution over word \(w\) being decoded, \(h^{\star}\) is the context vector, a function of the updated word attention \(\mu^{t}\), \(h_{n}\) is the encoder hidden state for the \(n\)th word. During pre-training, the loss is the negative log likelihood, we minimize the loss as  
\[L_{abs}=-\frac{1}{T}\sum_{t=1}^{T}\log P_{final}\left(w_{t}^{\star},\mu^{t}\right) \tag{19}\]  
where \(w_{t}^{\star}\) is the target word in the reference abstractive summary at the time step t. The coverage mechanism [6] is also used to prevent the abstractor from repeatedly putting the focus on the same point.