# A Text Abstraction Summary Model Based on BERT Word Embedding and Reinforcement Learning
## 3 Methods
### Extraction Model  
Our extraction model is motivated by [21; 22]. The main difference is that our extractor uses BERT as a sentence encoder and the document encoder adopts the self-attention mechanism. We take a similar computation method and make some changes by adding a unidirectional GRU. Each sentence of the document is visited sequentially to obtain a shortlist of remarkable sentences with high recall to further facilitate the abstractor is our objectives. The model consists of three components: a sentence encoder, a document encoder, and a sentence extractor. The sentence encoder adopts BERT as the encoder, a bidirectional GRU with self-attention is used to encode document, a unidirectional GRU is used to compute the summary representation. Then, the representation, the document vector and the hidden state of bi-GRU are involved in the computation of the sentence score. The architecture of the model is shown in Figure 3.  
Figure 3: Extraction network model illustration. After getting the vector representation of sentences by BERT encoder, we can summarize information of documents from both directions. It includes the forward GRU \(\overrightarrow{h_{t}}\) and a backward GRU \(\overleftarrow{h_{t}}\)  
\[\overrightarrow{h_{t}}=GRU\left(r_{t},h_{t-1}^{\rightharpoonup}\right) \tag{2}\]  
\[\overleftarrow{h_{t}}=GRU\left(r_{t},h_{t+1}^{\rightharpoonup}\right) \tag{3}\]  
where \(r_{t}\) is the sentence vector of \(t\)th sentence in the time step t.  
Both GRU and LSTM are based on RNN (Recurrent Neural Network), there is no evidence to show which one is the best [25; 26]. However, GRU is simpler, more efficient, fewer parameters and easier to implement. Therefore, we use a bidirectional GRU to encode the sentences in the documents and a unidirectional GRU to obtain the summary representation which taking account of decisions made previously. A GRU is a recurrent network with two gates, \(u_{g}\) called the update gate and \(r_{g}\) the reset gate, it can be described by the following equations  
\[u_{gj}=\sigma\left(W_{ux}x_{j}+W_{uth}h_{j-1}+b_{u}\right) \tag{4}\]  
\[r_{gj}=\sigma\left(W_{rx}x_{j}+W_{rh}h_{j-1}+b_{r}\right) \tag{5}\]  
\[h_{j}^{\prime}=tanh\left(W_{ln}x_{j}+W_{hh}\left(r_{gj}\odot h_{j-1}\right)+b _{h}\right) \tag{6}\]  
\[h_{j}=\left(1-u_{gj}\right)\odot h_{j}^{\prime}+u_{j}\odot h_{j-1} \tag{7}\]  
where the \(W\)'s and \(b\)'s are learnable parameters and \(h_{j}\) is the real valued hidden vector at time step j and \(x_{j}\) is the corresponding input vector namely aforementioned \(r_{i}\) and \(\odot\) denotes the Hadamard product. We concatenated the forward and the backward GRU hidden states to get the vector \(h_{t}\), which summarizes the information of the sentence \(s_{t}\) and its context, as in Equation (8),  
\[h_{t}=\left[\overleftarrow{h_{t}},\overleftarrow{h_{t}}\right] \tag{8}\]  
where \(d_{h}\) denotes the size of hidden vector, M is the number of sentences in the document, so the \(h_{D}\in R^{M+2d_{h}}\) denotes the whole GRU hidden states, as in Equation (9),  
\[H_{D}=\left(h_{1},h_{2},\ldots,h_{M}\right) \tag{9}\]  
We all know that we pay more or less attention to each sentence according to its contribution to the article. The representation of the whole document is modeled as a weighted sum of the concatenated hidden states of the bidirectional GRU by a self-attention mechanism [26]. We take the concatenated hidden states \(H_{D}\) as input and yield a vector of weights, \(a_{D}\), as output, calculated as shown in Equation (10),  
Figure 3:0  
where \(W_{s_{2}}\) and \(W_{s_{1}}\) are learnable parameters, \(W_{s_{2}}\in R^{k\geq 2d_{h}}\), \(W_{s_{2}}\in R^{k}\), k is a hyper-parameter can be set arbitrarily. The softmax() is the normalized function used to normalize the attention weights, which sum up to 1.