# An Abstractive Summarization Model Based on Joint-Attention Mechanism and a Priori Knowledge
## 3 Model
### Decoding Side
#### 3.4.2 Reinforcement Learning Based on Improved Pointer Networks  
During the training process of the codec, the decoder part inputs the correct reference summary, while during the prediction process, the encoder inputs the output of the previous time step, and once the output of the previous step is wrong, it will cause a series of subsequent deviations, leading to the problem of poor summary quality. This is the exposure bias problem. In addition, the model-generated summaries usually suffer from incoherence and semantic irrelevance, and such problems are caused by the non-uniformity of evaluation metrics and loss functions. For the above problems, the solution in this article is as follows.  
Reinforcement learning is used to describe and solve the problem of learning strategies to maximize the reward or achieve a specific goal by an intelligence agent during its interaction with the environment. The training method incorporating the reinforcement learning approach considers the supervised learning objective (maximum-likelihood method) and  
Figure 8: Improved pointer network.  
the RL strategy gradient objective for generating relationships between summary sentences and standard summary sentences under the Rouge criterion. The maximum-likelihood loss is the loss function used in the conventional supervised learning strategy; training by decreasing the loss function is comparable to increasing the likelihood that the real value will occur. According to the reference summary \(y^{*}\left\{y_{1}^{*},y_{2}^{*},...,y_{n}^{*}\right\}\), minimize the negative logarithmic likelihood function \(L_{ML}\).  
\[L_{ML}=-\sum_{t=1}^{n}logP(y_{1}^{*}|y_{1}^{*},y_{2}^{*},...,y_{t-1}^{*},w) \tag{24}\]  
The maximum likelihood only allows the generative model to learn similar representation patterns to the standard summary, while the Rouge metric allows for flexibility in sentence alignment. Therefore, we modify the loss function so that the loss function can be optimized for Rouge. Specifically, two independent output sequences \(y^{s}\) and \(\widehat{y}\) are generated in each training iteration. \(y^{s}\) is obtained by sampling from the \(P(y_{1}^{s}|y_{1}^{s},y_{2}^{s},...,y_{t-1}^{s},w)\)conditional probability distribution at each decoding time step, and \(\widehat{y}\) is obtained by maximizing the output probability distribution, which is also the model output result. We use the Rouge score as the reward function \(r(y)\). The generated abstracts are compared with the reference abstracts, compared by Rouge function values, and rewards are calculated. \(r(y)\) is calculated as the similarity evaluation Rouge score between the generated abstract \(y\) and the reference abstract \(y^{*}\).  
\[L_{RL}=(r(\widehat{y})-r(y^{s}))\sum_{t=1}^{n}logP(y_{1}^{*}|y_{1}^{*},y_{2}^{ *},...,y_{t-1}^{*},w) \tag{25}\]  
Minimizing \(L_{RL}\) is equivalent to maximizing the conditional likelihood of the sampled sequence \(y^{s}\), thus increasing the payoff expectation of the model. We finally mix the loss functions of the two components to obtain the overall objective function, with the following equation.  
\[L_{mixed}=\lambda L_{RL}+(1-\lambda)L_{ML} \tag{26}\]  
\(\lambda\) is used to balance \(L_{RL}\) and \(L_{ML}\). The loss function is introduced through reinforcement learning to effectively alleviate the exposure bias problem on the one hand, and to optimize the evaluation metrics and improve the evaluation metric scores of the model on the other hand. At this point, we have presented all parts of the generative summary model based on the joint-attention mechanism and a priori knowledge. Next, the experimental part is elaborated.