# An Abstractive Summarization Model Based on Joint-Attention Mechanism and a Priori Knowledge
## 5 Discussion  
To examine the function of each module in the overall model, we ran corresponding ablation experiments on the model. One or two modules are removed based on the experiment. After the WS, PK, JAM, and RL modules are removed, the fundamental model is JPATS_w/o_ALL. The PGN + Cov model in Table 1 is now identical to JPATS_w/o_ALL.  
The WS module of this model is intended to determine whether the hidden vectors generated by Bi-GRU deviate from the meaning of the original text and can assign higher weights to the important parts of the original text. In Table 4, we obtain the scores of the model without the WS module, and the results show that the scores are slightly worse than the original JPATS model. Again, the model with the JAM, PK, and RL modules removed scored slightly higher compared to the model with all modules removed. After comparing the two data, it is proved that the WS module plays the role of judging the matching and assigning weights in the model.  
The JAM module uses a joint-attention mechanism that is a fusion of the word-level attention mechanism and sentence-level attention mechanism. The joint-attention mechanism is computed separately with the state of the decoder at each moment, so that the model pays attention to both the keyword and the sentence in which the keyword is located. The captured word-level features and sentence-level features help the decoder to improve the quality of the generated summaries. In the absence of the JAM module, the Rouge score decreases by 0.88, 0.28, and 0.39, respectively. This shows that the JAM module helps the model to consider richer information and improve model performance. In addition, the Rouge score decreases more compared to the score after removing the WS module, indicating a more pronounced effect of the JAM module.  
The PK module mimics the human habit of combining background knowledge when reading, and uses the original semantic information without any training to give the model more original information, prompting the generation of summaries that are more in line with human reading habits. After removing the PK model, the Rouge score of the model showed a decrease. The scores still decreased compared to the model with the WS module removed, indicating that the PK module outperformed the WS module for the summary generation task.  
Based on previous studies on reinforcement learning, it is effective in improving Rouge scores. For this experiment, we also remove the RL module separately to study it. Removing the RL module resulted in a larger decrease in model scores than the model  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Models** & **Rouge-1** & **Rouge-2** & **Rouge-L** \\ \hline JPATS & 41.25 & 18.81 & 38.34 \\ JPATS\_w/o\_WS & 40.97 & 18.65 & 38.09 \\ JPATS\_w/o\_JM & 40.37 & 18.53 & 37.95 \\ JPATS\_w/o\_PK & 40.64 & 18.47 & 37.96 \\ JPATS\_w/o\_RL & 39.66 & 17.35 & 36.41 \\ JPATS\_w/o\_JAM + PK & 40.19 & 18.21 & 37.93 \\ JPATS\_w/o\_JAM + PK + RL & 39.86 & 17.79 & 37.88 \\ JPATS\_w/o\_ALL & 39.53 & 17.28 & 36.38 \\ \hline \hline \end{tabular}
\end{table}
Table 4: JPATS ablation experiment scores on dataset CNN/DailyMail.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Models** & **Rouge-1** & **Rouge-2** & **Rouge-L** \\ \hline ABS & 26.55 & 7.06 & 22.05 \\ ABS+ & 28.18 & 8.49 & 23.81 \\ RAS-Elman & 28.97 & 8.26 & 24.06 \\ SEAS & 29.21 & 9.56 & 25.51 \\ DEATS & 29.91 & 9.61 & 25.95 \\ JPATS & 29.98 & 9.80 & 26.12 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Rouge scores of JPATS on dataset DUC 2004.  
with the remaining modules removed, and we obtained the same conclusion as in previous studies. The effectiveness of the RL module for improving Rouge scores can also be derived from the scores of the model with JAM + PK removed and the model with JAM + PK + RL removed.