# Abstract Text Summarization with a Convolutional Seq2 seq Model
## 4 Experiments and Results
### Experimental Results  
We first describe the experimental results on Gigaword corpus, the detailed information is illustrated in Table 1. The numbers in bold are the best figures. From which we can observe that adding one sentence in the training datasets does help improve the performance, on both RNN and CNN models. Actually in practice adding more sentences results in worse performance, and we attribute it to the fact that the latter sentences are not that related to the summary. Our hierarchical convolutional seq2 seq model is more expressive than traditional model as CNN models outperform RNN models on both one sentence and two sentences conditions. We also observe that copying mechanism can aid the performance a lot with the highest src. copy rate. This means source document can help improving the summarization to a extend. The hierarchical attention mechanism can also improve the performance as it collaborates key words with key sentences simultaneously. No surprise that the CNN-2 sent-hieco obtains the best results as it utilizes both copying mechanism and hierarchical attention mechanism. The * indicates the statistical significant improvement of all the other models using a paired two-tailed t test with \(p<0.05\). The src. copy rate dropped comparing the CNN-2 sent-copy as hierarchical attention mechanism could improve the generative ability of the model, so it does not depend so much on the source. Actually, for ABS+, it has the highest src. copy rate whereas with worst performance,and we attribute this to the truth that it is lacking in generative ability as a good summarization model needs both generative and extractive abilities. Our model CNN-2 sent-hieco also beats the advanced model RASElman, GAN and BERT, which further proves the effectiveness of our model. Additionally, our model equipped with RBM achieves the even better results than our original CNN-2 sent-hieco model, which verifies that RBM pre-processing could help improve the performance of our model to a certain extend. As for the DUC corpus, we just ran the 1-sentence models as the source only contains one single sentence. Actually we only present the CNN-1 sent-hieco and CNN-1 sent-hieco-RBM results to save the space as they achieved the best performance among other variants. Table 2 illustrates the detailed experimental results. We can find that our model outperforms other models consistently with statistical significance, which proves our framework is a better alternative for the text summarization task. We also tested our models on the new testing dataset CNN/Daily Mail and we directly adopted our best model CNN-2 sent-hieco to compare with other state-of-the-art models. The results are presented in Table 3. From it we can observe that even changing the testing dataset, our model still consistently and statistically significantly outperforms other models, which proves that our model is suitable and scalable to other real-life datasets. \begin{table}
\begin{tabular}{l r r r r} \hline \hline
**Metric** & **Rouge-1** & **Rouge-2** & **Rouge-L** & **Src. Copy Rate** \\ \hline RNN-1 sent & 34.83 & 17.07 & 32.55 & 75.77 \\ RNN-2 sent & 35.68 & 17.34 & 32.64 & 76.48 \\ CNN-1 sent & 35.02 & 17.23 & 33.25 & 75.21 \\ CNN-2 sent & 35.87 & 17.38 & 33.36 & 76.53 \\ CNN-2 sent-copy & 36.32 & 18.03 & 33.87 & 84.12 \\ CNN-2 sent-hieco & 36.02 & 17.74 & 33.57 & 75.49 \\ CNN-2 sent-hieco & 37.02 & 18.21 & 34.07 & 80.03 \\ CNN-2 sent-hieco-RBM & * **37.95** & * **18.64** & * **35.11** & 80.74 \\ ABS+ & 29.89 & 11.94 & 27.05 & 92.06 \\ RASElman & 36.26 & 17.92 & 33.91 & 83.25 \\ GAN & 36.88 & 17.54 & 34.05 & 76.54 \\ BERT & 36.97 & 17.96 & 33.87 & 74.51 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The experimental results on Gigaword Corpus. * indicates the statistical significant improvement.