# Summarization of Spanish Talk Shows with Siamese Hierarchical Attention Networks
## 5 Experiments
The results of all the systems in Table 3 on the ES-NEWS corpus (in Spanish) considering Rouge-2 measure are in line with those on the English CNN/DailyMail corpus [15]; for instance, SHA-NN and Lead achieved 14.7 and 15.1 respectively on the CNN/DailyMail corpus. However, in terms of Rouge-1 and Rouge-L all the systems present a slight decrease of performances on the ES-NEWS corpus with respect to CNN/DailyMail corpus; for instance, SHA-NN and Lead achieved 35.4 and 37.3 respectively on the CNN/DailyMail corpus considering Rouge-1. This fact illustrates small differences between the two corpora that have affected the results of all the compared systems. Regarding SHA-NN system, the results show a good transferability between languages as we hypothesized. Using the summarization system trained in the above experimentation, we evaluated it with the LN24-SUMM test corpus, which contains 30 document-summary pairs (a small test set compared to the training set). Table 4 shows the results of the SHA-NN system compared to other summarization systems. This table shows that the results of our summarization system are better than those of the other systems at all levels of ROUGE, although it should be considered that the small size of the test set does not allow to obtain statistically significant results. It should be noted that when working with the ES-NEWS corpus, the Lead system, which consists of extracting the first 3 sentences of the article as a summary, outperforms the rest of the systems, including ours, as Table 3 shows. However, this system performed worse on LN24-SUMM corpus. This is due to the fact that in ES-NEWS corpus, unlike the LN24-SUMM corpus, the summary tends to be a very approximate version of the first sentences of the articles. Also, it is interesting that, although SHA-NN was trained under the bias to the first sentences (ES-NEWS), it is capable of generalizing when the relevant sentences are more scattered in the document (LN24-SUMM). In relation to the transferability between domains, it is possible to see that all the results, in terms of ROUGE, obtained on the LN24-SUMM corpus are higher than those obtained on the ES-NEWS corpus. That is because the reference summaries of the LN24-SUMM corpus have a very high density (i.e., they are composed by long extractive fragments of the transcribed talk shows) and a very low compression ratio in comparison to the ES-NEWS corpus, as it can be seen in Tables 1 and 2. Furthermore, it is interesting to see in Table 4 that in general, when the compression ratio of the generated summaries increases, the results in terms of ROUGE decrease. Our system provide the best trade-off Rouge/Compression among all systems. Moreover, although TextRank obtains the most similar results with respect to SHA-NN, it suffers from a very low compression ratio due to it tends to extract the longest sentences. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Rouge-1** & **Rouge-2** & **Rouge-L** & **Compression** \\ \hline SHA-NN & **46.0**\(\pm\) 4.38 & **29.0**\(\pm\) 5.94 & **42.2**\(\pm\) 4.46 & 7.0 \\ Lead & 33.2 \(\pm\) 4.86 & 17.4 \(\pm\) 5.86 & 29.9 \(\pm\) 5.17 & 13.4 \\ LexRank & 39.7 \(\pm\) 3.64 & 20.6 \(\pm\) 4.73 & 35.2 \(\pm\) 4.03 & 5.9 \\ TextRank & 43.3 \(\pm\) 5.76 & 27.0 \(\pm\) 7.65 & 39.7 \(\pm\) 6.10 & 3.9 \\ LSA & 36.1 \(\pm\) 4.60 & 15.8 \(\pm\) 5.86 & 31.2 \(\pm\) 5.10 & 9.4 \\ SumBasic & 31.8 \(\pm\) 5.21 & 14.4 \(\pm\) 5.45 & 28.7 \(\pm\) 4.94 & 24.7 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results on LN24-SUMM corpus with respect to the ground truth (full length Rouge \(F_{1}\)).