# Summarization of Spanish Talk Shows with Siamese Hierarchical Attention Networks
## 2 Corpora
### Es-NEWS Corpus  
We have built a corpus of written Spanish article-summary pairs--the ES-NEWS corpus. It consists of a set of 277,675 article-summary pairs extracted from 11 different Spanish online newspapers. ES-NEWS contains articles and summaries of news, sports, politics, culture and other subjects. We used this corpus to evaluate the transferability of the SHA-NN summarization system from English to Spanish and on the other hand, we used it to train the system that we apply to Spanish talk shows summarization. The ES-NEWS corpus is composed by newspaper articles extracted from around 1 million URLs, which were collected during the last week of June 2018. To enforce the diversity of summarization styles, 11 websites of relevant newspapers of Spain have been used. These newspapers are: _Elconfidencial_, _EconomiaDigital_, _HuffingtonPost_, _ABC_, _FormulaTV_, _EldiarioCantabria_, _Publico_, _Vozpopuli_, _Rioja2_, _PeriodistaDigital_ and _Eldiario_. It have been excluded newspapers that do not include highlights such as _ElMundo_, newspapers that only consider a single short highlight per article such as _ElPais_ and newspapers whose crawled content are not articles but web content (advertising, keywords, etc.) such as _EuropaPress_. Once all the URLs were crawled, following Reference [20], we have used the field _og:description_ to extract the highlights that, concatenated, were considered as summaries. We made a preprocess in order to remove noise such as duplicated URLs, empty summaries and articles and non-journalistic articles. All the text was lowercased and tokenized by using the Spanish version of Stanford CoreNLP. The corpus consists of 277,675 article-summary pairs. From it, training, development and test partitions have been defined, following similar proportions to CNN/Dailymail corpus [11] (90%, 5.5% and 4.5% respectively). Thus, resulting in a training set of 249,919 pairs, a development set of 15,266 pairs and a test set of 12,490 pairs. Some ES-NEWS corpus statistics are shown in Table 1.  
To make a comparison between the ES-NEWS and the NEWSROOM corpora, we have used the Extractive Fragment Coverage, Extractive Fragment Density and Compression Ratio measures. These metrics, proposed in Reference [20], aim to measure the overlapping between summaries and articles to analyze the diversity of summarization styles. The metrics are defined in Equations (1)-(3), where \(A\) is the sequence of words of the article, \(S\) is the sequence of words of the summary, \(\mathcal{F}\) is the set of \begin{table}
\begin{tabular}{l l} \hline \hline Dataset Size & 277,675 pairs \\ \hline Mean Article Length & 813.8 words \\ Mean Summary Length & 46.0 words \\ Mean Word Overlapping & 28.6 words \\ \hline Mean Extractive Fragment Coverage & 0.67 \\ Mean Extractive Fragment Density & 7.27 \\ Mean Compression Ratio & 20:1 \\ \hline Articles Vocabulary Size & 961,485 words \\ Summaries Vocabulary Size & 167,822 words \\ Overlapping Vocabulary & 157,863 words \\ \hline \hline \end{tabular}
\end{table} 
Table 1: ES-NEWS corpus statistics. common fragments, common sequences of words, in \(A\) and \(S\) computed using a greedy algorithm proposed in Reference [20] and \(|\cdot|\) stands for the length, in terms of words, of the sequences. \[\text{COVERAGE}(A,S)=\frac{1}{|S|}\sum_{f\in\mathcal{F}}|f| \tag{1}\]  
\[\text{DENSITY}(A,S)=\frac{1}{|S|}\sum_{f\in\mathcal{F}}|f|^{2} \tag{2}\]  
\[\text{COMPRESSION}(A,S)=\frac{|A|}{|S|} \tag{3}\]  
The Extractive Fragment Coverage is computed as the sum of the lengths of all the common fragments between the article and its summary divided by the size of the summary. Thus, the greater its value, the more common fragments or larger common fragments have been found between the article and its summary. The Extractive Fragment Density is a measure similar to the Extractive Fragment Coverage but using the square of common fragment lengths.