# Summarization of Spanish Talk Shows with Siamese Hierarchical Attention Networks
## 3 System Description  
The SHA-NN system [15] is based on Hierarchical Attention Networks (HAN) [21] trained in a Siamese way, where its left branch extracts representations for whole documents and its right branch extracts representations for summaries. HAN allows us to extract a vector representation for documents and summaries from the representations of their sentences. Moreover, the representation of each sentence is obtained from representations of their words. These representations are trained to address a binary classification task that consists of determining if a summary \(S\) is correct for a document \(D\). It acts as an intermediate task in order to extract the most relevant sentences of the document to make a summary. Figure 3 shows an outline of the system architecture. As input, the SHA-NN system uses \(d_{e}\)-dimensional skipgram word embeddings, trained from the training set of the ES-NEWS corpus, in order to represent both documents, \(D\in\mathbb{R}^{T\times W\times d_{e}}\) and summaries, \(S\in\mathbb{R}^{Q\times V\times d_{e}}\), where \(T\) and \(Q\) are the maximum number of sentences for document and summary and \(W\) and \(V\) are the maximum number of words per sentence for document and summary. These representations are used as input for the two Hierarchical Attention Networks (\(HAN_{1}\) and \(HAN_{2}\)) whose BLSTM layers are shared between them, both at sentence level \begin{table}
\begin{tabular}{l l} \hline \hline Dataset Size & 30 pairs \\ \hline Mean Article Length & 921.7 words \\ Mean Summary Length & 108.6 words \\ Mean Word Overlapping & 64.8 words \\ \hline Mean Extractive Fragment Coverage & 0.90 \\ Mean Extractive Fragment Density & 19.31 \\ Mean Compression Ratio & 8:1 \\ \hline Articles Vocabulary Size & 3969 words \\ Summaries Vocabulary Size & 1103 words \\ Overlapping Vocabulary & 1069 words \\ \hline \hline \end{tabular}
\end{table} 
Table 2: LN24-SUMM corpus statistics. Figure 3: SHA-NN Architecture. (\(BLSTM_{2}\) with dimensionality \(d_{s}=512\)) and at word level (\(BLSTM_{1}\) with dimensionality \(d_{w}=512\)). However, the attention mechanisms in both levels are not shared. From these inputs, \(H\in\mathbb{R}^{T\times d_{w}}\) and \(G\in\mathbb{R}^{T\times d_{w}}\) are computed, following Equations (4) and (6), as proposed in [22]. They are the output from the word level \(d_{w}\)-dimensional \(BLSTM_{1}\) with attention, where each row \(i\) is computed as the average of the hidden vectors of the sentence \(i\) attended by \(\alpha\in\mathbb{R}^{T\times W}\) Equation (5) and \(\beta\in\mathbb{R}^{Q\times V}\) Equation (7) for document and summary respectively. \[H_{i}=\sum_{j=0}^{W}BLSTM_{1}(D_{i1},...,D_{iW})_{j}\cdot\alpha_{ij} \tag{4}\]  
\[\alpha_{ij}=\frac{e^{tanh(W_{u}BLSTM_{1}(D_{i1},...,D_{iW})_{j}+b_{u})}}{\sum _{k=0}^{W}tanh(W_{u}BLSTM_{1}(D_{i1},...,D_{iW})_{k}+b_{u})} \tag{5}\]  
\[G_{i}=\sum_{j=0}^{V}BLSTM_{1}(S_{i1},...,S_{iV})_{j}\cdot\beta_{ij} \tag{6}\]  
\[\beta_{ij}=\frac{e^{tanh(W_{u}BLSTM_{1}(S_{i1},...,S_{iV})_{j}+b_{u})}}{\sum _{k=0}^{V}tanh(W_{v}BLSTM_{1}(S_{i1},...,S_{iV})_{k}+b_{v})} \tag{7}\]  
where \(W_{u}\in\mathbb{R}^{d_{w}}\), \(W_{v}\in\mathbb{R}^{d_{w}}\), are the weights of the attention mechanism for document and summary at word level. Once \(H\) and \(G\) are computed, \(r\in\mathbb{R}^{d_{s}}\) and \(p\in\mathbb{R}^{d_{s}}\) can be obtained, following Equations (8) and (10), similarly to the word level but using \(BLSTM_{2}\) and the attentions \(\hat{\alpha}\in\mathbb{R}^{T}\) and \(\hat{\beta}\in\mathbb{R}^{Q}\) for document and summary, respectively.