# Abstractive Sentence Compression with Event Attention
## 4 Abstractive Sentence Compression with Event Attention  
Event words in news articles play a key role in delivering the meaning of a source sentence, since an event means something that happens or occurs [34]. For instance, Table 1 shows compressed sentences of a source sentence \"_If IBM has **miscalculated** the demand, it will **suffer** badly as both the high operating costs and depreciation on the huge capital investment for the East Fishkill factory **drag** down earnings.\"_ in which the event words are expressed as bold. Since the event words contain main actions of the source sentence, they are usually preserved in the compressed sentence. As a result, the compressed sentences in Table 1 contain at least one of the words '_miscalculated_', '_suffer_', and '_drag_'. Therefore, the event words must be regarded as salient words in generating a compressed sentence. To focus on the event words of a source sentence in generating a compressed sentence, the event words should be first extracted. In this paper, the event words are extracted using the event extraction system proposed by [PERSON] et al. [35]. When a source sentence \(X=(x_{1},x_{2},x_{3},...,x_{n})\) is given, the system returns an event mask vector \(E_{X}=(e_{1},e_{2},e_{3},...,e_{n})\) where \(e_{i}\) is 1 if \(x_{i}\) is an event word and 0 otherwise. After that, a compressed sentence of \(X\) is generated by a sequence-to-sequence model. Figure 1 depicts the proposed sequence-to-sequence model to generate a compressed sentence from a vectorized source sentence \(\mathbf{X}\) and the event mask vector \(E_{X}\). For a given source sentence \(\mathbf{X}\), the encoder outputs the hidden state \(h_{t}\)'s and the source sentence representation \(Q\) by Equation (2). To compress the source sentence elaborately, two types of attention, which are _event attention_ and _global attention_, are adopted in the proposed model. The event attention allows the model to focus on the event words in the source sentence, while the global attention provides the overall understanding of the source sentence. The event attention weight \(\alpha_{i}\) of an encoder hidden state \(h_{i}\) is computed using \(h_{i}\), the source sentence representation \(Q\), and the event mask \(e_{i}\). That is,  
\[\alpha_{i} = \frac{exp(u_{i})}{\sum_{j=1}^{n}exp(u_{j})},\] \[u_{i} = V_{e}^{\top}g(W_{h}h_{i}+W_{q}Q+W_{e}e_{i}+b_{eattn}), \tag{5}\]  
where \(V_{e}\), \(W_{h}\), \(W_{q}\) and \(W_{e}\) are weight parameters and learned from training data. \(b_{eattn}\) is a bias vector, and \(g\) is a non-linear function. \begin{table}
\begin{tabular}{l l} \hline Source & If IBM has **miscalculated** the demand, it will **suffer** badly as both the high operating costs and depreciation on the huge capital investment for the East Fishkill factory **drag** down earnings. \\ \hline Compressed sentences & If IBM has **miscalculated** the demand, high operating costs and depreciation will **drag** down earnings. \\ \hline \end{tabular}
\end{table} 
Table 1: An example of a source sentence and its compressed sentences in MSR dataset. Figure 1: The overall process of abstractive sentence compression with event attention. Since the event attention helps the decoder focus on event words during decoding, it pays less attention to the salient words which deliver the overall meaning of the source sentence. Thus, an attention proposed by [PERSON] et al. [20] is adopted as a global attention of the proposed model. The reason the attention is adopted is that it is one of the widely used and standard attention [36]. When the decoder generates the \(t\)-th word, the global attention weight \(\beta_{i}^{t}\) of \(h_{i}\) is computed by  
\[\beta_{i}^{t} = \frac{exp(d_{i}^{t})}{\sum_{j=1}^{n}exp(d_{i}^{t})},\] \[d_{i}^{t} = V_{g}^{\top}g(W_{h^{\prime}}h_{i}+W_{s}s_{t-1}+b_{gattn}), \tag{6}\]  
where \(V_{g}\), \(W_{h^{\prime}}\) and \(W_{s}\) are weight parameters and learned from training data, \(b_{gattn}\) is a bias vector for the global attention, and \(g\) is a non-linear function.