# Abstractive Summarizers Become Emotional on News Summarization
## 7 Discussion  
We summarize the most important contributions and findings of this work in relation to the objectives stated in the introduction section. **Emotional content of summarization corpora**. First, we found that 99% of articles and 70% of summaries of the studied corpora contain at least one emotion. We also found that 12% in Xsum and 0.9% in Cnn/Dailymail of the reference summaries elicit at least one emotion that does not appear in the article. Second, we applied two measures, emotion density, and emotion ratio, to articles and summaries of both corpora and the results that we analyzed. Related to the articles, we observed that _trust_ concentrates the most significant number of articles with higher emotion densities. In contrast, _disgust_ concentrates the largest number of articles with lower emotion densities. Related to emotions in summaries, we noticed a similar behavior. In Xsum, the distributions are shifted toward higher values \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & & **Precision** & **Recall** & \
\begin{tabular}{c} **Samples w/o** \\ **Novel Words** \\ \end{tabular} &
\begin{tabular}{c} **Samples** \\ \(|\mathcal{E}_{\delta}|=\mathbf{0}\) \\ \end{tabular} & **Samples** \\ \hline \multirow{3}{*}{**CNNDM**} & **BART** & 84.73 & 38.79 & 7912 (68.9\%) & 2499 (21.7\%) & 1022 (8.9\%) \\  & **PEGASUS** & 85.82 & 36.75 & 6394 (55.6\%) & 3332 (29.0\%) & 1707 (14.9\%) \\  & **T5** & 84.99 & 34.47 & 7019 (61.1\%) & 3090 (26.9\%) & 1336 (11.6\%) \\ \hline \multirow{3}{*}{**XSUM**} & **BART** & 76.10 & 49.49 & 324 (2.9\%) & 3717 (32.8\%) & 6704 (59.1\%) \\  & **PEGASUS** & 77.73 & 51.01 & 279 (2.5\%) & 3685 (32.5\%) & 6781 (59.8\%) \\ \cline{1-1}  & **T5** & 75.30 & 47.21 & 354 (3.1\%) & 3901 (34.4\%) & 6554 (57.8\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Precision and recall of the emotions in the novel words generated by each model, compared to the emotions of the reference summaries. The number of samples without (w/o) novel words in the generated summary and w/o emotions in the novel words of the generated summary (\(|\mathcal{E}_{\delta}|=0\)) are also shown. The last column indicates the number of samples finally considered in the evaluation. We also show percentages of samples in the test sets. of emotion densities compared to Cnn/Dailymail. Regarding the emotion ratios, there is a tendency to overemphasize the emotion _fear_ in both corpora. In Cnn/Dailymail, the summaries tend to overemphasize emotions, especially the negative ones, while in Xsum they tend to overemphasize _fear_. **Emotional behavior of summarization models**. We introduced two new measures, _emotional coherence_ and _emotional bias_, to measure how the emotion densities and ratios of generated summaries correlate with those of the reference. We found that all the emotional coherences are higher than 0, suggesting positive relationships between the emotion densities. Abstractive models generally present a coherence higher than 0.5 in negative emotions: _fear_, _anger_, _sadness_, and _disgust_; and a coherence between 0.35 and 0.5 in the other emotions. Additionally, we found a higher emotional bias in Xsum than in Cnn/Dailymail. In Xsum, T5 is the abstractive model with the lowest emotional bias. Pegasus shows the highest emotional bias for almost all emotions in Cnn/Dailymail and Xsum. Also, we analyzed whether the novel words generated by the summarization models convey the emotions expected in their reference summaries. We observed that most of the novel words generated by the models have emotions that match those of the reference summaries. Interestingly, the recall is between 34% and 51%, which suggests that the emotions of the novel words are enough to cover, approximately, at least a third part of the emotions in the reference summaries. Finally, we should remark that the proposed methodology is valid for studying emotions in summarization regardless of the method used to detect emotions.