# FrameSum: Leveraging Framing Theory and Deep Learning for Enhanced News Text Summarization
## 3 News Text Summarization Model Based on Framing Theory
### A Summarization Model Incorporating Framework Information (FrameSum)
#### 3.3.2 Decoder with Framework Information  
1. Framework-Guided Decoding Strategy  
In the transformer decoder, the framework representation \(r\) of the current news is used as an additional input to guide the decoding process to focus on framework-related content. Specifically, at each time step in the decoder, the attention distribution \(\alpha_{t}\in R^{n}\) between the current hidden state \(s_{t}\) and \(r\) is first calculated through the attention mechanism:\[e_{ti}=\frac{(s_{t}W_{q})(r_{t}W_{t})^{T}}{\sqrt{d_{t}}}\left(i=1,2,\ \ldots,\ n\right) \tag{9}\] \[\alpha_{t}=softmax(e_{t})\]  
where \(W_{q}\), \(W_{k}\in R^{d_{model}\times d_{k}}\) are learnable attention parameters and \(d_{k}\) is the dimension of the attention head. The context vector \(c_{t}\in R^{d_{model}}\) at the current time step is obtained by weighted summation of \(r\) using the attention distribution \(\alpha_{t}\):  
\[c_{t}=\sum_{i=1}^{n}\alpha_{ti}r_{i} \tag{10}\]  
Then, \(c_{t}\) is concatenated with the current decoder hidden state \(s_{t}\) and passed through a feed-forward neural network to generate the output representation \(o_{t}\in R^{d_{model}}\) at the current time step:  
\[o_{t}=FFN([s_{t};c_{t}]) \tag{11}\]  
Finally, we use the output representation \(o_{t}\) to predict the vocabulary distribution \(p_{t}\in R^{|\mathrm{V}|}\) at the current time step:  
\[p_{t}=softmax(o_{t}W_{v}+b_{v}) \tag{12}\]  
where \(W_{v}\in R^{d_{model}\times|\mathrm{V}|},b_{v}\in R^{|\mathrm{V}|}\) are learnable output parameters and \(|\mathrm{V}|\) is the vocabulary size. Through the above framework-guided decoding strategy, the decoder can selectively focus on relevant content based on the framework attributes of the text during the summary generation process, thereby generating summaries that better align with the original theme. 2. Coverage Loss  
To improve the coverage of the summary over the news content, we introduce a coverage loss \(L_{cov}\) during the training process. First, we define the coverage vector \(g^{t}\in R^{n}\), which represents the accumulated attention of the decoder over each position of the original text in the previous \(t\) time steps:  
\[g^{t}=\sum_{\tau=1}^{t}\alpha_{\tau} \tag{13}\]  
The difference between the attention distribution \(\alpha_{t}\) at the current time step and the coverage vector \(g^{t-1}\) is calculated as the coverage loss:  
\[ \begin{array}{c}L_{cov}^{t}=\sum_{i=1}^{n}min\Big{(}\alpha_{ti},g_{i}^{t-1} \Big{)}\\ L_{cov}=\frac{1}{T}\sum_{t=1}^{T}L_{cov}^{t}\end{array}  \tag{14}\]  
where \(T\) is the total number of time steps in the decoder. By minimizing the coverage loss, the coverage of the summary over the original content can be improved, generating more comprehensive and complete summaries. 3. Framework Relevance Loss  
To ensure consistency between the generated summary and the original text in terms of the dominant framework, we introduce a framework relevance loss \(L_{fr}\) during the training process. Let \(y=(y_{1},y_{2},\ldots,y_{m})\) be the generated summary sequence. Using the pre-trained framework automatic identification model from Section 3.2.3, we perform framework prediction on the summary to obtain its probability distribution \(q\in R^{7}\) over each framework:  
\[q=FrameClassifier(y) \tag{15}\]  
The framework distribution \(q\) of the summary is compared with the true framework labels \(\hat{l}\in L\) of the original news text to calculate the framework relevance loss:  
\[L_{fr}=-\sum_{k=1}^{K}l_{k}log(q_{k}) \tag{16}\]where \(l_{k}\) is the true label of the original text on the \(k\)-th framework (1 indicates belonging to that framework, 0 indicates not belonging), and \(q_{k}\) is the predicted probability of the summary on the \(k\)-th framework. By minimizing the framework relevance loss, the thematic relevance of the summary can be improved, highlighting the core content of the original text. 4. Cross-Entropy Loss  
In the summary generation task, we use cross-entropy loss to optimize the generation quality of the model.