# FrameSum: Leveraging Framing Theory and Deep Learning for Enhanced News Text Summarization
## 4 Experiments
### News Framing Identification Experiments
#### 4.2.3 Experimental Results and Analysis  
Table 3 shows the performance of various models on the news framing identification dataset. It can be observed that deep learning-based methods generally outperform traditional machine learning methods. Among them, our proposed classification model that incorporates framing semantic features achieves the best performance on all metrics with a clear leading advantage.  
In comparison, traditional machine learning models such as NBM, SVM, and TF-IDF perform poorly on all metrics, with NBM having an F1 score of only 0.31. This may be because these models cannot effectively capture the semantic information and structural features of text. Although the deep learning-based RNN and transformer models show some improvement in accuracy and F1 score, there is still a considerable gap compared to BERT.  
NFRM demonstrates excellent performance advantages in the news framing identification task, which can be attributed to the following two points: First, the model incorporates rich framing semantic knowledge and characterizes the framing elements of news reports from multiple perspectives. Second, it employs the powerful pre-trained language model BERT as the encoder, enabling the model to fully extract deep semantic information from the text.  
In addition to model comparisons, we also designed ablation experiments to examine the impact of different framing features on framing classification. Specifically, we removed one category of framing features at a time and evaluated the changes in model performance. As shown in Figure 5, regardless of which category of features is removed, the F1 score of NFRM decreases to varying degrees, with the responsibility feature being the most significant. This indicates that causal attribution information is crucial for framing identification. In fact, many news reports analyze causes and identify responsible parties when describing events, which forms an important semantic basis for framing construction. Additionally, removing the morality feature also leads to a substantial performance loss, which is related to the frequent presence of moral judgments and value judgments in news reports.  
In contrast, removing the fact, conflict, human interest, economic and leadership features have a relatively smaller impact on model performance, seemingly suggesting that these features contribute less than the responsibility and morality features. However, this does not mean that these features are dispensable. In fact, although the influence of removing a single category of features is limited, the decline in model performance will be more significant if multiple categories of features are removed simultaneously. The construction of news frames is the result of the joint action of various semantic elements,  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1** \\ \hline NBM & 0.44 & 0.33 & 0.45 & 0.31 \\ SVM & 0.61 & 0.62 & 0.61 & 0.58 \\ TF-IDF & 0.62 & 0.69 & 0.61 & 0.56 \\ RNN & 0.47 & 0.37 & 0.49 & 0.34 \\ Transformer & 0.71 & 0.69 & 0.70 & 0.65 \\ BERT & 0.79 & 0.75 & 0.79 & 0.72 \\ Our Model & \multirow{2}{*}{0.92} & \multirow{2}{*}{0.87} & \multirow{2}{*}{0.89} & \multirow{2}{*}{0.90} \\ (NFRM) & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 3: Framing identification results.  
and we need to capture the interactive influence and holistic nature of framing elements from multiple perspectives.