# Multistage Mixed-Attention Unsupervised Keyword Extraction for Summary Generation
## 1 Introduction  
With the continuous development of large language models, text generation becomes a research hotspot in the field of natural language processing. It has gained a wide range of applications for tasks such as article creation, dialog generation, and conference summary. Summary generation is a task of text generation [1], and researchers use different techniques to generate summaries, including rewriting or extracting various information from documents [2]. Currently, lots of research results have been achieved in generating summaries for short texts [3, 4]. However, it is still difficult to quickly obtain high-quality summaries from long textual data such as long documents, news interviews, and conference content. Therefore, improving the quality of summaries of long texts is of great research importance [5].  
In the task of summary generation for long text, the current summary generation model has the problem of failing to comprehensively extract the important information of long text, and at the same time, there is redundant information in long text. Therefore, in order to better improve the generative ability of the summarization model, sentence filtering is usually used to remove redundant information in long texts during data preprocessing. To address the lack of an ability of summarization models to generate important information, based on deep learning models, supervised methods are designed by researchers to extract keywords, which require a large labeled training dataset. On the other hand, unsupervisedmethods do not require labeled datasets and have lower dataset requirements [6]. Therefore, unsupervised keyword extraction has been adopted by many scholars, and unsupervised keyword extraction methods based on text embedding, document embedding, and hybrid statistical embedding have emerged, such as MDERank [7], SIFRank [8], and KeyGames [9]. Although the above methods improve the extraction of keywords, the lower-weighted keywords may be used first for summarization inference, while the higher-weighted keywords fail to be applied, resulting in poor quality and readability of the summaries [10].  
Therefore, the self-attention mechanism has been used by many scholars to select keywords with higher weights. For example, when generating summaries for long texts or multiple documents, a self-attention mechanism is employed by the Transformer model [11] to capture the dependencies between keywords in the input texts or documents. It achieves a focus on some of the important keywords. However, when the extracted keywords receive a small attentional weight, it affects the complete encoding of the keywords as well as the representation of important information.  
The rest of this paper is organized as follows: Section 2 reviews the work related to this study. In Section 3, the framework of the summary generation model MSMAUKE-Summ\({}^{N}\) is given and the components of the framework are described in detail. Experimental results and analysis related to the MSMAUKE-Summ\({}^{N}\) model are shown in Section 4. Section 5 summarizes the conclusions of this paper and future research directions.