# Multistage Mixed-Attention Unsupervised Keyword Extraction for Summary Generation
## 2 Related Work
To address the inability of embedding-based keyword extraction models to fully utilize PLMs, an unsupervised keyword extraction method, PromptRank, was proposed by [PERSON] et al. [24]. It calculates the probability of keyword generation by the decoder based on prompts. Interpretable neural networks were used by [PERSON] et al. [25] for unsupervised keyword extraction, defining keywords as iconic words that are favorable for predicting the topic of a document. The above unsupervised approach simplifies the process of keyword extraction, but does not fully consider the importance of the keywords and the semantic relevance of the keywords to the sentences in the document. A mixed-attention model, AttentionRank, was proposed by [PERSON] et al. [26] to extract keywords from documents in an unsupervised manner. It uses self-attention to determine the importance of keywords in the context of a sentence and cross-attention to calculate the semantic relevance between keywords and sentences in a document. The above methods were able to adequately extract the keywords from the text, but failed to enable the keywords to be used correctly in the inference of the summaries, resulting in the absence of important information in the summaries. The uneven distribution of the extracted keywords in the original text and the lack of a rational method for their use lead to the fact that some keywords with lower weights may be used first in the inference of the summary, affecting the representation and distribution of important information in the summary. Therefore, in order to increase the detailed information of the summary, the targeted selection of keywords with higher weights to aid in inference is a critical step in summary generation. A method of selecting the best keywords was proposed by [PERSON] et al. [27] to iteratively select keywords through an attention scoring function to obtain the best results overall. Meanwhile, in order to select more representative keywords, a probabilistic method was proposed by [PERSON] et al. [28] for selecting representative keywords in a professional domain. A binary mixture model is used to obtain the distribution of keywords and select a representative subset of keywords. A keyword selection method was proposed by [PERSON] et al. [29]. The self-attentive head of the last layer of the pretraining encoder is clustered to select the most popular keywords from the original article. To address the problem of data sparsity in low-resource complex Named Entity Recognition (NER), an attention graph-aware keyword selection method was proposed by [PERSON] et al. [30]. It uses attention graph-assisted selective masking to retain named entities in the input sentence and select contextually relevant keywords. The above methods select the most representative keywords in the text, and although they enhance the local semantic representation of the summary, there is some repetition in the use of keywords. Aimed at the problems of high redundancy and poor comprehensiveness, accuracy, and readability in existing long text summaries, an \(N+1\) coarse-fine-grained multistage summary generation framework is constructed and a multistage mixed-attention unsupervised keyword extraction summary generation model (MSMAUKE-Summ\({}^{N}\)) is proposed. During data preprocessing, the sentence filtering layer, PureText, is set to remove useless information from the data samples to obtain high-quality data samples. A mixture of self-attention and cross-attention is used to consider the importance and semantic relevance of keywords at the sentence level and document level. Multistage iterative extraction of keywords is used to facilitate summarization inference to enrich the global semantic information of coarse-grained summaries, rationalize the use of keywords using the self-attention mechanism, select more weighted keywords to assist the summary, and increase the detailed information of the summary to enhance the local semantic representation of the fine-grained summary.