# TrendFlow: A Machine Learning Framework for Research Trend Analysis
## 2 Related Work
### Keyphrase Generation  
Keyphrase generation (KG) is a fundamental task in natural language processing (NLP) that aims to generate a set of keyphrases given a document. Keyphrases are concise and informative phrases that capture the main topics or themes of a document. They can be used for various applications such as document summarization, indexing, retrieval, clustering, and classification.  
Most traditional methods for Knowledge Graph (KG) construction primarily rely on extractive techniques, which involve selecting keyphrases from existing words or phrases in the document. Extractive methods typically follow a two-step process: candidate selection and ranking. In candidate selection, potential keyphrases are identified based on linguistic patterns, such as noun phrases or term frequency. Subsequently, in the ranking step, scores are assigned to the candidates using various features, including TF-IDF, position, length, or graph-based measures. Some examples of extractive methods are KEA ([PERSON] et al., 1999) [20], TextRank ([PERSON] and [PERSON], 2004) [21], and SingleRank ([PERSON] and [PERSON], 2008) [22].  
However, extractive methods have some limitations. First, they cannot generate novel keyphrases that do not appear in the document but are still relevant to its content. Second, they may miss important keyphrases that are expressed implicitly or paraphrased in the document. Third, they may produce redundant or overlapping keyphrases that convey similar information.  
To overcome these limitations, recent methods for KG are mostly generative, meaning that they can produce new words or phrases that are not necessarily present in the document.  
Generative methods mainly adopt the transformer-based sequence-to-sequence (seq2 seq) models, which employ encoder-decoder architectures with attention mechanisms to generate keyphrases [23]. Example models include T5 [24] and BART [25].  
Recently, a novel pre-trained model, known as KeyBART, has been introduced for acquiring rich keyphrase representations, resulting in enhanced keyphrase generation performance by harnessing the robust capabilities of the BART architecture [4]. Experimental findings demonstrate that KeyBART surpasses state-of-the-art methods in terms of performance for the keyphrase generation task.  
In this work, we concentrate on the KeyBART model. Given that our dataset is relatively small, comprised of only 50 training samples, fully fine-tuning KeyBART may lead to a forgetting problem [24]. Consequently, we investigate the incorporation of adapter techniques into the KeyBART model to address this issue.  
Adapters are lightweight modules that can be seamlessly integrated into the architecture of a pre-trained model. They enable efficient fine-tuning while addressing the forgetting problem. Comprising feed-forward neural networks, adapters adjust the activations of the model's hidden layers without modifying the original weights [6]. By leveraging adapters, the model can be fine-tuned on smaller datasets without compromising the crucial pre-trained knowledge, thereby preserving its generalization capabilities.  
In the context of KeyBART, integrating adapter techniques can potentially enhance the model's ability to generate meaningful keyphrases from a limited dataset. By preserving the pre-trained knowledge and adjusting the activations with adapters, KeyBART can effectively learn from the small dataset without losing its ability to generate coherent and relevant keyphrases.