# TrendFlow: A Machine Learning Framework for Research Trend Analysis
## 1 Introduction  
The field of scientific research is in a constant state of flux, with new technologies emerging at a rapid pace. Consequently, scholars face the challenge of staying abreast of the latest and most promising research directions in their respective fields. To provide a clearer understanding of this challenge, we introduce a novel machine learning task called research trend analysis (RTA). In RTA, the input consists of a set of literature and the output is comprised of pairs (cluster, keyphrases of the cluster). Each cluster represents a subset of original literature that shares the same research trend, and the keyphrases serve as indicators of that research trend. It is natural to observe that the literature within related research areas tends to be grouped in the same cluster. The size of the cluster, determined by the number of included papers, reflects the popularity of the corresponding research trend. The RTA problem can be viewed as a more challenging version of a topic modeling task, since topic modeling can only find \"keywords\" of documents, for example \"dogs\", instead of the keyphrases [1]. To this end, we do not focus on topic modeling tasks in this work. On the other hand, language models such as BERT [2] and GPT-3 [3] have revolutionized natural language processing (NLP) with their remarkable natural-language-understanding capabilities. They have been widely used for various text generation tasks, such as text summarization and keyphrase generation. Among them, KeyBART [4] is a generative pretrained language model that learns rich keyphrase representation and can improve many downstream NLP tasks, especially the keyphrase generation task. Motivated by these remarkable achievements, we propose TrendFlow, an innovative framework for RTA. TrendFlow follows a similar pattern to BERTopic [5], a topic modeling framework that clusters documents and identifies keywords using traditional topic modeling algorithms. However, TrendFlow distinguishes itself by employing advanced language models for generating keyphrases instead of relying on topic modeling algorithms. Forthis reason, we curate an RTA dataset containing 90 (abstract, trend) samples and present the KeyBART-adapter, a new model architecture that extends KeyBART using the adapter technique [6]. KeyBART-adapter freezes the parameters of KeyBART and only trains the adapters, which allows for efficient adaptation to research trend generation with limited training data and alleviates the forgetting problem of KeyBART. Furthermore, to enhance the user experience, TrendFlow supports automatic literature search on literature platforms (e.g., IEEE) based on user-specified queries and visualizes the final results. The general working pipeline of TrendFlow Is given as follows:  
1. TrendFlow starts by searching relevant literature on literature platforms such as IEEE using user-defined queries. 2. It then encodes the abstracts of the literature into embeddings (vectors) and clusters the embeddings. 3. It generates research trends from the abstracts for each cluster and visualizes the final results. The key contributions of this work are the following:  
1. We propose TrendFlow, a framework that solves the RTA problem. We have published the source code on GitHub ([[https://github.com/leoxiang66/research-analysis](https://github.com/leoxiang66/research-analysis)]([https://github.com/leoxiang66/research-analysis](https://github.com/leoxiang66/research-analysis)), accessed on 5 June 2023). 2. We annotate an RTA dataset that contains 90 (abstract, trend) samples and publish it on Huggingface ([[https://huggingface.co/datasets/Adapting/abstract-kephrases](https://huggingface.co/datasets/Adapting/abstract-kephrases)]([https://huggingface.co/datasets/Adapting/abstract-kephrases](https://huggingface.co/datasets/Adapting/abstract-kephrases)), accessed on 5 June 2023). 3. We present KeyBART-adapter, a new model that extends KeyBART with adapters for few-shot learning.