## 3 Applications of RAG-Based LLMs for Systematic Literature Reviews
### Applications of RAG-Based LLMs Focused on SLRs  
There is limited research on the application of RAG-based LLMs in SLRs. However, a notable study [47] introduces an innovative framework for automating research synthesis using fine-tuned domain-specific LLMs. This framework presents a method to automatically generate fine-tuned datasets from an SLR corpus. Initially, it employs GPT-4 to create question-and-answer (Q&A) pairs at two levels: paper summaries and individual paragraphs. Q&A pairs are then generated from the aggregated answers of the individual papers, equipping an LLM with both insights from specific studies and synthesized knowledge across the entire SLR corpus. Such fine-tuned models, based on the Mistral architecture, show substantial promise in streamlining the SLR process.  
LitLLM [48] is another RAG-based LLM framework designed for screening and summarizing scientific literature. An LLM is utilized to decompose a user-provided abstract into a set of keywords, which are then used to retrieve relevant papers through the Semantic Scholar API. The LLM employs an instructional permutation generation approach to re-rank the retrieved papers based on their relevance to the user's abstract and the abstracts of the papers. Finally, the LLM uses the re-ranked papers as augmented context to generate summaries. This study utilized GPT-3.5-turbo and GPT-4 models accessed via the OpenAI API.  
RefAI [49] is a RAG-based LLM tool developed for biomedical literature recommendations and summarization. Similarly, the GPT-4 model is employed to extract keywords from user queries, which are then used to retrieve paper metadata (including title, abstract, authors, publication year, journal, and DOI) through the PubMed API. A multivariate ranking algorithm ranks the literature based on the similarity between the query and paper abstracts, journal impact factor, citation count, and publication year. The similarity is computed using vector embeddings of user queries and paper abstracts, converted via sentence-transformers. Subsequently, the GPT-4 model generates literature summaries based on user queries and paper metadata. The relevance and quality of the literature recommendations, as well as the accuracy, comprehensiveness, and reference integration of the literature summarizations, are evaluated by a panel of biomedical professionals. These evaluations exhibit superior performance compared to baseline evaluations using GPT-4, Google Gemini, and ScholarAI.  
In summary, although all three studies utilize Retrieval-Augmented Generation (RAG)-based Large Language Models (LLMs) to enhance the Systematic Literature Review (SLR) process, they exhibit distinct methodologies and focal areas. The first study [47] prioritizes the creation of fine-tuned LLM datasets for research synthesis. In contrast, LitLLM [48] concentrates on refining literature retrieval and summarization through keyword extraction and a re-ranking approach. Meanwhile, RefAI [49] offers a specialized tool tailored for biomedical literature recommendations, employing a comprehensive ranking system. Collectively, these studies illustrate the diverse potential of RAG-based LLMs in automating aspects of the literature review process. However, it is important to note that these studies are conducted on relatively small datasets or within limited scopes. Furthermore, the outputs generated by the frameworks discussed in these studies are typically brief responses to specific questions or they give relatively generalized summaries, which do not equate to a thorough critique that is conducted on selected articles. This limitation underscores the necessity to develop a more robust framework that fully leverages the powerful and flexible capabilities of RAG-based LLMs to support and automate key SLR tasks. Such a framework should be trained and rigorously evaluated using large datasets across various domains to ensure its effectiveness and reliability.