S park RA : A Retrieval-Augmented Knowledge Service System Based on Spark Large Language Model (2024)

Abstract: Large language models (LLMs) have shown remarkable achievements across various language tasks. To enhance the performance of LLMs in scientific literature services, we developed the scientific literature LLM (SciLit-LLM) through pre-training and supervised fine-tuning on scientific literature, building upon the iFLYTEK Spark LLM. Furthermore, we present a knowledge service system Spark Research Assistant (SparkRA) based on our SciLit-LLM. SparkRA is accessible online and provides three primary functions: literature investigation, paper reading, and academic writing. As of July 30, 2024, SparkRA has garnered over 50,000 registered users, with a total usage count exceeding 1.3 million.

## introduction large language models (llms) have achieved significant success in natural language processing, including text generation and language understanding @xcite . owing to their strong capabilities, llms have shown immense potential across many downstream fields, such as education, medicine, and finance @xcite @xcite . as the performance of llms in scientific literature does not fully meet the needs of scholars, we developed a scientific literature llm (scilit-llm). we began by collecting a large dataset of scientific literature, including academic papers and patents, and performed data cleaning to ensure high-quality academic text. we then continued pre-training the open-source iflytek spark llm (13b) 2 using an autoregressive training task, followed by supervised fine-tuning, to create our scilit-llm. traditional knowledge service systems generally provide limited functionalities, such as the retrieval of scholarly articles and assistive reading services. in this paper, we introduce the spark research assistant (sparkra), a knowledge service system based on our scientific literature llm. sparkra offers a comprehensive, one-stop solution for scientific literature services. figure 1 depicts the process of constructing the sparkra system. the features of sparkra are as follows: experimental evaluation demonstrates that sparkra outperforms existing models, including gpt-3.5 and llama3-8b, across all tasks, establishing its efficacy in enhancing the productivity and accuracy of academic research activities. ## sparkra based on our scilit-llm, we developed a literature services system sparkra. this platform is comprised of three functions: literature investigation, paper reading, and academic writing. notably, sparkra is equipped to process inputs in both chinese and english, thereby catering to a diverse linguistic user base. the architecture of sparkra is shown in figure 2 and the demonstration video has been published on youtube 4 . ## experiments 4.1 experiment setting to validate the results of sparkra, we adopt the following llms as the baseline models: we evaluate the performance of models using the mean opinion score (mos) on a scale of 1 (poorest) to 5 (optimal), with evaluations conducted by more than five individuals per task. for the machine translation task, we also use the bleu metric @xcite for model evaluation. we gathered 100 academic parallel paragraphs from public chinese journals with chinese and english abstracts to serve as test sets. the highest results in the table are highlighted in bold, and the secondhighest results are underlined. to assess paper reading performance, we employ following two measures: to evaluate paper polishing and academic translation performance, we use three criteria: ## conclusion the sparkra system, built on the scilit-llm, provides a comprehensive solution for academic tasks, including literature investigation, paper reading, and academic writing. through extensive experiments, sparkra demonstrated superior performance compared to existing models like chatgpt, and even surpassed gpt-4 in specific tasks such as paper polishing, demonstrating its potential to enhance productivity for researchers and students with its precise and context-aware support for academic activities.