Automated Extraction of Molecular Interactions and Pathway Knowledge using Large Language Model, Galactica: Opportunities and Challenges (2023)

Abstract: Understanding protein interactions and pathway knowledge is essential for comprehending living systems and investigating the mechanisms underlying various biological functions and complex diseases. While numerous databases curate such biological data obtained from literature and other sources, they are not comprehensive and require considerable effort to maintain. One mitigation strategies can be utilizing large language models to automatically extract biological information and explore their potential in life science research. This study presents an initial investigation of the efficacy of utilizing a large language model, Galactica in life science research by assessing its performance on tasks involving protein interactions, pathways, and gene regulatory relation recognition. The paper details the results obtained from the model evaluation, highlights the findings, and discusses the opportunities and challenges.

## introduction a significant portion of contemporary molecular biology research is dedicated to studying and comprehending the roles and interactions of the countless proteins that form the fundamental building blocks of life. the prediction of protein structures and functions is essential in addressing crucial challenges in life science, including developing therapeutic solutions for various diseases. by speeding up drug discovery and development, such advancements could significantly enhance healthcare. the majority of proteins have undefined functions, and only a fraction of them have been unequivocally identified through arduous and intensive laboratory research. these established protein functions are used as a benchmark to predict functions computationally by analyzing dna and amino acid sequence homology across the continuously growing repository of protein sequences obtained from genome sequencing. to deeply understand protein functions, protein interaction information can be crucial, and many databases such as string, kegg, intact, biogrid, dip, and hprd have been established to gather and maintain pathway analysis and regulatory results obtained by lab experiments and from the scientific literature. regrettably, extracting information from the existing literature demands significant manual labor and is a time-consuming process. one viable solution to remedy this is to leverage efficient machine learning models that can accurately recognize such information in scientific texts. in recent years, large language models (llms) have gained significant attention in the natural language processing (nlp) field owing to their capability to execute complex language tasks, their flexibility, and their potential to generate responses similar to humans @xcite . their application in various domains and tasks, including knowledge extraction from texts, has yielded promising outcomes @xcite . our study aims to investigate the potential of llms in extracting pathway knowledge, protein interaction, and gene regulatory information. in this study, we have assessed the capability of galactica @xcite , a general-purpose scientific llm, to accomplish these biological tasks. although galactica did not yield optimal outcomes in our biology-related tasks, it exhibited the capacity to identify specific genes/proteins, pathways, and their interactions. our preliminary findings regarding this evaluation are presented in this paper. ## related work the field of biology encompasses challenging tasks such as identifying protein-protein interactions (ppis) and gene regulatory relations. additionally, pathway analysis is a crucial area of study as it documents the interactions between proteins and reflects important molecular biological processes, such as metabolic, signaling, protein interaction, and gene regulation processes. research such as that for expression-based disease diagnosis @xcite and the identification of disease markers @xcite suggests that tasks based on pathway activity can be more stable than tasks based solely on genes. the scientific literature of biological sciences serves as an important repository of essential knowledge that has yet to be effectively discovered. to address this, nlp models based on deep neural networks have been widely adopted for analysis of structural properties of proteins (vig et al., 2020), ppis @xcite , and pathway analysis @xcite . several studies showed that llms performed comparably to traditional neural network models that necessitate labeled training data and finetuning processes, resulting in significant time and effort savings while providing a universal model capable of managing multiple tasks simultaneously @xcite . the galactica llm @xcite has been trained on a massive amount of scientific literature and has successfully tackled biological understanding task such as sequence validation perplexity, functional keyword prediction, protein function description. hence, we aim to further examine the potential of galactica in the domain of biological scientific knowledge. ## experiments we investigated the potential of galactica for addressing biological tasks related to ppis, pathway knowledge, and gene regulatory relations. to accomplish this, string, kegg, indra databases were adopted, and the details pertaining to the utilized data for these tasks can be found in appendix a. in the context of a llm, the proper selection of the number of examples or shots is essential to ensure efficient engineering. for this purpose, an ablation study was conducted to identify the optimal number of shots for each task. the shot number associated with the highest performance in test samples was selected for implementation, as detailed in appendix b. additionally, prompt construction is another critical factor that merits ## discussion and conclusion this study evaluated the performance of galactica on various biological tasks using different types of database resources, including ppis, pathway knowledge, and regulatory relations. based on our findings, it was observed that the model faced greater difficulty in answering questions that contained limited information, in contrast to questions that provided contextual details and were more specific. while the galactica did not produce ideal results in our biology-related tasks, we observed that the model demonstrated the ability to recognize certain genes and proteins and their interactions. despite this, our approach presents a potential avenue for using the model, and we anticipate that our findings will assist domain scientists and researchers in employing the model for their applications and obtaining insights into the model's behavior based on experimental outcomes.