R e LM : Leveraging Language Models for Enhanced Chemical Reaction Prediction (2023)

Abstract: Predicting chemical reactions, a fundamental challenge in chemistry, involves forecasting the resulting products from a given reaction process. Conventional techniques, notably those employing Graph Neural Networks (GNNs), are often limited by insufficient training data and their inability to utilize textual information, undermining their applicability in real-world applications. In this work, we propose **ReLM**, a novel framework that leverages the chemical knowledge encoded in language models (LMs) to assist GNNs, thereby enhancing the accuracy of real-world chemical reaction predictions. To further enhance the model’s robustness and interpretability, we incorporate the confidence score strategy, enabling the LMs to self-assess the reliability of their predictions. Our experimental results demonstrate that ReLM improves the performance of state-of-the-art GNN-based methods across various chemical reaction datasets, especially in out-of-distribution settings. Codes are available at https://github.com/syr-cn/ReLM.

## introduction pre-trained language models (lms) possess a vast reserve of knowledge, coupled with impressive capabilities for logical inference @xcite @xcite @xcite . these advantages render lms useful for question-answering tasks, such as scientific inquiry and chemical inference @xcite . however, due to lms' black-box natures, distinguishing whether the answers stem from their inherent knowledge or are arbitrarily generated poses a significant challenge. furthermore, recent studies indicate that lms struggle with the graph structures of molecules in chemistryrelated tasks @xcite . recently, graph neural networks (gnns) have been widely used to address chemical reaction problems due to their ability to handle complex graph structures inherent in chemical compounds @xcite @xcite . however, gnn-based methods often suffer from limited and biased training data, resulting in poor performance in real-world applications that involve diverse reaction mechanisms. this is exemplified by the unsatisfactory performance of lo-calretro (chen and jung, 2021) on imidazo and nicolit (see results in a crucial question arises: can we develop a chemical reaction framework that synergistically integrates the advantages of both gnns and lms? in this work, we propose relm, a novel framework designed to conduct chemical reaction prediction by utilizing both pre-trained lms and gnns. relm enhances the prediction accuracy in out-ofdistribution (ood) scenarios by utilizing graph structure understanding of gnns and the natural language understanding capabilities of lms. more specifically, relm employs gnns to generate several high-probability candidate products. these products, along with appropriate in-context examples and descriptions of the reaction conditions, are then fed into the lms to facilitate accurate chemical reaction prediction (as depicted in figure 1 ). to further improve the robustness and interpretability of relm, we propose a prompting technique called the confidence score strategy (css). harnessing figure 1 : the overall framework of relm. the relm encompasses three key inputs for the language models: similar reactions (grey box), the target reaction along with its conditions (yellow box), and candidate products generated by gnns (green box). specifically, we select in-context examples with various confidence scores from the training samples that are nearest to the target reaction query. additionally, we choose the k-candidate products with the highest likelihood as predicted by the gnn. the intrinsic self-criticism capacity of lms, css boosts the model's performance without involving significant computational costs (see ## methodology in this section, we propose relm, a framework combining graph neural networks (gnns) and language models (lms) for chemical reaction analysis. relm employs gnns to generate answer candidates and in-context examples, then utilizes lms for analysis in the form of multiple-choice questions. we also propose the confidence score strategy, a generic prompting strategy to improve the reliability of lms. ## experimental results we aim to answer the following research questions: rq1: can relm improve the reaction prediction capability of graph neural networks on both out-of-distribution and in-distribution data? rq2: does the confidence strategy enhance the accuracy of relm? rq3: how does confidence score strategy influence the inference process of language models? ## conclusion despite the great success of molecular structure understanding, today's gnn-based reaction prediction methods are still far from being able to do real-world reaction analysis. in this work, we proposed relm, an in-context prompting method that utilizes both language models and graph neural networks for chemical reaction prediction. extensive experiments demonstrate the remarkable improvement of relm on various datasets indeed comes from the reasoning ability and self-criticism of lms.