R eact XT : Understanding Molecular “Reaction-ship” via Reaction-Contextualized Molecule-Text Pretraining (2024)

Abstract: Molecule-text modeling, which aims to facilitate molecule-relevant tasks with a textual interface and textual knowledge, is an emerging research direction. Beyond single molecules, studying reaction-text modeling holds promise for helping the synthesis of new materials and drugs. However, previous works mostly neglect reaction-text modeling: they primarily focus on modeling individual molecule-text pairs or learning chemical reactions without texts in context. Additionally, one key task of reaction-text modeling – experimental procedure prediction – is less explored due to the absence of an open-source dataset. The task is to predict step-by-step actions of conducting chemical experiments and is crucial to automating chemical synthesis. To resolve the challenges above, we propose a new pretraining method, ReactXT, for reaction-text modeling, and a new dataset, OpenExp, for experimental procedure prediction. Specifically, ReactXT features three types of input contexts to incrementally pretrain LMs. Each of the three input contexts corresponds to a pretraining task to improve the text-based understanding of either reactions or single molecules. ReactXT demonstrates consistent improvements in experimental procedure prediction and molecule captioning and offers competitive results in retrosynthesis. Our code is available at https://github.com/syr-cn/ReactXT.

## introduction multi-modal large language models (lms) have recently attracted extensive research attention. remarkably, in the vision-language domain, lms enhanced with visual encoders show impressive results in visual question-answering and image captioning @xcite . inspired by their successes, molecule-text modeling (mtm) becomes an emerging research field @xcite @xcite , aiming to build the natural language interface for molecular tasks, including text-guided molecule generation, molecule captioning, and molecule-text retrieval @xcite . building upon these mtm works, we study reaction-text modeling (rtm), aiming to improve lms' performance on reaction-relevant tasks. chemical reactions, involving the transformation of reactants into products, are fundamental to advancing drug discovery and material science @xcite . revisiting prior works, we identify key research gaps in both the learning paradigm and the evaluation benchmark for rtm: an open-source dataset for experimental procedure prediction is notably missing. as illustrated in figure 2 , experimental procedure prediction aims to deduce the 1. makesolution with @xmath0 (140 mg, 0.37 mmol) and @xmath1 (112 mg, 0.74 mmol) and @xmath2 (4 ml); 2. stir for 16 hours at 70°c; 3. concentrate; 4. drysolid under vacuum; 5. yield @xmath3 (82 mg, 51%). ## openexp: an open-source dataset for experimental procedure prediction here we briefly introduce openexp's curation process and defer the details to appendix a.1. openexp is sourced from chemical reaction databases of uspto-applications @xcite and @xcite . as illustrated in figure 2 , these databases include chemical reactions and the corresponding unstructured descriptions of experimental procedures. to convert these unstructured descriptions into structured action sequences, we first run the pragraph2action model from @xcite , and then conduct preprocessing following @xcite . the preprocessing is to remove low-quality data, eliminate duplicates, and construct molecule mapping between reactions and experimental procedures. specific preprocessing steps are summarized in as shown in to obtain insights on dataset quality, we invite two graduate students in chemistry to rate the alignment between the action sequences and their original descriptions, on a scale from 1 (lowest) to 5 (highest), as depicted in figure 5 . briefly, of the total 250 samples evaluated, 126 (≥ 50%) action sequences have at most 1 error (scores above 4), and 181 (≥ 50%) action sequences have at most 2 errors (scores above 3). our closer inspection shows that the one error in score-4 samples is usually a typo of material/action name, or a discrepancy of numerical value, and does not impede the overall execution. see appendix c.3.2 for details. ## experiment we empirically evaluate reactxt across three downstream tasks, including experimental procedural prediction, molecule captioning, and retrosynthesis. further, we include ablation studies showcasing the contributions of individual components. to ensure the significance of our experimental, we include statistical tests results in appendix c.2. ## conclusion and future works in this work, we explore reaction-text modeling to empower reaction-relevant tasks with textual interfaces and knowledge. we present reactxt, a pretraining method to learn chemical reactions within the context of the corresponding molecular textual descriptions. additionally, we propose a new dataset openexp to support open-source research for experimental procedure prediction. reactxt establishes the best performances across tasks of experimental procedure prediction and molecule captioning. it presents competitive performances for retrosynthesis. in future work, we plan to apply lms to learn the interactions among large molecules (e.g., proteins and nucleic acids), or introduce molecules' dynamics and 3d spatial structures for better molecule-language understanding @xcite .