# Language Models for Predicting Organic Synthesis Procedures
## 1 Introduction
The development of a chemical synthesis procedure involves detailed knowledge of the specific sequence of actions required, such as adding chemicals and performing the synthesis along with accurately defining their optimal conditions, like temperature, solvent choices, and atmospheric requirements. In classical chemistry, empirical knowledge and iterative testing have guided the determination of optimal synthesis conditions and steps [23]. The challenge is devising procedures for novel or minimally investigated reactions, due to a dependence on pre-existing databases. Recent advancements in DL for molecule generation also present new challenges and opportunities for chemists. These methods facilitate the discovery of compounds within previously underexplored chemical domains. Chemists need to create specific synthesis plans and procedures for practical laboratory applications based on these discoveries. The transition to machine learning (ML) provides a more effective method for analyzing chemical data, improving prediction for unexplored reactions. The task does not have many established conventions compared to challenges like language translation or text classification. The lack of established standards for creating synthesis methods requires the creation of flexible and adaptable methodologies. While synthesis procedure generation is not extensively studied, a transformer-based sequence-to-sequence model has been shown to attain a BLEU score of 54.7 through text-based representations by IBM [8]. Our methodology is a variant of the published methods and is closer to the research demonstrating the effectiveness of the FLAN-T5 model for the generation of esterification synthesis reactions, which attains a BLEU score of 51.82 [24]. Our paper aims to expand on the available knowledge and explore LLM applicability for the prediction of synthesis instructions for a large variety of organic synthesis types. Specifically, we assess the performance of LLMs by fine-tuning them with a curated dataset to predict a precise sequence of actions and parameters. Such a strategy offers an initial predictive means to scientists to optimize reaction conditions in advance and simplify the experimental process. This method seeks to enhance the efficiency of chemical synthesis, potentially reducing both the time and resources used in laboratory work. To enhance the state of the art in generating synthesis procedures, our research presents several pivotal contributions:  
* **Creation of the novel dataset. ** We compiled a unique dataset encompassing multiple reaction classes (-931,350 reactions in total). This dataset forms the foundation for the exploration of synthesis procedure prediction. * **Selective input strategy for enhanced predictive modeling. ** A key feature of our methodology is the intentional omission of ancillary compounds from the input to our models; we provide only the reactants and products, purposely leaving out any agents not specific to the reaction, such as gases, solvents, and catalysts. This approach adds complexity to the model's task, as it cannot rely solely on the given inputs to predict all necessary steps and parameters, including those related to ancillary compounds. However, this limitation results in outputs that are particularly useful for researchers, especially in the context of novel compounds for which detailed synthetic documentation is absent. In such cases, usually, the reactants and products are often the only known factors before laboratory experiments. * **LLMs suitability exploration and comparison. ** We investigated the suitability of different sizes of BART, T5, FLAN-T5, molT5, and classic sequence-to-sequence transformer models suitable for our text-to-text task. In the experiments, we first tested pre-trained models and then used configurations with randomly initialized models, allowing us to assess the impact of pre-trained weights.