NLP eople at Shared Task: An Ensembled Approach for Molecule Captioning from SMILES (2024)

Abstract: This paper presents our approach submitted to the Language + Molecules 2024 ( L+M-24 ) Shared Task in the Molecular Captioning track. The task involves generating captions that describe the properties of molecules that are provided in SMILES format.We propose a method for the task that decomposes the challenge of generating captions from SMILES into a classification problem,where we first predict the molecule’s properties. The molecules whose properties can be predicted with high accuracy show high translation metric scores in the caption generation by LLMs, while others produce low scores. Then we use the predicted properties to select the captions generated by different types of LLMs, and use that prediction as the final output. Our submission achieved an overall increase score of 15.21 on the dev set and 12.30 on the evaluation set, based on translation metrics and property metrics from the baseline.

## introduction molecular design is the process of devising molecules with desired properties and functions. while this is widely practiced in fields such as drug discovery, new materials, and chemical processes, predicting the properties of designed molecules remains a challenging problem. to tackle this problem, language models trained on molecular information have gained attention @xcite . the l+m-24 shared task @xcite involves translation between smiles @xcite , a string-encoded molecular format, and descriptive captions of the molecule's properties. the dataset covers four high-impact areas of molecular science: biomedical, human interaction, light and electricity, and agriculture and industry, providing pairs of molecules and their corresponding captions for these properties. an example of the data is shown in figure 1 . in this sample, specific diseases and protein properties figure 1 : a sample molecule depicted using rdkit @xcite and its caption from the training data. caption: the molecule is a jak inhibitor, immunomodulator, protein tyrosine kinase inhibitor, protein kinase inhibitor and belongs to the autoimmune disease treatment class of molecules. in this paper, we describe our submission to the molecular captioning track. we first address the properties of smiles as a multi-label classification problem. predicting properties is essential for molecule captioning and offers the following advantage: a lightweight model can be built that predicts the properties of the molecules compared to fine-tuning existing large transformer-based models. such an approach can get classification accuracy of 80% against experimental measurements with as little as 100 datapoints @xcite . we also fine-tune llasmol mistral and multitask text chemistry t5 @xcite models for the end-to-end molecular captioning. we obtain the system's output by selecting the generated captions from these models based on the predicted properties. we achieve an overall increase score from the molt5-small foot_0 baseline of 15.21 on the dev set and 12.30 on the eval set. in the next section, we discuss some related work that inspired our contributions to this shared task. ## related work text2mol @xcite stands out as a pioneering study integrating modalities between text and molecules. this task involves retrieving molecules using natural language descriptions as queries. they employ the scibert @xcite model to encode the text information and a graph convolutional network for the molecular information. the model is based on a cross-modal attention structure and successfully integrates the two modalities. molt5 @xcite is a t5 (raffel et al., 2020) based model that enables both molecule captioning and molecule generation, which generates smiles from natural language. the model is first trained using an objective that replaces corrupted spans. this task is performed on general text data in the form of the c4 corpus (colossal clean crawled corpus) as well as on smiles from the zinc-15 dataset @xcite . this pre-training procedure encourages the model to learn textual and chemical information. the model is then fine-tuned for molecule captioning and molecule generation using chebi-20 @xcite , which comprises approximately 33k text-molecule pairs. another t5 based model, text+chem t5 @xcite , aims at improving multitasking and multi-domain capabilities. this model is trained not only on smiles and caption pairs such as chebi-20 but also on reactionproducts pairs such as pistachio dataset used in @xcite , and experimental procedures dataset @xcite for chemical synthesis actions. it can perform multiple tasks beyond text2molecule and molecule2text translation, including mol2mol and text2text tasks. the mol2mol tasks contain forward reaction prediction, which predicts products from given reactants, and retrosynthesis, which predicts the necessary substances for synthesis from a given chemical compound. the text2text task consists of paragraph to action, which generates sequential steps to execute a described chemical reaction. a notable aspect of this model is its ability to perform all these tasks without additional fine-tuning, using a single model instead of individual specialised models for each task. this eliminates the need to develop tailored models for each domain, achieving a unified representation of the chemical domain with one model. ## system description figure 2 shows an overview of the submission system. first, we develop a classifier to predict properties from a given smiles string. the molecular properties are extracted using the evaluation script for the property metrics foot_1 by determining whether a predefined string is included in the tokenised captions using scibert_scivocab_uncased foot_2 . based on our analysis of the extracted properties, there are 1,084 unique properties present in the training data. since properties are extracted using string-matching, some occur together. some co-occurances are correct biochemically, like "biomedical disease -heart disease" and "biomedical disease -diabetic heart disease". others are not, like "biomedical disease -non-alcoholic fatty liver disease" and "human interaction and organoleptics -organoleptic effect relationsfatty". this leads to chemically incorrect labelling for some molecules. a molecule whose description is "this molecule impacts non-alcoholic fatty liver disease" is not necessarily fatty in the sense of organoleptic effects, yet it is always labelled as so. ## results and discussion in this section, we present the results of our classification models and generated captions using multitask t5 and llasmol mistral . ## conclusion in this work, we present our approach to the molecular captioning task. we propose combining a property classification model, llms for caption generation, and an ensemble method. our results show that molecules distinguished by property classification exhibit varying strengths and weaknesses depending on the model used. this approach achieved a translation increase score of 5.52 on the dev set and 5.68 on the eval set. for property metrics, we recorded an increase score of 18.44 on the dev set and 14.50 on the eval set. the overall increase score was 15.21 on the dev set and 12.30 on the eval set, ranking 4th in this shared task.