L ang2 M ol-Diff: A Diffusion-Based Generative Model for Language-to-Molecule Translation Leveraging SELFIES Representation (2024)

Abstract: Generating de novo molecules from textual descriptions is challenging due to potential issues with molecule validity in SMILES representation and limitations of autoregressive models. This work introduces Lang2Mol-Diff, a diffusion-based language-to-molecule generative model using the SELFIES representation. Specifically, Lang2Mol-Diff leverages the strengths of two state-of-the-art molecular generative models: BioT5 and TGM-DLM. By employing BioT5 to tokenize the SELFIES representation, Lang2Mol-Diff addresses the validity issues associated with SMILES strings. Additionally, it incorporates a text diffusion mechanism from TGM-DLM to overcome the limitations of autoregressive models in this domain. To the best of our knowledge, this is the first study to leverage the diffusion mechanism for text-based de novo molecule generation using the SELFIES molecular string representation. Performance evaluation on the L+M-24 benchmark dataset shows that Lang2Mol-Diff outperforms all existing methods for molecule generation in terms of validity. Our code and pre-processed data are available at https://github.com/nhattruongpham/mol-lang-bridge/tree/lang2mol/.

## introduction molecules, the elementary constituents of all matter, play a pivotal role in dictating the properties and functionalities that govern our world. the immense scale of chemical space, estimated to encompass around 10 33 molecules @xcite , presents a significant challenge for traditional methods in finding new medicine, materials, and chemical processes. this has driven the exploration of artificial intelligence models for efficient molecule finding. a key advancement lies in the confluence of natural language and molecular representations such as smiles (simplified molecular-input line-entry system) @xcite and selfies (self-referencing embedded strings) @xcite . these representations enable the seamless integration of natural language descriptions with corresponding molecular structures. by leveraging pre-trained language models and fine-tuning them on different benchmark datasets combining natural language and molecular string representations, researchers have successfully developed numerous downstream models capable of generating novel molecule structures based on textual descriptions outlining desired properties. besides, the success of diffusion models in image generation has spurred their application to text generation, and more recently, to the domain of molecular representation. in this research, we use the diffusion mechanism to address the limitations of autoregressive models, where errors from earlier predictions can propagate and magnify throughout the sequence and lead to inaccuracies, especially in long sequences. we also want to deal with validity issues in generating new molecules. the proposed method is a novel architecture that incorporates advancements in both the backbone model and the molecular representation. in essence, our key contributions are as follows: ## conclusion this work presents lang2mol-diff, a novel diffusion-based language-to-molecule generative model that addresses the challenges of de novo molecule generation from textual descriptions. by leveraging the strengths of biot5 for accurate tokenization of the selfies representation and incorporating a text diffusion mechanism inspired by