Transformer-based Approach for Predicting Chemical Compound Structures (2020)

Abstract: By predicting chemical compound structures from their names, we can better comprehend chemical compounds written in text and identify the same chemical compound given different notations for database creation. Previous methods have predicted the chemical compound structures from their names and represented them by Simplified Molecular Input Line Entry System (SMILES) strings. However, these methods mainly apply handcrafted rules, and cannot predict the structures of chemical compound names not covered by the rules. Instead of handcrafted rules, we propose Transformer-based models that predict SMILES strings from chemical compound names. We improve the conventional Transformer-based model by introducing two features: (1) a loss function that constrains the number of atoms of each element in the structure, and (2) a multi-task learning approach that predicts both SMILES strings and InChI strings (another string representation of chemical compound structures). In evaluation experiments, our methods achieved higher F-measures than previous rule-based approaches (Open Parser for Systematic IUPAC Nomenclature and two commercially used products), and the conventional Transformer-based model. We release the dataset used in this paper as a benchmark for the future research.

## introduction knowledge of chemical substances is necessary for developing new materials and drugs, and for synthesizing products from new materials. to utilize such knowledge, researchers have created databases containing the physical property values of chemical substances and the interrelationships among chemical substances. it is thought that several billions of chemical compounds exist @xcite , but only a portion of these are entered into chemical databases. even pubchem foot_1 , one of the largest databases of chemical compounds, includes the information of only approximately 100 million chemical compounds. moreover, databases for chemical domains are manually maintained, which consumes much time and cost. one of the time consuming processes is the integration of the same chemical compounds with different notations. for instance, a chemical structure can be derived from partial structures which are given notational variants, or the notation can fluctuate for a given chemical compound @xcite . therefore, a system that automatically predicts a chemical compound structure from its chemical compound names would improve the database creation procedure. structures are most commonly predicted from their notations by rule-based conversion methods @xcite . although rule-based conversion can accurately predict the structures of chemical compounds based on systematic nomenclatures such as the international union of pure and applied chemistry (iupac) foot_2 nomenclature, it often fails the structure prediction of chemical compound names that violate these nomenclatures (e.g., synonyms foot_3 ). to improve the low prediction performance of compounds with non-iupac names, we propose neural network-based models that predict chemical compound structures represented as simplified molecular input line entry system (smiles) @xcite strings from chemical compound names categorized as synonyms 5 . in this work, we use the transformer-based sequenceto-sequence neural network model @xcite for machine translation, which achieves a state-of-the-art performance in various tasks among the sequence-to-sequence neural network models such as recurrent neural network-based models. to improve the conventional transformer-based model, we introduce the following two chemicalstructure oriented features: for our experiments, we created a dataset from pubchem for predicting chemical compound structures represented by smiles strings from synonyms. the experimental results demonstrate the transformer-based conversion methods achieve higher f-measures than the existing rule-based methods. in addition, our two proposals (i.e., constraining the number of atoms of each element and multi-task learning of both smiles strings and inchi strings) improve the performance of the conventional transformer-based method. ## proposed methods this section presents our proposed methods, namely, our tokenizer training method and sequence-to-sequence models. let x and t be a set of chemical compound names and a set of smiles strings, respectively. we define a training dataset consisting of n samples as figure 2 overviews the transformer-based prediction of smiles strings from chemical compound names, @xmath0. chemical compound names, smiles, and inchi are long strings without explicit boundaries (such as white spaces in english text). therefore, to convert chemical compound names to smiles strings, we propose (a) training of a tokenizer and (b) a transformer-based approach. ## conclusions this paper introduced our transformer-based prediction methods, which convert chemical compound names to smiles strings trained with the constraint of the number of atoms of each element in the smiles string. we also proposed a multitask learning approach that simultaneously learns the conversions to smiles strings and inchi strings. in an experimental comparison evaluation, our proposed method achieved higher f-measures than the existing methods. in future work, we intend to explore various tokenization methods, and further improve the prediction performance. we also hope to apply the proposed loss function to multi-task learning.