Original: C HEMICAL N AMES S TANDARDIZATION USING N EURAL S EQUENCE TO S EQUENCE M ODEL
Revised: C HEMICAL N AMES S TANDARDIZATION USING N EURAL S EQUENCE TO S EQUENCE M ODEL

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Chemical information extraction is to convert chemical knowledge in text into true chemical database, which is a text processing task heavily relying on chemical compound name identiﬁcation and standardization.
Revised: Chemical information extraction is to convert chemical knowledge in text into true chemical database, which is a text processing task heavily relying on chemical compound name identiﬁcation and standardization.

Original: Once a systematic name for a chemical compound is given, it will naturally and much simply convert the name into the eventually required molecular formula.
Revised: Once a systematic name for a chemical compound is given, it will naturally and much simply convert the name into the eventually required molecular formula.

Original: However, for many chemical substances, they have been shown in many other names besides their systematic names which poses a great challenge for this task.
Revised: However, for many chemical substances, they have been shown in many other names besides their systematic names which poses a great challenge for this task.

Original: In this paper, we propose a framework to do the auto standardization from the non-systematic names to the corresponding systematic names by using the spelling error correction, byte pair encoding tokenization and neural sequence to sequence model.
Revised: In this paper, we propose a framework to do the auto standardization from the non-systematic names to the corresponding systematic names by using the spelling error correction, byte pair encoding tokenization and neural sequence to sequence model.

Original: Our framework is trained end to end and is fully data-driven.
Revised: Our framework is trained end to end and is fully data-driven.

Original: Our standardization accuracy on the test dataset achieves 54.04% which has a great improvement compared to previous state-of-the-art result.
Revised: Our standardization accuracy on the test dataset achieves 54.04% which has a great improvement compared to previous state-of-the-art result.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: There are more than 100 million named chemical substances in the world.
Revised: There are more than 100 million named chemical substances in the world.

Original: In order to uniquely identify every chemical substance, there are elaborate rules for assigning names to them on the basis of their structures.
Revised: In order to uniquely identify every chemical substance, there are elaborate rules for assigning names to them on the basis of their structures.

Original: These names are called systematic names.
Revised: These names are called systematic names.

Original: The rules for these names are deﬁned by International Union of Pure and Applied Chemistry (IUPAC) (Favre & Powell, 2013).
Revised: The rules for these names are deﬁned by International Union of Pure and Applied Chemistry (IUPAC) (Favre & Powell, 2013).

Original: However, besides the systematic name, there can be also many other names for a chemical substance due to many reasons.
Revised: However, besides the systematic name, there can be also many other names for a chemical substance due to many reasons.

Original: Firstly, many chemical are so much a part of our life that we know them by their familiar names which we call them common names or trivial names for the sake of simplicity.
Revised: Firstly, many chemical are so much a part of our life that we know them by their familiar names which we call them common names or trivial names for the sake of simplicity.

Original: For example, sucrose is a kind of sugar which we are very familiar with.
Revised: For example, sucrose is a kind of sugar which we are very familiar with.

Original: Its systematic name is much more complicated, which is (2R,3R,4S,5S,6R)-2-[(2S,3S,4S,5R)-3,4-dihydroxy-2,5bis(hydroxymethyl)oxolan-2-yl]oxy-6-(hydroxymethyl
Revised: Its systematic name is much more complicated, which is (2R,3R,4S,5S,6R)-2-[(2S,3S,4S,5R)-3,4-dihydroxy-2,5bis(hydroxymethyl)oxolan-2-yl]oxy-6-(hydroxymethyl

Original: )oxane-3,4,5-triol
Revised: )oxane-3,4,5-triol

Original: Secondly, in chemistry industry, especially in pharmaceutical industry, many producers always generate new names to a chemical substance in order to distinguish their products from those of their competitors.
Revised: Secondly, in chemistry industry, especially in pharmaceutical industry, many producers always generate new names to a chemical substance in order to distinguish their products from those of their competitors.

Original: We call these kind of names proprietary names.
Revised: We call these kind of names proprietary names.

Original: The most famous example is Aspirin.
Revised: The most famous example is Aspirin.

Original: Its systematic name is 2-Acetoxybenzoic acid .
Revised: Its systematic name is 2-Acetoxybenzoic acid .

Original: So due to the history reasons and idiomatic usages, a chemical substance can have many other names.
Revised: So due to the history reasons and idiomatic usages, a chemical substance can have many other names.

Original: Chemical information extraction is a research that extracts useful chemical knowledge in text and converts it into a database, which strongly relies on the unique standard chemical names.
Revised: Chemical information extraction is a research that extracts useful chemical knowledge in text and converts it into a database, which strongly relies on the unique standard chemical names.

Original: Nowadays, there are many chemical databases such as PubChem and SciFinder, which are designed to store chemical information including chemical names, chemical structures, molecular formulas and other relevant information.
Revised: Nowadays, there are many chemical databases such as PubChem and SciFinder, which are designed to store chemical information including chemical names, chemical structures, molecular formulas and other relevant information.

Original: For these databases, it is still an ongoing work to extract chemical information from chemical papers to update the databases.
Revised: For these databases, it is still an ongoing work to extract chemical information from chemical papers to update the databases.

Original: If all the chemical substances are expressed by the systematic names, it is easy to generate other information.
Revised: If all the chemical substances are expressed by the systematic names, it is easy to generate other information.

Original: For example, we can nearly perfectly convert the systematic name to other representations such as Simpliﬁed Molecular-Input Line-Entry
Revised: For example, we can nearly perfectly convert the systematic name to other representations such as Simpliﬁed Molecular-Input Line-Entry

Original: System (SMILES)
Revised: System (SMILES)

Original: (Weininger, 1988) and International Chemical Identiﬁer (InCHI)
Revised: (Weininger, 1988) and International Chemical Identiﬁer (InCHI)

Original: (Mcnaught, 2006) and then generate the structural formulas.
Revised: (Mcnaught, 2006) and then generate the structural formulas.

Original: Some online systems are already well developed for converting automatically systematic names to SMILES string with a very high precision such as Open Parser for Systematic IUPAC
Revised: Some online systems are already well developed for converting automatically systematic names to SMILES string with a very high precision such as Open Parser for Systematic IUPAC

Original: Nomenclature (OPSIN) (Lowe et al., 2011) developed by
Revised: Nomenclature (OPSIN) (Lowe et al., 2011) developed by

Original: [Table]
Revised: [Table]

Original: University of Cambridge 1 .
Revised: University of Cambridge 1 .

Original: Unfortunately, nowadays a great number of the chemical substances are expressed by their non-systematic names in chemical papers, which increases signiﬁcantly the difﬁculties for this task, so our work focuses on the standardization of non-systematic names.
Revised: Unfortunately, nowadays a great number of the chemical substances are expressed by their non-systematic names in chemical papers, which increases signiﬁcantly the difﬁculties for this task, so our work focuses on the standardization of non-systematic names.

Original: Examples of chemical information extraction are shown in Figure 1.
Revised: Examples of chemical information extraction are shown in Figure 1.

Original: In the following passage, we consider the differences between non-systematic names and systematic names as ”error” 2 .
Revised: In the following passage, we consider the differences between non-systematic names and systematic names as ”error” 2 .

Original: In view of natural language processing, the error types of non-systematic names can be summarized by four types: 1.
Revised: In view of natural language processing, the error types of non-systematic names can be summarized by four types: 1.

Original: Spelling error .
Revised: Spelling error .

Original: It means that non-systematic names just have slightly differences from systematic names in spelling; 2.
Revised: It means that non-systematic names just have slightly differences from systematic names in spelling; 2.

Original: Ordering error .
Revised: Ordering error .

Original: It means that the groups in a non-systematic name are in wrong order; 3.
Revised: It means that the groups in a non-systematic name are in wrong order; 3.

Original: Common name error .
Revised: Common name error .

Original: As mentioned above, many chemical substances have common names or proprietary names which look totally different from their systematic names; 4.
Revised: As mentioned above, many chemical substances have common names or proprietary names which look totally different from their systematic names; 4.

Original: Synonym error .
Revised: Synonym error .

Original: It means that the words in the nonsystematic names are different from those in the systematic names but they share the same root of word.
Revised: It means that the words in the nonsystematic names are different from those in the systematic names but they share the same root of word.

Original: In fact, it is the error type which happens most often.
Revised: In fact, it is the error type which happens most often.

Original: For example, 2-(Acetyloxy)benzoic Acid has synonyms Acetylsalicylic Acid and Acetysal and these three words share the same root of word ”Acety”.
Revised: For example, 2-(Acetyloxy)benzoic Acid has synonyms Acetylsalicylic Acid and Acetysal and these three words share the same root of word ”Acety”.

Original: Some examples of different types of errors are shown in Table 1.
Revised: Some examples of different types of errors are shown in Table 1.

Original: What is worth mentioning is that several types of error can appear at the same time in a single non-systematic name, especially for the ordering error and synonym error.
Revised: What is worth mentioning is that several types of error can appear at the same time in a single non-systematic name, especially for the ordering error and synonym error.

Original: The mixed types of error make this task very challenging.
Revised: The mixed types of error make this task very challenging.

Original: Based on these four error types, we propose a framework to convert automatically the non-systematic names to systematic names.
Revised: Based on these four error types, we propose a framework to convert automatically the non-systematic names to systematic names.

Original: Our framework is structured as followed: 1.
Revised: Our framework is structured as followed: 1.

Original: Spelling error correction.
Revised: Spelling error correction.

Original: It aims to correct the spelling errors; 2.
Revised: It aims to correct the spelling errors; 2.

Original: Byte pair encoding (BPE) tokenization.
Revised: Byte pair encoding (BPE) tokenization.

Original: It aims to split a name into small parts; 3.
Revised: It aims to split a name into small parts; 3.

Original: Sequence to sequence model.
Revised: Sequence to sequence model.

Original: It aims to ﬁx all the remaining ordering errors, common name errors and synonym errors.
Revised: It aims to ﬁx all the remaining ordering errors, common name errors and synonym errors.

Original: Actually, due to its great challenge, few work has been done on the chemical name standardization.
Revised: Actually, due to its great challenge, few work has been done on the chemical name standardization.

Original: To our best knowledge, Golebiewski et al.
Revised: To our best knowledge, Golebiewski et al.

Original: (2009) is the only work deserving a citation which developed an online system ChemHits to do the standardization basing on several transformation rules and the queries to online chemical databases.
Revised: (2009) is the only work deserving a citation which developed an online system ChemHits to do the standardization basing on several transformation rules and the queries to online chemical databases.

Original: The work of Golebiewski et al.
Revised: The work of Golebiewski et al.

Original: (2009) severely depends on chemical knowledge, limiting its application potential and effectiveness to some extent.
Revised: (2009) severely depends on chemical knowledge, limiting its application potential and effectiveness to some extent.

Original: Differently, we adopt sequence to sequence model that has been widely used on neural machine translation.
Revised: Differently, we adopt sequence to sequence model that has been widely used on neural machine translation.

Original: The reason why we apply the sequence to sequence model is that our task has some similarities with the machine translation problem.
Revised: The reason why we apply the sequence to sequence model is that our task has some similarities with the machine translation problem.

Original: In machine translation, there are source language and target language which correspond to the non-systematic names and the systematic names in our task.
Revised: In machine translation, there are source language and target language which correspond to the non-systematic names and the systematic names in our task.

Original: Two different languages can be different in: 1.
Revised: Two different languages can be different in: 1.

Original: Vocabularies, which corresponds to the common name error and synonym error; 2.
Revised: Vocabularies, which corresponds to the common name error and synonym error; 2.

Original: Word order, which corresponds to the ordering error.
Revised: Word order, which corresponds to the ordering error.

Original: Our framework is trained end-to-end, fully data-driven and without using external chemical knowledge.
Revised: Our framework is trained end-to-end, fully data-driven and without using external chemical knowledge.

Original: With this approach, we achieve an accuracy of 54.04% in our test data set.
Revised: With this approach, we achieve an accuracy of 54.04% in our test data set.

Original: Our work will be done on a corpus containing chemical names extracted from report of Chemical
Revised: Our work will be done on a corpus containing chemical names extracted from report of Chemical

Original: Journals with High Impact factors (CJHIF) 3 .
Revised: Journals with High Impact factors (CJHIF) 3 .

Original: The corpus is collected and checked by paid manual work.
Revised: The corpus is collected and checked by paid manual work.

Original: It is a parallel corpus which includes non-systematic names and systematic names of chemical substances.
Revised: It is a parallel corpus which includes non-systematic names and systematic names of chemical substances.

Original: In the following passage, we call a non-systematic name and the corresponding systematic name of a chemical substance data pair.
Revised: In the following passage, we call a non-systematic name and the corresponding systematic name of a chemical substance data pair.

Original: In our corpus, there are 384816 data pairs.
Revised: In our corpus, there are 384816 data pairs.

Original: In Figure 2, we give an overview of the distribution of the Levenshtein distance between the non-systematic names and the systematic names to show how different the non-systematic names and the systematic names are.
Revised: In Figure 2, we give an overview of the distribution of the Levenshtein distance between the non-systematic names and the systematic names to show how different the non-systematic names and the systematic names are.

Original: In the experiment, we use 80%, 19% and 1% data as training set, test set and development set respectively.
Revised: In the experiment, we use 80%, 19% and 1% data as training set, test set and development set respectively.

Original:  P ROPOSED FRAMEWORKS
Revised:  P ROPOSED FRAMEWORKS

Original: Our framework consists of spelling error correction, byte pair encoding tokenization and sequence to sequence model which can be summarized in Figure 3.
Revised: Our framework consists of spelling error correction, byte pair encoding tokenization and sequence to sequence model which can be summarized in Figure 3.

Original: 2. S PELLING ERROR CORRECTION
Revised: 2. S PELLING ERROR CORRECTION

Original: In this part, we aim to correct the spelling errors.
Revised: In this part, we aim to correct the spelling errors.

Original: Given a name of a chemical substance, we can separate it into different elemental words by all the non-alphabet characters.
Revised: Given a name of a chemical substance, we can separate it into different elemental words by all the non-alphabet characters.

Original: For example, 2-(chloroﬂuoro-methyl)-benzooxazole can be separated into chloro , ﬂuoro , methyl and benzooxazole .
Revised: For example, 2-(chloroﬂuoro-methyl)-benzooxazole can be separated into chloro , ﬂuoro , methyl and benzooxazole .

Original: To correct the spelling error, ﬁrstly we set up two vocabularies from the dataset: vocabulary of the systematic elemental words and of the non-systematic elemental words.
Revised: To correct the spelling error, ﬁrstly we set up two vocabularies from the dataset: vocabulary of the systematic elemental words and of the non-systematic elemental words.

Original: For the systematic elemental words, we just split all the systematic names to build the vocabulary.
Revised: For the systematic elemental words, we just split all the systematic names to build the vocabulary.

Original: For the non-systematic elemental words, ﬁrstly we use all the non-systematic names to build an elemental vocabulary, and then we just keep the elemental words which appear many times in the non-systematic names but outside the vocabulary of systematic elemental words and remove the rest.
Revised: For the non-systematic elemental words, ﬁrstly we use all the non-systematic names to build an elemental vocabulary, and then we just keep the elemental words which appear many times in the non-systematic names but outside the vocabulary of systematic elemental words and remove the rest.

Original: By this way, the vocabulary we build from the non-systematic names is the set of common names or synonyms.
Revised: By this way, the vocabulary we build from the non-systematic names is the set of common names or synonyms.

Original: We then combine these two vocabularies together to get a ﬁnal elemental vocabulary.
Revised: We then combine these two vocabularies together to get a ﬁnal elemental vocabulary.

Original: To do the correction search efﬁciently enough, we use BK-Tree (Burkhard & Keller, 1973) to structure the elemental vocabulary.
Revised: To do the correction search efﬁciently enough, we use BK-Tree (Burkhard & Keller, 1973) to structure the elemental vocabulary.

Original: BK-Tree is a tree structure which is widely used in spelling error correction.
Revised: BK-Tree is a tree structure which is widely used in spelling error correction.

Original: BK-Tree is deﬁned in the following way.
Revised: BK-Tree is deﬁned in the following way.

Original: An arbitrary vocabulary item a is selected as root node.
Revised: An arbitrary vocabulary item a is selected as root node.

Original: The root node may have zero or more subtrees.
Revised: The root node may have zero or more subtrees.

Original: The k-th subtree is recursively built of all vocabulary items b such that d ( a, b ) =
Revised: The k-th subtree is recursively built of all vocabulary items b such that d ( a, b ) =

Original: k where d ( a, b ) is the Levenshtein distance between a and b .
Revised: k where d ( a, b ) is the Levenshtein distance between a and b .

Original: Given a word and a threshold, BK-Tree can return rapidly, if possible, the vocabulary item which have the smallest Levenshtein distance with the given word and the Levenshtein distance is smaller than the threshold by using the triangle rules: | d ( a, b ) − d ( b, c ) | ≤ d ( a, c ) ≤ d ( a, b ) + d ( b, c ) .
Revised: Given a word and a threshold, BK-Tree can return rapidly, if possible, the vocabulary item which have the smallest Levenshtein distance with the given word and the Levenshtein distance is smaller than the threshold by using the triangle rules: | d ( a, b ) − d ( b, c ) | ≤ d ( a, c ) ≤ d ( a, b ) + d ( b, c ) .

Original: By using the BK-Tree, we can correct the spelling error of non-systematic names.
Revised: By using the BK-Tree, we can correct the spelling error of non-systematic names.

Original: Another advantage of using BK-Tree is that it is easy to insert new training data which makes it scalable.
Revised: Another advantage of using BK-Tree is that it is easy to insert new training data which makes it scalable.

Original: An example of BK-Tree built from a part of our dataset is shown in Figure 4.
Revised: An example of BK-Tree built from a part of our dataset is shown in Figure 4.

Original: At this stage, given a name of a chemical substance, we ﬁrstly separate it into elemental words and then input the elemental words one by one to the BK-Tree.
Revised: At this stage, given a name of a chemical substance, we ﬁrstly separate it into elemental words and then input the elemental words one by one to the BK-Tree.

Original: After the correction, we combine the elemental words to get the full name.
Revised: After the correction, we combine the elemental words to get the full name.

Original: In this step, a few non-systematic names can be directly corrected and some non-systematic names can be partially corrected.
Revised: In this step, a few non-systematic names can be directly corrected and some non-systematic names can be partially corrected.

Original: It is also helpful in the training of the sequence to sequence model because it can reduce the noise of elemental words.
Revised: It is also helpful in the training of the sequence to sequence model because it can reduce the noise of elemental words.

Original: The value on each edge is the Levenshtein distance between two nodes.
Revised: The value on each edge is the Levenshtein distance between two nodes.

Original: All the nodes in the same subtree of a node have the same Levenshtein distance to this node.
Revised: All the nodes in the same subtree of a node have the same Levenshtein distance to this node.

Original: For instance, the Levenshtein distances from dimethyl , diethyl , methan to methyl are all 2.
Revised: For instance, the Levenshtein distances from dimethyl , diethyl , methan to methyl are all 2.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: azanium
Revised: azanium

Original: [Table]
Revised: [Table]

Original: 2. T OKENIZATION BY B YTE P AIR E NCODING
Revised: 2. T OKENIZATION BY B YTE P AIR E NCODING

Original: To apply the sequence-to-sequence model, ﬁrstly we need to tokenize all the chemical names.
Revised: To apply the sequence-to-sequence model, ﬁrstly we need to tokenize all the chemical names.

Original: In this paper, we use Byte Pair Encoding (BPE) (Sennrich et al., 2015) to do the tokenization.
Revised: In this paper, we use Byte Pair Encoding (BPE) (Sennrich et al., 2015) to do the tokenization.

Original: Firstly, we initialize a symbol set by split all the names into characters.
Revised: Firstly, we initialize a symbol set by split all the names into characters.

Original: At this moment, the symbol set contains only the single characters.
Revised: At this moment, the symbol set contains only the single characters.

Original: Then we iteratively count all symbol pairs and replace each occurrence of the most frequent pair (X, Y) with a new symbol XY and add it to the symbol set.
Revised: Then we iteratively count all symbol pairs and replace each occurrence of the most frequent pair (X, Y) with a new symbol XY and add it to the symbol set.

Original: Each merge operation produces a new symbol.
Revised: Each merge operation produces a new symbol.

Original: The size of ﬁnal symbol set is equal to the size of initial character, plus the number of merge operations.
Revised: The size of ﬁnal symbol set is equal to the size of initial character, plus the number of merge operations.

Original: We then use the trained symbol vocabulary set to do the tokenization.
Revised: We then use the trained symbol vocabulary set to do the tokenization.

Original: The reasons why we choose BPE are as follow: Firstly, it can deal with out-of-vocabulary problem because the vocabulary set generated by BPE contains the vocabularies at character level.
Revised: The reasons why we choose BPE are as follow: Firstly, it can deal with out-of-vocabulary problem because the vocabulary set generated by BPE contains the vocabularies at character level.

Original: Secondly, it can separate a name into meaningful subwords because it can ﬁnd the small molecules which appear frequently in the corpus and tokenize a chemical name into the names of the small molecules.
Revised: Secondly, it can separate a name into meaningful subwords because it can ﬁnd the small molecules which appear frequently in the corpus and tokenize a chemical name into the names of the small molecules.

Original: Some examples of applying BPE to the chemical name are shown in Table 2.
Revised: Some examples of applying BPE to the chemical name are shown in Table 2.

Original: After the tokenization, we can use the split pairs to train the sequence to sequence model.
Revised: After the tokenization, we can use the split pairs to train the sequence to sequence model.

Original: 2. S EQUENCE TO SEQUENCE MODEL
Revised: 2. S EQUENCE TO SEQUENCE MODEL

Original: Sequence to sequence model (Sutskever et al., 2014) is widely used in machine translation.
Revised: Sequence to sequence model (Sutskever et al., 2014) is widely used in machine translation.

Original: In this work, we adapted an existing implementation OpenNMT (Klein et al., 2017) with a few modiﬁcations.
Revised: In this work, we adapted an existing implementation OpenNMT (Klein et al., 2017) with a few modiﬁcations.

Original: The sequence to sequence model consists of two recurrent neural networks (RNN) working together: (1) an encoder that gets the source sequences (here are the non-systematic names separated by BPE) and generates a context vector H , and (2) a decoder that uses this context vector to generate the target sequences (here are the corresponding systematic names).
Revised: The sequence to sequence model consists of two recurrent neural networks (RNN) working together: (1) an encoder that gets the source sequences (here are the non-systematic names separated by BPE) and generates a context vector H , and (2) a decoder that uses this context vector to generate the target sequences (here are the corresponding systematic names).

Original: For the encoder, we use a multilayers bidirectional LSTM (BiLSTM) (Graves & Schmidhuber, 2005).
Revised: For the encoder, we use a multilayers bidirectional LSTM (BiLSTM) (Graves & Schmidhuber, 2005).

Original: BiLSTM consists of two LSTMs: one that processes the sequence forward and the other backward, with their forward and backward hidden states −→ h t and ←− h t at each time step.
Revised: BiLSTM consists of two LSTMs: one that processes the sequence forward and the other backward, with their forward and backward hidden states −→ h t and ←− h t at each time step.

Original: The hidden state at time step t is just a concatenation of the two hidden states: h t = {−→ h t ; ←− h t } .
Revised: The hidden state at time step t is just a concatenation of the two hidden states: h t = {−→ h t ; ←− h t } .

Original: At the ﬁnal time step T of the encoder, by combining all the hidden states, we get the context vector H
Revised: At the ﬁnal time step T of the encoder, by combining all the hidden states, we get the context vector H

Original: = { h 1 , ..., h T } .
Revised: = { h 1 , ..., h T } .

Original: For the decoder, it gives the probability of an output sequence ˆ y =
Revised: For the decoder, it gives the probability of an output sequence ˆ y =

Original: { ˆ y i } :
Revised: { ˆ y i } :

Original: [Equation]
Revised: [Equation]

Original: and for a single token ˆ y t , the probability is calculated by
Revised: and for a single token ˆ y t , the probability is calculated by

Original: [Equation]
Revised: [Equation]

Original: where f is a multilayer LSTM; s t is the decoder’s hidden state at time step t ; and W c , W s are learned weights.
Revised: where f is a multilayer LSTM; s t is the decoder’s hidden state at time step t ; and W c , W s are learned weights.

Original: For the score function, we use the attention mechanism proposed by Luong et al.
Revised: For the score function, we use the attention mechanism proposed by Luong et al.

Original: [Equation]
Revised: [Equation]

Original: where W a are also learned weights.
Revised: where W a are also learned weights.

Original:  E XPERIMENTS 3. T RAINING DETAILS
Revised:  E XPERIMENTS 3. T RAINING DETAILS

Original: In our framework, at the spelling error correction stage, the only parameter is the threshold of the
Revised: In our framework, at the spelling error correction stage, the only parameter is the threshold of the

Original: BK-Tree.
Revised: BK-Tree.

Original: In the experiment, we have tried several threshold values: 1, 2 and 3.
Revised: In the experiment, we have tried several threshold values: 1, 2 and 3.

Original: At the BPE stage, the only parameter is the number of the merge operations.
Revised: At the BPE stage, the only parameter is the number of the merge operations.

Original: In the experiments, we have tried several values: 2500, 5000, 10000, 20000.
Revised: In the experiments, we have tried several values: 2500, 5000, 10000, 20000.

Original: For the sequence to sequence model, the dimensions of word embeddings and hidden states are both 500.
Revised: For the sequence to sequence model, the dimensions of word embeddings and hidden states are both 500.

Original: The vocabulary size is equal to the number of basic characters plus the number of merge operations of BPE.
Revised: The vocabulary size is equal to the number of basic characters plus the number of merge operations of BPE.

Original: The numbers of layers in encoder and decoder are both 2.
Revised: The numbers of layers in encoder and decoder are both 2.

Original: Before training the sequence to sequence model, we also do the spelling error correction for the non-systematic names in the training data.
Revised: Before training the sequence to sequence model, we also do the spelling error correction for the non-systematic names in the training data.

Original: During training, all parameters of the sequence to sequence model are trained jointly using stochastic gradient descent (SGD).
Revised: During training, all parameters of the sequence to sequence model are trained jointly using stochastic gradient descent (SGD).

Original: The loss function is a cross-entropy function, expressed as
Revised: The loss function is a cross-entropy function, expressed as

Original: [Equation]
Revised: [Equation]

Original: The loss was computed over an entire minibatch of the size 64 and then normalized.
Revised: The loss was computed over an entire minibatch of the size 64 and then normalized.

Original: The weights are initialized using a random uniform distribution ranging from -0.1 to 0.1.
Revised: The weights are initialized using a random uniform distribution ranging from -0.1 to 0.1.

Original: The initial learning rate is 1.0 and the decay will be applied with the factor 0.5 every epoch after and including epoch 8 or when the perplexity does not decrease on the validation set.
Revised: The initial learning rate is 1.0 and the decay will be applied with the factor 0.5 every epoch after and including epoch 8 or when the perplexity does not decrease on the validation set.

Original: The drop out rate is 0.3 and we train the model for 15 epochs.
Revised: The drop out rate is 0.3 and we train the model for 15 epochs.

Original: We set the beam size to 5 for the decoding.
Revised: We set the beam size to 5 for the decoding.

Original: For comparison, we also do another experiment by replacing sequence to sequence model with
Revised: For comparison, we also do another experiment by replacing sequence to sequence model with

Original: Statistical Machine Translation (SMT) model.
Revised: Statistical Machine Translation (SMT) model.

Original: In this experiment, we use implemented Moses system Koehn et al.
Revised: In this experiment, we use implemented Moses system Koehn et al.

Original: In the training, we limit the length of training sequences to 80 and apply the 3-grams language model by using KenLM (Heaﬁeld, 2011).
Revised: In the training, we limit the length of training sequences to 80 and apply the 3-grams language model by using KenLM (Heaﬁeld, 2011).

Original: The tokenization for the pairs we use is BPE with 5000 merge operations.
Revised: The tokenization for the pairs we use is BPE with 5000 merge operations.

Revised: Besides the spelling error correction, data augmentation is another technique for the neural model learning to deal with the noisy data (in this case, the noise is the spelling error).

Revised: For comparison, we also do the experiment of data augmentation.

Revised: For every non-systematic name, we insert an error into it for the probability of 0.025.

Revised: The error insertion has four types: we insert randomly a character in a random position; we randomly delete a character in a random position; we randomly exchange two characters; we randomly replace a character by another random character.

Revised: The four insertion methods are applied in an equal probability.

Original: 3. R ESULTS
Revised: 3. R ESULTS

Original: In the experiment, we measure the standardization quality with accuracy and BLEU score (Papineni et al., 2002).
Revised: In the experiment, we measure the standardization quality with accuracy and BLEU score (Papineni et al., 2002).

Original: Accuracy is calculated by the number of non-systematic names which are successfully standardized divided by the total number of non-systematic names.
Revised: Accuracy is calculated by the number of non-systematic names which are successfully standardized divided by the total number of non-systematic names.

Original: Note the accuracy that we adopt here is a very strong performance metric, as it equally means that the entire translated sentence is exactly matched for a machine translation task.
Revised: Note the accuracy that we adopt here is a very strong performance metric, as it equally means that the entire translated sentence is exactly matched for a machine translation task.

Original: The experiment results for different models on test dataset are shown in Table 3.
Revised: The experiment results for different models on test dataset are shown in Table 3.

Original: We can see that the combination of spelling error correction, BPE tokenization and sequence to sequence model achieves the best performance.
Revised: We can see that the combination of spelling error correction, BPE tokenization and sequence to sequence model achieves the best performance.

Original: Our framework has a great improvement compared to the SMT model and the ChemHits system.
Revised: Our framework has a great improvement compared to the SMT model and the ChemHits system.

Original: The latter is slightly better than just applying spelling error correction.
Revised: The latter is slightly better than just applying spelling error correction.

Original: The results for different numbers of BPE merge operation are shown in Table 4.
Revised: The results for different numbers of BPE merge operation are shown in Table 4.

Original: 5000 is the best value for this parameter.
Revised: 5000 is the best value for this parameter.

Original: 0 means a characterlevel sequence to sequence model.
Revised: 0 means a characterlevel sequence to sequence model.

Original: The results show the usefulness of BPE.
Revised: The results show the usefulness of BPE.

Original: The results for different
Revised: The results for different

Original: Levenshtein distance thresholds for the spelling error correction  are shown in Table 5.
Revised: Levenshtein distance thresholds for the spelling error correction and the result of data augmentation are shown in Table 5.

Original: We can see that spelling error correction is indeed helpful for our framework. 
Revised: We can see that spelling error correction is indeed helpful for our framework. [Table]

Revised: [Table]

Revised: Data augmentation also helps but does not perform as well as spelling error correction.

Original: Note that when the threshold is too large, the overcorrection might occur which reduces the standardization quality.
Revised: Note that when the threshold is too large, the overcorrection might occur which reduces the standardization quality.

Original: 3. A NALYSIS
Revised: 3. A NALYSIS

Original: Some examples of the non-systematic names which are successfully standardized are shown in Table 6.
Revised: Some examples of the non-systematic names which are successfully standardized are shown in Table 6.

Original: These 4 examples show what the sequence to sequence model can do.
Revised: These 4 examples show what the sequence to sequence model can do.

Original: In the ﬁrst example, the parentheses in the non-systematic name are replaced by another parentheses and brackets.
Revised: In the ﬁrst example, the parentheses in the non-systematic name are replaced by another parentheses and brackets.

Original: It means[Table] [Table] that the sequence to sequence model can ﬁx also the non-alphabet spelling errors.
Revised: It means that the sequence to sequence model can ﬁx also the non-alphabet spelling errors.

Original: The synonym error is also corrected: from 1-propanetriol to propane-1-thiol .
Revised: The synonym error is also corrected: from 1-propanetriol to propane-1-thiol .

Original: In the second example, the wrong order ethane,1,2-dichloro is corrected to 1,2-dichloroethane .
Revised: In the second example, the wrong order ethane,1,2-dichloro is corrected to 1,2-dichloroethane .

Original: In the third example, the mixture of ordering error and synonym error are corrected.
Revised: In the third example, the mixture of ordering error and synonym error are corrected.

Original: In the last example, P-anise alcohol is a proprietary name which looks like totally different from its systematic name but it is also successfully standardized.
Revised: In the last example, P-anise alcohol is a proprietary name which looks like totally different from its systematic name but it is also successfully standardized.

Original: To better illustrate how the sequence to sequence model works, here we give the visualization of attentions of an example, which is shown in Figure 5.
Revised: To better illustrate how the sequence to sequence model works, here we give the visualization of attentions of an example, which is shown in Figure 5.

Original: The non-systematic name is adenine,9methyl- (7ci,8ci) and the corresponding systematic name is 9-methyl-9H-purin-6-amine .
Revised: The non-systematic name is adenine,9methyl- (7ci,8ci) and the corresponding systematic name is 9-methyl-9H-purin-6-amine .

Original: In the nonsystematic name, adenine itself is also a chemical substance whose systematic name is 9H-purin-6amine .
Revised: In the nonsystematic name, adenine itself is also a chemical substance whose systematic name is 9H-purin-6amine .

Original: So it is a mixture of common name error and ordering error.
Revised: So it is a mixture of common name error and ordering error.

Original: From Figure 6, we can see that seq2seq model can ﬁnd the relation between adenine and 9H-purin-6-amine and can ﬁnd the right place for 9-methyl . 
Revised: From Figure 6, we can see that seq2seq model can ﬁnd the relation between adenine and 9H-purin-6-amine and can ﬁnd the right place for 9-methyl . [Table]

Revised: Example3-(dimethoxymethylsilyl)-1-propanetriol 3-[dimethoxy(methyl)silyl]propane-1-thiol

Revised: Exampleethane,1,2-dichloro 1,2-dichloroethane

Original:  3. L IMITATIONS
Revised: Example1-phenyl-3-methyl-4-benzoyl-1h-pyrazol-5(4h)-one 4-benzoyl-3-methyl-1-phenyl-4,5-dihydro-1H-pyrazol-5-one Example P-anise alcohol (4-methoxyphenyl)methanol 3. E RROR ANALYSIS

Revised: In this section, we will analyze the fail standardization attempts of our system.

Revised: Firstly, we randomly select 100 samples of failed attempts and label their error types manually and carefully.

Revised: The distribution over error types is shown in Table 7.

Revised: We can see that synonym error is the most confusing error type and our system performs well at spelling error.

Revised: As for the common error, since it is very hard to ﬁnd a rule between an unseen common name and its systematic name, our system also perform poorly at this error type.

Revised: Among these 100 samples, there are 10 samples which are nearly correct (only one or two characters different from the systematic name), 7 examples are totally incorrect (none of the subwords of prediction match the systematic name) and the rest are partially correct.

Revised: Some samples of failed attempts are shown in Table 8.

Revised: 3. L IMITATIONS

Original: We noticed that there are still nearly a half of the non-systematic names which are not successfully standardized.
Revised: We noticed that there are still nearly a half of the non-systematic names which are not successfully standardized.

Original: The accuracy for systematic names of different lengths are shown in Figure 6.
Revised: The accuracy for systematic names of different lengths are shown in Figure 6.

Original: We can see that our framework achieves the best performances for the systematic names of length between 20 and 40 while performing poorly for the systematic names of length bigger than 60 which account for 37% of our test dataset.
Revised: We can see that our framework achieves the best performances for the systematic names of length between 20 and 40 while performing poorly for the systematic names of length bigger than 60 which account for 37% of our test dataset.

Original: Another limitation of our model is that we do not take into account chemical rules in our model.
Revised: Another limitation of our model is that we do not take into account chemical rules in our model.

Original: For this reason, a few names generated by our model disobey the chemical rules and at the tokenization stage, some subwords generated by BPE are not explicable [Table] Example3-(dimethoxymethylsilyl -1-propanetriol
Revised: For this reason, a few names generated by our model disobey the chemical rules and at the tokenization stage, some subwords generated by BPE are not explicable as well.

Original: -[dimethoxy(methyl)silyl

Original: propane-1-thiol

Original: Exampleethane,1,2-dichloro 1,2-dichloroethane

Original: Example

Original: phenyl-3-methyl-4-benzoyl-1h-pyrazol-5(4h)-one 4-benzoyl-3-methyl-1-phenyl-4,5-dihydro-1H-pyrazol-5-one

Original: Example

Original: P-anise alcohol

Original: (4-methoxyphenyl)methanol

Original: as well. However, although there is lots of room for improvement, we still consider that our work is meaningful in practice.
Revised: However, although there is lots of room for improvement, we still consider that our work is meaningful in practice.

Original: Imagine that there is a long list of non-systematic names to be standardized
Revised: Imagine that there is a long list of non-systematic names to be standardized

Revised: [Table]

Revised: [Table]

Original: and we maintain a database which can check whether a name is systematic name or not.
Revised: and we maintain a database which can check whether a name is systematic name or not.

Original: In this case, we can process the non-systematic names by our system and check whether the results are correct.
Revised: In this case [Table] we can process the non-systematic names by our system and check whether the results are correct.

Original: After the processing, about 50% of non-systematic names are corrected which can save more than 50% manual work.
Revised: After the processing [Table] about 50% of non-systematic names are corrected which can save more than 50% manual work.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we propose a framework to automatically convert non-systematic names to systematic names .
Revised: In this work [Table] we propose a framework to automatically convert non-systematic names to systematic names .

Original: Our framework consists of spelling error correction, byte pair encoding tokenization and sequence to sequence model.
Revised: Our framework consists of spelling error correction, byte pair encoding tokenization and sequence to sequence model.

Original: Our framework achieves an accuracy of 54.04% on our dataset, which is far better than previous rule based system (nine times of accuracy) and thus enables the related chemical information extraction into more practical use stage.
Revised: Our framework achieves an accuracy of 54.04% on our dataset, which is far better than previous rule based system (nine times of accuracy) and thus enables the related chemical information extraction into more practical use stage.

Original: The advantage of our framework is that it is trained end to end, fully data-driven and independent of external chemical knowledge.
Revised: The advantage of our framework is that it is trained end to end, fully data-driven and independent of external chemical knowledge.

Original: This work starts a brand new research line for the related chemical information extraction as to our best knowledge.
Revised: This work starts a brand new research line for the related chemical information extraction as to our best knowledge.