Original: S YNONYM N ET :
Revised: S YNONYM N ET :

Original: M ULTI - CONTEXT
Revised: M ULTI - CONTEXT

Original: B ILATERAL
Revised: B ILATERAL

Original: M ATCHING FOR E NTITY S YNONYMS
Revised: M ATCHING FOR E NTITY S YNONYMS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Being able to automatically discover synonymous entities from a large free-text corpus has transformative effects on structured knowledge discovery.
Revised: Being able to automatically discover synonymous entities from a large free-text corpus has transformative effects on structured knowledge discovery.

Original: Existing works either require structured annotations, or fail to incorporate context information effectively, which lower the efﬁciency of information usage.
Revised: Existing works either require structured annotations, or fail to incorporate context information effectively, which lower the efﬁciency of information usage.

Original: In this paper, we propose a framework for synonym discovery from free-text corpus with minimal human annotation.
Revised: In this paper, we propose a framework for synonym discovery from free-text corpus without structured annotation.

Original: As one of the key components in synonym discovery, we introduce a novel neural network model S YNONYM N ET to determine whether or not two given entities are synonym with each other.
Revised: As one of the key components in synonym discovery, we introduce a novel neural network model S YNONYM N ET to determine whether or not two given entities are synonym with each other.

Original: Instead of using entities features, S YNONYM N ET makes use of multiple pieces of contexts in which the entity is mentioned, and compares the context-level similarity via a bilateral matching schema to determine synonymity.
Revised: Instead of using entities features, S YNONYM N ET makes use of multiple pieces of contexts in which the entity is mentioned, and compares the context-level similarity via a bilateral matching schema to determine synonymity.

Original: Experimental results demonstrate that the proposed model achieves state-of-the-art results on both generic and domain-speciﬁc synonym datasets: Wiki+Freebase, PubMed+UMLS and MedBook+MKG, with up to 4.16% improvement in terms of Area Under the Curve (AUC) and 3.19% in terms of Mean Average Precision (MAP) compare to the best baseline method.
Revised: Experimental results demonstrate that the proposed model achieves state-of-the-art results on both generic and domain-speciﬁc synonym datasets: Wiki+Freebase, PubMed+UMLS and MedBook+MKG, with up to 4.16% improvement in terms of Area Under the Curve (AUC) and 3.19% in terms of Mean Average Precision (MAP) compare to the best baseline method.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Discovering synonymous entities from a massive corpus is an indispensable task for automated knowledge discovery.
Revised: Discovering synonymous entities from a massive corpus is an indispensable task for automated knowledge discovery.

Original: For each entity, its synonyms refer to the entities that can be used interchangeably under certain contexts (Wikipedia contributors, 2018).
Revised: For each entity, its synonyms refer to the entities that can be used interchangeably under certain contexts.

Original: For example, Clogged Nose and Nasal Congestion are synonymous relative to the context they are mentioned.
Revised: For example, Clogged Nose and Nasal Congestion are synonymous relative to the context they are mentioned.

Original: Given two entities, the synonym discoveryis a taskthat determines how likely these two entities are synonym with each other.
Revised: Given two entities, the synonym discovery task determines how likely these two entities are synonym with each other.

Original: The main goal of synonym discovery is to learn a metric that distinguishes synonym entities from non-synonym ones.
Revised: The main goal of synonym discovery is to learn a metric that distinguishes synonym entities from non-synonym ones.

Original: The synonym discovery task is challenging to deal with, a part of which comes from the various entity expressions.
Revised: The synonym discovery task is challenging to deal with, a part of which due to the various entity expressions.

Original: For example, U.S.A / United States of America / United States / U.S.actually refer to the same entity but expressed quite differently.
Revised: For example, U.S.A / United States of America / United States / U.S. refer to the same entity but expressed quite differently.

Original: Recent works on synonym discovery focus on learning the similarity from entities and their character-level features (Neculoiu et al., 2016; Mueller & Thyagarajan, 2016).
Revised: Recent works on synonym discovery focus on learning the similarity from entities and their character-level features (Neculoiu et al., 2016; Mueller & Thyagarajan, 2016).

Original: These methods work well for synonyms that share a lot of character-level features like airplane / aeroplane or an entity and its abbreviation like Acquired Immune Deficiency Sydrome / AIDS .
Revised: These methods work well for synonyms that share a lot of character-level features like airplane / aeroplane or an entity and its abbreviation like Acquired Immune Deficiency Syndrome / AIDS .

Original: However, a much larger number of synonym entities in the real-world do not share a lot of character-level features, such as JD / law degree , or clogged nose and nasal congestion .
Revised: However, a much larger number of synonym entities in the real world do not share a lot of character-level features, such as JD / law degree , or clogged nose and nasal congestion .

Original: With only character-level features being used, these models hardly obtain the ability to differentiate entities that share similar semantics but are not alike verbatim.
Revised: With only character-level features being used, these models hardly obtain the ability to discriminate entities that share similar semantics but are not alike verbatim.

Original: Context information is crucial in representing entity synonymity, as the meaning of an entity can be better reﬂected by the contexts in which it appears.
Revised: Context information is helpful in indicating entity synonymity, as the meaning of an entity can be better reﬂected by the contexts in which it appears.

Original: Modeling the context for entity synonym usually suffers from following challenges: 1) Semantic Structure .
Revised: Modeling the context for entity synonym usually suffers from following challenges: 1) Semantic Structure .

Original: Context, as a snippet of natural language sentence, is essentially semantically structured.
Revised: Context, as a snippet of natural language sentence, is essentially semantically structured.

Original: Existing models such as skip-gram (Mikolov et al., 2013) simply models the context words of an entity as a bag of words.
Revised: Some existing models encode the semantic structures in the contexts implicitly during the entity representation learning (Mikolov et al., 2013; Pennington et al., 2014; Peters et al., 2018).

Original: The resulting entity representations embody some semantic information: entities with similar contexts are likely to live in proximity in the embedding space.
Revised: The context-aware entity representations embody meaningful semantics: entities with similar contexts are likely to live in proximity in the embedding space.

Original: However, it also tends to bring noisy entities in proximity when the structure information in the context is simply ignored.
Revised: Some other works extract and model contexts in an explicit manner with additional structured annotations.

Original: Additional annotation such as dependency parsing (Qu et al., 2017), user click information (Wei et al., 2009), or signed heterogeneous graphs (Ren & Cheng, 2015) are introduced to guide synonym discovery with additional annotation information.
Revised: Structured annotations such as dependency parsing (Qu et al., 2017), user click information (Wei et al., 2009), or signed heterogeneous graphs (Ren & Cheng, 2015) are introduced to guide synonym discovery.

Original: 2) Diverse Contexts .
Revised: 2) Diverse Contexts .

Original: An entity can be mentioned under a wide range of circumstances.
Revised: An entity can be mentioned under a wide range of circumstances.

Original: Previous works on context-based synonym discovery either focus on entity information only (Neculoiu et al., 2016; Mueller & Thyagarajan, 2016), or use a single piece of context for each entity (Liao et al., 2017;
Revised: Previous works on context-based synonym discovery either focus on entity information only (Neculoiu et al., 2016; Mueller & Thyagarajan, 2016), or use a single piece of context for each entity (Liao et al., 2017;

Original: Qu et al., 2017) to learn a similarity function for entity matching.
Revised: Qu et al., 2017) to learn a similarity function for entity matching.

Original: While in practice, similar context is only sufﬁcient but not necessary condition for context matching for synonym entities.
Revised: While in practice, similar context is only sufﬁcient but not necessary condition for context matching for synonym entities.

Original: Especially, in some domains such as medical, the context expression preference varies a lot from individuals.
Revised: Notably, in some domains such as medical, the context expression preference varies a lot from individuals.

Original: For example, sinus congestion is usually referred by medical professionals in  medical literature, while stuffy nose is often used by patients on social media.
Revised: For example, sinus congestion is usually referred by medical professionals in the medical literature, while stuffy nose is often used by patients on social media.

Original: It is also not practical to assume that each piece of context is equally informative to represent the meaning of an entity: a context may contribute differently when matched with different contexts of other entities.
Revised: It is also not practical to assume that each piece of context is equally informative to represent the meaning of an entity: a context may contribute differently when matched with different contexts of other entities.

Original: Thus it is imperative to focus on multiple pieces of contexts during the matching  for accuracy and robustness.
Revised: Thus it is imperative to focus on multiple pieces of contexts with a dynamic matching schema for accuracy and robustness.

Original: Inthe light of these challenges, we propose a framework to discover synonym entities from a massive corpus, with minimized annotation.
Revised: In light of these challenges, we propose a framework to discover synonym entities from a massive corpus without additional structured annotation.

Revised: Candidate entities are obtained from a massive text corpus unsupervisely.

Original: A novel neural network model S YNONYM N ET is proposed to detect entity synonyms based on two given entities via a bilateral matching among multiple pieces of contexts in which each entity appears.
Revised: A novel neural network model S YNONYM N ET is proposed to detect entity synonyms based on two given entities via a bilateral matching among multiple pieces of contexts in which each entity appears.

Original: A leaky unit is designed to explicitly alleviate the noises from uninformative context during the matching process.
Revised: A leaky unit is designed to explicitly alleviate the noises from uninformative context during the matching process.

Original: The contribution of this work is summarized as follows:
Revised: The contribution of this work is summarized as follows:

Original: • We introduce a framework for entity synonym discovery from massive text corpus without additional structured annotation.

Original: Unsupervisely trained candidate entities are generated and validated by the model.

Original: • We propose S YNONYM N ET , a context-aware  model to detect entity synonyms.
Revised: • We propose S YNONYM N ET , a context-aware bilateral matching model to detect entity synonyms.

Original: S YNONYM N ET utilizes multiple pieces of contexts in which each entity appears, and a bilateral matching schema with leaky units to distill the common contexts forany two given entities.
Revised: S YNONYM N ET utilizes multiple pieces of contexts in which each entity appears, and a bilateral matching schema with leaky units to distill informative contexts for two given entities.

Revised: • We introduce a synonym discovery framework that adopts S YNONYM N ET to obtain synonym entities from a free-text corpus without additional structured annotation.

Original: • Experiments on generic and domain-speciﬁc real-world datasets  demonstrate the effectiveness of the proposed model for synonym discovery.
Revised: • Experiments on generic and domain-speciﬁc real-world datasets in English and Chinese demonstrate the effectiveness of the proposed model for synonym discovery.

Original:  S YNONYM N ET
Revised:  S YNONYM N ET

Original: We introduce S YNONYM N ET , our proposed model that detects whether or not two entities are synonyms to each other based on a bilateral matching between multiple pieces of contexts in which entities appear.
Revised: We introduce S YNONYM N ET , our proposed model that detects whether or not two entities are synonyms to each other based on a bilateral matching between multiple pieces of contexts in which entities appear.

Original: Figure 1 gives an overview of the proposed model. 
Revised: Figure 1 gives an overview of the proposed model. [Figure]

Original: 2. C ONTEXT R ETRIEVER
Revised: 2. C ONTEXT R ETRIEVER

Original: For each entity e , the context retriever  fetches P pieces of contexts from the corpus D in which the entity appears.
Revised: For each entity e , the context retriever randomly fetches P pieces of contexts from the corpus D in which the entity appears.

Original: We denote the retrieved contexts for e as a set C
Revised: We denote the retrieved contexts for e as a set C

Original: = { c 0 , c 1 , ..., c P } , where
Revised: = { c 0 , c 1 , ..., c P } , where

Original: P is the number of context pieces.
Revised: P is the number of context pieces.

Original: Each piece of context c p ∈ C contains a sequence of words c p = ( w (0) p , w (1) p , ..., w ( T ) p ) , where T is the length of the context, which varies from one instance to another.
Revised: Each piece of context c p ∈ C contains a sequence of words c p = ( w (0) p , w (1) p , ..., w ( T ) p ) , where T is the length of the context, which varies from one instance to another.

Original: w ( t ) p is the t -th word in the p -th context retrieved for an entity e .
Revised: w ( t ) p is the t -th word in the p -th context retrieved for an entity e .

Original: 2. C ONFLUENCE C ONTEXT E NCODER
Revised: 2. C ONFLUENCE C ONTEXT E NCODER

Original: For the p -th context c p , an encoder tries to learn a continuous vector that represents the context.
Revised: For the p -th context c p , an encoder tries to learn a continuous vector that represents the context.

Original: For example, a recurrent neural network (RNN) such as a bidirectional LSTM (Bi-LSTM) (Hochreiter & Schmidhuber, 1997) can be applied to sequentially encode the context into hidden states:
Revised: For example, a recurrent neural network (RNN) such as a bidirectional LSTM (Bi-LSTM) (Hochreiter & Schmidhuber, 1997) can be applied to sequentially encode the context into hidden states:

Original: [Equation]
Revised: [Equation]

Original: where w ( t ) p is the word embedding vector used for the word w ( t ) p .
Revised: where w ( t ) p is the word embedding vector used for the word w ( t ) p .

Original: We could concatenate the last hidden state → h ( T ) p in the forward LSTM fw with the ﬁrst hidden state ← h ( 0 ) p from the backward LSTM bw to obtain the context vector h p for c p : h p =
Revised: We could concatenate the last hidden state → h ( T ) p in the forward LSTM fw with the ﬁrst hidden state ← h ( 0 ) p from the backward LSTM bw to obtain the context vector h p for c p : h p =

Original: [ → h ( T ) p , ← h ( 0 ) p ] .
Revised: [ → h ( T ) p , [Equation] h ( 0 ) p ] .

Original: However, such approach does not explicitly consider the location where the entity is mentioned in the context.
Revised: However, such approach does not explicitly consider the location where the entity is mentioned in the context.

Original: As the context becomes longer, it is getting risky to simply rely on the [Figure]
Revised: As the context becomes longer, it is getting risky to simply rely on the gate functions of LSTM to properly encode the context.

Original: We introduce an encoder architecture that models contexts for synonym discovery, namely the conﬂuence context encoder.
Revised: We introduce an encoder architecture that models contexts for synonym discovery, namely the conﬂuence context encoder.

Original: The conﬂuence context encoder learns to encode the local information around the entity from the raw context, without utilizing additional structured annotations.
Revised: The conﬂuence context encoder learns to encode the local information around the entity from the raw context, without utilizing additional structured annotations.

Original: It focuses on both forward and backward directions.
Revised: It focuses on both forward and backward directions.

Original: However, the encoding process for each direction ceases immediately after it goes beyond the entity word in the context: h p =
Revised: However, the encoding process for each direction ceases immediately after it goes beyond the entity word in the context: h p =

Original: [ → h ( t e ) p , ← h ( t e ) p ] , where t e is the index of the entity word e in the context and h p ∈ R 1 × d CE .
Revised: [ → h ( t e ) p , ← h ( t e ) p ] , where t e is the index of the entity word e in the context and h p ∈ R 1 × d CE .

Original: By doing this, the conﬂuence context encoder summarizes the context while explicitly considers entity’s location in the context, where no additional computation cost is introduced.
Revised: By doing this, the conﬂuence context encoder summarizes the context while explicitly considers entity’s location in the context, where no additional computation cost is introduced.

Original: Comparing with existing works for context modeling (Cambria et al., 2018) where the left context and right context are modeled separately, but with the entity word being discarded, the conﬂuence context encoder preserves entity mention information as well as the inter-dependencies between the left and right contexts.
Revised: Comparing with existing works for context modeling (Cambria et al., 2018) where the left context and right context are modeled separately, but with the entity word being discarded, the conﬂuence context encoder preserves entity mention information as well as the inter-dependencies between the left and right contexts.

Original: 2. B ILATERAL M ATCHING WITH L EAKY U NIT
Revised: 2. B ILATERAL M ATCHING WITH L EAKY U NIT

Original: Considering the base case, where we want to identify whether or not two entities, say e and k , are synonyms with each other, we propose to ﬁnd the consensus information from multiple pieces of contexts via a bilateral matching schema.
Revised: Considering the base case, where we want to identify whether or not two entities, say e and k , are synonyms with each other, we propose to ﬁnd the consensus information from multiple pieces of contexts via a bilateral matching schema.

Original: Recall that for entity e , P pieces of encoded contexts H
Revised: Recall that for entity e , P pieces of encoded contexts H

Original: = { h 1 , h 2 , ..., h P } are collected.
Revised: = { h 1 , h 2 , ..., h P } are randomly fetched.

Original: And for entity k , we denote Q pieces of encoded contexts being collected as G = { g 1 , g 2 , ..., g Q } .
Revised: And for entity k , we denote Q pieces of encoded contexts being fetched as G = { g 1 , g 2 , ..., g Q } .

Original: Instead of focusing on a single piece of context to determine entity synonym, we adopt a bilateral matching between multiple pieces of encoded contexts for both accuracy and robustness.
Revised: Instead of focusing on a single piece of context to determine entity synonym, we adopt a bilateral matching between multiple pieces of encoded contexts for both accuracy and robustness.

Original: H → G matching phrase:
Revised: H → G matching phrase:

Original: For each h p in H and g q in G , the matching score m p → q is calculated as:
Revised: For each h p in H and g q in G , the matching score m p → q is calculated as:

Original: [Equation]
Revised: [Equation]

Original:  where 
Revised: q ) , where W BM ∈ R d CE × d CE is a bi-linear weight matrix. Similarly, the H ← G matching phrase considers how much each context g q ∈ G could be useful to h p ∈ H : m p ← q =

Original: [Equation]
Revised: [Equation]

Original: is a bi-linear weight matrix
Revised: q

Original: [Equation]
Revised: [Equation]

Original: Similarly, the H ← G matching phrase considers how much each context g q ∈ G could be useful to
Revised: Note that P × Q matching needs to be conducted in total for each entity pair. We write the equations for each h p ∈ H and g q ∈ G for clarity.

Original: [Equation]
Revised: While in terms of implementation, the bilateral matching can be easily written in a matrix form, where a matrix multiplication is used HW BM G T ∈ R P × Q , and the matching score matrix M can be obtained by taking softmax on the HW BM G T matrix over certain axis (over 0-axis for M p → q , 1-axis for M p ← q ).

Original: However, not all contexts are informative during the matching for two given entities.
Revised: Not all contexts are informative during the matching for two given entities.

Original: For example, some contexts may contain intricate contextual information even if they mention the entity explicitly.
Revised: For example, some contexts may contain intricate contextual information even if they mention the entity explicitly.

Original: In this work, we introduce a leaky unit during the bilateral matching, so that uninformative context can be routed via the leaky unit rather than forced to be matched with any informative encoded contexts.
Revised: In this work, we introduce a leaky unit during the bilateral matching, so that uninformative context can be routed via the leaky unit rather than forced to be matched with any informative encoded contexts.

Original: The leaky unit is a domain-dependent vector l ∈ R 1 × d CE learned with the model.
Revised: The leaky unit is a domain-dependent vector l ∈ R 1 × d CE learned with the model.

Original: For simplicity, we keep l as a zero vector.
Revised: For simplicity, we keep l as a zero vector.

Original: If we use the H → G matching phrase as an example, the matching score from leaky unit l to the q -th encoded context in g q is:
Revised: If we use the H → G matching phrase as an example, the matching score from leaky unit l to the q -th encoded context in g q is:

Original: [Equation]
Revised: [Equation]

Original: Then, if there is any uninformative context in H , say the ˜ p -th encoded context, h ˜ p will contribute less when matched with g q due to the leaky effect: when h ˜ p is less informative than the leaky unit l .
Revised: Then, if there is any uninformative context in H , say the ˜ p -th encoded context, h ˜ p will contribute less when matched with g q due to the leaky effect: when h ˜ p is less informative than the leaky unit l .

Original: [Equation]
Revised: [Equation]

Original: 2. C ONTEXT A GGREGATION
Revised: 2. C ONTEXT A GGREGATION

Original: The informativeness of a context for an entity should not be a ﬁxed value, it heavily depends on the other entity and the other entity’sown contexts that we are comparing with.
Revised: The informativeness of a context for an entity should not be a ﬁxed value: it heavily depends on the other entity and the other entity’s contexts that we are comparing with.

Original: The bilateral matching scores indicate the matching among multiple pieces of encoded contexts for two entities.
Revised: The bilateral matching scores indicate the matching among multiple pieces of encoded contexts for two entities.

Original: For each piece of encoded context, say g q for the entity k , we use the highest matched score with its counterpart as the relative informativeness score of g q to k , denote as a q = max( m p → q | p ∈ P ) .
Revised: For each piece of encoded context, say g q for the entity k , we use the highest matched score with its counterpart as the relative informativeness score of g q to k , denote as a q = max( m p → q | p ∈ P ) .

Original: Then, we aggregate multiple pieces of encoded contexts for each entity to a global context based on the relative informativeness 
Revised: Then, we aggregate multiple pieces of encoded contexts for each entity to a global context based on the relative informativeness scores: for entity e :

Original: [Equation]
Revised: [Equation]

Original: Note that due to the leaky unit, less informative contexts are not forced to be heavily involved during the aggregation: the leaky unit may be more competitive than contexts that are less informative, thus assigned with larger matching scores.
Revised: Note that due to the leaky unit, less informative contexts are not forced to be heavily involved during the aggregation: the leaky unit may be more competitive than contexts that are less informative, thus assigned with larger matching scores.

Original: However, as the leaky unit is not used for aggregation, scores on informative contexts become more salient for the context aggregation.
Revised: However, as the leaky unit is not used for aggregation, scores on informative contexts become more salient for the context aggregation.

Original: 2. T RAINING O BJECTIVES
Revised: 2. T RAINING O BJECTIVES

Original: We introduce two architectures for training the S YNONYM N ET : a siamese architecture and a triplet architecture.
Revised: We introduce two architectures for training the S YNONYM N ET : a siamese architecture and a triplet architecture.

Original: Siamese Architecture
Revised: Siamese Architecture

Original: The Siamese architecture takes two entities e and k , along with their contexts
Revised: The Siamese architecture takes two entities e and k , along with their contexts

Original: H and G as the input.
Revised: H and G as the input.

Original: The following loss function L Siamese is used in training for the Siamese architecture:
Revised: The following loss function L Siamese is used in training for the Siamese architecture:

Original: [Equation]
Revised: [Equation]

Original: where it contains losses for two cases: L + ( e, k ) when e and k are synonyms to each other ( y = 1 ), and L − ( e, k ) when e and k are not ( y = 0 ).
Revised: where it contains losses for two cases: L + ( e, k ) when e and k are synonyms to each other ( y = 1 ), and L − ( e, k ) when e and k are not ( y = 0 ).

Original: Speciﬁcally,  we have
Revised: Speciﬁcally, inspired by Neculoiu et al. (2016), we have

Original: [Equation]
Revised: [Equation]

Original: where s ( · ) is a similarity function, e.g. cosine similarity, and m is the margin value.
Revised: where s ( · ) is a similarity function, e.g. cosine similarity, and m is the margin value.

Revised: L + ( e, k ) decreases monotonically as the similarity score becomes higher within the range of [-1,1].

Revised: L + ( e, k ) =

Revised: when s ( ¯h , ¯g ) = 1 .

Revised: For L − ( e, k ) , it remains zero when s ( ¯h , ¯g ) is smaller than a margin m .

Revised: Otherwise L − ( e, k ) increases as s ( ¯h , ¯g ) becomes larger.

Original: Triplet Architecture
Revised: Triplet Architecture

Original: The Siamese loss makes the model assign rational pairs with absolute high scores and irrational ones with low scores, while the rationality of entity synonym could be quite relative to the context.
Revised: The Siamese loss makes the model assign rational pairs with absolute high scores and irrational ones with low scores, while the rationality of entity synonym could be quite relative to the context.

Original: The triplet architecture learns a metric such that the global context ¯ h of an entity e is relatively closer to a global context ¯g + of its synonym entity, say k + , than it is to the global context ¯g − of a negative example ¯g − by some margin value m .
Revised: The triplet architecture learns a metric such that the global context ¯ h of an entity e is relatively closer to a global context ¯g + of its synonym entity, say k + , than it is to the global context ¯g − of a negative example ¯g − by some margin value m .

Original: The following loss function
Revised: The following loss function

Original: L Triplet is used in training for the Triplet architecture:
Revised: L Triplet is used in training for the Triplet architecture:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: [Equation]

Original: 2. I NFERENCE
Revised: 2. I NFERENCE

Original: The objective of the inference phase is to discover synonym entities for a given query entity from the corpus effectively.
Revised: The objective of the inference phase is to discover synonym entities for a given query entity from the corpus effectively.

Original: We utilize context-aware word representation learning models to obtain candidate entities that narrow down the search space and the S YNONYM N ET veriﬁes entity synonym by assigning a synonym score for two entities based on multiple pieces of contexts.
Revised: We utilize context-aware word representation learning models to obtain candidate entities that narrow down the search space. The S YNONYM N ET veriﬁes entity synonym by assigning a synonym score for two entities based on multiple pieces of contexts.

Original: The overall framework is described in Figure 2.
Revised: The overall framework is described in Figure 2.

Original: [Figure]
Revised: [Figure]

Original: When given a query entity e , it is tediousand very ineffective to verify its synonymity with all the other possible entities. In the ﬁrst step, we train entity representation unsupervisely from the massive corpus D using methods such as skipgram (Mikolov et al., 2013) or GloVe (Pennington et al., 2014).
Revised: is tedious

Original: An embedding matrix can be learned W EMBED ∈ R v × d EMBED , where v is the number of unique tokens in D .
Revised: An embedding matrix can be learned W EMBED ∈ R v × d EMBED , where v is the number of unique tokens in D .

Original: Although these unsupervised methods utilize the context information, they learn a representation for each word without considering the order of context words. Thus they are not directly applicable to entity synonym discovery, but they do serve as a way to obtain candidates as they tend to give entities with similar neighboring  words similar representations.
Revised: Although these unsupervised methods utilize the context information to learn semantically meaningful representations for entities, they are not directly applicable to entity synonym discovery. However, they do serve as an effective way to obtain candidates as they tend to give entities with similar neighboring context words similar representations.

Original: For example, nba championship , chicago black hawks and american league championship series have similar representations because they tend to share some similar neighboring words.
Revised: For example, nba championship , chicago black hawks and american league championship series have similar representations because they tend to share some similar neighboring words.

Original: In the second step, we construct a candidate entity list E NN by ﬁnding nearest neighbors of a query entity e in the entity embedding space of R d EMBED .
Revised: In the second step, we construct a candidate entity list E NN by ﬁnding nearest neighbors of a query entity e in the entity embedding space of R d EMBED .

Original: Ranking entities by their proximities with the query entity on the entity embedding space signiﬁcantly narrows down the search space for synonym discovery.
Revised: Ranking entities by their proximities with the query entity on the entity embedding space signiﬁcantly narrows down the search space for synonym discovery.

Original: For each candidate entity e NN ∈ E NN and the query entity e , we collect multiple pieces of contexts in which they are mentioned respectively, and feed them into the proposed S YNONYM N ET model.
Revised: For each candidate entity e NN ∈ E NN and the query entity e , we randomly fetch multiple pieces of contexts in which entities are mentioned, and feed them into the proposed S YNONYM N ET model.

Original: S YNONYM N ET calculates a score s ( e, e NN ) based on multiple pieces of contexts retrieved for e and e NN in the corpus D via bilateral matching with leaky units.
Revised: S YNONYM N ET calculates a score s ( e, e NN ) based on multiple pieces of contexts retrieved for e and e NN in the corpus D via bilateral matching with leaky units.

Original: The candidate entity e NN is considered as a synonym to the query entity e when it receives a higher score s ( e, e NN ) than other non-synonym entities, orsimply exceeds a speciﬁc threshold.
Revised: The candidate entity e NN is considered as a synonym to the query entity e when it receives a higher score s ( e, e NN ) than other non-synonym entities, or exceeds a speciﬁc threshold.

Original: In appendix A, we provide pseudo codes for the synonym discovery using S YNONYM N ET .
Revised: In appendix A, we provide pseudo codes for the synonym discovery using S YNONYM N ET .

Original:  E XPERIMENTS 3. E XPERIMENT S ETUP
Revised:  E XPERIMENTS 3. E XPERIMENT S ETUP

Original: Datasets Three datasets are prepared to show the effectiveness of the proposed model on synonym discovery.
Revised: Datasets Three datasets are prepared to show the effectiveness of the proposed model on synonym discovery.

Original: The Wiki dataset contains 6.8M documents from Wikipedia 1 and the synonym entitiesare obtained from Freebase 2 .
Revised: The Wiki dataset contains 6.8M documents from Wikipedia 1 with generic synonym entities obtained from Freebase 2 .

Original: The PubMed is an English dataset wherewe collect 0.82M research paper abstracts  from PubMed 3 and UMLS 4 contains existing entity synonym information in the medical domain.
Revised: The PubMed is an English dataset where 0.82M research paper abstracts are collected from PubMed 3 and UMLS 4 contains existing entity synonym information in the medical domain.

Revised: The Wiki + FreeBase and PubMed + UMLS are public available datasets used in previous synonym discovery tasks (Qu et al., 2017).

Original: The MedBook is a Chinese dataset  where  0.51M pieces of contextsare collected from Chinese medical textbooks as well as online medical question answering forums.
Revised: The MedBook is a Chinese dataset collected by authors where we collect 0.51M pieces of contexts from Chinese medical textbooks as well as online medical question answering forums.

Original: Synonym entities  are obtained from MKG, a medical knowledge graph.
Revised: Synonym entities in the medical domain are obtained from MKG, a medical knowledge graph.

Original: Table 1 shows the dataset statistics.
Revised: Table 1 shows the dataset statistics.

Original: [Table]
Revised: [Table]

Original: PreprocessingFor Wiki +Freebase and PubMed + UMLS, we adopt the Stanford CoreNLP package to do the tokenization.
Revised: Preprocessing Wiki +Freebase and PubMed + UMLS come with entities and synonym entity annotations, we adopt the Stanford CoreNLP package to do the tokenization.

Original: For MedBook, a Chinese word segmentation tool Jieba 5 is used to segment the context.
Revised: For MedBook, a Chinese word segmentation tool Jieba 5 is used to segment the corpus into meaningful entities and phrases.

Original: We  ﬁlter out entity if they appear in the document less than 5 times.
Revised: We remove redundant contexts in the corpus and ﬁlter out entities if they appear in the corpus less than ﬁve times.

Revised: For entity representations, the proposed model works with various unsupervised word embedding methods.

Original: For entity representations, we adopt skip-gram (Mikolov et al., 2013) with a dimension of 200, context window is set to 5, and a negative sampling of 5 words is used.
Revised: Here for simplicity, we adopt skip-gram (Mikolov et al., 2013) with a dimension of 200. Context window is set as 5 with a negative sampling of 5 words for training.

Original: Evaluation Metric For synonym detection using S YNONYM N ET and other alternatives, we train the models with existing synonym and randomly sampled entity pairs as negative samples.
Revised: Evaluation Metric For synonym detection using S YNONYM N ET and other alternatives, we train the models with existing synonym and randomly sampled entity pairs as negative samples.

Original: During testing, we also sample random entity pairs as negative samples to evaluate the performance.
Revised: During testing, we also sample random entity pairs as negative samples to evaluate the performance.

Original: Note that all test synonym entities are from synsets that are never observed in the training data.
Revised: Note that all test synonym entities are from synsets that are never observed in the training data.

Original: Thus the evaluations are done in atotally cold-start setting.
Revised: Thus the evaluations are done in a cold-start setting.

Original: Area under the curve (AUC) and Mean Average Precision (MAP) are used to evaluate the model.
Revised: The area under the curve (AUC) and Mean Average Precision (MAP) are used to evaluate the model.

Original: AUC is used to measure how well the models assign high scores to synonym entities and low scores to non-synonym entities.
Revised: AUC is used to measure how well the models assign high scores to synonym entities and low scores to non-synonym entities.

Original: An AUC of 1 indicates that there is a clear boundary between scores of synonym entities and non-synonym entities.
Revised: An AUC of 1 indicates that there is a clear boundary between scores of synonym entities and non-synonym entities.

Revised: Additionally, a single-tailed t-test is conducted to evaluate the signiﬁcance of performance improvements when we compare the proposed S YNONYM N ET model with all the other baselines.

Original: For synonym discovery during the inference phase, we obtain candidate entities E NN from Knearest neighbors of the query entity in the entity embedding space, and rerank them based on the output score s ( e, e NN ) of the S YNONYM N ET for each e NN ∈ E NN .
Revised: For synonym discovery during the inference phase, we obtain candidate entities E NN from Knearest neighbors of the query entity in the entity embedding space, and rerank them based on the output score s ( e, e NN ) of the S YNONYM N ET for each e NN ∈ E NN .

Original: We expect candidate entities in the top positions are more likely to be synonym with the query entity.
Revised: We expect candidate entities in the top positions are more likely to be synonym with the query entity.

Original: We report the precision at position K (P@K), recall at  K (R@K), and F1 score at position K (F1@K).
Revised: We report the precision at position K (P@K), recall at position K (R@K), and F1 score at position K (F1@K).

Original: Baselines We compare the proposed model with the following alternatives.
Revised: Baselines We compare the proposed model with the following alternatives.

Original: (1) word2vec
Revised: (1) word2vec

Original: (Mikolov et al., 2013): a word embedding approach based on entity representations learned from the skipgram algorithm.
Revised: (Mikolov et al., 2013): a word embedding approach based on entity representations learned from the skipgram algorithm.

Original: We use the learned word embedding to train a classiﬁer for synonym discovery.
Revised: We use the learned word embedding to train a classiﬁer for synonym discovery.

Original: A scoring function Score D ( u, v ) =
Revised: A scoring function Score D ( u, v ) =

Original: x u W x Tv is used as the objective.
Revised: x u W x Tv is used as the objective.

Original: (2) GloVe (Pennington et al., 2014): another word embedding approach.
Revised: (2) GloVe (Pennington et al., 2014): another word embedding approach.

Original: The entity representations are learned based on the
Revised: The entity representations are learned based on the

Original: GloVe algorithm.
Revised: GloVe algorithm.

Original: The classiﬁer is trained with the same scoring function Score D , but with the learned glove embedding for synonym discovery.
Revised: The classiﬁer is trained with the same scoring function Score D , but with the learned glove embedding for synonym discovery.

Original: (3) SRN (Neculoiu et al., 2016): a character-level approach that uses a siamese multi-layer bi-directional recurrent neural networks to encode the entity as a sequence of characters.
Revised: (3) SRN (Neculoiu et al., 2016): a character-level approach that uses a siamese multi-layer bi-directional recurrent neural networks to encode the entity as a sequence of characters.

Original: The hidden states are averaged to get an entity representation.
Revised: The hidden states are averaged to get an entity representation.

Original: Cosine similarity is used in the objective.
Revised: Cosine similarity is used in the objective.

Original: (4) MaLSTM (Mueller & Thyagarajan, 2016): another characterlevel approach.
Revised: (4) MaLSTM (Mueller & Thyagarajan, 2016): another characterlevel approach.

Original: We adopt MaLSTM by feeding the character-level sequence to the model.
Revised: We adopt MaLSTM by feeding the character-level sequence to the model.

Original: Unlike
Revised: Unlike

Original: SRN that uses Bi-LSTM, MaLSTM uses a single direction LSTM and l -1 norm is used to measure the distance between two entities.
Revised: SRN that uses Bi-LSTM, MaLSTM uses a single direction LSTM and l -1 norm is used to measure the distance between two entities.

Original: (5) DPE (Qu et al., 2017): a model that utilizes dependency parsing results as  structured annotation on a single piece of context for synonym discovery.
Revised: (5) DPE (Qu et al., 2017): a model that utilizes dependency parsing results as the structured annotation on a single piece of context for synonym discovery.

Original: S YNONYM N ET is the proposed model, we used siamese loss (Eq. 6) and triplet loss (Eq. 7) as the objectives, respectively.
Revised: S YNONYM N ET is the proposed model, we used siamese loss (Eq. 6) and triplet loss (Eq. 7) as the objectives, respectively.

Original: 3. P ERFORMANCE E VALUATION
Revised: 3. P ERFORMANCE E VALUATION

Original: We ﬁrst apply random search to obtain the best-performing hyperparameter setting on the validation set.
Revised: We ﬁrst apply random search to obtain the best-performing hyperparameter setting on the validation set.

Original: The hyperparameter settings as well as the sensitivity analysis are reported in Appendix B.
Revised: The hyperparameter settings, as well as the sensitivity analysis, are reported in Appendix B.

Original: We report Area Under the Curve (AUC) and Mean Average Precision (MAP) on three datasets in Table 2.
Revised: We report Area Under the Curve (AUC) and Mean Average Precision (MAP) on three datasets in Table 2.

Original: From the upper part of Table 2 we can see that S YNONYM N ET performances consistently better than other baselines on three datasets.
Revised: From the upper part of Table 2 we can see that S YNONYM N ET performances consistently better than other baselines on three datasets.

Original: S YNONYM N ET with the triplet training objective achieves the best performance on Wiki +Freebase, while the siamese objective works better on PubMed + UMLS and MedBook + MKG.
Revised: S YNONYM N ET with the triplet training objective achieves the best performance on Wiki +Freebase, while the Siamese objective works better on PubMed + UMLS and MedBook + MKG.

Original: Among the baselines, word2vec is generally performing better than GloVe.
Revised: Among the baselines, word2vec is generally performing better than GloVe.

Original: SRNs achieve decent performance on PubMed + UMLS and MedBook + MKG.
Revised: SRNs achieve decent performance on PubMed + UMLS and MedBook + MKG.

Original: This is probably because the synonym entities obtained from the medical domain tend to share more character-level similarities, such as 6-aminohexanoic acid and aminocaproic acid .
Revised: This is probably because the synonym entities obtained from the medical domain tend to share more character-level similarities, such as 6-aminohexanoic acid and aminocaproic acid .

Original: However, even if the character-level features are not explicitly used in our model, our model still performances better, by effectively modeling multiple pieces of context information.
Revised: However, even if the character-level features are not explicitly used in our model, our model still performances better, by matching multiple pieces of contexts effectively.

Original: DPE has the best performance among other baselines, by annotating each piece of context with dependency parsing results.
Revised: DPE has the best performance among other baselines, by annotating each piece of context with dependency parsing results.

Original: However, the dependency parsing results could be error-prone for the synonym discovery task, especially when two entities share the similar usage but with different semantics, such as NBA finals and NFL playoffs .
Revised: However, the dependency parsing results could be error-prone for the synonym discovery task, especially when two entities share the similar usage but with different semantics, such as NBA finals and NFL playoffs .

Original: Additional results are reported in Appendix C.
Revised: Additional results are reported in Appendix C.

Revised: We conduct statistical signiﬁcance tests to validate performance improvement.

Revised: The single-tailed ttest is performed for all experiments, which measures whether or not the results from the proposed model are signiﬁcantly better than ones from baselines.

Revised: The numbers with † markers in Tableindicate that the improvement is signiﬁcant with p < 0.05.

Original: [Table]
Revised: [Table]

Original: Besides numeric metrics, we also use box plots to represent the score distributions for each method on all three datasets in Figure 3.
Revised: Besides numeric metrics [Table] we also use box plots to represent the score distributions for each method on all three datasets in Figure 3.

Original: The red bars indicate scores on positive entity pairs that are synonym with each other, while the blue bars indicate scores on negative entity pairs.
Revised: The red bars indicate scores on positive entity pairs that are synonym with each other, while the blue bars indicate scores on negative entity pairs.

Original: A general conclusion is that our model assigns higher scores for synonymous entity pairs,which is marginally higher than other non-synonym entity pairs, when compared with other alternatives.
Revised: A general conclusion is that our model assigns higher scores for synonym entity pairs, marginally higher than other nonsynonym entity pairs when compared with other alternatives.

Original: [Figure]
Revised: [Figure]

Original: 3. A BLATION S TUDY
Revised: 3. A BLATION S TUDY

Original: To study the contribution of different modules of S YNONYM N ET for synonym discovery, we also report ablation test results in the lower part of Table 2.
Revised: To study the contribution of different modules of S YNONYM N ET for synonym discovery, we also report ablation test results in the lower part of Table 2.

Original: “w/o Conﬂuence Context Encoder” uses the
Revised: “w/o Conﬂuence Context Encoder” uses the

Original: Bi-LSTM as the context encoder.
Revised: Bi-LSTM as the context encoder.

Original: The last hidden states in both forward and backward directions in Bi-LSTM are concatenated; “w/o Leaky Unit” is the model without the ability to ignore uninformative contexts during the bilateral matching process: all contexts retrieved based on the entity, whether informative or not, are utilized in bilateral matching.
Revised: The last hidden states in both forward and backward directions in Bi-LSTM are concatenated; “w/o Leaky Unit” does not have the ability to ignore uninformative contexts during the bilateral matching process: all contexts retrieved based on the entity, whether informative or not, are utilized in bilateral matching.

Original: From the lower part of Table 2 we can see that both modules (Leaky Unit and Conﬂuence Encoder) contribute to the effectiveness of the model.
Revised: From the lower part of Table 2 we can see that both modules (Leaky Unit and Conﬂuence Encoder) contribute to the effectiveness of the model.

Original: The leaky unit is more effective when trained with the triplet objective, contributing 1.72% improvement in AUC and 2.61% improvement in MAP on the Wiki dataset.
Revised: The leaky unit contributes 1.72% improvement in AUC and 2.61% improvement in MAP on the Wiki dataset when trained with the triplet objective.

Original: The Conﬂuence Encoder gives the model an average of 3.17% improvement in AUC on all three datasets, and up to 5.17% improvement in MAP.
Revised: The Conﬂuence Encoder gives the model an average of 3.17% improvement in AUC on all three datasets, and up to 5.17% improvement in MAP.

Original:  R ELATED WORKS 4. S YNONYM D ISCOVERY
Revised:  R ELATED WORKS 4. S YNONYM D ISCOVERY

Original: The synonym discovery focuses on detecting entity synonyms.
Revised: The synonym discovery focuses on detecting entity synonyms.

Original: It can be seen as a task of learning the semantic similarities between entities.

Original: Most existing works try to achieve this goal by learning from structured information such the query logs (Ren & Cheng, 2015; Chaudhuri et al., 2009; Wei et al., 2009).
Revised: Most existing works try to achieve this goal by learning from structured information such as query logs (Ren & Cheng, 2015; Chaudhuri et al., 2009; Wei et al., 2009).

Original: While in this work, we focus on synonym discovery from free-text natural language contexts, which requires less annotation and is more challenging.
Revised: While in this work, we focus on synonym discovery from free-text natural language contexts, which requires less annotation and is more challenging.

Original: Some existing works try to detect entity synonyms by entity-level similarities (Lin et al., 2003; Roller et al., 2014; Neculoiu et al., 2016; Wieting et al., 2016).
Revised: Some existing works try to detect entity synonyms by entity-level similarities (Lin et al., 2003; Roller et al., 2014; Neculoiu et al., 2016; Wieting et al., 2016).

Original: For example, Roller et al.
Revised: For example, Roller et al.

Original: (2014) introduce distributional features for hypernymy detection.
Revised: (2014) introduce distributional features for hypernym detection.

Original: Neculoiu et al.
Revised: Neculoiu et al.

Original: (2016) use a Siamese structure that treats each entity as a sequence of characters, and uses a Bi-LSTM to encode the entity information.
Revised: (2016) use a Siamese structure that treats each entity as a sequence of characters, and uses a Bi-LSTM to encode the entity information.

Original: Such approach may be helpful for synonyms with similar spellings, or dealing with abbreviations.
Revised: Such approach may be helpful for synonyms with similar spellings, or dealing with abbreviations.

Original: Without considering the context information, it is hard for the aforementioned methods to infer synonyms that share similar semantics but are not alike verbatim, such as JD and law degree .
Revised: Without considering the context information, it is hard for the aforementioned methods to infer synonyms that share similar semantics but are not alike verbatim, such as JD and law degree .

Original: Various approaches (Snow et al., 2005; Sun & Grishman, 2010; Liao et al., 2017; Cambria et al., 2018) are proposed to incorporate context information to characterize entity mentions.
Revised: Various approaches (Snow et al., 2005; Sun & Grishman, 2010; Liao et al., 2017; Cambria et al., 2018) are proposed to incorporate context information to characterize entity mentions.

Original: Patty (Nakashole et al., 2012) introduce a pattern-based approach for entity extraction.
Revised: However, these models are not designed for synonym discovery.

Original: Qu et al.
Revised: Qu et al.

Original: (2017) propose to have additional annotation information, e.g. dependency parsing result, as the context of the entity.
Revised: (2017) utilize additional structured annotations, e.g. dependency parsing result, as the context of the entity for synonym discovery.

Original: Cambria et al. (2018) model the left and right contexts around an entity to discover conceptual primitives. However, these methods do not consider multiple pieces of contexts in which an entity may occur due to diverse expressions. Only one piece of context may possess strong modeling biases, which leads to less robustness and deteriorated performance.
Revised: While we aim to discover synonym entities from a free-text corpus without structured annotation.

Original: 4. S ENTENCE M ATCHING
Revised: 4. S ENTENCE M ATCHING

Original: There is another related research area that studies sentence matching.
Revised: There is another related research area that studies sentence matching.

Original: Early works try to learn a meaningful single vector to represent the sentence (Tan et al., 2015; Mueller & Thyagarajan, 2016).
Revised: Early works try to learn a meaningful single vector to represent the sentence (Tan et al., 2015; Mueller & Thyagarajan, 2016).

Original: These models do not consider the word-level interactions from two sentences during the matching.
Revised: These models do not consider the word-level interactions from two sentences during the matching.

Original: Wang & Jiang (2016); Wang et al.
Revised: Wang & Jiang (2016); Wang et al.

Original: (2016; 2017) introduce multiple instances for matching with varying granularities.
Revised: (2016; 2017) introduce multiple instances for matching with varying granularities.

Original: Although the above methods achieve decent performance on sentence-level matching, the sentence matching task is different from context modeling for synonym discovery in essence.
Revised: Although the above methods achieve decent performance on sentence-level matching, the sentence matching task is different from context modeling for synonym discovery in essence.

Original: Context matching focuses on local information, especially the words before and after the entity word; while the overall sentence could contain much more information, which is useful to represent the sentence-level semantics, but can be quite noisy for context modeling.
Revised: Context matching focuses on local information, especially the words before and after the entity word; while the overall sentence could contain much more information, which is useful to represent the sentence-level semantics, but can be quite noisy for context modeling.

Original: We adopt a conﬂuence encoder to model the context, which is able to aware of the location of an entity in the context while preserving information ﬂow from both left and right contexts.
Revised: We adopt a conﬂuence encoder to model the context, which is able to aware of the location of an entity in the context while preserving information ﬂow from both left and right contexts.

Original: Moreover, sentence matching models do not explicitly deal with uninformative instances:only maxpooling strategy or attention mechanism is introduced.
Revised: Moreover, sentence matching models do not explicitly deal with uninformative instances: maxpooling strategy and attention mechanism are introduced.

Original: The max-pooling strategy picks the most informative one and ignores all the other less informative ones.
Revised: The max-pooling strategy picks the most informative one and ignores all the other less informative ones.

Original: In context matching, such property could be unsatisfactory as an entity is usually associated with multiple contexts.
Revised: In context matching, such property could be unsatisfactory as an entity is usually associated with multiple contexts.

Original: We adopt a bilateral matching which involves a leaky unit to explicitly deal with uninformative contexts, so as to eliminate noisy contexts while preserving the expression diversity from multiple pieces of contexts.
Revised: We adopt a bilateral matching which involves a leaky unit to explicitly deal with uninformative contexts, so as to eliminate noisy contexts while preserving the expression diversity from multiple pieces of contexts.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this paper, we propose a framework for synonym discovery from free-text corpus with minimized human annotation.
Revised: In this paper, we propose a framework for synonym discovery from free-text corpus without structured annotation.

Original: A novel neural network model S YNONYM N ET is introduced for synonym detection, which tries to determine whether or not two given entities are synonym with each other.
Revised: A novel neural network model S YNONYM N ET is introduced for synonym detection, which tries to determine whether or not two given entities are synonym with each other.

Original: S YNONYM N ET makes use of multiple pieces of contexts in which the entity is mentioned, and compared the context-level similarity via a bilateral matching schema to determine synonymity.
Revised: S YNONYM N ET makes use of multiple pieces of contexts in which each entity is mentioned, and compares the context-level similarity via a bilateral matching schema to determine synonymity.

Original: Experiments on three real-world datasets show that the proposed method S YNONYM N ET has the ability to discover synonym entities effectively on both generic datasets (Wiki+Freebase), as well as domain-speciﬁc datasets (PubMed+UMLS  and MedBook+MKG) with an improvement up to 4.16% in AUC and 3.19% in MAP.
Revised: Experiments on three real-world datasets show that the proposed method S YNONYM N ET has the ability to discover synonym entities effectively on both generic datasets (Wiki+Freebase in English), as well as domain-speciﬁc datasets (PubMed+UMLS in English and MedBook+MKG in Chinese) with an improvement up to 4.16% in AUC and 3.19% in MAP.

Original: APPENDIX A P SEUDO C ODE
Revised: APPENDIX A P SEUDO C ODE

Original: Here we provide pseudo codes for the synonym discovery using S YNONYM N ET .
Revised: Here we provide pseudo codes for the synonym discovery using S YNONYM N ET .

Original: Data: Candidate entity e , Entity Word Embeddings W EMBED ∈
Revised: Data: Candidate entity e , Entity Word Embeddings W EMBED ∈

Original: R v × d , Document D
Revised: R v × d , Document D

Original: Result: Entity Set K where each k ∈ K is a synonym entity of e
Revised: Result: Entity Set K where each k ∈ K is a synonym entity of e

Original: E NN =
Revised: E NN =

Original: NearestNeighbor( e , W EMBED )
Revised: NearestNeighbor( e , W EMBED )

Original: Order E NN by the distance to e ; for e NN in E NN do Retrieve Contexts for e NN from Document D ;
Revised: Order E NN by the distance to e ; for e NN in E NN do Retrieve Contexts for e NN from Document D ;

Original: Apply S YNONYM N ET on e and e NN ; if s ( e, e NN ) > threshold then
Revised: Apply S YNONYM N ET on e and e NN ; if s ( e, e NN ) > threshold then

Original: Add e NN as a synonym of e to K ; end end
Revised: Add e NN as a synonym of e to K ; end end

Original: B H YPERPARAMETERS
Revised: B H YPERPARAMETERS

Original: We train the proposed model with a wide range of hyperparameter conﬁgurations, which are listed in Table 3.
Revised: We train the proposed model with a wide range of hyperparameter conﬁgurations, which are listed in Table 3.

Original: For the model architecture, we vary the number of  contexts P  for each entity from 1 to 20.
Revised: For the model architecture, we vary the number of randomly sampled contexts P = Q for each entity from 1 to 20.

Original: Each piece of context is chunked by a maximum length of T .
Revised: Each piece of context is chunked by a maximum length of T .

Original: For the conﬂuence context encoder, we vary the hidden dimension d CE from 8 to 1024.
Revised: For the conﬂuence context encoder, we vary the hidden dimension d CE from 8 to 1024.

Original: The margin value m in triplet loss function is varied from 0.1 to 1.75.
Revised: The margin value m in triplet loss function is varied from 0.1 to 1.75.

Original: For the training, we try different optimizers (Adam (Kingma & Ba, 2014), RMSProp (Tieleman & Hinton, 2012), adadelta (Zeiler, 2012), and Adagrad (Duchi et al., 2011)), with the learning rate varying from 0.0003 to 0.01.
Revised: For the training, we try different optimizers (Adam (Kingma & Ba, 2014), RMSProp (Tieleman & Hinton, 2012), adadelta (Zeiler, 2012), and Adagrad (Duchi et al., 2011)), with the learning rate varying from 0.0003 to 0.01.

Original: Different batch sizes are used to train the model.
Revised: Different batch sizes are used to train the model.

Original: HYPERPARAMETERS VALUE
Revised: HYPERPARAMETERS VALUE

Original: [Table] [Table] number) { 1, 3, 5, 10, 15, 20 }T (
Revised: P (context number) { 1, 3, 5, 10, 15, 20 }

Original: [Table]

Original:  length) { 10, 30, 50, 80 } [Table] [Table] [Table]
Revised: T (maximum context length) { 10, 30, 50, 80 } d CE (layer size) { 8, 16, 32, 64, 128, 256, 512, 1024 } m (margin) { 0.1, 0.25, 0.5, 0.75, 1.25, 1.5, 1.75 }

Original: } m

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: Optimizer { Adam, RMSProp, Adadelta
Revised: Optimizer { Adam, RMSProp

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Furthermore
Revised: Furthermore

Original: [Table]
Revised: [Table]

Original: we provide sensitivity analysis of the proposed model with different hyperparameters in Wiki + Freebase dataset in Figure 4.
Revised: we provide sensitivity analysis of the proposed model with different hyperparameters in Wiki + Freebase dataset in Figure 4.

Original: We ﬁrst apply random search to obtain the best-performing hyperparameter setting on the validation dataset, as shown in Table 4.
Revised: We ﬁrst apply random search to obtain the best-performing hyperparameter setting on the validation dataset, as shown in Table 4.

Original: Figure 4 shows the performance curves when we vary one hyperparameter while keeping the remaining ﬁxed.
Revised: Figure 4 shows the performance curves when we vary one hyperparameter while keeping the remaining ﬁxed.

Original: As the number of contexts P increases, the model generally performs better.
Revised: As the number of contexts P increases, the model generally performs better.

Revised: Due to limitations on computing resources, we are only able to verify the performance of up to 20 pieces of randomly sampled contexts

Original:  The model achieves the best AUC and MAP when the maximum context length T = 50 : longer contexts may introduce too much noise while shorter contexts may be less informative.
Revised: [Table] The model achieves the best AUC and MAP when the maximum context length T = 50 : longer contexts may introduce too much noise while shorter contexts may be less informative.

Original: A margin m of 0.75 is used for training with triplet objective.
Revised: A margin m of 0.75 is used for training with triplet objective.

Original: [Figure]
Revised: [Figure]

Original: C C ASE S TUDIES
Revised: C C ASE S TUDIES

Original: We provide additional results and case studies in this section.
Revised: We provide additional results and case studies in this section.

Original: Table 5 reports the performance in
Revised: Table 5 reports the performance in

Original: P @ K, R @
Revised: P @ K, R @

Original: K, and F

Revised: K, and F

Original: K. Table 6 shows a case for entity UNGA.
Revised: K. Table 6 shows a case for entity UNGA.

Original: The candidates are generated with pretrained word embedding using skip-gram.
Revised: The candidates are generated with pretrained word embedding using skip-gram.

Original: Wiki + Freebase
Revised: Wiki + Freebase

Original: PubMed + UMLS
Revised: PubMed + UMLS

Original: MedBook + MedKG
Revised: MedBook + MedKG

Original: P@K
Revised: P@K

Original: R@K
Revised: R@K

Original: F1@K
Revised: F1@K

Original: P@K
Revised: P@K

Original: R@K
Revised: R@K

Original: F1@K
Revised: F1@K

Original: P@K
Revised: P@K

Original: R@K
Revised: R@K

Original: F1@K
Revised: F1@K

Original: K= 0.
Revised: K= 0.

Original: K= 0. K= 0.
Revised: K= 0. K= 0.

Original: [Table]
Revised: [Table]