S yn S et E xpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discovery (2020)

Abstract: Entity set expansion and synonym discovery are two critical NLP tasks. Previous studies accomplish them separately, without exploring their interdependencies. In this work, we hypothesize that these two tasks are tightly coupled because two synonymous entities tend to have a similar likelihood of belonging to various semantic classes. This motivates us to design SynSetExpan, a novel framework that enables two tasks to mutually enhance each other. SynSetExpan uses a synonym discovery model to include popular entities’ infrequent synonyms into the set, which boosts the set expansion recall. Meanwhile, the set expansion model, being able to determine whether an entity belongs to a semantic class, can generate pseudo training data to fine-tune the synonym discovery model towards better accuracy. To facilitate the research on studying the interplays of these two tasks, we create the first large-scale Synonym-Enhanced Set Expansion (SE2) dataset via crowdsourcing. Extensive experiments on the SE2 dataset and previous benchmarks demonstrate the effectiveness of SynSetExpan for both entity set expansion and synonym discovery tasks.

## introduction entity set expansion (ese) aims to expand a small set of seed entities (e.g., {"united states", "canada"}) into a larger set of entities that belong to the same semantic class (i.e., country). entity synonym discovery (esd) intends to group all terms in a vocabulary that refer to the same realworld entity (e.g., "america" and "usa" refer to the same country) into a synonym set (hence called a synset). those discovered entities and synsets include rich knowledge and can benefit many downstream applications such as semantic search @xcite synsetexpan framework illinois il land of lincoln texas tx lone star state california ca golden state user provided et al., 2017), taxonomy construction @xcite , and online education @xcite . previous studies regard ese and esd as two independent tasks. many ese methods @xcite @xcite @xcite are developed to iteratively select and add the most confident entities into the set. a core challenge for ese is to find those infrequent long-tail entities in the target semantic class (e.g., "lone star state" in the class us_states) while filtering out false positive entities from other related classes (e.g., "austin" and "dallas" in the class city) as they will cause semantic shift to the set. meanwhile, various esd methods @xcite @xcite combine stringlevel features with embedding features to find a query term's synonyms from a given vocabulary or to cluster all vocabulary terms into synsets. a major challenge here is to combine those features with limited supervisions in a way that works for entities from all semantic classes. another challenge is how to scale a esd method to a large, extensive vocabulary that contains terms of varied qualities. to address the above challenges, we hypothesize that ese and esd are two tightly coupled tasks and can mutually enhance each other because two synonymous entities tend to have similar likelihoods of belonging to various semantic classes and vice versa. this hypothesis implies that (1) knowing the class membership of one entity enables us to infer the class membership of all its synonyms, and (2) two entities can be synonyms only if they belong to the same semantic class. for example, we may expand the us_states class from a seed set {"illinois", "texas", "california"}. an ese model can find frequent state full names (e.g., "wisconsin", "connecticut") but may miss those infrequent entities (e.g., "lone star state" and "golden state"). however, an esd model may predict "lone star state" is the synonym of "texas" and "golden state" is synonymous to "california" and directly adds them into the expanded set, which shows synonym information help set expansion. meanwhile, from the ese model outputs, we may infer "wisconsin", "wi" is a synonymous pair while "connecticut", "sc" is not, and use them to fine-tune an esd model on the fly. this relieves the burden of using one single esd model for all semantic classes and improves the esd model's inference efficiency because we refine the synonym search space from the entire vocabulary to only the ese model outputs. in this study, we propose synsetexpan, a novel framework jointly conducting two tasks (cf. fig. 1 ). to better leverage the limited supervision signals in seeds, we design synsetexpan as an iterative framework consisting of two components: (1) a ese model that ranks entities based on their probabilities of belonging to the target semantic class, and (2) a esd model that returns the probability of two entities being synonyms. in each iteration, we first apply the ese model to obtain an entity rank list from which we derive a set of pseudo training data to fine-tune the esd model. then, we use this fine-tuned model to find synonyms of entities in the currently expanded set and adjust the above rank list. finally, we add top-ranked entities in the adjusted rank list into the currently expanded set and start the next iteration. after the iterative process ends, we construct a synonym graph from the last iteration's output and extract entity synsets (including singleton synsets) as graph communities. as previous ese datasets are too small and contain no synonym information for evaluating our hypothesis, we create the first synonym enhanced set expansion (se2) benchmark dataset via crowdsourcing. this new dataset foot_0 is one magnitude larger than previous benchmarks. it contains a corpus of the entire wikipedia, a vocabulary of 1.5 million terms, and 1200 seed queries from 60 semantic classes of 6 different types (e.g., person, location, organization, etc.). contributions. in summary, this study makes the following contributions: (1) we hypothesize that ese and esd can mutually enhance each other and propose a novel framework synsetexpan to jointly conduct two tasks; (2) we construct a new large-scale dataset se2 that supports fair comparison across different methods and facilitates future research on both tasks; and (3) we conduct extensive experiments to verify our hypothesis and show the effectiveness of synsetexpan on both tasks. ## problem formulation we first introduce important concepts in this work, and then present our problem formulation. a term is a string (i.e., a word or a phrase) that refers to a real-world entity foot_1 . an entity synset is a set of terms that can be used to refer to the same realworld entity. for example, both "usa" and "america" can refer to the entity united states and thus compose an entity synset. we allow the singleton synset and a term may locate in multiple synsets due to its ambiguity. a semantic class is a set of entities that share a common characteristic and a vocabulary is a term list that can be either provided by users or derived from a corpus. problem formulation. given (1) a text corpus d, (2) a vocabulary v derived from d, and (3) a seed set of user-provided entity synonym sets s 0 that belong to the same semantic class c, we aim to (1) select a subset of entities v c from v that all belong to c; and (2) clusters all terms in v c into entity synsets s v c where the union of all clusters is equal to v c . in other words, we expand the seed set s 0 into a more complete set of entity synsets s 0 ∪ s v c that belong to the same semantic class c. a concrete example is presented in fig. 1 . ## the synsetexpan framework in this study, we hypothesize that entity set expansion and synonym discovery are two tightly coupled tasks and can mutually enhance each other. hypothesis 1. two synonymous entities tend to have similar likelihoods of belonging to various semantic classes and vice versa. the above hypothesis has two implications. first, if two entities e i and e j are synonyms and vocabulary v score 0.98 0.62 0.58 …… …… 0.91 0.94 0.57 0.78 0.85 entity …… lone star state golden state south korea …… chicago sc wi south carolina wisconsin … 6 1 3 200 2 5 … 4 rank 201 7 … south korea chicago … land of lincoln sc wi golden state lone star state south carolina wisconsin entity string match final merged entity rank list e i belongs to semantic class c, e j likely also belongs to class c even if it is currently outside c. this reveals how synonym information can help set expansion by directly introducing popular entities' infrequent synonyms into the set and thus increasing the expansion recall. the second implication is that if two entities are not from the same class c, then they are likely not synonyms. this shows how set expansion can help synonym discovery by restricting the synonym search space to set expansion outputs and generating additional training data to fine tune the synonym discovery model. at the beginning, when we only have limited seed information, this hypothesis may not be directly applicable as we do not have complete knowledge of either entity class memberships or entity synonyms. therefore, we design our synsetexpan as an iterative framework, shown in fig. 2 . framework overview. before the iterative process starts, we first learn a general synonym discovery model m 0 using distant supervision from a knowledge base (cf. sect. 3.1). then, in each iteration, we learn a set expansion model based on the currently expanded set e (initialized as all entities in user-provided seed synsets s 0 ) and apply it to obtain a rank list of entities in v, denoted as l se (cf. sect. 3.2). next, we generate pseudo training data from l se and use it to construct a new class-dependent synonym discovery model m c by fine-tuning m 0 . after that, for each entity in v, we apply m c to predict its probability of being the synonym of at least one entity in e and use such synonym information to adjust l se (cf. sect. 3.3). finally, we add top-ranked entities in the adjusted rank list into the current set and start the next itera-tion. after the iterative process ends, we identify entity synsets from the final iteration's output using a graph-based clustering method (cf. sect. 3.4). ## the se2 dataset to verify our hypothesis and evaluate the synsetexpan framework, we need a dataset that contains a corpus, a vocabulary with labeled synsets, a set of complete semantic classes, and a list of seed queries. however, to the best of our knowledge, there is no such a public benchmark dataset foot_3 . therefore, we build the first synonym enhanced set expansion (se2) benchmark dataset in this study foot_4 . ## conclusions this paper shows entity set expansion and synonym discovery are two tightly coupled tasks and can mutually enhance each other. we present synsetexpan, a novel framework jointly conducting two tasks, and se2 dataset, the first large-scale synonym-enhanced set expansion dataset. extensive experiments on se2 and several other benchmark datasets demonstrate the effectiveness of synsetexpan on both tasks. in the future, we plan to study how we can apply synsetexpan at the entity mention level for conducting contextualized synonym discovery and set expansion.