Predicting Institution Hierarchies with Set-based Models

ence research as well as in tools that track the research achievements and output . However , this structure is not consistently documented for all institutions in the world , motivating the need for automated construction methods . In this paper , we present a new task and model for predicting sub-institution/super-institution relationships based on their string names . The crux of our model is that it leverages learned , permutation invariant representations of various token subsets of institution name strings . Our model outperforms or matches non-set-based models and baselines . We also create a dataset for training and evaluating models for this task based on the publicly available relationships in the Global Research Identifier Database . 1 . Introduction . Academic , government and various industrial organizations are often hierarchically structured [ Fan et al. , 2012 ] . For example , the Canadian Institute for Theoretical Astrophysics is a sub-institution of University of Toronto and Harvard University is the super-institution of both Harvard Medical School and Harvard Divinity School . Science of science research focuses on interactions among various scientific agents ( researchers , academic institutions , etc . ) with the goal of developing tools and policies for accelerating science [ Fortunato et al. , 2018 ] . Institution hierarchies are used in the analysis of the funding of institutions and , when accompanied with temporal information , organizational hierarchies can also provide insights into the evolution / growth of institutions as well as fine-grained information about the mobility of researchers , capturing department and career focus changes [ Azoulay et al. , 2011 , Stephan , 2012 ] . Thus , correctly modeling organizational hierarchies is critical for tool development in science of science and other institutional research . The Global Research Identifier Database ( GRID ) is a resource meant to support such efforts . GRID is carefully curated and contains detailed information about research institutions , includ- ing their sub-institution/super-institution relationships . While the database is large and growing ( containing almost 100k institutions at the time of writing ) , there are institutions and relationships that are missing . For example , Montreal Institute for Learning Algorithms is not present in GRID , and the public hospital , John Hunter Children ’ s Hospital , in New South Wales is not connected to the Government of New South Wales nor Australia ’ s Government / healthcare system . Similarly , fine grained information , such as particular research laboratories or smaller departments within universities ’ colleges , are often absent . Relation extraction methods are typically used to create knowledge-bases ( KBs ) containing facts such as which institutions are parents/children of one another . OpenIE [ Etzioni et al. , 2011 , Fader et al. , 2011 , inter-alia ] and Universal Schema [ Riedel et al. , 2013 , Verga et al. , 2016 , Das et al. , 2017 , inter-alia ] , are able to extract complex relationship between entities from text . These methods are either based on link-prediction techniques or use rich contextual information to make accurate predictions . Unfortunately , such context is often unavailable . For example , funding agencies listed in research papers , assignees mentioned in patents , and author/inventor affiliations included in articles and BibTeX entries regularly appear as strings with little to no context relevant to making predictions about organization hierarchies . Access to a KB of institutions names and other features may be available , but these KBs often miss hierarchical relationships between those institutions . In this paper , we explore methods for predicting hierarchical relationships between institutions based solely on the spelling of the institution names and knowledge-base features . While some sub-institution/super-institution relationships can be predicted using simple token overlap between institution strings ( e.g. , Harvard Medical School sub-institution-of Harvard University ) , others require methods which can perform richer semantic understanding beyond simple token matching ( e.g. , Centre d ’ Investigation Clinique Pierre Drouin , Vandœuvre-lès-Nancy , France sub-institution-of Centre Hospitalier Universitaire de Nancy , Nancy , France ) . Predicting sub-institution/super-institution relationships without additional context is important when trying to complete KBs such as GRID without intensive data gathering for contextual evidence . We present a model that predicts sub-institution-of relationships using institution names , locations , and types . Inspired by the intuition that set-intersections and set-differences among the tokens of two institution names provide significant signal as to their relationship , our model learns to combine permutation invariant representations of token subets derived from institution names . Operating over embedded representations improves generalizability in comparison to techniques that operate on discrete tokens . In this work , we make the following 3 contributions : ( 1 ) we present a new set-based model that outperforms non-set-based models on sub-institution/super-institution relationship prediction , ( 2 ) we examine the difficulty of the task and show when set-based models work well , and ( 3 ) we create and release a new dataset derived from GRID to support sub-/super-institution relationship prediction . 2 . Background . 2.1 Dataset & Preprocessing . The Global Research Identifier Database ( GRID ) , contains information about 91,640 institutions from 217 countries1 . The institutions include universities , government agencies , companies , health 1 . We use the 2018-11-14 Release . care organizations and other places of research . GRID stores the name , location , and type ( university , hospital , government agency , etc ) , as well as the parent of each institution . The database includes 11,393 instances of the sub-institution relationship . Our goal is to develop models that predict the hierarchical structure of institutions . We use GRID to derive a new dataset that contains pairs of institutions that participate in the sub-institution relationship ( i.e. , positive examples ) and pairs of institutions that do not ( i.e. , negative examples ) . Constructing our dataset begins by building a graph where each node represents a GRID institution , and each edge represents a parent-child relationship among an institution pair . We partition the child-parent edges into a train/dev/test split by randomly assigning each connected component of institutions and all of its edges to either the train , development or test set ( using a 0.6/0.2/0.2 split ) . We construct negative examples in three ways : ( 1 ) non-sub-institutions in the same connected component , ( 2 ) non-sub-institutions in the same city in a different connected component , and ( 3 ) randomly sampled pairs . Figure 1 visualizes a connected component from GRID . Due to its overwhelming size ( 2981 sub-institutions ) with respect to other institutions in GRID , we exclude the connected component representing the United States Government from the dataset . 2.2 Institution Hierarchy Prediction . Let A and B be two institutions in GRID . Then , A sub-institution-of B , if A is a descendant of B according to GRID . In institution hierarchy prediction , we are given two GRID institutions , A and B , which includes their names , locations , and types , and asked to predict if the A sub-institution-of B . 3 . Set-based Models for Predicting Institution Hierarchies . In this section , we present a model for predicting whether one institution is a sub-institution of another . Our model makes use of Set Transformers [ Lee et al. , 2019 ] to model the token overlap between the institution names . 3.1 Sets of Institution Tokens . When predicting whether one institution is a sub-institution of another , the collection of tokens that appear in both institution names constitutes a strong signal . For example , consider the pair of institutions University of Pennsylvania and The Wharton School of the University of Pennsylvania . In this case , one institution name is a substring of the other , suggesting that in such cases the substring-institution name is a super-institution of the other . However , this heuristic is far too strict . For example , consider the institution pair The Ohio State University sub-institution-of University System of Ohio , where the substring relationship does not hold . In this example , we notice that the two strings share the tokens { Ohio , University } , though the tokens appear in different orders in the strings . This suggests treating institution names as unordered sets of tokens rather than ordered strings . Additionally , the most salient tokens appear in both institution names and less relevant tokens do not . This suggests that it is more likely for some tokens to appear in both names if two institutions participate in the sub-institution-of relationship than others . We present a model inspired by these insights , which is designed to predict whether one institution is a sub-institution of another . The model takes an ordered pair of institution names as input , ( s1 , s2 ) , and outputs a score : h ( s1 , s2 ) , with larger scores corresponding to greater model confidence in s1 sub-institution-of s2 . We use ts be the tokens of a string s. The model is a function of four sets , each of which can be built from the two institution names : ( 1 ) the first institution represented as a set of tokens ( ts1 ) , ( 2 ) the second institution represented as a set of tokens ( ts2 ) , ( 3 ) the intersection between the first and second institution ( ts1 ∩ ts2 ) , and ( 4 ) the set difference between the second institution and the first , ( s2 \ s1 ) . Let f : 2V → Rd be a function that returns the embedded representation of a token set , where V is the vocabulary defined by our model . Define the model , h , evaluated on two string , s1 and s2 , as follows : h ( s1 , s2 ) = 〈f ( ts2 ) , f ( ts2 ∩ ts1 ) 〉 − 〈f ( ts1 ) , f ( ts2 ∩ ts1 ) 〉 − 〈f ( ts2 ) , f ( ts2 \ ts1〉 ) ( 1 ) where 〈· , ·〉 is the inner product of the two vectors . The first term in Equation 1 computes the dot product between s2 , the candidate super-institution , and the intersection of the two institutions strings . As previously discussed , if all tokens of s2 appear in ts1 ∩ ts2 , then it is likely that s1 sub-institution-of s2 . The second term encodes a dual effect : if the embedded representation of s2 , i.e. , f ( t2 ) is similar to the embedding of the intersection of the tokens of s1 and s2 , i.e. , f ( ts1 ∩ ts2 ) , then s2 is likely a super-institution of s1 ( captured in the subtraction of the second term from the first ) . Together , the first two terms encode the intuition that children institutions tend to exhibit a higher degree of specificity than their parents . Finally , the third term in the model compares the tokens in s2 ( including tokens in the intersection ) to the tokens which are unique to s2 . If there is large overlap between these sets , it indicates that s2 is significantly different from s1 , i.e. , s1 is unlikely to be a sub-institution of s2 . The third term in Equation 1 compares the tokens in s2 ( and potentially in the intersection ) to the tokens which are unique to s2 . A large overlap between these sets indicates that s2 is sufficiently different from s1 , i.e. , s1 is unlikely to be a sub-institution of s2 . Figure 2 visualizes our model . Any set encoder function may be used for f . Our choice of this function is discussed further in Section 3.2 .

Summary: The paper shows how to infer the organisational structure of an institution. That is, it presents a model for predicting the is-ancestor relationships of institutions based on their string names. To this end, it makes use of Set-Transformers to model the token overlap between the institution names. This use is nice but also not highly original. The experimental evaluation is on a single dataset only. While the authors do present some examples, and overall hierarchy or something that provides some more insights into the learned model should be provided in order to show potential issues with transitivity and connected components. The evaluation only considers known pairs. But an organisational structure should also be consistent.  That is, the interesting motivation provided in the intro is not met in the experimental evaluation. Furthermore, the experimental protocol is  unclear. It seems to be a single training/test split, although one should consider several random splits or even some form of cross-validation, i.e., over the connected components given.