Equivariant Entity-Relationship Networks

1 INTRODUCTION . In the relational model of data , we have a set of entities , and one or more instances of each entity . These instances interact with each other through a set of fixed relations between entities . A set of attributes may be associated with each type of entity and relation.1 This simple idea is widely used to represent data , often in the form of a relational database , across a variety of domains , from shopping records , social networking data , and health records , to heterogeneous data from astronomical surveys . Learning and inference on relational data has been the topic of research in machine learning over the past decades . The relational model is closely related to first order and predicate logic , where the existence of a relation between instances becomes a truth statement about a world . The same formalism is used in AI through a probabilistic approach to logic , where the field of statistical relational learning has fused the relational model with the framework of probabilistic graphical models . Examples of such models include plate models , probabilistic relational models , Markov logic networks , and relational dependency networks ( Getoor & Taskar , 2007 ) . A closely related area that has enjoyed accelerated growth in recent years is relational and geometric deep learning , where the term “ relational ” is used to denote the inductive bias introduced by a graph structure . Although relational and graph-based terms are often used interchangeably in the machine learning community , they could refer to different data structures : graph-based models such as graph databases ( Robinson et al. , 2013 ) and knowledge graphs simply represent data as an attributed ( hyper- ) graph , while the relational model , common in databases , uses the entity-relation ( ER ) diagram ( Chen , 1976 ) to constrain the relations of each instance ( corresponding to a node ) , based on its entity-type ; see Fig . 1 ( a ) . Inspired by the success of equivariant deep learning , we use invariance and equivariance to encode the structure of relational data . This type of inductive bias informs a model ’ s behaviour under various transformations . Equivariant models have been successfully used for deep learning on data with a variety of structures , from translation equivariant images ( LeCun et al. , 1998 ) , to geometric settings ( Cohen et al. , 2018a ; 2019 ) , to discrete objects such as sets ( Zaheer et al. , 2017 ) , and graphs ( Kondor et al. , 2018b ) . By adopting this perspective , we present a maximally expressive feedforward layer that achieves equivariance w.r.t . exchangeabilities in relational data as it is expressed by the entity-relationship model . Our feedforward layer generalizes recently proposed layers for sets ( Zaheer et al. , 2017 ) , exchangeable tensors ( Hartford et al. , 2018 ) , and graphs ( Maron et al. , 2018 ) . 1An alternative terminology refers to instances and entities as entities and entity types . 2 REPRESENTATIONS OF RELATIONAL DATA . We represent the relational model by a set of entities D = { 1 , ... , D } and a set of relations R ⊆ 2D , indicating how the entities interact with one another . For each entity d ∈ D we have a set { 1 , .. , Nd } of instances of that entity . For each relation R ∈ R , where R = { d1 , . . . , d|R| } is a set of entities , we observe data in the form of a set of tuples XR = { 〈nd1 , . . . , nd|R| , x〉 | ndi ∈ [ Ndi ] , x ∈ RK } . That is , for each element of XR we associate the feature vector x with the relationship between the instances indexed by ndi for each entity di ∈ R. See Fig . 1 for a detailed , concrete example . Note that the singleton relation R = { di } can be used to incorporate individual entity attributes ( such as professors ’ evaluations in Fig . 1 ( a ) ) . In the most general case , we allow for both R , and any R ∈ R to be multisets ( i.e. , to contain duplicate entries ) . R is a multiset if we have multiple relations between the same set of entities . For example , we may have a SUPERVISES relation between STUDENTS and PROFESSORS , in addition to the WRITES REFERENCE relation . A particular relation R is a multiset if it contains multiple copies of the same entity . Such relations are ubiquitous in the real world , describing for example , the connection graph of a social network , the sale/purchase relationships between a group of companies , or , in our running example , the COURSE-COURSE relation capturing prerequisite information . For our derivations we make the simplifying assumption that each attribute x ∈ R is a scalar . Extension to x ∈ RK using multiple channels is trivial and discussed in Appendix F. Another common feature of relational data , the one-to-many relation , is addressed in Appendix E . 2.1 TUPLES , TABLES AND TENSORS . In relational databases the set of tuples XR is often represented using a table , with one row for each tuple ; see Fig . 1 ( b ) . An equivalent representation for XR is using a “ sparse ” |R|-dimensional tensor XR ∈ RNd1× ... ×Nd|R| , where each dimension of this tensor corresponds to an entity d ∈ R , and the the length of that dimension is the number of instances Nd . In other words 〈nd1 , . . . , nd|R| , x〉 ∈ X R ⇔ XRnd1 , ... , nd|R| = x . We work with this tensor representation of relational data . We use X = { XR | R ∈ R } to denote the set of all sparse tensors that define the relational data ( base ) ; see Fig . 1 ( c ) . For the following discussions around exchangeability and equivariance , we assume that for all R , XR are fully observed , dense tensors . Subsequently , we will discard this assumption and attempt to make predictions for ( any subset of ) the missing records . Note that relations R can be multisets . For simplicity , we handle this in the main text by considering equal elements as distinct through indexing ( e.g. , d ( i ) = d ( j ) ) , while leaving a formal definition of multisets for the supplementary material ; see Appendix B . 3 EXCHANGEABILITIES OF ENTITY-RELATIONSHIP DATA . Recall that in the representation X , each entity d ∈ { 1 , ... , D } has a set of instances indexed by nd ∈ { 1 , .. , Nd } . The ordering of { 1 , ... , Nd } is arbitrary , and we can shuffle these instances , affecting only the representation , and not the “ content ” of the relational data . However , in order to maintain consistency across data tables , we also have to shuffle all the tensors XR , where d ∈ R , using the same permutation applied to the tensor dimension corresponding to d. At a high level , this simple indifference to shuffling defines the exchangeabilities of relational data . A mathematical group formalizes this idea . A mathematical group is a set equipped with a binary operation between its members , such that the set and the operation satisfy closure , associativity , invertability and existence of a unique identity element . SM refers to the symmetric group , the group of all permutations of M objects . A natural representation for a member of this group gM ∈ SM , is a permutation matrix G ∈ { 0 , 1 } M×M . Here , the binary group operation is the same as the product of permutation matrices . In this notation , SNd is the group of all permutations of instances 1 , . . . Nd of entity d ∈ { 1 , .. , D } . To consider permutations to multiple dimensions of a data tensor we can use the direct product of groups . Given two groups G and H , the direct product G ×H is defined by ( g , h ) ∈ G ×H⇔ g ∈ G , h ∈ H and ( g , h ) ◦ ( g′ , h′ ) = ( g ◦ g′ , h ◦ h′ ) . ( 1 ) That is , the underlying set is the Cartesian product of the underlying sets of G and H , and the group operation is the component-wise operation . Observe that we can associate the group SN1 × . . .× SND with the D entities in a relational model , where each entity d has Nd instances . Intuitively , applying permutations from this group to the corresponding relational data should not affect the underlying contents , while applying permutations from outside this group should . To see this , consider the tensor representation of Fig . 1 ( c ) : permuting students , courses or professors shuffles rows or columns of X , but preserves its underlying content . However , arbitrary shuffling of the elements of this tensor could alter its content . Our goal is to define a feed-forward layer that is “ aware ” of this structure . For this , we first need to formalize the action of SN1 × . . .× SND on the vectorized form of X. Vectorization . For each tensor XR ∈ X , NR = ∏ d∈RNd refers to the total number of elements of tensor XR ( note that for now we are assuming that the tensors are dense ) . We will refer to N = ∑ R∈RNR as the number of elements of X . Then vec ( X R ) ∈ RNR refers to the vectorization of XR , obtained by successively stacking its elements along its dimensions , where the order of dimensions is given by d ∈ R. We use vec−1 ( · ) to refer to the inverse operation of vec ( · ) , so that vec−1 ( vec ( X ) ) = X . With a slight abuse of notation , we use vec ( X ) ∈ RN to refer to [ vec ( XR1 ) ; . . . ; vec ( XR|R| ) ] , the vectorized form of the entire relational data . The weight matrix W that we define later creates a feed-forward layer σ ( W vec ( X ) ) applied to this vector . Group Action . The action of g ∈ SN1 × . . .× SND on vec ( X ) ∈ RN , permutes the elements of vec ( X ) . Our objective is to define this group action by mapping SN1 × . . . × SND to a group of permutations of all N = ∑ R∈R ∏ d∈RNd entries of the database – i.e. , a homoomorphism into S N . To this end we need to use two types of matrix product . Let G ∈ RN1×N2 and H ∈ RN3×N4 be two matrices . The direct sum G⊕H is an ( N1 +N3 ) × ( N2 +N4 ) block-diagonal matrix , and the Kronecker product G⊗H is an ( N1N3 ) × ( N2N4 ) matrix : G⊕H .= ( G 0 0 H ) , G⊗H .= G1,1H . . . G1 , N2H ... . . . ... GN1,1H . . . GN1 , N2H . Note that in the special case that both G and H are permutation matrices , G⊕H and G⊗H will also be permutation matrices . Both of these matrix operations can represent the direct product of permutation groups . That is , given two permutation matrices G1 ∈ SN1 , and G2 ∈ SN2 , we can use both G1 ⊗ G2 and G1 ⊕ G2 to represent members of SN1 × SN2 . However , the resulting permutation matrices , can be interpreted as different actions : while the ( N1 + N2 ) × ( N1 + N2 ) direct sum matrix G1 ⊕G2 is a permutation of N1+N2 objects , the ( N1N2 ) × ( N1N2 ) Kronecker product matrix G1 ⊗G2 is a permutation of N1N2 objects . Claim 1 . Consider the vectorized relational data vec ( X ) of length N . The action of SN1 × . . .× SND on vec ( X ) is given by the following permutation group GX . = { ⊕ R∈R ⊗ d∈R Gd | ( G1 , . . . , GD ) ∈ SN1 × . . .× SND } . ( 2 ) where the order of relations in ⊕ R∈R is consistent with the ordering used for vectorization of X . Proof . The Kronecker product ⊗ d∈R G d when applied to vec ( XR ) , permutes the underlying tensor XR along the axes d ∈ R. Using direct sum , these permutations are applied to each tensor vec ( XR ) in vec ( X ) = [ vec ( XR1 ) ; . . . , vec ( XRD ) ] . The only constraint , enforced by ( 2 ) is to use the same permutation matrix Gd for all R when d ∈ R. Therefore any matrix-vector product GN vec ( X ) is a “ legal ” permutation of vec ( X ) , since it only shuffles the instances of each entity .

Summary: This paper defines a parameter-tying scheme for a general feed-forward network which respects the equivalence properties of a vector encoding of relational (ie database) data. As long as this parameter-tying is observed, the resultant feed-forward network is guaranteed to respect the equivariant properties of this data, meaning entity indexes can be permuted, and the output produced by the layer will be permuted in the correct manner. This results in a maximally expressive feed forward layer for this kind of data.