Question: What is the difference between the position matrix and role embedding? I assume both are used to distinguish entities at different positions in the hyper edge.
Evidence: As discussed in Sec. 1, designing the proper message function is more conducive to capturing relational patterns of given HKGs and pursuing high empirical performance. Therefore, the first task is to design an expressive search space so that message functions covering various relational patterns can be searched. However, the space of existing GNN searching methods (see Eq. 1 and Fig.) does not apply to HKGs and cannot cover the message functions of GNNs for KGs/HKGs (see Sec. 2.2). Thus, we focus on the search space design of message functions in this paper.
Recently, various message functions have been proposed for KGs/HKGs as presented in Sec. 2.2. From Fig., we can observe that those message functions are mainly different in these two aspects: 1) the operators (e.g., W , φ, γ) for computing hidden representations, 2) the structure of message functions that decides how computational operators are connected. For the operator selection, existing works manually tune them on different data sets, such as
Moreover, the structure of message functions for HKGs (StarE and G-MPNN) tends to be deeper and more complex than those for KGs (e.g., CompGCN). This is because the message function needs to process more information (e.g., more entities/roles) when facing the facts with higher arity. These observations motivate us to build space of operators and structures for message function search. Furthermore, we investigate more about the relationship between operators and relational patterns. Generally, the relational pattern can be represented as a certain correlation among r(pemu(e 1 , . . . , e n )) (check more in Appx. B.2), where pemu denotes the permutation. For example, r is symmetry if r(e j , e i ) must be true when r(e i , e j ) is true. Therefore, the message function in the search space must be able to handle such correlation in the form of r(pemu(e 1 , . . . , e n )). In the next, we first introduce the space of operators, and then discuss how they deal with r(pemu(e 1 , . . . , e n )).
• • Concatenate Operator γ(•): It mainly determines the concatenation way between embedding vectors. In this paper, we set O γ = {concat, mult, wsum}, where wsum is the weighted sum. In general, γ(•) can concatenate embeddings after the positional transform matrix W P (e) , i.e., encoding pemu(e 1 , . . . , e n ).
• Role Embedding r o : Unlike the positional information of entities encoded by W P (e) , the role embedding is utilized to model the semantic information of entities. For example, the roles in 2nd position of facts playCharacerIn(actor:ZacharyQuinto,character :Spock,movie:StarTrek) and actorAwardIn(actor:ZacharyQuinto,award: BC-BSFC,movie:StarTrek) are different though other entities and roles are same. Thus, the model should be able to capture the role of candidate entities after using W P (e) , e.g., BC-BSFC is unlike to be the 2nd entity of playCharacerIn since its role is award instead of character.
• Composition Operator φ(•): Following CompGCN, we utilize composition operator φ(•) to capture message between the node and edge embeddings before aggregation step. Note that φ actually encodes the interaction between r and pemu(e 1 , . . . , e n ). While CompGCN and StarE empirically selects the most proper φ(•), we include this operator into the space X . Thus, MSeaHKG is able to automatically search for the most suitable φ(•) in a more efficient way. We combine the settings of CompGCN and StarE to set O φ = {sub, mult, corr, rotat} (rotat (Sun, see others in Sec 2.2). Besides, φ not only occurs between primary relations with entities, but also captures the correlation between roles with entities. • Others: (1) The idt(h) = h operation allows inputs to skip one layer in the message function; (2) Unlike W P (e) , the transform matrix W processes the hidden representations.
Among above components, we fix the role embedding and positional transform matrix W P (e) in the message function (see Fig.
where θ k ij ∈ {0, 1} controls the connection between o k i with o k−1 j . Note that {h 0 i } n+1 i=1 = {W P (e1) e 1 , . . . , W P (en) e n , r} if role information is not available in the given HKG. From Fig. (c), we can observe that the structure of message functions is controlled by {θ k ij }. To avoid manual operation selection, we also search for concrete operations of two operators φ and γ. Given the operator set
Then, φ and γ perform the computation in Eq. 6 could be
The message function parameterized by θ mg is defined as: mg r(e 1 , . . . , e n );
where we simplify role embedding r o for simplicity and h K i is outputted by the last layer of Eq. 6. Intuitively, existing GNNs for KGs/HKGs are contained in the MSeaHKG space (compare Fig. and Fig.). Moreover, we include more discussions about the instantiations of other KG/HKGs models in the MSeaHKG space and the capability of handling relational patterns in Appx. B.2.
In addition to message function search, we also search for other operators (e.g., agg, comb, act) like existing GNN search methods as shown in Fig. (a) and (b). Let Θ = {θ mg , θ agg , . . . } be parameter set for all operators selection in our MPNN framework. Then, an architecture can be represented as X Θ = {mg(•; θ mg ), agg(•; θ agg ), • • • } (check other functions in Appx. C.1). And existing GNNs for KG/HKG embeddings actually can be represented by different instantiations of Θ. Overall, a GNN model X Θ encodes the given HKG G into embedding space ω = {E, R}, i.e., ω = X Θ (G).
Role Embedding ro: Unlike the positional information of entities encoded by WP (e) , the role embedding is utilized to model the semantic information of entities (Liu et al., 2021). For example, the roles in 2nd position of facts playCharacerIn(actor:ZacharyQuinto,character :Spock,movie:StarTrek) and actorAwardIn(actor:ZacharyQuinto,award: BC-BSFC,movie:StarTrek) are different though other entities and roles are same. Thus, the model should be able to capture the role of candidate entities after using WP (e) , e.g., BC-BSFC is unlike to be the 2nd entity of playCharacerIn since its role is award instead of character.
Answer: Overall, positional matrices encode the positional information of entities in hyper-relational facts, while role embeddings are utilized to model the semantic information of entities. Usually, the role represents the hypernym of entities.