Generalize to Fully Unseen Graphs: Learn Transferable Hyper-Relation Structures for Inductive Link Prediction

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-09 13:45:13

Abstract: Inductive link prediction aims to infer missing triples on unseen graphs, which contain unseen entities and relations during training. The performances of existing inductive inference methods were hindered by the limited generalization capability in fully unseen graphs, which is rooted in the neglect of the intrinsic graph structure. In this paper, we aim to enhance the model's generalization ability to unseen graphs and thus propose a novel Hyper-Relation aware multi-views model HyRel for learning the global transferable structure of graphs. Distinct from existing studies, we introduce a novel perspective focused on learning the inherent hyper-relation structure consisting of the relation positions and affinity. The hyper-relation structure is independent of specific entities, relations, or features, thus allowing for transferring the learned knowledge to any unseen graphs. We adopt a multi-view approach to model the hyper-relation structure. HyRel incorporates neighborhood learning on each view, capturing nuanced semantics of relative relation position. Meanwhile, dual views contrastive constraints are designed to enforce the robustness of transferable structural knowledge. To the best of our knowledge, our work makes one of the first attempts to generalize the learning of hyper-relation structures, offering high flexibility and ease of use without reliance on any external resources. HyRel demonstrates SOTA performance compared to existing methods under extensive inductive settings, particularly on fully unseen graphs, and validates the efficacy of learning hyper-relation structures for improving generalization. The code is available online at https://github.com/hncps6/HyRel.

Main paper: 
 related work: 
 Inductive link prediction with associated seen entities. Early exploration into inductive link prediction could be traced back to [13] and [29] . They require these unseen entities to be associated with seen entities in the training set. Therefore, such methods cannot be generalized to settings where entities are entirely unseen.Subgraph-based methods. These methods [20, 21, 25, 33] extract local subgraphs and employ GNN modules [3, 23] to enable relation prediction. While these methods could handle scenarios involving unseen entities, they cannot generalize to settings involving the emergence of unseen relations. [10] also falls into this category but could operate on inference graphs with unseen relations. The task handled by [10] is similar to ours; however, its method of extracting local subgraphs for each candidate entity to compute scores does not calculate embedding vectors, rendering it inapplicable for downstream tasks.Rule-guided and path formulation. These approaches aim to discover common logical rules or relation paths in graphs. As logical rules are not specific to particular entities, such methods could be generalized to handle inductive tasks involving unseen entities. Both [35] and [22] integrate parameter and structure learning of first-order logical rules into end-to-end differentiable models. [38] introduces a general framework for path representation learning, proposing new operators for aggregating node pairs and path embeddings. [37] constructs more complex relational digraphs than paths to capture local evidence. Language Models. Methods like [1, 7, 11] encode the textual descriptions of each entity using pretrained language models. These approaches utilize the corresponding text descriptions to obtain embedding for unseen entities. In our work, embeddings for entities and relations are solely derived from the graph structure, without relying on any external sources, aiming to be resource-efficient and generalizable.A method similar to ours. INGRAM [14] is similar to ours, as both approaches could handle the scenario of complete inductive link prediction, where entities and relations could be entirely unseen. INGRAM was the first to introduce the concept of affinity. However, it measures the affinity between two relations by the sum of the counts of shared head entities and shared tail entities. In addition, it overlooks the significant differences in affinity between different relative positions of relations. In other words, both the positional information and affinity between each relation should be considered simultaneously, as the affinity between relations varies depending on different relative positions. We comprehensively consider the relative positional information between relations and the affinity, aggregating fine-grained relational semantic information by enhancing the distinctiveness of each relative positional information(details will be provided in Section 4.2). 
preliminaries: 
 Next, we will explain the setup of the current task in this paper.The training graph is defined asG tra = (E tra , R tra , F tra ), where E tra is the set of entities, R tra is the set of relations, and F tra is the set of triples in the training graph G tra . We divide F tra into F sup and F opt , such that F tra = F sup ∪ F opt . F sup is a set of seen factual triples aimed at obtaining representations for all entities and relations in the training graph, while F opt is a set of factual triples used to calculate triple scores for the purpose of optimizing the model. G inf is the inference graph, defined asG inf = (E inf , R inf , T inf ), where E inf represents the set of entities, R inf represents the set of relations, and T inf represents the set of triples in the inference graph G inf . We divide T inf into three parts, T sup , T val , and T tes , such that T inf = T sup ∪ T val ∪ T tes . During testing, T sup is used to obtain embeddings for entities and relations in the inference graph. T val represents the validation set of the inference graph, and T tes represents the test set. After computing the embeddings for entities and relations using T sup , the corresponding embeddings are obtained for T val and T tes to perform validation and testing, respectively.We set the ratio of F sup to F opt in the training graph as 3:1, and the ratio of T sup , T val , and T tes in the inference graph as 3:1:1, by adapting the model to different graph structures through the transformation of triplets in F sup and F opt within each epoch during training, following the settings of previous work [1, 9, 14] . Entities in the inference graph are unseen to the training graph, meaning that all entities in the inference graph are new entities, i.e., E tra ∩ E inf = ∅. Furthermore, Our method can handle both cases where partial relations are unseen, i.e., R tra ≠ R inf , and where relations are completely unseen, i.e., R tra ∩ R inf = ∅. Inverse triples [24] induced by inverse relations are also encompassed within our model.  
defining hyper-relation structure: 
 We define hyper-relation structure by extracting structural information from the original graph. Firstly, we divided four views based on the distinct relative positions between relations. In other words, each view represents different relative positions between relations. Secondly, we computed the frequency of shared entities between relations at different relative positions and defined it as affinity. This definition is effective because affinitive relations tend to cluster around entities, as shown in Fig. 2 with kinship relations consistently clustering together. Finally, each view treats relations as nodes and the affinity as edges. Additionally, the affinity between the same relation pairs varies across different views.To obtain the four views, we first construct two matrices 𝑬 h ∈ R 𝑛×𝑚 and 𝑬 t ∈ R 𝑛×𝑚 , where 𝑛 represents the number of entities and 𝑚 represents the number of relations. These matrices indicate the frequency of each entity appearing as a head entity and tail entity across all relations. For example, 𝑬 h [𝑒 1 , 𝑟 2 ] represents the frequency of entity 𝑒 1 appearing as the head entity for relation 𝑟 2 in the original graph. Then, 𝑫 ℎ ∈ R 𝑛×𝑛 is the diagonal matrix representing the degree of entities as head entities, i.e., 𝑫 ℎ [𝑖, 𝑗] = 𝑗 𝑬 ℎ [𝑖, 𝑗]. Similarly, 𝑫 𝑡 ∈ R 𝑛×𝑛 is the diagonal matrix representing the degree of entities as tail entities. Finally, we define the adjacency matrices of the four views as:𝑨 𝑝 1 -𝑝 2 = 𝑬 𝑇 𝑝 1 𝑬 𝑝 2 𝑫 𝑝 1 𝑫 𝑝 2 (1)where 𝑝 1 -𝑝 The connections within each view are established based on the affinity within that specific view. Fig. 2 shows an example.… … … … … … … … … … … … … … … … … … … … … … … … Oxford A C D H G B EWe use Glorot initialization [12] for initializing relation embeddings, denoted as v 𝑖 ∈ R 𝑑𝑟 , where 𝑖 represents the 𝑖-th relation, and 𝑑𝑟 represents the dimension of the relation vectors. Then, the embeddings of relations are updated using the four views. Each relation in the four views obtains four different sets of neighboring relations and their corresponding affinity weights. We define the process of updating relations as follows:xEQUATIONwhere (𝑙) represents the 𝑙-th layer with 𝑙 ∈ {0, 1, • • • , 𝐿 -1}. x (0) 𝑖 = 𝑾 (𝑟𝑒𝑙 ) v 𝑖 , 𝑾 (𝑟𝑒𝑙 ) ∈ R 𝑑𝑟 ′ ×𝑑𝑟 is a learnable matrix,where 𝑜 (𝑖,𝑗 ) |𝑝 is the learnable parameter, and it is selected based on the ranking of 𝑎 𝑖 𝑗 within 𝑨 𝑝 1 -𝑝 2 , where a higher ranking indicates a stronger affinity. 𝜛 (𝑙 ) ∈ R 1×𝑑𝑟 ′ is a weight vector. Eq.( 5) provides a detailed definition of the relative attention coefficients 𝛼 (𝑙 )𝑖 𝑗 |𝑝 : 𝛼 (𝑙 ) 𝑖 𝑗 |𝑝 = exp 𝑏 (𝑙 ) 𝑖 𝑗 |𝑝 + 𝑡 𝑝 𝑟 𝑗 ′ ∈ N 𝑝 𝑖 exp 𝑏 (𝑙 ) 𝑖 𝑗 ′ |𝑝 + 𝑡 𝑝 (5)where 𝑡 𝑝 is the views-adaptive parameter. Due to the differing structures among the four views, the relative attention coefficients should be different accordingly and correspond to the structures of the four views. For instance, if the relative attention coefficient𝛼 (𝑙 ) 𝑖 𝑗 |ℎ-ℎ obtained in G ℎ-ℎ is equal to 𝛼 (𝑙 ) 𝑖 𝑗 |𝑡 -𝑡 in G 𝑡 -𝑡 , HyRel could differentiate them using 𝑡 𝑝 .In contrast to current inductive link prediction methods [4, 11, 17, 29] , we consider the different importance of relations across different relative positions. Specifically, each of the four views in HyRel has its own independent set of weight parameters: 𝑾 𝑝 , 𝑯 𝑝 , and views-adaptive parameter 𝑡 𝑝 . Hence, HyRel can aggregate more fine-grained semantic information by the differences between different views. Then, the final relation embeddings are computed based on the Eq.( 2), denoted asx 𝑖 = 𝑴 𝑟 x (𝐿) 𝑖 (𝑖 = 1, • • • , 𝑚), where 𝑴 𝑟 ∈ R 𝑑𝑟 ×𝑑𝑟 ′is a learnable mapping matrix.We design a novel contrastive learning, which imposes contrastive constraints on four views. Specifically, unlike the construction of negative sample pairs in common graph constrastive learning [16, 18, 30] , here we only consider the neighbors of nodes as negative pairs to avoid relational semantic confusion. In other words, the anchor's neighbors within the same view and across different views are treated as negative pairs. For instance, we define x 𝑖 |ℎ-ℎ between the ℎ-ℎ and 𝑡-ℎ views is formulated as follows:EQUATIONwhere sim (𝑝𝑜𝑠 ) represents the similarity between positive pairs, and is defined as:sim (𝑝𝑜𝑠 ) = 𝑒 𝜃 x (𝐿) 𝑖 |ℎ-ℎ ,x (𝐿)𝑖 |𝑡 -ℎ /𝜏 (7) And sim (𝑛𝑒𝑔) represents the similarity between negative pairs:EQUATION𝑗 |𝑡 -ℎ /𝜏 (8) where 𝜃 (•) is a similarity metric function(default inner product). 𝜏 is a temperature hyperparameter used to control the scale of the similarity.In the overall dual views contrastive constraints , the ℎ-ℎ view serves as the central view and is contrastive with the other three views, i.e., ℓ x(𝐿) 𝑟 |ℎ-ℎ , x (𝐿) 𝑟 |𝑡 -ℎ , ℓ x (𝐿) 𝑟 |ℎ-ℎ , x (𝐿) 𝑟 |𝑡 -𝑡 and ℓ x (𝐿) 𝑟 |ℎ-ℎ , x (𝐿)𝑟 |ℎ-𝑡 . In Eq. ( 6), we explained the loss formulation with 𝑟 𝑖 from the ℎ-ℎ view as the anchor. Similarly, when 𝑟 𝑖 from the other views is used as the anchor, the contrastive loss can be obtained in the same manner. To perform pairwise comparisons, the final contrastive loss is defined as follows:EQUATIONwhere P′ = {G 𝑡 -𝑡 , G ℎ-𝑡 , G 𝑡 -ℎ }. Section 5.6 explains that the selection of the central view has a minimal impact on performance.We use Glorot initialization [12] for initializing entity embeddings, denoted as e 𝑖 ∈ R 𝑑𝑒 , 𝑖 = 1, • • • , 𝑛, where 𝑑𝑒 is the dimension of the entity vectors. Then, similar to updating relation embeddings, the entity embeddings are updated by aggregating neighboring entities through a multi-head attention mechanism and residual connections.We define z R 𝑑𝑒 ′ ×𝑑𝑒 is a learnable matrix. The specific process for updating entity embeddings is as follows:EQUATIONwhere R 𝑗𝑖 is the set of relations from entity 𝑖 to 𝑗, 𝑾(𝑙 ) ∈ R 𝑑𝑒 ′ ×𝑑𝑒 ′ is the weight matrix. 𝛼 (𝑙 )𝑖 𝑗𝑟 is defined as the entity-level attention coefficient, which is computed by concatenating the feature vectors of neighboring entities and connected relations for each entity. To compute the attention weight for the self-loop of entities. We utilize the mean vector of the representation vectors of the relations adjacent to entities, which is similar to the strategies adopted in [14] . The mean embedding x 𝑒 = 𝑴 𝑒 z ( 𝐿) 𝑖 (𝑖 = 1, • • • , 𝑛), where 𝑴 𝑒 ∈ R 𝑑𝑒 ×𝑑𝑒 ′is a learnable mapping matrix.The model is trained to ensure that positive triples in F opt obtain higher scores compared to the sampled negative triples. We have employed a variant of the DistMult model [34] as the scoring function.The scoring function is defined as𝑓 𝑒 ′ 𝑖 , 𝑟, 𝑒 ′ 𝑗 = z 𝑇 𝑖 𝑑𝑖𝑎𝑔(𝑴 𝑟𝑒 x 𝑟 )z 𝑗 , where 𝑟 = (1, • • • , 𝑚), 𝑴 𝑟𝑒 ∈ R 𝑑𝑒 ×𝑑𝑟is a weight matrix and 𝑑𝑖𝑎𝑔(𝑴 𝑟𝑒 x 𝑟 ) represents a diagonal matrix. With this scoring function, we could derive a loss function specific to the link prediction.EQUATION)F ′ opt is a set of negative triples, which are generated by altering the head or tail entity of positive triples. The parameter 𝛾 serves as a threshold to differentiate the margin between positive and negative triples. By combining the contrastive constraints loss, we can obtain a composite loss function for optimizing our model:EQUATIONTable 1 : The performance of HyRel and baselines in inductive link prediction is evaluated on twelve datasets. The number following each dataset represents the proportion of new relations. * represents the results obtained by replicating the model on these datasets. + represents that we replicated the model, and for a fair comparison, we selected the best results reported in the paper.NL-100 NL-75 NL-50 NL-25 MRR Hit@10 Hit@1 MRR Hit@10 Hit@1 MRR Hit@10 Hit@1 MRR Hit@10 Hit@ MRR Hit@10 Hit@1 MRR Hit@10 Hit@1 MRR Hit@10 Hit@1 MRR Hit@10 Hit@ MRR Hit@10 Hit@1 MRR Hit@10 Hit@1 MRR Hit@10 Hit@1 MRR Hit@10 Hit@  
experiment 5.1 datasets: 
 The benchmarks used in our experiments are sourced from NELL-995 [32] , Wikidata68K [11] , and FB15K237 [26] , configured for inductive settings. Each benchmark is subdivided into four datasets, where the percentages of triples with new relations are 100%, 75%, 50%, and 25%. All entities in the G inf of the 12 datasets are not observed during training. For instance, in NL-100, all triples in the inference graph involve unseen relations. The detailed information of the datasets and hyperparameters settings is presented in supplementary materials.To assess the performance of different methods, we employ three commonly used evaluation metrics: MRR, Hit@10, and Hit@1. MRR represents the mean reciprocal rank, while Hit@10 and Hit@1 indicate the proportion of correct answers ranked within the top 10 and top 1 among all candidates. We compared the performance of our method with 16 methods on the task of inductive link prediction, including: INGRAM [14] , MaKEr [4] , GraIL [25] , INDIGO [20] , CoMPILE [21] , BLP [7] , QBLP [1] , RAILD [11] , CompGCN [28] , NodePiece [9] , NeuralLP [35] , DRUM [22] , at the same time. Thus the substantial performance degradation validates the rationale behind our approach. Some existing inductive link prediction methods alter the training regimes to enhance model generalization, leading to randomness in model outcomes. This situation also applies to how the training set is divided in each epoch. We compare HyRel with INGRAM, both of which are trained using this method of dividing the training set. As shown in Fig. 4 , our method exhibits significantly smaller error ranges across all three metrics on the NL-100 and NL-75 datasets compared to INGRAM. This indicates that our model's good performance is not a result of random chance but rather its ability to generate embeddings for unseen entities and relations more reasonably. In practice, the choice of central view in four views has minimal impact on performance, as shown in Table 4 with relevant experimental results. It can be observed that the impact of different central views on the experimental results is relatively small. Although there are fluctuations, they are within an acceptable range due to the presence of randomness.Relation pairs outlined with dashed ellipses in Fig. 5  
conclusion: 
 We introduce HyRel, a pioneering approach tailored for addressing inductive link prediction adopting the hyper-relation structure.HyRel operates independently, void of dependencies on pre-trained language models or external sources. Emphasizing the acquisition of robust reasoning capabilities, HyRel leverages the hyper-relation structure to effectively capture graph structural intricacies. This empowers HyRel to generate precise embeddings for unseen relations and entities. Moreover, the proposed dual views contrastive constraints are imposed across views, which alleviates semantic confusion of relations. Extensive experiments across diverse inductive settings validate HyRel's superior performance. HyRel offers a promising avenue for addressing the challenges of knowledge extrapolation in evolving KG environments.  
: 
 tail entity 𝑡 in the triple (ℎ, 𝑟, ?). Methods using low-dimensional embedding vectors [6, 31] have proven effective for link prediction. Entities and relations are transformed into embedding vectors, enabling inference of missing triples. However, these methods [2, 8, 19, 27, 36] follow the transductive setting, i.e. require entities and relations to be present in the training set. When entities or relations unseen during training occur, they no longer work.Inductive link prediction aims to infer missing triples on unseen graphs, which contain unseen entities and relations during training, even fully unseen. The unseen entities and relations mean they are not present in the training set. As shown in Fig. 1 (a), inference graph (i) and (ii) contain unseen entity 𝐺 haven't seen in training graph. All entities and relations in inference graphs (ii) are not seen in the training graph. An example of inductive link prediction is also shown in inference graph (ii), e.g. infer missing tail entity 𝐻 in the triple (𝐺, 𝐶𝑙𝑎𝑠𝑠𝑚𝑎𝑡𝑒_𝑜 𝑓 , ?). In inductive link prediction, there are two scenarios [10, 14] of inductive learning: (i) fully unseen entity set with partially unseen relations set during inference (semiinductive link prediction) and (ii) fully unseen entity set with fully unseen relations set (full-inductive link prediction). Inference graph (i) and inference graph (ii) in Fig. 1 (a) show these scenarios.A variety of inductive link prediction methods have been developed. Some works attempt to obtain embeddings for unseen entities from pre-trained language models or textual descriptions, or propagate information from seen entities to unseen entities through graph neural networks [5, 11, 17, 29] . However, additional textual descriptions or explicit associations between unseen entities and seen entities may not always be available in practical application scenarios. In addition, the approaches of learning transferable features through meta-learning or other subgraph partitioning methods [4, 10] , while not relying on external resources, lack access to the global structural information of the given knowledge graph. Recently, INGRAM [14] proposed to employ relation-level aggregation methods to address full inductive link prediction. The performances of such methods were hindered by the limited generalization capability.To address the above issue, we tend to learn transferable knowledge rather than rely on specific entities or relations, which is similar to the way humans think. Take the scenario in Fig. 1 (b) as an example, when inferring the triple (𝐵, 𝐻𝑢𝑠𝑏𝑎𝑛𝑑_𝑜 𝑓 , 𝐶), relevant information could be extracted from its associated relational structure, such as 𝑀𝑜𝑡ℎ𝑒𝑟 _𝑜 𝑓 and 𝐹𝑎𝑡ℎ𝑒𝑟 _𝑜 𝑓 , while other relations such as 𝑊 𝑜𝑟𝑘_𝑖𝑛 may be less relevant to the inference. This means that the relation 𝐻𝑢𝑠𝑏𝑎𝑛𝑑_𝑜 𝑓 is more affinitive with 𝑀𝑜𝑡ℎ𝑒𝑟 _𝑜 𝑓 and 𝐹𝑎𝑡ℎ𝑒𝑟 _𝑜 𝑓 than with 𝑊 𝑜𝑟𝑘_𝑖𝑛 in this inference. After identifying the affinitive relations, it's crucial to assess their relative positions. Specifically, if the relations 𝑀𝑜𝑡ℎ𝑒𝑟 _𝑜 𝑓 and 𝐹𝑎𝑡ℎ𝑒𝑟 _𝑜 𝑓 both directed towards entity 𝐴, it could be inferred that (𝐵, 𝐻𝑢𝑠𝑏𝑎𝑛𝑑_𝑜 𝑓 , 𝐶) holds true. Alternatively, if 𝑀𝑜𝑡ℎ𝑒𝑟 _𝑜 𝑓 and 𝐹𝑎𝑡ℎ𝑒𝑟 _𝑜 𝑓 are not both directed towards entity 𝐴, and the direction of 𝐹𝑎𝑡ℎ𝑒𝑟 _𝑜 𝑓 is not known, 𝐵 could be either the ℎ𝑢𝑠𝑏𝑎𝑛𝑑 or the 𝑔𝑟𝑎𝑛𝑑𝑐ℎ𝑖𝑙𝑑 of 𝐶.The core of the inference process outlined above is the recognition of affinity relations and the relative positions of these relations. In the inference graph (ii) of Fig. 1 (a), the validity of (𝐺, 𝐶𝑙𝑎𝑠𝑠𝑚𝑎𝑡𝑒_𝑜 𝑓 , 𝐻 ) could be determined according to the affinitive relations 𝑇 𝑒𝑎𝑐ℎ𝑒𝑟 _𝑜 𝑓 and 𝐺𝑟𝑎𝑑𝑢𝑎𝑡𝑒𝑑 with their relative directions. We refined affinitive relations and relative positions of relations, which are independent of any specific entity, relation, or graph, as Hyper-Relation Structure. It could be considered as transferable knowledge because it naturally exists in any graph. We argue that if the model learns this hyper-relation structure during training, it will enable the model to make inferences on unseen graphs, i.e., achieve semi-inductive link prediction and full inductive link prediction.To this end, we propose a novel Hyper-Relation aware multiviews approach for inductive link prediction (HyRel). It operates independently of pre-trained models or external resources, which provides high flexibility and ease of use. Specifically, we adopt a multi-view approach to model the hyper-relation structure, where different relative positions of relations form different views. In each view, relations are treated as nodes, and the affinity (the modeling of affinity between relations is formally defined in Section 4.1) between relations serves as edges. HyRel learns fine-grained semantics of relation contexts at different positions within each view and generalizes this learning to unseen graphs. Furthermore, we propose dual views contrastive constraints to reduce the impact of local information confusion, enhancing the robustness of learned intrinsic and transferable structural knowledge. To the best of our knowledge, our work makes one of the first attempts to generalize the learning of hyper-relation structures. Specifically, our contributions are summarized as follows:• Distinct from existing studies, we introduce a novel perspective focused on learning the hyper-relation structure inherent in graphs, allowing for transferring the learned knowledge to any unseen graphs. • We are the first to incorporate the affinity of different relative positions of relations into inductive link prediction. By leveraging the semantic differences between different relative positions, we aggregate more refined semantic information. • We propose dual views contrastive constraints to alleviate semantic confusion of relations and enforce the robustness of transferable structural knowledge.The experimental results in various inductive settings demonstrate that HyRel outperforms existing models in inductive link prediction, particularly excelling in challenging fully inductive tasks where entities and relations are entirely unseen. 
 
 Appendix:  


Decision: The paper presents an innovative approach to inductive link prediction, particularly in a fully inductive setting, through the HyRelGat model. The strengths highlighted by the reviewers include the method's generalizability across diverse inductive settings, effective update of relation representations, and dual views contrastive constraints that enhance performance. The ablation studies and case studies further substantiate the model's effectiveness, with results significantly outperforming baselines across multiple benchmarks. While the paper is well-organized and provides clear illustrations of motivation and architecture, it faces some limitations such as the need for broader benchmark comparisons and more detailed experimental settings to improve reproducibility and interpretability. Despite minor concerns about the code availability and dataset choices, the overall contributions and promising results support the acceptance of this paper.

Reviews: review: paper_summary: paper_summary: This paper introduces a hyper-relation aware multi-views model  HyRel for semi- and fully-inductive link prediction task. HyRel first constructs hyper-relation graphs with 4 views, and then update relation representation and entity representation based on these 4 views. It also propose a novel contrastive learning imposing contrastive constraints on 4 views. Experiments on existing benchmark of semi- and fully-inductive link prediction benchmarks show that HyRel performs well, especially on fully-inductive setting.
strength_weakness: strength_weakness: 1. This paper focus on an interesting and challenging task, inductive link prediction, especially in fully-inductive setting. 
2. The method proposed in this paper has a  quite strong generalizability, that could be used for diverse inductive setting with different new-relation ratio. 
3. The experiment results support and prove the effectiveness of the proposed method.
limitations: limitations: 1. The code is not available with the given link.
2. In the experiments of inductive inference with seen relations, why only NELL-995-v1 is chose for this experiments? as there are a lot of version of data are available.
paper_summary: paper_summary: The paper presents a method for inductive link prediction in knowledge graphs by learning transferable hyper-relation structures. It introduces the concept of hyper-relation structure to represent complex relationships between entities and proposes a model called HyRelGat that updates relation representations using graph attention networks. The model employs dual views contrastive constraints to enhance generalization across unseen graphs. Extensive experiments on various datasets demonstrate the effectiveness of the proposed approach in predicting links in fully unseen graphs, outperforming several baseline models. The study also includes ablation analysis, randomness analysis, and case studies to validate the model's performance and robustness.
strength_weakness: strength_weakness: The paper presents a novel approach for inductive link prediction by learning transferable hyper-relation structures, which is particularly useful for generalizing to unseen graphs. It introduces the HyRelGat model that effectively updates relation representations and applies dual views contrastive constraints for improved performance. The ablation study demonstrates the significance of each component, and case studies provide insights into the model's behavior. Overall, the method shows promising results in enhancing graph representation learning for relational data. Particularly, the experiments are sufficient enough to support the reliability of the proposed method.
limitations: limitations: Although it is appreciated that this paper can provide many benchmarking results between the proposed method and the available state-of-arts methods, I still suggest the author to consider categorize the available methods and select the typical several ones for benchmarking test.

Regarding "From Table 1, it can be observed that our method outperforms the baseline models in almost all of the 12 datasets", I have similar concern. I still suggest to give the readers more insights about the datasets. Why the author selects so many datasets for comparison? What is the point of each dataset concerning? Whether or not the proposed method can be proved to work well on this point? 

In session 6. Conclusion, "HyRel operates independently, void of dependencies on pre-trained language models or external sources." I suggest the author to provide the proof that using the proposed HyRel method, it can work well even combining the pre-trained language models. In my understanding, it is an unvoidable situation nowadays.
paper_summary: paper_summary: This paper proposes a called HyRel for inductive link prediction. HyRel aims to enhance generalization ability by learning the global transferable structure of graphs, specifically focusing on the inherent hyper-relation structure. The hyper-relation structure is independent of specific entities, relations, or features, allowing for knowledge transfer to any unseen graphs. Experiments on multiple benchmarks validate the efficacy of HyRel.
strength_weakness: strength_weakness: 1. This paper is easy to follow. The motivation and architecture of HyRel are clearly illustrated.

2. The results across multiple benchmarks are promising, significantly outperforming the baselines.

3. Extensive experiments are conducted to further analyze the proposed method.
limitations: limitations: 1. As the paper mostly follows INGRAM’s experiment settings, it would be valuable to include the performance of HyRel on other benchmarks such as FB15k-237 and WN18RR proposed by GraIL.

2. Honestly, I think this paper lacks relevance to multi-media or mutli-modality, please, may I ask, why the authors submit it to MM?

typo: Line 132: 'are' is missing.
paper_summary: paper_summary: This paper proposes a novel HyRel model to enhance the generalization ability of inductive link prediction in unseen graphs. HyRel improves the model's performance on completely unseen graphs by learning a globally transferable graph structure, particularly learning a hyper-relational structure consisting of relational positions and affinities. The authors mainly introduce a novel perspective that focuses on learning the hyper-relational structure inherent in graphs, enabling the learned knowledge to be transferred to any unseen graph. For the first time, affinities between relations at different relative positions are incorporated into inductive link prediction. Aggregate finer semantic information by exploiting the semantic differences between different relative positions. Dual-view contrast constraints are proposed to alleviate the semantic confusion of relations and enhance the robustness of transferable structural knowledge.
strength_weakness: strength_weakness: This article presents the practical application of the HyRel model, extracting the hyper-relational structure from the global structure of a given graph. By considering the affinities between relations at different relative positions, four views are constructed, each representing a different relative position of the relationship. Affinity is defined by calculating the frequency of shared entities between relationships at different relative positions. The authors proposed the HyRel-Gat mechanism for generating high-quality relation embeddings. Dual-view contrast constraints are designed to alleviate the semantic confusion of relationships and enhance the robustness of transferable structural knowledge, thereby inspiring new possibilities in the field of inductive link prediction.
limitations: limitations: Although the paper is well organized and logical, some parts must be changed and complemented.

 First, it is hoped that the author can provide a more detailed description of the experimental settings, including the data set used, data preprocessing steps, hyperparameter selection and other information, to increase the repeatability and credibility of the experimental results. 

Second, to improve the model's interpretability, it is recommended that a more in-depth explanation of the inner workings of the HyRel model be provided, including the role of key components and how they affect the final prediction results.