Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper argue that Conventional method can not utilize  neighborhood representation effectively for large-scale knowledge graph. The paper propose STARGRAPH, a node selection strategy, to find the superior node in two-hop subgraph. The results show that the proposed method get SOTA results in large-scale knowledge graph link prediction.
strength_and_weaknesses: 
Strength:

Experiment on the ogbl-wikikg2 link prediction (Table 4) get SOTA results.

Weakness:

The proposed method in the paper has limitations. The proposed method is to select better nodes in graph, which is a widely used idea [1][2][3]. Such an approach used on the knowledge graph lacks background and motivation. Also, as a KG-independent method, experiments on big graph are missing.

Treating the knowledge graph as a simple directed graph ignores the semantic properties of the knowledge graph, and such work is not conducive to the development of knowledge graph research. For example, whether the selected nodes make important at the knowledge level, the experiment lacks a case study to demonstrate this.

Compared to the previous approach [4], the proposed method experiments on knowledge graph related tasks and it is difficult to show that it is generalizable. 

[1]. Alsentzer E, Finlayson S, Li M, et al. Subgraph neural networks[J]. Advances in Neural Information Processing Systems, 2020, 33: 8017-8029.

[2]. Wang X, Ji H, Shi C, et al. Heterogeneous graph attention network[C]//The world wide web conference. 2019: 2022-2032.

[3]. Veličković P, Cucurull G, Casanova A, et al. Graph attention networks[J]. arXiv preprint arXiv:1710.10903, 2017.

[4]. Galkin M, Wu J, Denis E, et al. Nodepiece: Compositional and parameter-efficient representations of large knowledge graphs[J]. arXiv preprint arXiv:2106.12144, 2021.
clarity,_quality,_novelty_and_reproducibility: Well writing, Complete content, The proposed method works but lacks instructions.
summary_of_the_review: The proposed method is effective, but further reflection on KG is missing.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposed a StarGraph method to learn entity representations for large-scale knowledge graphs. The StarGraph first generates an incomplete two-hop neighborhood subgraph for each target node by picking part of the anchors and nodes within the 2-hop neighborhood. Then a transformer network is applied to obtain the entity representations. 

1. The transformer  network is able to mix the information of all nodes in the subgraph, but it fails to model the structure of the subgraph. Therefore, it seems not a good choice to use transformer  to encode graphs. It is neccessary to explain how the transformer  model the graph strucutre. Why do not you not use GNN networks?
2. When predicting the missing tail entity for (h,r,?), the methods need to sample and encode subgraphs for all the candidate tail entities. It seems  time-consuming. It is better to provide an analysis of  time complexity. 

strength_and_weaknesses: Weakness: 

1. The transformer  network is able to mix the information of all nodes in the subgraph, but it fails to model the structure of the subgraph. Therefore, it seems not a good choice to use transformer  to encode graphs. It is neccessary to explain how the transformer  model the graph strucutre. Why do not you not use GNN networks?
2. When predicting the missing tail entity for (h,r,?), the methods need to sample and encode subgraphs for all the candidate tail entities. It seems  time-consuming. It is better to provide an analysis of  time complexity. 

clarity,_quality,_novelty_and_reproducibility: This paper is generally well written. However, the following two questions should be further considered:
1. The transformer  network is able to mix the information of all nodes in the subgraph, but it fails to model the structure of the subgraph. Therefore, it seems not a good choice to use transformer  to encode graphs. It is neccessary to explain how the transformer  model the graph strucutre. Why do not you not use GNN networks?
2. When predicting the missing tail entity for (h,r,?), the methods need to sample and encode subgraphs for all the candidate tail entities. It seems  time-consuming. It is better to provide an analysis of  time complexity. 

summary_of_the_review: This paper is generally well written and the solution is ok. If the authors can clarity my questions, I will re-evaluate the paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Considering the current KG embedding methods of conventional shallow embedding models, which ignore the contextual information,   as well as the generic graph neural networks, which are difficult to expand to a KG with a large number of nodes and are very time-consuming, the authors propose a new knowledge representation learning method based on incomplete two-hop subgraph to leverage the neighborhood information with little computational complexity. 
strength_and_weaknesses: The strength of the paper:

In order to save time as well as keep much information from the neighbor nodes,  the authors propose a new method based on an incomplete 2-hop subgraph for knowledge representation learning.  

The weakness of the paper:
1. There is an error in the last two rows of the Table. 5 obviously.  
2. The proposed model is not evaluated enough on different datasets and different tasks. In the paper, the authors only evaluate the proposed method on two datasets on one task.
3. In fact, the proposed model is a variant of NodePiece and is not innovative enough.
4. In my opinion,  different types of anchors sampling methods are needed to be investigated in the ablation study.

clarity,_quality,_novelty_and_reproducibility: The paper is clear in expressing the authors’ ideas, while the proposed method is not innovative enough. 

As for the reproducibility of the paper, the authors describe related parts carefully in the paper, and it is not difficult to reproduce the experimental results from the paper. 
summary_of_the_review: 
The current version is not suitable for ICLR.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose a new knowledge representation method named StarGraph, which serves as an encoder and aims to generate high-quality entity embeddings for downstream score functions. The proposed StarGraph mainly consists of two stages: subgraph sampling and encoding, and the authors manage to optimize the efficiency in both stages. Experiments demonstrate that StarGraph achieves a new SOTA result on a large-scale knowledge graph benchmark (wik90m-v2), while being more parameter efficient than other baseline models.
strength_and_weaknesses: ## Strength
1. The idea of using subgraphs around entities to generate expressive entity embeddings is reasonable.
2. The authors propose a bag of tricks to reduce the computational overload for different steps of knowledge representation learning.
3. The proposed StarGraph significantly outperforms previous SOTA models on the wiki90m-v2 dataset.
4. Detailed experimental settings and codes are provided, so the reproducibility looks nice.


## Weaknesses
1. **Motivation**  
    1.1 Why we can use incomplete subgraphs is not well motivated. In the introduction, the authors use Figure 1(a) to demonstrate that complete subgraphs contain a lot of redundant information. However, is there any evidence that this information is redundant? Obviously, the completed subgraph in Figure 1(a) contains more topology information than the incompleted subgraph in Figure 1(b). Although using incomplete subgraphs may lead to competitive results, we can not tell which one is more suitable for knowledge representation learning just through a toy figure illustration.   

    1.2 Why we should use an anchor-based strategy is unclear. Even if using incomplete subgraphs is necessary for large-scale knowledge graphs, we can use a sample-based method (e.g., GraphSAGE) without anchors.
2. **Related Work**  
    The authors did a thorough review of related work. However, the differences between StarGraph and some existing works (e.g., NodePiece and GraphSAGE) are not fully discussed.
3. **Methodology**  
    3.1 The overall pipeline is unclear. I strongly suggest the authors provide a figure illustration for the overall architecture of StarGraph.

    3.2 In Section 3.2, the authors claim that they use a single transformer block. I am wondering if it is a standard transformer block, i.e., is there a residual connection and layer norm in the used module?

    3.3 The authors highlight two-hop subgraphs in the title. I am not sure whether two-hop is necessary. Can we use one-hop or three-hop subgraphs?

    3.4 The path information is captured by TripleRE in Section 3.2.2. However, it seems that the proposed framework is irrelevant to the path encoders. Therefore, I suggest the authors try more KGE models when encoding paths.

    3.5 I am curious about how to induce Equation (8) from (7). It would be better to provide a detailed derivation.
4. **Experiments**   
    4.1 It'd be better to put the main results before ablation studies.

    4.2 The Network ID in Table 2 is confusing.

    4.3 According to Table 3, the combination of anchors and neighbors already leads to the best test MRR, which demonstrates that path and center are unnecessary.

    4.4 As the authors claimed, the main contribution of this paper is a new graph encoder. Therefore, the graph encoder should be able to improve the performance of different decoders not only the proposed TripleRE', e.g., TransE, RotatE, AutoSF.



clarity,_quality,_novelty_and_reproducibility: ## Quality
Experiments in this paper are convincing and the proposed method shows strong performance. However, some concerns about motivation and ablations should be addressed. (See Strength And Weaknesses for more details.)

## Clarity 
This paper is overall well written, while the way to introduce Methodology can be improved.


## Originality 
Most of the used techniques are well-known, but the authors did a good job of combining them to achieve superior performance.

## Reproducibility    
Detailed experimental settings and codes are provided, so the reproducibility looks nice.
summary_of_the_review: This paper proposes an effective method to achieve new SOTA results on large-scale knowledge graph completion. Although the performance is strong, some concerns about motivation and ablations should be carefully addressed before this submission can be published. 

===============Update After Rebuttal===============

I have read the author's response (the authors did not upload a revised submission). However, my concerns about the motivation and methodology are not addressed. The author fails to provide sufficient evidence to explain why we must design the model as they do now. In my view, this submission looks more like a technical report than an academic paper. Thus, I am leaning to reject it.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a new approach for representation learning in knowledge graphs. It claims that other approaches do not sufficiently represent the information in the neighborhood of knowledge graph nodes. In order to achieve this, they sample the 2-hop-neighborhood and for target nodes. They claim that the method scales better than other methods which do consider the neighborhood of nodes.

The following strengths and weaknesses were identified by the reviewers.

Strenghts:
- Reasonable method for embedding the graph neighborhood
- Positive evaluation results

Weaknesses:
- Work is inspired bye.g.  NodePiece published at ICLR 2022 and the delta to this and other methods considered to be relatively small 
- Some further evaluations on larger graphs could be conducted to support the argument of scalability raised in the paper
- Some uncertainty about the methodology pointed out in parts of the reviews
justification_for_why_not_higher_score: The main reason for not rating the work higher is that it seems somewhat incremental relative to the previously published work on NodePiece and does not constitute a substantial step change to be published at ICLR.
Decision: Reject