Venue: AKBC 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper proposed a joint approach to learn embeddings for KG entities from both descriptions and KG structure, for the task of open-world knowledge graph completion. The main idea is separate modules for obtaining structural embeddings (of entities and relations in a KG) and description embeddings (of descriptions of possibly new entities), and a description projection module to project the description to a space where the structure and description embeddings are aligned (KG embeddings). 

(+) The approach was evaluated on three different datasets, showcasing the robustness of the approach.

(+) Comprehensive analysis of the proposed approach compared with several prior work on the topic.

(+) The differences with considered baselines are well explained.

(-) Some design choices of the approach need justification (see Q1).

Questions:
1. What is the disadvantage of using transformer-based model to represent an entity description when the description is short (ten of fewer words)? Why is there a need to use two different encoders depending on the description length?
2. In Section 6.2, four tasks were introduced but the last one was without any elaboration unlike the other three?
review_title: Interesting approach for jointly learning structural and textual embeddings for KGs, comprehensive experiments and analysis

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors propose a new method for computing Knowledge Graph Embeddings on KGs where textual descriptions of entities are available. They evaluate the effectiveness of their method on a number of downstream tasks, including open-world KG completion, closed-world KG completion and entity classification. They show that their method (FOlK) outperforms baselines across all tasks in key metrics.

The authors also perform several ablations to show effectiveness of different parts of their model. They also highlight areas in which their model is better than baselines such as OWE, and provide intuition for it.

However, there are several shortcomings in the way the paper is written and organized. Some of these are:

1. 'Cross domain alignment' Sec 4.3 is hard to understand. Authors say they 'treat an entity’s immediate graph neighbourhood as its context and use skipgram objective to maximize the probability of the entity given the neighbourhood entities'. However, this is nowhere reflected in the loss functions used. Loss function $\mathscr{L}$ as well which is used in eqn 2 and 3 hasn't been defined in the main text which makes things even more confusing. $\mathscr{L_{proj}}$ is not described anywhere. Training algorithm should be part of main text, since it is a vital differentiator from OWE.

2. DSA score is hard to understand, given description is not enough to be able to reproduce it.

3. 'FOlK is the first framework to use a transformer-based encoder to embed open-world entities'
KG-BERT (2019) uses BERT for encoding closed-world entities. Extending it to open-world entities is a straight forward solution and not a novelty.

4. 'the score for any triple must be of the order of the embedding dimensions'  Section 1, point 3-'Efficient Ranking'. What does this mean?

5. Whats the difference between FOlK(s) and FOlK(l)? It's not specified

6. A lot of the content that is being referred to in the main text lies in the supplementary material. The authors should move at least the most relevant stuff (such as training algorithm, loss functions) to main text.

It seems that the paper has been written in a hurry and is lacking sufficient description for the method, given that the main contribution is the proposed new method FOlK. A better method section and reorganizing is needed.


review_title: Interesting fusion of text and conventional KG Embeddings, consistent improvements over baselines for all datasets/metrics

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper presents an approach to learn representations of (open-world) entities in a KG with given textual descriptions of open-world entities. This paper describes a technique that jointly learns embeddings for KG entities from descriptions and KG structure for open-world knowledge graph completion. The technique is experimentally validated on the YAGO3-10-Open and WN18RR-Open datasets and it beats previous open world entity representation learning methods.

I think this paper is a decent focused contribution. However, I think it can be significantly improved in its presentation quality:
1. The drawbacks of existing approaches is not clearly described and thus it is hard to understand the contributions of this work
2. The method description is not complete. For example L_proj is not described. It is hard to understand the approach.
3. The descriptions of the DSA score is also fuzzy. It is not clear how I would implement this with just this description.

The method is taken from BiSkip. But it is hard to understand how the application is different here.

Statistical significance numbers should help in table 4.

I also have the following questions:
1. Why do we need a 2 phase learning approach. How does it compare to a one phase learning model?
2. how do you find the optimal value of alpha?
review_title: The paper is a decent focused contribution. However, the experimentation and the writing can be significantly improved.

Meta-review: This paper presents a new approach called FOIK to learn entity embeddings that exploits not just the KG structure but also the textual entity descriptions, evaluated on the task of open-world knowledge graph completion. Extensive experiments show the value of the approach.
The authors were able to address most reviewer feedback in the revised version that was uploaded to the system.
Decision: Accept