Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: In the paper, the authors propose to minimize the discrepancy between pairs of (conditional) neural axioms to align the embedding spaces of different KGs. This method is justified by the authors' study of all kinds of OWL2 properties. The author also studied the influence of margin $\lambda$ on less constrained/long-tail entities. The authors conducted experiments by adding the proposed model on top of the best models for entity alignment. The results are mixed, but the proposed model improves the SEA and RDGCN consistently. 

Reasons to accept:
1. This paper provides a theoretic point of view of the entity alignment task, which was mostly studied in empirical methods. The idea to align the axioms by minimizing Wasserstein distance is well-justified.
2. The experiment results are in favor of the intuitions.
2. The method described in this paper can be in principle adapted to any previous and future EEA scoring functions. 

Reasons to reject:
The idea of using adversarial training to align spaces, especially cross-lingual spaces, is based on the assumption of the large overlap between KGs. For KGs that are on very different domains, this method may include errors, as two heterogeneous KGs do not naturally fit in one unified space. The influence of overlap on this method is not well-studied. All of KGs used in the experiments are general domain KGs. 
review_title: Well justified EEA model

final_score: rating: 5: Marginally below acceptance threshold
review_content: --- Overall ---

This paper proposes an entity alignment framework that leverages the dependencies between entities and relations (reminiscent of TransR [Lin Y, et al. AAAI '15]) to further refine the results of conventional embedding-based alignment approach. 

Merits: The proposed framework is shown to be effective in improving the performance of baseline models.

Weaknesses: (1) the methodological contribution is limited. (2) the theoretical explanation part is trivial and contributes little scientific knowledge.


--- major comments ----

1.	The bound in Eq.5 seems meaningless since the assumption (i.e., one relation and one neighbour) on which the bound basis is too idealistic to meet in practice. 
2.	The explanations about the behaviour of embedding-based entity alignment (in both section 2.2 and section 3.1) are straight-forward and trivial, thus contribute little knowledge.
3.	In my point of view, section 2.1 and section 3.1 are too lengthy. It would be better to highlight the most important part i.e., loss function, while avoid emphasising too much on the detailed definitions and examples.


--- minor comments---
There are some typos and grammar mistakes, need to be proof-read carefully (e.g., “e_x^2” -> “e_y^2” in the paragraph just above Eq.6; “take X for example”-> “take X as an example”).

review_title: Empirically good, but the methodological contribution is limited.

final_score: rating: 5: Marginally below acceptance threshold
review_content: Overall Comments:
Entity alignment plays an important role in improving the quality of cross-lingual knowledge graphs. As one of the most important solutions, embedding-based methods aim at learning a semantic space where the unique entity cross knowledge graphs can have the closest distance. Most of research focus on entity-level granular, but discard the whole picture of embedding space of cross-lingual KGs. Besides the aligned entity pairs as the labelled data, this paper extended the labelled data with the conditional neural and basic axioms, which are actually sets of randomly selected entities or entities with the same relation type. Then the final objective is to align the cross-lingual knowledge graphs by both optimizing the distance of labelled entity pairs and neural axioms.

Clarity:
The presentation and organization of this paper is very difficult to follow.  Besides the grammar and type errors, there exists many concepts that not clear, which makes it difficult to understand the main idea of this work. For example, the concept of axiom and ontology are introduced before giving a formal definition. The claimed challenges, that have not been solved well by previous works ,are not convincing enough. In the 3rd paragraph, authors argued that previous research shows very good performance, but has not made on the theoretical analysis. After reading the whole draft, it's still a big question on the given theoretical analysis of this work. Taking the theorem 1 as example, it's more like a justification but not a theorem to show the connection between the proposed axiom and "ontology". Ontology provides an empirical structure to organize and classify the entities in the KGs. Its structure will be changed along with the KG in hand. From this paper, I can not find the connection between relation type alignment loss and the ontology. Please pay more attention to the writing and organization. It's an interesting work but not ready.

Questions for Rebuttal:
1. How to build the relation seed in this work? Are they labelled manually? If so, it will have flexibility issue for dealing multiple KGs.

2. Please compare to a recent proposed method [1] which also optimizes the distance between a group of entities from cross-lingual KGs. Different from this work, it's not condition on the relation type, but based on a randomly sampled group of entities.

Minor Comments:

1. In the paragraph around the Equation (6), the e_x^2 should be e_y^2.

2. Figure 3 shows the overall architecture of the proposed method. It should appear in the main content.  

References:

[1] Pei, Shichao, Lu Yu, and Xiangliang Zhang. "Improving cross-lingual entity alignment via optimal transport." International Joint Conferences on Artificial Intelligence Organization, 2019.
review_title: Interesting but not clear paper

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary: 
The paper proposes NeoEA, an approach that further constrains KG embedding with ontology knowledge. The paper first tries to summarize the existing embedding-based entity alignment methods, stating that most of the methods choose TransE as scoring functions. But their embedding features are not aligned well compared to the neural-based or composition-based loss function. The paper, therefore, solves this problem by developing a new NeoEA architecture which shows that adding a KG-invariant ontology knowledge can minimize such difference. The experiment shows the new constraints can improve state-of-the-art baselines.

Strengths:
+ The idea of using ontology constraint as an additional loss is new. The paper shows significant improvements when combining the newly designed loss with state-of-the-art baselines.
+ The overall paper is clear to understand. 

Weaknesses:
- The paper doesn't have a related work section. Figure 1 is a little bit messy. Especially for figure 1d, it would be better to make the figure a little bit larger. Moreover, it's unclear which color corresponds to the first KG, making readers confused. Section 2 should be merged into the introduction section.
- Some of the words are confusing such as neural axioms. The neural ontology alignment (which is stated in the appendix figure) is much more clear than the current Conditional Neural Axioms. Section 3 is not well-organized. The theorems and axioms should be propositions or some hypothesis. The usage of those words is a little bit wield here. The proof in the appendix is also not well defined.
- The experiment section is too short and not very informative. It would be better to include more comprehensive analysis such as providing some visualization before and after the new loss etc.

**Post-Rebuttal:
- I appreciate that the authors have conduct revisions on the current version. However, I think the current paper is probably still not strong enough for ICLR. 
review_title: Review for this paper

Meta-review: The authors study the problem of augmenting embedding-based entity alignment in knowledge graphs (KG) through the use of joint alignment with deduced neural ontologies (more specifically, alignment of the KG 'neural' axioms). Motivated by the observation that the representation between two potentially aligned entities must be bound by a minimal margin, which can be problematic when there are many potential alignments, they propose aligning neural axioms by Wasserstein distance-based loss between learned entity embeddings conditioned on the relation embeddings. Experiments are conducted on OpenEA against multiple strong baselines -- showing that adding the ontology alignment to these baselines improves the results.

== Pros ==
+ The addition of aligning (conditional) ontologies is ostensibly novel.
+ For KGs with sufficient entity/relation overlap, the proposed NeoEA method is applicable. 
+ NeoEA has been shown empirically to improve many SoTA methods.

== Cons == 
- While the theoretical justification is a welcome motivation, the reviewers did not find the theoretical arguments significant nor convincing.
- Overall, the narrative needs work to make the paper more self-contained and approachable for a broader range of readers. The reviewers (and myself) found many concepts and statements somewhat confusing and needing clearly context and contrast with existing works.

Evaluating along the requested dimensions:
- Quality: Conceptually, the core idea is interesting, well-motivated, original, and ostensibly effective. Empirically, NeoEA is shown able to improve upon several strong baseline (underlying) methods.  I believe that all of the reviewers find the work is interesting and promising. However, there were continuing concerns the strength/value of the described theory; it isn't clear if stronger theory isn't possible or if this just hasn't been fleshed out. 
- Clarity: Most of the reviewers (and myself) found the paper difficult to follow as a self-contained work in terms of concepts, clear definitions (e.g., \mathcal T isn't defined early on) and the actual applicability of the theory. The figures help, but even these need some work. A related work section (or more structured presentation of related work) might be clarifying along with running examples and a more unifying math presentation that captures existing and proposed work. After thinking about this more, it is actually a relative simple (in a good way) and clever idea. However, it took several readings and readings of related work to get there. Additionally, the fact that all of the reviewers were concerned about different limitations is concerning wrt clarity. Appendix B helps a bit and I believe can also be put into the main paper.
- Originality: As best as the reviewers and I can tell, we haven't seen this method applied to entity alignment despite this being a relatively mature subfield.
- Significance: The consensus seems to be that the approach could be a notable contribution to an important area. However, it also appears that most of the reviewers don't feel the paper is ready for publication at a top-tier venue yet.

As stated throughout this meta-review, there are several aspects to like about this work including the originality of the idea, strong motivation, and good empirical results. However, we all agreed that the paper isn't quite ready in its current form -- thus, I presently recommend reject for this submission.
Decision: Reject