Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The paper revisits limitations of relation-embedding models by taking losses into account in the derivation of these limitations. They propose and evaluate a new relation encoding (TransComplEx) and show that this encoding can address the limitations previously underlined in the literature when using the right loss.

There seems to be merit in distinguishing the loss when studying relation encoding but I think the paper's analysis lacks proper rigor as-is. A loss minimization won't make equalities in (3) and (5) hold exactly, which the analysis do not account for. A rewriting of the essential elements of the different proofs could make the arguments clearer.

Paper writing: 
* The manuscript should be improved with a thorough revision of the style and grammar. Example of mistakes include: extraneous or missing articles, incorrect verbs or tenses. 
* The 10-pages length is not beneficial, the recommended 8-pages could hold the same overall content.
* The option list on page 8 is very difficult to read and should be put in a table, e.g. in appendix.
* Parentheses are missing around many citations and equation references

Theory:
Equation (2) and (4) do not seem to bring much compared to the conditions in Table 1. Eq. (3) and (5) show "a" loss function rather than "the" loss function since multiple choices are possible. \gamma_1 should be set to 0 when it is 0 rather than staying in the equations.
* Minimizing the objective (3) and (5) will still not make the conditions in Table 1 hold exactly, because of slack variables.

Experiments:
- Can the authors provide examples of relations learned with RPTransComplEx# that go address the limitations L1...L6, validating experimentally the theoretical claims and showing that the gain with RPTransComplEx5 correspond to having learned these relations?
review_title: Official Blind Review #4

final_score: rating: 1: Reject
review_content: The paper analyses the effect of different loss functions for TransE and argues that certain limitations of TransE can be mitigated by chosing more appropriate loss functions. Furthermore, the paper proposes TransComplEx -- an adaption of ideas from ComplEx/HolE  to TransE -- to mitigate issues that can not be overcome by a simply chosing a different loss.

Analyzing the behavior and short-comings of commonly-used models can be an important contribution to advance the state-of-the-art. This paper focuses on the performance of TransE, which is a popular representation learning approach for knowledge graph completion and as such fits well into ICLR.

Unfortunately, the current version of the paper seems to have issues regarding methodology and novelty.

Regarding the experimental evaluation: The paper compares the results of TransComplEx and the different loss functions to results that have previously been published in this field (directly, without retraining). However, it seems from Section 5 (Dataset), that this paper is using a modified dataset, as the TransE models are only trained on high-confidence triples. All prior work that I checked doesn't seem to do this, and hence the numbers are not comparable.

Even more serious: Following again Section 5 (Dataset), it seems that the paper imputes all missing triples in the training set for symmetric and transitive relations ("grounding"). Hence,  the models get to see _all_ true triples for these relation types and as such the models in this paper are trained on the test set.

Regarding novelty: The short-comings of TransE and improvements to the loss have been discussed quite extensively in prior work. Using complex representations in TransComplEx seems also a straightforward application of the insights of ComplEx/Hole. As such, the main novelty would lie in the experimental results which, unfortunately, seem problematic.
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: In this paper, the authors investigate the main limitations of TransE in the light of loss function. The authors claim that their contributions consist of two parts: 1) proving that the proper selection of loss functions is vital in KGE; 2) proposing a model called TransComplEx. The results show that the proper selection of the loss function can mitigate the limitations of TransX (X=H, D, R, etc) models.

My major concerns are as follows.
1.	The motivation of TransComplEx and why it works are unclear in the paper.
2.	The experiments might be unconvincing. In the experiments, the authors claim that they implement RotatE [1] in their setting to make a fair comparison. However, with their setting, the performance of RotatE is much worse than that in the original paper [1]. Therefore, the experiments might be unfair to RotatE.
3.	There are some typos in this paper. For example, in Line 11 of Section 4.3, the comma should be a period; in Section 5, the "Dissuasion of Results" should be "Discussion of Results".

[1] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197, 2019.

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: 
Summary:
This paper list several limitations of translational-based Knowledge Graph embedding methods, TransE which have been identified by prior works and have theoretically/empirically shown that all limitations can be addressed by altering the loss function and shifting to Complex domain. The authors propose four variants of loss function which address the limitations and propose a method, RPTransComplEx which utilizes their observations for outperforming several existing Knowledge Graph embedding methods. Overall, the proposed method is well motivated and experimental results have been found to be consistent with the theoretical analysis.

Suggestions/Questions:

1. It would be great if hyperparameters listed in the “Experimental Setup” section could be presented in a table for better readability. 

2. In Section 2, the authors have mentioned that RotatE obtains SOTA results using a very large embedding dimension (1000). However, it gives very similar performance even with smaller dimensional embedding (such as 200) with 1000 negative samples. In Section 5, RotatE results with 200 dimension and 10 negative samples are reported for a fair comparison. Wouldn’t it be better to instead increase the number of negative samples in RPTransComplEx instead of decreasing negative samples in RotatE?

3. In Table 3, it is not clear why authors have not reported their performance on the WN18RR dataset for their methods. Also, the reported performance of TransE in [1] is much better than what is reported in the paper. 

[1] Sun, Zhiqing, Zhi-Hong Deng, Jian-Yun Nie and Jian Tang. “RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space.” ArXiv abs/1902.10197 (2019): n. pag.

review_title: Official Blind Review #3

Meta-review: The paper analyses the effect of different loss functions for TransE and argues that certain limitations of TransE can be mitigated by choosing more appropriate loss functions.  The submission then proposes TransComplEx to further improve results.  This paper received four reviews, with three recommending rejection, and one recommending weak acceptance.  A main concern was in the clarity of motivating the different models.  Another was in the relatively low performance of RotatE compared with [1], which was raised by multiple reviewers.  The authors provided extensive responses to the concerns raised by the reviewers.  However, at least the implementation of RotatE remains of concern, with the response of the authors indicating "Please note that we couldn’t use exactly the same setting of RotatE due to limitations in our infrastructure."  On the balance, a majority of reviewers felt that the paper was not suitable for publication in its current form.
Decision: Reject