Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a causal KGC model to alleviate the data bias issues that come from the data collection process, and further proposed an evaluation method to evaluate the causal KGC models on observation datasets. Moreover, the authors also provided a different perspective of viewing KGC from group theory. Numerical experiments showcased that the causal KGC models outperform traditional KGC models in most cases. 
strength_and_weaknesses: Strength: It's an interesting ideal of trying to incorporate causal inference techniques into traditional KGC models, and the group theory perspective of KGC is quite novel.

Weakness: 1.I agree that in some sense, data bias can be treated as confounders in the causal model, however, I do not think it's appropriate to simply define the confounders as the "artificially designed confounders" and "learnable confounders" in this paper. Confounders are the type of variables that have causal impact over both treatment and outcome variables, but in the definition of "artificially designed confounders" in this paper, they are essentially treated as the variables that are impacted by treatment variables. In other words, the directions of certain arcs in the causal graph changed.    
2. I appreciate the attempt of explaining KGC from the group theory perspective. However, I cannot see if there's an potential application or new insights from such explanation. 

Other comments: 1. Too much identical sentences in Abstract, Introduction and Conclusion.      
2. In multiple places, you wrote "In this paper, we propose causal KGC models to alleviate the issues ..." Here you should be more specific by saying "data bias issues".   
3. You mentioned "causal graph" is multiple places of this paper. However, in the causal KGC model, all causal graphs are essentially identical and they have the same topology as the graph (b) in Figure 1. The reason why causal graph in traditional causal inference is important is because, it gives people a direct way of viewing the relationship between variables, which can help people identify what are confounders, what are instrumental variables etc. By introduction causal model into KGC, here different triplets (h,r,t) will still have no connecting arcs between them, except the newly added confounder variables $(Z_h, Z_r, Z_t)$. For that reason, I do not think you should highlight the concept of "causal graph" in this paper at all.   
4. You mentioned a few times that "the main feature of causal is invariance and symmetry (Arjovsky et al., 2020)". However, I did not see that from your cited literature. Why is the main feature of causal is symmetry? A causes B means something totally different from B causes A.    
5. First sentence in Section 2.1, "let $\epsilon$ denote(s) and ... denote(s)".  
6. Bottom of page 4: "$P(y \mid h, \ldots, z_t)$ evaluates..." this is an incomplete sentence.   
7. Bottom of page 4: "confounders (is)" should be "are".   
8. When constructing the "learnable confounders", how to learn those neural networks?   
9. On page 5, $z_t'$ should be $z_{t'}$.  
10. Please add references to those ranking metrics $MRR$ and $H@N$.  
11. Should not use the word $Metric$ as the name of your evaluation metric.   
12. In Section 3.3, "should invariant" -> "should be an invariant".  
13. In Section 3.3, "which may (do) not match ..."  
14. Last sentence of Section 5.2: I cannot see the reason from the degree of data bias. Please explain more. 
clarity,_quality,_novelty_and_reproducibility: This paper is poorly written, bears with various typos and incomplete sentences, and the technical details are hard to follow. Overall, I suggest the authors to rewrite the Introduction and Background sections completely, and try not to use repeated sentences in Abstract, Introduction and Conclusion. 
summary_of_the_review: Based on my comments in the previous sections, even though this paper has certain merit of introducing causal model into KGC, but overall I do not think the contribution is significant enough to make this paper be accepted as the ICLR conference proceeding.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a novel approach to the task of knowledge graph completion, which draws on insights from the field of causal reasoning to obtain an adjustment that should - in principle - help with bias correction in the data underlying the knowledge graph. It considers two types of artificial confounders, one based on the log-popularity and another one, based on a one-layer neural network (aka a perceptron). 
strength_and_weaknesses: Strengths: The overall presentation is clear, and the method is explained in a logical way that is easy to follow. The abstract sets the stage, the introduction reviews relevant work, and the methods clarify the novelty of the work. The results are presented fairly. The conclusion points out that much future work remains.

Weaknesses: These are easier to list for specific sections, which I have done below.

Abstract: "our causal KGC models achieve better performance than traditional KGC models" - on what inputs?

Intro: "the world is driven by causality rather than correlation" - over-generalisation, especially as causality is often more difficult to observe

Background: the embedding isn't motivated or explained; is it externally specified? learned in the graph construction process? In the tensor representation of the causal graph each entity and relation is an element of a set, not a vector, so this needs a clear explanation/illustration.

Methods: 

1) While the entire process is motivated by causal inference and the removal of confounders, the simplifying assumptions ultimately amount to a. a frequency penalty and b. a learned entity-specific or relationship-specific penalty. Would it not have been easier to present the entire approach as a penalty-based one, and then provide the motivating causal inference-based derivation in the appendix?

2) The group-theoretic view does not provide any additional value or insight to the paper; I say this despite being a big fan of group theory myself. I recommend removing this section altogether.

3) A clearer differentiation from inverse propensity scoring should be provided by explicitly showing how IPS does something different from the frequency penalty introduced in the method.

Results: "different evaluation metrics" - it seems that you only used a single evaluation metric in the end, namely, Metric(-1,-1,-1)?

Discussion: "our model is [...] model-agnostic" is confusing; perhaps "our approach is [...]" instead?

Appendix: the standard deviation of frequency is not a great way to quantify bias; perhaps fitting a modified Zipf's law or computing a Gini coefficient could be more useful?
clarity,_quality,_novelty_and_reproducibility: Clarity is high, although a few grammar and language errors interfere at certain places (the use of "Roughly" is inconsistent with current use, in particular).

Quality is moderate as there are three datasets considered (good), but the results are evaluated with a single metric, and thus no "sensitivity analysis" is carried out (poor); there is also a throwaway reference to group theory which does not seem to advance the paper in any way.

Novelty is moderate insofar as similar methods have been previously proposed and some of them (especially IPS) perform comparably well.

Reproducibility is low as no source code or link to a repository containing it is provided (although the methods are described fairly clearly, this is insufficient for reproducing the results as the learned parameters are missing, as is the code and the RNG seeds used to generate them).
summary_of_the_review: Overall, a decent paper with a moderate advance over the current state-of-the-art; it is a "one-trick pony" paper, with a single idea being presented and implemented, and evaluated with a single metric.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The knowledge graph completion is a key to studying the correlations in data. This paper proposes to study the causality of the KGC from a causal graph point of view. The method is evaluated by empirical experiments. However, in total, the novelty of the approach and the profound impact are both limited. 


strength_and_weaknesses: Strength: The authors evaluate the proposed causal KCG by relatively comprehensive empirical experiments. 

Weaknesses: 

1. The preliminaries and background introduction for the SCM models should be more comprehensive, as this is the fundamental point for the later KGC model development. 

2. The logarithmic function definition for designed confounders is restrictive. Otherwise, the author should refer to the existing literature for such definition and evidence. 

3. In the one-layer neural network, is it able to approximate the complex confounder? This seems to disobey the universal approximation theorem in DNN. 

4. The score function $f(h,r,t)$ is not clear how affect the conditional probability $P(y|h,r,t,z_h,z_r,z_t)$. In addition, the linear decomposition might require the $g$ function is soothing. However, the authors seem not to address or mention this part in the paper.

5. The balancing of $\alpha_{h}, \alpha_{r}$ and $\alpha_{t}$ is crucial to balance the bias-variance tradeoff, the authors might be better to discuss this. 

6. The authors define a novel metric for evaluation purposes. I was wondering if is there any existing metric for better comparison.

7. The causal-distmult-2 outperforms the competing methods in FB15k-237. But other ones, like causal TransE-1, RotatE-1, and ComplEX-1 are best. Could the authors give some analysis of this phenomenon? 
clarity,_quality,_novelty_and_reproducibility: 1. The notation is abusive. For example, it would be better to use a different note for the head entity and tail entity. 

2. Figure 1. In the causal graph traditional KGC model, illustration example (a) is just one of the cases. The authors might give a more advanced structure in the existing literature for illustrating purposes. 

3. In related work, it would be better to provide closely related literature in explainable and interpretable knowledge graph methods. 


summary_of_the_review: The paper studies the causality of KGC and uses numerical experiments to justify the proposed method. However, the development of the method is not clear and the paper is not well-presented. This limits the potential impact of the paper.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The authors propose a causal knowledge graph completion (KGC) model to alleviate issues pertaining to mere correlation-based KGCs.
The reviewers felt that the paper’s writing and presentation of the material must be improved. They also had concerns about the significance of the work as for instance how it differs from other methods such as IPS. Perhaps a rewrite of the paper in light of reviewers concerns could improve and clarify the significance of the results.
justification_for_why_not_higher_score: It is not ready for publication. Neither the importance of the contributions are clear not the writing of the paper is suitable for a good venue.
Decision: Reject