Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper provides a differentiable method for learning temporal knowledge graphs in KGs. While differentiable rule-learning in the context of KG’s already exists, and rule-learning in the context of temporal KG’s already exists, this paper is the first to provide a differentiable rule learning method for temporal KGs.
strength_and_weaknesses: Thorough set of well-explained temporal features.

The addition of each feature is well-motivated, though the final set of features is quite complex and might benefit from more concrete running examples through the exposition.

Thorough set of experiments, though rather mixed results. The examples of learned rules are appreciated. Ablation with the “hard settings” in section 6.3 is also appreciated.
clarity,_quality,_novelty_and_reproducibility: Paper is generally well-written and understandable. Experiments are thorough, though additional data or code would be necessary to reproduce. Novelty is medium, being a fairly straightforward extension of existing work.

Seems to build a lot on the Timeplex framework, it’s good that the paper notes the specific differences. It might improve the presentation to present TimePlex and then present the additions, but this is probably not necessary. 

Typos / Nits:
Top of page 2, “Wididata12k”

For things like equation 4, a macro like \text{before} should be used instead of writing “before” in a math environment, as the latter looks like a multiplication of 6 variables.
summary_of_the_review: This is a well executed paper that straightforwardly follows up an existing method (TimePlex) with well motivated additions. The results are mixed but section 6.3 does a good job exploring the method in various settings. This paper would be a good addition to the literature.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method for learning temporal logical rules. Temporal logical rules are naturally discrete, this paper utilizes a differentiable framework for learning such rules. Specifically, it proposes a random walk mechanism and designs several operators for the temporal relations. It conducts experiments on several real-world datasets to demonstrate its effectiveness.  
strength_and_weaknesses: ### Strength
1. The problem of learning temporal logical rules is important. 
2. It improves the random walk in existing works from based on timestamps into based on intervals, which more flexible. 

### Weakness
1. The authors mention that StreamLearner and TLogic are both statistic learning methods, but what are the drawbacks of such methods? What type is the authors' method and what are the advantage?
2. The writing needs improvement, for example, it should emphasize in the introduction more the reason why this model is better than other TRL methods such as [1]. 
3. The experiments are not adequate, lacking efficiency study. 

[1] Liu Y, Ma Y, Hildebrandt M, et al. Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022, 36(4): 4120-4127.
clarity,_quality,_novelty_and_reproducibility: The codes are not provided. 
summary_of_the_review: The problem of TRL is crucial, but the core advantage of this method is unclear; furthermore, I have some concerns about reproducibility due to the lack of codes.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: 
The paper proposes a novel method to model temporal logical rules for knowledge graph completion. In specific, the authors first define constrained random walk for rule learning. Then, based on learned rules, the authors design a module to model various temporal features and apply rules for prediction.
================================================================
I have read the response and that solved most of my concerns. I would like to raise my score to 6.
strength_and_weaknesses: Strengths:
1. The motivation is reasonable.
2. The paper proposes a novel framework to model various temporal logical rules.
3. The experiments demonstrates the effectiveness of the proposed model.

Weaknesses:

The presentation needs further improvements and the paper is difficult to follow. For example:

 1) what is the difference between r and P? 
 2) The notations in Section 4 are not very clear, why sometimes using h and t for head and tail entities, and sometimes using x and y? 
 3) What does the operator M mean? An example may be helpful.
 4) The task formulation is actually similar with conventional link prediction without time interval. And, in table 1, why the ComplEx performs much better than TA-ComplEx, the temporal version of it? I would suggest an explanation of the dataset, so that we can better understand the necessity of modeling temporal information for link prediction.
clarity,_quality,_novelty_and_reproducibility: see above
summary_of_the_review: see above
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a differentiable temporal logical rule method, TILP for temporal knowledge graph. The proposal can achieve similar performance as state-of-the-art baselines, while can find helpful logic patterns under some restricted scenario which no previous work can find.
strength_and_weaknesses: ## Strength

- This method proposes a diiferentiable logic rule method on knowledge graph which can deal with time range related rules which has no previous works.

## Weakness

A lot of notations and explanations need to be refined for clarity.

-  In section 4, why call the first kind Markovian? I think Markovian refers to the case that current state is only determined by previous state. However, in your case, current random walk step is not determined by previous random walk step, instead, it is determined by a given relation type and given starting time which never changes in a random walk.
- Isn't Eq (6) wrong? In the left side, v is subscripted by length, while in the right side, v is subscripted by node identity.
- You should specify the dimension of each variable and function for Eq (8-13). It is very hard for me to understand the process there.
- What is $h_{0, l}$ in Eq (8)? Simply 0 as common RNN?
- Why this random walk embedding  Eq (8-12) only cares the context relation $r_h$, and ignores relations in the random walk?
- Can you provide the exact computation for all four temporal feature modeling? The definition is too vague for me to confidently understand.
- I do not see the usage of recurrence and duration in Eq (14-17).
- In section 5, it says that you will use all possible pathes between two nodes for both training and inference. So there is no "random" walk. Isn't this very costy?

Empirical result is not satisfying.

- On only two datasets, TILP does not show clearly better performance than TimePlex. Given the fact that this method is not incremental (does not need to store full history of temporal graph), this performance is not impressive.
- Although this paper shows TILP has the potential in restricted scenario, I am not sure if those hypothesis is interesting in tKG field or has no previous works, since I am unfamiliar with this field.
- It is unclear why TLogic works extremely worse since time range logics can be expressed by timestamp logics, thus I will expect similar performance.
clarity,_quality,_novelty_and_reproducibility: The clarity is bad for me, but may due to the lack of background.
Quality is fine.
The method and studying scenario is novel, however I am not sure if those scenarios are interesting due to the lack of background.
summary_of_the_review: This paper proposes a differentiable temporal logical rule method for temporal knowledge graph which has no previous works. However, the empirical gain is limited compare with existing baselines. The potential of restricted learning is not persuasive for me since those restricted scenario is only on two datasets, and no related works are provided to show community interest in those scenario.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper received a mixed range of reviews. It seems the paper indeed had some unclear parts as some of the reviewers have pointed out. However, I think many of these were addressed via rebuttal. The problem that the paper is tackling is an increasingly interesting problem in machine learning, and the paper brings a more robust insight into differentiable approaches for learning logic regarding temporal aspect. I think the paper is a great addition to the conference despite its weaknesses.

Strengths:
- Most reviewers agree that the paper's method is well-motivated and new (especially in that previous work has not tackled time range related rules).
- Most reviewers agree that the experiments are reasonable to support the paper's claim. 

Weaknesses:
- One of the reviewers was concerned that the paper does not discuss the efficiency of the method.
- Many reviewers were concerned with some unclear parts in the paper, many of which the authors seem to have clarified though.
- Some reviewers were concerned that the proposed method is not sufficiently better than its baseline, TimePlex. 
justification_for_why_not_higher_score: While the updated version is clearly better than the previous version, I think the original submission cannot be ignored when evaluating the recommendation type of the paper.
justification_for_why_not_lower_score: Given the characteristics of ICLR, I think we should encourage more submissions like this which might not be very novel but tackles an important problem and makes a significant improvement over an existing work.
Decision: Accept: poster