Venue: AKBC 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: ### Summary:

The paper addresses the problem of one-shot link prediction for temporal knowledge graphs (TKG). Specifically, it focuses on relations with very few triples. The proposed approach uses the framework of few-shot learning and episodic training on TKG using a temporal neighborhood encoder. The experiments demonstrate the proposed approach's effectiveness on three datasets against existing TKG models and few-shot learning models for static graphs.


### Strengths:
1. The paper presents a new setting for TKG link prediction. The paper also adapts the existing TKG datasets for the few-shot setting.
2. The proposed approach performs well across three datasets.
3. The problem is well-motivated, and the paper is well-written and easy to follow.

### Weaknesses:
1. Although it is a new setting, the temporal neighborhood encoder is similar to existing works (e.g., TeMP). Therefore, the novelty of the proposed method is limited. Further, it does not compare with alternate temporal encoders. Many TKG encoders are present in the literature, and it would have been interesting to evaluate their effectiveness in this setting.
2. The evaluation dataset only consists of relations with frequency >= 50. It would have been interesting to see the method's effectiveness on highly infrequent relations.


### Questions to the author:
- Did you also try relations with frequency <= 50? If yes, how did it perform?

### Presentation Improvements:
- The experiments section in the main paper is small. Probably, it can include concise versions of some experiments from the appendix.

review_title: New Setting for Temporal Knowledge Graph Link Prediction, but with limited experiments

final_score: rating: 6: Marginally above acceptance threshold
review_content: The work proposes a principled approach to knowledge completion in temporal knowledge graphs. This is achieved by two novelties — 1) a one-shot learning framework for TKGs that is based off similar approaches in static KGs and 2) a temporal neighborhood encoder that allows modeling of graphical and temporal dependencies, perhaps the key contribution of the work. Further, the work reports detailed exports highlighting the utility of their approach in addition to providing two additional benchmark datasets. 

Comments and Questions:
- Caption under Fig 3 can be improved — e.g. explaining the notation / input / output. 
- The paper is generally well-written and the described method is easy to follow
- The two architectural contributions proposed are not novel by themselves. However, their application in this setting is interesting and their utility, as demonstrated by experiments can be of use to the relevant community.
- Do the authors have intuition on the pros/cons of OAT-R or OAT-T for different types of data? Are there certain temporal properties that might favor one vs other? This might be useful to someone interested in applying the methodology for their problem. 

review_title: TKG Completion in one-shot setting

final_score: rating: 7: Good paper, accept
review_content: This paper proposes a new setting for temporal knowledge graphs ("TKG") --- one-shot link prediction for TKGs. The motivation is that knowledge graphs are long-tail (thus we need one-shot/few-shot link prediction) and for temporal knowledge graphs, the distribution of relational facts can be highly heterogeneous. The authors formalize the one-shot link prediction as an episodic learning problem, where in each episode there is a support set and a query set. The quadruples are divided into meta-train, meta-val and meta-test sets based on the timestamp. The authors also propose a new method to tackle the problem, including a neighborhood encoder to get entities' representation, with consideration of the graphs at previous timesteps, and a similarity network to get the similarity between query and support instances. The authors compare their model with several TKG or one-shot link prediction baselines on three datasets and show that the proposed model (OAT) achieves superior performance. The ablation study verifies the effectiveness of each component of the method.

Strength:

1. The paper is well motivated and the newly proposed setting is challenging and meaningful.
2. The proposed method is well described and is very intuitive.
3. The results are very strong and the ablation study is very comprehensive.

Weakness:

1. It would be nicer to provide some examples of relations and quadruples, especially for such a new setting.
2. The paper mainly talks about the one-shot setting. A few-shot setting and method would be more generalizable and more realistic in applications. 
3. Although the authors have already compared several baselines, it would be better to compare the proposed method to more recent TKG baselines (it is even more necessary considering the paper is setting a new direction).

Question:

1. Do you have any idea why OAT-T is much worse than OAT-R on ICEWS14 where it is better on the other datasets?

Misc.:

1. Missing space on page 3: "Xiong et. al. 2018,Du et. al. 2019"
2. Eq. (1): isn't Ti~T and Si, Qi the same thing?

review_title: A new setting for temporal knowledge graphs, 

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper first introduces the task of one-shot learning for temporal knowledge graph and constructs two benchmark datasets to conduct experiments and make comparisons. For addressing the problem, the author proposes One-shot Attention Temporal Graph Learning (OAT) which utilizes self-attention to encode temporal interactions and a scorer network for link predictions. The experimental results and ablation study confirm the superiority of the proposed OAT approach against several TKG and FSL baselines.

Strengths: 
1) The paper is generally well-written and easy to follow, the code and data are both provided for reproducibility.
2) This paper is well-motivated. One-shot learning for temporal knowledge graphs is an interesting research topic with practical scenarios and a wide range of applications. Two new publicly available benchmarks are also introduced by this paper and will benefit future research in this direction.
3) Compared with existing TKG models and few-shot learning methods for static graphs, the proposed OAT shows better performances on both GDELT and ICEWS benchmarks especially for sparse relations under the one-shot learning setting.

Weakness and Questions: 
1) Although cited in the paper, the author forgets to compare with two recent competitive baselines in the experiment section. They are TeMP (Wu, Jiapeng, et al. “TeMP: temporal message passing for temporal knowledge graph completion”, EMNLP 2020) and ATiSE (Xu, Chengjin, et al. “Temporal knowledge graph embedding model based on additive time series decomposition”, arXiv preprint arXiv:1911.07893, 2019). Moreover, another approach called DySAT (Sankar, Aravind, et al. “Dysat: Deep neural representation learning on dynamic graphs via self-attention networks.” WSDM 2020) is also a recent competitive TKG model, but the author didn’t cite and compare it with DySAT in the paper.
2) TeMP and DySAT both apply self-attention to encode/integrate temporal information from the graph entity representations. The only difference is the similarity network of OAT, which is designed for the one-shot learning setting. Hence, the technical novelty of the paper is relatively limited.
3) The format (mainly the font and header) of the paper didn’t follow the original AKBC template and should be updated.
review_title: The paper introduces a novel task and benchmark datasets, but lack of comparisons with several recent baseline methods

Meta-review: This paper addresses the problem of one-shot link prediction for temporal knowledge graphs (TKG), with a specific focus on sparse relations (ie with very few triples), using few-shot learning and episodic training on TKG with a temporal neighborhood encoder. While there were some questions around novelty and sufficiency of baselines, overall the paper presented very strong results in a very clear manner for a relevant and interesting problem.


Decision: Accept