Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: The paper presents empirical analysis of temporal event sequence modeling to encode positional information in a Transformer architecture. The paper also explores an alternative for sinusoidal positional embedding in Transformer architecture.
strengths_and_weaknesses: **Strengths**

- The paper is clear and concise.
- The paper provides the details of their architecture, training data statistics, and hyperparameters used which is a good step toward reproducibility.
- The authors did a thought study and applied their approach to specific domains(healthcare & education).
- The paper follows ICLR paper requirements.

**Weaknesses**

- The reported experiments are good but don’t highlight the impacts of the time-decaying encoding approach.
- No code is provided to support the conducted experiments.
suggested_changes: - The paper is written well and clearly but it would be great to conduct more experiments. An example could be to show how their model works without positional information vs sinusoidal encoding vs time-decaying encoding.
- It is unclear how much model size plays with encoding type. Did the time-relation perform worse than sinusoidal embedding because the model used for time-relation is smaller than later? Conducting more experiments would perhaps clarify your findings.
- The conclusion(and discussion) is concise but it would benefit from having your major findings and how they connect with your research question.
review_title: null

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: This paper provides an empirical evaluation of the transformer architecture for a binary classification task using a real-world education-related dataset. Two encodings have been compared, namely, sinusoidal positional encoding and time-decaying encoding. The authors recall particular attention to time-decaying encoding in the context of health and education data.
strengths_and_weaknesses: Strengths: The paper provides all details on the implementation of the transformer method and the used encodings.

Weaknesses: The trade-off between performance (measure with AUC/accuracy) and Memory is not very promising.
suggested_changes: - I suggest consistently using "quotes" for T1 and T2;
- I recommend answering the following questions: Are there any limitations that the authors identified? Rather than only comparing performance and memory consumption, are there other aspects that time-decaying encoding would be a better alternative?;
- The last phrase seems incompleted (This paper presents an empirical analysis of alternative encoding.). Alternative encoding for what?

Minor comments:
- Please correct the word "Including", which is capitalized in the abstract;
- Similarly, the word "Thus" is capitalized in Section 2 within the phrase "[...] with time Ebbinghaus (2013), Thus [...]".
review_title: null

Meta-review: Although reviewers considered this submission CCR, the meta-reviewer and reviewers identified areas for improvement. Please address all questions and revise the manuscript to make improvements.
Decision: Invite to revise