Paper: TRAM: Benchmarking Temporal Reasoning for Large Language Models (ICLR-2024)

Reviewer comment (Evaluation):
The task difficulty of the TRAM benchmark is not enough to serve the purpose for ongoing evaluations of future more powerful LLMs. 8 out of 10 tasks have close to or better than 90% accuracy when evaluated on the SOTA LLM GPT-4 (5S, CoT). For the same 8 tasks, the gap between GPT-4 and Human is within 5%. This leaves very little headroom for improvement to validate future more powerful LLMs.

Author response:
We appreciate your concern about the difficulty level of our TRAM benchmark. The purpose of this benchmark is to provide a comprehensive test that covers a wide range of temporal reasoning abilities for LLMs. As detailed in our paper, most tasks consist of multiple subtasks. Although the average SOTA performance is high, among the eight tasks you mentioned, which total 34 subtasks, there are 12 out of 34 subtasks where the GPT-4 (5S, CoT) setting performs at or below 80%. For instance, in this setting, the Ambiguity Resolution (calendar-shift) task achieves 53% accuracy, the Typical Time (comparison) task 77%, and the Arithmetic (date computation) task 78%. These results underscore specific areas that require further refinement, even in SOTA models.

Moreover, the performance gap between GPT-4 and human benchmarks warrants particular attention. While this gap may seem modest (within 5%), achieving even incremental improvements is notably challenging as models approach human-level accuracy. Therefore, these seemingly small gaps are, in fact, substantial indicators of areas where future LLMs can and need to improve.

Meanwhile, to address your concern and challenge the models more rigorously, we have considered some straightforward and effective methods, including introducing additional options and modifying question formats. To test the efficacy of these methods, we conducted case studies on the Causality and Storytelling tasks, which initially showed high performance. Under GPT-4 (5S, CoT), we reevaluated 50 samples from each task that were previously predicted correctly. When the distracting choices were contextually close to the correct answer, the model's performance dropped, with 9 out of 50 incorrect predictions for Causality and 17 out of 50 for Storytelling. In contrast, when distractors were clearly differentiated from the correct answer, the model had 5 out of 50 incorrect predictions for Causality and 13 out of 50 for Storytelling.

Even in a SOTA setting, our deliberate introduction of additional options can decrease model performance by more than 10%. Additionally, converting some tasks to a short-answer format has been shown to affect performance; for example, the SOTA performance on the Arithmetic task dropped from 94.3% to 89.6%. We are dedicated to continuously refining the tasks in our benchmark and to providing a variety of question formats to meet different levels of model testing. Updates to question sets and results will be regularly shared on our GitHub page.

Reviewer comment (Evaluation):
While the benchmark is designed specific for temporal reasoning, it is not entirely clear how much failure in the errors are due to the model's lack of general reasoning capability, rather than specific to the time dimension. A more careful separation and attribution to either general reasoning and temporal reasoning failures is needed to make the benchmark really useful for gauging temporal reasoning progress.

Author response:
We designed our temporal reasoning (TeR) benchmark, recognizing the importance in understanding and reasoning about time. Although TeR is a specialized form of general reasoning, our aim is to isolate and evaluate this unique aspect due to its broad use in everyday life. In our error analysis, we distinguish between general reasoning and TeR when evaluating LLM capabilities. For example, in the Frequency task with the question ‘If a person’s job contract has a renewal every 4 years, and they started working in 1857 and renewed it 3 times without gaps, until what year is their current contract valid’, GPT-4’s error was categorized as ‘Contextual Misjudgment’. The model incorrectly assumed that each 4-year contract period concludes at the beginning of the following year, instead of at the end of the fourth year. This error demonstrates a misunderstanding in processing temporal information, rather than a general reasoning flaw. While time-related mistakes may occur within a broader reasoning context, our primary objective is to gauge a model’s progress in TeR.

Reviewer comment (Novelty):
The difference between TRAM and previous temporal reasoning benchmarks is unclear: for instance Duration and Frequency are already covered by earlier benchmarks. A more detailed and convincing comparison is needed to establish the necessity of the newly introduced TRAM benchmark.

Author response:
**Question 1: TRAM vs. existing temporal reasoning benchmarks.**

| Dataset/Benchmark | Size  | Task Coverage                                                  |
|-------------------|-------|---------------------------------------------------------------|
| TRAM              | 526.7k| Ordering, Frequency, Duration, Typical Time, Ambiguity Resolution, Arithmetic, Relation, Temporal NLI, Causality, Storytelling |
| MenatQA           | 2.9k  | Scope, Order, Counterfactual                                   |
| TEMPREASON        | 52.8k | Facts                                                          |
| TEMPLAMA          | 50k   | Facts                                                          |
| Time-Sensitive QA | 41.2k | Facts                                                          |
| TIMEDIAL          | 1.1k  | Commonsense                                                    |
| MCTACO            | 13k   | Commonsense (duration, temporal ordering, typical time, frequency, stationarity) |

We acknowledge that aspects like Duration and Frequency are indeed covered by previous benchmarks. However, as these are foundational aspects in temporal reasoning (TeR), including these tasks is necessary and essential for a comprehensive TeR benchmark. Compared to earlier works, our dataset offers a much larger data size and broader task coverage, encompassing 10 main tasks and 38 subtasks. For instance, in the classic tasks of ‘Duration’ and ‘Frequency’, earlier works mainly cover Facts and Commonsense. In contrast, our work includes 7 subtasks for ‘Duration’ and 6 subtasks for ‘Frequency’, delving into more nuanced aspects of these temporal elements, which are less emphasized by earlier works. Consequently, the extensive data size of TRAM and the intricate design of its subtasks represent a notable change in both the depth and breadth of temporal reasoning (TeR) evaluation, compared to existing benchmarks.

Reviewer comment (Evaluation):
I like good benchmarks to be hard. I'm a bit concerned that SOTA performance on this benchmark starts at 84%; this perhaps suggests that the benchmark isn't hard enough.

Author response:
We appreciate your concern about the difficulty level of our TRAM benchmark. The purpose of this benchmark is to provide a comprehensive test that covers a wide range of temporal reasoning abilities for LLMs. As detailed in our paper, most tasks consist of multiple subtasks. Although the average state-of-the-art (SOTA) performance is high, particularly for 8 out of 10 tasks with accuracies close to or exceeding 90%, there are 12 out of 35 subtasks where the GPT-4 (5S, CoT) setting performs at or below 80%. For instance, in this setting, the Ambiguity Resolution (calendar-shift) task achieves 53% accuracy, the Typical Time (comparison) task 77%, and the Arithmetic (date computation) task 78%. These results underscore specific areas that require further refinement, even in SOTA models.

Meanwhile, to address your concern and challenge the models more rigorously, we have considered some straightforward and effective methods, including introducing additional options and modifying question formats. To test the efficacy of these methods, we conducted case studies on the Causality and Storytelling tasks, which initially showed high performance. Under GPT-4 (5S, CoT), we reevaluated 50 samples from each task that were previously predicted correctly. When the distracting choices were contextually close to the correct answer, the model's performance dropped, with 9 out of 50 incorrect predictions for Causality and 17 out of 50 for Storytelling. In contrast, when distractors were clearly differentiated from the correct answer, the model had 5 out of 50 incorrect predictions for Causality and 13 out of 50 for Storytelling.

Even in a SOTA setting, our deliberate introduction of additional options can decrease model performance by more than 10%. Additionally, converting some tasks to a short-answer format has been shown to affect performance; for example, the SOTA performance on the Arithmetic task dropped from 94.3% to 89.6%. We are dedicated to continuously refining the tasks in our benchmark and to providing a variety of question formats to meet different levels of model testing. Updates to question sets and results will be regularly shared on our GitHub page.

Reviewer comment (Evaluation):
The benchmark currently is only in the form of multi-choice questions.

Author response:
We employ multiple-choice questions (MCQs) in TRAM to align with established practices in LLM benchmarks, such as the AI2 Reasoning Challenge (ARC) and Massive Multitask Language Understanding (MMLU), which also utilize MCQs. Additionally, the MCQ format is the most straightforward method for evaluating LLMs. Nevertheless, we recognize the potential limitations of this format. On our GitHub page, we have provided an alternative version of the benchmark in a short-answer format where appropriate, except for tasks like storytelling, ambiguity resolution, and causality. For these three types of questions, I have retained the MCQ format, as other formats could lead to answers that are too open-ended and difficult to evaluate. As part of our ongoing research, we remain open to incorporating more diverse question formats and adding more challenging questions to each task. These updates will be made available on our GitHub page, adapting to the evolving needs of the research community and the developing capabilities of LLMs.

Reviewer comment (Evaluation):
The sizes of different categories of problems are imbalanced. For example, causality is of only 600 problems. This might render the benchmarking evaluation results misguiding, especially for the pretraining-finetuning paradigms.

Author response:
Our causality task is inspired by the Choice of Plausible Alternatives (COPA) dataset, which itself comprises only 1,000 questions. Given that COPA has set a precedent in the field, particularly for tasks centered around causal reasoning, the number of questions it contains serves as a benchmark for what can be considered sufficient for a robust evaluation. Furthermore, 93.5% of the questions in our original causality task are meticulously crafted by hand. This manual process, while ensuring the precision and applicability of each question, also limits the feasible scale of our dataset. To address the limited size of the causality category, we have added an additional, mirrored instance for each original instance. This expands our causality task to a total of 1,200 questions. Each mirrored instance uses the same alternatives as the corresponding original instance but introduces a new premise that matches the incorrect alternative of the original instance. We have updated the relevant information on this specific task in our revised paper, mainly including updated BERT-based results for causality and the task description. We are committed to continuously refining and potentially expanding this task, with updates to the question set being regularly posted on our GitHub page. This effort is specifically aimed at mitigating issues associated with the pretraining-finetuning paradigms, particularly those arising from limited training samples.

Reviewer comment (Writing):
For the terms "commonsense", "analogy inference", "comparison" and so on, it would be better to have a formal definition and ensure that the datasets and the reasoning follow the formal definitions with verifiable criteria (automatically verifiable would be even better).

Author response:
Following your suggestion, we have added a glossary with definitions of all tasks and subtasks in our benchmark to the revised paper (Appendix E, pages 22-23).

Reviewer comment (Experiments):
Page 7, it would be better to include a more comprehensive list of the settings for human expert annotators evaluation. For example, how the experts are drawn from the population, how to ensure they are capable experts or their level of expertise. How will an ordinary person performance compare to experts? Such a systematic study of human experiments might also provide hints for comparing human performance variation with LLM performance variation.

Author response:
Following your suggestion, we have added more details about human experts in our revised paper (Appendix A, page 12).

**Selection of Expert Annotators**: Our selection criteria for expert annotators emphasized expertise in temporal reasoning as well as quantitative analysis. This necessitated including professionals with advanced degrees (M.S. or Ph.D.) in cognitive science and psychology, for their insights into the nuances of human temporal cognition, and in statistics, mathematics, and computer science, for their ability to handle the analytical rigor required by many tasks in our benchmark.

**Expertise Verification**: We conducted a comprehensive screening process to ensure high levels of expertise. This process involved validating educational qualifications, reviewing related professional or research experience in time perception and quantitative problem-solving, and administering a preliminary assessment involving temporal reasoning problems. One random problem per subtask (38 problems in total) was used, and the criterion for passing the screening was an average accuracy of more than 92% (a maximum of three incorrect problems).

**Comparison with Unspecialized Individuals**: We also assessed multiple human non-specialists using our benchmark sourced from Amazon Mechanical Turk. They responded to the same set of 1,900 questions as the experts. The accuracy rate of these non-specialists across all tasks was 62.5%.

Reviewer comment (Reproducibility):
Page 18, appendix B, for self-contained please detail a little bit more for SP with examples. The whole section is essentially about CoT with little information regarding how SP is constructed.

Author response:
Thank you for pointing out the missing pieces. In accordance with your suggestions, we have added more details about SP with corresponding examples on Page 18 (Appendix B) of our revised manuscript.

Reviewer comment (Reproducibility):
As the paper primarily focuses on the area of datasets and benchmarks in large language models, it is better to provide an anonymous GitHub page, for example (https://anonymous.4open.science/) with code for dataset curation and empirical evaluation, as well as simple documentation on running the LLM’s assessments.

Author response:
**Weakness 1: GitHub page.**

Following your suggestion, we have set up an anonymous GitHub page. Here is the link: https://anonymous.4open.science/r/TRAM-Benchmark-596D/.

Reviewer comment (Experiments):
At this point, the overall contribution of the dataset curation done by the authors is unclear. It is better to provide some examples for comparing the differences between the source dataset and the provided curated dataset. With some manual comparison between the shared supplementary materials and the source dataset repos (https://github.com/CogComp/MCTACO/tree/master) and (https://rajpurkar.github.io/SQuAD-explorer/), the author seems to simply reformulate multiple original Yes/No questions into one multiple-choice question. For example, in the ‘frequency’ task, the original questions from MCTACO look like the following sequence of Yes/No questions and answers (Q1–Q8). However, the provided dataset in this paper takes these 8 original questions and reformulates them in frequency.csv as one question with three candidate answers, where some answers (e.g., Answer B: "he says for example") do not correlate with time, which is caused by errors from the original data source. It’s better to provide an overall evaluation of the data quality and to explain the advantages of converting several existing Yes/No questions to one multiple-choice question.

Author response:
**Weakness 2/Question 2: Overall contribution of the work.**

 Thank you for your insightful suggestion about clarifying the contribution of our work and the rationale behind the transformation of existing data sources into the TRAM dataset. The following table provides a comparison between our TRAM and previous temporal reasoning datasets, including task coverage and data size. The order of the prior work listed is from the most recent to the earlier ones.

| Dataset/Benchmark | Size  | Task Coverage                                                  |
|-------------------|-------|---------------------------------------------------------------|
| TRAM              | 526.7k| Ordering, Frequency, Duration, Typical Time, Ambiguity Resolution, Arithmetic, Relation, Temporal NLI, Causality, Storytelling |
| MenatQA           | 2.9k  | Scope, Order, Counterfactual                                   |
| TEMPREASON        | 52.8k | Facts                                                          |
| TEMPLAMA          | 50k   | Facts                                                          |
| Time-Sensitive QA | 41.2k | Facts                                                          |
| TIMEDIAL          | 1.1k  | Commonsense                                                    |
| MCTACO            | 13k   | Commonsense (duration, temporal ordering, typical time, frequency, stationarity) |

Compared to earlier works, our dataset offers a much larger data size and broader task coverage, encompassing 10 main tasks and 38 subtasks. For instance, in the classic tasks of ‘Duration’ and ‘Frequency’, earlier works mainly cover Facts and Commonsense. In contrast, our work includes 7 subtasks for ‘Duration’ and 6 subtasks for ‘Frequency’, delving into more nuanced aspects of these temporal elements, which are less emphasized by earlier works. Consequently, the extensive data size of TRAM and the intricate design of its subtasks represent a notable change in both the depth and breadth of temporal reasoning (TeR) evaluation, compared to existing benchmarks.

Following your suggestion, we have included examples in Appendix F (pages 23-24) that compare our curated dataset with the source datasets, specifically MCTACO and SQuAD. For information on data quality, please refer to the datasheets available on our anonymous GitHub page. The reformulation of original data sources into the TRAM benchmark offers several advantages:

**Increased Complexity and Depth**: By converting simple Yes/No questions to multiple-choice format, we inherently increase the complexity of each question. This change demands that models not only identify the correct answer but also discern and reject multiple plausible but incorrect options, thereby enhancing the depth of temporal reasoning evaluation.

**Reduction in Guesswork**: The multiple-choice format reduces the likelihood of correct guesses compared to Yes/No questions, demanding a more nuanced understanding and reasoning, and thus providing a clearer assessment of a model's temporal reasoning capabilities.

In summary, we have chosen to use existing benchmarks due to their wide acceptance within the NLP community. The multiple-choice format, effective for evaluating LLMs as seen in benchmarks like the AI2 Reasoning Challenge (ARC) and Massive Multitask Language Understanding (MMLU), introduces more complexity for the model and challenges it to discern subtle differences in temporal reasoning contexts. Our benchmark combines the strengths of existing benchmarks with extensive manual work, setting a new standard for assessing temporal reasoning in LLMs. To address the evolving needs of advanced model evaluation, we will regularly share updates to our question sets and results on our GitHub page.

Reviewer comment (Experiments):
It is better to provide an overall description of the dataset following the datasheet for datasets [1] (or other similar sources), as this may address most of the concerns.

Author response:
**Weakness 3/Question 1: Overall dataset description.**

Thank you for providing the insightful advice. Following your suggestions, we have created a datasheet for the TRAM benchmark, which is available on the anonymous GitHub page (datasheet_for_TRAM_benchmark.pdf). Please let us know if there are any additional aspects you think would be worthwhile to incorporate into the datasheet. We are open to suggestions and will gladly update it accordingly.

Reviewer comment (Experiments):
The experimental results show a disparity in performance across different Large Language Models (LLMs), and the integration of chain-of-thought prompting results in only minor improvements. GPT-4 appears to outperform every other model by a large margin. It is unclear whether the provided model is already trained on the source dataset (MCTACO) from which this benchmark dataset is derived. For example, GPT-3.5 did not provide information about MCTACO when prompted about it, suggesting unfamiliarity, whereas GPT-4 gave a detailed description of the dataset. It is better to provide some justifications for this disparity and the potential prior exposure of some models to the source dataset.

Author response:
**Weakness 4/Question 3: Performance disparity among LLMs.**

Thank you for your feedback regarding the use of questions from existing datasets in our benchmark. We acknowledge this as a limitation of our work and understand the issues it might pose, particularly in terms of performance disparity among different LLMs. Similar to the GLUE and SuperGLUE benchmarks, our adoption of pre-existing datasets is based on their implicit acceptance within the NLP community. For questions sourced from existing datasets, we conducted significant preprocessing and reformulation to align them with the specific objectives and format of TRAM. Our preprocessing involved adapting the questions to effectively test temporal reasoning capabilities and reformatting them to fit the structure of our benchmark. This process included manually adding distracting or confusing options, filtering out irrelevant questions for quality control, and reformulating problems. The performance disparity, especially the superior results of GPT-4, can be attributed to varying architectures, training dataset scales, and sizes of the models. While GPT-4 may have been exposed to these datasets during pretraining, our manual work in reformulating the tasks aims to introduce new dimensions and challenges, thus minimizing any potential advantage. To address both this issue and the need for advanced LLM evaluation, we have undertaken the following actions and plans:

1.  We have conducted some case studies on tasks in our benchmark, including Causality and Storytelling, by a) adding additional options and b) modifying question formats. The reason for experimenting with a) is to challenge models, potentially confuse them, and test the nuanced reasoning capabilities of LLMs. On the other hand, b) is implemented to avoid the possibility that models can guess instead of truly reasoning. These direct methods have proven effective in our case studies, as demonstrated by the decrease in model performance. Meanwhile, even if models like GPT-4 have been exposed to existing datasets during pretraining, we can still present additional challenges, such as confusing options and challenging problem formats, to test their abilities.

2. We will prepare and craft more challenging questions for our benchmark, such as reframing existing tasks by introducing novel contexts or counterfactual elements, and introducing a set of novel tasks that require non-linear temporal reasoning. These tasks might include analyzing events in reverse chronological order or understanding cyclical time concepts, which are less likely to be present in standard training datasets. We are dedicated to continuously refining the tasks in our benchmark and providing a variety of question formats to meet different levels of model testing. Updates to question sets and results will be regularly shared on our GitHub page.

Reviewer comment (Experiments):
There are many specifically designed models to solve temporal reasoning. None of these models are included in the benchmarks. Without these, it is difficult to compare results between LLMs and RoBERTa or BERT. What goodness that LLMs bring in which tasks compared to these special models which are smaller compared to LLMs? [1] Yuan, Weizhe, and Pengfei Liu. "reStructured Pre-training." (2022) [2] Ben Zhou, Kyle Richardson, Qiang Ning, Tushar Khot, Ashish Sabharwal, and Dan Roth. (2021). "Temporal Reasoning on Implicit Events from Distant Supervision."

Author response:
Thank you for highlighting the need for a comparison with domain-specific models. Following your suggestion, we have conducted additional experiments using a temporal reasoning model (https://huggingface.co/GAIR/rst-temporal-reasoning-11b), referred to as RST-TeR below, where the training and testing sets are the same as those used for BERT-based models. We have also included results from RoBERTa-large and GPT-4 for reference and comparison:

| Model            | Order | Freq. | Dur. | Typical Time | Amb. Res. | Arith. | Rel.       | NLI        | Caus.      | Story | Average | 
|-------------------|-------|------|------|--------------|-----------|--------|------------|------------|------------|-------|---------|
| GPT-4 (5S, CoT)  | 71.5        | 93.7 | 93.0         | 90.2      | 89.8   | 94.3       | 69.5/69.1  | 90.7/91.0  | 100.0 | 96.3    | 87.4       |
| RoBERTa-large    | 55.5        | 57.7 | 55.4         | 60.0      | 41.0   | 29.1       | 90.0/90.0  | 70.0/70.3  | 88.0  | 84.0    | 65.9       |
| RST-TeR          | 54.5        | 56.2 | 52.3         | 58.7      | 39.8   | 31.6       | 91.5/91.6  | 68.2/68.7  | 87.5  | 82.2    | 65.2       |

Based on these results, RST-TeR achieves performance on par with the RoBERTa-large model. However, there is a significant gap between this domain-specific model and LLMs like GPT-4, except in the Relation (Rel.) task, where RST-TeR is specifically trained. GPT-4’s extensive training across varied contexts contributes to its robust performance, showcasing its superior generalization capabilities. Overall, LLMs bring depth and versatility to temporal reasoning, enabling them to handle a range of tasks that demand a comprehensive and integrative approach, an aspect where specialized models may be more limited.

Reviewer comment (Experiments):
How does the length of questions in MCQ affect performance? And adding more options, will it confuse models?

Author response:
Thank you for raising the insightful question about how the length of questions and the number of options in MCQs affect model performance. Here are the statistics for the average length of questions, options, and their combined average for each task:

| Category             | Avg. Question Length | Avg. Option Length | Avg. Combined Length |
|----------------------|----------------------|--------------------|----------------------|
| Ordering           |        56.8                 | 7.8                | 64.6                 |
| Frequency            | 25.3                 | 9.3                | 34.6                 |
| Duration             | 41.7                 | 8.0                | 49.7                 |
| Typical Time         | 98.1                 | 4.6                | 102.7                |
| Ambiguity Resolution | 20.7                 | 5.2                | 25.9                 |
| Arithmetic           | 13.3                 | 6.9                | 20.2                 |
| Relation             | 53.9                 | 3.3                | 57.2                 |
| NLI                  | 44.1                 | 3.0                | 47.1                 |
| Causality            | 14.2                 | 16.5               | 30.7                 |
| Storytelling               | 48.3                 | 17.7               | 66.0                 |

Based on our experiments, we have the following observations:

**Length of questions vs. Performance**: There is no direct correlation between question/option length and model performance. For instance, tasks with longer questions such as Typical Time and Storytelling have some of the highest accuracies, indicating that the additional context from longer questions can be beneficial. Conversely, shorter tasks like Arithmetic and Ambiguity Resolution do not necessarily result in higher performance, suggesting that brevity, which may reduce context, does not automatically improve performance. For the tasks in our study, as there is no substantial difference in length, the intrinsic task complexity appears to have a more pronounced effect on the final performance rather than the question length.

**Length of options vs. Performance**: Longer option lengths in tasks like Causality and Storytelling do not lead to uniform decreases in performance, implying that detailed options may provide valuable context that helps the model deduce the correct answer. However, if the added length introduces ambiguity or irrelevant details, it could confuse the model.

**Number of options vs. Performance**: Based on our manual preprocessing procedure for constructing distracting options and the experimental results, we find that adding more options can indeed confuse the model. Considering the complexity differences among various tasks, we additionally conducted case studies on tasks that initially performed well, specifically on Causality and Storytelling tasks that lacked a sufficient number of options. We used GPT-4 to evaluate 50 samples for each task that were originally predicted correctly. We considered two scenarios. First, if the distracting choices are closely related to the correct answer contextually, the model incorrectly predicted 9 out of 50 for Causality and 17 out of 50 for Storytelling. Conversely, if there is a clear distinction between the distractions and the correct answer, the model incorrectly predicted 5 out of 50 for Causality and 13 out of 50 for Storytelling. Thus, we conclude that option similarity has a more significant impact on performance than the mere number of options.