Timo: Towards Better Temporal Reasoning for Language Models

Authors: zhaochen  su
jun  zhang
tong  zhu
  xiaoye qu
juntao  li
min  zhang
yu  cheng
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-22 09:12:10

Abstract: Reasoning about time is essential for Large Language Models (LLMs) to understand the world. Previous works focus on solving specific tasks, primarily on time-sensitive question answering.
While these methods have proven effective, they cannot generalize to a wider spectrum of temporal reasoning tasks.
Therefore, we propose a crucial question: Can we build a universal framework to handle a variety of temporal reasoning tasks?
To that end, we systematically study 38 temporal reasoning tasks.
Based on the observation that 19 tasks are directly related to mathematics, we first leverage the available mathematical dataset to set a solid foundation for temporal reasoning.
However, the in-depth study indicates that focusing solely on mathematical enhancement falls short of addressing pure temporal reasoning tasks. To mitigate this limitation, we propose a simple but effective self-critic temporal optimization method to enhance the model's temporal reasoning capabilities without sacrificing general task abilities.
Finally, we develop Timo, a model designed to excel in temporal reasoning at the 7B and 13B scales. Notably, Timo outperforms the counterpart LLMs by 10.0 and 7.6 in average accuracy scores and achieves the new state-of-the-art (SOTA) performance of comparable size. Extensive experiments further validate our framework's effectiveness and its generalization across diverse temporal tasks. The code is available at https://github.com/zhaochen0110/Timo.

Main paper: 
 introduction: 
 Large Language Models (LLMs) have achieved remarkable success in various reasoning tasks (Zhao et al., 2023; Chang et al., 2023; Su et al., 2024) , such as mathematical, commonsense, and symbolic reasoning. Despite these advances, LLMs face significant challenges in temporal reasoning (Chen et al., 2021; Tan et al., 2023a) , which is crucial in human perception. Compared to other reasoning tasks that focus solely on one specific reasoning ability, temporal reasoning is an integrated task that requires arithmetic (Zhu et al., 2023a) , logic (Mishra et al., 2022a) and world knowledge (Wei et al., 2022) .Prior efforts to improve the temporal reasoning capacity of LLMs focus mainly on timesensitive question-answering (Chen et al., 2021) , and utilize methods such as step-by-step reasoning (Zhu et al., 2023b; Li et al., 2023) and ruled-based supervised fine-tuning (SFT) (Tan et al., 2023a; Yuan et al., 2023b) . More recent studies expand the scope of temporal tasks to include basic temporal concepts understanding (e.g., duration), intricate temporal interpretations (e.g., relation) and computations (e.g., arithmetic) (Wang & Zhao, 2023) . Due to their task-specific nature, the aforementioned methods exhibit limited generalization across the wider spectrum of temporal tasks. To address these limitations, we explore a crucial question: Can we build a universal framework to handle various temporal reasoning tasks? To tackle this, we face the following challenges: (1) integrating different temporal reasoning tasks into a unified framework; (2) generating and selecting the high-quality training dataset automatically; (3) improving the comprehensive temporal reasoning abilities while maintaining its general performance.In response to these challenges, we first systematically study 38 subtasks within the temporal reasoning benchmark proposed by Wang & Zhao (2023) . As shown in Figure 1 , our analysis reveals that 19 tasks are directly related to mathematical reasoning (i.e., mathematical time tasks). For example, when "identifies the next leap year following 2024", mathematical skills are required to calculate the results. The rest are categorized as pure temporal reasoning tasks, focusing solely on temporal reasoning without using mathematical abilities. Meanwhile, mathematical reasoning stands out with its diverse and rich instruction tuning datasets compared to temporal reasoning (Cobbe et al., 2021; Mishra et al., 2022b; Yue et al., 2023) . Therefore, it is intuitive to build a generalist temporal reasoning framework based on math-enhanced LLMs, setting a solid foundation for temporal reasoning skills. However, our in-depth study indicates that focusing solely on mathematical enhancement through supervised fine-tuning falls short of addressing pure-time tasks. To bridge this gap, we further introduce a simple but effective method to obtain comprehensive temporal reasoning abilities. Specifically, we propose a self-critic method to generate and select the high-quality temporal preference pairs, which are then utilized for enhancing model temporal capabilities through preference optimization. Finally, we propose a unified temporal reasoning framework, namely Timo. With this framework, our model achieves superior performance among 38 temporal tasks, as depicted in Figure 2 .In our experiments, we train LLaMA2 models at both 7B and 13B scales with our framework, which results in Timo-7B and Timo-13B. These two models demonstrate a substantial improvement of 10.0 and 7.6 in average accuracy scores over the base models, respectively. Our comprehensive analysis indicates that our framework successfully integrates substantial mathematical knowledge along with temporal information. Extensive experiments further verify the effectiveness of our method in preserving general task capabilities and maintaining robustness under different scenarios. To sum up, our contributions are shown below:• We systematically study a variety of temporal reasoning tasks and discover the inner correlation between time and mathematics, where temporal reasoning could benefit from mathematics instructions; • We make the first attempt to build a unified framework to address 38 temporal tasks.Specifically, upon mastering mathematical reasoning capabilities, we propose a simple but effective self-critic temporal optimization method to strengthen the temporal reasoning capabilities comprehensively; • The proposed framework outperforms 10.0 and 7.6 scores over the baselines, establishing as the new SOTA model of comparable sizes. Besides, our models consistently enhance the temporal reasoning capabilities without sacrificing general task performance. 
analysis on temporal reasoning benchmark: 
 Wang & Zhao (2023) provides a comprehensive collection of 38 subtasks centered around temporal reasoning tasks. It is widely observed that a substantial portion of these tasks relies on mathematical skills for calculating and reasoning about time. For example, within the Frequency category, the Computation subtask requires the calculation of event frequencies or intervals. In the Ambiguity Resolution task, mathematics provides a standardized method of time representation, such as the 24-hour format and date calculation formulas, enabling different temporal expressions to be accurately understood and converted. Based on these observations, we categorize temporal tasks into two categories. The specific subtasks under each category are shown in Figure 1 . Below is our classification:• Mathematical Time Tasks (Math-time tasks): These are temporal reasoning tasks that necessitate mathematical skills, such as calculating time frequencies, converting time shifts, comparing time sequences, determining time intervals, estimating durations, and so on. This category encompasses a total of 19 subtasks. • Pure Time Tasks (Pure-time tasks): These tasks require only temporal reasoning abilities for resolution and include reasoning about temporal commonsense, applications in realworld scenarios, temporal natural language inference (NLI) and so on. This category also contains 19 subtasks.Inspired by Wei et al. (2022) , we construct Math-CoT for each temporal task to establish a connection between mathematics and temporal tasks. We utilize the MathInstruct dataset (Yue et al., 2023) , which comprises a diversified collection of mathematical problems with detailed rationales. From this dataset, we select five mathematical question-CoT pairs and employ GPT-4 to generate Math-CoT rationales by mimicking mathematical reasoning. Since pure-time questions lack mathematical rationales, Math-CoT is specifically designed for math-time tasks. We compare Math-CoT with two prompting methods: (1) Few-shot, which samples five question-answer pairs per task, and (2) CoT (Wei et al., 2022) , where GPT-4 is used to generate step-by-step rationales for each task. We conduct the experiments using LLaMA2-7B under the 5-shot setting and report the accuracy for each task. As shown in Figure 3 , integrating mathematical reasoning into temporal tasks leads to a significant enhancement in model performance, with Math-CoT outperforming traditional prompting methods in all math-time tasks.Given the established correlation between mathematics and temporal reasoning, it is intuitive to instruct models in mastering mathematical reasoning to establish a solid foundation for advanced temporal reasoning abilities. This connection motivates our investigation into how varying degrees of mathematical instruction influence model performance. Specifically, we select 180k mathematical CoT rationales from the MathInstruct and perform scaling experiments by fine-tuning the LLaMA2-7B with different volumes of math instructions (i.e., 0, 50k, 100k, 150k, and 180k). We evaluate the models on both math-time tasks and puretime tasks under the 5-shot setting. The results are shown in Figure 4 . After supervised fine-tuning on 50k math instruction tuning instances, the model exhibits a notable improvement in performing math-time tasks, with accuracy increasing from 56.4 to 63.3. However, It is worth noting that this enhancement in mathematical skills has a minimal impact on pure-time tasks, with a maximum enhancement of 1.9. Additionally, our analysis indicates a declining trend in performance across both task categories as the volume of math instructions increases. We believe this decline results from overfitting to mathematical tasks due to excessive data, adversely impacting the model's temporal reasoning capability (Mishra et al., 2022a) . 
self-critic temporal task optimization: 
 In the previous section, we discovered that focusing solely on mathematical enhancement falls short of addressing pure-time tasks. To mitigate this limitation, we introduce a simple but effective self-critic optimization framework to equip the model with comprehensive temporal reasoning abilities. The pipeline of our proposed framework is detailed in Figure 5 .Given the mathematical model L, we start by generating a set of N candidate responsesY i = {y 1 i , y 2 i , . . . , y N i } for each input prompt x i .Given the golden label g i for each prompt x i , we divide Y i into the correct response set R + i and the incorrect response set R - i :EQUATIONwhere align(y n i , g i ) is a function that returns true if the response y n i aligns with the golden label g i , and false otherwise. Inspired by the LLM-as-a-Judge mechanism (Zheng et al., 2023; Yuan et al., 2024; Qu et al., 2024b) , we utilize mathematical model L directly as a reward model to identify high-quality response pairs. Notably, we introduce a novel hierarchical scoring method, which is specifically designed for evaluating responses to temporal tasks and contains five key aspects: (1) relevance and basic temporal reasoning;(2) understanding of temporal aspects; (3) application of internal temporal knowledge; (4) direct and well-organized addressing of the question; (5) insightfulness and advanced reasoning. To choose the higher quality pair from the correct set R + i and wrong set R - i , wePublished as a conference paper at COLM 2024 prioritize the response that utilizes the model's temporal reasoning to the fullest extent. The criteria for our evaluation prompts are illustrated in Figure 10 and 11. For each criterion a response meets, a point is awarded. We prompt the model L to assign a score r n i ∈ [0, 5] to each response y n i , quantifying its quality across the above dimensions. The temporal preference pair (y + i , y - i ) is formed by selecting the top-scoring response from the correct set R + i as y + i and from the incorrect set R - i as y - i . We then utilize these pairs to perform direct preference optimization (DPO) by optimizing the following loss function:EQUATIONwhere y + i is favored over y - i , and β is a hyperparameter. 
experimental setup: 
 Training Setup We use LLaMA2 7B and 13B (Touvron et al., 2023) as our base pre-trained model. For SFT, we select 100k instances from MathInstruct (Yue et al., 2023) , the most representative dataset for mathematical reasoning instruction tuning. For self-critic temporal optimization, we focus on pure temporal reasoning tasks, which encompass 19 subtasks. We reserve 100 instances for evaluation and utilize the remaining data for training. If a subtask contains fewer than 5,000 samples, we maintain all of them. Otherwise, we randomly select 5,000 instances. In total, we use 35,655 instances for optimization.We conduct a comprehensive evaluation across all temporal reasoning tasks, encompassing a total of 38 tasks. Following Tan et al. (2023a) , we assess the model performance on 100 examples for each task, amounting to a total of 3,800 instances. Consistent with prior work (Qu et al., 2024a; Xia et al., 2024) , we evaluate the model's temporal abilities under the 5-shot setting and utilize greedy decoding (i.e., temperature = 0) for generating model's responses. We extract the prediction from the response and calculate the accuracy of each subtask.We utilize four/eight NVIDIA Tesla A100 GPUs to train models.To facilitate parallel training, we employ DeepSpeed Zero-Stage 3 (Ren et al., 2021) and FlashAttention2 (Dao, 2023) . For SFT, we use a learning rate of 2e-5, a batch size of 128, and a cosine scheduler with a 3% warm-up period for 2 epochs. For candidate response generation, we sample N = 5 candidate responses with temperature T = 0.8, p = 0.95. When evaluating candidate responses, as there is variance to these scores, in our experiments we also use sampled decoding (with the same parameters) and generate these evaluations multiple (3) times and take the average. For DPO, we follow the hyper-parameters from Tunstall et al. ( 2023) with a batch size 32, learning rate 5e-7, a warm ratio of 0.1 using linear warmup scheduler for 9 epochs.To ensure the fairness of the experiments, we select baseline models built upon the foundational model LLaMA2. The baselines are selected based on the following dimensions:• LLMs for Temporal Reasoning: TimeLLaMA (Yuan et al., 2023b) utilizes the proposed Reinforcement Learning from Evol-Instruct Feedback (RLEIF) (Xu et al., 2023) to enhance its mathematical reasoning capabilities. (3) ToRA (Gou et al., 2024) , a series of Tool-integrated Reasoning LLM Agents, is designed to solve challenging mathematical reasoning problems. • LLMs for Code Generation: Previous work indicates that the usage of code enhances the model's ability to solve reasoning tasks (Gao et al., 2023) . We select the following popular code models as our baselines: (1) CodeLLaMA (Roziere et al., 2023) , a family of LLMs for code generation and programming-related tasks.(2) WizardCoder (Luo et al., 2023b ) is similar to WizardMath and adapts the RLEIF method within the domain of coding.Table 1 presents the results of Timo across 38 temporal reasoning tasks. Results categorized by Math-time and Pure-time tasks are shown in Appendix G in Table 9 and Table 10 . From the results, we observe: (1) Timo surpasses counterpart LLMs in average accuracy of 10.0 and 7.6 scores, and outperforms other competitive math-solving and code-solving models with a clear margin, achieving the SOTA results on average. We also discover that Timo-7B consistently outperforms all 13B models in average performance, achieving a maximum performance gain of 7.1. (2) Mathematical models do not show significant advantages in solving math-related tasks. This phenomenon is also observed in LLMs enhanced for coding abilities and temporal prediction capabilities. It indicates that excessive training on specific abilities leads the model to overfit in task-centric enhancements, diminishing its performance in other areas (Jha et al., 2023) .(3) It is worth noting that Timo underperforms MAmmoTH in the Arithmetic task (i.e., scoring 66.3 vs 67.0) when evaluated under the 13B model size parameter. The superior performance of MAmmoTH can be attributed to its advanced general math-solving abilities, which facilitate more accurate computations in time-related scenarios. However, other mathematical models like ToRA and WizardMath do not achieve the same effectiveness in handling the Arithmetic task. A detailed case study for illustration is in Appendix B.  
further analysis on our framework: 
 In our framework, we initially train a mathematical model, i.e., MathLLaMA. Then, we optimize its pure temporal reasoning abilities to derive the final Timo model. In this section, we first compare the performance of these two stages. Then, we delve into these models through the lens of token distribution shift and detailed case analysis.We compare the model's performance on both mathtime tasks and pure-time tasks. The results are shown in Figure 6 . Compared to the foundational model LLaMA, MathLLaMA demonstrates superior performance in mathrelated tasks and surpasses the LLaMA in the majority of pure-time tasks, achieving higher scores in 6 out of 9 tasks. This improvement is attributed to the advanced logical and reasoning skills developed through mathematical instruction tuning (Mishra et al., 2022a) .When compared to Timo and MathLLaMA, our framework demonstrates strong generalization capabilities, achieving significant improvement in pure-time tasks, with only minimal performance degradation in the arithmetic task. Additionally, it is worth noting that Timo outperforms MathLLaMA in various math-time tasks (i.e., Ambiguity Resolution, Duration and Frequency). This improvement is attributed to our framework's ability to learn generalized temporal features.To understand the learning process and the differences between the different stages of our framework, we follow the methodology proposed by Lin et al. (2024) to analyze through the lens of token distribution shift. We analyze three pairs of models at the 7B scale: LLaMA vs MathLLaMA, MathLLaMA vs Timo, and LLaMA vs Timo. The results are shown in Figure 7 . Notably, we observe the largest token distribution shift when transitioning from LLaMA to Timo. Furthermore, we investigate the top 200 most frequently shifted tokens, labeling math-related tokens in red and time-related tokens in green. The transition from LLaMA to MathLLaMA primarily features changes in math-related tokens. Conversely, the switch from MathLLaMA to Timo is characterized by the presence of time-related tokens. When compared to LLaMA, Timo exhibits shifts in both math-related and time-related tokens, demonstrating a profound capacity to integrate substantial mathematical knowledge along with the temporal information.Case Analysis As shown in Table 2 , we present a case analysis to provide a clear and intuitive demonstration of Timo's superior performance. In math-time tasks, both MathLLaMA and Timo effectively integrate temporal knowledge with computational capabilities to give the correct CoT and answer. However, LLaMA produces an incorrect result due to the Figure 7 : Token distribution shift on different stages of our proposed temporal task optimization method. The ratios of unshifted, marginal, and shifted tokens are colored (%).Frequently shifted tokens are shown below, where math-related tokens are labeled in red and time-related tokens are highlighted in green.error in time calculation, which indicates the importance of mathematical skills in solving math-time tasks. We also In our provided case of the pure-time tasks, both MathLLaMA and LLaMA fail to grasp the sequence of events, i.e., the timing of Amy's laundry activities.On the other hand, Timo demonstrates a strong understanding and application of temporal reasoning, accurately tracking the sequence and timing of Amy's activities and giving the correct answer. Overall, these cases vividly demonstrate Timo's comprehensive capabilities in temporal reasoning across different temporal task types. 
more detailed study: 
 Performance Comparison between Timo and OpenAI GPT Models We compare Timo-13B with the current most powerful LLMs, i.e., GPT-3.5 and GPT-4. Specifically, we use the gpt-3.5-turbo-1106 and gpt-4-1106-preview and set the temperature to 0 for consistent evaluation. The results are shown in Figure 6 . Despite its relatively small size of 13B parameters, Timo demonstrates impressive performance on pure-time tasks, outperforming GPT-3.5 in 7 out of 9 tasks and surpassing GPT-4 in 5 out of 9 tasks. Notably, Timo exceeds GPT-4 by a significant margin of 38 accuracy scores in the Relation task. Although there has been a significant improvement in pure-time tasks, the performance on math-time tasks suggests that there is still room for further enhancement. This is attributed to the foundational model's weaker mathematical reasoning capabilities. We leave it as future work to further improve the model's temporal reasoning abilities by better integrating mathematics capabilities. In our framework, we design a series of criteria to assess the standard of responses and obtain high-quality temporal preference pairs. To verify the effectiveness of our criteria, we compare our prompting approach with the widely adopted self-rewarding strategy (Yuan et al., 2024) and the random selection strategy. As shown in Table 3 , our strategy outperforms others in both math-time and pure-time tasks, highlighting its superiority in evaluating the quality of generated responses across different types of temporal challenges. Calculating the weeks from January 1st to October 27th, it's 27 days. Dividing 27 by 7 places October 27th in the 4th week. However, depending on the week starting day and system used, it can be considered as the 39th week. The answer is (B).We calculate the total days from January 1st to October 27th, which amounts to 31 (Jan.) + 28 (Feb.) + 31 (Mar.) + 30 (Apr.) + 31 (May.) + 30 (Jun.) + 31 (Jul.) + 31 (Aug.) + 30 (Sept.) + 27 (Oct.), which equals 300 days. Dividing 300 by 7, we find that October 27th falls into the 43rd week of the year. The answer is (A).Calculating the weeks from January 1st to October 27th, the total days are 31 (Jan.) + 28 (Feb.) + 31 (Mar.) + 30 (Apr.) + 31 (May.) + 30 (Jun.) + 31 (Jul.) + 31 (Aug.) + 30 (Sept.) + 27 (Oct.), summing up to 300 days. Dividing 300 by 7 places October 27th in the 43rd week of the year. The answer is (A).Table 2 : A case study for math-time tasks to intuitively show the effectiveness of Timo with temporal direct preference optimization. The right and wrong steps are colored by blue and red, respectively. An example of pure-time tasks is shown in Table 11 .ure 8. The experimental results indicate that the trained models consistently outperform in handling time-related tasks compared to their corresponding mathematical models, highlighting our method's capability to enhance temporal reasoning across different mathematical training backgrounds.To verify the model's ability to retain its original capabilities, we utilize the lm-evaluation-harness (Gao et al., 2021) to evaluate its performance on six typical downstream tasks: 5-shot MMLU (Hendrycks et al., 2020) , 25-shot ARC Challenge (Clark et al., 2018) , 5-shot GSM8K (Cobbe et al., 2021) , 10-shot HellaSwag (Zellers et al., 2019) , 5-shot Winogrande (Sakaguchi et al., 2021) and 0-shot TruthfulQA (Lin et al., 2022) . In addition to comparing with LLaMA and MathLLaMA, we introduce Timo-SFT, which mirrors our framework in all aspects except for its training methodology. Specifically, Timo-SFT is supervised fine-tuned with the chosen responses in the selected preference pairs. The results are shown in Figure 9 . We surprisingly discover that Timo outperforms other baselines in the reasoning and general knowledge ability tasks. Error analysis shows that our model aligns with the base model for 97% of the correct responses. This consistency indicates that our Timo effectively preserves general task knowledge, demonstrating remarkable generalization capabilities. 
related work: 
 Temporal Reasoning in LLMs Time is a crucial dimension in our physical world (Lazaridou et al., 2021; Su et al., 2022; 2023; Zhao et al., 2024) . Despite the advanced capabilities of LLMs in various tasks, their reasoning abilities are still underdeveloped (Su et al., 2024; Qiao et al., 2023; Huang & Chang, 2023; Sun et al.; Chu et al., 2023) . Temporal reasoning, which is fundamental for humans to understand the world, is an important task in reasoning and has gained substantial research focus (Pustejovsky, 2003; UzZaman et al., 2012; Huang et al., 2024) . However, existing works often specialize in limited aspects of temporal reasoning, such as frequency (Zhou et al., 2019) , duration (Zhang & Choi, 2021) , or event-time relations (Chen et al., 2021; Tan et al., 2023a) . In our work, we address a comprehensive scope of temporal reasoning, including various levels and dimensions of understanding about time (Wang & Zhao, 2023) . Differing from prior approaches that rely on external knowledge (Yuan et al., 2023a; Tan et al., 2023b; Xiong et al., 2024) or impose temporal constraints (Li et al., 2023; Zhu et al., 2023b) Preference Optimization for LLMs Preference optimization approaches typically involve training a fixed reward model based on preference data, and then utilizing the reward model to train via reinforcement learning (RL) (Schulman et al., 2017; Ziegler et al., 2019; Stiennon et al., 2020; Bai et al., 2022) . To simplify this process, Direct Preference Optimization (DPO) (Rafailov et al., 2023) is introduced to avoid training the reward model entirely, and instead directly train the LLM using preference pairs. Building on this approach, recent works explore automatic optimization and self-correction in LLMs (Pan et al., 2023; Ji et al., 2024) . This involves two key steps: instructing LLMs to self-generate their training dataset (Wang et al., 2023; Taori et al., 2023; Tunstall et al., 2023) and serving LLMs as reward models (Fernandes et al., 2023; Saha et al., 2023; Dubois et al., 2024) to select high-quality data. The self-generated data optimization enables models to iteratively improve their performance through a self-rewarding mechanism (Yuan et al., 2024) . Inspired by the above works, we introduce a self-critic temporal optimization method that leverages the inherent capabilities of the model itself to achieve significant improvements in all temporal tasks. 
conclusion: 
 In this paper, we consider the problem of building a universal framework to strengthen the temporal reasoning capabilities of LLMs. Through systematic investigation, we discover a close relationship between mathematics and temporal reasoning. Building upon this insight, we introduce a self-critic temporal optimization method to equip the model with comprehensive temporal reasoning abilities. The Timo model, trained within our proposed framework, indicates significant generalizability across 38 temporal tasks, establishing as the new SOTA model of comparable sizes. Extensive experiments further demonstrate the effectiveness of our framework in maintaining general task abilities. 
 
 Appendix:  


Decision: The reviewers agree that this paper has value and would contribute to the program in the conference. There were some concerns by one reviewer about not having experiments on held-out temporal reasoning datasets. The authors provided a proper rebuttal with new experiments that made the reviewer reconsider their position.

Reviews: review: paper_summary: paper_summary: This paper proposes a model, Timo, achieving new SOTA at 7B and 13B scales, which leverages the mathematical datasets to set a solid foundation model for temporal reasoning and enhance with self-critic temporal optimization method.
strength_weakness: strength_weakness: 1. More generalization of the framework should be validated in other reasoning tasks.
paper_summary: paper_summary: Comments after Author Discussion
Reasoning about time is essential for Large Language Models (LLMs) to understand the world. In this paper, the authors propose a simple but effective self-critic temporal optimization method (TIMO) to enhance the model’s temporal reasoning capabilities without sacrificing general task abilities. It’s designed to excel in temporal reasoning at the 7B and 13B scales and outperforms the LLaMA2 by 10 and 7.6 in average accuracy scores on 38 Temporal reasoning tasks.
strength_weakness: strength_weakness: 1.In section ‘Main Results’, there is lack of analysis on comparison with the other models.
2. In section ‘Ablation Analysis of Framework’, there is lack of analysis on comparison with the GPT4 model. Why do the performance differences between TIMO and gpt4 differ on different tasks? The author should make appropriate analysis .
3. In table 1, the result ‘83.0’ of column ‘Pure-time tasks-AMB.’ is not the best result. Suggest the authors to highlight the second results in the table as well.
4. In case study, there is a certain ambiguity in the first case (Math-CoT lacks prerequisites ‘1652 Jan 1st is Monday’). Suggest the authors to replace other case.
questions: questions: None
paper_summary: paper_summary: This paper presents a framework to improve the general temporal reasoning capabilities of LLMs. The authors empirically verify the intuition that temporal reasoning is closely related to math reasoning, which motivates the proposed approach that adopts the LLM-as-Judge scheme using a math LLM and temporal reasoning instructions to produce temporal-related pairwise preference dataset, which is later adopted for training a DPO model called TIMO. The authors conduct experiments on TRAM (a temporal reasoning benchmark consisting of 10 categories) and show that TIMO outperforms the math LLM under most cases.
strength_weakness: strength_weakness: 1. The final DPO step uses the data distilled and evaluated by the math-LLM through the LLM-as-Judge framework. However, there is no comparison with adopting a non-math LLM (e.g., vanilla LLaMA or generally-SFT LLaMA) for generating such data. Existing work on weak-to-strong generalization suggests that distilling data from a weaker or equivalent LLM benefits a stronger LLM. Thus, it is possible that the improvements shown by TIMO results from the LLM-as-Judge framework rather than the math LLM (which is considered as the main contribution of this work).

2. Among all models that are compared against TIMO, TIMO is the only one that has been trained on the instructions from TRAM. So one concern is whether the improvements of TIMO over math-LLM result from better general temporal reasoning or specialized adaptation on TRAM. Some evaluations on other held-out temporal reasoning datasets could be much more convincing.

Comments: title: reply to rebuttal
Looking forward to further discussion
Looking forward to further discussion
Looking forward to further discussion
comment: Thanks for the response. I have no questions about the rebuttal and the further exploration of the generalization of the framework in more temporal reasoning tasks should be added in the revision. And I will keep the score unchanged.
Thank the authors for providing more information and results of the proposed model.
I agree that the new information and results can address some of the issues.
I will raise my score.
Dear Reviewer P1tK,

To address your concerns, we have conducted experiments on vanilla LLaMA-7B and LLaMA2-7B-chat to validate the importance of the math LLM in our framework. Additionally, we have evaluated TIMO on MCTACO and TempReason to demonstrate its effectiveness.

Since the rebuttal period is closing very soon, could you please check our response to see whether it mitigates your concerns?

Feel free to reach out with any further questions or suggestions!

Best,

Authors of TIMO
Dear Reviewer QQVe,

To address your concerns, we have provided additional model comparisons, improved case studies, and conducted new experiments on LLaMA2-8B to verify the effectiveness of TIMO.

With the rebuttal period closing soon, could you please check the response to see whether it mitigates your concerns?

If you have any further inquiries or suggestions, please do not hesitate to reach out to us!

Best,

Authors of TIMO
Dear Reviewer Q4nZ,

Thanks for your review! Considering the response deadline is approaching, we hope to have more discussions with you. 

If you have any further inquiries or suggestions, please do not hesitate to reach out to us!

Best,

Authors of TIMO