Title: A Semantic Parsing Framework for End-to-End Time Normalization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Time Normalization

Temporal Information Extraction
Abstract: Time normalization is the task of converting natural language temporal expressions into machine-readable representations. It underpins many downstream applications in information retrieval, question answering, and clinical decision-making. Traditional systems based on the ISO-TimeML schema limit expressivity and struggle with complex constructs such as compositional, event-relative, and multi-span time expressions. In this work, we introduce a novel formulation of time normalization as a code generation task grounded in the SCATE framework, which defines temporal semantics through symbolic and compositional operators. We implement a fully executable SCATE Python library and demonstrate that large language models (LLMs) can generate executable SCATE code. Leveraging this capability, we develop an automatic data augmentation pipeline using LLMs to synthesize large-scale annotated data with code-level validation. Our experiments show that small, locally deployable models trained on this augmented data can achieve strong performance, outperforming even their LLM parents and enabling practical, accurate, and interpretable time normalization.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel end-to-end approach to time normalization by casting it as a code generation task within the SCATE (Semantically Compositional Annotation of Temporal Expressions) framerwork. The authors implement SCATE as a python library and show that large language models can generate valid code directly from natural language. They introduce a data augmentation pipeline in which LLMs generate annotated examples that are automatically validated through code execution. Using this synthetic data, they fine-tune a smaller, local model and report that it outperforms the LLMs originally used to generate the training data. Overall, the work presents a scalable, interpretable, and efficient method for handling complex temporal expressions.\n\nThe authors propose an innovative data augmentation strategy where LLMs generate synthetic annotated data (sentence, temporal expression, SCATE code) that is validated through runtime execution of the code, ensuring correctness. They demonstrate that a small, locally deployable model (Qwen2.5-0.5B) fine-tuned on this augmented dataset can outperform the LLMs used for data generation, achieving competitive performance while being computationally efficient and interpretable.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. Methodological Innovation:\nThe formulation of time normalization as a code generation problem using executable SCATE operators is novel. By directly outputting executable code, the system offers clear interpretability and determinism, unlike black-box sequence-to-sequence models.\n\n2. Practical Implementation:\nThe authors fully implement SCATE as a Python library with composable, object-oriented design, making the semantics both human-readable and machine-executable. The paper carefully details the class structures for intervals, repeating intervals, periods, and operators, ensuring transparency.\n\n3. LLM-driven Data Augmentation with Validation:\nThe automatic data augmentation pipeline cleverly uses LLMs to generate large-scale annotated data, filtered via code execution to guarantee syntactic and semantic correctness. This reduces reliance on scarce expert-annotated datasets and improves scalability.\n\nWeaknesses:\nDataset and Domain limitations:\n\nThe approach is evaluated solely on the TempEval-2013 dataset. The paper does not address how the system performs in other domains or languages beyond english. \n\nThe system struggles with accurately identifying temporal expression spans. \n\nPublic release not available yet\n\nPaper does not explore open source LLM alternatives, focuses mainly on Claude and GPT-4.1\n\nQUESTIONS:\nHave you considered decoupling span detection from code generation via a two-stage pipeline? Would introducing a lightweight span classifier before code generation improve performance and reduce missed expressions?\n\nDoes the framework work on non-english languages?\n\nCould your executable code generation approach extend to other semantic parsing or reasoning tasks such as event extraction? \n\nAny failure analysis on the discarded code samples that failed execution? \n\nAny biases from the LLM generated dataset?\n\nLIMITATIONS:\nThe limitations in the paper are well-articulated, including:\n\nRestriction to TempEval-2013 for SCATE evaluation.\n\nLack of multilingual or open-domain benchmarks.\n\nHeavy reliance on proprietary LLMs for data generation.\n\nPerformance gaps in span detection.\n\nPending open-source release of code and datasets.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel approach to time normalization by reformulating it as a code generation problem following the SCATE framework. Traditional time normalization approaches that are based on ISO-TimeML schema struggle with complex temporal expressions like event-relative or compositional phrases. The presented method addresses this by implementing a fully executable SCATE Python library that translates temporal expressions expressed as text into structured and interpretable Python code, which then gets executed to produce deterministic normalized time normalization.\n\nContributions of this paper include:\n- It implements SCATE as a Python library, enabling deterministic execution of normalized time intervals;\n- It proposes data augmentation by using LLMs to generate SCATE code from text following a well-crafted prompt, creating over 8,000 new annotations of text-code pairs;\n- It presents empirical results of fine-tuning a small-scale model (Qwen2.5-0.5B) on the augmented data, showing that such a model can outperform the LLMs used for data generation and providing further analysis.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The method is original and has novelty according to the related work discussion. The idea of formulating time normalization as a code generation problem makes a lot of sense (i.e., the proposed python implementation essentially serves as a DSL for the time normalization task). This said, I am not very familiar with related work so cannot judge with confidence if the authors have missed important related work.\n- The paper is presented well and is very easy to follow and understand.\n- The empirical results are overall positive, showing that the approach is promising. The derived data as well as the model can be a useful resource for the community.\n\nWeaknesses:\n- The biggest weakness of this work is that the experiment section is brief and not particularly thorough, leaving some questions unanswered. As an example, it is hard to know how this method compares to previous work in a similar evaluation setup. I'll include more thoughts in the questions section below.\n- The problem being studied is relatively niche and is only impactful to a small community, limiting its overall significance.\n- Some in-depth analysis with examples can be provided to help the readers better understand the behaviors and limitations of the model.\n\nQUESTIONS:\nAs I mentioned, a main weakness of this paper is that the experiment section lacks thoroughness. My questions below are mainly around a result of that:\n- How does the proposed method compare to previous methods in a similar evaluation setting?\n\n- The authors show that in order for the 0.5B to learn the task, the augmented data (8k+) has to be used with the training set (300 examples), or otherwise the performance is close to zero. This suggests some data efficiency issues that can be addressed with options below. Have these been explored?\n  * efficient fine-tuning methods (LoRA), which typically works better with less data;\n  * using code-finetuned checkpoint instead of general instruction-tuned model.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a novel formulation of the time normalization task as a code generation task. They contribute a comprehensive python library for executing the existing SCATE framework, which represents temporal expressions using temporal operators. They then investigate the use of large language models to generate SCATE code within their novel python library, and present an analysis of errors made by their best-performing model.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The proposed SCATE Python library is extensive, and a valuable resource to the community.\n* The author's design of the SCATE library is well-suited for their downstream LLM code generation goal, as their class and operation names are semantically meaningful and likely to be interpretable by LLMs.\n* The ultimate goal of having a small model for both time expression extraction and SCATE code generation is well-motivated, and results present a solid baseline upon which future work can improve.\n* The evaluation of LLM performance is solid, with an interesting error analysis.\n\nWeaknesses:\n* The decision to jointly extract time expressions and generate SCATE code with one LLM pass needs more motivation, and perhaps an additional experimental result. While the authors offer that span recognition may be a direction for future work on lines 335-338, it's worth noting that techniques for extraction tasks with LLMs are generally improving (see 1, 2). It seems appropriate to evaluate whether using a specialized prompt for LLM time expression extraction and then generating SCATE code separately would further improve performance (albeit at the cost of an additional model pass), especially since this appears to be a source of frequent errors (see lines 324-326).\n\nQUESTIONS:\n* (See weakness #1) Were there any attempts to evaluate time expression identification and code generation separately?\n* Why are results so much worse on the test set? If I'm understanding correctly, Claude 3.7 + SCATE Prompt achieves 0.52 F1 on the test set, and 0.66 F1 on the training set. If this is due to block sampling in the creation of the TempEval-2013, this should be mentioned briefly in the manuscript. If the authors hypothesize that this is due to training data contamination, perhaps this should be noted in the manuscript as well.\n* Would it be possible to use existing time normalization systems (based on the ISO-TimeML framework) as additional baselines for this test set? That would further the argument for the necessity of the SCATE framework made on lines 25-37.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSummary\n- The authors develop a python library 'SCATE' for time normalization using an existing research paper- Sematically Compositional Annotation of Temporal Expressions (Bethard et al 2016). They utilize the framework outlined in Bethard et al to develop code representations. Using an existing dataset (TempEval 2013), they implement LLM-based methods including data augmentation  to explore the extraction of time normalized information.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n--\n- The manuscript reads well. There is detailed documentation for the implementation of the framework from Bethard et al including python class objects, functions. \n- The common ambiguities that arise from the framework are well documented in the appendix.\n\nWeaknesses\n--\nThere is limited novelty in the manuscript and several weaknesses.\n- There is no benchmarking of existing methods for time normalization against the proposed framework. (See questions)\n- There is limited experimentation on various prompting techniques and is restricted to 2. \n- The authors do not benchmark the LLM-based extraction method against baseline fine-tuning methods using BERT.\n\nQUESTIONS:\n- Figure 1 is vague and terse. The inputs and outputs to the LLM could be better outlined.\n- Which task dataset (A or B or C) in TempEval2013 was used for benchmarking? Description in lines 247-261miss that information. Were the methods outlined in the paper relevant? Were they tested before using this LLM-based approach?\n- Table 1 seems ambiguous. why weren't the performance of prompting techniques outlined in table 2- interval few-shot, SCATE, measured? What type of prompting techniques were used.\n- Were there performance improvements in run-time using the proposed method. Were Xu et al and Lapara et al frameworks studied? LLMs are great to use. What are the advantages of using the authors' proposed method against theirs? \n- Appendix B and C outline the prompts that were studied. Clarification- was the first prompt the full markdown page (13-35) Was the length of the prompts an issue? What happened if a smaller prompt was provided? Was the complete Markdown file generated by the authors or were LLMs used to refine the prompt? \n - The performance on Tasks A, B and ABC in TempEval paper have methods outlined. how does the proposed method compare to these rule-based and DL-based methods work?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}