Discourse-Aware In-Context Learning for Temporal Expression Normalization (2024)

Abstract: Temporal expression (TE) normalization is a well-studied problem. However, the predominately used rule-based systems are highly restricted to specific settings, and upcoming machine learning approaches suffer from a lack of labeled data. In this work, we explore the feasibility of proprietary and open-source large language models (LLMs) for TE normalization using in-context learning to inject task, document, and example information into the model. We explore various sample selection strategies to retrieve the most relevant set of examples. By using a window-based prompt design approach, we can perform TE normalization across sentences, while leveraging the LLM knowledge without training the model.Our experiments show competitive results to models designed for this task. In particular, our method achieves large performance improvements for non-standard settings by dynamically including relevant examples during inference.

## introduction temporal tagging is a challenging problem for building information extraction pipelines. traditionally, it involves first the identification of temporal expressions (tes) from text (extraction), followed by a mapping to a well-defined format such as timeml (normalization). previously, approaches to dealing with this problem involved curating handwritten rules @xcite often limiting their applicability to new domains and new languages. more recently, deep neural networks were trained for many tasks across domains and languages @xcite . however, they require an increasing amount of training data. in contrast, the advent of large language models (llms) @xcite overview of our proposed in-context learning approach for temporal expression normalization. given a test input, we retrieve similar text representations from the train set. we combine both of them along with a running context window of previous predictions and feed it to a language model along with instructions. @xcite led to strong zero-and few-shot capabilities by transferring knowledge for specific downstream nlp tasks like named entity recognition, question-answering, or sequence classification. therefore, making use of a recent llm without training is a compelling strategy to deal with data scarcity in multilingual setups and also to diversify utility across multiple domains. in this work, we explore the proprietary gpt-3.5-turbo model as well as the open-source zephyr model @xcite for te normalization. for both models, our discourse-aware approach (see figure 1 ) leverages in-context learning using few-shot examples and a document-level temporal context window. we explore various sample selection strategies for prompting tailored toward the te normalization task and show that standard sentence-level processing might not be suitable to capture all the necessary long-range context dependencies and discourse information. our broad evaluation across six domains and seven languages demonstrates the competitiveness of our method to dedicated normalization models. in particular, our analysis reveals the benefits of our method in settings when the target document is more distant from their training data. ## background and related work temporal tagging is a two-step process consisting of te extraction from textual documents, followed by the normalization into a standard format. we follow the timeml annotation guidelines @xcite , which define four temporal types, namely date, time, duration, and set. for the normalization, we focus on the value attribute that captures the most important temporal semantics of a te. while explicit tes include all necessary information for the normalization, e.g., "may 24, 2024", others require further knowledge or temporal discourse information. for example, implicit expressions like "easter 2024" require semantic knowledge, and relative expressions like "tomorrow" rely on an anchoring date, e.g., the document creation time (dct). under-specified expressions are missing the relation to the anchor and cannot be fully normalized by the given context. 1 te extraction and reasoning. te extraction has been handled as a sequence labeling problem through trained language model sequence taggers @xcite . @xcite utilize bert to identify temporal relations in text. @xcite investigate temporal reasoning capabilities of recent llms. in extractionrelated tasks, prior works explore gpt's abilities for event extraction, specifically relation extraction @xcite @xcite . however, no prior work studies the feasibility of te normalization using llms. for solving te normalization, several rulebased systems have been proposed such as hei-deltime @xcite and su-time (chang and manning, 2012), while other systems rely on context-free grammars @xcite . however, both approaches rely on highly language-specific resources. in contrast, deep-learning-based models have demonstrated robust and generalizable performance across languages for the normalization @xcite . however, this system required a careful design of the neural network and large-scale training. instead, we rely on the transfer learning abilities of llms by providing selected examples to learn from. few-shot learning. with the advent of powerful pre-trained language models, @xcite discovered that these models can be utilized for solving tasks without task-specific training. by 1 for further details, we refer to @xcite . ## approach in-context learning (icl) utilizes few-shot examples to learn the downstream task @xcite . we follow this approach and describe our selection strategies along with prompting formats relevant to te normalization. ## experiments this section describes our experimental setup, the results, and broad analyses of various settings. ## conclusions in this paper, we demonstrated that recent llms are capable of temporal expression normalization when being prompted with an appropriate in-context learning method. our discourse-aware prompt allows the llm to capture important context information while still being generic enough to provide general task descriptions. our experiments across domains and languages showcase the competitive performance of our method compared to specifically designed normalization models and outperforms them when the target document is more distant from their underlying training data.