Can ISO 24617-1 go clinical? Extending a General-Domain Scheme to Medical Narratives (2025)

Abstract: The definition of rigorous and well-structured annotation schemes is a key element in the advancement of Natural Language Processing (NLP). This paper aims to compare the performance of a general-purpose annotation scheme — Text2Story, based on the ISO 24617-1 standard — with that of a domain-specific scheme — i2b2 — in the context of clinical narrative annotation; and to assess the feasibility of harmonizing ISO 24617-1, originally designed for general-domain applications, with a specialized extension tailored to the medical domain. Based on the results of this comparative analysis, we present Med2Story, a medical-specific extension of ISO 24617-1 developed to address the particularities of clinical text annotation.

## introduction developing robust and coherent annotation schemes is key to the advancement of natural language processing (nlp). these schemes provide formalized frameworks that define which linguistic or domain-specific phenomena are to be annotated, and how such information should be consistently represented across datasets. by standardizing the annotation process, they ensure that labeled data is meaningful and interpretable to downstream algorithms @xcite . throughout the years, several annotation frameworks have been developed providing structured labels and attributes for morphosyntactic @xcite , semantic roles @xcite @xcite , coreference @xcite , temporal @xcite and discourse relations @xcite information. additionally, multi-layer annotation schemes that can cover different linguistic phenomena @xcite @xcite have been proposed, thus enabling a more overarching linguistic representation. concurrently, the growing demand for annotated schemes has heightened the need for standardization and interoperability. initiatives such as the iso 24617 -semantic annotation framework (iso tc37/sc4, 2012) support the development of reusable annotation models, thereby promoting consistency and facilitating comparative evaluation across datasets @xcite . for the most part, these annotation schemes are domaingeneral, designed to capture linguistic structure and meaning in any type of text. nevertheless, some domains, such as the medical field, require more specialized annotation approaches. due to the complexity and specificity of clinical language and concepts, task-specific annotation schemes are essential. these schemes are designed to capture entities such as medical conditions, medications @xcite , negation and uncertainty @xcite , and temporal information @xcite . such domain-focused schemes are crucial for enabling effective information extraction in clinical settings, ultimately supporting decision-making and research in healthcare. choosing between a general-purpose and a domain-specific annotation scheme is a critical design decision that significantly affects the quality, applicability, and transferability of annotated datasets. each approach offers distinct advantages and limitations, depending on the project's objectives, the nature of the source texts, and the intended downstream applications. this paper pursues two main objectives: (1) to compare the performance of a general-purpose annotation scheme with that of a domain-specific scheme in the context of annotating clinical narratives; and (2) to explore the feasibility of harmonizing iso 24617-1, a general-domain scheme, with a specialized medical branch. to that end, we introduce med2story, an extension of iso 24617 tailored to the medical domain. the creation of med2story will enable the systematization of data relevant to different domains: in linguistics, by supporting the study of issues such as the aspectual properties of event-denoting nouns; in computational research, by facilitating the training of models for the extraction of medical information; and in medicine, by promoting the detection of patterns and the transformation of unstructured data into structured data that is important to clinical research. the remainder of this paper is structured as follows. section 2 presents an overview of general-purpose and domain-specific annotation schemes. section 3 describes the experimental setup, including the methodology, dataset, annotation schemes, and key findings. section 4 introduces the med2story annotation framework. finally, section 5 concludes the paper and outlines directions for future work. ## related work over the past several decades, numerous annotation schemes have been developed to address the representation of grammatical and domain-specific information in textual data. within the scope of this study, we distinguish between general-purpose annotation schemes, which aim to capture linguistic structures and meaning in a domain-agnostic manner, and domain-specific annotation schemes, which are tailored to encode specialized knowl-edge relevant to particular fields. general-purpose schemes tend to offer broader linguistic coverage, often requiring detailed linguistic expertise for accurate annotation. in contrast, domain-specific schemes are typically narrower in focus and demand specialized domain knowledge (medical, economic) for effective annotation. while many annotation schemes have concentrated on individual linguistic levels, such as morphological, syntactic, semantic, or pragmatic features, there have also been efforts to develop comprehensive, multilayer frameworks that encompass several of these dimensions. among these, the universal dependencies (ud) framework @xcite stands out for its typologically-informed approach to morphosyntactic annotation, enabling cross-linguistic comparison. in the realm of semantic annotation, frameworks such as the abstract meaning representation (amr) @xcite and, more recently, the uniform meaning representation (umr) @xcite focus on modeling multiple layers of meaning. umr extends amr to facilitate document-level semantic annotation, incorporating semantic roles, temporal relations, and discourse structures. another important contribution is the iso 24617-semantic annotation framework (se-maf), which includes multiple modules for the annotation of temporal, referential, spatial, quantificational, and semantic-role-related information, among others. this framework offers a languageagnostic, interoperable and theoretical neutral architecture, allowing for its adaptation across languages with minimal modification. the text2story annotation scheme @xcite , developed in compliance with iso 24617 standards, is also a multilayer framework applied to the annotation of morphosyntactic and semantic information in european portuguese texts. turning to domain-specific annotation, particularly within the biomedical and clinical domain, the focus of the present study, there have been efforts to develop annotation schemes that encode both domain-relevant and grammatical information. for instance, @xcite created an annotation scheme with syntactic and semantic layers alongside medical concepts. @xcite annotated a dataset of spanish clinical records with semantic groups. the merlot corpus @xcite comprises 500 french clinical narratives annotated for linguistic, seman-tic, and structural features. the i2b2 annotation guidelines @xcite include clinical and temporal annotations. @xcite developed semclinbr, comprising 1,000 clinical notes in brazilian portuguese with semantic annotations, while, for european portuguese, @xcite compiled a set of 281 clinical case texts annotated for medical entities. despite these advances, most existing clinical annotation resources exhibit several limitations. as noted by @xcite , inconsistencies are common, and comprehensive annotation encompassing both domain-specific and grammatical features is lacking. both general-purpose and domain-specific annotation schemes possess distinct strengths and limitations, which we assessed though the experiment described in the following section. ## the experiment 3.1 the methodology to assess the efficacy of both general-domain and domain-specific annotation schemes in capturing temporal information within medical reports, an experimental study was conducted utilizing two distinct schemes: the text2story scheme @xcite and the i2b2 scheme @xcite . as outlined in section 2, the former is predicated on iso standards 24617, whereas the latter was expressly designed for annotating clinical texts in english. the selection of the text2story annotation scheme was motivated by its comprehensive nature, interoperability, languageagnostic capabilities, and the potential for integrating, in a harmonized fashion, annotations across multiple semantic layers -such as referential, semantic roles, and spatial information, although the current focus is solely on its temporal module. conversely, the i2b2 scheme was chosen due to its extensive validation and demonstrated capacity to encode not only specific medical information but also temporal features inherent in medical reports. following the selection of the annotation scheme, six pseudonymized clinical reports from patients diagnosed with acute myeloid leukemia (aml) written in european portuguese were annotated using the two distinct schemes. subsequently, the annotation outputs were systematically analyzed to evaluate the respective strengths and limitations of each approach. based on this comparative analysis, the most effective strategy for annotating both grammatical structures and domain-specific information in medical reports was identified. ## the text2story medical branch -med2story after the comparative analysis between the i2b2 and text2story annotation schemes, the decision was made to develop an extension of the iso-based text2story framework foot_1 . this decision was motivated by text2story's effectiveness in capturing morphosyntactic and grammatical phenomena, in contrast to its limitations in representing specialized clinical knowledge. on the other hand, while i2b2 includes medical domain categories, it was found to be overly broad and insufficiently granular, reducing the accuracy of clinical annotation. furthermore, i2b2 lacks systematic support for annotating morphosyntactic and semantic linguistic features, making it less suitable for deeper linguistic analyses. the first development step consisted in defining a set of labels that could rigorously capture relevant clinical information. a fundamental distinction was introduced between two types of events: general event and specialized event. events classified as general event retained the original text2story attributes -namely, class, type, tense, aspect, polarity, vform, modality, and pos -suitable for linguistic description of clinical narratives. specialized events incorporated medical domain-specific attributes, allowing for a more detailed and meaningful representation of clinical content. all events were annotated as general events, and only those conveying clinical content were additionally annotated as specialized events. the selection of clinical labels was conducted in collaboration with a hematologist from ipo-porto, who participated in validating the clinically relevant categories. this phase was based on the analysis of a corpus of 40 pseudonymized medical reports from patients diagnosed with aml, including discharge summaries, general reports, and consultation notes. as for nominal events, only the pos and polarity attributes were annotated under the general event layer, since current literature does not yet offer viable solutions for aspectual annotation of nouns. the definition of medical domain labels was guided by the principles of the umls metathesaurus ontology @xcite , widely recognized as a systematic reference for organizing biomedical terminology. additionally, contributions from @xcite , whose research on the same corpus proposed a preliminary set of clinically validated categories, were considered. some of these categories were retained, while others were adapted or refined to align with the goals of this scheme. the final set of medical categories included: sign or symptom, personal history (with subcategories: past medical history, comorbidity, and undefined), intercurrence, examination, examination result, principal diagnosis, characterization of the disease, medical procedure, treatment, drug administration route, and treatment response. these categories addressed two key gaps in previous schemes: the lack of clinical categories in iso-based text2story and the excessive generality of i2b2, as exemplified by the use of the generic label test to annotate both examinations and their results. regarding annotation scope, only occurrences, states, or circumstances with temporal relevance were annotated as events, following the approach of text2story and iso 24617-1. entities such as medications, organs, institutions, or healthcare services were not annotated at the event layer, but instead in the referential layer, as participants in events. two modifications were introduced to the grammatical attributes: relative pronoun under pos; and the extension of negative polarity to include cases of implicit polarity. as for temporal expressions, their annotation was not addressed in depth due to its complexity and the need for a more robust framework based on iso 24617-1. instead, text2story's guidelines were followed, with the addition of two specific attributes: admission time (date of patient admission), and discharge time (date of hospital discharge). the attribute doctime (report creation date) was retained. when discharge date and report creation date coincide, only the discharge time label should be used. finally, regarding tlink, we followed the guidelines of the text2story annotation scheme. tlink are established between events, between events and temporal expressions, and between temporal expressions. their annotation proceeds from the last event in the linear order of discourse to the first, thereby ensuring relational consistency across annotations. this rule applies only in cases where transitivity can be verified. in general, transitivity is preserved, which enables the temporal localization of all events. moreover, a direct link is established between each event and the temporal expression that situates it within the discourse timeline. the i2b2 approach, which systematically links all events with anchors such as doctime, admission time, or discharge time was not adopted, as this practice proved redundant and of limited informational value. instead, it is proposed that only events with no explicit or definite temporal relation be linked to those anchors. additionally, the tlink after overlap and their mirror before overlap were introduced, ensuring that links are consistently established left-to-right, or from the event to the temporal expression, in alignment with the other tlink. figure 1 shows an annotation example using the med2story scheme. the complete scheme, guide-lines, decision tree, and appendices are available in the associated github repository. ## conclusion in this study, we set out to compare the performance of a general-purpose annotation scheme -text2story, based on iso standard 24617-1 -with that of a domain-specific scheme, i2b2, in the context of annotating clinical narratives. the results show that the text2story annotation scheme is applicable to this type of text. however, it proves to be insufficiently informative with respect to domain-specific medical categories, highlighting the need to create new specialized tags. on the other hand, its capacity to represent morphosyntactic and semantic information is notably robust. as for the i2b2 scheme, although it enables the annotation of medical information, its tags are overly broad, and it provides limited detail at both the morphosyntactic and semantic levels. given these limitations, we developed a medicalspecific extension of the iso 24617-1 scheme, called med2story, designed to meet the requirements of the clinical domain. from a conceptual and structural perspective, the iso standard is robust and comprehensive, allowing for proper integration of domain-specific aspects related to the medical field. although certain tags and attributes could be refined to represent information more precisely -such as the introduction of the tlink after overlap/before overlap -iso 24617-1 is designed to accommodate extensions for more specialized information, namely through the inclusion of tags derived from the medical ontology. in future research, the proposed scheme will be applied to a set of medical reports in european portuguese. building on this, we intend to create a parallel dataset by translating these reports into other languages, with the aim of evaluating the applicability and robustness of the approach across different linguistic contexts. furthermore, we propose extending med2story with additional annotation layers, particularly referential annotation, to enhance its descriptive and analytical scope.