Defining and Learning Refined Temporal Relations in the Clinical Narrative (2020)

Abstract: We present refinements over existing temporal relation annotations in the Electronic Medical Record clinical narrative. We refined the THYME corpus annotations to more faithfully represent nuanced temporality and nuanced temporal-coreferential relations. The main contributions are in re-defining CONTAINS and OVERLAP relations into CONTAINS, CONTAINS-SUBEVENT, OVERLAP and NOTED-ON. We demonstrate that these refinements lead to substantial gains in learnability for state-of-the-art transformer models as compared to previously reported results on the original THYME corpus. We thus establish a baseline for the automatic extraction of these refined temporal relations. Although our study is done on clinical narrative, we believe it addresses far-reaching challenges that are corpus- and domain- agnostic.

## introduction temporal relation extraction and reasoning in the clinical domain continues to be a primary area of interest due to the potential impact on disease understanding and, ultimately, patient care. a significant body of text available for this purpose is the thyme (temporal histories of your medical events) corpus @xcite , consisting of 594 clinical and pathology notes on colon cancer patients and 600 radiology, oncology and clinical notes on brain cancer patients, all from the electronic medical record (emr) of a leading us medical center. this dataset has previously undergone a variety of annotation efforts, most notably temporal annotation @xcite . it has been part of several semeval shared tasks such as clinical tempeval @xcite where state-of-the-art results have been established. our goal was to utilize this thyme corpus to enable the extraction of more extensive patient timelines by manually creating cross-document links that built off the pre-existing single file annotations. @xcite discuss that a subset of the thyme temporal annotations contributed to incompatible temporal inferences, thus reducing their ability to support meaningful temporal reasoning. accuracy and informativeness of temporal relation gold annotations are essential for their effectiveness in training a system for temporal relation extraction. we build on this work by offering an in-depth discussion of three key temporal relations -contains, contains-subevent (abbreviated as @xcite , and noted-on -and how the addition of the last two types enhances the learnability of the temporal relations by resolving the conflicting temporal information in the original annotations. while these revisions are corpus-specific, the reasoning behind them has far-reaching implications for automated timeline extraction. since the cross-document linking task inherently deals with multiple, discrete narratives, it exposes the practical impact of discourse context on word sense (different narratives have different goals, which in turn influences meaning interpretation). this is discussed in detail in section 4. we empirically found it essential to take changes in discourse context into account and suggest the same would be true for any annotation project that is interested in temporal reasoning, particularly those dealing with longer timelines (i.e., beyond the single-document level). recent developments in natural language processing establish neural approaches and more specifically transformer-based methods as the state of the art. pre-trained models such as bert @xcite , biobert @xcite , xlnet @xcite , albert @xcite , roberta @xcite , bart @xcite , and spanbert @xcite report significant gains on multiple tasks. thus, we demonstrate the learnability of the refined temporal relations in the context of these recent methodological developments. ## dataset the 594 notes that make up the colon cancer part of the thyme corpus are grouped into sets, each set pertaining to a single patient and consisting of three notes written at different times during the patient's course of care. these notes had been previously annotated for five different intradocument temporal relations (before, overlap, begins-on, ends-on and contains), a subset of the iso-timeml temporal link (tlink) types @xcite 1 . to keep annotation manageable and circumvent massively inferential temporal linking, the thyme guidelines constrained tlink creation to events within the same sentence or adjacent sentences, and specifically prohibited tlinking across sections (these are clinically-delineated sections separated from each other by numerical section ids -history of present illness is section 20103, vital signs is section 20110, etc.) linguistic evidence for creating these tlinks included local cues such as temporal 1 this approach to temporal annotation in fact evolved directly from the iso-timeml model, in collaboration with prof. james pustejovsky. the basic formal ontology has introduced similar relations @xcite , and a posthoc mapping could be done between the relations proposed here and those in bfo 2.0. however, we defer to more philosophical scholars to determine the similarities and differences between iso timeml and bfo 2.0. additionally, the notes had been separately annotated for intra-document coreference (identical) and bridging (set-subset, whole-part) relations, which were later merged with the temporal annotations @xcite . temporal relations alone are insufficient for timeline extraction; coreference relations are also necessary (is the tumor seen in september the same as the one seen in march?). pursuant to our goal of reasoning over largescale timelines, we built off these pre-existing within-note annotations by manually adding coreference and bridging links across each set of three notes. in the process, we discovered a subset of the original contains relations contributed to temporally-conflicting information, which led to the addition of two new tlinks: noted-on and con-sub @xcite . we discuss below how these updates contribute to more accurate and comprehensive temporal relations which facilitated cross-document linking. as such, this is one of the few studies in clinical nlp for cross-document temporal relation annotations (see @xcite ; also see @xcite for general domain cross-document temporal annotation discussions). ## refined temporal relation: noted-on the thyme guidelines specified that tests should always be in a contains relation with their results, as in (1), observing that it is inferential to say that something seen on the test exists before or after the test. we agree with the authors in part, particularly for a project that did not have coreference relations and that relied heavily on explicitly-temporal cues (after, during, before, etc.). however, once the coreference links had been merged with the temporal annotations, more information was revealed about the temporal nature of the events. for example, metastases in (1) might well be identical to a later mention, such as: 2. february 20, 2009, liver metastases resected with clear margins. the merged thyme and coreference annotations 2 for (1) and (2) were as follows: together, these relations entail that the same liver abnormalities were temporally contained by january 17, 2009 and temporally overlapped february 20, 2009 -a logical impossibility. this situation was extremely frequent in the data, reducing the informativeness of the contains links and the timeline as a whole. we therefore manually converted these links to a subtype of overlap relation, noted-on. noted-on conveys temporal overlap between events and additionally says that event a (result) was observed on/by event b (test). (1a) was therefore re-annotated as follows: 3. metastases noted-on ct the application of this link was so constrained that it was fairly easy to annotate; we therefore added it as a single-annotated post-processing step. 4 refined temporal relation: contains-subevent 4.1 motivation and annotation early in the cross-document annotation pilot, we found the pre-existing intra-document schema categories were insufficient for dealing with cross-narrative phenomena, which in turn led to the addition of the new con-sub relation. consider: 4. note a: 2009: patient presents with recurrent adenocarcinoma. ct abdomen and 2 note: this and all examples in this paper have been fabricated for patient privacy. however, they are all linguistically similar to actual sentences from the corpus. pelvis ordered for further staging of this colon cancer. i. adenocarcinoma ident cancer note b: successful removal of primary tumor in 2004. recurrent adenocarcinoma is inoperable. patient here today to discuss other treatment options for his colon cancer. ii. cancer contains tumor iii. cancer contains adenocarcinoma both sets of intra-document links are pragmatically appropriate. discourse contexts can expand or reduce the level of granularity at which a sense is interpreted @xcite ; also see @xcite . in note a, the text supports a coarse-grained interpretation of adenocarcinoma, or what hovy et al., 2013 term a "wide" reading; it refers generally to the patient's cancer. note b, however, requires a fine-grained ("narrow") interpretation -adenocarcinoma here refers specifically to the new, inoperable tumor and is contrasted with the original, resected tumor. the quandary for the cross-document task lies in whether to link adenocarcinoma in a as identical to adenocarcinoma in b. an identical relation entails logical impossibilities: assuming we also link cancer a as ident to cancer b , the combined within-and crossdocument relations now say the recurrent adenocarcinoma temporally contains itself and the primary tumor which was removed years earlier. this reduces the meaningfulness of the contains links and therefore the timeline. on the other hand, leaving the two adenocarcinoma references unlinked fails to capture the significant semantic relation between two identical strings (recurrent adenocarcinoma) that do in fact refer, on some level of granularity, to the same real-world event. in either case, annotators are stymied. clearly, the pre-existing intra-document schema categories, specifically the binary coreference choice (@xmath0, were insufficient for dealing with the sense variation and nuanced event structure exposed by multiple narratives. we therefore introduced con-sub (based on @xcite as a subtype of the contains tlink type. while contains conveys only temporal containment, con-sub additionally says that event b is intrinsically part of the structure of event a foot_0 . the difference may be seen in (5): 5. during patient's neoadjuvant treatment, she was in a car accident which delayed cycle 4. a. treatment contains accident, delayed b. treatment con-sub cycle we were then able to re-annotate the intradocument relations for both notes a and b above as cancer con-sub adenocarcinoma, which streamlined cross-document decisions -cancer a ident cancer b and adenocarcinoma a ident adenocarcinoma b -while preserving the "quasiidentical" relation @xcite between the cancer and adenocarcinoma terms. while this solution does not fully resolve the problem of binary annotation choices, it does provide more "wiggle room" along the meaning spectrum @xcite ) by introducing a third value -two mentions may be identical, non-identical, or mereologically (part-whole) related. in keeping with our primary goal of enabling timeline extraction, we implemented con-sub as a tlink since it conveys true temporal containment. con-sub, however, differs from other tlinks, which were constrained by proximity and lexical cues, as discussed in section 2. the fact that con-sub also represents structural information allowed us to treat it like a coreference/bridging relation in terms of permissible textual evidence for link creation: namely, semantic scripts. these may be defined as: "a stereotypical sequence of events" @xcite or "prototypical schematic sequences of events" @xcite . we can expect a surgery, for example, to consist of certain, typical subevents (incisions, subprocedures, anesthesia administration, etc.), which therefore enables annotators to look throughout the whole document for lexical items with meanings that fit those subevents. the concept of semantic scripts is what facilitates attainable long-distance coreference /bridging linking, and therefore, long-distance con-sub linking. this is obviously not the case for nonsubevent contains relations. we were therefore able to revise the thyme annotations to accommodate con-sub in two ways: first, we converted contains links to con-sub as appropriate, e.g.: treatment contains radiation became treatment con-sub radiation. secondly, we added certain longdistance con-sub links for which there were no pre-existing contains annotations. these changes were made via a double-blind annotation process followed by an adjudication pass. the annotation team for the entire project (including the cross-document stage) consisted of nine annotators, eight of whom either had or were obtaining undergraduate or graduate degrees in linguistics. the ninth annotator was a physician who received on-the-job linguistics training and focused primarily on annotation subtasks that demanded considerable medical knowledge. additionally, we consulted regularly with an oncologist and a medical coder with a decade of nlp annotation experience. ## summary of refined thyme relations the pre-existing contains annotations were revised in part through the addition of two new links, both of which convey temporal and nontemporal information. the original contains and overlap relations were therefore reimagined as supertypes 8 , each consisting of two subtypes. all links are described in in the next section we demonstrate the learnability of the refined thyme+ temporal relations with state-of-the-art transformer methods. we report results that establish baselines for the thyme+ corpus for further methods development and offer insights into the challenges we faced which we view as exciting venues for future research.