Adapting A bstract M eaning R epresentation Parsing to the Clinical Narrative – the SPRING THYME parser (2024)

Abstract: This paper is dedicated to the design and evaluation of the first AMR parser tailored for clinical notes. Our objective was to facilitate the precise transformation of the clinical notes into structured AMR expressions, thereby enhancing the interpretability and usability of clinical text data at scale. Leveraging the colon cancer dataset from the Temporal Histories of Your Medical Events (THYME) corpus, we adapted a state-of-the-art AMR parser utilizing continuous training. Our approach incorporates data augmentation techniques to enhance the accuracy of AMR structure predictions. Notably, through this learning strategy, our parser achieved an impressive F1 score of 88% on the THYME corpus’s colon cancer dataset. Moreover, our research delved into the efficacy of data required for domain adaptation within the realm of clinical notes, presenting domain adaptation data requirements for AMR parsing. This exploration not only underscores the parser’s robust performance but also highlights its potential in facilitating a deeper understanding of clinical narratives through structured semantic representations.

## introduction abstract meaning representation @xcite (amr)is a highly adaptable and expressive framework designed to capture the semantics of natural language expressions. automatic amr parsing is a natural language processing (nlp) method that translates natural language inputs into formal amr expressions -representations which have proven to be useful across a wide range of downstream applications @xcite @xcite @xcite including those in the biomedical domain @xcite . formally, amr expressions take the form of labeled, rooted, directed, and acyclic graphs, @xmath0, e), where v represents the set of amr nodes, which can be of type predicate, abstract concept and attributes; e represents the possible semantic relations between nodes such as prototypical agent and patient denoted by arg0 and arg1. the amr graph structure underpinned by neo-davidsonian semantics can then effectively encapsulate the abstract concepts, relationships, and entities present in individual sentences or utterances. from a practical standpoint, amr expressions encompass the semantic content typically addressed by individual representation schemes such as semantic role labeling @xcite , named entities @xcite , and coreference chains @xcite , thereby unifying these diverse aspects of meaning into a single comprehensive representation. figure 1 illustrates an amr expression selected from the clinical domain. as figure 1 demonstrates, concepts including events, entities and properties are captured as nodes in the graph, while the relations among the concepts are captured by labeled edges connecting the nodes. events are represented using propbank frames @xcite , and the semantic relations of both entities and events to these predicates are specified either by a frame's numbered argument or one of the relations from amr's role inventory. for example, the see-09 predicate represents the event of "visit/consultation by a medical professional." in this case, the agent of the seeing event is "dr. chandler bing", represented by see-09's arg0 relation, and the semantic role of patient for the event is "she" indicated by the arg1 semantic relation. amr graphs also specify the temporal information in a formal way. in the above example, the time of the seeing event is specified by two temporal modifier subgraphs. it is a conjunction of "after now" and "within this week" which makes "later this week" a concrete time range. amr parsers based on pretrained large language models and sequence-to-sequence (encoderdecoder) architectures have demonstrated impressive accuracy when trained and evaluated on standard datasets. the use of amr parsers has contributed to improved performance across a range of nlp tasks including question answering @xcite , information retrieval @xcite , knowledge-graph construction @xcite , and text generation @xcite . these successes have sparked growing interest in employing amr in domains that diverge from the existing training data, such as human-robot interaction tasks, educational applications involving classroom discourse analysis, and diverse biomedical use cases. unfortunately, as language form and meaning deviate from the general language captured in generic training data, parsing performance shows a rapid decline. this decline stems from disparities in vocabulary, syntax, and overall discourse structure. addressing these challenges necessitates dedicated human expert annotation efforts to create domain-specific amr resources. however, such endeavors can be costly and time-consuming. hence, the preference lies in maximizing the utilization of existing data and parsers and adapting them to new domains, rather than building entirely new systems from scratch. the contributions of this paper include: ## data supervised training data for amr parsers consists of pairs of linguistic expressions along with their associated human annotated gold-standard amr expressions. the current standard dataset for amr development is amr 3.0 @xcite available from the linguistic data consortium as ldc2020t02. this general domain dataset is the basis for our baseline efforts prior to domain adaptation. amr 3.0 consists of over 59k english expressions from a variety of broadcast conversations, newswire, weblogs, web discussion forums, fiction and web text. to facilitate evaluation and model comparison, amr 3.0 is divided into standard training, development and test splits consisting of 55,635, 1,722, and 1,898 expressions respectively. to adapt amr to the clinical narrative, we developed 8,327 in-domain amrs (separate paper with detailed description under review) on a subset of the thyme colon cancer corpus @xcite . the colon cancer part of the thyme corpus consists of 594 de-identified physicians' notes for 198 patients with colon cancer. each patient is represented by one pathology note and two clinical notes. the corpus has undergone several prior annotation efforts, including temporal and coreference annotation @xcite @xcite and entity tagging as defined by the unified medical language system (umls @xcite ). as part of our amr annotation process, we adopted seven clinical-domain named entity (ne) types (anatomical-site, clinical-attribute, devices, disease-disorder, medications-drugs, signsymptom) from the umls project and relied heavily on the umls in classifying many amr concepts. like other genre-specific amr tasks @xcite , we found it necessary to modify the standard amr annotation approach to support meaningful annotation of domain-unique linguistic phenomena. two phenomena are pervasive in the clinical narrative. first, physician notes frequently drop eventive mentions when they are inferable by human readers. for example, "declines tetanus" does not mean the patient declined having tetanus; they declined a tetanus immunization. we expanded amr's guidelines to permit explicit rendering of certain implicit concepts like the immunization: second, like other specialized domains, clinical texts are rife with semantically dense noun phrases (nps) @xcite . in amr, nps must be treated in one of two ways: either all components are extracted and related (white @xmath1, or they are analyzed as single units of meaning, i.e., nes (white house). however, semantic compositionality exists on a spectrum @xcite , and many specialized nps in particular strain the adequacy of a binary approach. this can be seen even in simple clinical nps: one annotator might decide "blood pressure" is a single, cohesive unit of meaning and annotate it as an ne, while another might decide "pressure" is an extractable property of "blood". to address this, we implemented a two-pass strategy: in the first pass, for nps that fell under one of the clinical ne types mentioned above, an experienced annotator made these compositionality judgments and added each unique phrase to a searchable, phrasal ne dictionary along with an amr fragment that "defined" the compositionality for each phrase. annotators then referenced the dictionary when building the amr graphs in the second pass. this approach supported consistency and speed of annotation. finally, the thyme corpus contains frequent repetition of many other multiword expressions and phrases. for extremely formulaic phrases, such as those found in vital signs sections (@xmath2.60 cm, e.g.), we implemented a template-filling script that deterministically produced the amrs, again saving significant manual annotation time. of the 8,327 amrs, 1,640 were produced by this script; the rest were created manually. the final 8,327 thyme-amr data are split into training, development and test sets randomly with 4,955, 1,641 and 1,731 sentence-amr pairs, respectively. all of the model training is conducted on the training set of the amr 3.0 and thyme amr corpora. we show the inter annotator agreement between three annotators on 107 thyme-amrs in ## methods we treat the amr parsing task as a supervised machine learning problem and train a parameterized model to map natural language expressions to their corresponding amr graphs. various model architectures and training methods and paradigms have been employed over the years @xcite @xcite @xcite @xcite @xcite , resulting in a continuous improvement in the state of the art on the general domain amr dataset(i.e. amr 2.0 and 3.0 corpus (ldc2020t2)). however, these improvements are highly dependent on the availability of significant amounts of annotated training data hampering the development of parsers for specific genres and languages other than english. our approach here is to leverage an existing high-performance parser and adapt it to the clinical domain using the modest amount of domainspecific training data described in the last section. meanwhile, the great advances of the pre-trained foundational models has introduced a new modeling paradigm in the field of nlp as well as to structure-prediction problems such as amr parsing. in particular, the sequence-to-sequence modeling, originally developed for machine translation, has proven a highly effective approach for amr parsing @xcite @xcite . in this approach, two neural network components are involved: an encoder, which takes the natural language sentence as input and maps it to a continuous manifold as a sequence of high-dimensional vectors, and a decoder, which takes the embedded sentence representation vectors and maps them to the output embedding space, corresponding to the target sequence tokens. here we make use of the spring parser @xcite , one of the state-of-the-art amr parsers on amr 3.0 evaluation. the underlying pre-trained language model is bart-large @xcite , a transformer-based language model that has been trained using a set of denoising pre-training objectives, such as a masked language modeling objective and a document reconstruction objective, on general domain unlabeled english text. the neural network architecture relies on the self-attention and cross-attention mechanism to learn patterns from natural language texts. this pre-trained model is then fine-tuned on the amr 3.0 training data to map english inputs to linearized amr graphs, which consist of a sequence of amr tokens. we show the linearization correspondence of an amr graph to its sequence of amr tokens in figure 2 . a critical aspect of using sequence-to-sequence models for structured prediction tasks, like parsing, is transforming the task itself. in amr parsing, the amr graph is converted into a sequence of tokens through a linearization algorithm. note that the vocabulary of the decoder differs from that of the encoder model, as the target sequence consists of amr-specific tokens such as the relations arg0 and arg1, and predicates like test-01. during fine-tuning, we utilize the vocabulary derived from the amr 3.0 corpus, which ensures consistency and accuracy in the parsing process. the parsing problem is then to convert an input text sequence into a valid sequence of amr tokens that can be deterministically transformed into a directed amr graph. the overall spring approach is depicted in figure 3 . given a high-performing spring model, we adapt it to the thyme domain by finetuning on the thyme-amr training set (4,955 expressions). here, fine-tuning involves continuous gradient-based updates to the original model parameters with a small learning rate (5 × 10 -6 ) with batch size to be 20, we keep the maximum sequence length to be 1024.. ## experiments we present the domain adaption training experiments in this section to show the characteristics of the text from thyme corpus when it comes to amr parser developement. ## discussion our results have highlighted the advantages of employing data augmentation techniques for domain adaptation fine-tuning. this opens up the possibility for additional follow-up studies, including the incorporation of data from domain-specific propbank roleset development. for instance, in the case of thyme, leveraging example sentences for newly added named-entity types like "anatomicalsite" could prove beneficial. initializing the word embedding vectors with such domain-specific concepts would enable a better fit with the pre-trained foundational models. future investigations involving more sophisticated foundational models and data augmentation approaches hold great promise for enhancing amr parsing in the medical domain and other specialized domains. by harnessing the capabilities of cutting-edge language models and innovative data augmentation strategies, we can expect significant advancements in semantic parsing tasks and domain adaptation techniques. with these advances, amr parses have wide applicability to core information extraction tasks from the clinical narrative such as entity recognition, negation detection, uncertainty detection, coreference, temporality and relation extraction. ## conclusion in our investigation, we have presented substantial evidence highlighting the critical role of domainspecific amr annotations in the context of domain adaptation. our findings illuminate how variances in the distribution between original and target domains can precipitate a marked decline in the performance of amr parsing. this phenomenon underscores the challenge of catastrophic forgetting, a significant hurdle in the training of neural network models where new learning can disrupt previously acquired knowledge. to counteract this issue, we demonstrated the critical role of data augmentation techniques. specifically, by integrating domain-specific examples into the training dataset, we significantly bolstered the model's capability to acclimate to the nuances of the new domain while preserving its proficiency in the original domain. this strategic approach of coupling domain-specific annotation with thoughtful data augmentation has emerged as a formidable solution, ensuring both the robustness and accuracy of amr parsing across different domain adaptation scenarios. our study reaffirms the indispensability of domain-specific annotation in achieving effective domain adaptation and also supports data augmentation as an essential tool in maintaining a delicate balance between learning new domain characteristics and retaining essential knowledge from the original domain. this balanced approach provides a promising avenue for future research and development in the field of amr parsing, potentially paving the way for more nuanced and adaptable ai systems capable of navigating other domains with limited data yet maintain robustness.