Domain Adaptation and Instance Selection for Disease Syndrome Classification over Veterinary Clinical Notes (2020)

Abstract: Identifying the reasons for antibiotic administration in veterinary records is a critical component of understanding antimicrobial usage patterns. This informs antimicrobial stewardship programs designed to fight antimicrobial resistance, a major health crisis affecting both humans and animals in which veterinarians have an important role to play. We propose a document classification approach to determine the reason for administration of a given drug, with particular focus on domain adaptation from one drug to another, and instance selection to minimize annotation effort.

## introduction microorganisms -such as bacteria, fungi, and viruses -were a major cause of death until the discovery of antibiotics @xcite . however, antimicrobial resistance ("amr") to these drugs has been detected since their introduction to clinical practice @xcite , and risen dramatically over the last decade to be considered an emergent global phenomenon and major public health problem @xcite . companion animals are capable of acquiring and exchanging multidrug-resistant pathogens with humans, and may serve as a reservoir of amr @xcite @xcite . in addition, amr is associated with worse animal health and welfare outcomes in veterinary medicine @xcite . "antimicrobial stewardship" is broadly used to refer to the implementation of a program for responsible antimicrobial usage, and has been demonstrated to be an effective means of reducing amr in hospital settings @xcite @xcite . a key part of antimicrobial stewardship is having the ability to monitor antimicrobial usage patterns, including which antibiotic history: examination: still extremely pruritic. there is no frank blood visible. and does not appear to be overt inflammation of skin inside eac. laboratory: assessment: much improved but still concnered there might be some residual pain/infection. this may be exac by persistent oilinesss from pmp over the last week. treatment: cefovecin 1mg/kg sc owner will also use advocate; advised needs to lose weight. to be 7kg plan: owner may return to recheck in ten days at completion of cefo duration. figure 1 : sample clinical note, in which the indication of use for cefovecin would be ear disorder is given and the reason -or "indication" -for its use. this data is generally captured within free text clinical records created at the time of consult. the primary objective of this paper is to develop text categorization methods to automatically label clinical records with the indication for antibiotic use. we perform this research over the vetcompass australia corpus, a large dataset of veterinary clinical records from over 180 of the 3,222 clinical practices in australia which contains over 15 million clinical records and 1.3 billion tokens @xcite . an example of a typical clinical note is shown in figure 1 . we aim to map the indication for an antimicrobial into a standardized format such as veterinary nomenclature (venom) @xcite , and in doing so, facilitate population-scale quantification of antimicrobial usage patterns. as illustrated in figure 1 , the data is domain specific, and expert annotators are required to label the training data. this motivates the use of approaches to minimize the amount of annotation effort required, with specific interest in adapting models developed for one drug to a novel drug. previous analysis of this dataset has focused on labeling the antibiotic associated with each clinical note @xcite . in that study, it was found that cefovecin along with amoxycillin clavulanate and cephalexin were the top 3 antibiotics used. as cefovecin was the most commonly used antimicrobial with the most critical significance for the development of amr, it was targeted for additional studies to understand the specific indications of use. the indication of use was manually labeled in 5,008 records. however, there were still over 79,000 clinical records with instances of cefovecin administration that did not have labels, in addition to over 1.1 million other clinical records involving other antimicrobial drug administrations missing labels. having only a single type of antimicrobial agent labeled causes challenges for training a model to classify the indication of use for other antimicrobials, as antimicrobials vary in how and why they are used, with the form of drug administration @xcite and different indications of use creating distinct contexts that can be seen as sub-domains. therefore, models that allow for the transfer of knowledge between the sub-domains of the various antimicrobials are required to effectively label the indication of use. to explore the interaction between learning methods and the resource constraints on labeling, we develop models using the complete set of labels we had available, but also models derived using only labels that can be created within two hours, following the paradigm of @xcite . specifically, our work explores methods to improve the performance of classifying the indication for an antibiotic administration in veterinary records of dogs and cats. in addition to classifying the indication of use, we explore how data selection can be used to improve the transfer of knowledge derived from labeled data of a single antimicrobial agent to the context of other agents. we also release our code, and select pre-trained models used in this study at: https://github. com/havocy28/vetbert . ## related work clinical coding of medical documents has been previously done with a variety of methods @xcite @xcite . additionally, classifying diseases and medications in clinical text has been addressed in shared tasks for human texts @xcite . previous methods have also been explored for extracting the antimicrobials used, out of veterinary prescription labels, associated with the clinical records @xcite , and labeling of diseases in veterinary clinical records @xcite as well exploring methods for negation of diseases for addressing false positives @xcite . our work expands on this work by linking the indication of use to an antimicrobial being administered for that diagnosis. contextualized language models have recently gained much popularity due to their ability to greatly improve the representation of texts with fewer training instances, thereby transferring more efficiently between domains @xcite . pre-training these language models on large amounts of text data specific to a given domain, such as clinical records or biomedical literature, has also been shown to further improve the performance in biomedical domains with unique vocabularies @xcite . these models can also accomplish many tasks in an unsupervised manner. for example, @xcite showed that free text questions could be fed through a language model and generate the correct answer in many cases. in our experiments, we demonstrate the usefulness of contextualized language models by pre-training bert on a large set of veterinary clinical records, and further explore its usefulness for domain adaptation through instance selection. domain adaptation is a task which has a long history in nlp @xcite @xcite . there has been further work demonstrating the usefulness of reducing the covariance between domains through adversarial learning @xcite . more recently, it has been shown that domain adversarial training can be effectively done using contextualized models, such as bert, through using a two-step domain-discriminative data selection @xcite . we adapt these methods to our task to create a more generalizable model that can adapt between domains more effectively. previous experiments have used active learning to improve clinical concept extraction with weak supervision @xcite . our work expands on this work through combining approaches to domain adaptation and the effective use of a small number of labels through the development of additional instance selection methods. ## approach in this section we detail our approach, as illustrated in figure 3 . ## conclusions and future work in conclusion, we proposed a range of methods to transfer knowledge derived from labeled data for one antimicrobial agent to other agents, considering the additional constraint of a limited annotation resource time of two hours. while the in-domain accuracy of 83% exceeds the raw inter-annotator agreement of 80% (cohen's @xmath0.78) on the source domain, transfer to other classes is still substantially lower with an average of 76% between the two classes. this shows that while the accuracy on classifying diseases is on par with human classifications for a single disease, there is still room for improvement on transferability to new data subdomains. the primary question is whether the labels created are good enough to report the reason for antibiotic administration in epidemiological reporting and antimicrobial stewardship guidelines. while the labels for why cefovecin was administered were better than the current standard of using expert annotations, our results indicate that accuracy varies substantially depending on the antibiotic being administered, and testing of the accuracy for each individual antibiotic should be evaluated prior to reporting the results based on labels generated by any model. in future research, these methods could be im-proved through utilization of available resources such as umls or drugbank to identify clinical use guidelines for antibiotics, to allow for training or adapting a model with few or no annotations. additionally, further work is required to apply these models into a data pipeline to create labels for vet-compass data to enable analysis of the key reasons for antimicrobial administration in veterinary hospitals across australia.