P harm MT : A Neural Machine Translation Approach to Simplify Prescription Directions (2020)

Abstract: The language used by physicians and health professionals in prescription directions includes medical jargon and implicit directives and causes much confusion among patients. Human intervention to simplify the language at the pharmacies may introduce additional errors that can lead to potentially severe health outcomes. We propose a novel machine translation-based approach, PharmMT, to automatically and reliably simplify prescription directions into patient-friendly language, thereby significantly reducing pharmacist workload. We evaluate the proposed approach over a dataset consisting of over 530K prescriptions obtained from a large mail-order pharmacy. The end-to-end system achieves a BLEU score of 60.27 against the reference directions generated by pharmacists, a 39.6% relative improvement over the rule-based normalization. Pharmacists judged 94.3% of the simplified directions as usable as-is or with minimal changes. This work demonstrates the feasibility of a machine translation-based tool for simplifying prescription directions in real-life.

## introduction adverse drug events stemming from medication errors are a vital cause of concern in patient care and are estimated to cost us$42 billion annually or roughly 1% of total global expenditure. in the us alone, medication errors cause one death every day and are responsible for over 700,000 visits to the emergency department and over 100,000 hospitalizations each year @xcite @xcite . one of the frequent sources of medication errors in the us is the directions on the 1.91 billion electronic prescriptions (e-prescriptions) transmitted annually @xcite @xcite . the style and language used in eprescriptions are highly variable and often filled with medical jargon. for example, in a recent study @xcite , the authors noted that the direction "take 1 tablet by mouth once daily" was represented in 832 different ways. the study also found that 10.1% of e-prescriptions contained incorrect or confusing language. pharmacists play a vital role as intermediaries between physicians and patients by translating the rich medical jargon in the e-prescriptions written by physicians to patient-comprehensible directions on the prescription labels printed on pill bottles. however, human translation is time-consuming and subject to errors, potentially leading to medication errors and other patient safety risks due to prescription ambiguity. in this paper, we propose a machine translationbased system, called pharmmt, to simplify the e-prescription directions authored by physicians into patient-friendly language. the system aims to automate the translation and normalization of e-prescription directions and reduce the pharmacists' overall workload. we investigate multiple neural network-based models, including transformer-based models and bi-directional lstm models, rule-based approaches, and a hybrid model combining neural network-based models with a rule-based backoff. we train and evaluate the proposed system over a dataset of over 530k paired e-prescriptions and their humantranslated text, obtained from a large mail-order pharmacy. using automated measures to evaluate machine translation output, we compare the pharmmt system against a rule-based approach developed based on domain knowledge from pharmacists. our results show that pharmmt performs significantly better than the rule-based baseline. manual evaluation by pharmacists also shows a high potential to directly apply the proposed approach in pharmacies to translate e-prescription directions. the contributions of this work are: 1. we develop a neural machine translation model for simplifying e-prescriptions and build an end-to-end system to generate normalized, patient-friendly, and usable translations. the model achieves a bleu score of 60.27 against the reference directions generated by pharmacists, and 94.3% of simplified prescriptions are judged as usable as-is or with minimal changes by pharmacists. ## related work prior work on automated approaches for translating e-prescription directions mainly focused on information extraction models relied on handwritten rules or linguistic signals found in prescription free text. tools such as metamap @xcite and medlee @xcite extract and organize clinical information in text documents using external knowledge sources, such as the unified medical language system (umls). other systems, such as fable @xcite , employed a conditional random fields-based model to recognize medication entities. other researchers have proposed rule-based approaches to normalize and simplify directions using task-specific knowledge such as common abbreviations used in prescriptions @xcite . while machine translation-based approaches have not yet been proposed for translating eprescription directions, prior works such as @xcite @xcite have suggested solving machine translation tasks without the need for explicitly-defined rules. neural machine translation (nmt) models have been shown to be able to learn contextual rules automatically from large corpora and produce higher quality translations @xcite @xcite . other researchers, such as @xcite , have shown that while statistical machine translation methods mainly focused on lexical rules to minimize sentence complexity, nmt models could capture richer syntactic information @xcite . researchers studying deep neural network models have explored multiple encoder-decoder frameworks, such as transformer-based networks @xcite , and recurrent neural networks (rnn), including long short-term memory (lstm) models @xcite and gated recurrent units (gru) @xcite . rnn units have been used to encode source sentences into fixed-length representations and then decoded into reference sentences @xcite . models with deep lstm-based rnn units have shown the benefits of using deeper structure @xcite . in other works, @xcite introduced a convolution neural network with an attention-based mechanism to learn long-range dependency. in recent work, @xcite developed a novel transformer-based architecture using attention mechanism without recurrence or convolution, resulting in a state-of-the-art performance for many natural language processing tasks, such as recognizing textual entailment, sentiment analysis, and natural language inference. @xcite @xcite 3 simplifying e-prescription directions we frame the challenge of simplifying eprescription directions as a machine translation task from physician-authored directions ("source") to patient-facing text authored by pharmacists ("reference"). this monolingual translation task focuses on replacing highly-abbreviated medical jargon with patient-friendly vocabulary, simplifying cryptic expressions, and normalizing them so that they can be used with minimal changes by the pharmacists. e-prescription directions consists of specific components related to the prescribed drug, viz., dosage, form, route, duration, frequency, and reason for prescribed use. for example, the eprescription "2 puffs orally q 4 hrs x90 dys wheeze" specifies that the patient should inhale 2 (dosage) puffs (form) by mouth (route) every 4 hours (frequency) for 90 days (duration) for wheezing (reason). while not all components are present in every e-prescription direction, some components are critical and need to be stated explicitly. ## results we present our results first by evaluating the nmt module by comparing the two proposed architectures using the automated bleu and meteor scores. after finalizing the best performing nmt model, we compare its results against a rule-based baseline and demonstrate the significance of the individual stages through an ablation study. finally, we report the performance of the end-to-end system based on the manual evaluation. ## conclusion we proposed and developed a machine translationbased approach, called pharmmt, to simplify eprescription directions. we systematically evaluated the individual stages and the overall approach over a large mail-order pharmacy data corpus. our results showed that an ensemble model with a bidirectional lstm encoder and an lstm decoder, trained over a clinical-domain word embedding representations, achieved the best overall bleu score of 60.27. ner-based numeric check and graceful backoff ensure information consistency and the normalization stage helps generate patient-friendly directions. qualitative evaluation by domain experts showed that 94.3% of the simplified directions could be used as-is or with minimal changes. these results indicate that the proposed approach could be deployed in practice to automate the simplification of prescription directions. and variability of patient directions in electronic prescriptions in the ambulatory care setting. journal of managed care specialty pharmacy, 24(7):691-699. sevinj yolchuyeva, géza németh, and bálint gyires-tóth. 2018. text normalization with convolutional neural networks. international journal of speech technology, 21(3):589-600. xinyan zhao and v.g.vinod vydiswaran. 2020. lex-transfer: break data discrepancy with lexically enhanced extraction transfer learning for prescription parsing. under review at the journal of the american medical informatics association. yifan zheng, yun jiang, michael p. dorsch, yuting ding, v.g. vinod vydiswaran, and corey a. lester. 2020. work effort, readability and quality of pharmacy transcription of patient directions from electronic prescriptions: a retrospective observational cohort analysis. in bmj quality & safety.