P rompt EHR : Conditional Electronic Healthcare Records Generation with Prompt Learning (2022)

Abstract: Accessing longitudinal multimodal Electronic Healthcare Records (EHRs) is challenging due to privacy concerns, which hinders the use of ML for healthcare applications. Synthetic EHRs generation bypasses the need to share sensitive real patient records. However, existing methods generate single-modal EHRs by unconditional generation or by longitudinal inference, which falls short of low flexibility and makes unrealistic EHRs. In this work, we propose to formulate EHRs generation as a text-to-text translation task by language models (LMs), which suffices to highly flexible event imputation during generation. We also design prompt learning to control the generation conditioned by numerical and categorical demographic features. We evaluate synthetic EHRs quality by two perplexity measures accounting for their longitudinal pattern (longitudinal imputation perplexity, lpl) and the connections cross modalities (cross-modality imputation perplexity, mpl). Moreover, we utilize two adversaries: membership and attribute inference attacks for privacy-preserving evaluation. Experiments on MIMIC-III data demonstrate the superiority of our methods on realistic EHRs generation (53.1% decrease of lpl and 45.3% decrease of mpl on average compared to the best baselines) with low privacy risks. Software is available at https://github.com/RyanWangZf/PromptEHR.

## introduction the prevalence of electronic patient healthcare records fuel the development of machine learning models for many healthcare applications @xcite @xcite . however, sharing ehr data usually undergoes strict and expensive de-identification and administration processes thus being difficult. although there have been attempts on perturbing potentially identifiable attributes as the deidentification step @xcite , they were figure 1 : a conceptual demonstration of how promptehr works more flexible than all existing works. v t indicates the t-th visit; dx, med are short for diagnosis and medication events; red are the targets to generate. our method (promptehr) is amenable to four new conditional generation ways thus more controllable and flexible. deep generative models like gans @xcite and vaes @xcite have become popular for unconditional ehrs generation @xcite and longitudinal ehrs generation @xcite for diagnosis codes. however, ehrs are often multimodal with different types of events, including diagnoses, procedures, medications, and also patient baseline demographic features like age and gender @xcite . gans & vaes usually struggle to model complex multimodal and non-gaussian distributions as well as sparse onehot-encoded vectors @xcite . by contrast, generative language models (lms) are proved highly powerful to represent large and complex distributions on discrete data (e.g., texts) @xcite , which makes them promising for ehrs generation. in this work, we propose to leverage generative language models (lms) for ehrs generation. we try to generate a sequence of visits with mixed types of events, e.g., diagnosis and medications. as fig. 1 shows, previous works make unconditional generation for single-modal static ehrs @xcite or for single-modal longitudinal ehrs @xcite . however, real ehrs are heterogeneous with multiple types of temporal events and have baseline patient features, e.g., demographic information. we seek to ( 1 • we design prompt learning for controllable and flexible ehrs generation with lms. ## methods in this section, we elaborate on the main framework of promptehr, including the problem setting, workflow, and training tasks formulation. next, we discuss the strategies for generating diverse synthetic ehrs with minor loss of quality. then, we present the recipe proposed for the evaluation for both quality and privacy-preserving ability of the ehrs generation models. ## experiments in this section, we designed experiments to answer the following questions. ## conclusion in this paper, we study how to leverage real ehrs to train a prompt learning based generative language model for synthetic ehrs generation, namely promptehr. unlike previous ehrs generation methods, promptehr is able to learn from and generate heterogeneous ehrs. to evaluate its performance, we draw the idea of perplexity from the text generation literature and propose two per-plexity measures: spatial and temporal perplexity. experiments on mimic-iii data demonstrates the quality of generated ehrs are better than the baselines. the synthetic data provides both utility and privacy for downstream healthcare applications.