JMIR MEDICAL INFORMATICS

McDonough et al

Original Paper

Optimizing Antihypertensive Medication Classification in Electronic
Health Record-Based Data: Classification System Development
and Methodological Comparison
Caitrin W McDonough1, MS, PhD; Steven M Smith1, PharmD, MPH; Rhonda M Cooper-DeHoff1,2, PharmD, MS;
William R Hogan3, MD, MS
1

Department of Pharmacotherapy and Translational Research, College of Pharmacy, University of Florida, Gainesville, FL, United States

2

Division of Cardiovascular Medicine, Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, United States

3

Department of Health Outcomes & Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, United States

Corresponding Author:
Caitrin W McDonough, MS, PhD
Department of Pharmacotherapy and Translational ResearchCollege of Pharmacy
University of Florida
PO Box 100486
Gainesville, FL,
United States
Phone: 1 3522736435
Fax: 1 3522736121
Email: cmcdonough@cop.ufl.edu

Abstract
Background: Computable phenotypes have the ability to utilize data within the electronic health record (EHR) to identify
patients with certain characteristics. Many computable phenotypes rely on multiple types of data within the EHR including
prescription drug information. Hypertension (HTN)-related computable phenotypes are particularly dependent on the correctclassification of antihypertensive prescription drug information, as well as corresponding diagnoses and blood pressure information.
Objective: This study aimed to create an antihypertensive drug classification system to be utilized with EHR-based data as partof HTN-related computable phenotypes.
Methods: We compared 4 different antihypertensive drug classification systems based off of 4 different methodologies andterminologies, including 3 RxNorm Concept Unique Identifier (RxCUI)–based classifications and 1 medication name–basedclassification. The RxCUI-based classifications utilized data from (1) the Drug Ontology, (2) the new Medication Reference
Terminology, and (3) the Anatomical Therapeutic Chemical Classification System and DrugBank, whereas the medication
name–based classification relied on antihypertensive drug names. Each classification system was applied to EHR-based prescription
drug data from hypertensive patients in the OneFlorida Data Trust.
Results: There were 13,627 unique RxCUIs and 8025 unique medication names from the 13,879,046 prescriptions. We observeda broad overlap between the 4 methods, with 84.1% (691/822) to 95.3% (695/729) of terms overlapping pairwise between the
different classification methods. Key differences arose from drug products with multiple dosage forms, drug products with anindication of benign prostatic hyperplasia, drug products that contain more than 1 ingredient (combination products), and terms
within the classification systems corresponding to retired or obsolete RxCUIs.
Conclusions: In total, 2 antihypertensive drug classifications were constructed, one based on RxCUIs and one based on medication
name, that can be used in future computable phenotypes that require antihypertensive drug classifications.
(JMIR Med Inform 2020;8(2):e14777) doi: 10.2196/14777
KEYWORDS
antihypertensive agents; electronic health records; classification; RxNorm; phenotype

http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 1
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

McDonough et al

Introduction

Methods

BackgroundResources

Electronic health records (EHRs) contain a wealth of clinicalinformation, and the development of tools to perform structured
queries of common data models (CDMs) to standardized data
formats has allowed researchers to utilize EHR data fordiscriminating complex clinical phenotypes with highmeasurement validity [1-4]. Measuring such complex
phenotypes often requires integration of multiple streams of
EHR data, including diagnoses or procedures, clinicalmeasurements (eg, vitals and laboratory parameters), andmedications or other exposures [5]. Additionally, the accurate
measurement of these complex phenotypes relies on accurate
measurement and classification of these individual components[6,7]. The classification of medication use in this context offers
unique challenges because many medications have multiple
indications and dosage forms. Furthermore, existing therapeuticclassification systems generally group drug entities in ways that
may only partially correlate with intended use [8], eg, numerous
medications block beta-adrenergic receptors (beta-blockers),
and such an effect can lower blood pressure (BP), but not all
beta-blockers are used clinically as antihypertensives. Evenamong beta-blockers that are used clinically as
antihypertensives, not all dosage forms effectively treat systemic
hypertension (HTN) [9]. Thus, the mere presence of a drug
entity in the prescribing record may not be sufficient to meet
medication-related criteria in a complex phenotype.

RxNormObjectives
Accordingly, we aimed to design a set of standardized
antihypertensive drug codes to be usable and maintainable byboth ourselves and others in the research community [10]. We
performed this study in the context of optimizing medication
classification for use in HTN-related computable phenotypes.
To properly identify patients who are taking antihypertensive
medications, the prescription drug information from the EHR
must be properly classified into antihypertensive therapeutic
indication and antihypertensive drug classes. We designed ourstudy by utilizing concepts (RxNorm and medication name)present in the prescription drug information from data models
common in the United States (Informatics for Integrating
Biology & the Bedside [i2b2], The National Patient Centered
Clinical Research Network [PCORnet] CDM, and The
Observational Medical Outcomes Partnership [OMOP] CDM)
[1,3,4]. The main objective of this study was to report thedevelopment of a set of standardized drug codes and names foruse in querying EHR data for antihypertensive medicationprescriptions. The second objective was to compare different
methods and resources for creating the antihypertensive drugclassification. The third and final objective was to explore thecoverage of different methods and resources (for creating theclassification) on querying a large EHR-based dataset.

RxNorm is a standardized terminology to represent drugs. It
was developed by the US National Library of Medicine (NLM)
in 2002 and represents medications through normalized names
for clinical drugs, which include ingredient or ingredientsstrength or strengths, and dose form [11,12]. The normalized
names allow equivalent drug terms from different source
vocabularies to be grouped together under the same RxNorm
Concept Unique Identifier (RxCUI) [11]. RxNorm was designed
to support electronic prescribing, mapping between different
drug vocabularies, and the development of medication-related
clinical decision support rules [11,12]. RxNorm can be accessed
through an application programming interface (API), theRxNorm Navigator (RxNav) [12-14]. Currently, RxNorm
integrates drug terminology from many sources including mostdrug knowledge bases (eg, Multum, Micromedex, and First
DataBank), standard terminologies (eg, SNOMED CT and
MeSH), the new Medication Reference Terminology(MED-RT), federal agencies in the United States (eg, Food andDrug Administration Structured Product labels), andinternational drug resources such as the Anatomical Therapeutic
Chemical (ATC) Classification System and DrugBank [11,15].

Drug Ontology
The Drug Ontology (DrOn) is a formal representation of drug
products, drug ingredients, mechanisms of action, therapeutic
indications, strengths, and dosage forms based on the OWL2Web Ontology Language [8,16-18]. These representations werecreated to allow researchers to query drug datasets, whichusually come from EHRs or health insurance claim databases
and typically use RxCUIs to identify different aspects of drug
products (ie, ingredient or ingredients, dose forms, and strength
or strengths), and National Drug Codes to identify individual
drug products and the manufacturer and packaging thereof[16,17]. Currently, DrOn represents a drug’s therapeutic
indication as being a property of a drug product, which is acomposite of one or more drug ingredients and excipients,
whereas its mechanism of action (MoA) is represented as aproperty of the chemical compound or chemical compounds
that constitutes the drug’s ingredients [8,16,17]. DrOn has a
hand-curated component and a component built automaticallyfrom RxNorm [16-18]. At present, DrOn contains
representations for several mechanisms of action belonging tovarious antihypertensive drugs classes [8,16,17]. It also contains
class representations for the drug products and ingredients,
dosage forms, strengths, and therapeutic indications.

Data Source
OneFlorida and the OneFlorida Hypertension
Population
The OneFlorida Clinical Research Consortium is a statewide
network of health systems, providers, and payers covering morethan 74% of Florida’s population [19]. The network’s catchment
area covers all 67 Florida counties and allows for the facilitation

http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 2
(page number not for citation purposes)

JMIR MEDICAL INFORMATICSof clinical and translational research in health care settings andcommunities throughout the state. OneFlorida houses a Data
Trust, which contains longitudinal EHR data on approximately
14 million Floridians (approximately 66%), mapped to the
PCORnet CDM [19,20]. The HTN population within OneFlorida
was defined as all adults (aged ≥18 years) with ≥1 HTN
diagnosis from an outpatient encounter, defined as International
Classification of Diseases (ICD)-9 code 401.x (Essential HTN)
or ICD-10 code I10 (Essential [primary] HTN). The data utilized
for this study were extracted from the OneFlorida Data Truston December 14, 2017, in the PCORnet CDM, version 3.0, andincluded EHR data from June 2000 to July 2017, with 99.97%
from encounters occurring from January 2011 onward. All(100%) of the prescription drug data were from January 2011
onward.

Prescription Drug Data
All prescription drug data for the HTN population were extracted
from the Prescribing Table, including information on raw
medication name and RxCUI. A total of 2 drug lists were createdfrom this dataset. The first contained all unique raw medication
names (see Drug Classification by Ingredient Name) derived
directly from source EHRs, ie, not cleaned or curated duringmapping to the PCORnet CDM. The second contained all unique
RxCUIs (see Drug Classification by RxCUI utilizing DrOn andDrug Classification by RxCUI utilizing RxClass), which may
be derived from source EHRs or created during mapping to the
PCORnet CDM.

Drug Classification by Ingredient Name
A Medication Name Classification was constructed forantihypertensive medications, utilizing drug ingredient names.
A summary of the features included is available in Multimedia
Appendix 1. Both brand name and generic name were included.
The list was constructed through manual curation, including
methods from prior antihypertensive drug classifications [21,22].
The manually curated list was also reviewed by authors withbiomedical informatics expertise (CM) and HTN
pharmacotherapy expertise (CM and RC). The Medication NameClassification contains 286 drug ingredient names, further
classified by antihypertensive drug class (eg, beta-blockers,
angiotensin II receptor inhibitors [ARBs], calcium channel
blockers [CCBs], etc).
To apply the Medication Name Classification to the OneFlorida
raw medication list, the first word in the field was extracted as
the ingredient name, with additional coding to capturecombination medications (eg, adding underscores between thefirst and second words) and strings where the ingredient was
not the first word. The antihypertensive Medication NameClassification was merged with the unique raw medication
names from the OneFlorida dataset to map the drugs by
antihypertensive drug class. All of the raw medication namesthat did not merge with the Medication Name Classification
were discarded (eg, statins, insulin, etc).

Drug Classification by RxNorm Concept Unique
Identifier Utilizing the Drug Ontology
The DrOn RxCUI Classification was constructed throughmultiple steps (Multimedia Appendix 1). First, all RxCUIs with
http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

McDonough et ala therapeutic indication for HTN (antihypertensive function)
were extracted from DrOn using the dron-query tool [8]. Then,
separate lists of RxCUIs were extracted for drugs with any of
the following mechanisms of action: angiotensin-converting
enzyme (ACE)–inhibitor, ARB, beta-blocker, CCB, loop
diuretic, and thiazide and thiazide-like diuretic. These lists werethen merged to assign an antihypertensive MoA to each RxCUI
with a therapeutic indication for HTN. Combination drug
products were assigned multiple mechanisms of action,
representing each ingredient. The list was then manuallyreviewed, and mechanisms of action were added for drug
products with mechanisms of action not currently represented
in DrOn. These included the following: aldosterone antagonists,
direct renin inhibitors, alpha-1 blockers, potassium-sparing
diuretics, vasodilators, centrally acting agents, and other agents.
The list was then reviewed by authors with biomedical
informatics expertise (CM and WH) and HTN pharmacotherapy
expertise (CM, SS, and RC). The DrOn RxCUI Classification
contains 2543 antihypertensive RxCUIs, of SCDF, SCD, and
SBD term types, organized by antihypertensive drug class or
drug classes.
The unique RxCUIs from the OneFlorida dataset were mergedwith the DrOn RxCUI Classification to map the drugs by
antihypertensive drug class. All of the RxCUIs that did not
merge with the DrOn RxCUI Classification were discarded (eg,statins, insulin, etc).

Drug Classification by RxNorm Concept Unique
Identifier Utilizing RxClass
The unique list of RxCUIs extracted from OneFlorida were also
mapped utilizing the RxClass API on RxMix [23]. RxCUIs withless than 4 digits were removed (n=25). The function,“getClassByRxNormDrugId,” was used to obtain the drugclasses for a specified drug identifier. In total, 2 different
relationship sources (relaSource) were tested: ATC and
MED-RT (Multimedia Appendix 1). Within MED-RT, thefollowing relationships (rela) were selected: “has_MoA” and“may_treat.” The unique RxCUIs from the OneFlorida dataset
were classified using the ATC and MED-RT relationship sources
through the batch input mode.

Comparison Between Drug Classifications
The different drug classification methods were compared
pairwise by calculating percent coverage and by reviewing theoverlapping and nonoverlapping sets of RxCUIs among them.
For all classification methods, the percent of antihypertensive
drugs covered was calculated as the number of antihypertensive
medications mapped by the classification method divided by
the total number of unique terms (Raw Name or RxCUI)
contained in the OneFlorida Prescribing Table. Within the ATC
relationship source from RxClass, the antihypertensive classes
were selected from the “name” field. A complete list of the ATC
relationships included as antihypertensive drugs is available inMultimedia Appendix 2. Within the MED-RT relationshipsource from RxClass, 2 steps were used to selectantihypertensive drugs: (1) RxCUIs were selected with the“may_treat” HTN relationship and (2) RxCUIs were further
filtered based on the “has_MoA” relationship, including only
those drugs from antihypertensive classes. A complete list of
JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 3
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

McDonough et al

the MED-RT “has_MoA” relationships included as
antihypertensive drugs is available in Multimedia Appendix 3.
Within the MED-RT relationship source from RxClass, theresults from “may_treat” relationship were used to calculate
total coverage. Differences between the antihypertensive
coverage results were identified by pairwise comparisons andmerges between the DrOn RxCUI Classification and every otherclassification.

coverage by each method was calculated as number ofprescription records mapped to an antihypertensive drug classby each map divided by the total number of prescriptions.
Differences between the coverage results were identified bypairwise comparison. Additionally, summary-level frequency
counts and percentages by the antihypertensive drug class werealso calculated. All coding for mapping the drug data and thesummary statistics was conducted using SAS version 9.4 (SAS).

On the basis of our review of the overlapping andnonoverlapping sets of names and RxCUIs among theclassifications, we created a version 1.0 of 2 finalizedclassifications—one for ingredient names and one for
RxCUIs—for use by us and other researchers. We publishedthem on GitHub [10] to make them findable and reusable andto enable community contributions to future versions.

Results

Application of the Drug Classifications to OneFlorida

In total, there were 13,879,046 prescriptions from these patientsover the study period (January 2011 to July 2017; approximately
6.6 years). These prescriptions consisted of 13,627 unique
RxCUIs and 8025 unique first words from the raw medication
name string (Table 1).

The Medication Name Classification and the DrOn RxCUIClassification were applied to all of the prescription drug datafrom the OneFlorida HTN population. All available prescriptions
from January 2011 onward were considered. Antihypertensive

Data Source
At the time of the data extraction, there were 1,188,977 patientsin the OneFlorida Data Trust with an ICD-9 or ICD-10 diagnosis
code for HTN (Table 1).

Table 1. Counts of data attributes in the OneFlorida hypertensive patient prescribing table dataset.
Data attributes

Values, N

Hypertensive patients

1,188,977

Prescription records

13,879,046

Unique RxCUIsaUnique Raw Med Nameba13,6278025

RxCUI: RxNorm Concept Unique Identifier.

bUnique first word of the Raw_Med_Name field, after additional data cleaning and quality control steps.

Drug Classification by Ingredient Name
The initial Medication Name Classification contained 286
antihypertensive medications that are mapped to 35
antihypertensive medication classes or combination medication
classes (eg, ACE inhibitors, ARBs, CCBs, CCB-ARB
combinations, etc). We chose to exclude timolol to be

http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

conservative. On the basis of this classification system, it is
impossible to distinguish between oral and ophthalmic products.
An excerpt from the Medication Name Classification is shownin Table 2, whereas the full map is available in Multimedia
Appendix 4. Each entry consists of an arbitrary code, the single
word drug name, the generic name (if applicable), and the drugclass.

JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 4
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

McDonough et al

Table 2. Excerpt from Medication Name Classification.

a

Codea

Drug_Name

Genericb

Drug_Class

1

Benazepril

—c

ACEd

2

Lotensinbenazepril

ACE

3

Captopril

—

ACE

4

Capotencaptopril

ACE

5

Enalapril

—

ACE

6

Enalaprilat

enalapril

ACE

7

Fosinopril

—

ACE

8

Monoprilfosinopril

ACE

9

Lisinopril

—

ACE

10

Prinivillisinopril

ACE

11

Zestrillisinopril

ACE

Full classification list available in Multimedia Appendix 4.

bGeneric drug name included for brand name drugs.

c

Not applicable for generic drugs.

dACE: angiotensin-converting enzyme inhibitor.

Drug Classification by RxNorm Concept Unique
Identifier Utilizing the Drug Ontology
The initial DrOn RxCUI Classification contained 2543
antihypertensive RxCUIs that were mapped to 46

antihypertensive medication classes or combination medication
classes. Each RxCUI entry contains the RxCUI, the drug
product, and the drug class. An excerpt of the DrOn RxCUIClassification is shown in Table 3, and the full map is available
in Multimedia Appendix 5.

Table 3. Excerpt from the Drug Ontology RxNorm Concept Unique Identifier classification.

aRxCUIa,b

Drug_Product

Rx_Norm_Drug_Class

858926

Enalapril Maleate 20 MG Chewable Tablet

ACEc

378269Enalapril Chewable Tablet

ACE

858810

Enalapril Maleate 20 MG Oral Tablet

ACE

858804

Enalapril Maleate 2.5 MG Oral Tablet

ACE

858821

Enalapril Maleate 1.25 MG/ML Injectable Solution

ACE

858817

Enalapril Maleate 10 MG Oral Tablet

ACE

858813

Enalapril Maleate 5 MG Oral Tablet

ACE

372007

Enalapril Oral Tablet

ACE

378288

Enalapril Injectable Solution

ACE

246264

Enalaprilat 1 MG/ML Injectable Solution

ACE

374378

Enalaprilat Injectable Solution

ACE

252820

Enalaprilat 0.625 MG/ML Injectable Solution

ACE

204404

Enalaprilat 1.25 MG/ML Injectable Solution

ACE

Full map available in Multimedia Appendix 5.

b

RxCUI: RxNorm Concept Unique Identifier.

c

ACE: angiotensin-converting enzyme inhibitor.

Comparison Between Drug Classifications
The percent of drugs covered by each antihypertensive map isshown in Table 4. Using the Medication Name Classification,
207 ingredient terms were mapped to antihypertensive drug
http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

classes out of a total of 8025 unique raw medication names(2.58%; Table 4). When the DrOn RxCUI Classification wasapplied to the unique RxCUIs (n=13,627), 729 were successfully
mapped to antihypertensive drug classes (5.35%; Table 4).
Through the RxClass API, using ATC as the relationship source,
JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 5
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

McDonough et al

822 out of 13,602 RxCUIs were mapped to antihypertensive
drug classes (6.04%; Table 4). Finally, when the RxClass API
was used with MED-RT as the relationship source, 792 RxCUIs
had the relationship of “may_treat” HTN and a “has_MoA” that
corresponded to an antihypertensive drug class (792/13,602,
5.82%; Table 4).
When the DrOn RxCUI Classification was compared with theother classifications, they broadly overlapped; however, there
were some key differences (Figure 1). When the DrOn RxCUIClassification was compared with the other RxCUI-based maps
constructed through the RxClass API, 691 terms overlappedwith the RxClass-ATC map and 683 terms overlapped with the
RxClass–MED-RT map. There were 38 terms and 46 terms thatwere unique to the DrOn RxCUI map when compared with the

RxClass-ATC map and RxClass–MED-RT map, respectively.
Additionally, there were 131 terms that were unique to the
RxClass-ATC map, and 109 terms that were unique to the
RxClass–MED-RT map. Of these 131 terms and 109 terms,
respectively, there were 135 unique terms, with 105 overlapping
between the RxClass-ATC map and the RxClass–MED-RT
map.
Of the 135 unique RxCUIs from the RxClass ATC and MED-RT
maps, 29 had a term type of IN, MIN, or PIN, representing
antihypertensive medications. These term types were notincluded in the DrOn RxCUI map owing to their low level of
specificity. Next, there were 24 RxCUIs that were either
ophthalmic or topical drug products. Examples of these areshown in Table 5.

Table 4. Drug coverage by each classification method.
Classification method

Medication Name

Input, N

Antihypertensive
Mapped, n

Coverage, %Raw Name

8025

207

2.58

b

RxCUI

13,627729

5.35

RxClass-ATCc

RxCUI

13,602822

6.04

RxClass–MED-RTd,e

RXCUI

13,602

792

5.82

a

DrOn RxCUIaInput term

DrOn: Drug Ontology.

b

RxCUI: RxNorm Concept Unique Identifier.

c

ATC: Anatomical Therapeutic Chemical.

d

MED-RT Antihypertensive coverage was mapped using 2 steps: (1) "may_treat" hypertension and (2) mechanism of action of an antihypertensive
drug class.
e

MED-RT: Medication Reference Terminology.

Figure 1. Comparisons of the Drug Ontology RxNorm Concept Unique Identifier Classification to the other classification methods. Results are shownfor the number overlapping between the methods (Both) and the numbers unique to each method. ATC: Anatomical Therapeutic Chemical; MED-RT:
Medication Reference Terminology.

http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 6
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

McDonough et al

Table 5. Examples of RxNorm Concept Unique Identifiers inappropriately mapped as antihypertensives utilizing the RxMix tools.

a

RxCUIa DrugName hasMOA

ConceptName

DoseForm

207371Minoxidil

Potassium Channel Interactions

Minoxidil 20 MG/ML Topical Solution (Rogaine)

Topical solution

208560

Timolol

Adrenergic beta1-Antagonists

Timolol 2.5 MG/ML Ophthalmic Solution (Betimol)

Ophthalmic solution

213729Betaxolol

Adrenergic beta1-Antagonists

Betaxolol 2.5 MG/ML Ophthalmic Suspension (Betoptic S)

Ophthalmic suspension

RxCUI: RxNorm Concept Unique Identifier.

Of the remaining 82 RxCUIs, 11 were RxCUIs corresponding
to alpha-blockers primarily indicated for benign prostatic
hyperplasia, 6 were RxCUIs for sacubitril/valsartan (Entresto)
indicated for chronic heart failure with reduced ejection fraction,
1 was an RxCUI corresponding to an injectable antihypertensive
product that is rarely used to treat HTN outside of a hypertensivecrisis, and 1 was the IN term-type RxCUI for Potassium. A totalof 63 were true antihypertensive drugs that were inadvertently
omitted from the DrOn RxCUI map.

Following the comparisons between the different drugclassifications, 96 RxCUIs were added to the DrOn RxCUIClassification, and 15 antihypertensive medication names wereadded to the Medication Name Classification. The DrOn RxCUIClassification version 1.0 contains 2639 RxCUIs and is available
in Multimedia Appendix 6, and the Medication NameClassification version 1.0 contains 301 antihypertensive
medication names and is available in Multimedia Appendix 7[10].

When the DrOn RxCUI Classification was compared with theMedication Name Classification using the DrOn RxCUIClassification as the reference, 695 terms overlapped, and 34
terms were unique to the DrOn RxCUI Classification (Figure1). When the DrOn RxCUI Classification was compared with
the Medication Name Classification using the Medication NameClassification as the reference, 195 terms overlapped, and 12
terms were unique to the Medication Name Classification. In
total, 11 of the 12 terms that were unique to the Medication
Name Classification were also included in the 135 terms thatwere unique to the RxClass ATC and MED-RT maps. The otherterm represented the brand name (Loniten) for the vasodilator
minoxidil and was not present in any of the other classifications.

Application of the Drug Classifications to OneFlorida

From the comparisons, the DrOn RxCUI Classification had 71
unique RxCUIs/terms that were not present in the RxClass
MED-RT, RxClass ATC, and/or Medication Nameclassifications. Many were combination products (n=29), andthe remaining 42 RxCUIs were spread across the otherantihypertensive drug classes: ACE inhibitor (n=1),
alpha-blocker (n=1), ARB (n=1), beta-blocker (n=17), CCB(n=11), loop diuretic (n=1), thiazide diuretic (n=3), andvasodilator (n=7).

The results from applying the DrOn RxCUI Classification v1.0
and the Medication Name Classification v1.0 to the prescribing
information from the OneFlorida HTN population are shownin Table 6.
Overall, the methods performed very similarly, withapproximately 15% (2,080,685 versus 2,089,557/13,879,046)
of the total prescriptions mapping to antihypertensive drugs.
The DrOn RxCUI Classification v1.0 was able to map 8872
more prescription records to an antihypertensive drug classcompared with the Medication Name Classification v1.0 (Table
6). When the different methods were compared byantihypertensive class, all classes were within 1% of each other(eg, 443,835/2,089,557, 21.24% vs 443,540/2,080,685, 21.32%
of prescriptions mapped to the ACE inhibitor class using the
DrOn RxCUI Classification method and Medication NameClassification method, respectively). Table 7 shows the specificantihypertensive class or classes each prescription was mapped
to using the DrOn RxCUI Classification v1.0. The majority ofthe prescriptions mapped to a single antihypertensive class (eg,ACE inhibitors, beta-bockers; Table 7). However, 10.42%(217,682/2,089,557) mapped to combination antihypertensive
products (Table 7).

Table 6. Summary of the methods to the OneFlorida Antihypertensive Prescribing Dataset.
Classification methodaAntihypertensive (N=13,879,046)
Mapped, n

Coverage, %Medication Name

2,080,685

14.99

DrOna RxCUIb

2,089,557

15.06

DrOn: Drug Ontology.

b

RxCUI: RxNorm Concept Unique Identifier.

http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 7
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

McDonough et al

Table 7. Application of the Drug Ontology RxNorm Concept Unique Identifier Classification to the OneFlorida antihypertensive prescribing dataset(N=2,089,557).

aAntihypertensive Drug Class or Classesa

Frequency, n (%)

ACEb

443,835 (21.24)

Beta-blocker

411,721 (19.70)

Calcium Channel Blocker

360,653 (17.26)

Diuretic (Thiazide/Thiazide Like)

217,474 (10.41)

ARBc

178,252 (8.53)

Loop Diuretic

115,931 (5.55)

Diuretic/ACE Combo

92,275 (4.42)

Diuretic/ARB Combo

63,010 (3.02)

Centrally Acting

56,501 (2.70)

Vasodilator

30,665 (1.47)

Aldosterone Antagonist

29,214 (1.40)

Alpha Blocker

26,995 (1.29)

Results are shown for antihypertensive drug classes that represent ≥1% of all antihypertensive prescriptions among the OneFlorida HTN population.

b

ACE: angiotensin-converting enzyme inhibitorc

ARB: angiotensin II receptor inhibitor.

Discussion
We created 2 different medication classification systems forantihypertensive drugs: one utilizing medication names and theother utilizing RxCUIs. After comparing these classification
systems to each other, and to existing drug class terminologies
available through RxNorm, we identified key areas that canlead to misclassification of antihypertensive medications anddrug classes. Most misclassifications stemmed from failure todiscriminate between dosage forms or issues related to primary
indications of drugs (eg, selection of drugs that are primarily
indicated for benign prostatic hyperplasia).
To illustrate, timolol is a beta-blocker that has both oral andophthalmic dosage forms. The oral form is used to treat HTN,
whereas the ophthalmic form is used to treat glaucoma [9,24].
An ideal antihypertensive drug classification system would
include the oral form of timolol, but not the ophthalmic form.
This is exactly what we see with the DrOn RxCUI Classification,
as DrOn represents the therapeutic indication (HTN) for oral
timolol separate from its MoA (beta-blocker) [8]. In contrast,
all forms of timolol were included in our analysis that utilized
the ATC and MED-RT terminologies. ATC only modeled thedrug class, with all forms of timolol included as beta-blocking
agents. MED-RT allowed for filtering through both a MoA
relationship and a may-treat relationship. However, all formsof timolol, both oral and ophthalmic are mapped back to theingredient term type for timolol, which has the may-treat
relationship with HTN. Although this method allows for
simplicity in mapping multiple drug products, it assigns anincorrect therapeutic indication (for our purposes) to theophthalmic forms of timolol. We observed similar
misclassification for other medications that have multiple
clinical uses beyond HTN (eg, the vasodilator minoxidil andthe alpha-1-blockers silodosin, alfuzosin, and tamsulosin).
http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

Minoxidil has oral dosage forms to treat HTN and topical foams
to treat hair loss [25,26], whereas silodosin, alfuzosin, andtamsulosin are all alpha-1-blockers that are indicated for thetreatment of benign prostatic hyperplasia as opposed to HTN[27,28].
Retired RxCUIs, or those that have been remapped to otherclasses, were classified by DrOn but not by MED-RT or ATC.
This illustrates the need for maps and terminologies that include
retired and obsolete RxCUIs as many of our longitudinal data
sources include these. The data source used in this study contains
data from January 2011 to July 2017, and contained 1170
RxCUIs that have been retired and 421 that have been remapped.
We conducted this work in the context of optimizing
antihypertensive medication classification for use in
HTN-related computable phenotypes. In other disease states
where there are not as many options for pharmaceuticaltreatment, the classification of drug products may not be ascomplicated. However, in the case of HTN, and particularly thecomplex clinical phenotype of resistant hypertension (RHTN)[29-31], there are multiple classes of antihypertensive drugs
with different mechanisms of action [30,32]. With RHTN, it isnecessary to properly assign each antihypertensive medication
to a medication class to determine if a patient meets thedefinition for RHTN, which is classically defined as requiring
4 or more antihypertensive drugs from different antihypertensive
drug classes to achieve BP control [30,31]. This also illustrates
the need to include a subject matter expert (eg, PharmD) during
the creation of drug classification systems.
We also observed differences in the classification of
combination drug products. There were some combination
antihypertensive drug products that were not identified in the
MED-RT terminology through the methods that we utilized inthis study. Additionally, when utilizing a classification system
JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 8
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
based on medication name, all possible permutations of the
combination name must be considered and included in the map(eg, HCTZ-metoprolol, hydrochlorothiazide-metoprolol,
metoprolol-HCTZ, and metoprolol-hydrochlorothiazide).
Without each of these permutations, it is possible to miss certain
combination products. Finally, as a phenotype such as RHTN
is determined partially, or fully, based on the antihypertensive
drug count, a correct drug count must be assigned to eachcombination product. We have added this field into our DrOn
RxCUI Classification v1.0.
Our study is not without limitation; currently, we do not havea gold standard antihypertensive medication classification list.
We selected different data sources and compared classification
based on these sources. However, we only used terminologies
available through the US NLM. We have not included
terminologies maintained by other groups (eg, American
Hospital Formulary Service Pharmacologic-Therapeutic
Classification) [33]. In addition, to classify based off of RxCUI,
an RxCUI must be present. This may require additional workin some health care systems or datasets. We did not explore thetemporal relationship between HTN diagnosis andantihypertensive medication prescription within our data. This
could have resulted in the inclusion of medication data that wasprescribed for indications other than HTN (eg, heart failure,atrial fibrillation, etc). We can examine this relationship morein our future work. Additionally, we did not explore the effect

McDonough et alof the different methods and resources for creating the drugclassification on the number of patients with RHTN within
OneFlorida. We chose our study methodology because we firstneeded to create and validate this antihypertensive drugclassification before we could use it in creating and validating
our RHTN computable phenotype. Once our RHTN computablephenotype is validated, we can subsequently explore the effectof the different drug classification methods on the RHTN
phenotype in future work. Other future work will include
incorporating RxNorm Term Type (SCDF, SCD, SBD, etc),
indicating retired RxCUIs, expanding to other disease states(diabetes, chronic kidney disease, heart failure, etc), andexploring the application to clinical decision support within the
EHR [34,35].
In conclusion, we created and compared 4 different drugclassification methods, focusing on the classification ofantihypertensive drug products. We observed key differencesin how each classification system handled drug products withmultiple dosage forms, drug products with indications for benign
prostatic hyperplasia, drug products that contain multiple
antihypertensive ingredients (combination drug products), andRxCUIs that have been retired or remapped. We constructed 2
antihypertensive drug classification systems, 1 based off ofRxCUIs and 1 based off of medication names. These areavailable for public use [10], and we will continue to updatethem through our own research.

Acknowledgments
Support for this project came from National Institutes of Health (NIH) grants KL2 TR001429 (CM), K01 HL141690 (CM), and
K01 HL138172 (SS). Additionally, the research reported in this publication was supported in part by the OneFlorida Clinical
Data Network, funded by the Patient-Centered Outcomes Research Institute (PCORI) #CDRN-1501-26692, in part by the
OneFlorida Cancer Control Alliance, funded by the Florida Department of Health’s James and Esther King Biomedical Research
Program #4KB16, and in part by the University of Florida Clinical and Translational Science Institute, which is supported in partby the NIH National Center for Advancing Translational Sciences under award number UL1TR001427. The content is solely the
responsibility of the authors and does not necessarily represent the official views of the PCORI, its Board of Governors or
Methodology, the OneFlorida Clinical Research Consortium, the University of Florida’s Clinical and Translational Science
Institute, the Florida Department of Health, or the NIH.

Conflicts of Interest
WRH is one of the creators of DrOn.

Multimedia Appendix 1
Features of each Classification Method.
[XLSX File (Microsoft Excel File), 9 KB-Multimedia Appendix 1]

Multimedia Appendix 2
List of Anatomical Therapeutic Chemical relationships included as antihypertensive drugs.
[XLSX File (Microsoft Excel File), 9 KB-Multimedia Appendix 2]

Multimedia Appendix 3
List of Medication Reference Terminology has_MoA relationships included as antihypertensive drugs.
[XLSX File (Microsoft Excel File), 9 KB-Multimedia Appendix 3]

Multimedia Appendix 4
Initial Medication Name Classification.
http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 9
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

McDonough et al[XLSX File (Microsoft Excel File), 16 KB-Multimedia Appendix 4]

Multimedia Appendix 5
Initial Drug Ontology RxNorm Concept Unique Identifier Classification.
[XLSX File (Microsoft Excel File), 80 KB-Multimedia Appendix 5]

Multimedia Appendix 6
Drug Ontology RxNorm Concept Unique Identifier Classification version 1.0.
[XLSX File (Microsoft Excel File), 103 KB-Multimedia Appendix 6]

Multimedia Appendix 7
Medication Name Classification version 1.0.
[XLSX File (Microsoft Excel File), 17 KB-Multimedia Appendix 7]

References
1.
2.

3.
4.

5.
6.

7.

8.

9.

10.
11.
12.
13.
14.
15.
16.
17.
18.

Klann JG, Abend A, Raghavan VA, Mandl KD, Murphy SN. Data interchange using i2b2. J Am Med Inform Assoc 2016
Sep;23(5):909-915 [FREE Full text] [doi: 10.1093/jamia/ocv188] [Medline: 26911824]
Fleurence RL, Curtis LH, Califf RM, Platt R, Selby JV, Brown JS. Launching PCORnet, a national patient-centered clinicalresearch network. J Am Med Inform Assoc 2014;21(4):578-582 [FREE Full text] [doi: 10.1136/amiajnl-2014-002747][Medline: 24821743]
PCORnet. PCORnet. PCORnet Common Data Model (CDM) URL: https://pcornet.org/wp-content/uploads/2019/09/
PCORnet-Common-Data-Model-v51-2019_09_12.pdf [accessed 2020-01-03]
Klann JG, Joss MA, Embree K, Murphy SN. Data model harmonization for the All Of Us Research Program: Transformingi2b2 data into the OMOP common data model. PLoS One 2019;14(2):e0212463 [FREE Full text] [doi:10.1371/journal.pone.0212463] [Medline: 30779778]
Wei W, Denny JC. Extracting research-quality phenotypes from electronic health records to support precision medicine.
Genome Med 2015;7(1):41 [FREE Full text] [doi: 10.1186/s13073-015-0166-y] [Medline: 25937834]
Newton KM, Peissig PL, Kho AN, Bielinski SJ, Berg RL, Choudhary V, et al. Validation of electronic medical record-based
phenotyping algorithms: results and lessons learned from the eMERGE network. J Am Med Inform Assoc 2013
Jun;20(e1):e147-e154 [FREE Full text] [doi: 10.1136/amiajnl-2012-000896] [Medline: 23531748]
Mo H, Thompson WK, Rasmussen LV, Pacheco JA, Jiang G, Kiefer R, et al. Desiderata for computable representationsof electronic health records-driven phenotype algorithms. J Am Med Inform Assoc 2015 Nov;22(6):1220-1230 [FREE Fulltext] [doi: 10.1093/jamia/ocv112] [Medline: 26342218]
Hogan WR, Hanna J, Hicks A, Amirova S, Bramblett B, Diller M, et al. Therapeutic indications and other use-case-driven
updates in the drug ontology: anti-malarials, anti-hypertensives, opioid analgesics, and a large term request. J Biomed
Semantics 2017 Mar 3;8(1):10 [FREE Full text] [doi: 10.1186/s13326-017-0121-5] [Medline: 28253937]
Frampton JE, Perry CM. Topical dorzolamide 2%/timolol 0.5% ophthalmic solution: a review of its use in the treatmentof glaucoma and ocular hypertension. Drugs Aging 2006;23(12):977-995. [doi: 10.2165/00002512-200623120-00005][Medline: 17154662]
McDonough CW, Smith SM, Cooper-DeHoff RM, Hogan WR. GitHub. 2019. Antihypertensive Medication Classification(AntiHTNMedClassification) URL: https://github.com/caitrinmcd/AntiHTNMedClassification [accessed 2020-01-03]
Nelson SJ, Zeng K, Kilbourne J, Powell T, Moore R. Normalized names for clinical drugs: RxNorm at 6 years. J Am Med
Inform Assoc 2011;18(4):441-448 [FREE Full text] [doi: 10.1136/amiajnl-2011-000116] [Medline: 21515544]
Bodenreider O, Cornet R, Vreeman DJ. Recent Developments in Clinical Terminologies - SNOMED CT, LOINC, andRxNorm. Yearb Med Inform 2018 Aug;27(1):129-139 [FREE Full text] [doi: 10.1055/s-0038-1667077] [Medline: 30157516]
Peters L, Bodenreider O. Using the RxNorm web services API for quality assurance purposes. AMIA Annu Symp Proc
2008 Nov;6:591-595 [FREE Full text] [Medline: 18999038]
RxNorm Navigator (RxNav). URL: https://rxnav.nlm.nih.gov/ [accessed 2019-12-18]
Bodenreider O, Rodriguez LM. Analyzing US prescription lists with RxNorm and the ATC/DDD Index. AMIA Annu
Symp Proc 2014;2014:297-306 [FREE Full text] [Medline: 25954332]
Hogan WR, Hanna J, Joseph E, Brochhausen M. Towards a consistent and scientifically accurate drug ontology. CEUR
Workshop Proc 2013;1060:68-73 [FREE Full text] [Medline: 27867326]
Hanna J, Joseph E, Brochhausen M, Hogan WR. Building a drug ontology based on RxNorm and other sources. J Biomed
Semantics 2013 Dec 18;4(1):44 [FREE Full text] [doi: 10.1186/2041-1480-4-44] [Medline: 24345026]
Hanna J, Bian J, Hogan WR. An accurate and precise representation of drug ingredients. J Biomed Semantics 2016;7:7[FREE Full text] [doi: 10.1186/s13326-016-0048-2] [Medline: 27096073]

http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 10(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
19.

20.
21.
22.

23.
24.
25.
26.
27.

28.

29.
30.

31.

32.

33.
34.

35.

McDonough et al

Shenkman E, Hurt M, Hogan W, Carrasquillo O, Smith S, Brickman A, et al. OneFlorida Clinical Research Consortium:
Linking a Clinical and Translational Science Institute With a Community-Based Distributive Medical Education Model.
Acad Med 2018 Mar;93(3):451-455 [FREE Full text] [doi: 10.1097/ACM.0000000000002029] [Medline: 29045273]
Collins FS, Hudson KL, Briggs JP, Lauer MS. PCORnet: turning a dream into reality. J Am Med Inform Assoc
2014;21(4):576-577 [FREE Full text] [doi: 10.1136/amiajnl-2014-002864] [Medline: 24821744]
Sattari M, Cauley JA, Garvan C, Johnson KC, LaMonte MJ, Li W, et al. Osteoporosis in the Women's Health Initiative:
Another Treatment Gap? Am J Med 2017 Aug;130(8):937-948. [doi: 10.1016/j.amjmed.2017.02.042] [Medline: 28366425]
Dumitrescu L, Ritchie MD, Denny JC, El Rouby NM, McDonough CW, Bradford Y, et al. Genome-wide study of resistant
hypertension identified from electronic health records. PLoS One 2017;12(2):e0171745 [FREE Full text] [doi:10.1371/journal.pone.0171745] [Medline: 28222112]
Medical Ontology Research. RxMix URL: https://mor.nlm.nih.gov/RxMix/ [accessed 2019-12-18]
Harris FJ, Tonkin M, Pratt C, DeMaria AN, Amsterdam EA, Mason DT. Short- and long-term therapy of mild essential
hypertension with timolol. Clin Pharmacol Ther 1981 Dec;30(6):765-772. [doi: 10.1038/clpt.1981.236] [Medline: 7030579]
Pettinger WA. Minoxidil and the treatment of severe hypertension. N Engl J Med 1980 Oct 16;303(16):922-926. [doi:10.1056/NEJM198010163031607] [Medline: 6997744]
Rumsfield JA, West DP, Fiedler-Weiss VC. Topical minoxidil therapy for hair regrowth. Clin Pharm 1987 May;6(5):386-392.
[Medline: 3311578]
Jung JH, Kim J, MacDonald R, Reddy B, Kim MH, Dahm P. Silodosin for the treatment of lower urinary tract symptoms
in men with benign prostatic hyperplasia. Cochrane Database Syst Rev 2017 Nov 22;11:CD012615 [FREE Full text] [doi:10.1002/14651858.CD012615.pub2] [Medline: 29161773]
Maldonado-Ávila M, Manzanilla-García HA, Sierra-Ramírez JA, Carrillo-Ruiz JD, González-Valle JC, Rosas-Nava E, etal. A comparative study on the use of tamsulosin versus alfuzosin in spontaneous micturition recovery after transurethralcatheter removal in patients with benign prostatic growth. Int Urol Nephrol 2014 Apr;46(4):687-690. [doi:10.1007/s11255-013-0515-y] [Medline: 24061764]
Achelrod D, Wenzel U, Frey S. Systematic review and meta-analysis of the prevalence of resistant hypertension in treatedhypertensive populations. Am J Hypertens 2015 Mar;28(3):355-361. [doi: 10.1093/ajh/hpu151] [Medline: 25156625]
Carey RM, Calhoun DA, Bakris GL, Brook RD, Daugherty SL, Dennison-Himmelfarb CR, American Heart Association
Professional/Public Education and Publications Committee of the Council on Hypertension; Council on Cardiovascularand Stroke Nursing; Council on Clinical Cardiology; Council on Genomic and Precision Medicine; Council on Peripheral
Vascular Disease; Council on Quality of Care and Outcomes Research;Stroke Council. Resistant Hypertension: Detection,
Evaluation, and Management: A Scientific Statement From the American Heart Association. Hypertension 2018
Nov;72(5):e53-e90 [FREE Full text] [doi: 10.1161/HYP.0000000000000084] [Medline: 30354828]
Calhoun DA, Jones D, Textor S, Goff DC, Murphy TP, Toto RD, et al. Resistant hypertension: diagnosis, evaluation, andtreatment. A scientific statement from the American Heart Association Professional Education Committee of the Councilfor High Blood Pressure Research. Hypertension 2008 Jun;51(6):1403-1419. [doi:10.1161/HYPERTENSIONAHA.108.189141] [Medline: 18391085]
Whelton PK, Carey RM, Aronow WS, Casey DE, Collins KJ, Himmelfarb CD, et al. 2017
ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA Guideline for the Prevention, Detection, Evaluation,and Management of High Blood Pressure in Adults: Executive Summary: A Report of the American College of
Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Hypertension 2018 Jun;71(6):1269-1324.
[doi: 10.1161/HYP.0000000000000066] [Medline: 29133354]
American Society Of Health System Pharmacists (ASHP). AHFS Drug Information 2019. Bethesda, Maryland: Amer Soc
Of Health System; 2019.
Young A, Tordoff J, Dovey S, Reith D, Lloyd H, Tilyard M, et al. Using an Electronic Decision Support Tool to Reduce
Inappropriate Polypharmacy and Optimize Medicines: Rationale and Methods. JMIR Res Protoc 2016 Jun 10;5(2):e105[FREE Full text] [doi: 10.2196/resprot.5543] [Medline: 27288200]
Dixon BE, Alzeer AH, Phillips EO, Marrero DG. Integration of provider, pharmacy, and patient-reported data to improvemedication adherence for type 2 diabetes: a controlled before-after pilot study. JMIR Med Inform 2016 Feb 8;4(1):e4 [FREEFull text] [doi: 10.2196/medinform.4739] [Medline: 26858218]

Abbreviations
ACE: angiotensin-converting enzyme
API: application programming interface
ARB: angiotensin II receptor inhibitor
ATC: anatomical therapeutic chemical
BP: blood pressure
CCB: calcium channel blocker
CDM: common data model
http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 11(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

McDonough et al

DrOn: drug ontology
EHR: electronic health record
HTN: hypertension
ICD: International Classification of Diseases
MED-RT: Medication Reference Terminology
MoA: mechanism of action
NIH: National Institutes of Health
NLM: National Library of Medicine
PCORI: Patient-Centered Outcomes Research Institute
PCORnet: The National Patient Centered Clinical Research Network
RHTN: resistant hypertension
RxCUI: RxNorm Concept Unique Identifier

Edited by G Eysenbach; submitted 21.05.19; peer-reviewed by K Tingay, Y Chu; comments to author 29.09.19; revised version received
18.10.19; accepted 15.12.19; published 27.02.20
Please cite as:
McDonough CW, Smith SM, Cooper-DeHoff RM, Hogan WR
Optimizing Antihypertensive Medication Classification in Electronic Health Record-Based Data: Classification System Development
and Methodological Comparison
JMIR Med Inform 2020;8(2):e14777
URL: http://medinform.jmir.org/2020/2/e14777/
doi: 10.2196/14777
PMID: 32130152©Caitrin W McDonough, Steven M Smith, Rhonda M Cooper-DeHoff, William R Hogan. Originally published in JMIR Medical
Informatics (http://medinform.jmir.org), 27.02.2020. This is an open-access article distributed under the terms of the Creative
Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The
complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright andlicense information must be included.

http://medinform.jmir.org/2020/2/e14777/

XSL• FO
RenderX

JMIR Med Inform 2020 | vol. 8 | iss. 2 | e14777 | p. 12(page number not for citation purposes)