JMIR MEDICAL INFORMATICS

Usui et al

Original Paper

Extraction and Standardization of Patient Complaints from
Electronic Medication Histories for Pharmacovigilance: Natural
Language Processing Analysis in Japanese
Misa Usui1, BA; Eiji Aramaki2, PhD; Tomohide Iwao2, MS; Shoko Wakamiya2, PhD; Tohru Sakamoto3, BA; Mayumi
Mochizuki4,5, PhD
1Division of Hospital Pharmacy Science, Graduate School of Pharmaceutical Sciences, Keio University, Tokyo, Japan

2

Social Computing Lab, Graduate School of Information Science, Nara Institute of Science and Technology, Nara, Japan

3

Holon Co, Ltd., Hiroshima, Japan

4

Division of Hospital Pharmacy Science, Faculty of Pharmacy, Keio University, Tokyo, Japan

5

Department of Pharmacy, Keio University Hospital, Tokyo, Japan

Abstract
Background: Despite the growing number of studies using natural language processing for pharmacovigilance, there are few
reports on manipulating free text patient information in Japanese.
Objective: This study aimed to establish a method of extracting and standardizing patient complaints from electronic medication
histories accumulated in a Japanese community pharmacy for the detection of possible adverse drug event (ADE) signals.
Methods: Subjective information included in electronic medication history data provided by a Japanese pharmacy operating in
Hiroshima, Japan from September 1, 2015 to August 31, 2016, was used as patients’ complaints. We formulated search rules
based on morphological analysis and daily (nonmedical) speech and developed a system that automatically executes the searchrules and annotates free text data with International Classification of Diseases, Tenth Revision (ICD-10) codes. The performanceof the system was evaluated through comparisons with data manually annotated by health care workers for a data set of 5000
complaints.
Results: Of 5000 complaints, the system annotated 2236 complaints with ICD-10 codes, whereas health care workers annotated2348 statements. There was a match in the annotation of 1480 complaints between the system and manual work. System performance
was .66 regarding precision, .63 in recall, and .65 for the F-measure.
Conclusions: Our results suggest that the system may be helpful in extracting and standardizing patients’ speech related tosymptoms from massive amounts of free text data, replacing manual work. After improving the extraction accuracy, we expect
to utilize this system to detect signals of possible ADEs from patients’ complaints in the future.
(JMIR Med Inform 2018;6(3):e11021) doi: 10.2196/11021
KEYWORDS
adverse drug events; natural language processing; medical informatics; medication history; pharmacovigilance

Introduction
Background
Adverse drug events (ADEs) are any untoward injuries resulting
from the use of a drug [1]. They occur in around 18% of
inpatients [1-4] and are a significant burden on health care andsociety. The ADEs are a cause of morbidity, and mortality andtheir economic loss is estimated at US $177.4 billion annuallyin the US [5]. In the field of pharmacovigilance, postmarketingsurveillance such as spontaneous reporting is important for thedetection of ADEs because clinical trials have limitations
including patient sample size, population, and administration
period [6].
http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

The need to understand patients’ subjective complaints and touse other sources in pharmacovigilance has increased. Unlike
health care providers, patients use various expressions andterminology to describe their situations. Direct reporting frompatients is helpful in understanding their detailed symptoms andimpacts on quality of life, which medical professionals tend tooverlook [7-9]. For example, analysis of the content ofcomments posted on patients’ online community pages revealed
unknown long-term symptoms of antidepressant withdrawal[10]. The Maintenance and Support Services Organization
developed the Patient-Friendly Term List [11] based on themost frequent ADEs reported by patients and consumers tofacilitate direct patient reporting of ADEs to regulators and thepharmaceutical industry. Despite its importance, little work has
JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 1
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et al

been done on exploring patient records until recently due totheir unstructured, time-consuming data format.

Methods

Natural language processing (NLP) is the automatic
manipulation of natural language such as narrative text andspeech for extraction and structuring [12]. Numerous attempts
have been made to use NLP in electronic health records (EHRs),social media, medical literature, or existing reporting systems[13-29]. Those studies found that NLP could identify various
points for the assessment of medications (eg, inactive
medication, nonadherence, patients’ mentions of ADEs).

Concept of the System

In Japan, text analysis and automated detection of medical
events from EHRs have been reported [30], and a tool for
disease entity encoding was developed [31]. However, these 2
studies intended only to manipulate clinical text provided byhealth care professionals using medical terminology. No
previous study dealt with patients’ complaints in their own
words in Japanese.

Prior Work
Nikfarjam et al [25] introduced a machine learning-based
extraction system using conditional random fields (CRFs) for
user posts on DailyStrength (precision: .86, recall: .78,
F-measure: .82) and Twitter (precision: .76, recall: .68,
F-measure: .72) to detect adverse drug reaction (ADR) signals.
Freifeld et al [28] classified Twitter posts (precision: .72, recall:.72) to compare product-event pairs with the US Food and Drug
Administration Adverse Event Reporting System (FAERS) data.
In the mining of patients’ reports, Topaz et al [26] used alinguistic-based approach comparing EHRs (clinicians’ reports)and social media (patients’ mentions) for 2 common drugs.
White et al [27] used search log data for the identification of
ADE signals and a comparison with FAERS data resulted inhigh concordance as determined by the Area Under the Curve
Receiver Operating Characteristics curve of .82. Denecke et al[29] collected data from multiple media sites with keyword lists
and classified texts as relevant/irrelevant using support vector
machines.
Although no previous studies have been completed in Japanese,
Aramaki et al [30] reported on a system to extract medical event
information from Japanese EHRs based on CRFs (precision:.85, recall: .77, F-measure: .81). The text source in their study
was written in medical terminology, mainly by physicians. No
lexicon to standardize patients’ informal expressions such as
the Patient-Free Term List [11] and the work of Freifeld et al[28] has been published in Japanese.

Study Aim
This study aimed to develop techniques to establish a method
for extracting and standardizing patient complaints fromelectronic medication history data (EMHD) accumulated in a
Japanese community pharmacy for the detection of possible
ADE signals.

We propose a system that automatically extracts andstandardizes patient complaints (Figure 1). In this system,
subjective information included in the medication histories
collected from a pharmacy is input data, and data in which
International Classification of Diseases, Tenth Revision(ICD-10) codes are attached to patient expressions are outputs.
A dictionary-based method was adopted for extraction andstandardization. The processing steps in the system are as
follows. First, morphological analysis is performed on input
data. Next, the search rules are applied to split data. In the searchrules, morpheme combinations in general expressions and thecorresponding ICD-10 codes are described for each line, andexclusion rules are set for some ICD-10 codes. When a patient
expression satisfies the search rules, a corresponding ICD-10
code is given. Procedures for creating the search and exclusion
rules and system development procedures are detailed in “Search
Rules” and “System Development.”

Data Sources
The EMHD stored in a community pharmacy were used as thesource of patients’ comments. When pharmacists dispense
prescription drugs to patients, they are required to record theresults of medication instructions and patients’queries/responses. A medication history in Japan is typically
written in the “SOAP” format, which consists of 4 sections:“Subjective information” (complaints of the patient), “Objective
information” (objective indicators such as laboratory findings
or names of drugs prescribed), “Assessment” (the pharmacist’s
findings on the occurrence of ADRs, interactions, or doubt aboutprescription instructions), and “Plan” (action plan of thepharmacist derived from the assessment).
Although patients do not write the medication history, of those
4 sections the “Subjective information” appeared to be the mostappropriate text source, because pharmacists complete that
section in the patients’ own words.
Patients’ comments were extracted from the EMHD of acommunity pharmacy operated by Holon Co, Ltd, Hiroshima,
Japan. This company operates a chain of 14 pharmacies, andthe data used in this study mainly came from a single one. The
study period was from September 1, 2015 to August 31, 2016.
Personal information such as patients’ names and birth dates
were anonymized before analysis.
Information on the hospitals or clinics that issued prescriptions
for which the subjective information used in this study wasderived is shown in Table 1. The pharmacy filled a total of42,120 prescriptions during the study period for the top 9
prescribing hospitals or clinics. The number of prescriptions
from medical institution A was the highest (18,273/42,120,
43.5%). Clinic A specializes in otolaryngology, and the patientsare older adults who often complain of dizziness or hearing loss.
Table 2 shows the items recorded in the EMHD, while Figure
2 is an example of a recording object.

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 2
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et al

Figure 1. Concept of the system. ICD-10: International Classification of Diseases, Tenth Revision..

Table 1. Backgrounds of the prescriptions used in this study (N=42,120).
Hospital/ clinic

Specialty

Main patient characteristics

Prescriptions, n (%)

A

Otolaryngology

•
•
•
•

ElderlyDizziness
Tinnitus
Hearing loss

18,273 (43.38)

B

General medicine; cardiology•••

Elderly
Heart disease
Hypertension

5356 (12.72)

C

General medicine; gastroenterology; cardiology •••Elderly
Digestive tract diseases
Circulatory diseases

537 (1.27)

D

General medicine; Kampoa medicine••

Elderly989 (2.35)

E

Neuropsychiatry•

Wide range of age-groups

377 (0.90)

F

Neuropsychiatry•

Wide range of age-groups

563 (1.34)

G

Obstetrics and gynecology•

Gynecological conditions

649 (1.54)

H

Obstetrics and gynecology•

Infertility treatment

4206 (9.99)

I

Breast surgery•

Breast cancer patients visiting for diagnosis andpostoperative care

2608 (6.19)

Otherb

Various•

Various8562 (20.33)

Kampoa medicines (for >half of the patients)a

The word “Kampo” means herbal medicine in Japanese. The term “Kampo Shoseiryuto” is commonly used to treat watery nasal discharge, nasal
congestion, watery sputum, and sneezing.
b

These are medical institutions that are not the major clinics “A” to “I” from which this pharmacy receives prescriptions.

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 3
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et al

Table 2. Items recorded in the electronic medication history data.
Category

Content

Identification

Identification of patient

Name

Name of patient

Sex

Male/female

BirthaYear/month/day

Special instructions

Notes as required (eg, disease entity)

Concomitant drugs
Drug name

YJ-codeb or product name

Dosing period

Year/month/start date/end datePrescription data
Dispensing dateYear/month/day

Drug name

YJ-code or product name

Dosage/administrationc

Free text

Medication counselingaCounseling dateYear/month/day

Subjective informationd

Free text

Objective informatione

Free text

Assessmentf

Free text

Plang

Free text

Only the year of birth data was used.

b

YJ-codes identify prescription drugs covered by insurance in Japan.

c

Not used in this study.

dIncludes patient complaints.

e

Includes laboratory data.

fIncludes pharmacists’ assessments of patient conditions.

gIncludes pharmacists’ plans for prescription questions, patient education, and follow-up.

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 4
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et al

Figure 2. Example of a medication history.

Search Rules
We created search rules to identify the appropriate ICD-10 codefrom the free text in the “Subjective information” section anddeveloped a coding system that annotates the ICD-10 codes
within patient complaints. The ICD-10 was originally an
English-based system but is also used in Japan. It was translated
into Japanese by the World Health Organization, and a coding
rulebook was published. For example, in Medis [32] the ICD-10
is given as the basic classification code, and coding matched as
closely as possible to clinical interpretation is undertaken.
Although it may be possible to use the Medical Dictionary forRegulatory Activities (MedDRA) or the International
Classification of Primary Care as a medical code system, we
http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

adopted ICD-10 in this study because it is used for insurance
claims in Japan and because many coders are familiar with
ICD-10.
In developing the system, a nurse with 10 years of experiencein the field of terminal care and a medical coder with 20 yearsof experience created the search rules based on the expressions
in the “Subjective information” section. A programmer readthe search rules and developed a program to accommodate new
expressions. Search rules were created by a combination ofmorphological analysis and common expressions.
The search rules govern the pattern for analyzing comments
included in the subjective information. The rules were saved in
Microsoft Excel format with the corresponding disease entity
JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 5
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
category and ICD-10 codes. For example, to search for “D69.9:
Hemorrhagic condition, unspecified,” the search strings are “(出血|しゅっけつ|血)+(傾向|けいこう|し易く|しやすく|し易
い|しやすい|出やすい|止まりにくい|とまりにくい|止まらない|とまらない).” In English, this would translate to“(bleeding|blood)+(tendency|easy to|hard to stop|won’t stop|notstop).” Written Japanese utilizes 3 orthographic systems:
Chinese characters, hiragana, and katakana. Therefore, theactual search strings are longer than in English. All rules areshown in Multimedia Appendix 1. The rule-making steps areshown in Textbox 1. We repeated this process 5 times over 1
month in order to refine the search rules.
The nurse first checked the free text recorded in the “Subjective
information” section and selected complaints referring topatients’ symptoms. Then words related to ICD-10 codes weremanually extracted from the complaints. Finally, the extracted
words were added sequentially to the search string for each
ICD-10 code. The search strings consist of patterns of word
combinations using “|” (logical sum) or “+” (logical product).
At present, a maximum of 3 words/terms can be combined ina string separated by “+” signs. For example, from the text
“blood pressure today was a little high,” the terms “bloodpressure” and “a little high” were extracted, and the systemannotated the text with the ICD-10 code “I10: hypertension.”
However, some text found in the “Subjective information”
section could not be annotated with an ICD-10 code even though
it followed the search rules. Therefore, we set exclusion rules
for some codes, which were created following the sameprocedure as for the search rules but were only applied when ahealth care worker could visually confirm the keyword for
exclusion. For the previous example of “D69.9: Hemorrhagic
condition, unspecified,” terms with “(-|ない|なし|無い|無し),”
in English, “(-|no|none|negative|never|don’t)” were excluded

Usui et aleven if they included search strings. For example, “(血がとまりにくい),” in English, “(the bleeding won’t stop),” wasannotated as D69.9, but “(血がとまりにくいことはない),”in English, “(I never felt the bleeding wouldn’t stop),” wasexcluded.

System Development
The system developed extracts complaints related to patients’
symptoms from the “Subjective information” section of EMHD
automatically and annotated each complaint with the ICD-10
code using the search rules above. During system development,
we used Perl as the programming language and MeCab [33] asa morphological analyzer. The Microsoft Excel format was used
for subsequent analysis.
The development procedure can be summarized as follows:
1.
2.

3.

4.

Subjective information was extracted from each saved
Microsoft Excel file
Morphological analysis was performed to extract subjective
information, separating the text with spaces into minimum
meaningful units of words/terms
After the processes above were performed, the subjective
information was copied back into a Microsoft Excel file.
Search rules and exclusion rules were applied to thesubjective information by analyzing each complaint andsearching for the ICD-10 code
If an appropriately matching ICD-10 code was found, thecomplaint was annotated with the ICD-10 code and thecorresponding disease entity

The coding system adapts the search rules (shown in Multimedia
Appendix 1) in order from the top. If an adaptable rule is found,
the result of ICD-10 coding is output. If multiple rules arematched, all of them are output in the results.

Textbox 1. Rule-making steps.
1.

Make seed rules

2.

Apply seed rules to development set

3.

Error analysis performed by two rule curators (the nurse and medical coder)4.

New rules added by the programmer, who converts the error analysis to general expressions

5.

Repeat steps 3 and 4

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 6
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et al

Figure 3. System interface screenshot. ICD-10: International Classification of Diseases, Tenth Revision.

Optimization of System Performance
For optimal performance of the system, the system-annotateddisease entities should ideally match the entities manuallyannotated by health care professionals. As mentioned above,the more thoroughly the search rules are satisfied, the moreaccurate the system. Therefore, we reviewed the search rules
multiple times to determine the most appropriate ones toimprove the accuracy of the system.
In this study, we did not attempt machine learning for thedetection of relevant terms to match ICD-10 codes. By adding
search rules as appropriate, free text can be automaticallyassociated with ICD-10 codes via the system.

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

Experiment
An evaluation experiment was conducted to confirm theperformance of the system. Five thousand complaints from thesubjective information were processed, and 323 search rules
were created. In the experiment, health care workers (1 nurse
and 1 pharmacist) first independently annotated the 5000
complaints manually with the ICD-10 codes. Second, 108
mismatched annotations were excluded, and the data from theremaining 2348 were used as correct answers for the subsequent
step. Finally, the system with 323 search rules was applied to
the 5000 complaints.
The subjective information used in this study consisted ofmultiple sentences, and thus several patient expressions were
JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 7
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et alobtained from one “Subjective information” section. Since eachpatient expression is linked to the ICD-10 code, multiple ICD-10
codes are assigned to a single “Subjective information” section.
In evaluating the system in this study, if one of the plural ICD-10
codes differed from the manual result, it was judged that allother coding for that entry was incorrect (unmatched). Figure
3 shows an actual system execution screen.

F-measure was calculated by taking the harmonic mean between
precision and recall.

Based on the results of this experiment, the precision, recall,and F-measure of the system were calculated [34,35]. Precision
was calculated by dividing the matched number (the number of“Subjective information” sections for which manual coding andsystem coding had the same result) by the searched number (thenumber of “Subjective information” sections that the systemannotated with ICD-10 codes). Recall was calculated by dividing
the matched number by the correct answers (the number of“Subjective information” sections manually coded). The

Results

Ethical Considerations
This study was approved by the Ethics Committees on Human
Research of the Faculty of Pharmacy, Keio University and Nara
Institute of Science and Technology.

Examples of correct answer data and system execution results
are shown in Table 3. From 5000 complaints, 2348 ICD-10
codes were extracted by health care workers. The system
extracted 2236 codes, 1480 of which matched the manual results.
The system performed .662 for precision, .630 for recall and.646 for F-measure. Table 4 shows precision and recall for the
10 most frequent symptoms extracted by health care workers.

Table 3. Comparison between manual and system extraction of ICD-10 codes for patient complaints from typical examples.
Patient complaints (Original text of Japanese is attached)

Manual results

めまい は 起こっ て い ない です 。 暑 さ で フラー っと する R42: Dizziness and giddiこと が ある 。
ness; R26.0: Ataxic gait

System results

Matching

R42: Dizziness and giddinessNot matched

J00: Acute nasopharyngitis(common cold); R51: Headache

MatchedNo vertigo/dizziness. Sometimes I feel unsteady due to the heat.
昨日 の 夜 から 頭 の 後ろ が 重い よう な 感じ が する 。 風邪 J00: Acute nasopharyngitis
か と 思っ て 受診 。
(common cold); R51:
From the night before last, the back of my head felt heavy. I probably Headachecaught cold, so I saw a doctor.
私 よく 走っ ちゃう ん だ けど 坂 で 躓い て ね 。 ちょっと な Q82.5: Congenital nonneoのに 大きな アザ ガ でき た 。 整形 外科 の 先生 は 血 流 の 薬 plastic nevus
が ある から って 。 で も 飲ま ない と だめ だ と いわ れ た。

Q82.5: Congenital nonneoplastic Matchednevus

I was in a hurry, stumbled, and fell. A big bruise came up. My orthopedist said it may be caused by my blood thinner, though I have tocontinue that medicine.
血圧 は ちょっと 高かっ た の 。 １ ４ ０ 後半 。 家 だっ たら I10: Essential (primary) hy- I10: Essential (primary) hyperten- Matched
１ ３ ０ くらい なん だ けど 。 整形 外科 で もらっ た パップ pertension; L259: Unspeci- sion; L259: Unspecified contact
は かぶれ て しまっ た 。
fied contact dermatitis, un- dermatitis, unspecified cause
specified cause
My blood pressure was a bit high, in the upper 140s, although it’saround 130 when I’m at home. I got a rash from plasters prescribedby orthopedics.
風邪 を 引き まし た 。 この間 、 ヘルペス の 薬 を X月Y日 か J00: Acute nasopharyngitis;
ら 飲ん で X’月 Y’日くらい から ２ ． ３ 日 、 夜 の トイレ で R32: Unspecified urinary
尿 が 我慢 でき ず に もらし そう な こと が 続い て しんどかっ incontinence
た 。 あんな こと は 初めて 。 薬 情 を 見 たら 腎臓 に 悪く なる と 書い て ある から この 薬 の せい だ と 思う 。
I caught a cold. The other day I started taking an antiherpes prescription from mm/dd, and few days afterward I felt like I was going to
pee in my pants and was really exhausted. I never had such an experience. Looking at the drug information, this medicine may have bad
effects on the kidney. Maybe it’s due to this medicine.

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

J00: Acute nasopharyngitis;
Not matched
B02.9: Zoster without complication; R32: Unspecified urinary
incontinence; R53: Malaise andfatigue; N28.9: Disorder of kidney and ureter, unspecified;
R94.4: Abnormal results of kidney function studies

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 8
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et al

Table 4. Precision and recall for the 10 most frequent symptoms.

a

Rank

ICD-10aMatched, n (%)

Correct answer, n (%)

Searched, n (%)

Precision

Recall1

R42 (Dizziness and giddiness)

146 (9.86)

173 (7.37)

254 (11.36)

.575

.844

2

J00 (Acute nasopharyngitisb)

189 (12.77)

208 (8.86)

226 (10.11)

.836

.909

3

R52.9 (Pain, unspecified)

108 (7.29)

146 (6.22)

199 (8.90)

.543

.740

4

F19.6 (Mental and behavioral disordersc)

142 (9.59)

168 (7.16)

197 (8.81)

.721

.845

5

R05 (Cough)114 (7.70)

134 (5.71)

143 (6.40)

.797

.851

6

H931 (Tinnitus)

99 (6.68)

131 (5.58)

134 (6.00)

.739

.756

7

R26.0 (Ataxic gait)

41 (2.77)

59 (2.51)

120 (5.37)

.341

.695

8

I10 (Essential [primary] hypertension)

51 (3.44)

89 (3.79)

87 (3.89)

.586

.573

9

F51.1 (Nonorganic hypersomnia)

39 (2.63)

46 (1.96)

85 (3.80)

.459

.848

10

R53 (Malaise and fatigue)

40 (2.70)

59 (2.51)

81 (3.62)

.494

.678

ICD-10: International Classification of Diseases, Tenth Revision.

bCommon cold.

c

Due to multiple drug use and use of other psychoactive substances.

Textbox 2. Six reasons for unmatched results. ICD-10: International Classification of Diseases, Tenth Revision.
1. Misdetection of negation or possible event
System misread an expression including negation or possible event as a symptom that actually occurred (eg, “dizziness has not occurred,” “If I feel
dizzy”)
2. Misdetection of a clinical test item
System mistook a clinical test term as a patient symptom (eg, “test for dizziness”)
3. Misdetection of drug class name
System mistook the name of the drug class as a patient symptom (eg, “painkiller” mistaken for “R529: Pain, unspecified”)
4. Misdetection of unrelated words
System mistook unrelated words as a patient symptom (eg, “I’m getting old” mistaken for “R54: Senility”)
5. False negative
System missed a word that indicates a patient symptom
6. Inappropriate ICD-10 code
System failed to choose the appropriate ICD-10 code even if it extracted words related to a patient symptom

The results indicated that the average performance of the system
was .66 for precision, .63 for recall, and .65 for the F-measure.
Comparing the performance for each symptom, the precisionof “dizziness and giddiness,” “pain, unspecified,” and “ataxicgait” was especially low. We identified 6 reasons for theunmatched results for these 3 symptoms, as shown in Textbox2.

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

Table 5 details the unmatched results and typical examples for
3 symptoms. The main reason for discordance between manualand system coding was misdetection of negation or possible
event in “R42: dizziness and giddiness” (79/108 results, 73.1%)
and “R26.0: ataxic gait” (71/79 results, 90%), whereas
misdetection of drug class name was the most common in“R52.9: pain, unspecified” (28/91 results, 31%).

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 9
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et al

Table 5. Details of unmatched results and typical examples for 3 symptoms.
Symptom and categoryn (%)

Example of Patient complaints (Original text of Japanese is attached)

Manual results

System results眩暈 は 起き て い ない が 、 ２ 回 くらい ふらつき が あっ R26.0: Ataxic
た 。 今日 Ｄｒ に 診 て もらっ たら 大丈夫 だ と 言わ れ gait
た。

R42: Dizzinessand giddiness;
R26.0: Ataxic
gait

R42: Dizziness and giddiness (N=108)
1 (Misdetection of nega- 79 (73.1)
tion or possible event)I didn‘t feel dizzy but staggered about twice. Today I saw a doctor
and be said there was no problem.
2 (Misdetection of clinical test item)

5 (4.6)

なぜ か ニセルゴリン だけ １ ０ 錠 余っ てる 。 調子 は—
なんとも ない 。 眩暈 も 検査 し た けど 問題 ない と 言
わ れ た し 、 聞こえ も 悪く なっ て ない と 言わ れ た 。
I don’t know why, but I have 10 leftover Nicergoline pills. I feelfine. I was examined for dizziness and no problem was found. I
was also told that there was no problem with my hearing.

3 (Misdetection of drugclass name)8 (7.4)

めまい の 薬 は 昼 飲め ない こと が 多く て 残り が あるの 。 ずっと 仕事 だ から 飲む の が 難しく て 。

—

I have leftover motion sickness medicine because I often don’ttake it in the daytime. I work all day and it’s hard to take it.
4 (Misdetection of unrelated words)

4 (3.7)

母 の 介護 で 忙しかっ た けど 秋 に 亡くなっ て 、 葬式 R42: Dizzinessとか で また 忙しく て あまり 薬 が 飲め て なかっ た の and giddinessで 残り は あり ます 。 めまい も そんなに ひどく なっ てなかっ た です 。
I was busy taking care of my mother. She passed away this autumn and I got busy with the funeral arrangements, etc. I didn’t
have time, so there are still some pills. The dizziness didn’t get
much worse.

R42: Dizzinessand giddiness;
H91.9: Unspecified hearing loss

R42: Dizzinessand giddiness;
R13:Dysphagia

R42: Dizzinessand giddiness;
R99: Other ill-defined and unspecified causes ofmortality

5 (False negative)

11 (10.2)

めまい は 起こっ て い ない です 。 暑 さ で フラー っとする こと が ある 。

R42: Dizzinessand giddiness;
No vertigo/dizziness. Sometimes I feel unsteady due to the heat. R26.0: Ataxic
gait

R42: Dizzinessand giddiness6 (Inappropriate ICD10code)

1 (0.9)ふらふら する の は 最近 落ち着い てる 。 Ｄｒ に 言わ れ R42: Dizzinessて 、 起き上がる とき も ゆっくり 起きる よう に し たら and giddiness;
眩暈 あまり 起き なく なっ た 。
H81.1: Benign
paroxysmal vertiThe dizziness is getting better. My doctor told me to stand upgoslowly and when I tried that I didn’t feel dizzy.

R42: Dizzinessand giddiness;
R26.0: Ataxic
gait

R52.9: Pain, unspecified (N=91)
1 (Misdetection of nega- 20 (22.0)
tion or possible event)血圧 １ １ ０ くらい 。 ふらつく こと は ない です 。 筋 R53: Malaise and M79.1: Myalgia;
肉 痛 も ない です 。 気 に なっ た こと は ない が 、 言わ fatigue
R52.9: Pain, unれ て みれ ば 、 たまに だるく なる こと が あり ます 。
specified; R53:
Malaise and faBP is around 110. No dizziness and no muscle pain. But now
tigue
that you ask, I sometimes feel malaise.

3 (Misdetection of drugclass name)

痛く ない から 神経 痛 の 薬 は もう いら ない 。 納豆 が —本当は 大好き な ん だ けど 、 ワーファリン 錠 飲ん でるから 止め てる 。 納豆 関係 ない 薬 も ある らしい けど、 値段 が 高い らしい ね 。

28 (30.8)

II don’t feel pain, so I don’t need the nerve medicine any longer.
I really love natto (fermented soybeans), but have to avoid it because I take warfarin. I think there is some kind of medicine thatis not affected by eating natto, but I hear it’s really expensive.
4 (Misdetection of unrelated words)

8 (8.8)

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

耳鳴り と のど が 痛い 。
My ears are ringing and I have a sore throat.

H93.1: Tinnitus;
J02.9: Acutepharyngitis, unspecified

R52.9: Pain, unspecified; M79.2:
Neuralgia andneuritis, unspecified

H92.0: Otalgia;
H92.1: Otorrhea;
R52.9: Pain, unspecified

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 10(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et al

Symptom and categoryn (%)

Example of Patient complaints (Original text of Japanese is attached)

Manual results

5 (False negative)

9 (9.9)

胃 が 痛い 、 のん で も 治ら ん から 先生 に 相談 し た 。
腰 も い たい ん よ ね 。 ロキソニン は 手持ち 無くなった 。 今回 で て ない ？ あら 。 ボラザ Ｇ 軟膏 は ２ １ 本
持っ てる けど ちょっと 足り ない かも ね 。

R10.1: Pain localized to upper abdomen; R52.9:
Pain, unspecified;
M54.5: Low backMy stomach hurts. I took some medicine but didn’t feel betterso I saw a doctor. My lower back also hurts. I’m out of Loxopro- pain
fen, so did the doctor give me a new prescription? I have 21 tubesof ointment but that may not be enough.

R101: Pain localized to upper abdomen; R52.9:
Pain, unspecified

6 (Inappropriate ICD10code)

26 (28.6)

前 の 粉薬 を 止め たら 尿 の 出 は 治っ た ん だ けど 、他 の 薬 を 続け て い たら 手の甲 と 関節 が ピリピリ 痛
み 始め て 驚い た 。 飲む の を 辞め たら 治っ た よ 。

M25.5: Pain injoint; 52.9: Pain,
unspecified

M25.5: Pain injoint; M79.6:
Pain in limb

System results

After I stopped taking the powdered medicine, my urine flow
improved but I’m still taking the other medicine. I suddenly felt
a sharp pain on the back of my hand and in the joints, but that
stopped after I quit taking the medicine.
R26.0: Ataxic gait (N=79)
1 (Misdetection of nega- 71 (89.9)
tion or possible event)

カルデナリン 錠 が 中止 に なっ て 、 フラ ツキ は あれ—
っきり よく なっ た 。 血圧 の 薬 は 内科 で かるい の が追加 に なり まし た 。 眼科 は 今 は かかっ て なく 、 内科 で 目薬 出し て もらっ て ます 。

R26.0: Ataxic
gait

After I stopped taking Doxazosin, the light-headedness got better.
A new mild blood pressure medicine was prescribed by the doctor. I don’t go to the eye doctor anymore, so I have my regular
doctor prescribe eyedrops.
4 (Misdetection of unrelated words)

2 (2.5)

かるい フラ ツキ ある ん だ けど 大丈夫 。 内科 は いって ない 。 先生 行け って 言わ ない から 大丈夫 な ん だと 思う 。 セロクラール と パルトックス 錠 は わかる から 赤 線 は ひか なく て いい よ 。

R26.0: Ataxic
gait

I stagger a little bit, but that’s OK. I didn’t go to the Internal
Medicine Department because my doctor didn’t tell me to, so it
probably isn’t a problem. I know which pills are Cerocral andPantethine, so you don’t have to draw a red line on the bottles
for me.
5 (False negative)

5 (6.3)血圧 は いつも より 良かっ た 。 病院 で 血糖 が ６ ８ だっ E16.2: Hypoglyた から 飴 玉 食べ た 。 ブドウ糖 も 持っ てる 。 ふわっ caemia, unspeciと する 症状 も あっ た 。
fied; R26.0:
My blood pressure was better than usual. My blood glucose was Ataxic gait

R26.0: Ataxic
gait; R21: Rashand other nonspecific skin eruption

R26.0: Ataxic
gait

68, so I ate some candy. I carry glucose tablets with me, too. I
felt weightless.
6 (Inappropriate ICD10code)

1 (1.3)

頭 が フラフラ する ので 医師 に 相談 し た 。 リリカ が効き 過ぎ て いる の で は ない か と の こと 。

R42: Dizzinessand giddinessR26.0: Ataxic
gait

I felt woozy and went to the doctor. He thought that the Pregabalin was too strong.

Discussion

are at least adequate as the first step in possible ADE signal
detection.

Principal Results

This was the first attempt to standardize patients’ expressions
with the Japanese version of ICD-10 and to use the “Subjective
Information” section in the medication history as a source. The
advantage of using the medication history is its structured format
and data storability. The medication history is recorded forpatient monitoring including side effects. Its features provide
more specialized information relevant to possible ADEs thansocial media like Twitter or EHRs in hospitals. Moreover, thenumber of pharmacies in Japan is increasing [36,37], andpharmacists are required to record patient medication histories
for health insurance claims. Thus, huge amounts of data on

Nikfarjam et al [25] and Aramaki et al [30] used CRFs, andFreifeld et al [28] used a tree-based dictionary matchingalgorithm for extracting the terms. Our approach involved
rule-based searching, which is much simpler but less tolerantof orthographic variants. Additionally, differences in linguisticfeatures might have contributed to the gap between the resultsof the present study and nonJapanese ones [25,28]. In written
Japanese, words are not separated by spaces, and therefore theaccuracy of extraction is affected by the quality ofmorphological analysis. Considering these points, the results

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 11(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
patients’ medication are available, making medication histories
an appropriate source for ADE signal detection.
It is not necessary for ADEs to have causal relationships withdrugs, whereas ADRs must have a reasonable association withdrug use. Using patient records to detect ADRs is a major
challenge because causality cannot be readily assessed; however,
it is also important to detect potential ADE signals.
In this study, some text could not be annotated with ICD-10
codes. As compared with the performance of health care
professionals, our newly developed system performed at levelsof .66 for precision, .63 for recall, and .65 for the F-measure.
These values are relatively lower than in previous studies[25,28,30], likely due to differences in methodology. As
explained in the Experiment section, if one of the many ICD-10
codes was different from the manual result, all other coding was
regarded as incorrect (unmatched) for that entry. This is one
reason why the F-measure was lower than in previous research.
There was also insufficient specific information about thecondition of each patient. Because the majority of patients are
not medical experts, they describe their symptoms in everyday
language, which is more equivocal and more inflected than
medical terminology. Nikfarjam et al [25] reported similar
aspects of ambiguity and lack of context in patients’ wording.
The dialect spoken can affect the subjective information,although, of 5,000 complaints analyzed in this study, only 7
were recorded in a regional dialect. This is probably related to
the nature of the text. Although it is recommended thatpharmacists record patients’ statements exactly, it is possiblethat they replace dialect expressions with standard wording to
make the information easier to understand by others later.
Regarding standardization across languages, the present system
could be applied to other languages to some extent by translating
the morphemes used for the search rules or by adding or refining
the search rules later.

Limitations
There were some limitations of this study. First, qualitative
differences in the text data could have occurred. The “Subjective
information” section is filled in by pharmacists, and therefore
they may interpret and summarize patients’ comments when
they record them. To ensure that the medication histories of all
patients are recorded during the daily business hours ofcommunity pharmacies, in some cases fixed-form complaint
set phrases and excerpts of comments may be relied on to
decrease the time needed to complete the “Subjective
information” section. It is therefore possible that the finer

Usui et alnuances patients hope to convey are altered or lost during theprocess. Qualitative differences were also noted among
pharmacists for the contents of the “Subjective information”
section. Some wrote about symptoms using explicit medical
terminology (eg, “back pain and knee pain were unabated”).
Others included general information unrelated to symptoms (eg,greetings and general conversation transcribed word for word).
Second, it was difficult for the system to determine whether the
extracted keyword was related to patients’ symptoms or thoseof others. For example, from the sentence “My friend had
hypertension,” the system may extract “hypertension,” although
it is unrelated to the speaker’s condition. This point should be
improved by revising the search rules after consultation withregulatory experts or using machine learning to deal with
ambiguity.
Also, since only 1 of 14 pharmacies in a single chain participatedin this study, there is a possibility that the search rules were
optimized for patients receiving prescriptions from specificmedical departments. In the experimental results, the mostfrequent ICD-10 code was “dizziness and giddiness.” As shownin Table 1, the target pharmacy frequently dispenses
prescriptions from otolaryngologists, and the results may reflect
this potential bias. Before the practical application of the system,
it is necessary to improve the search rules by considering awider range of medication histories including data from othercommunity pharmacies.
ICD-10 codes were used as normalization terms for patients’complaints regarding their symptoms because they are widely
available and understood, but MedDRA is thought to be moresuitable for extracting information on ADRs and for signal
detection. We are currently enhancing the system toaccommodate MedDRA terms.

Conclusions
In this study, we developed an automated system to extract
terms related to symptoms from the verbal complaints ofJapanese patients. As a result of an evaluation experimentcomparing automated with manual extraction, the system
performed at the level of .66 in precision, .63 in recall, and .65
for the F-measure. Although the accuracy of the system was
not satisfactory, our results suggest that it might be useful inextracting and standardizing patients’ expressions related tosymptoms from massive amounts of free text data instead of
performing those procedures manually. After improving theextraction accuracy, we expect to utilize this system to detect
the signals of ADRs from patients’ complaints in the future.

Acknowledgments
The authors wish to thank Holon Co, Ltd For agreeing to participate in this study by providing anonymized EMHD records from
September 1, 2015, to August 31, 2016. This research was supported by AMED under grant #JP16mk0101058.

Conflicts of Interest
One of the authors (TS) is an employee of Holon Co, Ltd.

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 12(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et alMultimedia Appendix 1
Search rules for the program.
[XLSX File (Microsoft Excel File), 40KB-Multimedia Appendix 1]

References
1.
2.
3.
4.
5.
6.
7.
8.

9.

10.

11.
12.
13.

14.
15.

16.

17.
18.
19.
20.

21.

22.

Nebeker JR, Barach P, Samore MH. Clarifying adverse drug events: a clinician's guide to terminology, documentation, andreporting. Ann Intern Med 2004 May 18;140(10):795-801. [Medline: 15148066]
Venulet J. Progress in Drug Research. Basel: Birkhauser Verlag, Basel; 1997:233-292.
Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis ofprospective studies. JAMA 1998 Apr 15;279(15):1200-1205. [Medline: 9555760]
Budnitz DS, Lovegrove MC, Shehab N, Richards CL. Emergency hospitalizations for adverse drug events in older Americans.
N Engl J Med 2011 Nov 24;365(21):2002-2012. [doi: 10.1056/NEJMsa1103053] [Medline: 22111719]
Ernst FR, Grizzle AJ. Drug-related morbidity and mortality: updating the cost-of-illness model. J Am Pharm Assoc (Wash)
2001 Apr;41(2):192-199. [Medline: 11297331]
Rogers AS. Adverse drug events: identification and attribution. Drug Intell Clin Pharm 1987 Nov;21(11):915-920. [Medline:
3678067]
Basch E. The missing voice of patients in drug-safety reporting. N Engl J Med 2010 Mar 11;362(10):865-869 [FREE Fulltext] [doi: 10.1056/NEJMp0911494] [Medline: 20220181]
Barbara AM, Loeb M, Dolovich L, Brazil K, Russell M. Agreement between self-report and medical records on signs andsymptoms of respiratory illness. Prim Care Respir J 2012 Jun;21(2):145-152 [FREE Full text] [doi: 10.4104/pcrj.2011.00098][Medline: 22273629]
Avery AJ, Anderson C, Bond CM, Fortnum H, Gifford A, Hannaford PC, et al. Evaluation of patient reporting of adverse
drug reactions to the UK 'Yellow Card Scheme': literature review, descriptive and qualitative analyses, and questionnaire
surveys. Health Technol Assess 2011 May;15(20):1-234, iii [FREE Full text] [doi: 10.3310/hta15200] [Medline: 21545758]
Belaise C, Gatti A, Chouinard V, Chouinard G. Patient online report of selective serotonin reuptake inhibitor-induced
persistent postwithdrawal anxiety and mood disorders. Psychother Psychosom 2012;81(6):386-388 [FREE Full text] [doi:10.1159/000341178] [Medline: 22964821]
Patient-Friendly TL(V2. The MSSO. 0) URL: https://www.meddra.org/patient-friendly-term-list [accessed 2018-07-07][WebCite Cache ID 70iTr7Lyt]
Freund PR, Rowell LB, Murphy TM, Hobbs SF, Butler SH. Blockade of the pressor response to muscle ischemia by sensory
nerve block in man. Am J Physiol 1979 Oct;237(4):H433-H439. [doi: 10.1152/ajpheart.1979.237.4.H433] [Medline: 495728]
Wong A, Plasek JM, Montecalvo SP, Zhou L. Natural Language Processing and Its Implications for the Future of Medication
Safety: A Narrative Review of Recent Advances and Challenges. Pharmacotherapy 2018 Jun 09. [doi: 10.1002/phar.2151][Medline: 29884988]
Uzuner O, South BR, Shen S, DuVall SL. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. J
Am Med Inform Assoc 2011;18(5):552-556 [FREE Full text] [doi: 10.1136/amiajnl-2011-000203] [Medline: 21685143]
Li Y, Salmasian H, Harpaz R, Chase H, Friedman C. Determining the reasons for medication prescriptions in the EHR
using knowledge and natural language processing. AMIA Annu Symp Proc 2011;2011:768-776 [FREE Full text] [Medline:
22195134]
Rudd RA, Aleshire N, Zibbell JE, Gladden RM. Increases in Drug and Opioid Overdose Deaths--United States, 2000-2014.
MMWR Morb Mortal Wkly Rep 2016 Jan 01;64(50-51):1378-1382 [FREE Full text] [doi: 10.15585/mmwr.mm6450a3][Medline: 26720857]
Shang N, Xu H, Rindflesch TC, Cohen T. Identifying plausible adverse drug reactions using knowledge extracted from theliterature. J Biomed Inform 2014 Dec;52:293-310 [FREE Full text] [doi: 10.1016/j.jbi.2014.07.011] [Medline: 25046831]
Shetty KD, Dalal SR. Using information mining of the medical literature to improve drug safety. J Am Med Inform Assoc
2011;18(5):668-674 [FREE Full text] [doi: 10.1136/amiajnl-2011-000096] [Medline: 21546507]
Hazell L, Shakir SAW. Under-reporting of adverse drug reactions : a systematic review. Drug Saf 2006;29(5):385-396.
[Medline: 16689555]
Botsis T, Nguyen MD, Woo EJ, Markatou M, Ball R. Text mining for the Vaccine Adverse Event Reporting System:
medical text classification using informative feature selection. J Am Med Inform Assoc 2011;18(5):631-638 [FREE Fulltext] [doi: 10.1136/amiajnl-2010-000022] [Medline: 21709163]
Xu H, Stenner SP, Doan S, Johnson KB, Waitman LR, Denny JC. MedEx: a medication information extraction system forclinical narratives. J Am Med Inform Assoc 2010;17(1):19-24 [FREE Full text] [doi: 10.1197/jamia.M3378] [Medline:
20064797]
Carrell DS, Cronkite D, Palmer RE, Saunders K, Gross DE, Masters ET, et al. Using natural language processing to identify
problem usage of prescription opioids. Int J Med Inform 2015 Dec;84(12):1057-1064. [doi: 10.1016/j.ijmedinf.2015.09.002][Medline: 26456569]

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 13(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
23.

24.

25.

26.

27.
28.

29.
30.

31.
32.
33.
34.
35.
36.
37.

Usui et al

Haerian K, Varn D, Vaidya S, Ena L, Chase HS, Friedman C. Detection of pharmacovigilance-related adverse events using
electronic health records and automated methods. Clin Pharmacol Ther 2012 Aug;92(2):228-234 [FREE Full text] [doi:10.1038/clpt.2012.54] [Medline: 22713699]
Gonzalez-Hernandez G, Sarker A, O'Connor K, Savova G. Capturing the Patient's Perspective: a Review of Advances in
Natural Language Processing of Health-Related Text. Yearb Med Inform 2017 Aug;26(1):214-227 [FREE Full text] [doi:10.15265/IY-2017-029] [Medline: 29063568]
Nikfarjam A, Sarker A, O'Connor K, Ginn R, Gonzalez G. Pharmacovigilance from social media: mining adverse drug
reaction mentions using sequence labeling with word embedding cluster features. J Am Med Inform Assoc 2015
May;22(3):671-681 [FREE Full text] [doi: 10.1093/jamia/ocu041] [Medline: 25755127]
Topaz M, Lai K, Dhopeshwarkar N, Seger DL, Sa'adon R, Goss F, et al. Clinicians' Reports in Electronic Health Records
Versus Patients' Concerns in Social Media: A Pilot Study of Adverse Drug Reactions of Aspirin and Atorvastatin. DrugSaf 2016 Mar;39(3):241-250. [doi: 10.1007/s40264-015-0381-x] [Medline: 26715498]
White RW, Wang S, Pant A, Harpaz R, Shukla P, Sun W, et al. Early identification of adverse drug reactions from searchlog data. J Biomed Inform 2016 Feb;59:42-48. [doi: 10.1016/j.jbi.2015.11.005] [Medline: 26610385]
Freifeld CC, Brownstein JS, Menone CM, Bao W, Filice R, Kass-Hout T, et al. Digital drug safety surveillance: monitoring
pharmaceutical products in twitter. Drug Saf 2014 May;37(5):343-350 [FREE Full text] [doi: 10.1007/s40264-014-0155-x][Medline: 24777653]
Denecke K, Krieck M, Otrusina L, Smrz P, Dolog P, Nejdl W, et al. How to exploit twitter for public health monitoring?
Methods Inf Med 2013;52(4):326-339. [doi: 10.3414/ME12-02-0010] [Medline: 23877537]
Aramaki E, Miura Y, Tonoike M, Ohkuma T, Mashuichi H, Ohe K. TEXT2TABLE: Medical text summarization system
based on named entity recognition and modality identification. 2009 Presented at: Workshop on BioNLP; June 4-5,2009;
Colorado p. 185-192.
Aramaki E, Yano K, Wakamiya S. MedEx/J: A One-Scan Simple and Fast NLP Tool for Japanese Clinical Texts. Stud
Health Technol Inform 2017;245:285-288. [Medline: 29295100]
Medis. Standard disease name master for ICD-10 URL: http://www2.medis.or.jp/stdcd/byomei/byomei.html [accessed
2018-07-06] [WebCite Cache ID 70iJsldRa]
MeCab. MeCab: Yet Another Part-of-Speech and Morphological Analyzer URL: http://taku910.github.io/mecab/ [accessed
2018-05-10] [WebCite Cache ID 6zJAQ4y1e]Friedman C, Hripcsak G. Evaluating natural language processors in the clinical domain. Methods Inf Med 1998
Nov;37(4-5):334-344. [Medline: 9865031]
Hripcsak G, Rothschild AS. Agreement, the f-measure, and reliability in information retrieval. J Am Med Inform Assoc
2005;12(3):296-298 [FREE Full text] [doi: 10.1197/jamia.M1733] [Medline: 15684123]
Ministry of Health, Labour and Welfare of Japan. Report on Public Health Administration and Services FY2016 URL:
https://www.mhlw.go.jp/english/database/db-hss/rophas.html [accessed 2018-07-06] [WebCite Cache ID 70iKpMh52]
Japan Pharmaceutical Society. Report on insurance dispensing trends URL: http://www.nichiyaku.or.jp/activities/division/
faqShinchoku.html [accessed 2018-07-06] [WebCite Cache ID 70iLDq7Pi]

Abbreviations
ADE: adverse drug event
ADR: adverse drug reaction
AMED: Japan Agency for Medical Research and Development
CRF: conditional random fields
EHR: electronic health record
EMHD: electronic medication history data
FAERS: US Food and Drug Administration Adverse Event Reporting System
ICD-10: International Classification of Diseases, Tenth Revision
MedDRA: Medical Dictionary for Regulatory Activities
NLP: natural language processing

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 14(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Usui et al

Edited by G Eysenbach; submitted 12.05.18; peer-reviewed by MA Mayer, KNB Nor Aripin; comments to author 13.06.18; revised
version received 07.08.18; accepted 25.08.18; published 27.09.18
Please cite as:
Usui M, Aramaki E, Iwao T, Wakamiya S, Sakamoto T, Mochizuki M
Extraction and Standardization of Patient Complaints from Electronic Medication Histories for Pharmacovigilance: Natural Language
Processing Analysis in Japanese
JMIR Med Inform 2018;6(3):e11021
URL: http://medinform.jmir.org/2018/3/e11021/
doi: 10.2196/11021
PMID: 30262450©Misa Usui, Eiji Aramaki, Tomohide Iwao, Shoko Wakamiya, Tohru Sakamoto, Mayumi Mochizuki. Originally published in
JMIR Medical Informatics (http://medinform.jmir.org), 27.09.2018. This is an open-access article distributed under the terms of
the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use,
distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly
cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as thiscopyright and license information must be included.

http://medinform.jmir.org/2018/3/e11021/

XSL• FO
RenderX

JMIR Med Inform 2018 | vol. 6 | iss. 3 | e11021 | p. 15(page number not for citation purposes)