Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019

Proposal on Assuring the Reliability ofPatient Registry Data for Use
in Application Dossiers of
Pharmaceuticals and Medical Devices.
［January 2019］

AMED Research on the Regulatory Science of Pharmaceuticals and Medical Devices
Title of research and development: Utilization of real world evidence using patientregistry data to support regulatory decision‒making［Grant number JP18mk0101068］

Introduction
For rare diseases and in other cases where traditional
and/or orthodox randomized controlled clinical trials are
dif ficult to conduct, ef for ts have been made in recent
years to explore the use of the data from existing patient

main ones assuming they are used for individual drugs,
etc.
（1）Market research/Investigation of feasibility of clinical trials（2）Preparation of protocols of clinical trials/postmarketing clinical trials

registries for marketing approval applications of pharma-（3）Recruitment of candidate patients for participation

ceuticals, medical devices and regenerative, cellular ther-in clinical trials/postmarketing clinical trials

apy and gene therapy products（hereinafter referred to as（4）External controls（including historical controls）in“drugs, etc.”）; or for re‒examination applications（includ-

clinical trials/investigator‒initiated clinical trials

ing use‒results survey of medical devices）（hereinafter（hereinafter referred to as“clinical trials, etc.”）inreferred to as“approval applications, etc.”）; or for evalua-

the field of rare diseases or other fields or on medi-tion of drugs, etc. as“real‒world data.”

cal devices or other products, for which it is diffi-

Patient registries are designed/operated by diverse

cult to implement the development by randomized

parties, including governments, healthcare institutions

controlled trials at the significance level/statisticaland academic societies, for various medical and scientificpower that is commonly used in confirmatory tri-

purposes. Data of relevant patients, such as those relatedals, under the following conditions:

to medical practice, are accumulated in databases in an●randomized controlled trials at a less strict sig-

integrated manner according to those purposes. There arenificance level/with a lower statistical power are

different ways to use patient registries for the development of drugs, etc. The 5 usages presented below are thedifficult to conduct●a control group is not necessar y in the same

（Note）This document was prepared by the research team by compiling the research results from fiscal years 2016 to 2018.

s23

Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019study but information on an external control isreliability of the data needs to be ensured at a level consid-

useful as supporting data for evaluation of drugs,

ered sufficient in light of the purpose of the use. However,etc.

it is also not necessarily adequate to define the same level（5）Postmarketing surveillanceof requirements as those for clinical trials, etc. in the use

For the purpose of approval applications, etc., patientof patient registries for approval applications, etc. Instead,registr y data are expected to be used in an application

it is desirable to clarify the level specific to patient regis-

dossier for（4）and in a re‒examination application dossier

tries. Nevertheless, patient registries vary in their charac-

for（5）above. In particular, the use for（4）is roughly

teristics and so do the methods for using them; it is there-

divided into（ⅰ）use as an alternative to the data fromfore unclear how the reliability of the data should be

randomized controlled clinical trials that ser ve as evi-

ensured, which may inhibit the utilization of other wise

dence for efficacy;（ⅱ）use as a supplement, in terms ofpotentially useful data.

safety and other information, for the clinical data package;

Focusing on the use of patient registry data for a newand（ⅲ）use as a source of data on actual medical practicedrug application dossier or a re‒examination application

when these data are difficult to obtain in clinical trials, etc.

dossier（hereinafter referred to as“application dossier,For each patient registry, whether or not its data canetc.”）, we will present in this document（hereinafter

be used for approval applications, etc. depends on the pur-referred to as“this proposal”）the basic ideas for thepose and status of its design/operation. Because there are

requirements for the design and operation of patient regis-

extremely diverse ways to use patient registries depend-

tries that should be met in the use of these registries foring on the therapeutic areas or purposes, and the regis-

the purpose of approval applications, etc. of drugs, etc., astries are not usually designed/operated for approval appli-

well as possible requirements that applicant companies

cations, etc., the quality of the accumulated data varies. As

should meet in the use of patient registries for approvalrequired in“Ethical Guidelines for Medical and Healthapplications, etc. of drugs, etc.: the former requirements

Research Involving Human Subjects（28 MEXT/RPB Noti-

are presented for parties that design/operate patient reg-

fication No. 406, MHLW/MS/HSD Notification No. 0228‒

istries（hereinafter referred to as“patient registry hold-1, and MHLW/HPB Notification No. 0228‒1 dated Febru-

ers”）and the latter requirements are presented for users

ary 28, 2017, issued by the Director General of Researchof these patient registries for approval applications, etc. ofPromotion Bureau, the Ministr y of Education, Culture,drugs, etc. We hope that, with the help of this proposal,

Spor ts, Science and Technology, the Director of theuse of patient registries for approval applications, etc. will

Health Sciences Division, Minister’s Secretariat, the Min-be considered more often and there will be more chance

istry of Health, Labour and Welfare, and the Director Gen-of actual use.

eral of Health Policy Bureau, the Ministr y of Health,Labour and Welfare）”, researchers should ensure the reliability of data‒they have to make sure that the information

1．Objectives

used in their research and the records related to the infor-

In this proposal, we list the points that patient regis-

mation concerned are accurate. However, there will be

try holders and the users of patient registries for approval

different opinions regarding the degree and the method of

applications, etc. of drugs, etc. need to consider in order tothis assurance, depending on the situation. In light ofappropriately ensure the reliability of the data being used

these circumstances, the concept of data reliability assur-

or expected to be used for such purposes. In doing so, weance in the use of patient registries for approval applica-

divided the points intoⅰ）those related to the design/

tions, etc., needs to be carefully discussed. Decisions on

operation of patient registries andⅱ）those related to con-the level of data reliability required for patient registries

firming that the reliability of the information presented inand the method for ensuring this reliability should be

an application dossier, etc. was ensured at a level consid-

made according to the purposes of the use of the regis-

ered sufficient in light of the purpose of use.

tries. Therefore, it is difficult and inappropriate to define

Ethical issues associated with the use of patient regis-

uniform reliability criteria for all purposes. When using

tries for approval applications, etc., are impor tant andpatient registr y data for approval applications, etc., theneed to be handled appropriately. However, the details ofs24Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019

these issues will not be handled in this proposal. More-Regulations for Law on Securing Quality, Ef ficacy andover, use for approval applications, etc. may raise issues

Safety of Pharmaceuticals, Medical Devices, Regenerativethat are difficult to handle in accordance with the current

and Cellular Therapy Products, Gene Therapy Products,

Japanese GCP ordinance. We therefore consider that it is

and Cosmetics［hereinafter referred to as“Enforcement

necessary to separately discuss how the regulatory sys-

Regulations”］for drugs, Article 114‒22 of Enforcement

tem should respond to such use. This discussion is also

Regulations for medical devices, Article 137‒25 for regen-outside the scope of tasks conducted by our researcherative, cellular therapy and gene therapy products and,team and is therefore not included in this proposal.

during re‒examination, Article 61 of Enforcement Regulations for drugs, Article 114‒42 for medical devices and2．Parties to which this proposal is applicableThis proposal is applicable toⅰ）the patient registry

Article 137‒42 for regenerative, cellular therapy and genetherapy products）andⅱ）confirmation in accordance with
GCP（Good Clinical Practice）（Figure 1）.

holders who provide or are considering providing the

In the confirmation in accordance with the standards

patient registry data concerned to the parties that plan toof reliability, the accuracy, integrity/completeness, andfile marketing approval applications, etc. of drugs, etc.

preservation of the data
（specifically, the data that serve as（hereinafter referred to as“applicants”）for the use inevidence）attached to the approval application are con-application dossiers, etc. that are classified into the cate-firmed. Main evidence data include the materials or othergories（4）or（5）, mentioned above in“Introduction”andforms of data that clinical trial sponsors should retain. Anⅱ）the parties（applicants）that use or are considering

example is evidence data related to quality assurance ofusing the concerned patient registry data for application

the data, including those in the data management plan/

dossiers, etc., in a manner classified into the categories

report and the statistical analysis plan/report. Case report（4）or（5）mentioned above. This proposal is not appli-

forms and monitoring records at all healthcare facilities

cable to patient registry holders or parties that use patient

are also considered to be main evidence data. Among

registries that do not meet the above conditions.

these evidence data, some, including the data from caseThis proposal is also not applicable to cases including

report forms at healthcare facilities, need to be checked in

（1）
（2）
,
, and（3）above, clinical research where the use isterms of the compliance with GCP. This compliance is

not for the purpose of application dossiers, etc., or cases

checked in on‒site inspections at a cer tain number ofwhere the patient registr y concer ned is originallyselected healthcare facilities. Specifically, evidence data

designed to fulfill the approval conditions and is subject tothat should be retained at healthcare facilities, including

other regulations including the GPSP（Good Post‒market-

medical records, laboratory test slips, and patient diaries,

ing Study Practice）ordinance. From the perspective ofare checked. Overall, there are 2 purposes of this inspec-

regulatory sciences, patient registry data may be used, fortion: one is to ensure the scientific quality of clinical trials

example, to evaluate the adequacy of alternative endpoints

and the reliability of the results; the other is to protect thefor individual diseases, not assuming development or reg-human rights and safety of the subjects. From the latterulatory approval applications, etc. of specific drugs, etc.

perspective, the process involved and other aspects of theSuch use is not dealt with this proposal.

acquisition of consent from subjects are also confirmed.

3．Concept of data reliability
3.1

3.2

Standards of reliability and confirmation process
for use of patient registries in application dossiers,

Standards of reliability of application dossiers, etc.

etc.

in general and process of confirmation

When confirming the reliability of patient registr y

The Pharmaceuticals and Medical Devices Agency

data, it is unrealistic to apply all the requirements related（PMDA）confirms the reliability of application data fromto on‒site inspection for GCP compliance（although theirclinical trials, etc. attached to the approval applicationnecessity cannot be denied）, because studies using

using roughly 2 processes: ⅰ）
confirmation in accordance

patient registries are planned and conducted differently

with the standards of reliability（Article 43 of Enforcement

than clinical trials, which are intervention studies. Nevers25

Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019

Figure 1

Over view of data reliability assurance in clinical trials of drugs―GCP standard and standards of reliability

theless, as patient registries are intended for medical/sci-

postmarketing sur veillance, the GPSP ordinance isentific purposes, they need to be designed/operated in aapplied, while for the criteria for postmarketing clinical tri-

manner in which data reliability is sufficiently ensured inals, the GPSP and GCP ordinances are applied.

light of the actual status of consultation and follow‒up of

It should be noted that unlike clinical trials, many

patients in each therapeutic area, as well as the purposespatient registries are not planned/operated by pharma-

regarding the use of patient registries. In this sense, it isceutical companies, and thus the relationship between theconsidered rational to confirm data reliability by referring

clinical trial sponsor（pharmaceutical company）and theto the standards of reliability.

contract research organization assumed in Article 12 of

For this reason, we do not list in this proposal the

the GCP ordinance is hardly applicable to the relationshippossible requirements of compliance review in accor-

between the applicant（pharmaceutical company）and thedance with GCP and with the standards of reliability.

patient registry holder. Even if the contract signed by theInstead, regarding the items for which inspection or com-

applicant（pharmaceutical company）and the patient reg-pliance review is carried out, we maintain the structural

istry holder in the above relationship corresponds to Arti-

relationships between patient registries and clinical trials

cle 12 of the GCP ordinance, the actual contract depends

when discussing the following 2 aspects: ⅰ）matterson the usage of data and the relationship between the par-related to the design/operation of patient registries（rela-ties involved in the contract, and may be similar to thetionship between healthcare facilities and patient registry

contract for data provision or other matters between aholders, Figure 2,（a））, andⅱ）matters necessar y forpharmaceutical company and a sponsor‒investigator in

determining that the reliability of the information pre-cases of utilization of the data from an investigator‒initi-

sented in application dossiers, etc. is ensured at a levelated clinical trial, or the contract between a pharmaceuti-

considered suf ficient in light of the purpose of its use

cal company and a business operator handling medical（relationship between patient registry holders and appli-

information database（a DB business）based on the GPSP

cants and relationship between applicants and application

ordinance（Figure 3）. The relationship between a health-dossiers, etc., Figure 2,（b）and（c））. For the criteria forcare facility and a patient registry holder generally varies

s26

Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019
Evidence data at
healthcare facilities
Source data
（medicalrecords, test slips, patient
diaries, etc.）（a）
Data managed bypatient registry holder（b）Snapshot of patientregistry data received
by applicant（c）Data attached
to NDA form
（NDA dossier）（a）Matters related to the design/operation of patient registries（relationship between healthcare facility and patient registry holder）（b）Matters necessary for the judgement that the reliability of the information presented
in application dossiers, etc. is ensured at a level considered sufficient in light of purposeof the use（relationship between patient registry holder and applicant ）（c）Matters necessary for the judgement that the reliability of the information presented
in application dossiers, etc. is ensured at a level considered sufficient in light of the
purpose of the use（relationship between applicant and application dossier, etc.）

Figure 2

Figure 3

Points of reliability assurance between parties involved in patientregistries

Diagram of contractual relationshipaccording to the presence or absence of a contract. For

tical companies may range from those similar to the ones

some patient registries, the holder has a direct relation-

in investigator‒initiated clinical trials shown in Figure 3

ship not with a healthcare facility, but with patients. That isto those similar to the ones in postmarketing databaseto say, because of the diversity in how patient registries

surveys. However, one point of view is that in use cases ofare planned/operated, their relationships with pharmaceu-

approval applications, etc. that are beyond the scope diss27

Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019
Table 1

Responses of involved parties categorized by purpose/usage

PurposeUsage（a）Between healthcare
facility and patient registry
holderOther than approval （1）Market research/Investigation of feasibility of cliniapplications, etc.
cal trials

Confirmation of achievementof the standard specified
jointly by the healthcare facility and the patient registr y（2）Preparation of protocols holder
for clinical trials/postmarketing clinical trials（b）Between patientregistry holder andapplicant（or health carecompany, etc.）（c）Between applicant
and application
dossier, etc.

Confirmation of the absenceof a gap between the qualityof information the patientregistry can provide and thequality of information
requested by the company,
etc.

－

－

－

（3）Recruitment of candidate
patients for participation inclinical trials/postmarketing clinical trials
Approval applica- （4）External controls（includtions, etc.
ing historical controls）inclinical trials, etc.
（including postmarketing clinical
trials）in the field of rarediseases and other fields inwhich development byregular clinical trials, etc.
is difficult

Confirmation of consistency
between the source data at
healthcare facilities and thepatient registry data which are
based on GCP（for example,
confirmation of consistency
with the source data under acertain condition; if confirmation is dif ficult, the reason
should be explained）

Direct confirmation by the Compliance review
applicant of the status ofoperation and other aspectsof the patient registry（5）Postmarketing sur veillance

Confirmation that the patientregistry is designed and operated in a way that the source
data at the healthcare facility
are appropriately registered in
the registry（if confirmation isdifficult, the reason should be
explained）

Direct confirmation by the Compliance review
applicant of the status ofoperation and other aspectsof the patient registry（ a ）Matters related to the design and operation of patient registries（b）Matters necessary for the judgement that the reliability of the information presented in application dossiers, etc. is ensured at a level sufficient in light of the purpose of the use（ c ）Matters necessary for the judgement that the reliability of the information presented in application dossiers, etc. is ensured at a level sufficient in light of the purpose of the use

cussed in this proposal, the formatting of contracts, han-

3.3

Relationship between usage of patient registries anddling of data, and other aspects should be performed inreliability confirmation process

the same way as in clinical trials, complying with the GCP

For the usage of patient registries in（1）to（3）inordinance. In other words, it is crucial whether or not the“Introduction,”it should not be problematic in Figure 2

use case is within the scope discussed in this proposal. To（a）if appropriate measures are taken between the health-resolve this difference in opinions, case‒by‒case discus-care facility and the patient registry holder, and no prob-

sions of specific situations are necessary. It is importantlems related to（c）should arise. For（4）and（5）, certain

for the applicant and/or patient registry holder to proac-

responses need to be taken for（a）（see Table 1）. How-

tively consult PMDA regarding these specific situations.

ever, the standard required for（4）and（5）is not the sameas the standard assuming the requirements corresponding to those in GCP or GPSP. In principle, under the qual-s28Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019ity management system created according to the thera-cants when they prepare the data sets at the time of data

peutic area or the characteristics of the patient registry atlock or data cutoff, when they prepare the data sets forthe time the registry is established, reliability needs to beanalysis, and, if analysis results are provided to outsideassured from the perspective of（a）, through the operation

parties, when they compile the analysis results（includingof the registry and other related tasks in accordance with

output sheets, tables, figures, and other forms）and pre-various written procedures listed as examples in“4. Mat-pare the analysis reports. Applicants or regulatory author-

ters related to design/operation of patient registries:

ities may request that patient registr y holders provide

examples of written procedures and other materials of

documents confirming the above reliability assurance.

patient registry holders（for reference）”.
（ⅱ）Method for assuring the reliability of patient registry
3.4

Concept of reliability in use as“control group, etc.

data on individual patientsin clinical trials, etc. in the field of rare diseases

Unlike the case in which summary statistics are citedand other fields in which development by regularfrom published papers or other sources for use as anclinical trials, etc. is difficult”

external control, the use of data on individual patients pro-（ⅰ）Circumstances in the development of drugs, etc. forvides researchers with the advantage of being able to per-rare diseases and other diseases in which dr ug

form detailed/precise statistical analysis of patient back-development by regular clinical trials, etc. is difficult

ground factors, clinical information and other data, instead

In this proposal, it is assumed that the main circum-of having only the information regarding endpoints. To

stances in which patient registries are used are: ⅰ）“themake this possible, the reliability of data in patient regis-

field of rare diseases or other fields in which randomized

tries has to be ensured at a level considered sufficient incontrolled trials at the significance level/statistical power

light of purpose of their use. For this reason, when usage

commonly used in confirmatory trials and other clinical（4）is intended, certain methods for data reliability assur-

trials designed/planned acceptable by regulatory authori-ance, for example, Source Document Verification（SDV）,

ties are dif ficult to conduct,”in other words, circum-

are supposed to be per formed by the patient registr y

stances in which regular development is difficult, andⅱ）

holder; that is, sampling and checking of the accuracy of“the circumstances/fields in which controlled trials used

the source documents, including medical records and theto be difficult, but it is considered appropriate to conduct

data registered in the patient registry. If data are obtained

controlled trials with concurrent control formed using

not by manual input but through a computer system thatpatient registries.”

retrieves the data from electronic charts or other records,

The main purpose of this proposal is to clarify thecurrently obscure conditions and points of discussion for

confirmation that the system is working as designed canbe one method of reliability assurance.

data usage in situations where it is difficult to collect suffi-

Depending on the therapeutic area and the form ofcient data under a common development policy. There-

patient registry, SDV or other methods may be difficult tofore, this proposal does not discuss the use of patient reg-

perform. This is because the data concerned may not be

istries for straightfor ward data collection in situations

recorded in the source document, or because confirming

where sufficient data collection is possible.

the record in the source document may not be possible forIf the same level of reliability as in clinical trials, etc.

reasons including the following:is required in the above situation where sufficient data col-‒The system for patient follow‒up in daily medical prac-lection is difficult, this requirement defeats the purpose oftice varies by disease.

using patient registries; the need to rely on patient regis-‒Medical records are not intended to include informa-tries is assumed in this discussion. In addition, thistion that is necessary for potential future analyses, but

requirement would make it difficult to use the registries.

rather information that is necessary in daily medical

Nevertheless, as long as registries are used for approvalpractice, and therefore these records may not be con-applications, etc., data reliability needs to be ensured.

sidered to be source data.

Therefore, patient registry holders must maintain the reli-‒When a patient registry is established without assum-

ability of data or other information they provide to appli-

ing the use of its data for approval applications, etc.,s29

Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019

patient consent may not have been obtained for view-

sent needs to be obtained for the collection/utilization of

ing of medical records and other information by thedata and for the confirmation of the source document by

applicant or other par ties. If this consent was not

involved parties.

obtained, SDV, which is considered as one option to
directly confirm the status of the operation and otheraspects of the patient registry by the applicant, is difficult to perform.

（ⅲ）Different levels of rigor required for data reliability
assurance
Assuming that SDV or other methods are necessary,‒The patients concerned are not necessarily treated at

the level of rigor required needs to be discussed not only

the same facilities throughout their clinical course, andconsidering the regulatory systemic aspects and feasibility

there are limitations regarding the collection of infor-

mentioned above, but also in relation to the level of preci-

mation from other facilities.

sion required for the specific usage and the strength ofIn the above cases, the reason it is difficult to assure data

unavoidable bias.

reliability using the patient registry holder’s SDV or otherFirst, for both rare diseases and other diseases, when

methods needs to be clarified by the applicant, as does theerrors and their proportions and other characteristics inadequacy of the use of the relevant patient registr y forthe data concerned are identified by SDV or other meth-approval applications, etc. Alternatively, a rule can be set

ods, it is easy to determine whether or not the data in thespecifying that cer tain information registered in therelevant patient registry are sufficiently precise given thepatient registry is handled as source data.

purpose of the data usage. For example, when the capture

However, even when the source document is speci-

propor tion of an outcome event is not 100％, but thefied, it is not acceptable to perform SDV in a formal man-

uncertainty involved can be estimated, it is possible toner only. Because of the characteristics of patient regis-

make an inference in the statistical analysis that takes thetries, the following issues would arise.

uncertainty into account. If the precision for the final infer-‒Unlike in clinical trials, etc., endpoints, obser vationence is sufficient even when the uncertainty in the event

schedules, analysis methods, and other details are notcapture is taken into account, an event capture proportion

specified in advance; therefore the definition of the

below 100％ would not in itself pose any problem in thesource document that provides the source data forsense that erroneous judgement can be avoided. In other

patient registries may be ambivalent, as may the infor-words, what is important is not the absence of errors inmation itself.

data, but the fact that the quality and quantity of errors are‒Unlike in clinical trials, etc., it is difficult to determine

maintained within a range that does not lead to erroneous

what cost is acceptable for reliability assurance when it

judgements based on the data. In order to make suchis uncertain if the data are going to be used; even if

judgements possible, methods for reliability assurance

they are used, the purpose of the use or requirements

and associated records are beneficial. This is similar tofor the use, including the level of accuracy/precision ofscientific experiments where the measurement error of

the registries, are uncertain.

each measurement device used in the experiments is not

Therefore, it may be necessar y to check, in light of thezero, but there is no problem if the measurement error

purpose of data usage, if there are any problems with the

can be identified and the level of overall error resulting

analysis plan and evaluation of the analysis results.

from the accumulation of individual errors still allows forMoreover, patient registr y holders should, if they

conclusions to be drawn at a sufficient level of precision.

assume there will be a future use for an application dos-

Additionally, regarding patient registries for whichsier, etc., establish or review the design/operation system

the timing of data collection is not specified in advance,of the registries. For example, holders can introduce aunlike interventional studies, attention should be paid tomechanism to retain the records that serves as the source

the impact of bias associated with data collection. Patient

data for the patient registry data, in forms including theregistry data include 1）the data related to patient back-source document and alternative documents. They canground before the start of the treatment with the drugs,

also attempt to obtain the consent of patients for possible

etc. that are to be evaluated, and 2）the data related to theuse of their data. In addition, depending on the usage, con-outcome measured after the start of the treatment. For thes30

Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019latter, whether or not data are collected is likely to be

discussed how to scientifically conceptualize and respondassociated with each patient’s condition, that is to say, withto the limits of data reliability assurance that necessarily

the outcome of treatment with the drug, etc. Unlike inter-

arise despite maximum measures taken in light of the data

ventional studies in which the timing of tests is specified

usage.

in advance and the tests are per formed regardless ofpatient condition, in cases of patient registries physicians（ⅳ）Data storage

decide whether or not to perform certain tests according

In addition to the above, appropriate storage of data is

to patient condition. Therefore, analyzing the obtained

important. In“Ethical Guidelines for Medical and Healthdata as they are may give rise to bias in the evaluation of

Research Involving Human Subjects,”storage of informa-treatment results. Moreover, patients’ conditions maytion is mentioned in relation to research reliability assur-

sometimes affect whether or not information from certain

ance. Some research institutions have a specific systempatients is registered into patient registries. In such cases,for storing the data sets and other information that serve

there may be a concern regarding the appropriateness ofas evidence in research papers and other materials.

registered patients as comparators. Even when complete

Patient registry holders need to specify the data storage

consistency between source documents and data in asite and the person responsible for the storage to ensure

patient registry is confirmed by SDV or other methods,

the appropriate storage of the data concerned, even when

the data might not be usable because of the bias men-the data are used for approval applications, etc.

tioned above, depending on how the data will be used.
Thus, the level of rigor required for data reliability
assurance depends on how the patient registry data areused in approval applications, etc.
（how they are characterized compared with other data in clinical data package,

4．Matters related to design/operation of registries:
examples of written procedures and other materials
that patient registr y holders should prepare（for reference）

importance placed on them in PMDA review, etc.）, disease, status of existing therapeutic methods, characteris-

The written standard operating procedures and othertics of drugs, etc., statistical characteristics of the usedmaterials that patient registr y holders should prepare

data, including the level of precision required for discus-

when data are used for postmarketing surveys are listed insion, and other factors. In addition, patient registries differ

the attachment to“Points to Note regarding Reliability

in their design/operation, which makes it dif ficult toAssurance in Postmarketing Database Surveys for Drugs”

ensure the reliability of data using a uniform standard. For（MHLW/PSEHB/PED Notification No. 0221‒1 dated Feb-

this reason, the level of data quality should be objectively

ruary 21, 2018, issued by the Director of Pharmaceuticaldefined by the quality management system in each patient

Evaluation Division, Pharmaceutical Safety and Environ-registry, and, in discussion/consultation with PMDA andmental Health Bureau, the Ministry of Health, Labour andother parties, the following matters should be addressed:

Welfare）. Patient registry holders should refer to this list

1）matters for which applicants should present their opin-

when preparing and following the written procedures andions in discussion, 2）matters for which PMDA should

other materials.

present its opinions, and 3）matters for which patient reg-

However, the written procedures and other materials

istry holders should present their opinions, as necessary.

necessary for the conduct of clinical trials, etc. that utilize

If consultation experience accumulates to a certain
extent in the future and consultation cases are classified
into certain types and the result is disclosed by PMDA,

patient registries should be discussed further.
The items with ＊ are not included in the above notification.

the issues that should be addressed in consultation areclarified/specified for both applicants/pharmaceutical

1 ）Rules regarding establishment/managementcompanies and patient registr y holders. Discussion○ Organizational structure: Responsible person, man-should take place about the feasibility of clarifying theager, locus of responsibility for facilities, education/issues to be addressed in consultation with PMDA andtraining and other matters at patient registry holders.

about related systemic improvement. In this section, we○ Management of outsourcees: The procedures or plans
s31

Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019

to confirm the appropriate management of outsour-○ Procedures for modifying the standards of this task:

cees by patient registr y holders in cases where

Procedures and process of modifying the standards

patient registry holders outsource some of their tasks.

of data cleaning.
○ Management of outsourcees: Procedures and plans＊for confirming the appropriate management of out-

registries

sourcees by patient registry holders in cases where

2 ） Standards/procedures for entering data into patient○ Roles of the depar tment in charge and persons inpatient registry holders outsource some of their tasks.

charge: Responsibilities of patient registr y holders
and persons involved in the tasks, including the4 ）Standards/procedures for coding

healthcare facilities and other parties that provide the○ Roles of the depar tment in charge and persons in

data, are specified in written procedures.
○ Receipt, entry/import of data: Methods for provisionand receipt of data（via network, media, etc.）, proce-charge: Departments and responsibilities of the person responsible for this task, manager of this task andperson in charge of this task.

dures for input/import, handling of entered data and○ Standards/procedures of this task: Data to be codedother details are specified in written procedures. For

and details specified for each step taken by patient

data entry, in particular, the preparation of detailed

registry holders to code the data（importing data col-procedures regarding handling of entered data andlected from information sources, preparing data setsother tasks leads to data quality assurance.

for analysis, etc.）.

○ Anonymization of data（anonymization with a decod-○ Preparation of coding list and other materials（mas-ing index retained at the facility as necessary）: Theter）to be used for coding: Frequency and methods ofanonymization method used in the patient registry is

updating the masters concerned.

described in written procedures. In anonymization of○ Procedures for modification of the standards for thispersonal information, whether or not the decoding

task: Procedures and process of modification of theindex should be retained at the facility depends onstandards for coding by patient registry holders.

the purpose of the collection of patient registry data

○ Management of outsourcees: Procedures and plans

and the usage of the data.

for confirming the appropriate management of out-

○ Method of confirming that data entry/import is cor-sourcees by patient registry holders in cases where

rect: Method of confirming the correctness of data

patient registr y holders outsource some of theirentry/import is described in written procedures.

tasks.

○ Management of outsourcees: Written standard operating procedures and plans for confirming the appropri-5 ）Rules/procedures related to security

ate management of outsourcees by patient registr y○ Roles of the depar tment in charge and persons inholders when patient registr y data are used forcharge: Departments and responsibilities of the per-approval applications, etc., in which patient registry

son responsible for this task, manager of this task andholders outsource some of their tasks.

person in charge of this task.
○ Rules for information security

3 ）Standards/procedures for data cleaning（ⅰ）Rules for management of logging into and out of○ Roles of the depar tment in charge and persons inhealthcare information databases: Rules forcharge: Departments and responsibilities of the per-entry/exit are set at the level considered neces-son responsible for this task, manager of this task andsary in light of the healthcare data handled inperson in charge of this task.

the healthcare information databases, the struc-○ Standards/procedures of this task: Data items subject

ture of the system and the operation method toto cleaning and details specified for each step taken

specify the preparation/use/storage of the log‒

by patient registry holders to implement the cleaning（importing data collected from information sources,
preparing data sets for analysis, etc.）.
s32

in/log‒out record.
（ⅱ）Rules for management of users: Range of users
and management methods, including userJpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019account setup, are specified.

○ Systems to be validated, methods and procedures for（ⅲ）Rules for access control: Setup and control ofvalidation: Systems to be validated and the interval,

authorization of user access in accordance withtechniques and other details of validation ar e

the importance of healthcare data are specified.

described in the plan.

（ⅳ）Rules for network security: Rules are set for network security.

○ Results of validation: Validation result report is prepared.

○ Rules for other types of security: Rules related to theplans for continuation of business, etc. are set.

9 ）Rules for verifying the appropriate preparation of data
sets for analysis or analysis results.

6 ）Rules/procedures related to data backup and recovery○ Roles of the depar tment in charge and persons incharge: Departments and responsibilities of the per-○ Roles of the depar tment in charge and persons in

son responsible for this task, manager of this task andcharge: Departments and responsibilities of the per-person in charge of this task; preparation to be veri-

son responsible for this task, manager of this task andfied/methods/procedures of this task; interval, meth-person in charge of this task.

ods and other details of this task.

○ Data to be backed up/frequency/generation manage-○ Report of results of this task.

ment/destination: Rules including those regarding○ Management of outsourcees: In cases where patient

the back‒up or update frequency of the whole health-registry holders outsource some of their tasks, proce-care information database, number of generations todures and plans for confirming the appropriate man-be stored, media used for backup, storage location

agement of outsourcees by patient registry holders.

and storage period.
○ Plans and procedures for recovery: Plans for recovery, including specific details and procedures.
○ Recovery testing: Record of the results of recovery
testing.

10）Rules regarding plans related to quality control/
report of confirmation results.
○ Roles of the depar tment in charge and persons incharge: Departments and responsibilities of the person responsible for this task, manager of this task and7 ）Rules regarding quality control of healthcare data collected from information sources○ Roles of the depar tment in charge and persons incharge: Persons in charge of this task and theirperson in charge of this task.
○ Rules regarding quality control plans（including rules
regarding what is subject to quality control）andreport（including confirmation results）.

responsibilities at the information sources and patientregistry holders.

11）Rules regarding quality assurance○ Provision and receipt, entry and import of healthcare○ Roles of the depar tment in charge and persons in

data: Methods for provision and receipt of healthcare

charge: Departments and responsibilities of the per-data（via network, media, etc.）, and procedures for

son responsible for this task, manager of this task andentr y（including standards for input［dictionaries,

person in charge of this task.

etc.］）and import.
○ Input to/output from healthcare information data-○ Rules regarding quality assurance（including rules
regarding how quality assurance is defined）.

bases: Method for confirmation of correct entr y/import of healthcare data.

12）Rules regarding storage of records related to preparation of application dossier for re‒examination, etc.

＊

8 ） Plan/report for validation of computer systems○ Roles of the depar tment in charge and persons in○ Person who performs validation, person responsible

charge: Departments and responsibilities of the per-for validation: Record is prepared specifying the nameson responsible for this task, manager of this task andof the person who performed validation and that ofperson in charge of this task.

the person responsible for validation.

○ Records to be stored: Rules regarding the preparations33

Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019of data sets for analysis or analysis results and the2 ）Management of patient registrydata or other materials prepared during the investiga-○ In approval applications, the computer system con-tion.
○ Procedures for storage: Rules regarding location, procedures, period and other details of storage.

cerned is required to comply with the“Guideline on
Use of Electromagnetic Records/Electronic Signatures in Applications, etc. for Approvals, Licenses, etc.

○ Procedures for transfer: In cases of transfer, rulesof Dr ugs, etc.”（MHLW/PMSB Notification No.

regarding destination and procedures of transfer.

0401022 dated April 1, 2005, issued by the Director○ Procedures for disposal: Rules regarding procedures

General of Phar maceutical and Medical Safety

and other details of disposal.

Bureau, the Ministry of Health, Labour and Welfare）（however, the level of compliance differs according to13）Rules regarding education and training of persons
including those involved in establishment/management

the purpose of use）.
○ Regarding the above requirement, there is an opinion
that the system should comply with the guideline in○ Timing and total number of hours of education andthe same way as clinical trials. Therefore, it is desir-training: Rules regarding timing and total number ofable to consult PMDA in advance when the policy ishours of education and training.
○ People to be educated and trained: Rules regarding

determined in each situation.
○ In re‒examination applications, compliance with thepeople to be educated and trained on each subject.

guideline is not required except in cases of postmar-○ Persons in charge of education: Rules regarding the

keting clinical trials. However, it is appropriate toperson in charge of each education and training session.
○ Content of education and training: Rules regarding
the content of education and training sessions.

establish the system by referring to this guideline.
○ In order to operate/manage patient registries in amanner that the source document, including medical○ Evaluation of results of education and training: Rules

records, can be retrospectively confirmed, decoding

regarding the record summarizing the results of edu-

indices for each healthcare facility need to be pre-cation and training sessions.

pared and appropriately stored.
●Because it is considered difficult to specify uni-

14）＊Record of ethical consideration

versal rules regarding who stores decoding indices and how to store them given the diversity of

5．Points to note regarding data reliability assurance（matters applicants should address to ensurethat the information presented in application dossiers, etc. are consistent with the objective）

registry formats, operations and organizations,the tables should at least be utilized according tothe“Ethical Guidelines for Medical and HealthResearch Involving Human Subjects.”●Moreover, when the use of the patient registry

The following points need to be examined when afor an application dossier, etc. is planned, it isdecision is made to use the registr y concerned for annecessary to reach an agreement with the appli-

application dossier.

cant regarding the possibility of confirming thesource document and, if possible, details of the1 ）System of operation and other aspects of the patient

confirmation.

registry○ Necessary arrangements for audit, access to data andother matters should be made with the applicant.

3 ）Quality assurance○ The patient registry holder needs to ensure the qual-○ System/procedures that enable the applicant to con-ity of the data or other materials provided to the appli-

firm that the patient registry is properly and smoothly

cant in accordance with the rules regarding the qual-

operated/managed should be prepared in advance.

ity assurance system that were specified in advance.

○ From the perspective of personal information, appro-○ The patient registr y holder needs to store related

priate management of data is necessary.
s34

records.

Jpn Pharmacol Ther（薬理と治療）vol. 47 suppl. 1 2019

○ The patient registr y holder also needs to preparematerials regarding the quality of the patient registry
so that these can be presented to the applicant.

this proposal does not assume that the sponsor‒
investigator plays the same role as the applicant.
○ Nevertheless, in cases of the use of patient registries
with investigator‒initiated clinical trials, it is desirable

4 ）Storage of records

for the sponsor‒investigator to examine related mat-

○ It is assumed that records storage is implemented inters in advance so that the potential applicant or otheraccordance with the“Ethical Guidelines for Medical

parties can deal with the requirements listed in thisand Health Research Involving Human Subjects.”

proposal at the time of approval applications or other

However, as the data are used for application dos-

types of applications; or discuss contracts that will besiers, etc., it is necessary to reach an agreement withnecessar y in the future and other related matters

the applicant regarding the storage method and dura-with parties, including the applicant, when the investi-tion for the records such that they will demonstrategator‒initiated clinical trial is being planned.

appropriate data reliability assurance.
6．Definition of terms5 ）Contract with applicant○ In cases where data are used for approval applications,a contract between the patient registry holder and theapplicant based on GCP needs to be signed, depending on the situation.
○ In cases where data are used for re‒examination appli-

Terms used in this proposal are defined as shown
below.
Term

Definition

Patient registry

Database established for the purpose of collectionof healthcare data, including those on a particular disease or disease group and their pharmacologic and nonpharmacologic treatment

cations, a contract between the patient registr yholder and the applicant that complies with GPSP
（Paragraph 2, Article 6 and, if necessary, Article 10）
needs to be signed.
○ For both contracts, it is assumed that the patient registry holder and the healthcare facility/participating

Patient registry Party that designs/operates a patient registry
holder
ApplicantParty that files marketing approval applications,etc. of drugs, etc. using the data from a patientregistry

Data cleaning

To improve a database by deleting/correcting
the data

CodingReplace data, including those on disease conditions, names/use of drugs, etc. and results/
course of treatment with numbers and codes for
efficient computer processing

Drugs, etc.

Pharmaceuticals, medical devices, regenerative,
cellular therapy and gene therapy products

patients gave consent to the use of the data for
approval applications or re‒examination applications
by the applicant, at an appropriate level.
○ The range of data that the patient registry holder provides to the applicant needs to be determined inadvance.
○ In cases of the use of patient registries with investigator‒initiated clinical trials, the question of whether or
not the sponsor‒investigator should play the samerole as the applicant may arise in consideration of thefact that the result of investigator‒initiated clinical trials may be used for regulatory approval applications,

Approval appli- Marketing approval applications and re‒examications, etc.
nation applications（including use‒results
assessment of medical devices）

etc. in the future. However, as the sponsor‒investigator cannot implement the approval applications, etc.,
This document was prepared by the team for AMED Research on Regulatory Science of Pharmaceuticals and Medical Devices
from fiscal year 2016 to fiscal year 2018, entitled“Utilization of Real World Evidence Using Patient Registry Data to Support
Regulatory Decision‒Making”（Grant number, JP18mk0101068）.
Based on the document compiling the research results in fiscal year 2016, the team conducted further research in fiscal years
2017 and 2018 and made some corrections, resulting in the final version of this document（January 2019 version）.
s35*Reprinted with permission from the Japan Society of Clinical Trials and Research