Clin Eval 49（1）2021

Original article

A study on a strategic model
to implement remote auditing ＊Naoki Tsutsumi 1）
Kotone Matsuyama 2）
Hirotaka Nagashima 3）
Kyoko Imamura 1）
1）Social Cooperation Program of IT Healthcare, Graduate School of Pharmaceutical Sciences,
The University of Tokyo
2）Department of Health Policy and Management, Nippon Medical School3）Clinical Research Center, Tokyo Center Clinic

Abstract

Background：The concept of a risk-based approach 1 〜 5) has already been introduced in preparation for theaudit plan in pharmaceutical companies, and the quality assurance in the workplace and regulatory
authorities has been implemented in a well-defined manner 6, 7).
Remote auditing is considered as a new method to optimize the risk-based approach, but its
implementation has raised concerns from both the auditors and the auditees, which have become an
obstacle to achieving remote auditing.
In this study, we identified concerns about remote auditing by questionnaire surveys and developed astrategic model to implement remote auditing to address such concerns.
Methods：First, we developed a questionnaire to confirm the current status of the GCP audit and to
identify concerns regarding remote auditing. Second, feasibility and usefulness of remote auditing wereassessed in a series of verification experiments using Information and Communication Technology (ICT)
tools. Finally, the expected benefits of remote auditing were also investigated by distributing aquestionnaire survey to audit sponsors.
Results：A total of 18 pharmaceutical companies, 47 clinical investigator sites, and 14 contract researchorganizations (CROs) participated in the survey. All investigator site audits (ISAs) were performed on-site.
An auditing model addressing the pre-identified concerns including data access, data security, andremote auditing methods was subjected to verification experiments. The model was found to provide thesame level of audit as a conventional on-site audit in terms of the number of observations. The difficulties
and measures to overcome in the implementation of future remote auditing were made clear, as well as thepotential benefits.
Conclusions：Based on the results of the survey, a strategic model to implement remote auditing wasdeveloped to address existing concerns. The remote auditing model was found as feasible and useful as
conventional on-site audits. Significant benefits were expected by introducing this model.

Key words
remote auditing，questionnaire survey，risk-based approach，Information and Communication Technology(ICT)，data security，data privacy
Rinsho Hyoka（Clinical Evaluation）．2021；49（1）：29−39．

＊ This article is first published on the journal website followed by the print issue after adjusting the number of pages.
− 29 −

臨床評価49 巻 1 号

2021

contract research organizations (CROs) to viewthe current status of audit implementation and to
identify concerns about remote auditing. Surveys
included questions regarding the background andstatus of conducted GCP audits, audit implementation system, problems found during the general
audit and specific questions about remote auditing.
The first online survey was conducted from 30th
May to 14th July 2020. The questionnaires weresent to 18 pharmaceutical companies (9 Japanesecompanies and 9 foreign capital companies) among
the top 10 Japanese companies and foreign capitalcompanies in terms of R & D expenditure in 2019
based on market research information, etc., asaudit sponsors, 520 clinical investigator sites registered in the Massive Network for Clinical Trials(MNCT) as auditees, and 47 CROs registered in
the Japan CRO Association also as auditees. The
second online survey was sent to the same 18 companies from 16th October to 21st October 2020,
focusing on their expected benefits of remote
auditing and implementation status of PMDA
inspection during the COVID-19 outbreak.

1．Introduction
Remote auditing is a method for efficient use of
the resources to achieve reduction in costs andrisks associated with travel for continuous quality
assurance. In recent years, examples of remote
auditing in GCP system audits have been introduced, but full-scale implementation was notreported in the world for investigator site audits(ISAs). The primary reason for this poor implementation could be to avoid uncertainty in data
security associated with remote data verification.
However, in 2020 with the COVID-19 outbreak,academic institutions started to report theirefforts 8〜11) to accept remote monitoring at theirclinical investigator sites. Yet systematic remote
data verification, document review, facility tours,
and interviews have not been performed withremote auditing.

2．Method
In this study, we first conducted a questionnaire
survey asking the status of GCP audits in Japanand their concerns about remote auditing. Basedon the survey results, we examined the feasibility
and usefulness of a new remote auditing method todevelop a model applicable to all audit procedures
using an effective combination of ICT tools.
Elements of data privacy and data security protection were carefully considered in our experiments
when comparing remote auditing with conventional on-site audits to organize identified concerns.

2.2

Verification experiments of remote
auditing
Verification experiments of remote auditing
were conducted at two clinical investigator sites
with experience in clinical trials and with cooperation in this study as shown with the audit plans(Table 1). Prior to conducting the audit, risk assessments were performed at each of the clinical investigator sites with their clinical trial protocols
according to the Clinical Research Audit Procedure
Guide 12) and remotely audited if its overall risk
was seen as low.
In this study, auditor with nine years of experience in more than 55 ISAs and 8 system audits
examined four major operational components of
GCP audits, namely document review, data review,

2.1

Questionnaire surveys on the status of
GCP audits
Questionnaire surveys were conducted in Japan
with audit sponsors, clinical investigator sites, and− 30 −

Clin Eval 49（1）2021

interview, and facility tour referring to the PMDA’ sremote inspection concept 13).
After developing a confidentiality agreement, an
audit in four categories (document review, data
review, interview, and facility tour) was carried out
using the remote auditing method, followed by theconventional on-site auditing method. The numberof specified observations and the thoughts of theauditees were compared between these two auditing methods.
The secured, multi-functional hands-free cloudsystem “T-4PO (e-Sense)” 14, 15), developed by

Rozetta and Tobishima Corporation for the construction industry, was adopted for the document
review, data review, and facility tour during the
remote audit. This system was used with ITC
devices, such as smart glasses, smartphones, andtablets. The usefulness of these devices was verified at each audit. In particular, the “VUZIX M400”
smart glasses were selected with its excellent Wi-Fi
sensitivity, fit, weight, voice recognition, andspeaker function 16). A conceptual diagram of theremote auditing system is as shown in Fig. 1.
In our remote auditing, the clinical investigator

Table 1 Audit plans
Audit No. 01

Date of audit: 08 Sep 2020 and 10 Sep 2020

Title: A randomized controlled trial of best supportive care (BSC) versus photodynamic therapy with
ME2906 and PNL6405PLC for patients with peripheral lung cancer
Principal investigator: Jitsuo Usuda
Audit No. 02

Location: Nippon Medical School Hospital

Date of audit: 17 Sep 2020

Title: Hybrid virtual clinical research in Japanese patients with type 2 diabetes mellitus
Principal investigator: Hirotaka Nagashima

Location: Tokyo Center Clinic

Scope:– Principal Investigator oversight– Informed consent process– Source data verification– IRB approvals and communications– Investigational product handling– Safety reporting– Investigator Site File– Monitoring and Facility

Fig. 1 Conceptual diagram of the remote auditing system− 31 −

臨床評価49 巻 1 号

2021

site and the auditor site were connected online, andthe audit procedures were examined without visiting the sites. An on-site agent was assigned by theaudit sponsor to show trial-related documents anddata to the ICT tools to avoid disruption in thedaily operations of trial staff at the site. The surveillance camera was used at the auditor site tomonitor the auditor from the clinical investigatorsite and the viewpoint of information security
management. For the staff interview during the
remote audit, Zoom was used as a familiar web
conferencing system at the clinical investigatorsite. To ensure security during the interviews, necessary measures were taken such as generating apasscode and locking the meeting room duringuse. The usefulness of three types of ICT devices(smart glasses, smartphone, and tablet) was examined at each audit to see if they met their specifictasks during the remote auditing.

(30%) responded to the survey.
3.1.1 Implementation status of GCP audits
All pharmaceutical companies had experience inperforming the ISAs as audit sponsors. There wasno significant difference in the number of audits
between Japanese and foreign capital companies
in any of the audit categories.
Implementation of GCP audits by the sponsors
was examined in relation to the average number ofannual audits, method of audits, number of auditors per audit, number of days per audit, and actual
number of audits. The average number of audits
per year is shown in Fig. 2. It ranged from 11 to 30,
except for one Japanese company and two foreign
capital companies. All ISAs were performed by
the on-site audit method. In terms of the number ofauditors, all Japanese companies had two to three
employees, while four out of nine foreign capitalcompanies (44%) had one employee. The numberof days per audit was two to three in all Japaneseand foreign capital companies.
3.1.2 Experience of CRO system audits
For CRO system audits by the sponsors, sevenout of nine (78%) Japanese companies responded
that they conducted an average of one to five audits

3．Results
3.1 Response to the questionnaire surveys
Eighteen pharmaceutical companies (100%), 47
clinical investigator sites (19%), and 14 CROs

Fig. 2 Average number of annual ISAs by sponsor− 32 −

Clin Eval 49（1）2021

Fig. 3 Implementation status of CRO system audits by sponsors

per year. Eight out of nine (89%) foreign capitalcompanies conducted the same volume of audits(Fig. 3). Regarding the method of auditing, only
one Japanese company carried out audits by remote
auditing, while the others conducted on-site
audits.
Out of 14 CROs, more than half of them
responded that their audits were conducted on-site,
but 6 companies (43%) experienced a combinationof remote and on-site audits.
3.1.3 Experience of ISAs
Out of 97 clinical investigator sites, 75 sites(77%) experienced ISAs, of which 70 received anaverage of one to five audits a year. All audits wereconducted on-site. Out of the 75 clinical investigator sites, 69 (92%) were visited by two to threeauditors. Fifty-three sites (71%) responded thateach ISA lasted two to three days.
3.1.4 Concerns about remote auditing
All of the audit sponsors (100%), 42 out of 97(43%) clinical investigator sites and 9 out of 15(60%) CROs reported some concerns about remote
auditing (Table 2). Their primary concerns weredata access, data security, audit method, andincrease in their workload.

Table 2 Concerns about remote auditing
Concerns
Any concerns

Audit sponsors
n=18

Clinical Investigator sites
n=97

CROs
n=15

18 (100%)

42 (43%)9 (60%)

Components of concerns（ multiple responses permitted）
Data access

12

11

7

Data security

6

13

1
1

Audit method

510Increase of workload

4

8

-

Limit of remote auditing

-

6

2

General2--

Lack of industry consensus

-

1

-

− 33 −

臨床評価49 巻 1 号

2021

glasses, smartphone, tablet, PC) were adopted by
the auditor, on-site agents, and auditees during theverification experiments. The preferred combination of ICT tools by their users was different
depending on the auditing process. In the processof document review, data review, and facility tour,
the “smart glasses + smart glasses” combination
received the highest evaluation at both sites (Table
3).
The effectiveness of the measures in this remote
auditing was evaluated at the clinical investigatorsites. As a result, all concerns existing prior to theverification experiments were resolved after using
the remote auditing.

3.2 Results of audit experiments
3.2.1 Risk assessment
The risk assessment was made prior to conducting Audit No. 01 and Audit No. 02. As the overallrisk was evaluated as low in both audits, the remote
audit model was applied to all audit procedures(document review, data review, interview, andfacility tour).
3.2.2 Verification experiments in remote
auditing
Audit No. 01 and Audit No. 02 were conductedusing a complete remote auditing method in comparison with conventional on-site audit procedures(Fig. 4).
Three observations (filing of the essential documents, documentation of staff qualifications, andmanagement of investigational product) weremade in Audit No. 01, whereas four observations(version control of the informed consent, agreement with the external laboratory, source documentation, and data reporting in case report form)in Audit No. 02 were identified during the remote
auditing. There were no additional observations at
the subsequent on-site audit procedures in either
Audit No. 01 or No. 02.
Several combinations of ICT tools (smart

3.3 Benefits of remote auditing
We mainly evaluated our remote auditing modelin terms of its feasibility and quality in the verification experiments. The benefits expected upon
the introduction of remote auditing were examinedin cost, delivery (time), travel-related risk, andflexibility by additional questionnaire surveys with
13 audit sponsors. Expectation of cost saving bychanging from conventional on-site auditing toremote auditing is shown in Fig. 5. In ISAs in
Japan, all audit sponsors except for one (8%)

Fig. 4 Use of ICTs for data review− 34 −

Clin Eval 49（1）2021

Table 3 Combination of devices for remote auditing
Conbination of devices for remote auditing（◎：Recommended, 〇：Acceptable, △：Conditionally, ×：Not acceptable）
Procedure

Clinical Investigator site / Auditor site

Comment

Smart glasses / Smart glasses / Smartphone / Smartphone /Smart glasses
PC
Smart glasses
PC

Tablet / PC

Document review◎（◎/◎）〇（◎/〇）〇（〇/◎）〇（〇/〇）△（△/〇）

Depends

Data review◎（◎/◎）△（◎/△）〇（〇/◎）△（〇/△）△（△/△）

Security first

Facility tour◎（◎/◎）〇（◎/〇）△（△/◎）△（△/〇）△（△/〇）

Convenience first

×（×/×）

×（×/◎）

×（〇/×）〇（〇/◎）◎（◎/◎）

Convenience first

1）

Interview 2）

1) Pharmacy, laboratory, archiving room etc.
2) Principal investigator, clinical trial office, IRB office, CRC, IT manager etc.

answered that they could save at least 500,000
Japanese yen (USD 4,776) a year and eight of
those expected to save more than one million
Japanese yen (USD 9,552) a year. In overseas
ISAs, 6 out of 13 audit sponsors expected savingsof at least two million Japanese yen (USD 19,104)a year.
The expected travel time saved by changing
from conventional on-site auditing to remote audit-

ing was more than 15 days/year in five sponsors(38%), 7–15 days/year in four sponsors (31%),
and less than 7 days/year in four sponsors (31%) in
domestic ISAs. In overseas ISAs, almost 70% ofaudit sponsors responded that 7–15 days/year
could be saved by remote auditing.
All audit sponsors responded that remote audits
could also reduce travel-related risks such as infection, natural disasters, and traffic accidents associ-Fig. 5 Expected cost savings by moving to remote auditing in ISAs

− 35 −

臨床評価49 巻 1 号

2021

ated with their business trips. The most anticipatedimprovement in flexibility was “duration of audits”(11) followed by “place of audit” (10), “agenda ofaudit” (9), and “adjustment in audit schedule” (8).

firmed. We believe this success comes from thestrategic approach of this model based on the identified concerns about remote audits in our study
and appropriate measures were incorporated as a
response.
After the completion of a series of experiments,
new challenges were observed for the future implementation of this remote auditing model in the realworld. The biggest challenge will be to assign
appropriate on-site agents. In the case of clinical
trials sponsored by a pharmaceutical company, it
could be relatively easy to solve the problem byassigning the Clinical Research Associates (CRAs)
to deal with that role along with their routine monitoring activity, but in the case of investigator-initiated clinical trials or clinical research, it should
be considered individually.
However, the status of digitalizing clinical trial
procedures at the clinical investigator sites may
affect the feasibility of remote auditing. In recent
years, the introduction of electronic document
management systems for clinical trials has become
popular, but according to our survey conducted in
May 2020, only 27% of Institutional Review
Boards (IRBs) have applied such a system 17). As
the local IRB is the most common type in Japan, it
may suggest a limited level of digitalization anddifficulty in applying remote auditing at the clinical investigator sites.
The electronic document management system is
expected to reduce the number of documents to bereviewed at the clinical investigator sites, which
will lead into more efficient audits regardless ofthe method of auditing (remote or on-site). We
hope that more investigator sites will apply thisremote auditing model to perform continuous quality assurance activities and build on best practices.

4．Discussion
4.1 Implementation status of GCP audits
With regards to the audit sponsors, Japanesecompanies were likely to spend more resources(more auditors per audit) on ISAs than foreign
capital companies. Their audit method was limited
to on-site by visiting the clinical investigator sites
in ISAs. It was also found that complete remote
audits were not performed in CRO system auditsthat did not involve reviewing a subject’s personal
data or medical information. If this remote audit
model is accepted by audit sponsors and auditees,
it will bring a change in quality assurance work.
This will also lead to the realization of continuous
quality assurance activities in the future.
4.2

Verification experiments of remote
auditing
The main features of this remote auditing model
are the utilization of the ICT tools and visualization of the security countermeasures. In this system, documents and data are reviewed and facility
tours are conducted with direct viewing via ICT
tools in cooperation with the on-site agent without
intruding into the system of the clinical investigator site, such as viewing electronic medical records
from outside. Therefore, neither the sponsor nor
the clinical investigator sites need to introduce an
expensive system for the purpose of data verification.
The hypothesis of no difference in the number offindings between remote and on-site auditing wasproven in our study, and the feasibility and usefulness of our remote auditing model were also con− 36 −

Clin Eval 49（1）2021

4.3 Expected benefit of remote auditing
While reducing travel-related risk by introducing a remote audit model was clearly predicted, theexpectation of reducing cost and time was also significant. In addition to those benefits, this model
can be applied not only to GCP audits but also to
any GxP audits, clinical trial monitoring, and regulatory inspections. It is easy to use as this model
does not require procurement of an expensive system or external access of the auditee’s electronicsystems.

5．Conclusion
A strategic model to implement remote auditing
was developed to address the concerns identified
in the questionnaire survey. In verification experiments, our remote auditing model was found toprovide the same level of audit as a conventional
on-site audit in terms of the number of observations. In addition, the security measures taken werereassuring. The difficulties and measures to overcome in the implementation of future remote auditing were made clear, as well as the potential
benefits.

4.4

PMDA inspection under the COVID-19
outbreakIn Japan, the inspection of clinical investigatorsites has been suspended since March 2020, except
for the investigation of sponsors 18〜20). Normally,
PMDA inspection at the sponsor’s office is conducted in approximately six hours. Due to the
COVID-19 outbreak, however, current inspections
are conducted remotely by using web conferencing systems, file sharing technology, document
viewing using cameras, etc. Our survey confirmedthat the total time required for PMDA inspection
was significantly shorter than in the past. In ourstudy, eight companies underwent PMDA inspection between April and October 2020, during
which the common inspection time was one tothree hours. This may be due to the changes in aseries of inspection procedures associated with theremote practice, although there is a concern thatthe quality assurance by PMDA might be affected.
Measures should be established to enable immediate resumption of inspections for clinical investigator sites and to ensure that the inspection system
is appropriate. We hope our remote audit model
can help.

Conflict of Interest
The authors have no conflicts of interest directly relevant to the content of this article.
Acknowledgments
We would like to thank the Massive Network for
Clinical Trials, the Japan CRO Association, Takeda
Pharmaceutical Company Limited, Otsuka Holdings Co.,
Ltd., Astellas Pharma Inc., Daiichi Sankyo Company,
Limited, Eisai Co., Ltd., Sumitomo Dainippon PharmaCo., Ltd., Chugai Pharmaceutical Co., Ltd., Mitsubishi
Tanabe Pharma Corporation, Ono Pharmaceutical Co.,
Ltd., Shionogi Co., Ltd., Merck Biopharma Co., Ltd.,
Novartis Pharma K.K., Janssen Pharmaceutical K.K.,
Pfizer Japan Inc., Sanofi K.K., AstraZeneca K.K., BMS
K.K., Bayer Yakuhin, Ltd., A2 Healthcare Corporation,
accerise, Inc., CAC Croit Corporation, Clinipace K.K.,
CMIC Co., Ltd., DOT WORLD CO., LTD., EPS
Corporation, EPS International Holdings Co., Ltd.,InCROM CRO Inc., intellim Corporation, IQVIA Services
Japan K.K., PAREXEL International Inc., PRA Health
Sciences K.K., SRD Co., Ltd., Syneos Health Clinical
K.K., Nippon Medical School, Tokyo Center Clinic,
Rozetta Corp., and Proactive Consulting Inc.

− 37 −

臨床評価49 巻 1 号

2021

Clinical Sites for Sponsor’s Preparation for FDA/EMA Inspection. 2014. Available from: https://www.
pharmasug.org/proceedings/2014/BB/PharmaSUG2014-BB18.pdf
8) Nami Hashimoto, et al. Effort of introducing remote
SDV. Poster presentation No. 119 at the conference
on CRC and clinical trials 2020 in Nagasaki, 3-16
Nov 2020.
9) Kazue Tanabe, et al, Implementation Structure ofClinical Trial and effort to remote SDV outbreak.
Poster presentation No. 121 at the conference on
CRC and clinical trials 2020 in Nagasaki, 3-16 Nov
2020.
10) Hiroto Okuda, et al. Development of a remote SDV
Environment using Web conferencing system Response of Medical Institutions against to COVID19 outbreak. Poster presentation No. 126 at theconference on CRC and clinical trials 2020 in
Nagasaki, 3-16 Nov 2020.
11) Mayumi Shirase, et al. Implementation of remote
SDV - competitive system with COVID-19. Poster
presentation No. 132 at the conference on CRC andclinical trials 2020 in Nagasaki, 3-16 Nov 2020.
12) Japan Society of Quality Assurance. Guidance ofaudit do clinical research. Mar 2018. Japanese.
Available from: https://www.jsqa.com/wp2/wp-content/
uploads/2018/03/03_Audit-procedures-guide_
Clinical-research_JSQA-C4B_20180323.pdf
13) Pharmaceuticals and Medical Devices Agency.
Immediate action for implementation of GCP/GPSP
inspection under CIVID-19. Dec 2020, revised Jan
2021. Japanese. Available from: https://www.pmda.
go.jp/files/000235011.pdf
14) Rozetta. T-4PO e-Sense. Rozetta in-house document.
15) Rozetta. Handling of confidential information of
T-4PO. Rozetta in-house document.
16) Rozetta. Specification comparison of Smart glasses.
Rozetta in-house document.
17) Naoki Tsutsumi, et al. Utilization of Information andCommunication Technology in IRBs as a countermeasure against the COVID-19 outbreak and futurechallenges. Rinsho Hyoka (Clinical Evaluation).
2020; 48(2): 355-66. Available from: http://cont.o.

References
1) U.S. Department of Health and Human Services
Food and Drug Administration. Guidance forIndustry. Oversight of Clinical Investigations ̶ A
Risk-Based Approach to Monitoring. Aug 2013.
Available from: https://www.fda.gov/media/116754/
download2) Office of Pharmaceutical Quality. Understanding
CDER’s Risk-Based Site Selection Model (CDER
Manual of Policies and Procedures (MAPP) 5014.1).
Sep 26 2018. Available from: https://www.fda.gov/
media/116004/download
3) Medicines and Healthcare products Regulatory
Agency. Guidance for GLP facilities on the implementation and maintenance of a risk based Quality
Assurance programme. Sep 2015. Available from:https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/
file/464086/GLP_QA_programme.pdf
4) Medicines and Healthcare products Regulatory
Agency. Guidance: Good pharmacovigilance practice (GPvP). Comply with good pharmacovigilance
practice and prepare for an inspection. Dec 18 2014,
Last updated Jan 11 2021. Available from: https://www.gov.uk/guidance/good-pharmacovigilancepractice-gpvp5) European Medicines Agency. Reflection paper onrisk based quality management in clinical trials. Nov
18 2013. Available from: https://www.ema.europa.
eu/en/documents/scientific-guideline/reflectionpaper-risk-based-quality-management-clinical-trials_
en.pdf6) Medicines and Healthcare products Regulatory
Agency. Guidance: Good clinical practice for clinical
trials. How to show MHRA you’re meeting goodclinical practice (GCP) standards and what to expectfrom an inspection. Dec 18 2014, Last updated Jul 31
2020. Available from: https://www.gov.uk/guidance/
good-clinical-practice-for-clinical-trials#inspectionsunder-the-risk-based-compliance-programme7) Xiangchen (Bob) Cui. PharmaSUG 2014 ̶ BB18:
Risk-Based Approach to Identifying and Selecting− 38 −

Clin Eval 49（1）2021

oo7.jp/48_2/p355-66.pdf
18) Pharmaceuticals and Medical Devices Agency. Immediate action for inspection. Feb 2020. Japanese. Available from: https://www.pmda.go.jp/files/000237851.
pdf19) Pharmaceuticals and Medical Devices Agency.
Implementation of remote inspection of pharmaceuticals and regenerative medicine products. Nov 16
2020. Japanese. Available from: https://www.pmda.
go.jp/files/000237602.pdf＊

20) Pharmaceuticals and Medical Devices Agency.
Document specified in paragraph 5 of “Implementation of remote inspection of pharmaceuticals andregenerative medicine products”. Dec 1 2020.
Japanese. Available from: https://www.pmda.go.jp/
files/000237852.pdf(Submitted December 15, 2020)(Accepted February 1, 2021)(Published March 3, 2021)＊− 39 −＊