Coordination matters:
Puzzles of science, technology, andinnovation policy implementation
Tohru Yoshioka-Kobayashi (Tokyo Foundation for Policy Research /Hitotsubashi University)

Akio Kurokawa (Tokyo Foundation for Policy Research / Kanagawa Universityof Human Services)An ideal causal model of science/technology-led
innovation policy
Innovation policy
with well-defined
goal

Implementation
by regulations or
funding programs

Promotion ofresearch anddevelopment

Commitment ofrecipients
Innovation
implementation

Social value
creation frominnovation@ The Tokyo Foundation for Policy Research
All rights reserved.

A puzzle of innovation policy implementation:
A level of targeting
Concentrated model

Portfolio management model

MissionMissionSpecific strategy

Project

Relevant project

Project

Project• Concentrated investment (bigger
resource allocation)

Pros• Minimization of uncertainty risk

• Necessity of coordination of(or selection from) interest groups• Risks of complete failureCons• Potential conflicts among projects• Redundant investment@ The Tokyo Foundation for Policy Research
All rights reserved.

Another puzzle of innovation policy implementation:
A key actor of coordination/portfolio management
Top-down approach

Agency approach

Private-led approach

Politicians

Ministries

Ministries

Ministries/
Departments

Government
agencies

Firms/NGOs

Pros• Bigger resource
allocation• Legitimacy• Certain expertise• Expertise• FlexibilityCons• Transparency• Expertise• (Fixed design)• Transparency• Legitimacy• Amount of(or anti-trust issue)
resource allocation • Conflicts of(or political power)
strategy@ The Tokyo Foundation for Policy Research
All rights reserved.

Challenges of innovation policy implementationby combinations of models and approaches
Regulation

Funding

Consortium

Advocacy group /industry
association

Firm

Firm

Top-down approach

Agency approach

Private-led approach

Concentrated model

Selection/coordination
capability

Transparency

Selection/coordination
capability of key actor

Portfolio
management model

Portfolio management
capability

Divided investments (Legitimacy of

Selfregulations

Fund manager
Selfregulations

Regulation

portfolio manager) or(government's
selection capability)

Project

Funding

Project@ The Tokyo Foundation for Policy Research
All rights reserved.

An underlying issue in selection/coordination(from multi-disciplinary perspectives)• Objections to high uncertainty / highly technical challenge• ...often leads compromise (i.e., safe and acceptable goal)(Takeishi et al., 2010: Innovation study)• ...are raised by taxpayers and/or auditing agencies (Lerner, 2012:
Public policy study)• Peer pressures under collective decision making process• ...often leads acceptable, well-known, and myopic targes (Stephen &Zubcsek, 2015; Creativity study)
Takeishi, A., Aoshima, Y., & Karube, M. (2010). Reasons for innovation: Legitimizing resource mobilization for innovation in the cases of the Okochi Memorial
Prize Winners. In Dynamics of knowledge, corporate systems and innovation (pp. 165-189). Springer, Berlin, Heidelberg.; Lerner, J. (2012). Boulevard of
Broken Dreams: Why Public Efforts to Boost Entrepreneurship and Venture Capital Have Failed and What to Do About It.; Stephen, A. T., & Zubcsek, P. P.
(2015). People offer better ideas when they can’t see what others suggest. Harvard Business Review, 24.;

@ The Tokyo Foundation for Policy Research
All rights reserved.

Another underlying issue in selection/coordination:
Conflict of business strategies• One goal with multiple paths• Example: Autonomous driving
Intelligent
transportation systemapproach

Cloud computing
approach

Edge computing
approach...requires large infrastructure
investments...requires secure and fast
wireless telecommunication
technology (e.g., 5G)...requires high performancebattery and semiconductor,
and smart AI software.

(Source of illustrations) Freepik from flaticon@ The Tokyo Foundation for Policy Research
All rights reserved.

Consequences of two underlying issues
Concentrated model

Portfolio management model

Safe and
acceptable goal

Hard tocoordinate

Lack of quality
identification
capability

Dividedinvestment

Red ocean(as a business
goal)

Weak
commitment

Inefficiency

Insufficientinvestment(Source of illustrations) Freepik and surang from flaticon

These issues can be resolved by adevelopment of quality
identification capability@ The Tokyo Foundation for Policy Research
All rights reserved.

Learnings from Japanese national project in 1970s/80s• They took hierarchic portfolio model
Tier 1 Consortium
Firm

State-owned Firm

Tier 2 Consortium
Firm

Firm

Firm

Tier 2 Consortium
Firm

Firm

At the later phase of the project,
the government shifted its targets
to those in Tier 2 consortium
Shimamoto, M. (2014). Keikaku no Sohatsu: Sunshine keikaku to taiyoukou hatsuden [Co-creation of Innovation Project: Insights from Solar Energy R&D in
Sunshine Project]. Yuhikaku.
@ The Tokyo Foundation for Policy Research
All rights reserved.

Questions for discussion• How do the private sector coordinate their strategic behavior
in the collaborative regulation design or R&D activities?
• Trade-off: Strategic isomorphism (with competitors)
vs Decreased necessity for coordination
Poor competitiveness potential as a business• How do both the government and private sector maintainlegitimacy of private sector-involved regulation or innovation
program?
• Trade-off : Flexibility vs legitimacy@ The Tokyo Foundation for Policy Research
All rights reserved.

Case Study: AI medical device• Medical devices using AI technology have unique characteristics such as theblack box nature of the deep learning algorithm and the possibility of
constant performance changes due to automatic learning.
• The following issues need to be addressed during the regulatory approval
review:i.

How to verify performance on an ongoing basis
The performance of AI medical devices must be specified and verifiable in a medically and statistically valid manner.

ii.

Quality assurance in the event of performance changes
It is necessary to specify the factors that affect the performance of AI as a medical device, as well as the range andlimitations within which effects may be confirmed.

iii.

Approaches to regulatory procedures
The issue of application to medical devices; if that's outside the current definition of a medical device, how should therules be changed?

iv.

Responsibility
In the event of a serious accident, is it the responsibility of the medical practitioner, equipment manufacturer or
program designer?
@ The Tokyo Foundation for Policy Research
All rights reserved.

Features of the introduction of regulations on
AI medical devices
The most fundamental Question
How to regulate product groups with new functions and capabilities?
• The process of introducing regulations for AI medical devices was roughly as follows.
i. Regulators attempted to apply the existing regulatory framework, SaMD
(Software as Medical Device), to AI medical devices.
ii. However, it was found that it could not be properly evaluated due to functions
beyond what was expected for a programmed medical device.
iii. Therefore, the regulators decided to apply the rules of the current system for thetime being while developing new rules.
• In the absence of appropriate regulatory measures, there is no alternative but to takeprovisional measures in the form of applying the existing rules.
• The result was that any SaMD with even a slight change in functionality would
have to be approved as a new and different product.
• Even if the products concerned were to enter the market, it would be practically
impossible to approve them.
@ The Tokyo Foundation for Policy Research
All rights reserved.

The process of introducing new regulations in Japan
Major Events
Oct 2015

PMDA restructures its consultation and review system for medical devices in light of technological innovations in robotics,
ICT, etc.

Mar 2016

Ministry of Health, Labour and Welfare (MHLW) published guidance on issues related to the review of medical device
programs that support diagnosis.

Dec 2016

Ministry of Health, Labour and Welfare (MHLW) helds opinion exchange meetings with diagnostic imaging device
manufacturers and others regarding the use of AI.

Jun 2017

The Ministry of Health, Labour and Welfare (MHLW) released a report on the results of a roundtable discussion on
promoting the use of AI in the healthcare sector.

Dec 2017

PMDA's Scientific Committee AI Subcommittee compiled "Issues and Recommendations on AI-Based Medical Diagnosis
Systems and Medical Devices 2017

May 2018

The report of the PMDA's Scientific Committee on AI was also published in an English-language academic journal.
Chinzei K, Shimizu A, Mori K, Harada K, Takeda H, Hashizume M et al. Regulatory Science on AI-based Medical Devices
and Systems. Adv Biomed Eng. 2018; 7: 118—123.

Dec 2018

Ministry of Health, Labour and Welfare (MHLW) issued a notice on the relationship between the use of programs to
support diagnosis and treatment using artificial intelligence (AI) and the provisions of Article 17 of the Medical
Practitioners Act.

Mar 2019

The Ministry of Health, Labour and Welfare (MHLW) published a report on next-generation evaluation indicators for
medical imaging diagnostic support systems using artificial intelligence technology.

Sep 2020

The Pharmaceutical and Medical Devices Law was amended to introduce a new system for approval of medical devices
using AI technology according to their characteristics.
@ The Tokyo Foundation for Policy Research
All rights reserved.

The process of introducing new regulations in Japan
Major Events
Oct 2015

PMDA restructures its consultation and review system for medical devices in light of technological innovations in robotics,
ICT, etc.

Mar 2016

Ministry of Health, Labour and Welfare (MHLW) published guidance on issues related to the review of medical device
programs that support diagnosis.

Dec 2016

Ministry of Health, Labour and Welfare (MHLW) helds opinion exchange meetings with diagnostic imaging device
manufacturers and others regarding the use of AI.

Jun 2017

The Ministry of Health, Labour and Welfare (MHLW) released a report on the results of a roundtable discussion on
promoting the use of AI in the healthcare sector.

Dec 2017

PMDA's Scientific Committee AI Subcommittee compiled "Issues and Recommendations on AI-Based Medical
Diagnosis Systems and Medical Devices 2017

May 2018

The report of the PMDA's Scientific Committee on AI was also published in an English-language academic journal.
Chinzei K, Shimizu A, Mori K, Harada K, Takeda H, Hashizume M et al. Regulatory Science on AI-based Medical Devices
and Systems. Adv Biomed Eng. 2018; 7: 118—123.

Dec 2018

Ministry of Health, Labour and Welfare (MHLW) issued a notice on the relationship between the use of programs to
support diagnosis and treatment using artificial intelligence (AI) and the provisions of Article 17 of the Medical
Practitioners Act.

Mar 2019

The Ministry of Health, Labour and Welfare (MHLW) published a report on next-generation evaluation indicators for
medical imaging diagnostic support systems using artificial intelligence technology.

Sep 2020

The Pharmaceutical and Medical Device Act was amended to introduce a new system for approval of medical
devices using AI technology according to their characteristics.
@ The Tokyo Foundation for Policy Research
All rights reserved.

A major turning point• In December 2017, the AI Expert Committee of the Scientific Committee
established by the Pharmaceuticals and Medical Devices Agency (PMDA)
compiled the "Issues and Recommendations on AI-based Medical
Diagnostic Systems and Medical Devices 2017" (PMDA Report).
• The PMDA report outlined the challenges in applying AI to
medical devices from the perspective of regulatory science.
i.

Necessity of evaluation methods and systems in line withthe characteristics of AI

ii. Necessity of considering evaluation index for diagnostic
imaging devices using AI technology• With this report as a starting point, concrete progress wasmade in the formation of rules for AI medical devices in Japan.
@ The Tokyo Foundation for Policy Research
All rights reserved.

Revision of the Pharmaceuticaland Medical Device Act• An approval system has been introduced to address the characteristics of AI medical
devices, whose performance is constantly changing.
• There is no longer a need to obtain a new approval as a separate product when thefunctions change.
• Change plan confirmation procedure system: advance notification system for
performance changes• AI medical devices can be subject to confirmation by the Minister of Health, Labor andWelfare regarding the plan for changes in performance, manufacturing method, etc.,
among the approved items (how to collect and manage data and learning in the futurewhen first approved, etc.), making it possible to submit a plan for the part ofperformance and process that is expected to improve the device in advance.
• In the case of an application for approval of a change in accordance with the confirmed
plan, the review will include an investigation into whether the change is in accordancewith the plan (whether the device has sufficient performance after being improved inaccordance with the plan submitted in advance), instead of an investigation into
quality, efficacy, or safety.
• This will shorten the review period from about six months to about one to two months.
ログラム医療機器としての想定を超える機能を評価できない（継続的な性能変化@ The Tokyo Foundation for Policy Research
All rights reserved.

Trends in the United States• Similar to Japan, the U.S. Food and Drug Administration (FDA) also took a lot oftime to establish and institutionalize a method for evaluating medical devices with
continuous performance changes.
•

The FDA divided AI medical devices into two categories, "adaptive" and "locked",defining the former as those that continuously collect, analyze, and learn data throughuse of the device, and whose algorithms constantly adapt and change to theenvironment, and the latter as those that can lock in the algorithms learned by the AI,
so that once the algorithms are locked in place, their functions do not change.

• The FDA's regulation took a major step forward with the release of a discussion paper
titled "Proposed Regulatory Framework for Modifications to Artificial
Intelligence/Machine Learning-Based Software as a Medical Device" in April
2019. In this paper, the FDA proposed a basic regulatory framework for "adaptive" AI
medical devices.
• In February 2020, the FDA approved the marketing of an adaptive product that had
been submitted under the new "Predetermined Change Control Plan," after
coordinating with stakeholders through a series of public workshops.
• In other words, all AI medical device products approved before February 2020 were
"locked".医療機器としての想定を超える機能を評価できない（継続的な性能変化
Source: FDA, “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning-Based Software as a Medical Device” available athttps://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-Learning-Discussion-Paper.pdf@ The Tokyo Foundation for Policy Research
All rights reserved.

Trends in the United States
FDA’s new regulatory approach• The FDA proposed a new quality control concept called "Good Machine Learning
Practice (GMLP)" for the introduction of AI medical device regulations.
• The GMLP is a policy that requires a high level ofquality control on the part of development andmanufacturing companies at each stage of the product(Data selection and management, model training andtuning, model validation, etc.) in order to ensure highquality in compliance with the regulations.
• This approach is quite unique and qualitatively
different from that in Japan, as it includes not only theperformance of the device itself, but also the quality
control aspects of the product lifecycle, such as theappropriateness of the data used and thereasonableness of program updates.
Source: FDA, “Proposed Regulatory Framework for Modifications to Artificial

Intelligence/Machine Learning-Based Software as a Medical Device” available athttps://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-andMachine-Learning-Discussion-Paper.pdf@ The Tokyo Foundation for Policy Research
All rights reserved.

Findings from the case study:（Ⅰ）Introduction of regulations in each country and international harmonization• Difficulties in introducing regulations for emerging technologies in a timely manner• In both Japan and the US, it took a lot of time to introduce regulations for AI medical
devices.

• Domestic review process in each country• As for new regulations for emerging technologies, it seems that each country has
established its own domestic rules while checking international trends.
• Therefore, it seems that the regulatory trends in the US did not directly trigger theformation of rules in Japan.

• Future development of harmonization• Harmonization is expected to proceed after the development of domestic rules in eachcountry has been completed.
• The FDA is already working with regulatory authorities in several countries toaccelerate the standardization of GMLP.
• January 2021, FDA releases action plan for AI and machine learning based SaMD.
• In October 2021, the U.S. Food and Drug Administration (FDA), Health Canada, and the U.K.'s
Medicines and Healthcare products Regulatory Agency (MHRA) issued new guidance, “Good
Machine Learning Practice for Medical Device Development: Guiding Principles”.
@ The Tokyo Foundation for Policy Research
All rights reserved.

Findings from the case study:（Ⅱ） Regulatory lag and lack of technology forecasting for regulation• In Japan, unlike other technological fields, funding and regulation of R&D in the
medical field has not been planned and implemented based on proper technologicalforecasting.
• The PMDA report was the trigger, and the research for regulations on AI medical
devices was not started early on.

• The delay in the introduction of regulations for emerging technologies such as AI
medical devices is a problem of “regulatory lag”.
• In order to avoid delays in regulations for emerging technologies, it is necessary toaccurately forecast technology trends and invest in R&D related to evaluation
technologies based on the forecasts at the right time.
• In September 2019, the PMDA established the Horizon Scanning Implementation
Guidelines. The PMDA has introduced horizon scanning on a trial basis.
• AMED, Japan's version of NIH, plans to establish a new strategic division(SCARDA)
that will be responsible for strategic budget allocation.
@ The Tokyo Foundation for Policy Research
All rights reserved.

Findings from the case study:（Ⅲ） Limitations of the regulator's capability• Limitations in the ability of the MHLW
• In designing new regulations for emerging technologies, it is difficult for even the
MHLW's technical officers to fully understand the technical characteristics and planevaluation methods for a highly specialized set of knowledge and technical information,
including AI.

• Limitations of street-level capacity of agencies• It is not possible to evaluate emerging technologies simply by applying existing
frameworks, standards, and procedures to them. They can't deal with anything that
isn't clearly defined in the rules.

• Dependence on external resources• The PMDA, which is only an agency (the entity that implements regulations, not theentity that plans them), is doing the actual agenda-setting for regulations.
• PMDA itself does not have research capacity, so it mobilizes experts as external
resources in the form of scientific committees.
*The National Institute of Health Sciences is primarily responsible for research on regulatory science.

@ The Tokyo Foundation for Policy Research
All rights reserved.

Findings from the case study:（Ⅳ） Low commitment from the private sector• Rule making led by academia• " Academia" such as the PMDA Scientific Committee played a major role in theregulation review process. The private sector's commitment to the regulatory review
process was extremely low.

• Potential asymmetry of information• There should be a clear information asymmetry between the regulation review sideand the development side. It is not "academia" that develops the actual products andservices related to AI, but private companies that own the products.

• Consideration of regulations through industry-academia-government collaboration• In the process of developing regulations related to regenerative medicine, Forum for
Innovative Regenerative Medicine（FIRM） and The Japanese Society for Regenerative
Medicine（JSRM） participated in the collaborative design of regulations, but in thecase of AI medical devices, the process was very different.
• Now is the time for industry, government, and academia to seriously consider ways todesign regulations in a more coordinated manner. Such an approach also needs to be
introduced on a trial basis and reviewed as necessary.
@ The Tokyo Foundation for Policy Research
All rights reserved.

Corporate Participationin the Regulatory Planning Process• There are a number of concerns regarding the participation of companies in the regulatory
drafting process.
• Intellectual property confidentiality needs to be protected in the provision of
information and knowledge about emerging technologies from companies. Otherwise,companies will not be able to participate.
• On the other hand, it is also essential to consider how to control the possibility thatcompanies will try to design regulations in their favor. For example, there is apotential incentive on the part of companies to reduce the level of evidence required toassess safety and efficacy in their favor.
• Even with the participation of corporations, legitimacy is required in the design andimplementation of regulations.
• To ensure legitimacy, a transparent process and evidence-based review under
government initiative is needed.
• In order to go beyond the knowledge and position of one company, it may be necessary
to adjust horizontally as an "industry".
• The challenge will be how to design regulations in a transparent process while protecting
the intellectual property of companies.
@ The Tokyo Foundation for Policy Research
All rights reserved.