Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Marketing Submission 
Recommendations for a 
Predetermined Change Control Plan 
for Artificial Intelligence/Machine 
Learning (AI/ML)-Enabled Device 
Software Functions
 
______________________________________________________________________________ 
Draft Guidance for Industry and 
Food and Drug Administration Staff 
 
DRAFT GUIDANCE 
This draft guidance document is being distributed for comment purposes 
only. 
Document issued on April 3, 2023. 
 
You should submit comments and suggestions regarding this draft document within 90 days of 
publication in the Federal Register of the notice announcing the availability of the draft 
guidance. Submit electronic comments to https://www.regulations.gov. Submit written 
comments to the Dockets Management Staff, Food and Drug Administration, 5630 Fishers Lane, 
Room 1061, (HFA-305), Rockville, MD 20852. Identify all comments with the docket number 
listed in the notice of availability that publishes in the Federal Register. 
 
For questions about this document regarding CDRH-regulated devices, contact 
digitalhealth@fda.hhs.gov. For questions about this document regarding CBER-regulated 
devices, contact ocod@fda.hhs.gov. For questions about this document regarding CDER-
regulated products, contact druginfo@fda.hhs.gov. For questions about this document regarding 
combination products, contact the Office of Combination Products at combination@fda.gov. 
 
U.S. Department of Health and Human Services 
Food and Drug Administration 
Center for Devices and Radiological Health  
Center for Biologics Evaluation and Research 
Center for Drug Evaluation and Research 
Office of Combination Products in the Office of the Commissioner  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Preface 
 
Additional Copies 
 
CDRH 
 
Additional copies are available from the Internet. You may also send an email request to CDRH-
Guidance@fda.hhs.gov to receive a copy of the draft guidance. Please include the document 
number GUI00020049 and complete title of the guidance in the request. 
 
CBER 
 
Additional copies are available from the Center for Biologics Evaluation and Research (CBER), 
Office of Communication, Outreach, and Development (OCOD), 10903 New Hampshire Ave., 
Bldg. 71, Room 3128, Silver Spring, MD 20993-0002, or by calling 1-800-835-4709 or 240-402-
8010, by email, ocod@fda.hhs.gov, or from the Internet at https://www.fda.gov/vaccines-blood-
biologics/guidance-compliance-regulatory-information-biologics/biologics-guidances. 
 
CDER 
 
Additional copies are available from the Center for Drug Evaluation and Research, Office of 
Communications, Division of Drug Information, 10001 New Hampshire Ave., Hillandale Bldg., 
4th Floor, Silver Spring, MD 20993-0002, or by calling 855-543-3784 or 301-796-3400, or by 
email, druginfo@fda.hhs.gov, or from the Internet at https://www.fda.gov/drugs/guidance-
compliance-regulatory-information/guidances-drugs. 
 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Table of Contents
 
I.  Introduction .......................................................................................................................... 1 
 
II.  Background .......................................................................................................................... 2 
III.  Scope .................................................................................................................................... 5 
IV.  Definitions............................................................................................................................ 8 
A.  Software Functions ........................................................................................................... 8 
B.  Data Sets ........................................................................................................................... 8 
C.  Predetermined Change Control Plan ................................................................................ 9 
V.  Policy for Predetermined Change Control Plans ................................................................. 9 
A.  Components of a PCCP .................................................................................................. 10 
B.  Establishing a PCCP ...................................................................................................... 11 
C.  Identifying a PCCP in a Marketing Submission ............................................................ 12 
D.  Utilizing an Authorized PCCP to Implement Device Modifications ............................. 13 
E.  Modifying a PCCP for an Authorized Device ............................................................... 15 
VI.  Description of Modifications ............................................................................................. 16 
A.  Goals of the Description of Modifications Section ........................................................ 16 
B.  Content of the Description of Modifications Section..................................................... 16 
C.  Types of Modifications .................................................................................................. 17 
VII.  Modification Protocol ........................................................................................................ 18 
A.  Goals of the Modification Protocol Section ................................................................... 18 
B.  Content of the Modification Protocol Section................................................................ 20 
(1) Data management practices ........................................................................................ 20 
(2) Re-training practices ................................................................................................... 21 
(3) Performance evaluation .............................................................................................. 22 
(4) Update procedures ...................................................................................................... 22 
C.  Traceability Between the Description of Modifications Section and the Modification 
Protocol Section ........................................................................................................................ 23 
VIII.  Impact Assessment............................................................................................................. 24 
Appendix A: Example Elements of Modification Protocol Components for ML-DSFs .............. 26 
(1) Data Management ....................................................................................................... 26 
(2) Re-Training ................................................................................................................. 28 
(3) Performance Evaluation ............................................................................................. 29  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
(4) Update Procedures ...................................................................................................... 30 
Appendix B: Example ML-DSF Scenarios Employing PCCPs ................................................... 33 
(1) Patient Monitor Software............................................................................................ 33 
(2) Skin Lesion Software.................................................................................................. 35 
(3) Ventilator Settings Software ....................................................................................... 36 
(4) Image Acquisition Assistance Device ........................................................................ 37 
(5) Feeding Tube Placement Radiograph Analysis Software .......................................... 38 
 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Marketing Submission 
1 
Recommendations for a 
2 
Predetermined Change Control Plan 
3 
for Artificial Intelligence/Machine 
4 
Learning (AI/ML)-Enabled Device 
5 
Software Functions 
6 
7  ______________________________________________________________________________ 
Draft Guidance for Industry and 
8 
Food and Drug Administration Staff 
9 
10   
11  This draft guidance, when finalized, will represent the current thinking of the Food and Drug 
12  Administration (FDA or Agency) on this topic. It does not establish any rights for any person 
13  and is not binding on FDA or the public. You can use an alternative approach if it satisfies the 
14  requirements of the applicable statutes and regulations. To discuss an alternative approach, 
15  contact the FDA staff or Office responsible for this guidance as listed on the title page.  
16   
17  I.  Introduction 
18  FDA has a longstanding commitment to develop and apply innovative approaches to the 
19  regulation of medical device software and other digital health technologies to ensure their safety 
20  and effectiveness.1 As technology continues to advance all facets of healthcare, medical software 
21  incorporating artificial intelligence (AI), and specifically the subset of AI known as machine 
22  learning (ML) (henceforth referred to as machine learning-enabled device software functions or 
23  ML-DSFs), has become an important part of many medical devices. This draft guidance is 
 
1 FDA regulates software that meets the definition of a device, which is defined in section 201(h)(1) of the Federal 
Food, Drug, and Cosmetic Act (FD&C Act) as an instrument, apparatus, implement, machine, contrivance, 
implant, in vitro reagent, or other similar or related article, including any component, part, or accessory, which is  
recognized in the official National Formulary, or the United States Pharmacopoeia, or any supplement to them, 
intended for use in the diagnosis of disease or other conditions, or in the cure, mitigation, treatment, or prevention of 
disease, in man or other animals, or intended to affect the structure or any function of the body of man or other 
animals, and which does not achieve its primary intended purposes through chemical action within or on the body of 
man or other animals and which is not dependent upon being metabolized for the achievement of its primary 
intended purposes. The term device does not include software functions excluded pursuant to section 520(o) of 
the FD&C Act. 
1  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
24  intended to provide a forward-thinking approach to promote the development of safe and 
25  effective medical devices that use ML models trained by ML algorithms. 
26   
27  ML-enabled technologies have the potential to transform healthcare by deriving new and 
28  important insights from the vast amount of data generated during the delivery of healthcare every 
29  day. Medical device manufacturers2 are using ML technologies to innovate their products to 
30  better assist healthcare providers and improve patient care. Examples of ML applications in 
31  medicine include earlier disease detection and diagnosis, development of personalized 
32  diagnostics and therapeutics, and development of assistive functions to improve the use of 
33  devices with the goal of improving user and patient experience.  
34   
35  FDA recognizes that the development of ML-DSFs is an iterative process. This draft guidance 
36  proposes a least burdensome approach to support iterative improvement through modifications to 
37  an ML-DSF while continuing to provide a reasonable assurance of device safety and 
38  effectiveness. As such, this draft guidance demonstrates FDAs broader commitment to 
39  developing innovative approaches to the regulation of device software functions as a whole. 
40  Specifically, this draft guidance provides recommendations on the information to be included in 
41  a Predetermined Change Control Plan (PCCP) provided in a marketing submission for an ML-
42  DSF. This draft guidance recommends that a PCCP describe the planned ML-DSF 
43  modifications; the associated methodology to develop, implement, and validate3 those 
44  modifications; and an assessment of the impact of those modifications. The PCCP is reviewed as 
45  part of a marketing submission to ensure the continued safety and effectiveness of the device 
46  without necessitating additional marketing submissions for implementing each modification 
47  described in the PCCP. 
48   
49  In general, FDAs guidance documents do not establish legally enforceable responsibilities. 
50  Instead, guidances describe the Agencys current thinking on a topic and should be viewed only 
51  as recommendations, unless specific regulatory or statutory requirements are cited. The use of 
52  the word should in Agency guidances means that something is suggested or recommended, but 
53  not required. 
54   
55  II.  Background 
56  In April 2019, FDA published the Proposed Regulatory Framework for Modifications to 
57  Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) 
 
2 For the purposes of this guidance, manufacturer is used in accordance with the definitions of manufacturer in 21 
CFR Parts 803, 806, 807, and 820 and as described in FDAs guidance Policy for Device Software Functions and 
Mobile Medical Applications (DSF-MMA) available at https://www.fda.gov/regulatory-information/search-fda-
guidance-documents/policy-device-software-functions-and-mobile-medical-applications. 
3 For the purposes of this guidance, the term validation is being used as it is defined in 21 CFR 820.3(z), which 
states [v]alidation means confirmation by examination and provision of objective evidence that the particular 
requirements for a specific intended use can be consistently fulfilled. See Section IV. for more information on 
definitions used for the purposes of this guidance. 
2  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
58  - Discussion Paper and Request for Feedback (2019 discussion paper).4 The 2019 discussion 
59  paper describes FDAs foundation for a potential approach to premarket review for AI/ML-
60  driven software modifications. The ideas delineated in the 2019 discussion paper leveraged 
61  practices from our current premarket programs and relied on the International Medical Device 
62  Regulators Forums risk categorization principles,5 the FDAs benefit-risk framework,6 risk 
63  management principles described in the Software Modifications guidance,7 and the total product 
64  lifecycle approach envisioned in the Digital Health Software Precertification (Pre-Cert) Pilot 
65  Program.8 
66   
67  ML can allow software to learn through data, without being explicitly programmed, to perform a 
68  task. One of the greatest potential benefits of ML resides in the ability to improve ML model 
69  performance through iterative modifications, including by learning from real-world data. To 
70  support the iterative development of ML-DSFs, and as part of the proposed framework presented 
71  in the 2019 discussion paper, FDA described a Predetermined Change Control Plan that could 
72  be included in a marketing submission for a device that is or includes an ML-DSF (referred to 
73  interchangeably as an ML-DSF or a device).9 In this draft guidance, we provide 
74  recommendations on the marketing submission content for a PCCP, which generally includes: 1) 
75  a detailed description of the specific, planned device modifications; 2) the associated 
76  methodology to develop, validate, and implement those modifications in a manner that ensures 
77  the continued safety and effectiveness of the device across relevant patient populations, referred 
78  to as the Modification Protocol; and 3) an Impact Assessment to describe the assessment of the 
79  benefits and risks of the planned modifications and risk mitigations. 
80   
81  The 2019 discussion paper received a substantial amount of feedback from a wide array of 
82  stakeholders. General comments were received, as well as specific responses to 18 questions 
 
4 Available at https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-
machine-learning-software-medical-device. 
5 See FDAs website on Global Approach to Software as a Medical Device available at 
https://www.fda.gov/medical-devices/software-medical-device-samd/global-approach-software-medical-device and 
IMDRFs SaMD Risk Categorization Framework available at 
http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-
141013.pdf. 
6 See FDAs guidance Factors to Consider When Making Benefit-Risk Determinations in Medical Device 
Premarket Approval and De Novo Classifications available at https://www.fda.gov/regulatory-information/search-
fda-guidance-documents/factors-consider-when-making-benefit-risk-determinations-medical-device-premarket-
approval-and-de and FDAs guidance Benefit-Risk Factors to Consider When Determining Substantial Equivalence 
in Premarket Notifications (510(k)) with Different Technological Characteristics available at 
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/benefit-risk-factors-consider-when-
determining-substantial-equivalence-premarket-notifications-510k. 
7 See FDAs guidance Deciding When to Submit a 510(k) for a Software Change to an Existing Device available 
at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/deciding-when-submit-510k-
software-change-existing-device. 
8 See FDAs website on Digital Health Software Precertification (Pre-Cert) Program available at 
https://www.fda.gov/medical-devices/digital-health-center-excellence/digital-health-software-precertification-pre-
cert-program. 
9 For the purposes of this guidance, the terms ML-DSF and device are used interchangeably. Additionally, 
reference to an ML-DSF is referring to a software function that meets the definition of device, as defined in 
section 201(h) of the FD&C Act. See Section IV. for details on definitions. 
3  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
83  posed in the 2019 discussion paper.10 Additionally, numerous articles in peer-reviewed journals 
84  discuss or reference the framework proposed in the 2019 discussion paper.11  
85   
86  FDA has also held a number of public meetings and workshops on AI/ML topics. On February 
87  25-26, 2020, FDA held a Public Workshop on the Evolving Role of Artificial Intelligence in 
88  Radiological Imaging12 to discuss emerging applications of AI in radiological imaging, 
89  including AI devices intended to automate the diagnostic radiology workflow as well as guide 
90  image acquisition. At this workshop, the Agency worked with interested stakeholders, including 
91  patients, to identify both benefits and risks associated with the use of AI in radiological imaging, 
92  and discussed best practices for the validation of fully automated radiological imaging software 
93  and image acquisition devices.  
94   
95  On October 22, 2020, FDA held a Patient Engagement Advisory Committee meeting on 
96  Artificial Intelligence and Machine Learning in Medical Devices13 to further elicit input from a 
97  diverse group of patients on AI/ML technologies. The Committee provided recommendations on 
98  AI/ML-enabled medical devices and how to foster patient trust in them, considering the diverse 
99  populations in which they are and will be used.  
100   
101  On October 14, 2021, FDA held a Public Workshop on Transparency of Artificial 
102  Intelligence/Machine Learning-enabled Medical Devices14 for patients, caregivers, and 
103  providers. The purpose of the workshop was to 1) identify unique considerations in achieving 
104  transparency for users of AI/ML-enabled medical devices and ways in which transparency might 
105  enhance the safety and effectiveness of these devices; and 2) gather input from various 
106  stakeholders on the types of information that would be helpful for manufacturers to include in 
107  the labeling and public facing information of AI/ML-enabled medical devices, as well as other 
108  potential mechanisms for information sharing. 
109   
110  FDA continues to receive an increasing number of marketing submissions and pre-submissions 
111  for devices leveraging ML technologies, and the Agency expects this to increase over time. 
112  Moreover, since the 2019 discussion papers publication, there has been strong interest in 
113  utilizing PCCPs for AI/ML-enabled medical devices.  
114   
115  In light of the public health need to facilitate innovation for ML-DSFs while providing 
116  appropriate oversight for them, on January 12, 2021, the CDRH Digital Health Center of 
 
10 For more information, see the 2019 discussion papers public docket available at 
https://www.regulations.gov/document/FDA-2019-N-1185-0001. 
11 For example, Gerke S et al., The need for a system view to regulate artificial intelligence/machine learning-
based software as medical device, NPJ Digit Med 3, 53 (2020); Harvey et al., How the FDA Regulates AI, 
Academic Radiology 27, 58-61 (2020); and Subbaswamy et al., From development to deployment: dataset shift, 
causality, and shift-stable models in health AI, Biostatistics 21, 345-352 (2020). 
12 Available at https://www.fda.gov/medical-devices/workshops-conferences-medical-devices/public-workshop-
evolving-role-artificial-intelligence-radiological-imaging-02252020-02262020. 
13 Available at https://www.fda.gov/advisory-committees/advisory-committee-calendar/october-22-2020-patient-
engagement-advisory-committee-meeting-announcement-10222020-10222020. 
14 Available at https://www.fda.gov/medical-devices/workshops-conferences-medical-devices/virtual-public-
workshop-transparency-artificial-intelligencemachine-learning-enabled-medical-devices. 
4  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
117  Excellence issued FDAs Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a 
118  Medical Device Action Plan (the Action Plan).15 The Action Plan describes FDAs strategy for 
119  addressing AI/ML-enabled medical devices in a holistic, collaborative, and multidisciplinary 
120  manner. An important pillar of the Action Plan is the further advancement of the tailored 
121  regulatory framework for ML-enabled medical devices that was proposed in the 2019 discussion 
122  paper.  
123   
124  Further, on October 4, 2022, the White House released a Blueprint for an AI Bill of Rights,16 
125  which outlined five principles that should guide the design, use, and deployment of automated 
126  systems. These five principles discuss: safe and effective systems; algorithmic discrimination 
127  protections; data privacy; notice and explanation; and human alternatives, consideration, and 
128  fallback. This draft guidance is consistent with and promotes the principles described in the 
129  Blueprint for an AI Bill of Rights.  
130   
131  Additionally, section 3308 of the Food and Drug Omnibus Reform Act of 2022, Title III of 
132  Division FF of the Consolidated Appropriations Act, 2023, Pub. L. No. 117-328 (FDORA), 
133  enacted on December 29, 2022, added section 515C Predetermined Change Control Plans for 
134  Devices to the FD&C Act. Section 515C provides FDA with express authority to approve or 
135  clear PCCPs for devices requiring premarket approval or premarket notification. For example, 
136  section 515C provides that supplemental applications (section 515C(a)) and new premarket 
137  notifications (section 515C(b)) are not required for a change to a device that would otherwise 
138  require a premarket approval supplement or new premarket notification if the change is 
139  consistent with a PCCP previously approved or cleared by FDA. Section 515C also provides that 
140  FDA may require that a PCCP include labeling for safe and effective use of a device as such 
141  device changes pursuant to such plan, notification requirements if the device does not function as 
142  intended pursuant to such plan, and performance requirements for changes made under the 
143  plan.17 In this draft guidance, we provide recommendations on the marketing submission content 
144  for PCCPs, which are based on the statute and feedback obtained through our various 
145  interactions with stakeholders. 
146   
147  III.  Scope 
148  This draft guidance is applicable to ML-DSFs that the manufacturer intends to modify over time. 
149  This includes ML-DSFs for which modifications to the ML model are implemented 
150  automatically (i.e., for which the modifications are implemented automatically by software), as 
151  well as for ML-DSFs for which modifications to the ML model are implemented manually (i.e., 
152  involving steps that require human input, action, review, and/or decision-making, and therefore 
153  are not implemented automatically).  
154   
 
15 Available at https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-
machine-learning-software-medical-device. 
16 Available at https://www.whitehouse.gov/ostp/ai-bill-of-rights/.  
17 Sections 515C(a)(3) and (b)(3) of the FD&C Act. 
5  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
155  This draft guidance describes an approach that would often be least burdensome and would 
156  support the ability to modify an ML-DSF while continuing to provide a reasonable assurance of 
157  safety and effectiveness across relevant patient populations. Specifically, this draft guidance 
158  proposes recommendations on the information to be included in the PCCP in a marketing 
159  submission18 for a device that is or includes an ML-DSF. For the purposes of this guidance, the 
160  term PCCP refers to a plan that includes device modifications that would otherwise require a 
161  premarket approval supplement, De Novo submission, or a new premarket notification. A plan 
162  that contains only minor modifications that would not require a new submission is outside the 
163  scope of this guidance. For more information on whether a modification would require a new 
164  submission, see Deciding When to Submit a 510(k) for a Software Change to an Existing 
165  Device19 and Deciding When to Submit a 510(k) for a Change to an Existing Device20 or 
166  Modifications to Devices Subject to Premarket Approval (PMA) - The PMA Supplement 
167  Decision-Making Process21 guidances. Throughout this guidance, a PCCP that has been 
168  reviewed and established through a device marketing authorization is also referred to as an 
169  authorized PCCP. See Section IV. for more information on definitions used for the purposes of 
170  this guidance. 
171   
172  By including a PCCP in a marketing submission, manufacturers can proactively pre-specify and 
173  seek premarket authorization22 for intended modifications (and their method of implementation) 
174  to an ML-DSF without necessitating additional marketing submissions for each modification 
175  delineated and implemented in accordance with the PCCP. In other words, a PCCP, as part of a 
176  marketing submission, is intended to provide a means to implement modifications to an ML-DSF 
177  that generally would otherwise require additional marketing submissions prior to 
178  implementation.23  
179   
180  Modifications to an ML-DSF that could significantly affect,24 or that could affect,25 the safety or 
181  effectiveness of the device, unless those modifications are covered by a PCCP, require premarket 
182  authorization.26 Premarket authorization for an ML-DSF with a PCCP may be established 
 
18 For purposes of this guidance, unless otherwise stated, the term marketing submission includes premarket 
notification (510(k)) submission, De Novo Classification request, and Premarket Approval (PMA) application. 
19 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/deciding-when-submit-
510k-software-change-existing-device.  
20 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/deciding-when-submit-
510k-change-existing-device. 
21 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/modifications-devices-
subject-premarket-approval-pma-pma-supplement-decision-making-process. 
22 For the purposes of this guidance, the term authorization includes clearance of a 510(k) submission, grant of a 
De Novo Classification request, or approval of a PMA application. 
23 21 CFR 807.81(a)(3) and 21 CFR 814.39(a). 
24 21 CFR 807.81(a)(3). 
25 21 CFR 814.39(a). 
26 In accordance with 21 CFR 807.81(a)(3), a 510(k) is required for significant changes or modifications to a device 
and include 1) those that could significantly affect the safety or effectiveness of the device, e.g., a significant 
change or modification in design, material, chemical composition, energy source, or manufacturing process or 
include 2) a major change or modification in the intended use of the device. In accordance with 21 CFR 814.39(a), 
a PMA supplement is required for change[s] affecting the safety or effectiveness of the device unless an exception 
 
6  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
183  through the 510(k) pathway (see section 515C(b) of the FD&C Act), PMA pathway (see section 
184  515C(a) of the FD&C Act), or the De Novo pathway (see section 513(f)(2) of the FD&C Act).27 
185  FDA considers the PCCP to be part of the technological characteristics of the device. For devices 
186  subject to 510(k) requirements, the determination of substantial equivalence includes, among 
187  other requirements, a comparison of the technological characteristics of the predicate device to 
188  the technological characteristics of the subject device.28 In making a determination of substantial 
189  equivalence where the predicate device was authorized with a PCCP, the subject device must be 
190  compared to the version of the predicate device cleared or approved prior to changes made under 
191  the PCCP.29  
192   
193  Generally, the recommendations in this guidance apply to the device constituent part of a 
194  combination product30 (such as drug-device and biologic-device combination products) when the 
195  device constituent part31 is or includes an ML-DSF. Early engagement with the appropriate 
196  CBER or CDER Divisions is recommended for drug-device and biologic-device combination 
197  products that include an ML-DSF.32 For more information, contact the FDA review division that 
198  will have the lead review for the combination product.  
199   
200  FDA encourages early engagement regarding a proposed PCCP with the FDA review division 
201  that will review the ML-DSF; in particular, early engagement could be especially helpful for 
202  certain ML-DSFs, including combination products and high-risk, life-sustaining, life-supporting, 
203  or implantable devices. FDA encourages manufacturers to leverage the Q-Submission process 
204  for obtaining FDA feedback on a proposed PCCP for an ML-DSF.33 
205    
206  This draft guidance is not intended to provide a complete description of what may be necessary 
207  to include in a marketing submission for an ML-DSF. It is also not intended to delineate the 
208  types of modifications the Agency would consider acceptable in a PCCP. However, some 
209  considerations on modification types are described, including examples of questions for 
 
applies (see 21 CFR 814.39). For simplicity, in this guidance, we state significantly affect the safety or 
effectiveness of the device for when a marketing submission may be required, significant modifications, and 
similar phrasings, aligning with 21 CFR 807.81(a)(3). However, for devices subject to PMA requirements, the 
broader requirement pursuant to 21 CFR 814.39(a) of a change affecting the safety or effectiveness applies. 
27 The De Novo classification process allows FDA to classify a device into class I or II when general controls or 
general controls and special controls provide a reasonable assurance of safety and effectiveness, but for which there 
is no legally marketed predicate. The De Novo pathway, therefore, allows FDA to develop special controls that 
provide a reasonable assurance of the safety and effectiveness. At this time, FDA expects that if it authorizes an ML-
DSF with a PCCP via the De Novo pathway, the Agency would develop appropriate special controls, which may 
include specific requirements for a PCCP.   
28 See section 513(i) of the FD&C Act. 
29 See section 515C(c) of the FD&C Act. 
30 21 CFR 3.2(e).  
31 21 CFR 4.2. 
32 See FDAs guidance Requesting FDA Feedback on Combination Products available at 
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/requesting-fda-feedback-combination-
products. 
33 See FDAs guidance Requests for Feedback and Meetings for Medical Device Submissions: The Q-Submission 
Program available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/requests-
feedback-and-meetings-medical-device-submissions-q-submission-program. 
7  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
210  consideration that manufacturers should address in a Modification Protocol (see Appendix A) 
211  and illustrations of different ML-DSF scenarios where a PCCP could be employed (see 
212  Appendix B). The FDA review division with purview over the device under review will 
213  determine the acceptability of a proposed PCCP. That is, the FDA review division will determine 
214  whether the scope of the modifications is appropriate for inclusion in a PCCP and what evidence 
215  and information are required to support proposed modifications in a marketing submission.34 
216   
217  IV.  Definitions  
218  This section defines certain terms as they are used for the purposes of this guidance. 
219  A.  Software Functions 
220  Device Software Function (DSF): A software function that meets the device definition in 
221  section 201(h) of the FD&C Act.35,36 As discussed in other FDA guidances, the term function 
222  is a distinct purpose of the product, which could be the intended use or a subset of the intended 
223  use of the product.37 
224   
225  Machine Learning-Enabled Device Software Function (ML-DSF): A device software 
226  function (as defined above) that implements an ML model trained with ML techniques.  
227   
228  B.  Data Sets 
229  Training Data: These data are used by the ML-DSF manufacturer in procedures and ML 
230  training algorithms to build an ML model, including to define model weights, connections, and 
231  components. Training the ML model happens after the exploratory phase of ML-DSF 
232  development. These data typically should be representative of the proposed intended use 
233  populations (e.g., with respect to race, ethnicity, disease severity, gender, age, etc.). The result is 
234  an operational ML-DSF. 
235   
236  Tuning Data: These data are typically used by the ML-DSF manufacturer to evaluate a small 
237  number of trained ML-DSFs in order to explore, for example different architectures or 
238  hyperparameters. The tuning phase is the last phase before ML-DSF testing and is often 
239  considered part of the training process. The ML community sometimes refers to the tuning data 
240  and phase with the word validation. However, we recommend that the word validation not 
 
34 21 CFR 807.81(a)(3) and 21 CFR 814.39(a). 
35 See footnote 1. 
36 Device software functions may include Software as a Medical Device (SaMD) and Software in a Medical Device 
(SiMD). See FDAs website on Software as a Medical Device (SaMD), available at https://www.fda.gov/medical-
devices/digital-health-center-excellence/software-medical-device-samd. 
37 See FDAs guidance Multiple Function Device Products: Policy and Considerations available at 
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/multiple-function-device-products-
policy-and-considerations. 
8  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
241  be used when referring to data or operations related to training or tuning ML models intended for 
242  medical applications.38 
243   
244  Testing Data: These data are used to characterize the performance of the ML-DSF. The testing 
245  phase is expected to provide data to establish a reasonable assurance of safety and effectiveness 
246  before an ML-DSF is marketed. Testing data should be independent of data used for training and 
247  tuning and should be from multiple sites different from those that were used to generate training 
248  and tuning data. 
249   
250  C.  Predetermined Change Control Plan 
251  Predetermined Change Control Plan (PCCP): The documentation describing what 
252  modifications will be made to the ML-DSF and how the modifications will be assessed. The 
253  modifications described in the PCCP include device changes that would otherwise require a 
254  PMA supplement, De Novo submission, or new 510(k) notification. The PCCP includes a 
255  Description of Modifications, Modification Protocol, and Impact Assessment.  
256  Authorized Predetermined Change Control Plan (Authorized PCCP): A PCCP that has been 
257  reviewed and established through a device marketing authorization. An authorized PCCP is a 
258  technological characteristic of the authorized device with which it was established.  
259  Modification Protocol: The documentation describing the methods that will be followed when 
260  developing, validating, and implementing modifications delineated in the Description of 
261  Modifications section of the PCCP. The Modification Protocol includes the verification and 
262  validation activities (including pre-defined acceptance criteria) for those modifications and is 
263  intended to provide a step-by-step delineation of how the modifications proposed in the PCCP 
264  will be implemented while ensuring the device remains safe and effective. 
265  Impact Assessment: The documentation of the assessment of the benefits and risks of 
266  implementing a proposed PCCP, as well as the plan for risk mitigation. It could be a separate 
267  section of a PCCP or incorporated into the Modification Protocol, whichever is least burdensome 
268  for a particular marketing submission.  
269   
270  V.  Policy for Predetermined Change Control Plans  
271  Software development is an iterative process, and FDA appreciates that manufacturers of device 
272  software functions strive to continually improve and update their devices. Manufacturers should 
273  evaluate the impact of modifications to their devices and must submit a marketing submission 
274  when device modifications affect the intended use of the device or could significantly affect the 
275  safety or effectiveness of the device.39 
 
38 The way the term validation is used in the ML community to refer to training or tuning data sets is not 
consistent with the broad understanding of the word validation, defined in 21 CFR 820.3(z) as confirmation by 
examination and provision of objective evidence that the particular requirements for a specific intended use can be 
consistently fulfilled. 
39 21 CFR 807.81(a)(3) and 21 CFR 814.39(a). 
9  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
276  An authorized PCCP specifies planned modifications that, if not included in a PCCP, could 
277  otherwise require a new marketing submission pursuant to 21 CFR 807.81(a)(3) and 21 CFR 
278  814.39(a), and in accordance with Deciding When to Submit a 510(k) for a Software Change to 
279  an Existing Device40 and Deciding When to Submit a 510(k) for a Change to an Existing 
280  Device41 or Modifications to Devices Subject to Premarket Approval (PMA) - The PMA 
281  Supplement Decision-Making Process42 guidances (hereafter referred to as the device 
282  modifications guidances). An authorized PCCP includes the range of FDA-authorized 
283  specifications for the characteristics and performance of the planned modifications to the device 
284  (as part of the detailed Description of Modifications section included in the PCCP), along with 
285  the associated verification and validation testing and acceptance criteria to assure the device 
286  remains safe and effective across relevant patient populations (as part of the Modification 
287  Protocol section included in the PCCP), and documentation of the assessment of the benefits and 
288  risks of implementing a proposed PCCP (which could be a separate Impact Assessment section 
289  of a PCCP or incorporated into the Modification Protocol). Because modifications made to an 
290  ML-DSF in accordance with an authorized PCCP were reviewed and authorized through the 
291  marketing submission containing the PCCP, the modifications can be implemented to the ML-
292  DSF without triggering the need for a new marketing submission under 21 CFR 807.81(a)(3) and 
293  21 CFR 814.39(a), and in accordance with the device modifications guidances. 
294   
295  Deviations from the authorized PCCP reviewed in the marketing submission could significantly 
296  affect the safety or effectiveness of the device. For example, when implementing a modification 
297  that is a deviation from the PCCP, the devices clinical functionality or performance 
298  specifications could be compromised. Deviations from the PCCP include instances where the 
299  PCCP is not followed or cannot be followed (e.g., issues related to the Modification Protocol, 
300  such as data management, re-training, or performance failure). Accordingly, modifications made 
301  to an ML-DSF that are not specified in, or implemented in accordance with, the authorized 
302  PCCP (i.e., the manufacturer deviates from the authorized PCCP when implementing the 
303  modification) likely require a new marketing submission.43, 44 In such a circumstance, continued 
304  distribution of the ML-DSF without submitting a new marketing submission would constitute 
305  adulteration and misbranding under sections 501(f)(1)(B) and 502(o) of the FD&C Act, 
306  respectively. Adulteration and misbranding are prohibited acts under section 301 of the FD&C 
307  Act, and where appropriate, FDA may take legal or regulatory action against violations of 
308  prohibited acts, including, without limitation, seizure or injunction. 
309   
310  A.  Components of a PCCP 
 
40 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/deciding-when-submit-
510k-software-change-existing-device.  
41 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/deciding-when-submit-
510k-change-existing-device. 
42 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/modifications-devices-
subject-premarket-approval-pma-pma-supplement-decision-making-process. 
43 21 CFR 807.81(a)(3) and 21 CFR 814.39(a). 
44 See Section V.D. below for further details on implementing device modifications that may or may not require a 
new marketing submission in accordance with 21 CFR 807.81(a)(3) and 21 CFR 814.39(a). 
10  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
311  A PCCP should consist of a detailed Description of Modifications, a Modification Protocol, and 
312  an Impact Assessment (see Sections VI.  VIII.) because these components are intended to 
313  provide FDA with the information needed for our review of the proposed modifications. The 
314  detailed Description of Modifications should outline the modifications that will be made to the 
315  ML-DSF, and the Modification Protocol should describe the verification and validation activities 
316  (including pre-defined acceptance criteria) that will support those modifications. The Impact 
317  Assessment helps to tie the Description of Modifications to the Modification Protocol in that the 
318  Impact Assessment identifies the benefits and risks introduced by the specified, planned 
319  modifications and how the verification and validation activities of the Modification Protocol will 
320  continue to assure the safety and effectiveness of the device. The detailed Description of 
321  Modifications, Modification Protocol, and Impact Assessment are all components of a PCCP and 
322  should exist in tandem. 
323   
324  B.  Establishing a PCCP 
325  Premarket authorization for an ML-DSF with a PCCP must be established through the 510(k) 
326  pathway, De Novo pathway, or PMA pathway, as appropriate, as a PCCP must be reviewed and 
327  established as part of a marketing authorization for a device prior to a manufacturer 
328  implementing any modifications under that PCCP.45 For 510(k) submissions, in making a 
329  determination of substantial equivalence where the predicate device was authorized with a 
330  PCCP, the subject device must be compared to the version of the predicate device cleared or 
331  approved prior to changes made under the PCCP.46 
332   
333  For a manufacturer who would like to establish a new PCCP for a previously authorized device 
334  with a PCCP, the marketing submission must include the appropriate marketing submission 
335  requirements47 and the proposed PCCP for the device (see Section V.E.).48 FDA intends to focus 
336  its review on the aspects of the device that are most significantly modified.49 For example, if the 
337  device has been relatively unchanged since FDAs prior authorization and a new PCCP is 
338  proposed, FDA would focus its review on the proposed PCCP. 
339   
340  An authorized PCCP is a technological characteristic of the authorized device with which it was 
341  established. If the authorized device is significantly modified other than as specified in the 
342  authorized PCCP, a new marketing submission is required for the significantly modified 
343  device.50 In that case, for the original authorized PCCP to apply to the significantly modified 
344  device, the PCCP will need to be reviewed and established as part of such marketing submission.  
345   
 
45 See sections 513(f)(2) and 515C of the FD&C Act. 
46 See section 515C(c) of the FD&C Act. 
47 See, e.g., 21 CFR 807.87, 21 CFR 860.220, or 21 CFR 814.20. 
48 E.g., through a Traditional 510(k) for a device that has already been authorized, or a PMA supplement, such as a 
panel-track supplement or 180-day supplement. 
49 Note that focus of the review is not intended to imply a review of the PCCP only; rather, the focus on the PCCP 
is as a significant change to the device that could affect the safety or effectiveness of the device. 
50 21 CFR 807.81(a)(3) and 21 CFR 814.39(a). 
11  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
346  FDA encourages manufacturers to leverage the Q-Submission process for obtaining FDA 
347  feedback on a proposed PCCP prior to submitting a marketing submission. For more information 
348  on the Q-Submission process, refer to the Requests for Feedback and Meetings for Medical 
349  Device Submissions: The Q-Submission Program guidance.51 While manufacturers are 
350  encouraged to discuss their plans through a Pre-Submission, PCCPs are not authorized in Pre-
351  Submissions. 
352   
353  C.  Identifying a PCCP in a Marketing Submission 
354  The PCCP should be included as a standalone section within the marketing submission. 
355  Additionally, it should be prominently included and discussed in the cover letter and included in 
356  the marketing submissions table of contents as Predetermined Change Control Plan. The 
357  PCCP should be discussed in the marketing submission as part of the device description, 
358  labeling, and relevant sections used for determining substantial equivalence or reasonable 
359  assurance of safety and effectiveness. Any information pertaining to the PCCP content included 
360  outside of the PCCP section should be referenced within the PCCP section.  
361   
362  Device labeling must comply with applicable statutes and regulations,52 which includes adequate 
363  directions for use.53 FDA may require that a change control plan include labeling required for 
364  safe and effective use of the device as such device changes pursuant to such plan.54 For ML-
365  DSFs with an authorized PCCP, the labeling should explain that the device incorporates machine 
366  learning and has a PCCP so that users are aware that the device may require the user to perform 
367  software updates, and that such software updates may modify the devices performance, inputs, 
368  or use. Information on the ML-DSF and its PCCP in the labeling is important in order for a user 
369  to use the device safely and for the purposes for which it is intended. In particular, this 
370  information may be necessary for a user to understand changes in the device and to continue to 
371  use the device safely and effectively across relevant populations as the device changes pursuant 
372  to the PCCP.  
373    
 
51 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/requests-feedback-and-
meetings-medical-device-submissions-q-submission-program.  
52 21 CFR 801 (Labeling) and 21 CFR 809 (In Vitro Diagnostic Products for Human Use). See, e.g., 21 CFR 801.5 
(requiring that labeling include adequate directions for use); 21 CFR 801.109(c) (for prescription devices, requiring 
that labeling include any relevant hazards, contraindications, side effects, and precautions under which practitioners 
licensed by law to administer the device can use the device safely and for the purpose for which it is intended); and 
21 CFR 809.10(b)(6) (for in vitro diagnostic products, requiring labeling accompanying any instruments use or 
function, installation procedures, performance characteristics and specifications, service and maintenance 
information, etc.). 
53 21 CFR 801.5 (requiring that labeling include adequate directions for use). 
54 See sections 515C(a)(3), 515C(b)(3), and 513(f)(2) of the FD&C Act. 
12  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
374  The PCCP should be described in the 510(k) summary,55,56 De Novo decision summary,57 or 
375  PMA summary of safety and effectiveness document (SSED) and approval order.58 Details of the 
376  PCCP should be included in sufficient detail in the public-facing documents to support 
377  transparency to users of FDAs determination of substantial equivalence or reasonable assurance 
378  of safety and effectiveness for the device and its range of FDA-authorized specifications. 
379   
380  D.  Utilizing an Authorized PCCP to Implement Device 
381  Modifications 
382  Once a PCCP has been reviewed and established through a marketing submission, the PCCP is 
383  considered part of the marketing authorization. In general, a PCCP should be evaluated within 
384  the existing risk management framework of the device and implemented in accordance with the 
385  manufacturers quality system. 
386   
387  Figure 1 depicts the process for implementing a modification to a device with an authorized 
388  PCCP. Manufacturers should first consider whether the modification is or is not consistent with 
389  the authorized PCCP; a modification is considered consistent with the authorized PCCP when 
390  the modification has been specified in the Description of Modifications included in the PCCP 
391  and has been implemented in accordance with the Modification Protocol. If the modification is 
392  consistent with the authorized PCCP, a new marketing submission is not necessary, the 
393  modification can be implemented in accordance with the Modification Protocol, and the 
394  manufacturer should document that modification and the analysis in accordance with the 
395  manufacturers quality system.59 
396   
397  As described previously, an authorized PCCP specifies planned modifications that, if not 
398  included in a PCCP, would otherwise require a new marketing submission pursuant to 21 CFR 
399  807.81(a)(3) and 21 CFR 814.39(a), and in accordance with the device modifications guidances. 
400  Therefore, if the modification is not consistent with the authorized PCCP  including if the 
401  specific modification is not included in the authorized PCCP or if the modification is included in 
 
55 In accordance with 21 CFR 807.92, a 510(k) summary shall be in sufficient detail to provide an understanding of 
the basis for a determination of substantial equivalence. This includes, but is not limited to, a description of the 
device, and for those 510(k) submissions in which a determination of substantial equivalence is also based on an 
assessment of performance data, non-clinical tests, and clinical tests. 
56 If a sponsor chooses to submit a 510(k) Statement rather than 510(k) Summary, the sponsor should provide to 
requestors all PCCP information in the 510(k) that supports transparency to users of FDAs determination of 
substantial equivalence for the device and its range of FDA-authorized specifications, as such information 
constitutes safety and effectiveness information. See 21 CFR 807.93 for requirements on the content and format of a 
510(k) Statement. 
57 The De Novo decision summary is intended to present an objective and balanced summary of the scientific 
evidence that served as the basis for the FDA's decision to grant a De Novo request; see 
https://www.fda.gov/medical-devices/premarket-submissions/de-novo-classification-request. 
58 In accordance with 21 CFR 814.9(e), FDA will make available to the public ... a detailed summary of 
information submitted to FDA respecting the safety and effectiveness of the device that is the subject of the PMA 
and that is the basis for the order. 
59 Manufacturers are required to comply with the quality system regulation (21 CFR 820). The device and PCCP 
must be implemented consistent with 21 CFR 820, including, but not limited to: 21 CFR 820.30 Design controls; 21 
CFR 820.90 Nonconforming products; and 21 CFR 820.100 Corrective and preventive action. 
13  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
402  the authorized PCCP but is not implemented in accordance with the methods and specifications 
403  described in the Modification Protocol  the manufacturer should then proceed to evaluate the 
404  modification in accordance with applicable laws and regulations and the device modifications 
405  guidances, if applicable, and proceed accordingly. If, after review of applicable laws and 
406  regulations, a new marketing submission is required,60 then the manufacturer must submit the 
407  appropriate marketing submission before the modified device is marketed. The appropriate 
408  marketing submission could request authorization for 1) a device modification effected through a 
409  change to the authorized PCCP61 (see Section V.E.); or 2) a device modification not 
410  implemented through a PCCP; or 3) both. In each of these cases, a marketing submission for the 
411  device modification must include the appropriate marketing submission requirements62 for the 
412  device. As noted in Section V.B., an authorized PCCP is a technological characteristic of its 
413  authorized device with which it was established. Therefore, for the original authorized PCCP to 
414  apply to the modified device, the manufacturer must include the proposed PCCP in the 
415  marketing submission for the device modification;63 the PCCP is reviewed and established as 
416  part of the review of the modified device. 
417   
418  See Appendix B for example scenarios for implementing modifications to a device with an 
419  authorized PCCP. Manufacturers may contact the appropriate FDA review division (e.g., through 
420  a Pre-Submission)64 for a discussion about the proposed modification and whether it may be 
421  considered consistent with the current, authorized PCCP.  
422   
 
60 See, e.g., 21 CFR 807.81(a)(3) and 21 CFR 814.39(a). 
61 A change to the authorized PCCP could include a change in Description of Modifications, the Modification 
Protocol, and/or the Impact Assessment, as appropriate. 
62 See, e.g., 21 CFR 807.87, 21 CFR 860.220, or 21 CFR 814.20. 
63 See sections 513(f)(2) and 515C of the FD&C Act. 
64 See FDAs guidance Requests for Feedback and Meetings for Medical Device Submissions: The Q-Submission 
Program available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/requests-
feedback-and-meetings-medical-device-submissions-q-submission-program.  
14  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Is the modification
(1) specified in the Description of 
Modifications
Document
and
START Yes in accordance with Quality 
(2) implemented in conformance with 
System
the methods and specifications in the 
Modification Protocol 
of an authorized PCCP?
 No
Following review of the 
Submit marketing 
applicable laws and regulations, 
Yes submission for 
 is a new marketing submission 
the modification*
required?
No
*For the modified device to have a PCCP, a PCCP should be submitted with the 
423  marketing submission so that the device and PCCP can be authorized together.   
424   
425  Figure 1: Implementing a Modification to a Device with an Authorized PCCP 
426   
427  E.  Modifying a PCCP for an Authorized Device 
428  Because the modifications described in the PCCP include device changes that would otherwise 
429  require a PMA supplement, De Novo submission, or new 510(k) premarket notification, at this 
430  time, FDA believes that modifications to an authorized PCCP will generally constitute changes 
431  to the ML-DSF that require a new marketing submission for the device, which will include the 
432  modified PCCP.65 In other words, FDA expects that the modified PCCP will need to be reviewed 
433  and established as part of the premarket review of the modified device because a modification to 
434  the PCCP will generally significantly affect the safety or effectiveness of the device.66 In those 
435  cases where modifications to the PCCP are the only significant modifications introduced since 
436  FDAs prior authorization for the device and its PCCP, FDA intends to focus its review on the 
437  proposed PCCP. For a manufacturer who would like to modify a PCCP for a previously 
438  authorized device with a PCCP, the marketing submission must include the appropriate 
439  marketing submission requirements for the device and the proposed PCCP.67 
440   
 
65 Section 510(l)(1) and section 515(d)(5)(A)(i) of the FD&C Act. 
66 See, e.g., 21 CFR 807.87, 21 CFR 860.220, or 21 CFR 814.20. 
67 See sections 513(f)(2) and 515C of the FD&C Act.  
15  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
441  VI.  Description of Modifications 
442  As introduced above, a description of each planned modification to an ML-DSF should be 
443  included in the Description of Modifications section of a PCCP. The detailed description should 
444  describe changes to the device characteristics and performance resulting from implementation of 
445  the modifications. To ensure an efficient review, FDA recommends that a PCCP include only a 
446  limited number of modifications that are specific, and that can be verified and validated.  
447   
448  A.  Goals of the Description of Modifications Section  
449  A dedicated Description of Modifications section in a PCCP identifies the specific, planned 
450  modifications to the ML-DSF that the manufacturer intends to implement. The Description of 
451  Modifications draws a range of FDA-authorized specifications around the initial characteristics 
452  and performance of the device that, following the agreed upon verification and validation 
453  described in the Modification Protocol, can be implemented without a new marketing 
454  submission. Upon FDA review of a PCCP, it is possible that FDA may determine that a 
455  Modification Protocol supports some but not all modifications identified in a PCCP; in such 
456  cases, only those modifications that are appropriate in the FDAs findings of substantial 
457  equivalence or reasonable assurance of safety and effectiveness would be included in the 
458  authorized PCCP.
 
459   
460  B.  Content of the Description of Modifications Section 
461  The Description of Modifications should enumerate the list of individual proposed device 
462  modifications discussed in the PCCP, as well as the specific rationale for the change to each part 
463  of the ML-DSF that is planned to be modified. In some situations, a Description of Modifications 
464  will consist of multiple modifications. It may be helpful to reference the labeling changes that are 
465  associated with each modification in the Description of Modifications section (such labeling 
466  changes should be included in a Modification Protocol, as described in Section VII.B.). 
467   
468  FDA recommends that a PCCP include modifications that are specific, and that can be verified 
469  and validated. Modifications should also be presented at a level of detail that permits 
470  understanding of the specific modifications that will be made to the ML-DSF. Each modification 
471  should be linked to a specific performance evaluation activity within the Modification Protocol 
472  (for an example, see Table 1 in Section VII.C.).  
473   
474  The Description of Modifications should clearly state if the planned modifications are proposed 
475  to be implemented automatically (i.e., whether the modifications are implemented automatically 
476  by software) or whether modifications are implemented manually (i.e., involving steps that 
477  require human input, action, review, and/or decision-making, and therefore are not implemented 
478  automatically). Understanding that this is an evolving area, FDA is proposing to consider PCCPs 
479  for ML-DSFs where modifications are implemented automatically to the extent the Agency can 
480  properly review them for substantial equivalence to the predicate or a reasonable assurance of 
481  safety and effectiveness. The Agency recognizes that this subset of ML-DSFs has an additional 
482  degree of complexity; as with all ML-DSFs, FDA will consider the benefit-risk assessment, and 
16  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
483  the Agencys current experience applying this policy when reviewing PCCPs containing 
484  automatically implemented modifications.  
485   
486  The Description of Modifications should also clearly specify if the proposed modifications will 
487  be implemented in a uniform manner across all devices on the market (sometimes referred to as 
488  homogenous or global changes, or global adaptations) or implemented differently on different 
489  devices on the market based on, for example, the unique characteristics of a specific clinical site 
490  or individual patients (sometimes referred to as heterogenous or local changes, or local 
491  adaptations). For local adaptations, the Description of Modifications should include describing 
492  what local factors or conditions warrant a local change. 
493   
494  C.  Types of Modifications 
495  Modifications that are appropriate for a PCCP include those that are intended to maintain or 
496  improve the safety or effectiveness of the device. Modifications proposed within the Description 
497  of Modifications should be able to be verified and validated within the existing quality system of 
498  the device. As such, not all modifications may be appropriate for inclusion within a PCCP. Types 
499  of modifications that may be acceptable within a PCCP include: 
500   
501  (i) modifications related to quantitative measures of ML-DSF performance specifications; 
502  (ii) modifications related to device inputs to the ML-DSF; and 
503  (iii) limited modifications related to the devices use and performance (e.g., for use 
504  within a specific subpopulation). 
505   
506  Examples of modifications related to quantitative measures of ML-DSF performance 
507  specifications include improvements to analytical and clinical performance resulting from re-
508  training the ML model based on new data within the intended use population from the same type 
509  and range of input signal. 
510   
511  Modifications related to device inputs to the ML-DSF may involve expanding the algorithm to 
512  include new sources of the same signal type (e.g., different makes, models, or versions of a data 
513  acquisition system) or limited modifications related to new types of inputs. 
514   
515  Modifications related to the devices use and performance could include authorization of a 
516  device for a specific subset of a population within the originally indicated population based on 
517  re-training on a larger data set for that subpopulation that was not previously available.  
518   
519  All modifications included in a PCCP must maintain the device within the devices intended 
520  use.68 At this time, FDA expects that modifications included in a PCCP should also maintain the 
521  device within the devices indications for use.69 As with modifications to the intended use, FDA 
 
68 Section 515C(a)(2) and section 515C(b)(2) FD&C Act. 
69 FDA has a long-standing policy of applying the definition of indications for use in the PMA regulation at 21 CFR 
814.20(b)(3)(i) in the same way in the 510(k) context. See the FDA guidance The 510(k) Program: Evaluating 
Substantial Equivalence in Premarket Notifications [510(k)], available at https://www.fda.gov/regulatory-
 
17  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
522  believes modifications to the indications for use would not allow the device to remain safe and 
523  effective.70 
524   
525  Recognizing there is a spectrum of risk for devices, for the purposes of reviewing a PCCP, FDA 
526  intends to, among other considerations, take into account the benefit-risk profile of the specific 
527  device that is the subject of the PCCP, the specific modifications being proposed, and its 
528  experience applying this policy across different device types. See Appendix B for examples of 
529  modifications. 
530   
531  VII. Modification Protocol 
532  The Modification Protocol includes the documentation describing the methods that will be 
533  followed when developing, validating, and implementing modifications delineated in the 
534  Description of Modifications section of the PCCP. The Modification Protocol includes the 
535  verification and validation activities (including pre-defined acceptance criteria) for those 
536  modifications and is intended to provide a step-by-step delineation of how the modifications 
537  proposed in the PCCP will be implemented while assuring the device remains safe and effective. 
538  Documentation of modifications verified and validated per the Modification Protocol must be 
539  compliant with the quality system (QS) regulation.71 The QS regulation requires manufacturers 
540  of finished medical devices to review and approve modifications to device design and production 
541  (21 CFR 820.30 and 820.70) and document changes and approvals in the device master record 
542  (21 CFR 820.181). 
543   
544  A.  Goals of the Modification Protocol Section 
545  Whereas the Description of Modifications outlines the planned modifications to an ML-DSF, the 
546  Modification Protocol describes the methods that will be followed when developing, validating, 
547  and implementing those modifications, to ensure the device remains safe and effective. The 
548  methods described in the Modification Protocol should be consistent with and support the 
549  modifications outlined in the Description of Modifications. 
550   
551  The goals of the Modification Protocol are to: 
552   
553    Identify the methods and data used to develop, validate, and implement all proposed 
554  modifications; 
555    Identify the test methods, data, statistical analyses, and specified acceptance criteria 
556  for all proposed modifications; 
557    Ensure that the information that would otherwise be generated and submitted to the 
558  Agency (i.e., if the modifications were implemented on a device that did not have an 
559  authorized PCCP) will be generated by the manufacturer for each modification and 
 
information/search-fda-guidance-documents/510k-program-evaluating-substantial-equivalence-premarket-
notifications-510k. 
70 Section 515C(a)(2)(A) and section 515C(b)(2) FD&C Act. 
71 21 CFR Part 820. 
18  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
560  maintained consistent with recordkeeping requirements72 and in accordance with the 
561  manufacturers QS; 
562    Ensure that the risks that have been identified in the Impact Assessment as related to 
563  modifications detailed in the Description of Modifications (including the update 
564  process and communication to users) will be mitigated; and  
565    Be least burdensome for the manufacturer to develop and for FDA to review. This 
566  includes being traceable and specific to the modifications detailed in the Description 
567  of Modifications section and sufficiently comprehensive to support specific 
568  modifications. 
569   
570  This draft guidance identifies four primary components of a Modification Protocol, described 
571  below, that outline a manufacturers 1) data management practices, 2) re-training practices, 3) 
572  performance evaluation protocols, and 4) update procedures, including communication and 
573  transparency to users and real-world monitoring plans, for each modification in a PCCP. In 
574  FDAs experience, these four components generally provide FDA with the information needed to 
575  evaluate the PCCP. For a particular marketing submission, additional components of a 
576  Modification Protocol may be important to provide a reasonable assurance of safety and 
577  effectiveness. 
578   
579  In some cases, the same methods in each component of the Modification Protocol may support 
580  all modifications in a PCCP for a device. In other cases, the same methods in each component 
581  may not be adequate for every modification in a PCCP. It is important to note, however, that for 
582  each planned modification provided in the Description of Modifications, FDA recommends that 
583  each of the four primary components of a Modification Protocol be addressed. Additionally, the 
584  Modification Protocol should include a description of how its proposed methods are similar to or 
585  different from methods used elsewhere in the marketing submission. For example, if the 
586  validation methods in the Modification Protocol represent a subset of the original testing for the 
587  device, or if the acceptance criteria for the validation are different, manufacturers should 
588  describe these differences and provide a justification. The justification for a different 
589  methodology may include references to other marketing submissions where the methodology 
590  was used for similar modifications.  
591   
592  As noted above in Section V.E., manufacturers should follow their risk management processes to 
593  develop a Modification Protocol that considers each modification. Upon FDA review of a PCCP, 
594  FDA may determine that the Modification Protocol supports some but not all proposed 
595  modifications identified in the PCCP. In such cases, only those modifications that were 
596  supportable by the proposed Modification Protocol would be appropriate for inclusion in the 
597  PCCP and would become part of the authorized PCCP. Those modifications that were not 
598  supportable by the proposed Modification Protocol, or that FDA considered inappropriate for 
599  other reasons (e.g., the modification changes the devices indications for use), would be removed 
600  from the PCCP. 
601   
 
72 21 CFR 820 Subpart M  Records. 
19  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
602  B.  Content of the Modification Protocol Section 
603  As part of their Modification Protocol, manufacturers should outline the methods for each 
604  component described below. Example elements of each of the four Modification Protocol 
605  components are provided in Appendix A.  
606   
607  (1)  Data management practices  
608  What they are: ML-DSF training and testing typically utilize data that include the inputs (e.g., 
609  medical images) that will be used by the device and often utilize a label or reference standard 
610  that is determined through a reference standard determination and/or an annotation process. 
611  Training data and testing data are sequestered to prevent overfitting and misquotes of test 
612  performance. The training and testing methods aim to identify and eliminate bias in the data 
613  (which may be inherent from historical datasets) and to improve the robustness and resilience of 
614  these algorithms to withstand changing clinical inputs and conditions. To support modifications 
615  to an ML-DSF that may need training and/or testing, it is anticipated that new data (i.e., data that 
616  were not used to develop the initial ML-DSF) will be collected. The data management practices 
617  in a Modification Protocol should outline how those new data will be collected, annotated, 
618  curated, stored, retained,73 controlled, and used by the manufacturer for each modification. The 
619  data management practices in a Modification Protocol should also clarify the relationship 
620  between the Modification Protocol data and the data used to train and test the initial and 
621  subsequent versions of the ML-DSF. It should also describe the control methods employed to 
622  curb the potential for data or performance information leaking into the development process 
623  during modification development or assessment. 
624   
625  Why they are recommended: This information allows FDA to understand the manufacturers data 
626  management practices that will be used to support each modification to an ML-DSF, including 1) 
627  how the manufacturer plans to obtain and use training and testing data that are complete and 
628  representative of the proposed intended use populations (e.g., with respect to race, ethnicity, 
629  disease severity, gender, age, etc.74); 2) whether identifiable subpopulations will be adequately 
630  represented, including intersectional groups, and separated into training and testing sets to 
631  minimize ML model bias; 3) how training and testing data will be sequestered to prevent 
632  overfitting and misquotes of test performance; 4) how older data will be complemented or 
633  replaced by newer data so that the performance is representative of the current patient population 
634  and standard of care; 5) whether the reference standard represents the best available process for 
635  determining the ground truth; and 6) how the data management practices may reduce the 
636  potential to produce discriminatory outcomes. A clear explanation of data management practices 
637  also provides assurance to FDA that modifications to the ML-DSF are based on data that are 
 
73 The QS regulation requires manufacturers to retain all records for a period of time equivalent to the design and 
expected life of the device, but in no case less than 2 years from the date of release for commercial distribution by 
the manufacturer (21 CFR 820.180(b)). 
74 We recommend that manufacturers consider additional characteristics, such as those described in the Blueprint for 
an AI Bill of Rights (https://www.whitehouse.gov/ostp/ai-bill-of-rights/): race, color, ethnicity, sex (including 
pregnancy, childbirth, and related medical conditions, gender identity, intersex status, and sexual orientation), 
religion, age, national origin, disability, veteran status, and genetic information. 
20  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
638  representative of the device indications for use. This includes information regarding products 
639  that will be used to generate data as inputs for the ML-DSF, patient populations in which the 
640  device will be used, and clinical scenarios where the device will be used. 
641   
642  What manufacturers should include in a submission: Examples of the types of information 
643  manufacturers should provide in a Modification Protocol describing their data management 
644  practices are provided in Appendix A. In general, this information should describe: how data will 
645  be collected, including clinical study protocols with inclusion/exclusion criteria; information on 
646  how data will be processed, stored, and retained;75 the process that will be followed to determine 
647  the reference standard; the quality assurance process related to the data; the data sequestration 
648  strategies that will be followed during data collection to separate the data into training and 
649  testing sets; and the protocols in place to prevent access during the training and tuning process to 
650  data intended for performance testing. 
651   
652  (2)  Re-training practices  
653  What they are: ML software generally involves multiple processing steps from the point the ML-
654  DSF receives the input data to the point it provides an output. The re-training practices 
655  component of a Modification Protocol should identify the processing steps that are subject to 
656  change for each modification and the methods that will be used by the manufacturer to 
657  implement modifications to the ML-DSF. In addition, if re-training involves ML architecture 
658  modifications (e.g., in a neural network, modifications to training hyperparameters or the number 
659  of nodes, layers, etc.), the re-training practices component of a Modification Protocol should also 
660  describe the rationale or the justification for each specific architecture modification. 
661   
662  Why they are recommended: Information on the manufacturers re-training practices allows FDA 
663  to understand how the proposed modifications will be achieved through re-training, to determine 
664  if modifications are implemented following appropriate, well-defined practices,76 and to 
665  determine if the performance evaluation and update procedures (discussed below) support the 
666  modifications. Information on the manufacturers re-training practices is typically provided in 
667  the device description of a marketing submission for the majority of ML-DSFs that FDA 
668  reviews. The specifics of what should be included in this component of the Modification 
669  Protocol will depend on the type of modification and specific device.  
670   
671  What manufacturers should include in a submission: Examples of the types of information 
672  manufacturers should provide in a Modification Protocol describing their re-training practices 
673  are provided in Appendix A. In general, this information should identify the objective of the re-
674  training process, provide a description of the ML model, identify the device components that 
675  may be modified, outline the practices that will be followed (e.g., data sequestration strategies 
676  during re-training), and identify any triggers for re-training (e.g., when new data reaches a 
677  certain size or when a drift in data is observed over time). 
 
75 See 21 CFR 820.180(b). 
76 For example, FDA has published a document on Good Machine Learning Practice for Medical Device 
Development: Guiding Principles, available at https://www.fda.gov/medical-devices/software-medical-device-
samd/good-machine-learning-practice-medical-device-development-guiding-principles.  
21  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
678   
679  (3)  Performance evaluation  
680  What they are: The FDA may require that performance requirements for changes made under the 
681  plan be provided in a PCCP.77 Performance evaluation methods should describe the processes 
682  that will be followed to validate that the modified ML-DSF will meet the specifications 
683  identified as part of a specific modification, in addition to maintaining the specifications that are 
684  not part of the modification but may be impacted by the modification. Performance evaluation 
685  should include the plans for verification and validation of the entire device following ML-DSF 
686  modifications for each individual modification and in aggregate for all implemented 
687  modifications. This includes, but is not limited to, ML model testing protocols comparing the 
688  newly modified device to both the original device (the version of the device without any 
689  modifications implemented) and the last modified version of the device. For example, for device 
690  software functions that drive hardware functionality, performance evaluation should include not 
691  only the device software functions, but also the effect of the modifications on hardware 
692  functionality. The content of this section in a Modification Protocol should provide details on the 
693  study design, performance metrics, pre-defined acceptance criteria, and statistical tests for each 
694  planned modification. More comprehensive testing can potentially support a broader set of 
695  proposed modifications.  
696   
697  Why they are recommended: Information regarding the manufacturers performance evaluation 
698  methods allow FDA to confirm that appropriate study designs, including performance metrics 
699  and statistical tests, will be used to evaluate the effect of modifications on overall device 
700  performance. Performance evaluation of the device is essential to ensure that specified 
701  acceptance criteria for all proposed modifications will continue to be met over the range of FDA-
702  authorized specifications.  
703   
704  What manufacturers should include in a submission: Examples of the types of information 
705  manufacturers should provide in a Modification Protocol describing their performance evaluation 
706  are provided in Appendix A. In general, this information should describe how performance 
707  evaluation will be triggered; how sequestered test data representative of the clinical population 
708  and intended use will be applied for testing; what performance metrics will be computed; and 
709  what statistical analysis plans will be employed to test hypotheses relevant to performance 
710  objectives for each modification. In addition, the Modification Protocol should affirmatively 
711  state that if there is a failure in performance evaluation for a specific modification, the failure(s) 
712  will be recorded, and the modification will not be implemented.  
713   
714  (4)  Update procedures  
715  What they are: Data management practices, re-training practices, and performance evaluation 
716  described above largely relate to making and testing modifications to the ML-DSF. Once these 
717  meet the performance objectives, manufacturers will need to update the ML-DSF to implement 
718  the modifications and communicate information to users about the modifications that is needed 
 
77 Section 515C(a)(3) and (b)(3) of the FD&C Act. 
22  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
719  to safely use the device. The update procedures in a Modification Protocol should describe how 
720  manufacturers will update their devices to implement the modifications, provide appropriate 
721  transparency to users, and, if appropriate, updated user training about the modifications and 
722  perform real-world monitoring, including notification requirements if the device does not 
723  function as intended pursuant to the authorized PCCP.78 The PCCP should include a description 
724  of any labeling changes that will result from the implementation of the modifications. The 
725  available labeling must include adequate directions for use and reflect information about the 
726  current version(s) of the ML-DSF available to the user, including information regarding site-
727  specific modifications.79 The labeling should not reflect information on modifications to the ML-
728  DSF that have not been implemented in the available version, because it could cause confusion 
729  and would be misleading. If such information is included in the labeling, the ML-DSF could be 
730  deemed misbranded.80 
731   
732  Why they are recommended: Information on the manufacturers update procedures allows FDA 
733  to understand 1) how risks from implementing modifications may be mitigated by the update 
734  process; 2) how communication regarding the device updates will be provided to users (e.g., so 
735  that updates in device output results will be correctly interpreted by users); 3) how the device 
736  operation will remain reliable after the update; and 4) how all stakeholders will be kept up-to-
737  date about device functionality and performance. In addition, it is important for FDA to 
738  understand how potential risks associated with the update process, itself, may be mitigated.  
739   
740  What manufacturers should include in a submission: Examples of the types of information 
741  manufacturers should provide in a Modification Protocol describing their update procedures are 
742  provided in Appendix A. In general, this information should include 1) confirmation that the 
743  verification and validation plans for the modified version of the device are the same as those that 
744  have been performed for the version of the device prior to the implementation of the 
745  modifications, or identification of any differences between the two plans; 2) a description of how 
746  software updates will be implemented; 3) a description of how legacy users will be affected by 
747  the software update (if applicable); and 4) a description of how modifications will be 
748  communicated to the users, including transparency on any differences in performance, 
749  differences in performance testing methods, and/or known issues that were addressed in the 
750  update (e.g., whether there is an improvement in performance in a subpopulation of patients). 
751  Communication of performance changes should be consistent with performance evaluation 
752  described in the Modification Protocol. 
753   
754  C.  Traceability Between the Description of Modifications 
755  Section and the Modification Protocol Section 
 
78 Section 515C(a)(3) and (b)(3) of the FD&C Act. 
79 See 21 CFR 801.5, requiring that labeling include adequate directions for use including statements of all 
conditions, purposes, or uses for which the device is intended. 
80 See section 502(a)(1) of the FD&C Act, stating that a medical device is deemed misbranded if its labeling is false 
or misleading in any particular. 
23  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
756  The PCCP should clearly delineate which parts of the Modification Protocol are applicable to 
757  each modification within the Description of Modifications. For a PCCP with multiple 
758  modifications, this may be accomplished through a traceability table; a sample traceability table 
759  is provided below in Table 1. This sample traceability table provides an example of how a 
760  manufacturer can depict the traceability between the Description of Modifications and 
761  Modification Protocol, as well as how to provide clear references to where within the PCCP this 
762  information is located in a marketing submission.  
763   
764  Table 1. Example of Description of Modifications to Modification Protocol Traceability 
765  Table  
766   
767  Table 1: A traceability table can help to identify where each method supporting each modification may be found in the marketing 
768  submission. 
  Modification Protocol Component 
Data management  Re-training  Performance  Update 
Modification 
practices  practices  evaluation  procedures 
Method A   Method D   Method G   Method J  
Modification #1 
(see Section X.A)  (see Section X.D)  (see Section X.G)  (see Section X.J) 
Method A   Method E   Method H   Method J  
Modification #2 
(see Section X.A)  (see Section X.E)  (see Section X.H)  (see Section X.J) 
Method B   Method F   Method I   Method J  
Modification #3 
(see Section X.B)  (see Section X.F)  (see Section X.I)  (see Section X.J) 
769   
770  VIII.  Impact Assessment 
771  An Impact Assessment, in the context of a PCCP, is the documentation of the assessment of the 
772  benefits and risks of implementing a PCCP for an ML-DSF, as well as the mitigations of those 
773  risks. The manufacturers existing quality system should be used as the framework in which to 
774  conduct an Impact Assessment for the modifications set forth in the PCCP.  
775   
776  Documentation for an Impact Assessment provided to the Agency in a marketing submission 
777  containing a PCCP should 1) compare the version of the device with each modification 
778  implemented to the version of the device without any modifications implemented, 2) discuss the 
779  benefits and risks, including risks of social harm, of each individual modification, and 3) discuss 
780  how the activities proposed within the Modification Protocol continue to reasonably ensure the 
781  safety and effectiveness of the device. The Impact Assessment documentation in a marketing 
782  submission should also discuss 4) how the implementation of one modification impacts the 
783  implementation of another, and 5) the collective impact of implementing all modifications. FDA 
24  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
784  believes it is important to address these elements in an Impact Assessment in order to 
785  demonstrate that the combination of the proposed modifications is unlikely to introduce 
786  additional, unmitigated risks, and that the safety and effectiveness of the device under review is 
787  maintained as modifications are implemented. 
788   
789  Impact Assessment documentation in a marketing submission should discuss how the individual 
790  modifications included in the PCCP impact not only the ML-DSF, but also how they impact the 
791  overall functionality of the device, including how they impact other device software functions, as 
792  well as device hardware. Additionally, if the ML-DSF is a device function of a multiple function 
793  device product, we recommend considering FDAs guidance Multiple Function Device Products: 
794  Policy and Considerations81 to determine if any information should be included in the Impact 
795  Assessment documentation in a marketing submission (and the marketing submission overall) so 
796  that FDA may assess the impact of the other function(s) on the safety or effectiveness of the 
797  ML-DSF and modifications to the ML-DSF. 
798   
799  The Agency acknowledges that for some devices, it may be least burdensome to include the 
800  content of the Impact Assessment within the Modification Protocol rather than as a separate 
801  section within the PCCP.
 
81 See FDAs guidance Multiple Function Device Products: Policy and Considerations available at 
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/multiple-function-device-products-
policy-and-considerations. 
25  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Appendix A: Example Elements of Modification Protocol 
802 
Components for ML-DSFs 
803 
804  In general, a Modification Protocol that is included as part of a PCCP in a marketing submission 
805  should include four components that outline a manufacturers 1) data management practices, 2) 
806  re-training practices, 3) performance evaluation protocols, and 4) update procedures, for each 
807  modification in the Description of Modifications for the ML-DSF. However, manufacturers may 
808  include other or additional components if they believe that their proposed protocols do not fit 
809  into any of these four components. Examples of questions for consideration and the types of 
810  information manufacturers should provide in these components of a Modification Protocol are 
811  provided below. 
812   
813  Note that this is a developing area, and as FDA gains experience, these example questions may 
814  change. The items below are also not an exhaustive list of topics that a manufacturer is expected 
815  to cover, and all questions may not apply to all marketing submissions. Likewise, the Agency 
816  may request additional Modification Protocol components or information to be included in a 
817  PCCP for some device types so that the Agency can review the PCCP as part of the marketing 
818  submission.82 Some sections of a Modification Protocol may be more or less detailed depending 
819  on the complexity and risks of each modification in the PCCP. 
820   
821  (1)  Data Management  
822  Different data can be collected and used for training and testing ML model updates. For 
823  collection of new training and testing data, the Modification Protocol should include how the 
824  data will be used (e.g., for ML model development or testing), and how the data management 
825  supports these uses. 
826  a.  Collection protocols  
827  1.a.1.  For each modification, what are the inclusion/exclusion criteria for data 
828  collection, and how are they linked to the intended use population?  
829  1.a.2.  What is the intended distribution of your data set along covariates describing the 
830  patient population (e.g., sex, age, race, height, weight, disease conditions) and 
831  data acquisition conditions (e.g., sites, data acquisition devices/methods, imaging 
832  and reconstruction protocols)? Is this distribution representative of the intended 
833  use population, including intersectional groups? 
834  1.a.3.  Will the data be collected prospectively or retrospectively? Will the data set 
835  include consecutive cases within a given date range? Otherwise, if random 
836  sampling is planned, what method or technique will be used and how will it 
837  account for bias and randomness? 
838  1.a.4.  Are there any plans for enrichment or stratified sampling to include specific 
839  patient subgroups (e.g., sex, age, race)? 
 
82 Such information would be required in the marketing submission pursuant to 21 CFR 807.81(a)(3) and 21 CFR 
814.39(a). 
26  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
840  1.a.5.  What is the number and geographical distribution of data collection sites?  
841  1.a.6.  What are the measures to mitigate potential unwanted bias in learning or 
842  performance estimation, for example, due to issues related to new training or 
843  testing data, respectively?  
844  1.a.7.  What are the strategies and measures to understand and mitigate potential biases 
845  in the data, such as those due to historical inequalities to medical treatment access 
846  by different populations? 
847  1.a.8.  What are the strategies to ensure data sets remain relevant over time with respect 
848  to changes in, for example, data acquisition technologies or protocols, clinical 
849  practice, patient populations, and disease conditions? 
850  1.a.9.  Are data collection, storage, retention, and use protocols in compliance with 
851  regulations for human subject protections and requirements for clinical 
852  investigations (e.g., pursuant to 21 CFR Part 812, 45 CFR Part 46, 21 CFR Part 
853  50, and 21 CFR Part 506, as applicable)? 
854  1.a.10. For ML-DSFs that use input data from dedicated acquisition systems (e.g., 
855  software device functions in a patient monitor that uses connected sensors), is the 
856  data acquired with the systems and settings with which the ML-DSF will be used? 
857  For device software functions that use input data from different acquisition 
858  systems (e.g., interoperable medical devices), does the data acquired meet the 
859  input specifications of the ML-DSF? 
860  b.  Assurance of Data Quality  
861  1.b.1.  What techniques will be employed to fortify data consistency and completeness?  
862  1.b.2.  What are the strategies used to promote data authenticity, transparency, and 
863  integrity? 
864  1.b.3.  How will potentially missing data elements within a case/record be handled?  
865  1.b.4.  Are there criteria for including/excluding cases/records based on data quality (in 
866  addition to inclusion/exclusion criteria listed in 1.a.1. above), and if so, what are 
867  the criteria and rationale?  
868  1.b.5.  If data might be excluded as a result of the quality assurance process, what 
869  methods are planned to minimize the impact on the generalizability of training 
870  and accuracy of testing?  
871  1.b.6.  Are there strategies to trace a data issue to an individual record? What are the 
872  strategies to identify and investigate data issues? 
873  1.b.7.  Will data that are obsolete or no longer needed be removed? 
874  1.b.8.  What controls are in place to prevent and identify unauthorized access or 
875  manipulation of the training and testing data sets? For example, what controls are 
876  in place to prevent malicious addition or deletion of data for the purpose of 
877  adversarial machine learning? 
878  1.b.9.  Is there an automated process to ensure data quality?  
879  c.  Reference standard determination  
880  For the purposes of this guidance, the reference standard is the best available truth standard to 
881  define the true condition for each patient/case/record. The reference standard may be used in 
882  device training, device testing, or both.  
27  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
883   
884  1.c.1.  What is the justification for the method for the determination of the reference 
885  standard? 
886  1.c.2.  If the reference standard is based on evaluations from clinicians, what was the 
887  grading protocol used (e.g., what are the total number of clinicians who 
888  participated and their qualifications; what data are these clinicians provided with; 
889  and how are the clinicians evaluations collected/adjudicated for determining the 
890  reference standard)?  
891  1.c.3.  What is the strategy for addressing cases where results obtained using a reference 
892  standard may be equivocal or missing?  
893  1.c.4.  What is the uncertainty inherent in your reference standard? 
894  1.c.5.  Will any of the methods for determining the reference standard be automated? 
895  Are there differences between the reference standards used for training versus 
896  testing? 
897  1.c.6.  Are there differences between the reference standard used to support the initial 
898  ML-DSF marketing authorization and the reference standard being applied to 
899  update the ML model? 
900  d.  Sequestration of test data sets  
901  For the purposes of this guidance, sequestration of test data sets means that manufacturers do not 
902  have access to the test data set for the purpose of ML-DSF development.  
903   
904  1.d.1.  What strategies will be employed at the outset of data collection to shield the test 
905  data set from the ML-DSF development? 
906  1.d.2.  What are the specific procedures to be followed so that the test data set remains 
907  sequestered during re-training?  
908  1.d.3.  If test data are planned to be used multiple times for performance evaluation, what 
909  measures are in place to prevent unwanted bias from being introduced through 
910  ML model manufacturers learning substantial information about the test data set 
911  and results?  
912   
913  (2)  Re-Training 
914  a.  Re-training objectives and focus 
915  2.a.1.  How are the modifications presented in the Description of Modifications in the 
916  PCCP related to the planned re-training methods?  
917  2.a.2.  Which parts of the ML-DSF are planned to be modified (e.g., transfer learning, 
918  data pre-processing, data augmentation, only a certain set of coefficients, ML 
919  architecture and hyper-parameters, loss functions, optimization methods and 
920  criteria, types of ML model inputs and outputs), and what are the details of the 
921  planned modifications to the ML-DSF design? What is the specific rationale for 
922  the change to each part that is planned to be modified? 
923  2.a.3.  For each part of the ML-DSF that will be modified, is ML model re-training 
924  needed to achieve the modifications specified in the PCCP? 
28  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
925  2.a.4.  If re-training applies to only certain parts of the ML-DSF, what are the plans to 
926  ensure that other functions or software components are not affected? 
927  b.  Re-training implementation:  
928  2.b.1.  What are the triggers for re-training (e.g., when new data reaches a certain size, 
929  when a drift is observed, periodically in time)?  
930  2.b.2.  What strategies will be employed to identify and limit overfitting?  
931  2.b.3.  Are there risks related to ML model bias introduced by re-training a modified ML 
932  model, and if so, what are planned mitigations? 
933   
934  (3)  Performance Evaluation  
935  a.  Triggers to initiate performance evaluation  
936  3.a.1.  What are the triggers for initiating performance evaluation of a re-trained ML 
937  model or modified ML-DSF (e.g., re-training shows a certain performance level 
938  on the training or validation data, test data reaches a certain size, periodically in 
939  time)?  
940  3.a.2.  How frequently is this expected to occur?  
941  b.  Assessment metrics and elements  
942  3.b.1.  How is the Data Management Plan in (1) above applied to produce the testing 
943  data for performance evaluation that are different from any training or tuning 
944  data? 
945  3.b.2.  What metrics will be computed to understand device performance?  
946  3.b.3.  How do these metrics demonstrate that the modified device can be safely used?  
947  3.b.4.  How will the metrics provide a comprehensive assessment of device performance 
948  and patient safety?  
949  3.b.5.  What corner cases (i.e., cases outside the norm) or known challenging scenarios 
950  will be evaluated? 
951  c.  Statistical analysis plans  
952  3.c.1.  What is the plan for evaluating equivalent or improved performance with respect 
953  to previously validated versions, including the original version, of the ML-DSF?  
954  3.c.2.  What are the high-risk subpopulations and subgroups (e.g., sex, gender 
955  differences, acquisition protocols) that need to be evaluated?  
956  3.c.3.  How will this evaluation be used to support labeling specifications?  
957  3.c.4.  How will you test that performance in one area (e.g., sensitivity) does not result in 
958  degrading performance in another (e.g., specificity)?  
959  3.c.5.  How will the sample size be determined? 
960  3.c.6.  Is the primary analysis based on the intention-to-diagnose population (no study 
961  subjects will be excluded) or the per-protocol population (subjects with protocol 
962  violations will be excluded)? 
963  3.c.7.  How is variability in the reference standard accounted for (e.g., in the case of 
964  reader variability when clinical interpretation is used)? When the reference 
29  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
965  standard may be imperfect (e.g., sometimes includes a diagnostic error), are errors 
966  made by the imperfect reference standard conditionally independent of errors 
967  made by the ML-DSF, or are they positively correlated?  
968  3.c.8.  How will missing data be addressed in analysis?  
969  d.  Performance targets  
970  3.d.1.  What are the acceptance criteria?  
971  3.d.2.  What clinical considerations were used to develop the acceptance criteria? 
972  3.d.3.  How will the acceptance criteria support that the modification will be successfully 
973  implemented? 
974  e.  Additional testing needs  
975  3.e.1.  Is database testing sufficient to address the risks associated with the proposed 
976  modification (e.g., does the user need to interact with the device to evaluate the 
977  performance or address a clinical risk or, for software that is part of a hardware 
978  device,83 how is the effect of the modification on hardware functionality 
979  evaluated)? 
980  3.e.2.  How may clinical usability need to be addressed for a modification? 
981   
982  (4)  Update Procedures 
983  a.  Software verification and validation  
984  4.a.1.  Does the proposed modification necessitate a different software verification and 
985  validation plan from that used for the version of the device without any 
986  modifications implemented?  
987  4.a.2.  What type of testing will be performed? Will the modified device be validated to 
988  function in an integrated environment? 
989  4.a.3.  If the device includes other device functions in addition to the ML-DSF, how will 
990  the impact of modifications to the ML-DSF on the other device functions be 
991  evaluated? 
992  4.a.4.  If the device includes other functions in addition to the ML-DSF, how will the 
993  other functions impact on the safety or effectiveness of the modified ML-DSF 
994  be evaluated? 
995  b.  When and how updates will be implemented  
996  4.b.1.  How will the decision be made on when to perform an update? What is the 
997  expected timeline for implementing the modifications? Is there a set frequency of 
998  updates?  
999  4.b.2.  When and how will an update be implemented (e.g., automatically when the 
1000  device is not being used, manually by users or hospital technicians, or both 
 
83 For the purposes of this guidance, part of a hardware device means the software is used to control a device, or 
the software is necessary for a hardware device to achieve its intended use. 
30  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
1001  manually and automatically)? What is the basis on which the mechanism of 
1002  implementation is dependent?   
1003  4.b.3.  For ML-DSF updates to reusable medical equipment, how will the device 
1004  operation, including function of critical safety features (e.g., medical device 
1005  alarms), be verified following the update? 
1006  4.b.4.  Will updates be made globally (i.e., the same update applied to all devices in the 
1007  field) or locally (e.g., the devices may be modified for a patient/provider/care 
1008  unit/hospital)?  
1009  4.b.5.  What cybersecurity protocols84 will be applied during updates? 
1010  c.  Communication and transparency to users  
1011  4.c.1.  How will the PCCP be described in the public summary document and/or 
1012  labeling? 
1013  4.c.2.  How will updates be communicated to users, including, but not limited to, in 
1014  updated labeling (e.g., release notes)?  
1015  4.c.3.  What information about modifications to the device (e.g., performance) will be 
1016  communicated to the user? 
1017  4.c.4.  How will version information be presented to the user when reviewing device 
1018  outputs? 
1019  4.c.5.  Will users have the option to review labeling before implementing an update?  
1020  4.c.6.  How will any known biases or other performance issues with the potential to 
1021  result in individual or social harms be disclosed, including, but not limited to, in 
1022  labeling?  
1023  4.c.7.  How will any changes in performance related to known issues or sources of bias 
1024  be communicated to the user, including, but not limited to, in labeling? 
1025  4.c.8.  What information about the population and methods for validation will be 
1026  provided?  
1027  4.c.9.  If patient data from previous device use is available and can be rerun on an 
1028  updated ML model, will this activity be performed for the available data and will 
1029  those updated results be available to patients and users? Is there a plan to 
1030  communicate if patient results before and after an update would provide clinically 
1031  meaningful differences? 
1032  d.  Device monitoring plan  
1033  4.d.1.  How will adverse events be tracked for different updates?  
1034  4.d.2.  Is there a plan to monitor real-world device performance (beyond general 
1035  controls) and, if not, why is it not necessary?  
1036  4.d.3.  How will changes in performance in patient subpopulations be identified?  
 
84 For recommendations related to cybersecurity, please consult FDA guidance documents on this topic, including 
Content of Premarket Submissions for Management of Cybersecurity in Medical Devices (available at 
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/content-premarket-submissions-
management-cybersecurity-medical-devices) and Postmarket Management of Cybersecurity in Medical Devices 
(available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/postmarket-management-
cybersecurity-medical-devices).  
31  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
1037  4.d.4.  What will be the response to the identification of previously unknown risks or 
1038  previously unrecognized high-risk patient subpopulations?  
1039  4.d.5.  What is the strategy to respond to unexpected performance deficiencies or other 
1040  hazards, or to higher levels of adverse events, as compared with previous 
1041  iterations of the device? 
1042  4.d.6.  How will errors in diagnosis (i.e., misdiagnosis), attributable or partially 
1043  attributable to the device that do not meet the criteria for an adverse event, be 
1044  tracked? 
 
1045   
32  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Appendix B: Example ML-DSF Scenarios Employing 
1046 
PCCPs  
1047 
1048  The examples in this appendix illustrate different ML-DSF scenarios where a PCCP could be 
1049  employed. Due to the complexity of ML-DSFs, all examples are hypothetical and do not reflect 
1050  any specific authorized device.  
1051   
1052  Each example begins with a brief description of an authorized device, its intended use, and one 
1053  summary of a modification from the Description of Modifications in its authorized PCCP (in the 
1054  examples, denoted as Brief Overview of Pre-Specified Modification). Please note that the 
1055  provided summaries of the devices and modifications in this appendix are not intended to reflect 
1056  the complete content or detail expected in a Description of Modifications section in a PCCP. 
1057  Rather, proposed modifications should be described in much greater detail in a PCCP, consistent 
1058  with the recommendations provided throughout this guidance. The post-authorization 
1059  modification scenarios are described to illustrate how the PCCP would be implemented. A 
1060  distinction is drawn between post-authorization modifications that 1) would be acceptable for the 
1061  authorized device with a PCCP and could be implemented without a new marketing submission 
1062  or 2) would not be acceptable for the authorized device with a PCCP and likely requires a new 
1063  marketing submission85 before the device could be introduced into interstate commerce.  
1064   
1065  Due to the complexity of ML-DSFs, it is not practical to describe all relevant considerations, or a 
1066  complete PCCP, for the limited examples presented below. Therefore, while these examples 
1067  highlight important concepts that could inform the development and utility of a PCCP, the PCCP 
1068  will be specific to the circumstances of a particular ML-DSF, based on factors including a 
1069  scientifically valid assessment of benefits and risks.  
1070   
1071  FDA recommends that the PCCP strategy be discussed with the appropriate FDA review division 
1072  through the Q-Submission program86 prior to submitting a marketing submission containing a 
1073  PCCP. As part of a marketing submission, the manufacturer should provide a PCCP, consisting 
1074  of a Description of Modifications (Section VI), a Modification Protocol (Section VII), and an 
1075  Impact Assessment (Section VIII).  
1076   
1077  (1)  Patient Monitor Software 
1078   
1079  Background: 
1080   
1081  The device is an ML-DSF intended for use in high-acuity healthcare environments (e.g., an 
1082  intensive care unit). The software obtains physiological signals (e.g., electrocardiogram, blood 
1083  pressure, pulse oximetry) from a primary patient monitor. The physiological signals are 
 
85 21 CFR 807.81(a)(3) or 21 CFR 814.39(a). 
86 See FDAs guidance Requests for Feedback and Meetings for Medical Device Submissions: The Q-Submission 
Program available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/requests-
feedback-and-meetings-medical-device-submissions-q-submission-program.  
33  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
1084  processed and analyzed by an ML model to detect patterns that occur at the onset of physiologic 
1085  instability. When physiologic instability is detected, an audible alarm signal is generated to 
1086  indicate that prompt clinical action is needed to prevent potential harm to the patient. The ML-
1087  enabled medical device was authorized with a PCCP. 
1088   
1089  Brief Overview of Pre-Specified Modification: 
1090   
1091  The manufacturer would like to re-train the ML model with more data to reduce the false alarm 
1092  rate while maintaining or increasing sensitivity to the onset of physiologic instability. The 
1093  baseline sensitivity is y%. The manufacturer would like to demonstrate a significant 
1094  improvement in the false-alarm rate while the sensitivity remains within a pre-specified  
1095  non-inferiority margin of z% when compared with the original device, i.e., the version of the 
1096  device without any modifications.87  
1097   
1098   
1099  Post-Authorization Modification Scenarios: 
1100   
1101  Modification Scenario 1: Modification related to quantitative measures of device performance, 
1102  as specified in the PCCP, and implemented in accordance with the PCCP 
1103   
1104  In accordance with the Modification Protocol, data were collected and used to re-train the 
1105  ML model. The modified ML model was tested per the methods specified in the 
1106  Modification Protocol. The results demonstrated that the false alarm rate was 
1107  significantly reduced while the mean sensitivity estimate was statistically within the 
1108  proposed non-inferiority margin of the baseline sensitivity y%. Labeling was updated in 
1109  accordance with the modified ML-DSF performance, and communication was provided 
1110  to the device users. Because the device modification was specified in the PCCP, and it 
1111  was implemented in conformance with the PCCP, the device modification would not 
1112  require a new marketing submission. The manufacturer should document the 
1113  modification that was specified in the PCCP in accordance with their quality system. 
1114   
1115  Modification Scenario 2: Modification beyond quantitative measures of device performance, 
1116  which was not specified in the PCCP  
1117   
1118  In accordance with the Modification Protocol, the manufacturer re-trained their ML 
1119  model using additional data to improve the sensitivity. Analytical validation 
1120  demonstrated that the revised ML model has the same false alarm rate and sensitivity as 
1121  the previous version. However, the manufacturer also noticed that the modified ML 
1122  model maintained the same sensitivity and can now also predict physiologic instability in 
1123  advance of its onset, which the previous version of the ML model could not do. The 
1124  manufacturer would like to update the devices indications for use to reflect this 
1125  additional performance claim to predict physiologic instability in advance of its onset, 
 
87 The values in this example are shown as variable terms. A completed PCCP should include specific criteria 
whenever possible.  
34  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
1126  which was not previously included in the PCCP. The methods used for analysis, 
1127  performance, and statistics were not specified in the PCCP for predicting a future state. 
1128  Because this modification that was not included in the PCCP could significantly affect 
1129  the safety or effectiveness of the device, a new marketing submission would be required. 
1130   
1131  (2)  Skin Lesion Software 
1132   
1133  Background: 
1134   
1135  The device is an ML-DSF that analyzes images of skin lesions by identifying and characterizing 
1136  its features (e.g., color, quantification of area change over time) to aid in diagnosis. It was 
1137  validated with a specific camera and is intended to be used by a primary healthcare provider. The 
1138  device was authorized with a PCCP. 
1139   
1140  Brief Overview of Pre-Specified Modification: 
1141   
1142  The manufacturer would like to extend the ML-DSF for use on additional general-purpose 
1143  computing platforms, including smartphones and tablets. The general-purpose computing 
1144  platform must include a two-dimensional camera that meets the minimum specifications defined 
1145  in the PCCP. The updated device must achieve a minimum performance defined in the 
1146  Modification Protocol using a specified methodology.  
1147   
1148  Post-Authorization Modification Scenarios: 
1149   
1150  Modification Scenario 1: Modification in input, as specified in the PCCP and implemented in 
1151  accordance with the PCCP 
1152   
1153  The manufacturers analytical validation demonstrated the ML-DSF can be deployed on 
1154  two additional smartphones that have image acquisition specifications that meet the 
1155  minimum specifications provided in the PCCP. The analytical performance using the new 
1156  image acquisition systems was found to be statistically equivalent to the baseline 
1157  performance, as specified in the Modification Protocol. Labeling was updated to reflect 
1158  the new ML-DSF compatibility with the additional smartphones, which may increase 
1159  access of the ML-DSF in the healthcare community. Communication updates on device 
1160  compatibility were also provided. Because the device modification was specified in the 
1161  PCCP, and it was implemented in conformance with the PCCP, the device modification 
1162  would not require a new marketing submission. The manufacturer should document the 
1163  modification that was specified in the PCCP in accordance with their quality system. 
1164   
1165  Modification Scenario 2: Modification in input that was not specified in the PCCP  
1166   
1167  The manufacturer would like to deploy a modified ML model that uses images captured 
1168  by a thermographic camera; however, the new camera technology was not specified in 
1169  the PCCP. Because this modification that was not included in the PCCP could 
35  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
1170  significantly affect the safety or effectiveness of the device, a new marketing submission 
1171  would be required. 
1172   
1173  Modification Scenario 3: Modification related to the devices use and performance, which was 
1174  not specified in the PCCP 
1175   
1176  The manufacturer would like to distribute a new version of the ML-DSF that is patient-
1177  facing. The ML-DSF would provide an analysis of the physiological characteristics of 
1178  skin lesions, as it does currently, and direct patients to follow-up with a dermatologist 
1179  based on the preliminary analysis of the malignancy of the skin lesion. The modification 
1180  introduces many new, unconsidered risks that were not yet mitigated in the current 
1181  PCCP, given that the modified ML-DSF will be patient-facing. Because this modification 
1182  that was not included in the PCCP could significantly affect the safety or effectiveness of 
1183  the device, a new marketing submission would be required. 
1184   
1185  (3)  Ventilator Settings Software 
1186   
1187  Background: 
1188   
1189  The device is an ML-DSF intended for use in the healthcare or home-use setting. The ML-DSF 
1190  recommends the ideal ventilation parameters based on input data interpretation, which can then 
1191  be programmed into the ventilator by a healthcare provider. The manufacturer proposes 
1192  modifications to the ML-DSF to improve performance within the original indications. The 
1193  device was authorized with a PCCP. 
1194    
1195  Brief Overview of Pre-Specified Modification: 
1196   
1197  The manufacturer would like to re-train the ML model to optimize site-specific performance for 
1198  a specific subset of patients with a particular condition, for whom sufficient data were not 
1199  previously available. Specifically, the manufacturer would like to modify the ML model to 
1200  improve its ability to optimize ventilator settings for minute volume and tidal volume to reduce 
1201  the variability to x% within the specified range to improve treatment outcomes for that subset 
1202  of patients at different sites. 
1203   
1204  Post-Authorization Modification Scenarios: 
1205   
1206  Modification Scenario 1: Modification related to the devices use and performance in a subset of 
1207  the patient population, which was specified in the PCCP and implemented in accordance with 
1208  the PCCP 
1209 
 
1210  The manufacturer re-trained and re-validated the ML model on newly acquired data in a 
1211  subpopulation of patients with a particular disorder. As evidenced by additional clinical 
1212  performance data collected and analyzed per the Modification Protocol, the re-training on 
1213  new data improved the reliability and precision of ventilator setting recommendations, 
36  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
1214  showing improvements and specializations to improve site-specific ventilator operation. 
1215  The updated recommendations were validated against patient outcomes and adverse 
1216  events that may occur due to ventilator setting inaccuracies following the methods in the 
1217  Modification Protocol. The adverse event rates and outcomes acceptance criteria were 
1218  established in the Modification Protocol, and as such, were used to validate the updated 
1219  ML model. The ML-DSF was updated to implement the re-trained ML model and the 
1220  labeling was updated for clarity to inform users how the updated ML model accounts for 
1221  local experience and prevalence. The implementation of this modification was done only 
1222  at applicable sites. Because this device modification was specified in the PCCP, and it 
1223  was implemented in conformance with the PCCP, the device modification would not 
1224  require a new marketing submission. The manufacturer should document the 
1225  modification that was specified in the PCCP in accordance with their quality system. 
1226   
1227  Modification Scenario 2: Modification related to devices use and performance in a subset of the 
1228  patient population, which was specified in the PCCP, but was not implemented in conformance 
1229  with the PCCP  
1230     
1231  The manufacturer re-trained and re-validated the ML model on newly acquired data, but 
1232  was unable to fulfill the protocol because the manufacturer had to implement a reference 
1233  standard that was different from the one described in the Modification Protocol. Even 
1234  though the modification was specified in the PCCP, it was not implemented in 
1235  conformance with the PCCP. Because this modification that was not implemented in 
1236  conformance with the PCCP could significantly affect the safety or effectiveness of the 
1237  device, a new marketing submission would be required. 
1238   
1239  (4)  Image Acquisition Assistance Device 
1240  Background:  
1241  The ML-DSF is integrated into an imaging system and is intended to assist healthcare providers 
1242  during acquisition of ultrasound images of the shoulder region in adult and pediatric populations 
1243  by highlighting portions of the image where it detects a potential abnormality in real time. The 
1244  ML-DSF interfaces with the device acquisition system, analyzes its output using an ML model, 
1245  provides real-time alerts to the operator if an abnormality is detected, and automatically adjusts 
1246  parameters in the device acquisition system during image acquisition to optimize the imaging. 
1247  The device does not provide a diagnosis. The ML-enabled medical device was authorized with a 
1248  PCCP. 
1249  Brief Overview of Pre-Specified Modification: 
1250  The manufacturer would like to re-train their ML model to further optimize the accuracy of the 
1251  abnormality detection. The PCCP pre-specifies that both the sensitivity and specificity will be 
1252  shown to be significantly superior for abnormality identification during the shoulder exam. 
37  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
1253  Post-Authorization Modification Scenario:  
1254  Modification Scenario 1: Modification related to quantitative measures of device performance, 
1255  as specified in the PCCP, and implemented in accordance with the PCCP 
1256  In accordance with the Modification Protocol, imaging data were collected and used to 
1257  re-train the ML model. The modified ML model was tested according to a specified test 
1258  protocol in the Modification Protocol. The results demonstrated that the sensitivity and 
1259  specificity for abnormality identification met statistical superiority pre-specifications. 
1260  Labeling was updated in accordance with the modified device performance, and 
1261  communication was provided to the device users. Because the device modification was 
1262  specified in the PCCP, and because it was implemented in conformance with the PCCP, 
1263  the device modification would not require a new marketing submission. The 
1264  manufacturer should document the modification that was specified in the PCCP in 
1265  accordance with their quality system. 
1266  Modification Scenario 2: Modification related to the devices use and performance that was not 
1267  specified in the PCCP 
1268  The manufacturer used new images to re-train the ML model and would like to update 
1269  their labeling to reflect improved performance in the same shoulder region in a subset of 
1270  the pediatric patient population identified in the devices indications for use. However, 
1271  the modification was not specified in the PCCP. Because this modification that was not 
1272  included in the PCCP could significantly affect the safety or effectiveness of the device, a 
1273  new marketing submission would be required.  
1274   
1275  (5)  Feeding Tube Placement Radiograph Analysis Software 
1276   
1277  Background: 
1278   
1279  The device is an ML-DSF that analyzes chest radiographs from hospitalized patients to evaluate 
1280  feeding tube placement. The ML-DSF reorders the radiologists review queue so that 
1281  radiographs identified as having a higher likelihood of feeding tube misplacement are placed 
1282  higher in the queue. The device was authorized with a PCCP. 
1283   
1284  Brief Overview of Pre-Specified Modification: 
1285   
1286  The manufacturer would like to improve ML model performance by increasing sensitivity to 
1287  misplaced feeding tubes by re-training on new data. The baseline sensitivity is x%.  
1288   
1289  Additionally, the manufacturer would like to modify the device so that it would notify nursing 
1290  staff to check on the patient, in parallel with prioritization in the review queue. This modification 
1291  requires achieving a sensitivity of z%.  
1292   
38  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
1293  Post-Authorization Modification Scenarios: 
1294   
1295  Modification Scenario 1: Modification related to quantitative measures of device performance, 
1296  and modification related to devices use and performance, which were both specified in the 
1297  PCCP and implemented in accordance with the PCCP 
1298   
1299  The manufacturer re-trained and re-validated the ML model on newly acquired data as 
1300  described in the Modification Protocol, which significantly improved the ML-DSF 
1301  sensitivity from x% to z% to detect incorrect feeding tube placements. The improved 
1302  sensitivity achieved the required sensitivity (z%) statistically to enable the nursing staff 
1303  notification function, and that notification function was enabled. Labeling of the device 
1304  was changed in accordance with the PCCP. Because the device modification was 
1305  specified in the PCCP, and it was implemented in conformance with the PCCP, the 
1306  device modification would not require a new marketing submission. The manufacturer 
1307  should document the modification that was specified in the PCCP in accordance with 
1308  their quality system. 
1309   
1310  Modification Scenario 2: Modification related to devices use and performance that was not 
1311  specified in the PCCP  
1312   
1313  The manufacturer used the same database of images to re-train the ML model to identify 
1314  pneumonia on chest radiographs. The pneumonia identification function was found to 
1315  have the same sensitivity and specificity as the feeding tube ML model. The 
1316  manufacturer would like to employ the new pneumonia identification function feature 
1317  alongside the feeding tube placement ML model in radiograph triage. The modification 
1318  was not specified in the PCCP. Because this modification that was not included in the 
1319  PCCP could significantly affect the safety or effectiveness of the device, a new marketing 
1320  submission would be required. 
39