Submission of Quality 
Metrics Data
  
Guidance for Industry 
 
 
 
 
 
 
DRAFT GUIDANCE
 
 
  This guidance document is being distributed for comment purposes only. 
 
Comments and suggestions regarding this draft document should be submitted within 60 days of 
publication in the Federal Register of the notice announcing the availability of the draft 
guidance.  Submit electronic comments to http://www.regulations.gov.  Submit written 
comments to the Division of Dockets Management (HFA-305), Food and Drug Administration, 
5630 Fishers Lane, rm. 1061, Rockville, MD  20852.  All comments should be identified with 
the docket number listed in the notice of availability that publishes in the Federal Register. 
 
For questions regarding this draft document contact (CDER) Tara Gooen Bizjak at 301-796-3257 
or (CBER) Office of Communication, Outreach and Development at 1-800-835-4709 or 240-
402-8010.  
 
 
U.S. Department of Health and Human Services 
Food and Drug Administration 
Center for Drug Evaluation and Research (CDER) 
Center for Biologics Evaluation and Research (CBER) 
 
November 2016 
 
Pharmaceutical Quality/CMC 
 Current Good Manufacturing P ractices (CGMPs) 
 
Revision 1
1063761 RevDft 
   
Submission of Quality 
Metrics Data
  
Guidance for Industry 
 
Additional copies are available from: 
Office of Communications, Division of Drug Information  
Center for Drug Evaluation and Research 
Food and Drug Administration 
10001 New Hampshire Ave., Hillandale Bldg., 4th Floor  
Silver Spring, MD 20993-0002  
Phone: 855-543-3784 or 301-796-3400; Fax: 301-431-6353 
Email: druginfo@fda.hhs.gov  
http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/default.htm 
and/or 
Office of Communication, Outreach and Development  
Center for Biologics Evaluation and Research 
 Food and Drug Administration 
10903 New Hampshire Ave., Bldg. 71, Room 3128 
Silver Spring, MD 20993 
Phone: 800-835-4709 or 240-402-8010 
Email: ocod@fda.hhs.gov 
http://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guidances/default.htm    
 
 
 
 
 
 
 
U.S. Department of Health and Human Services 
Food and Drug Administration 
Center for Drug Evaluation and Research (CDER) 
Center for Biologics Evaluation and Research (CBER) 
 
November 2016 
 
Pharmaceutical Quality/CMC 
 Current Good Manufacturing Practices (CGMPs) 
 
Revision 1 
 
    Contains Nonbinding Recommendations 
Draft  Not for Implementation 
TABLE OF CONTENTS 
 
 
I.  INTRODUCTION............................................................................................................. 1 
II.  BACKGROUND ............................................................................................................... 2 
A.  Modernization of Regulatory Oversight of Drug Quality and Promotion of  
  Post-Approval Improvements ....................................................................................................... 2 
B.  Quality Metrics Data  Regulatory Foundation .......................................................................... 3 
III.  REPORTING OF QUALITY DATA AND CALCULATION OF QUALITY 
METRICS .......................................................................................................................... 5 
A.  Who Reports and Who May Contribute to a Report ................................................................. 5 
B.  Quality Metrics that FDA Intends to Calculate .......................................................................... 7 
C.  Quality Metrics Data that May Be Reported .............................................................................. 7 
D.  How to Submit Comments Within a Quality Metric Data Report and How to  
  Pose Questions to FDA ................................................................................................................ 11 
E.  How to Report Quality Metrics Data to FDA ............................................................................ 11 
IV.   THE USE OF QUALITY METRICS AND PUBLIC REPORTING ........................ 12 
A.  How FDA Intends to Use Quality Metrics ................................................................................. 12 
B.  Quality Metric Reporters List .................................................................................................... 13 
GLOSSARY................................................................................................................................. 16 
APPENDIX A:  APPLICABLE IDENTIFYING INFORMATION AND QUALITY 
METRIC DATA ELEMENTS FOR PRODUCT REPORTS AND SITE REPORTS ......... 17 
APPENDIX B:  EXAMPLES ..................................................................................................... 26 
    Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
1  Submission of Quality Metrics Data 
1
2  Guidance for Industry  
3   
4   
5  This draft guidance, when finalized, will represent the current thinking of the Food and Drug 
6  Administration (FDA or Agency) on this topic.  It does not create any rights for any person and is not 
7  binding on FDA or the public.  You can use an alternative approach if it satisfies the requirements of the 
8  applicable statutes and regulations.  To discuss an alternative approach, contact the FDA staff responsible 
9  for this guidance as listed on the title page.   
10   
11   
12   
13   
14  I.  INTRODUCTION  
15   
16  Quality metrics are used throughout the drugs and biologics2 industry to monitor quality control 
17  systems and processes and drive continuous improvement efforts in drug manufacturing.  These 
18  metrics can also be useful to FDA:  to help develop compliance and inspection policies and 
19  practices, such as risk-based inspection scheduling of drug manufacturers; to improve the 
20  Agencys ability to predict, and therefore, possibly mitigate, future drug shortages; and to 
21  encourage the pharmaceutical industry to implement state-of-the-art, innovative quality 
22  management systems for pharmaceutical manufacturing.  This revised draft guidance includes an 
23  explanation of how the Center for Drug Evaluation and Research (CDER) and the Center for 
24  Biologics Evaluation and Research (CBER) intend to utilize submitted data and quality metrics 
25  to help ensure that their policies and practices continue to support continuous improvement and 
26  innovation in the pharmaceutical manufacturing industry.   
27   
28  In order to achieve these goals, FDA is initiating a quality metrics reporting program.3  As 
29  described in this guidance, FDA is initiating a voluntary reporting phase of the FDA quality 
30  metrics reporting program.4  In the voluntary reporting phase of the program, FDA expects to 
31  learn more about a limited set of quality metrics, associated analytics, and improve the FDA 
32  quality metrics reporting program. 
33     
34  During the voluntary phase of the reporting program, FDA will accept voluntarily submissions of 
35  data from owners and operators of human drug establishments.  FDA expects that the large 
                                                 
1 This guidance has been prepared by the Center for Drug Evaluation and Research (CDER) and the Center for 
Biologics Evaluation and Research (CBER) at the Food and Drug Administration. 
2 This guidance uses the terms drugs to refer to both drugs and biologics. 
3 FDA issued a draft guidance regarding the collection of quality metrics on July 28, 2015.  In response to comments 
received in the public docket (FDA-2015-D-2537), FDA is replacing the draft guidance published in 2015 with this 
revised draft.   
4 More details about the timing of the program are in the notice announcing the availability of this draft guidance in 
the Federal Register.  
  1  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
36  majority of voluntary reports will be submitted by establishments engaged in the manufacture, 
37  preparation, propagation, compounding, or processing of finished dosage forms (FDF) of 
38  covered drug products or active pharmaceutical ingredients (API) used in the manufacture of 
39  covered drug products.5   
40   
41  The voluntary reporting phase of the program described in this guidance is not focused on 
42  reporting from certain CDER regulated manufacturers (i.e., compounders operating under 
43  section 503A or registered as outsourcing facilities under section 503B of the Federal Food, 
44  Drug, and Cosmetic Act (FD&C Act) or CBER regulated manufacturers of blood and blood 
45  components for transfusion, vaccines, in vitro diagnostics,6 cell therapy products, gene therapy 
46  products, allergenic extracts, human cells, tissues, and cellular and tissue based products).7   
47   
48  In general, FDAs guidance documents do not establish legally enforceable responsibilities.  
49  Instead, guidances describe the Agencys current thinking on a topic and should be viewed only 
50  as recommendations, unless specific regulatory or statutory requirements are cited.  The use of 
51  the word should in Agency guidances means that something is suggested or recommended, but 
52  not required.  Also, in this guidance, the use of the word should is used to indicate an FDA 
53  preference to promote consistent reporting and counting of quality metrics data.8 
54   
55   
56  II.  BACKGROUND 
57   
58  A.  Modernization of Regulatory Oversight of Drug Quality and Promotion of 
59  Post-Approval Improvements 
60   
61  FDAs approach to quality oversight has evolved in recent years.  CDER and CBER are 
62  committed to supporting the modernization of pharmaceutical manufacturing as part of the 
63  Agencys mission to protect and promote public health.  This effort is also part of a long-term 
64  strategy to mitigate drug shortages by addressing the underlying causes of shortages, as noted in 
65  FDAs Strategic Plan for Preventing and Mitigating Drug Shortages.9  In 2002, FDA launched 
66  an initiative entitled Pharmaceutical cGMPs for the 21st Century:  A Risk-Based Approach, to 
67  encourage the implementation of a modern, risk-based pharmaceutical quality assessment 
                                                 
5 The terms covered drug product and covered establishment are defined in section III.A. 
6 This guidance is not applicable to biological products that meet the definition of a device in section 201(h) of the 
FD&C Act (21 U.S.C. 321(h)). 
7 The guidance does apply to licensed biological products that are plasma derived products, including recombinant 
and transgenic versions of plasma derivatives. 
8 FDA intends to accept voluntary reports with quality metrics data that are inconsistent with the metrics and 
definitions in this guidance, as well as reports about establishments and products that are not the focus of the 
voluntary reporting phase of the quality metrics program as described in this guidance.  However, as the data 
submitted in a manner inconsistent with the definitions and recommendations in this guidance may not be 
comparable with submissions from other reporters, we: (1) do not intend to include these reporters on the quality 
metrics reporters list, and (2) may not be able to integrate the submission of the report into FDAs risk-based 
inspection model.  Submissions will be evaluated on a case-by-case basis. 
9 See FDAs Strategic Plan for Preventing and Mitigating Drug Shortages at: 
http://www.fda.gov/downloads/Drugs/DrugSafety/DrugShortages/UCM372566.pdf. 
  2  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
68  system.10  The initiative was published with several goals, including ensuring that regulatory 
69  review, compliance, and inspection policies support continuous improvement and innovation in 
70  the pharmaceutical manufacturing industry.  Since publication of the Pharmaceutical cGMPs for 
71  the 21st Century, CDER has promoted a vision of  a maximally efficient, agile, flexible 
72  manufacturing sector that reliably produces high-quality drug products without extensive 
73  regulatory oversight.11 
74   
75  FDA encourages manufacturers to routinely use additional quality metrics beyond the metrics 
76  described in this guidance in performing product and establishment specific evaluations.12  The 
77  selected metrics are not intended to be an all-inclusive set of the quality metrics that 
78  manufacturers may find useful to assess a product and manufacturers state of quality.   
79   
80  B.  Quality Metrics Data  Regulatory Foundation 
81   
82  FDA understands that establishments involved in the manufacture, preparation, propagation, or 
83  processing of human drugs, including oversight to ensure quality,13 currently use quality metrics 
84  as part of the process validation lifecycle and pharmaceutical quality system (PQS) assessment.14  
85  The metrics described in this guidance could be a part of such oversight.  
86   
87  As described in FDAs process validation guidance, manufacturers depend on information and 
88  knowledge from product and process development as the basis for establishing an approach to 
89  control of the manufacturing process (i.e., a control strategy) that results in products with the 
                                                 
10 See Pharmaceutical cGMPs for the 21st Century:  A Risk-Based Approach at: 
http://www.fda.gov/Drugs/DevelopmentApprovalProcess/Manufacturing/QuestionsandAnswersonCurrentGoodMan
ufacturingPracticescGMPforDrugs/ucm137175.htm. 
11 See FDA Pharmaceutical Quality Oversight:  One Quality Voice at 
http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDER/UCM4426
66.pdf. 
12 One type of evaluation is an internal, independent audit and review of processes and procedures to determine 
whether established protocols and procedures have been followed.  FDAs Compliance Policy Guide Sec. 130.300, 
FDA Access to Results of Quality Assurance Program Audits and Inspections (June 2, 2007) describes our policy 
that during routine inspections and investigations, FDA will not review or copy these specific reports and records to 
encourage firms to conduct candid and meaningful audits and inspections.  The voluntary submission of quality 
metrics data described in this guidance will be for specific data that are maintained on-site, routinely reviewed 
during inspections, and not subject to a request for the results of an internal audit. 
http://www.fda.gov/iceci/compliancemanuals/compliancepolicyguidancemanual/ucm073841.htm.   
13 Food and Drug Administration Safety and Innovation Act of 2012 (FDASIA) section 711 added text to section 
501 of the FD&C Act clarifying that, for the purposes of paragraph 501(a)(2)(B), the term current good 
manufacturing practice includes the implementation of oversight and controls over the manufacture of drugs to 
ensure quality, including managing the risk of establishing the safety of raw materials, materials used in the 
manufacturing of drugs, and finished drug products. 
14 Refer to FDA guidance for industry Process Validation: General Principles and Practices (Rev 1).  We update 
guidances periodically. To make sure you have the most recent version of a guidance, check the FDA Drugs 
guidance Web page at 
http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/default.htm. 
  3  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
90  desired quality attributes.15  Once a control strategy has been successfully implemented, 
91  manufacturers are expected to maintain the process in a state of control over the life of the 
92  process, even as materials, equipment, production environment, personnel, and manufacturing 
93  procedures change.16  Current good manufacturing practice (CGMP) for human drugs require 
94  manufacturers to have an ongoing program to maintain and evaluate product and process data 
95  that relate to product quality.17  Best practice for this ongoing assessment is continued process 
96  verification,18 which should include a Periodic Product Review (PPR), conducted at least 
97  annually, in which data collected includes relevant process trends and quality of incoming 
98  materials or components, in-process materials, and finished products.  Some establishments may 
99  call this evaluation an Annual Product Review (if conducted annually) or a Product Quality 
100  Review,19 for finished drug products or APIs, respectively.  We expect that most of the quality 
101  metrics data described in this guidance will be collected by establishments already as part of 
102  conducting the PPR.  
103   
104  Under Title VII section 706 of the Food and Drug Administration Safety and Innovation Act 
105  (FDASIA) Public Law No. 112-144, FDA may require the submission of any records or other 
106  information that FDA may inspect under section 704 of the FD&C Act, in advance or in lieu of 
107  an inspection by requesting the records or information from a person that owns or operates an 
108  establishment that is engaged in the manufacture, preparation, propagation, compounding, or 
109  processing of a drug.  The quality metrics data described in this guidance is information of the 
110  type that FDA may inspect under section 704 of the FD&C Act.  However, FDA does not intend 
111  to require the submission of information pursuant to section 704(a)(4) of the FD&C Act in 
112  implementing the voluntary phase of the quality metrics reporting program.  FDA does not 
113  intend to take enforcement action based on errors in a quality metrics data submission made as a 
114  part of this voluntary phase of the reporting program, provided the submission is made in good 
115  faith.   
116   
117  Section 510(h)(3) of the FD&C Act requires a risk-based inspection schedule for drug 
118  establishments according to the known safety risks posed by establishments that are required to 
119  register.  These risks are based on certain factors described in section 510(h)(4)(A-F), including 
120  the inherent risk of the drug manufactured, prepared, propagated, compounded, or processed at 
121  the establishment and other factors.  FDA intends to analyze the calculated quality metrics to 
122  support its understanding of the safety risks of manufacturing establishments and products, and 
123  as the basis for criteria it deems necessary and appropriate for allocating inspection resources. 
                                                 
15 Refer to FDA guidance for industry Process Validation: General Principles and Practices (Rev 1) for a 
description of other sections of 21 CFR part 211 that set forth requirements related to aspects of process validation. 
16 FDASIA section 711 added text to section 501 of the FD&C Act clarifying that, for the purposes of paragraph 
501(a)(2)(B), the term current good manufacturing practice includes the implementation of oversight and controls 
over the manufacture of drugs to ensure quality, including managing the risk of and establishing the safety of raw 
materials, materials used in the manufacturing of drugs, and finished drug products. 
17 See 21 CFR 211.180(e) and section 501(a)(2)(B) of the FD&C Act (21 U.S.C. 351(a)(2)(B)). 
18 Refer to FDA guidance for industry Process Validation: General Principles and Practices (Rev 1). 
19 The Product Quality Review of APIs is comparable to the Annual Product Review conducted for finished drug 
products under 21 CFR 211.180(e).  Refer to FDA guidance for industry Q7A Good Manufacturing Practice 
Guidance for Active Pharmaceutical Ingredients.  
  4  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
124   
125   
126  III.  REPORTING OF QUALITY DATA AND CALCULATION OF QUALITY 
127  METRICS 
128   
129  A.  Who Reports and Who May Contribute to a Report 
130   
131  1.  Covered Establishments and Covered Drug Products  
132   
133  Except as noted below, owners and operators of each establishment that is engaged in the 
134  manufacture, preparation, propagation, compounding, or processing of a covered drug product, 
135  or an API used in the manufacture of a covered drug product, may submit quality metrics data.  
136  For purposes of this guidance, we will refer to the types of establishments whose owners or 
137  operators directly or indirectly submit reports  as covered establishments.  
138   
139  For purposes of reporting a covered drug product or an API used in the manufacture of a covered 
140  drug product, a covered drug product is: 
141   
142    subject to an approved application under section 505 of the FD&C Act or under 
o
143  section 351 of the Public Health Service Act (PHS) Act, 
144    
145    marketed pursuant to an OTC monograph, or 
o
146   
147    a marketed unapproved finished drug product. 
o
148   
149  Covered establishments also include (but are not limited to) contract laboratories, contract 
150  sterilizers, contract packagers,20 and other establishments, as appropriate, engaged in the 
151  manufacture, preparation, propagation, compounding, or processing of a covered drug product or 
152  API used in a covered drug product.   
153   
154  2.   Who Reports for Covered Establishments 
155   
156  This guidance describes two types of quality metric data reports:  (1) product reports submitted 
157  by product reporting establishments,21 and (2) site reports submitted by site reporting 
158  establishments.  We encourage reports from product reporting establishments and site reporting 
159  establishments.  FDA prefers for all covered establishments to work with a product reporting 
160  establishment and report data for the covered drug product so that the product reporting 
161  establishment submits a single product report that includes data from all covered establishments. 
                                                 
20 Contract re-packagers that purchase product and repackage it into a different primary packaging configuration are 
included (e.g., large bottles of tablets repackaged into unit dose blister packs).  Contract re-packagers that purchase 
product and repackage into secondary or tertiary packaging are not included. 
21 A product reporting establishment is one establishment who will already possess or have access to all of the 
quality metrics data needed to submit such reports.  It is further defined in section III.A.2.a. 
  5  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
162  Compilation of data into a single product report will facilitate data analysis and identification of 
163  product specific issues (e.g., potential loss in drug supply).   
164   
165  a.  Submission of a product report by a product reporting establishment 
166   
167  The subject of a product report will generally be a covered drug product or an API used in the 
168  manufacture of a covered drug product.  The report may include quality metrics data from each 
169  covered establishment within the manufacturing supply chain that has the data described in this 
170  guidance.  FDA believes that, as part of its responsibility for oversight and controls over the 
171  manufacture of drugs to ensure quality, one establishment will already possess or have access to 
172  all of the quality metrics data needed to submit such reports  for example, through contract or 
173  because all of the covered establishments with quality metrics data related to a covered drug 
174  product or API used in the manufacture of a covered drug product will be under common 
175  ownership or control.22  This establishment should combine the data so that a single report is 
176  submitted.  For example, a single API may be the subject of a stand-alone product report, as 
177  APIs are often supplied to multiple customers and finished drug product manufacturers often use 
178  multiple API suppliers. 
179   
180  In this guidance, we refer to the covered establishments that submit product reports to FDA as 
181  product reporting establishments.  If a product reporting establishment is gathering data from 
182  covered establishments in the manufacturing supply chain for a particular product for the 
183  purpose of submitting a product report, but data is not available for a covered establishment, 
184  FDA prefers that the product report clearly identifies the covered establishment and that specific 
185  data was not received.23 
186   
187  FDA believes that the quality control unit (QCU)24 in each reporting establishment for a covered 
188  drug product or API used in a covered drug product will generally be best positioned to compile 
189  reports for submission to FDA, considering the QCU responsibilities and authorities for the 
190  oversight of drugs as described in 21 CFR 211.22.25  
191   
192  b.  Submission of a site report by a site reporting establishment 
193   
194  If the covered establishment prefers to report directly or is unsure if all products and data will be 
195  reported via a product report, the covered establishment may elect to submit a site report.  In this 
196  guidance, we refer to the covered establishments that submit site reports to FDA as site 
197  reporting establishments.    
198   
                                                 
22 See, e.g., FDASIA section 711; 21 CFR 200.10(b). 
23 Refer to Appendix A.1, A.2, A.3, and A.4. 
24 For the purpose of this guidance, the term quality control unit is synonymous with quality unit. 
25 For APIs, these responsibilities are described in FDA guidance for industry Q7A Good Manufacturing Practice 
Guidance for Active Pharmaceutical Ingredients (section 2.2). 
  6  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
199  The subject of a site report is a single covered establishment.  A complete report would list all 
200  covered products with associated quality metric data specific to each product manufactured at the 
201  subject establishment as described in this guidance.26   
202   
203  B.  Quality Metrics that FDA Intends to Calculate 
204   
205  The following set of quality metrics that FDA intends to calculate based on industry reporting 
206  was developed with stakeholder input.  FDA used the following selection criteria in developing 
207  the set of data that it is inviting covered establishments to submit: (1) objective data to provide 
208  consistency in reporting, (2) of the type contained in records subject to inspection under section 
209  704 of the FD&C Act, and (3) a valuable component in assessing the overall effectiveness of a 
210  PQS, within reasonable limits, and in a reasonable manner, while avoiding an undue reporting 
211  burden.  FDA believes that these quality metrics data, in conjunction with other data accessible 
212  to FDA, provide important information about operational reliability.   
213   
214  Using reported data described in the following section, FDA intends to calculate quality metrics 
215  for each product and covered establishment, where applicable:  
216   
217    Lot Acceptance Rate (LAR) as an indicator of manufacturing process performance.  
218  LAR = the number of accepted lots in a timeframe divided by the number of lots started 
219  by the same covered establishment in the current reporting timeframe.  
220     
221    Product Quality Complaint Rate (PQCR) as an indicator of patient or customer 
222  feedback.  PQCR = the number of product quality complaints received for the product 
223  divided by the total number of dosage units distributed in the current reporting timeframe.    
224   
225    Invalidated Out-of-Specification (OOS) Rate (IOOSR) as an indicator of the operation 
226  of a laboratory.  IOOSR = the number of OOS test results for lot release27 and long-term 
227  stability testing invalidated by the covered establishment due to an aberration of the 
228  measurement process divided by the total number of lot release and long-term stability 
229  OOS test results in the current reporting timeframe.28,29 
230   
231  C.  Quality Metrics Data that May Be Reported  
232     
233  Section IV.B describes the types of metrics FDA intends to calculate and the associated data that 
234  may be submitted to calculate and understand each metric.  FDA encourages product reporting 
235  establishments to submit product reports, segmented by covered establishment, where possible.30  
                                                 
26 Refer to Appendix A.5, A.6, A.7, and A.8. 
27 This term does not refer to samples and protocols under 21 CFR 610.2. 
28 Reference this guidances Glossary for OOS result (e.g., lot release tests and long-term stability tests only).  A 
single result (e.g., one value on a Certificate of Analysis) may result in only one OOS test result.   
29 The metric measures invalidated lot release OOS results and long-term stability OOS results, separately. 
30 FDA anticipates that  data relevant to contract laboratories will generally be limited to the number of OOS results, 
the number of lot release and stability tests conducted, and the number of invalidated OOS. 
  7  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
236  The quality metrics data described in this draft guidance is developed and maintained in the 
237  course of manufacturing drugs in compliance with CGMP.  In general, the information that FDA 
238  will receive is maintained in accordance with 21 CFR 211 subpart J and evaluated under 21 CFR 
239  211.180(e).  For non-finished drug products (e.g., APIs), refer to section 501(a)(2)(B) of the 
240  FD&C Act and FDA guidance for industry Q7 Good Manufacturing Practice Guidance for 
241  Active Pharmaceutical Ingredients.  Data that is summed and reported as described in this 
242  section is in a readily accessible format for analysis. 
243   
244  Reporting of data related to lots of drugs that are imported, intended for import into the United 
245  States, or manufactured in the United States is preferred.  However, FDA recognizes that it may 
246  not be possible for some covered establishments to identify started lots, rejected lots, and OOS 
247  results that are specific to drugs that are imported, intended for import, or manufactured in the 
248  United States.  Further, lots manufactured outside of the United States may be split after 
249  manufacturing is completed and a portion is imported, or intended for import into the United 
250  States.  In these instances, if the manufacturing process uses the same process and controls data 
251  for lots that are not specific to those that are imported, intended for import, or manufactured in 
252  the United States, the report could include both data from lots not imported or intended for 
253  import to the United States with the data from lots imported or intended for import to the United 
254  States for the lot acceptance and invalidated OOS metrics.  The selection of drugs that are either:  
255  (1) imported, intended for import, or manufactured in the United States, or (2) all drugs using the 
256  same manufacturing process and controls which are not necessarily imported, intended for 
257  import, or manufactured in the United States, should remain consistent within and across 
258  reporting cycles, unless otherwise specified.  Product quality complaint data should be related to 
259  drugs that are imported, intended for import or manufactured in the United States.   
260   
261  Reporting of data should include all manufacturing operations, including testing, which would be 
262  included in a PPR (e.g., lots intended for commercial distribution, post-approval clinical trial lots 
263  when the same manufacturing process and controls are used as for commercial lots). 
264   
265  (1) Lot Acceptance Rate (LAR) Data: 
266   
267    The number of saleable lots started which are intended for primary packaging or 
268  distribution.  
269   
270    The number of saleable lots released for primary packaging or distribution.  
271   
272    The number of saleable lots started which are intended for primary packaging or 
273  distribution and were rejected. 
274   
275    The number of lots started of in-process and packaging product lots which are intended 
276  for distributed product. 
277   
278    The number of in-process and packaging product lots released which are intended for 
279  distributed product. 
280   
  8  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
281    The number of in-process and packaging product lots which were intended for distributed 
282  product and were rejected. 
283   
284  Specific criteria for the LAR data: 
285   
286    Examples of saleable lots include bulk tablets, filled vials, bulk milled in-process material 
287  if manufacturing is performed at another covered establishment, bulk API, and bulk 
288  intermediate API if further manufacturing is performed at another covered establishment. 
289   
290    A lot may be subdivided or grouped after the first started lot is initiated.  Each subsequent 
291  subdivision or grouping is considered a separate lot.  
292   
293    Examples of packaging product lots include multiple packaging configurations of bulk 
294  tablets (e.g., small bottles, large bottles, blisters) and labeling filled sterile vials with 
295  multiple labels (e.g., intended for different countries).  The packaging operation can be 
296  stand-alone lots or included in an existing lot.   
297   
298    In general, FDA anticipates that the number of lots started minus the sum of lots released 
299  and lots rejected will equal the total number of lots pending disposition (e.g., work in 
300  progress, lots evaluated for batch release, lots pending disposition due to quality-related 
301  discrepancies).  We recognize that there are rare instances when this construct will not be 
302  valid (e.g., lots pending disposition for an extended period) and we encourage the use of 
303  the comment text box to explain the occurrence of such an anomaly.   
304   
305  (2) Invalidated OOS Rate Data (IOOSR): 
306   
307    The number of lot release test OOS and long-term stability OOS results for the finished 
308  drug product or API where the long-term stability test supports the labeled expiration 
309  date.  
310   
311    The total number of lot release and long-term stability tests conducted for the finished 
312  drug product or API where the long-term stability test supports the labeled expiration 
313  date. 
314   
315    The number of OOS results for lot release tests and long-term stability tests for the 
316  finished drug product or API where the source of the OOS result is identified as an 
317  aberration of the measurement process and where the stability test supports the labeled 
318  expiration date. 
319   
320  Specific criteria for the IOOSR data: 
321   
  9  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
322    An investigation must be conducted whenever an OOS result is obtained.31   For the 
323  purpose of the quality metrics program, the following  OOS results should be counted:  
324  (1) finished drug product and API and long-term stability test results only, and (2) all 
325  finished drug product and API and long-term stability test results that initially indicate 
326  OOS, even if the source of the OOS is investigated and determined to be an aberration of 
327  the measurement process.  See FDA guidance for Industry Investigating Out-of-
328  Specification (OOS) Test Results for Pharmaceutical Production (October 2006), section 
329  III, and FDA guidance for industry Sterile Drugs Products Produced by Aseptic 
330  Processing  Current Good Manufacturing Practice (September 2004), section XI.   
331   
332    The number of total tests is a measurement tool that:  (1) provides context for the 
333  invalidated OOS rate, and (2) provides a secondary metric for manufacturing 
334  performance and the ability to produce product within limits (lot release and long-term 
335  stability OOS results investigated as a manufacturing aberration divided by the total 
336  number of lot release and long-term stability tests performed in the same current 
337  reporting period).   
338   
339    For the purpose of this program, an OOS result should be counted on the day that the test 
340  result is completed or the day that an OOS investigation is initiated. 
341   
342    A test includes a single analytical result for lot release or a stability timepoint with an 
343  established limit (e.g., analytical chemistry, release sterility test).  For example:  (1) for 
344  lot release, the final content uniformity result as reported on a Certificate of Analysis is 
345  considered one test; (2) for a stability timepoint, each test performed in the timepoint 
346  would count as an individual test.   
347   
348    A covered establishment that manufactures API used in a covered drug product is not 
349  expected to report stability OOS results.   
350   
351    For stability testing, only tests that support real-time stability of the product should be 
352  counted (i.e., accelerated stability testing is excluded).  
353   
354    If a lot release or long-term stability test is conducted multiple times for a lot (e.g., a 
355  retest), each test should be counted. 
356   
357    FDA recognizes the importance of other types of testing not discussed in this guidance 
358  (e.g., in-process testing, environmental testing, raw material and packaging component 
359  testing).  However, results of these tests should not be counted in this report. 
360   
361  (3) Product Quality Complaint Rate (PQCR): 
362   
363    The number of product quality complaints received for the product. 
                                                 
31 See 21 CFR 211.192 and section 501(a)(2)(B) of the FD&C Act. 
  10  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
364   
365    The total number of dosage units distributed for the product. 
366   
367  Specific criteria for the PQCR data: 
368   
369    The total number of all product quality complaints is based on the definition in the 
370  glossary.  This number does not include multiple counting of the same product quality 
371  complaint if the complaint receiver forwards the complaint to individual manufacturers 
372  for further investigation.  This number does include all potential quality issues, such as 
373  subpotency (e.g., a patient report of lack of effect). 
374   
375    The total number of dosage units distributed for the product is defined in the glossary. 
376   
377  D.  How to Submit Comments Within a Quality Metric Data Report and How to 
378  Pose Questions to FDA 
379   
380  Reporting establishments may submit a 300-word text comment to provide an explanation of 
381  submitted data or report plans for improvement.  FDA may refer to the comments if unusual data 
382  or trends are identified, or in preparation for an on-site inspection.  The submission of comments 
383  is optional.  In the future, FDA may consider establishing a set of codes to standardize the 
384  comments. 
385   
386  Comments may describe special situations, such as natural disasters, the use of emerging 
387  technology, or describe the manufacturing supply chain or a plan for improvement.  For 
388  example, an unexpected decrease in lot acceptance rate may be due to a situation outside the 
389  control of the facility (e.g., an act of nature such as a storm or fire).  For emerging technology, 
390  the use of new, in-line analytical technology used for real time release testing with increased 
391  sensitivity might result in better detection of in-process OOS results used for Real Time Release 
392  Testing and thus, a temporary increase in total OOS results.  However, improved detection that 
393  allows for the diversion and rejection of poor quality product will provide improved assurance of 
394  quality.  In this instance, it may be appropriate to provide an explanation that new, improved 
395  technology was implemented and that there is data demonstrating that more robust product was 
396  released to the market as a result of this change (e.g., increased lot uniformity would be 
397  appropriate).   
398   
399  Upon gathering this data, any questions that a covered establishment may have about their 
400  specific situation can be sent to OPQ-OS-QualityMetrics@fda.hhs.gov.  
401   
402  E.  How to Report Quality Metrics Data to FDA 
403   
404  To facilitate the quality metrics reporters list as described in section IV.B, a defined reporting 
405  period (e.g., a single calendar year) is needed to reduce discrepancies between site and product 
406  reporting.  Therefore, reporting establishments may submit quality metrics data reports where the 
  11  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
407  data is segmented on a quarterly basis throughout a single calendar year.32  FDA expects to begin 
408  the data analysis when the portal is closed and then publish initial findings and the quality metric 
409  reporters list on the FDA Web site.   
410   
411  Appendix A of the draft guidance is a quality component list with the information for submission 
412  into the electronic portal as well as a description of applicable quality metrics data elements 
413  relevant for different business segments/types.  The associated Technical Conformance Guide 
414  describes additional technical details.33   
415   
416   
417  IV.   THE USE OF QUALITY METRICS AND PUBLIC REPORTING  
418   
419  A.  How FDA Intends to Use Quality Metrics  
420   
421  FDA intends to use data from the quality metrics reporting program to focus the use of FDA 
422  resources on the areas of highest risk to public health (e.g., risk-based inspection scheduling). 
423  Specifically, we intend to: 
424   
425    establish a signal detection program as one factor in identifying establishments and 
426  products that may pose significant risk to consumers;  
427    identify situations in which there may be a risk for drug supply disruption;  
428    improve the effectiveness of establishment inspections; and  
429    improve FDAs evaluation of drug manufacturing and control operations.   
430   
431  Shortages of drugs can pose a significant public health threat; delaying, and in some cases even 
432  denying, critically needed care for patients.  Taking action to reduce drug shortages remains a 
433  top priority for FDA.  The Agency has found that the majority of drug shortages stem from 
434  quality concernssubstandard manufacturing facilities or processes are discovered, or 
435  significant quality defects are identified in finished drug product, necessitating remediation 
436  efforts to fix the issue, which in turn, may interrupt production and cause a drug shortage.  FDA 
437  intends to use quality metrics, along with other measures, to identify potential shortage signals 
438  and engage proactively with manufacturers to mitigate the likelihood of occurrence.  
439   
440  FDA may not be able to accomplish the overall goals of an FDA quality metrics reporting 
441  program, as described in this draft guidance, from voluntary reporting alone.  If FDA does not 
442  receive a large body of data from reporting establishments, the ways in which the Agency can 
443  use the information may be limited.  For example, data received may not constitute a 
444  representative sample of the industry.  Further, a self-selection bias may increase the risk of 
445  signaling an outlier where none exists.  For these reasons, we expect to use the information 
446  collected to specifically focus on: (1) working with establishments towards early resolution of 
                                                 
32 More details about the timing of the program are in the notice announcing the availability of this draft guidance in 
the Federal Register. 
33 See http://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/Manufacturing/UCM508464.pdf. 
  12  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
447  potential quality problems and to reduce the likelihood that the establishments operations will 
448  be disrupted and impact the drug supply, (2) helping to prepare for and direct our inspections, 
449  and (3) using the calculated metrics as an element of the post-approval manufacturing change 
450  reporting program with an emphasis on encouraging lifecycle manufacturing improvement.   
451   
452  While FDA recognizes the value of quality metrics, we also recognize that the individual data 
453  points and metrics described in this guidance, either individually or in combination, do not 
454  definitively quantitate the quality of the establishment or its products.  Further, FDA continues to 
455  encourage the adoption of emerging technology.  We request comments on implementing new 
456  technology while maintaining robust quality metrics programs.  
457   
458  FDA intends to publish an analysis of the quality metrics data  received on the FDA Web site to 
459  share what the Agency has learned from the voluntary phase of the reporting program, and how 
460  analyzing these data has affected the frequency of CGMP inspections and the ability of the 
461  Agency to address potential drug shortage situations.  We also intend to provide opportunities for 
462  participating establishments to provide feedback and additional comments, as well as share 
463  knowledge from ongoing, industry-driven quality metrics programs. 
464   
465  B.  Quality Metric Reporters List 
466   
467  FDA intends to publish a list of the names of establishments that voluntarily report all or a subset 
468  of quality data as described in this guidance (i.e., product reporting establishments and site 
469  reporting establishments).  We believe that there is a benefit to publicly sharing the names of 
470  establishments that voluntarily choose to submit these quality data to FDA because, through their 
471  participation, these establishments demonstrate a willingness to proactively engage with the 
472  Agency in pursuit of the goals described in this guidance.  Participation in this voluntary 
473  reporting phase of the program also demonstrates a commitment to increasing transparency 
474  between industry and FDA and a contribution to improving quality monitoring throughout the 
475  industry.   
476   
477  This list may be useful to establishments within the pharmaceutical manufacturing industry when 
478  selecting contract manufacturers and component suppliers as one element of robust outsourcer or 
479  supplier selection (e.g., past inspection and regulatory authority history, audits of the facility and 
480  associated systems, and analytical testing).  This list may also be useful for healthcare purchasing 
481  organizations, healthcare providers, patients, and consumers in sourcing drugs when used in 
482  conjunction with other information (e.g., inspection history).  The list will provide information 
483  about whether an establishment voluntarily submitted quality metrics data to the Agency, and if 
484  so how much data was submitted.  It should be noted that inclusion on the list is not an indication 
485  of FDAs evaluation of the submitted data.   
486   
487  The Agency will identify participating establishments on FDAs Web Site according to the 
488  following recognition categories:   
489   
490    For Product Reporting Establishments (finished drug product reporter or API reporter): 
491   
  13  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
492    Product Reporter Top Tier:  If complete data supporting all metrics were 
o
493  included for each covered establishment in the manufacturing supply chain for 
494  all covered drug products (or APIs used in the manufacture of a covered drug 
495  product) for the full year reporting period 
496   
497    Product Reporter Mid Tier:  If all covered establishments in the manufacturing 
o
498  supply chain for all covered products were identified in the report, and complete 
499  quality metric data was provided from at least one of the establishments for each 
500  covered drug products (or APIs used in the manufacture of a covered drug 
501  product) for the full year reporting period 
502   
503    Product Supply Chain Reporter:34  If all covered establishments in the 
o
504  manufacturing supply chain for all covered drug products (or APIs used in the 
505  manufacture of a covered drug product) were identified in the report  
506   
507    For Site Reporting Establishments (finished drug product reporter or API reporter):35  
508   
509    Site Reporter Top Tier:  If complete data supporting all metrics were included 
o
510  for all covered drug products (or APIs used in the manufacture of a covered drug 
511  product) for the full year reporting period 
512   
513    Site Reporter Mid Tier:  If complete data supporting all metrics were included 
o
514  for at least one covered drug product (or API used in the manufacture of a 
515  covered drug product) manufactured at an establishment for the full year 
516  reporting period 
517   
518  For example, if product reporting establishment Company ABC submitted a report identifying all 
519  covered establishments in the manufacturing supply chain for all covered drug products (or APIs 
520  used in the manufacture of a covered drug product), but did not provide quality metrics data, 
521  Company ABC would have a Product Supply Chain Reporter designation.  If product reporting 
522  establishment Company ABC submitted a report identifying all establishments in the 
523  manufacturing supply chain for all covered drug products (or APIs used in the manufacture of a 
524  covered drug product), and metrics data was provided from the primary manufacturing 
525  establishment for each product or API, but incomplete data was submitted from the other 
526  establishments in the manufacturing supply chain, Company ABC would have a Product 
527  Reporter Mid Tier designation.  If product reporting establishment Company ABC submitted a 
528  complete report for the data listed above for all covered drug products (or APIs used in the 
529  manufacture of a covered drug product), Company ABC would have a Product Reporter Top 
530  Tier designation.   
                                                 
34 Product Supply Chain Reporter is defined for the purpose of FDAs quality metric reporting program and is not 
associated with Title II of the Drug Quality and Security Act, the Drug Supply Chain Security Act (DSCSA). 
35 An establishment may be considered a site reporting establishment by either: (1) directly submitting data to FDA 
(not applicable for product reporting establishments), or (2) indirectly submitting data to FDA via a product report, 
submitted by a product reporting establishment. 
  14  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
531   
532  For site reporters, if contract manufacturer Company XYZ manufactures 30 covered drug 
533  products and submitted a report with at least one covered drug product produced at the 
534  establishment and data supporting all metrics, Company XYZ would have a Site Reporter Mid 
535  Tier designation.  If the report contains data for all 30 products and all metrics for each covered 
536  drug product, Company XYZ would have a Site Reporter Top Tier designation.  Alternatively, 
537  if Company XYZ submitted data to reporting establishments and the data covers each product 
538  manufactured at the site, and the submitted product reports reference this establishment, 
539  Company XYZ would also have a Site Reporter Top Tier designation.  
540   
541  FDA does not intend to publicly disclose information submitted to the Agency as part of the 
542  voluntary phase of the quality metrics program that is exempt from disclosure under the Freedom 
543  of Information Act as confidential commercial information, e.g., information that would reveal 
544  nonpublic commercial relationships and production volumes.   
  15  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
545  GLOSSARY 
546   
547  Active Pharmaceutical Ingredient (API)36  any substance that is intended for incorporation 
548  into a finished drug product and is intended to furnish pharmacological activity or other direct 
549  effect in the diagnosis, cure, mitigation, treatment, or prevention of disease, or to affect the 
550  structure or any function of the body.  Active pharmaceutical ingredient does not include 
551  intermediates used in the synthesis of the substance.  The term includes those components that 
552  may undergo chemical change in the manufacture of the drug product and be present in the drug 
553  product in a modified form intended to furnish the specified activity or effect. 
554   
555  Batch  a specific quantity of a drug or other material that is intended to have uniform character 
556  and quality, within specified limits, and is produced according to a single manufacturing order 
557  during the same cycle of manufacture.37  A batch may be comprised of one lot or multiple lots. 
558   
559  Continued Process Verification  A process validation activity where ongoing assurance is 
560  gained during routine production that the process remains in a state of control.38 
561   
562  Critical Quality Attribute (CQA)  A physical, chemical, biological, or microbiological 
563  property or characteristic that should be within an appropriate limit, range, or distribution to 
564  ensure the desired product quality.39   
565   
566  Dosage Units  the total number of individual dosage units (e.g., 100,000 tablets, 50,000 vials, 
567  50 kg), distributed or shipped under the approved application or product family (for non-
568  application products) to customers, including distributors.40   
569   
570  Establishment  a place of business under one management at one general physical location.  
571  The term includes, among others, independent laboratories that engage in control activities for a 
572  registered drug establishment (e.g., consulting laboratories).41 
573   
574  Finished Dosage Form (FDF)  the physical manifestation of a drug product that contains the 
575  active ingredient(s) and/or inactive ingredients that are intended to be delivered to the patient.  
576  Examples include tablets, capsules, vials, solutions, creams, or ointments.42 
577   
578  Finished Drug Product  a finished dosage form (FDF) (e.g., tablet, capsule, or solution) that 
579  contains at least one active pharmaceutical ingredient, generally, but not necessarily, in 
580  association with other ingredients in finished package form suitable for distribution to 
                                                 
36 Refer to 21 CFR 207.1 (effective November 29, 2016) and 21 CFR 210.3(b)(7). 
37 See 21 CFR 210.3(b)(2). 
38 Refer to FDA guidance for industry Process Validation: General Principles and Practices (Rev 1). 
39 Refer to FDA guidance for industry Q8(R2) Pharmaceutical Development.  
40 See 21 CFR 314.81(b)(2)(ii)(a), 211.196. 
41 See 21 CFR 207.1 (effective November 29, 2016). 
42 Refer to dose form as defined in ISO 11616:2012(en), Health informatics  Identification of medicinal products 
 Data elements and structures for the unique identification and exchange of regulated pharmaceutical product 
information. 
  16  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
581  pharmacies, hospitals, or other sellers or dispensers of the drug product to patients or 
582  consumers.43 
583   
584  Long-term testing  Stability studies under the recommended storage condition for the retest 
585  period or shelf life proposed (or approved) for labeling.44 
586   
587  Lot  a batch, or a specific identified portion of a batch, having uniform character and quality 
588  within specified limits; or, in the case of a drug product produced by continuous process, it is a 
589  specific identified amount produced in a unit of time or quantity in a manner that assures its 
590  having uniform character and quality within specified limits.45 
591   
592  Accepted Lot  a started lot which has been released for distribution or for the next stage 
593  of processing.  If the lot is released with an unexpectedly low yield due to an assignable 
594  root cause and the associated investigation supports the release of the lot, it should be 
595  considered an accepted lot.46  Investigations into low yield results should be thorough and 
596  managed by the quality unit.  If a lot number is closed, the lot is transferred to a new lot 
597  number, and subsequently released, only the original lot should be counted.  An accepted 
598  lot should be counted on the day of the final disposition decision.  It may be possible that 
599  an accepted lot is no longer considered accepted (e.g., a stability failure, a quality 
600  problem identified by a contract packager, or in the marketplace).  In this case, the lot 
601  should no longer be counted as an accepted lot.  If the change in disposition decision is 
602  after submission of quality data, the reporter may submit an amendment and it would be 
603  helpful if the amendment is available for discussion during a future on-site inspection. 
604   
605  Started Lot  a lot intended for commercial use for which the manufacturer has issued a 
606  lot number, physically charged API (for finished drug manufacturers) or primary starting 
607  materials (for API manufacturers), and there will be a disposition decision.47  If the 
608  manufacturing spans multiple time segments (quarters), the started lot should be counted 
609  when the lot number is issued or the API or primary starting material is physically 
610  charged.  If unique lot numbers are issued for different packaging configurations, each lot 
611  number should be counted.  
612   
                                                 
43 See 21 CFR 207.1 (effective November 29, 2016). 
44 See FDA guidance for industry Q1A(R2) Stability Testing of New Drug Substances and Products. 
45 See 21 CFR 210.3(b)(10). 
46 For example:  (1) if the power fails halfway through a tableting operation and a portion of the manufactured 
tablets are acceptable to release for distribution, this is considered an accepted lot, (2) if an API lot is reworked and 
released under the original lot number, the lot is considered an accepted lot, (3) for continuous manufacturing, if 
there was an unplanned shut down of the line due to quality reasons, this would be not be considered an accepted lot, 
(4) if the entire lot is rejected due to an OOS, the lot would not be considered an accepted lot, and (5) if the entire lot 
is rejected due to a potential contamination, the lot would not be an accepted lot. 
47 See 21 CFR 211.101.  
  17  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
613  Lot Release Test  includes all tests of conformance to final specifications, including all real 
614  time release tests, and all in-process tests that act as a surrogate for final lot release (e.g., real 
615  time release testing is approved in the application).48,49 
616   
617  Out-of-Specification (OOS) Result  all test results that fall outside the specifications or 
618  acceptance criteria established in drug applications, drug master file, official compendia, or by 
619  the manufacturer.50  An investigation must be conducted whenever an OOS result is obtained.51   
620  For the purpose of the quality metrics program, the following test events should be counted:  (1) 
621  lot release, including in-process tests that act as a surrogate for a lot release test,52 and long-term 
622  stability test results only and, (2) all lot release and long-term stability test results, even if the 
623  source of the OOS is later determined to be due to a measurement aberration.53  
624   
625  Invalidated OOS  any out-of-specification result where the investigation identifies the 
626  source of the OOS result as an aberration of the measurement process.  Invalidation of a 
627  discrete test result may be done only upon the observation and documentation of a test 
628  event that can reasonably be determined to have caused the OOS result.54  For the 
629  purpose of the quality metrics program, the following test events should be included:  (1) 
630  lot release55 and stability test results only and, (2) all lot release and stability test results 
631  that initially appear as OOS, even if invalidated by a subsequent laboratory investigation. 
632   
633  Periodic Product Review  an evaluation, conducted at least annually, of the quality standards 
634  of a drug product to determine the need for changes in drug product specifications or 
635  manufacturing or control procedures.56 
636   
637  Product Family  for finished drug products, any combination of National Drug Code (NDC) 
638  product code segments where the API and FDF is the same (i.e., a product family could be 
639  multiple strengths or only a single strength).57  For APIs, the product family is defined by the 
640  NDC product code segment.  A product family is defined for the purpose of grouping non-
641  application drugs for the submission of quality metric data.  Grouping is likely consistent with 
642  how products are grouped for the Periodic Product Review (e.g., Annual Product Review).58 
                                                 
48 See 21 CFR 211.165.  
49 This term does not refer to samples and protocols under 21 CFR 610.2.  
50 See FDA guidance for industry Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical 
Production.   
51 See 21 CFR 211.192 and section 501(a)(2)(B) of the FD&C Act. 
52 For example, if a near infrared (NIR) spectroscopy-based method is approved for testing active content of core 
tablets for release as an alternative to testing active content on finished tablets by traditional high-performance liquid 
chromatography (HPLC) method, and the NIR result is reported on the Certificate of Analysis, this test should be 
counted as a single analytical result and OOS result, as appropriate, for the purpose of this guidance. 
53 Each test may also be defined as a single analytical result listed on the Certificate of Analysis. 
54 See 21 CFR 211.160(a) and FDA guidance for industry Investigating Out-of-Specification (OOS) Test Results for 
Pharmaceutical Production. 
55 This term does not refer to samples and protocols under 21 CFR 610.2. 
56 See 21 CFR 211.180(e).   
57 See 21 CFR 207.35. 
58 See 21 CFR 211.180(e). 
  18  Draft  Not for Implementation  
Contains Nonbinding Recommendations 
 
643   
644  Product Quality Complaint  a complaint involving any possible, including actual, failure of a 
645  drug to meet any of its specifications designed to ensure that any drug conforms to appropriate 
646  standards of identity strength, quality, and purity.59 
                                                 
59 See, e.g., 21 CFR 211.160(b); 211.198.  
  19  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
647  APPENDIX A:  APPLICABLE IDENTIFYING INFORMATION AND QUALITY 
648  METRIC DATA ELEMENTS FOR PRODUCT REPORTS AND SITE REPORTS 
649   
650  This appendix provides clarity on which identifying information and quality metric data elements 
651  are applicable for submission in the voluntary phase of the quality metrics reporting program.  
652  Technical details of quality metric data submissions are provided in the Technical Conformance 
653  Guide.60  Data standards are available for certain identifying information elements (e.g., dose 
654  forms, business operations).61 
655   
656  Appendix A is separated into eight (8) subparts.  Each subpart corresponds to a different 
657  combination of report type, establishment type, and product type, as described in this draft 
658  guidance.  Specifically: 
659   
660    Product Report, segmented by all sites62 
661    Application Product 
o
662    Finished Drug Product:  Appendix A.1 
663    API:  Appendix A.2 
664    Non-Application Product 
o
665    Finished Drug Product:  Appendix A.3 
666    API:  Appendix A.4 
667   
668    Site Report, segmented by products 
669    Manufacturing with product quality oversight responsibilities only: Appendix A.5 
o
670    Manufacturer with testing responsibilities:  Appendix A.6 
o
671    Manufacturer without testing responsibilities:  Appendix A.7 
o
672    Manufacturer with testing responsibilities only:  Appendix A.8 
o
                                                 
60 See http://www.fda.gov/downloads/Drugs/DevelopmentApprovalProcess/Manufacturing/UCM508464.pdf. 
61 See http://www.fda.gov/forindustry/datastandards/structuredproductlabeling/ucm162038.htm.  
62 For a product report, when information was not provided by a contract facility, the corresponding data elements 
should be marked as not provided.   
 
  17  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
673  Appendix A.1:  Applicable Inputs for a Product Report Submission, Application Product, Finished Drug Product 
674   
 
Product Name  Rx/OTC  OTC Monograph  Product Type  Applicant Name  Application Type  Application Number  NDC Product Code Number(s)  Reporting Time Period  Quarter  Dose Form  Active Ingredient  Supply Chain/Process Stage Code  FEI/DUN  Started: In-process/Packaging  Started: Saleable  Rejected: In-process/Packaging  Rejected: Saleable  Released: In-process/Packaging  Released: Saleable  Number of quality complaints  Number of Dosage Units Distributed  Sum of Release test and Stability test OOS results where the source of the OOS result is identified as an aberration of the measurement process  Sum of Release test and Stability test OOS results  Sum of all Release and Stability Tests 
Product 
Reporting 
Establishment 
[Manufacturing  X  X  N/A  X  X  X  X  N/A  X  X  X  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  N/A 
with oversight 
responsibilities 
only] 
     
                                             
Contract 
Manufacturer 
performing  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  X  X  X  X  X  X  X  X  X  X  X 
release or 
stability testing 
     
                                             
Contract 
Manufacturer 
not performing 
release or 
N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  X  X  X  X  X  X  X  X  N/A  N/A  N/A 
stability testing 
(FDF, 
packaging, 
sterilizing, etc.) 
     
                                             
Contract 
Laboratory 
performing 
N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  X 
release or 
stability testing 
only 
675  X = Input is applicable to report; N/A = Input is not applicable to report 
  18  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
676  Appendix A.2:  Applicable Inputs for a Product Report Submission, Application Product, API 
677   
 
Product Name  Rx/OTC  OTC Monograph  Product Type  Applicant Name  Application Type  Application Number  NDC Product Code Number(s)  Reporting Time Period  Quarter  Dose Form  Active Ingredient  Supply Chain/Process Stage Code  FEI/DUN  Started: In-process/Packaging  Started: Saleable  Rejected: In-process/Packaging  Rejected: Saleable  Released: In-process/Packaging  Released: Saleable  Number of quality complaints  Number of Dosage Units Distributed  Sum of Release test and Stability test OOS results where the source of the OOS result is identified as an aberration of the measurement process  Sum of Release test and Stability test OOS results  Sum of all Release and Stability Tests 
Product 
Reporting 
Establishment 
[Manufacturing  X  N/A  N/A  X  X  X  X  N/A  X  X  X  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  N/A 
with oversight 
responsibilities 
only] 
   
                                               
Contract 
Manufacturer 
performing  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  X  X  X  X  X  X  X  X  X  X  X 
release or 
stability testing 
   
                                               
Contract 
Manufacturer 
not performing 
release or 
N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  X  X  X  X  X  X  X  X  N/A  N/A  N/A 
stability testing 
(FDF, 
packaging, 
sterilizing, etc.) 
   
                                               
Contract 
Laboratory 
performing 
N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  X 
release or 
stability testing 
only 
678  X = Input is applicable to report; N/A = Input is not applicable to report 
  19  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
679  Appendix A.3:  Applicable Inputs for a Product Report Submission, Non-application Product, Finished Drug Product 
680   
 
Product Name  Rx/OTC  OTC Monograph  Product Type  Applicant Name  Application Type  Application Number  NDC Product Code Number(s)  Reporting Time Period  Quarter  Dose Form  Active Ingredient  Supply Chain/Process Stage Code  FEI/DUN  Started: In-process/Packaging  Started: Saleable  Rejected: In-process/Packaging  Rejected: Saleable  Released: In-process/Packaging  Released: Saleable  Number of quality complaints  Number of Dosage Units Distributed  Sum of Release test and Stability test OOS results where the source of the OOS result is identified as an aberration of the measurement process  Sum of Release test and Stability test OOS results  Sum of all Release and Stability Tests 
Product 
Reporting 
Establishment 
[Manufacturing  X  X  X  X  N/A  N/A  N/A  X  X  X  X  X  X  X  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  N/A 
with oversight 
responsibilities 
only] 
     
                                             
Contract 
Manufacturer 
performing  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  X  X  X  X  X  X  X  X  X  X  X 
release or 
stability testing 
     
                                             
Contract 
Manufacturer 
not performing 
release or 
N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  X  X  X  X  X  X  X  X  N/A  N/A  N/A 
stability testing 
(FDF, 
packaging, 
sterilizing, etc.) 
     
                                             
Contract 
Laboratory 
performing 
N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  X 
release or 
stability testing 
only 
681  X = Input is applicable to report; N/A = Input is not applicable to report 
  20  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
682  Appendix A.4:  Applicable Inputs for a Product Report Submission, Non-application Product, API 
683   
 
Product Name  Rx/OTC  OTC Monograph  Product Type  Applicant Name  Application Type  Application Number  NDC Product Code Number(s)  Reporting Time Period  Quarter  Dose Form  Active Ingredient  Supply Chain/Process Stage Code  FEI/DUN  Started: In-process/Packaging  Started: Saleable  Rejected: In-process/Packaging  Rejected: Saleable  Released: In-process/Packaging  Released: Saleable  Number of quality complaints  Number of Dosage Units Distributed  Sum of Release test and Stability test OOS results where the source of the OOS result is identified as an aberration of the measurement process  Sum of Release test and Stability test OOS results  Sum of all Release and Stability Tests 
Product 
Reporting 
Establishment 
[Manufacturing  X  X  X  X  N/A  N/A  N/A  X  X  X  X  X  X  X  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  N/A 
with oversight 
responsibilities 
only] 
   
                                               
Contract 
Manufacturer 
performing  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  X  X  X  X  X  X  X  X  X  X  X 
release or 
stability testing 
   
                                               
Contract 
Manufacturer 
not performing 
release or 
N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  X  X  X  X  X  X  X  X  N/A  N/A  N/A 
stability testing 
(FDF, 
packaging, 
sterilizing, etc.) 
   
                                               
Contract 
Laboratory 
performing 
N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  X 
release or 
stability testing 
only 
684  X = Input is applicable to report; N/A = Input is not applicable to report 
  21  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
685  Appendix A.5:  Applicable Inputs for a Site Report Submission, Manufacturer with oversight responsibilities only (e.g., 
686  application holder) 
687   
  Product Name  Rx/OTC  OTC Monograph  Product Type  Applicant Name  Application Type  Application Number  NDC Product Code Number(s)  Reporting Time Period  Quarter  Dose Form  Active Ingredient  Supply Chain/Process Stage Code  FEI/DUN  Started: In-process/Packaging  Started: Saleable  Rejected: In-process/ Packaging  Rejected: Saleable  Released: In-process/Packaging  Released: Saleable  Number of quality complaints  Number of Dosage Units Distributed  Sum of Release test and Stability test OOS results where the source of the OOS result is identified as an aberration of the  measurement process Sum of Release test and Stability test OOS results  Sum of all Release and Stability Tests 
Finished 
Drug 
X  X  N/A  X  X  X  X  N/A  X  X  X  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  N/A 
Product  
Application 
   
                                               
Finished 
Drug 
Product   X  X  X  X  N/A  N/A  N/A  X  X  X  X  X  X  X  X  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  N/A 
Non-
application 
   
                                               
API  
X  N/A  N/A  X  X  X  X  N/A  X  X  X  N/A  X  X  X  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  N/A 
Application 
   
                                               
API  Non 
X  N/A  X  X  N/A  N/A  N/A  X  X  X  X  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  X  X  N/A  N/A  N/A 
Application 
688  X = Input is applicable to report; N/A = Input is not applicable to report 
689   
690   
691   
692   
693   
694   
695   
696   
  22  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
697  Appendix A.6:  Applicable Inputs for a Site Report Submission, Manufacturer that perform testing  
698   
  Product Name  Rx/OTC  OTC Monograph  Product Type  Applicant Name  Application Type  Application Number  NDC Product Code Number(s)  Reporting Time Period  Quarter  Dose Form  Active Ingredient  Supply Chain/Process Stage Code  FEI/DUN  Started: In-process/Packaging  Started: Saleable  Rejected: In-process/Packaging  Rejected: Saleable  Released: In-process/Packaging  Released: Saleable  Number of quality complaints  Number of Dosage Units Distributed  Sum of Release test and Stability test OOS results where the source of the OOS result is identified as an aberration of the measurement process  Sum of Release test and Stability test OOS results  Sum of all Release and Stability Tests 
Finished 
Drug 
X  X  N/A  X  X  X  X  N/A  X  X  X  N/A  X  X  X  X  X  X  X  X  X  X  X  X  X 
Product  
Application 
   
                                               
Finished 
Drug 
Product  X  X  X  X  N/A  N/A  N/A  X  X  X  X  X  X  X  X  X  X  X  X  X  X  X  X  X  X 
Non-
application 
   
                                               
API  
X  N/A  N/A  X  X  X  X  N/A  X  X  X  N/A  X  X  X  X  X  X  X  X  X  X  X  X  X 
Application 
   
                                               
API  Non 
X  N/A  X  X  N/A  N/A  N/A  X  X  X  X  N/A  X  X  X  X  X  X  X  X  X  X  X  X  X 
Application 
699  X = Input is applicable to report; N/A = Input is not applicable to report 
700   
701   
702   
703   
704   
  23  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
705  Appendix A.7:  Applicable Inputs for a Site Report Submission, Manufacturer that does not perform testing  
706   
 
Product Name  Rx/OTC  OTC Monograph  Product Type  Applicant Name  Application Type  Application Number  NDC Product Code Number(s)  Reporting Time Period  Quarter  Dose Form  Active Ingredient  Supply Chain/Process Stage Code  FEI/DUN  Started: In-process/Packaging  Started: Saleable  Rejected: In-process/Packaging  Rejected: Saleable  Released: In-process/Packaging  Released: Saleable  Number of quality complaints  Number of Dosage Units Distributed  Sum of Release test and Stability test OOS results where the source of the OOS result is identified as an  aberration of the measurement process Sum of Release test and Stability test OOS results  Sum of all Release and Stability Tests 
Finished 
Drug 
X  X  N/A  X  X  X  X  N/A  X  X  X  N/A  X  X  X  X  X  X  X  X  X  X  N/A  N/A  N/A 
Product 
Application 
   
                                               
Finished 
Drug 
Product  X  X  X  X  N/A  N/A  N/A  X  X  X  X  X  X  X  X  X  X  X  X  X  X  X  N/A  N/A  N/A 
Non-
application 
   
                                               
API  
X  N/A  N/A  X  X  X  X  N/A  X  X  X  N/A  X  X  X  X  X  X  X  X  X  X  N/A  N/A  N/A 
Application 
   
                                               
API  Non 
X  N/A  X  X  N/A  N/A  N/A  X  X  X  X  N/A  X  X  X  X  X  X  X  X  X  X  N/A  N/A  N/A 
Application 
707  X = Input is applicable to report; N/A = Input is not applicable to report 
708   
709   
710   
711   
712   
713   
714 
  24  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
715  Appendix A.8:  Applicable Inputs for a Site Report Submission, Manufacturer with Testing Only  
716   
` 
n 
Product Name  Rx/OTC  OTC Monograph  Product Type  Applicant Name  Application Type  Application Number  NDC Product Code Number(s)  Reporting Time Period  Quarter  Dose Form  Active Ingredient  Supply Chain/Process Stage Code  FEI/DUN  Started: In-process/Packaging  Started: Saleable  Rejected: In-process/Packaging  Rejected: Saleable  Released: In-process/Packaging  Released: Saleable  Number of quality complaints  Number of Dosage Units Distributed  Sum of Release test and Stability test OOS results here the source of the OOS result is identified as a aberration of the measurement process Sum of Release test and Stability test OOS results  Sum of all Release and Stability Tests 
w
Finished 
Drug 
X  X  N/A  X  X  X  X  N/A  X  X  X  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  X 
Products 
Application 
   
                                               
Finished 
Drug 
Product   X  X  X  X  N/A  N/A  N/A  X  X  X  X  X  X  X  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  X 
Non-
application 
   
                                               
API  
X  N/A  N/A  X  X  X  X  N/A  X  X  X  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  X 
Application 
   
                                               
API  Non 
X  N/A  X  X  N/A  N/A  N/A  X  X  X  X  N/A  X  X  N/A  N/A  N/A  N/A  N/A  N/A  N/A  N/A  X  X  X 
Application 
717  X = Input is applicable to report; N/A = Input is not applicable to report
  25  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
718  APPENDIX B:  EXAMPLES 
719   
720  (1)  Lot Acceptance Rate 
721  a.  An establishment manufactures a product where six small in-process lots are 
722  combined into a single unit operation to make one saleable lot (e.g., tablet, liquid, 
723  filled vial).  Two saleable lots are then combined into one packaging lot.   
724  Assuming all lots that are started are released: 
725    In-process and packaging lots started and released: 13 [six lots from the 
726  first saleable lot, six lots from the second saleable lot, and the single 
727  packaging lot] 
728    Saleable lots started and released: 2 
729  b.  An establishment manufactures one saleable lot that is separated into five 
730  packaged lots.   
731  Assuming all lots that are started are released: 
732    In-process and packaging lots started and released: 5 
733    Saleable lots started and released: 1 
734  c.  For an OTC monograph product, one batch of saleable product is packaged into 
735  an unlabeled primary pack and the primary pack is subsequently labeled and 
736  placed into secondary packaging at three different packagers.  In this scenario, all 
737  four of these facilities are considered covered establishments (one for the bulk 
738  manufacturing and three for primary labeling).  For the manufacturer of the 
739  unlabeled primary pack OTC product, the unlabeled primary pack lots are 
740  saleable lots.  The lots which are distributed by each packaging establishment are 
741  also saleable lots.   
742  d.  Facility A manufactures the product and Facility B packages the product.  Facility 
743  B discovers a defect that leads to the rejection of the lot; the defect was due to the 
744  manufacturing at Facility A.  In this situation, Facility A should not count this 
745  product lot as a released lot, despite the initial release.  For Facility B, if the 
746  defect was discovered upon incoming acceptance testing and the packaging lot 
747  was not yet started, the lot should not be counted.  If a packaging lot was started, 
748  it should be counted as a lot started, not as a released lot.   
749  e.  For a non-functional or functional film-coated tablet where the coating process 
750  consists of multiple separate coating pan loads, the count of lots depends on 
751  whether the separate pan loads are considered unique lots or if the loads are part 
  26  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
752  of a single started lot.  For either functional or non-functional coatings, samples 
753  collected and testing for finished drug product release should be representative of 
754  the lot.  
755  f.  Facility A initiates manufacturing of Product Z in the last quarter of the reporting 
756  cycle or ceases manufacturing of Product Y in the first quarter of the reporting 
757  cycle  An explanation of the partial year can be described in the comment field.  
758  The product report or site report would be considered complete for that product. 
759  (2)  Product Quality Complaint Rate 
760  a.  If a lot is distributed to five customers and all customers report the same 
761  complaint, this should be counted as five complaints. 
762   
763  b.  If a lot is distributed and a single customer submits the same complaint from 
764  different departments, only a single complaint should be counted.  If submitting a 
765  site report, the covered establishment may choose to include this complaint in 
766  their data if it is the least burdensome option. 
767   
768  c.  A lot is distributed to three regions and a complaint is received on that lot from a 
769  region outside of the United States.  In this instance, the complaint does not need 
770  to be reported as part of the quality metrics program.  The covered establishment 
771  may choose to include this complaint if it could be applicable to product imported 
772  or intended for import to the United States or its territories. 
773   
774  d.  For a site report by a packager, if a complaint is received and potentially due to 
775  the packagers operations (e.g., discolored tablet or powder), the complaint should 
776  be counted by the site reporting establishment.   
777   
778  (3)  Invalidated Out of Specification (OOS) Result Rate 
779  a.  Regarding analytical tests with multiple sample preparations or injections 
780  involved in the test to generate the final result, one test is represented by a single 
781  analytical result with an established limit.  For example, one content uniformity 
782  test proceeding to stage two may have 30 invalidated results.  Only one OOS 
783  result would be counted.   
784   
785  b.  If two samples from one lot are tested with two injections each and there is one 
786  result reported on the Certificate of Analysis, this is considered one release test.  
787   
788  c.  If an OOS result occurs during in-process testing for a test that is considered a 
789  real time release test, this is considered a release OOS result for the purpose of 
790  this guidance. 
  27  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
791  d.  If more than one OOS result is observed during finished drug product testing 
792  (e.g., the lot fails both assay and uniformity), this is considered multiple release 
793  OOS results. 
794  e.  50 kg of an API is packaged into five 10 kg packages and three to five of the five 
795  containers are tested; the Certificate of Analysis reports the average.  If one or 
796  more of the container results is OOS for the same attribute, the establishment 
797  should initiate an OOS investigation and count these OOS results as a single OOS 
798  result.  A single API container with an OOS result should result in an 
799  investigation for the lot in its entirety.  After the investigation is complete, 
800  subsequent retesting should be counted as a new release test.   
801  f.  Company A does not declare an OOS result until the laboratory investigation 
802  proves the result is valid.63  If invalid, and the original result is not labeled as an 
803  OOS, there will be no record of invalidating an OOS result, thus resulting in a 
804  lower Invalidated OOS Rate for Company A.  For the purpose of the quality 
805  metrics program, a lot release OOS result should be counted prior to the 
806  laboratory investigation, in accordance with the term OOS result as defined in 
807  this guidance.  Furthermore, these type of results should be evaluated as part of 
808  the PPR to determine the need for changes in drug product specifications or 
809  manufacturing or control procedures.64   
                                                 
63 It should be noted that this practice is inconsistent with the recommendations outlined in FDA guidance for 
industry Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical Production.  
64 Refer to 21 CFR 211.180(e). 
  28