# Presentation: 502553

## Details of FDA’s Pilot Risk-Ranking Model for GMP Inspections

- Nga Tran, Dr.P.H., Exponent
- Brian J. Hasselbalch, FDA
- July  2004

- March 16, 2004

## Application of Risk Ranking

- Regulatory programs:
- EPA, Cal EPA
- USDA
- Risk management
- DoD
- Industry

- March 16, 2004

## Examples of Risk Ranking Models

**Examples of Risk Ranking Models**

- March 16, 2004

## A regulatory decision tool 
Foundation for the EPA’s Resource Conservation and Recovery Act Persistent, Bioaccumulative, and Toxic (PBT) List of chemicals. 
Framework of expert judgment 
Identify chemicals or emissions of potential concern 
Using key physical-chemical properties and associated cutoff criteria.  

- Foundation for the EPA’s Resource Conservation and Recovery Act Persistent, Bioaccumulative, and Toxic (PBT) List of chemicals.
- Framework of expert judgment
- Identify chemicals or emissions of potential concern
- Using key physical-chemical properties and associated cutoff criteria.

**EPA Waste Minimization Prioritization Tool (WMPT)**

- March 16, 2004

## EPA Waste Minimization Prioritization Tool (WMPT) -- Scoring system 

**EPA Waste Minimization Prioritization Tool (WMPT) -- Scoring system**

- March 16, 2004

## The TRI Relative Risk-Based Environmental Indicator 

**The TRI Relative Risk-Based Environmental Indicator **
**(Bowves et al, 1997)**** **

- March 16, 2004

## A Hierarchical Framework of Scoring and Ranking Methodology, Pennington and Yue (2001) 

***A Hierarchical Framework of Scoring and Ranking Methodology, *****Pennington and Yue (2001)**** **

- March 16, 2004

## DoD

- Comparative risk pre-deployment
- Rank locations based on potential chemical risks
- Risk score is a function of volume and inherent chemical toxicity
- Comparative risk during deployment
- Ranking and comparing disparate risks
- Quick/simple tools

- March 16, 2004

## Risk Assessment Matrix

**U.S. Army Field Manual 3-100.12.**

**E – Extremely high risk**
**H – High risk**
**M – Moderate risk**
**L – Low risk**

- March 16, 2004

## Risk Level Definitions

**(US CHPPM, Technical Guidance-248) **** **

- March 16, 2004

## Hazard Severity Ranking Chart for Military Deployments 

**(US CHPPM, Technical Guidance-248) **

- March 16, 2004

## Chemical Hazard Probability Ranking Chart for Military Deployments

**(US CHPPM, Technical Guidance 230)**

- March 16, 2004

## Consumer Product Example

- Ranking to target products for which additional toxicity testing of High Production Volume chemical is needed.
- Laundry detergent, consumer care products, etc..
- Many products, limited resources
- Ranking based on exposure to consumer products
- “High end” product use frequency (e.g., number of times use per day) as surrogate for exposure

- March 16, 2004

## Example Consumer Product Exposure Model for Ranking

**Example Consumer Product Exposure Model for Ranking**
**Dermal Direct Contact**

- March 16, 2004

## Ranking of  Consumer Product by Exposures to High Production Volume Chemical A 

- March 16, 2004

## Food Microbiological Hazard Example Van Gerwen et al (2000) 

- SIEFE: Stepwise and Interactive Evaluation of Food Safety by an Expert System.
- A decision support system for microbiological risk assessment for food products and their production processes
- Risks are first assessed broadly, using order of magnitude estimates, i.e. level 1 risk assessment.
- Characteristic numbers are used to quantitatively characterize microbial behavior during the production process to highlight the major risk-determining phenomena

- March 16, 2004

## Categorization of the probability of having problems, growth and inactivation by characteristic numbers 

**PC (probability characterization); SC (step characterization); OC is the characteristic number for occurrence characteristic; HC for health problem characterization.**

- March 16, 2004

## Food risk ranking model (Ross and Sumner, 2002) 

- Tool to compare food-borne risks for ranking and prioritizing risks from diverse sources.
- 11 questions relating to 3 main factors:
- severity of hazard
- likelihood of disease causing dose of hazard being present in a meal
- probability of exposure to the hazard

- March 16, 2004

## Slide 19

- March 16, 2004

## Comments

**Comments**

- March 16, 2004

## Comments

**Comments**

**Weights**

- March 16, 2004

## USDA- Food Safety and Inspection Service  

**"Inspector Optimization System" (IOS)**
- Basis of assigning the processing inspector workforce to provide for greater flexibility in targeting inspection resources to areas of greater public health risk and concern
- Report to Congress on Risk-Based Inspection to the United States House and Senate Appropriations Committees, March 2, 2001
**Each plant receives a Food Safety Hazard Coefficient (HC)**
- Inherent hazards
- Expert Elicitation to rank inherent hazards

- March 16, 2004

## Food Safety Hazard Coefficient 

**Species variable** designed to reflect the _inherent_ biological, chemical, and physical hazards in meat and poultry arriving at inspected establishments
- P**rocess variable** captures the _inherent _hazards (biological, chemical, and physical) of the establishment’s operations
- Take into account how a process normally works, the likelihood of fluctuations or deviations from the norm, the effect these fluctuations or deviations may have on hazards in the product, and the potential resulting implications for public health as the product leaves the establishment.
**Volume variable – **facility size** **correlated with the number of persons who consume food from the establishment (exposure).

- March 16, 2004

## Expert elicitation

- Steers/Heifers      Lambs/Sheep/GoatsVeal/Calves         Young ChickensCows/Bulls          Young TurkeysMarket Hogs       Older PoultrySows/Boars         Ducks/Geese

- To determine the species and process hazards
- Two groups of experts  (species and processing)
- Government, academia, and industry.

**Expert elicitation**

- March 16, 2004

## Process Categories Used in Hazard Elicitation

- Raw product, ground              Heat-treated, shelf-stable product
- Raw product, not ground        Heat-treated, but not fully cooked, not shelf-stable
- Slaughter                                Fully cooked, not shelf-stable product
- Thermally processed,              Not heat-treated, shelf-stable product commercially-sterile
- Secondary inhibitors,
- not shelf-stable product

- March 16, 2004

## Slide 26

- March 16, 2004

## The CDER-OC process

- Began one year ago
- Internal experts from CDER, CVM, CBER, and ORA (Risk Mgmt & Workplan WG)
- Generated a list of risk factors that are descriptors of site risks for consideration in development of site selection model
- Qualitatively valued: High, Medium, Low

- March 16, 2004

## Examples of risk factors

|  | Factor | Risk |
| --- | --- | --- |
|  | Administered through membranes | Medium |
|  | Aseptically produced finished dosage forms | High |
|  | Dermatological OTC liquids/creams/ointments/powders with no daily dose limit | Low |
|  | Excipients | Low |
|  | Aseptic fill using isolator (self-contained isolators and form-fill-seal), non-terminal sterilized | Medium |
|  | Aseptic sterile filtration and filling (non-isolator, non-terminal sterilization) | High |
|  | Assembly/fabrication processing (as opposed to batch processing) | High |
|  | Cartoning / packaging | Low |
|  | Large number of approvals/time | Low |
|  | Complaints received suggesting Class I or Class II health hazard situations | High |
|  | Contract micronizers | Medium |
|  | First-time applicants/filers | Medium |

- March 16, 2004

## Develop Risk Ranking Framework

- High level risk management begins with organization
- Categorized list of risk factors into 3 components
- Product
- Process
- Facility

- March 16, 2004

## Categorization of risk factors, e.g.

| Component | Factor |  |
| --- | --- | --- |
| Product | Administered through membranes |  |
|  | Aseptically produced finished dosage forms |  |
|  | Dermatological OTC liquids/creams/ointments/powders with no daily dose limit |  |
|  | Excipients |  |
| Process | Aseptic fill using isolator (self-contained isolators and form-fill-seal), non-terminal sterilized |  |
|  | Aseptic sterile filtration and filling (non-isolator, non-terminal sterilization) |  |
|  | Assembly/fabrication processing (as opposed to batch processing) |  |
|  | Cartoning / packaging |  |
| Facility |  Large number of approvals/time |  |
|  | Complaints received suggesting Class I or Class II health hazard situations |  |
|  | Contract micronizers |  |
|  | First-time applicants/filers |  |

- March 16, 2004

## A RISK-BASED FRAMEWORK FOR PRIORITIZING SITES FOR cGMP INSPECTION

- PRODUCT

- PROCESS

- FACILITY

**A RISK-BASED FRAMEWORK FOR PRIORITIZING SITES FOR cGMP INSPECTION**

- Site risk potential (SRP):
- A function of the risk potentials for the PRODUCT, PROCESS and FACILITY component risk factors.
- The SRP is a weighted combination of the PRODUCT, PROCESS and FACILITY risk potentials
- Semi-quantitative risk potentials represented as  “high,” “medium,” or “low,” and are numerically discrete variables in calculations.

- March 16, 2004

## Drilling Down to Component Factors

- Selected factors for product, process and facility components
- Assigned weights
- Empirically derived
- Expert judgment
- Developed logical algorithm to combine factors to calculate a site score – *The Model*

- March 16, 2004

## Drilling Down and Populating the Product, Facility, and Process Components

- March 16, 2004

## Product Component Factors

- STERILITY

- MEDICAL GAS

- 1997 – 2003 Recall

- Severity
- HHE Class III, II, I

- Frequency

- Intrinsic Factors

- Site-Product Score

**SRP (Site Risk Potential)**

- Site-Process Score

- Site-Facility Score

- Product Scores

**Product Component Factors**

- March 16, 2004

## Product Component Data

- Data source: FACTS (Field Accomplishments and Compliance Tracking System)
- Site information/identifier
- Product codes
- CDER Recall Database
- Product codes
- Rx/OTC and Classification of Hazard
- Years:
- Recall: 1997-2004
- Product: 2000-2004

- March 16, 2004

## Recall Weight Matrix

- March 16, 2004

## Product Categorization

- Need to link recall data to sites
- Indirect linkage -- correlate product codes in recall data with product codes in site database
- Recall weights assigned to Recall-FACTs harmonized product codes

- March 16, 2004

## Examples of Product Code Correlation

| FACTS Process Indicator Code letter | Recall System Product Codes | Recall Code Description | General Category |
| --- | --- | --- | --- |
| Q | OAER | Aerosol, Inhalation/Systemic | inhaled |
| S | BULK | Bulk Pharmaceutical | API/Excipient |
| R | OPWRR | Oral Solid For Reconstitution | oral, solid, immediate release capsules/powders |
| A | OTABE | Tablet, Effervescent | oral, solid, immediate release tablets |
| A | OTABS | Tablet, Sublingual/Buccal | oral, solid, immediate release tablets |
| C | TEMR | Tablet, Extended/Modified Rel | oral, extended-release |
| B | OTABX | Tablet, Extended Release | oral, extended-release |
| E | OCAP | Oral Capsule | oral, solid, immediate release capsules/powders |
| E | OCAPH | Capsule, Hard Gelatin | oral, solid, immediate release capsules/powders |
| F | OCAPX | Oral Capsule Extended Release | oral, extended-release |
| L | TSPY | Topical Spray | topical, dermal |
| L | TSUS | Topical Suspension, Mixture | topical, dermal |
| L | TTCT | Topical Tincture | topical, dermal |

- March 16, 2004

## Facility Component Factors

- (est.)

- History of Compliance and Violation

- Site-Product Score

**SRP (Site Risk Potential**

- Site-Process Score

- Site-Facility Score

**Facility Component Factors**

- History of
-  inspection

- March 16, 2004

## Facility Component Data

- Data source: FACTS (Field Accomplishments and Compliance Tracking System)
- Years:  2000-2004
- All sites, foreign and domestic

- March 16, 2004

## Process Component Factors

- Relevant inherent process risk factors

- Site-Process Score

**SRP (Site Risk Potential)**

- Site-Facility Score

**Process Component Factors**

- Site-Product Score

- Producti, facilityj

- March 16, 2004

## Process Component Data

- No data, but do have staff with experience and knowledge
-   ***expert elicitation survey***
- Elicitation survey drafting began Nov. 2003
- Inter-Center/ORA workgroup effort
- ORA; CVM; CBER; CDER; CDER; OC

- March 16, 2004

## Expert Elicitation Survey Challenges

- What are the relevant **process**-related risk factors? (i.e., sources of variability and poor quality)
- What, if any, unit operations are more liable to a loss of control or at risk to contamination?
- What products? Answers are product dependent
- Large number of products
- Identify “mutually exclusive” category of products -- tradeoffs in number of product categories and responder burden

- March 16, 2004

## Starting point methodology... ISPE Baseline Pharmaceutical Engineering Guide, vol. 2, Oral Solid Dosage Forms 

**Starting point methodology... ****ISPE Baseline Pharmaceutical Engineering Guide, vol. 2, Oral Solid Dosage Forms **

- March 16, 2004

## Defining product types and unit operations

- Product types
- Several CDER/agency coding schemes; too much
- Types selected are representative of most product types in the market
- Codes grouped into product types with similar unit ops
- Distinguish high/low actives
- Unit operations
- WG member knowledge +
*Remington: Science and Practice of Pharmacy*, 20th edition
*Modern Pharmaceutics*, 3rd edition
*Pharmaceutical Process Validation*, 3rd edition
- ISPE Baseline Pharmaceutical Engineering Guide, vol. 2, *Oral Solid Dosage Forms* (1st edition, 1998)

- March 16, 2004

## Slide 46

| Product group | DRLS / EES / FACTS profile class codes | Profile class description |
| --- | --- | --- |
| Solid oral immediate release | CHG | Capsules, Prompt Release |
|  | CSG | Capsules, Soft Gelatin |
|  | TCM | Tablets, Prompt Release |
| Solid oral modified release | TTR | Tablets, Extended Release |
|  | CTR | Capsules, Modified Release |
|  | TCT | Tablets, Delayed Response |
| Powders | POW | Powders (Includes Oral and Topical) |
| Semisolids | OIN | Ointment, Non-sterile (Includes Cream, Jelly, Paste) |
|  | SUP | Suppositories |
| Liquids, nonsterile (solution / suspension) | LIQ | Liquids (Solutions, Suspensions, Elixers and Tinctures) |
| Liquids, sterile | SVL | Small Volume Parenterals (Lypholized) |
|  | SVS | Sterile-Filled Small Volume Parenterals (Drugs) |
|  | SVT | Terminally Sterilized Small Parenterals |
|  | LVP | Large Volume Parenterals |
|  | SNI | Sterile Non-Injectable |
| Biotech | CBI | Biotechnology Crude Drug |
|  | TRP | Therapeutic Recombinant Products |
| Transdermal | TDP | Transdermal Patches |
| MDI | ADM | Aerosol Dispensed Medication |
| API synthesis | CEX | Plant/Animal Extraction Crude Drug |
|  | CRU | Crude Bulk Drugs - Non-Synthesized |
|  | CSN | Non-Sterile Bulk by Chemical Synthesis |
|  | CSS | Sterile Bulk by Chemical Synthesis |
| API fermentation | CFN | Non-Sterile Bulk by Fermentation Crude Drug |
|  | CFS | Sterile Bulk by Fermentation Crude Drug |

- March 16, 2004

## Slide 47

- March 16, 2004

## PROCESS RISK FACTORS

**A)  Process control**
- To what degree does this unit of operation contribute to **variability in quality** of the final product?
- 2)    How difficult is it to **maintain this unit of operation** in a state of control?
- 3)    If a problem does occur, how **reliable are the current detection** methods?

**B) Contamination**
- 4)  Is this unit of operation more or less **vulnerable to contamination from previous product**?
- 5)  Is this unit of operation more or less **vulnerable to contamination from the environment**?

- March 16, 2004

## Slide 49

| Solid oral drugs, immediate release (i.e., tablets, capsules), "High Active" |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Variables |  | Process control |  |  | Contamination |  |
| Questions |  | 1) To what degree does this unit of operation contribute to variability in quality of the final product?  | 2) How difficult is it to maintain this unit of operation in a state of control? | 3) If a problem does occur, how reliable are the current detection methods?   | 4) Is this unit of operation more or less vulnerable to contamination from previous product? | 5) Is this unit of operation more or less vulnerable to contamination from the environment? |
| Answering Choices |  | 1 = minimal 2 = minimal to moderate3 = moderate to high4 = high to very high5 = very high | 1 = slightly difficult2 = slightly to moderately difficult3 = moderately difficult4 = moderately to very difficult5 = very difficult | 1 = very reliable2 = very to moderately reliable3 = moderately reliable4 =moderately to slightly reliable5 = slightly reliable | 1 = slightly vulnerable2 = slightly to moderately vulnerable3 = moderately vulnerable4 = moderately to very vulnerable5 = very vulnerable | 1 = slightly vulnerable2 = slightly to moderately vulnerable3 = moderately vulnerable4 = moderately to very vulnerable5 = very vulnerable |
| Units of Operation | Measuring |   |   |   |   |   |
|  | Mixing/Blending |   |   |   |   |   |
|  | Wet Granulation |   |   |   |   |   |
|  | Dry Granulation |   |   |   |   |   |
|  | Milling |   |   |   |   |   |
|  | Drying |   |   |   |   |   |
|  | Compression (Tablet)  |   |   |   |   |   |
|  | Encapsulation (Hard Gel) |   |   |   |   |   |
|  | Coating |   |   |   |   |   |
|  | Pelleting |   |   |   |   |   |
|  | Primary packaging |   |   |   |   |   |

- March 16, 2004

## The survey also asks experts to rank products alone (not by unit of operation) by the general areas of concern:

**The survey also asks experts to rank products alone (not by unit of operation) by the general areas of concern:**

- March 16, 2004

## Survey Status

- Delivery By e-mail
- 50 FDA experts participated
- reviewers from CDER
- senior CDER compliance staff
- senior ORA field staff
- 90% response rate
- Results being analyzed

- March 16, 2004

## Data analysis

- Exploratory Data Analysis (EDA)
- Product rank consistent with unit of operation drill down data – correlation
- Developing process weights based on unit of operation drill down survey data
- K-mean cluster analysis
- Average rank per product category per question
- 	a) Process controls:  questions 1, 2 and 3
- 	b) Contamination potential: questions 4 and 5
- Coefficient of Variance (CV) Weighted average rank per product category per question
- Principal component and fuzzy arithmetic for expert categorical data analysis

- March 16, 2004

## Process Component Factors

- Process Controls

- Site-Process Score

**SRP (Site Risk Potential)**

- Site-Facility Score

**Process Component Factors**

- Site-Product Score

- Variability in quality
- Survey question 1

- Maintain state of control
- Survey question 2

- Previous product
- Survey question 4

- Environmental
- Survey question 5

- Reliable detection
- Survey question 3

- March 16, 2004

## Plain language summary 

- A site will tend to be less frequently selected for inspection if it:
- has been inspected recently and/or relatively few previous violations of GMPs and/or a smaller volume of product (facility weight);
- makes non-sterile, OTC drugs, and/or other product types that are not associated with a high frequency of serious recalls (product weight);
- makes products estimated to be relatively straightforward to manufacture and not vulnerable to contamination (process weight)

- March 16, 2004

## Current and Future Risk Factors in Site Score

| Modules | Factors in Baseline  SPR Score |  | Factors to Update SPR Score (including mitigating factors) |  | Field Inputs to Update SPR Score |  |
| --- | --- | --- | --- | --- | --- | --- |
| Product | Rx | x  | Recalls | x | Never covered? | future |
|  | Sterile | x | Other Defects | future |   |   |
|  | Not medical gas | x | Complaint | future |   |   |
|  | “Must work right” | future |   |   |   |   |
|  | Therapeutic range | future |   |   |   |   |
|  | Higher intrinsic toxicity, e.g., enhanced risk mgt controls, require periodic patient monitoring | future |   |   |   |   |
| Process | Expert Elicitation | x | Enhanced controls (e.g., PAT, process capability metrics) | future | New construc.? | future |
| Facility | Compliance history | x |   |   | Corp. changes? | future |
|  | History of Inspection  | x |  |  |  |  |
|  | Volume (est.) | x |   |   |  |   |
|  | Successful practices study | future |  |  |  |  |

- March 16, 2004

## Questions to the Subcommittee

***Can you identify alternative approaches that would systematically prioritize manufacturing sites for GMP inspections?***
***In what areas would additional data provide the most value added in prioritizing manufacturing sites for GMP inspections?***
***Are there other metrics that should be incorporated, e.g., measuring process control?***

- March 16, 2004