# Presentation: 002562

## Risk Ranking and Filtering and Its Role in Risk Management 

- H. Gregg Claycamp, Ph.D., CHP
- Center for Veterinary Medicine
- July 21, 2004

*The materials presented here are opinions of the author and do not represent policy of the FDA.*

## Risk is a Concept

_*Risk*_* is intuitive and familiar to everyone, yet it can be sophisticated and elusive when organizations seek definitions of risk for specific risk management programs.*

## Risk Management

- Risk *assessment* is not a single process, but  “a systematic approach to organizing and analysing scientific knowledge and information” that supports a risk decision. 	NRC (1994)
- Risk *management* is a systematic process for the identification, assessment, control and communication of risks to life, property, or other valued objects.

## Premises

*As a broad *_*concept*_*, risk inherently has many possible meanings depending on the individual or organization. *
- Any effort as complex in scope as the FDA’s risk initiative necessarily defines risk at different contextual levels and can do so without departure from the mission to reduce, manage or control risk to public health.

## Levels of Risk Management

## Multiple Levels of Risk Management

- As used here,
- “high-level” refers to broadly-based, general and principle-driven approaches.
- “low-level” refers to detailed, specific and discipline-driven approaches.
- There is a hierarchy in processes and systems.
- Risk Ranking and Filtering is a high-level approach (or *process*).
*Examples...*

## Sources of Risk from a Medical Product

**Source:** adapted from FDA  (1999). *Managing the Risks from Medical Product Use*.

## Sources of Risk from a Medical Product

- Medication or Device Error

- Product Defects

- Preventable Adverse Events

- Injury or Death

- Unexpected Consequences

**Source:** basic model adapted from FDA  (1999). *Managing the Risks from Medical Product Use*.

- Links?

## Dual Impact of Quality Systems 

- Dual Impact of Quality Systems

- Medication and Device Error

- Product Defects

- Preventable Adverse Events

- Injury or Death

**Source:** modified from FDA (1999). *Managing the Risks from Medical Product Use*.

*Quality Systems can decrease the chances of manufacturing product defects; and, given that defects can occur, QS can also decrease the chances that a defective product will reach a patient.*

## Risk Tools Supporting Quality Systems 

- Product Defects

**FMEA**

**Fault Trees**

**HACCP**

**PRA**

**RCA**

**Others**

*These tools are helpful for focusing on assessing and managing risks given a specific product or product class.*

## Risk Tools for High-Level Prioritization Among Many Products

**Hierarchical Holographic Modeling**

**Risk Ranking and Filtering**

**Risk Matrices**

*Higher level tools are needed for higher level risk questions, e.g., prioritization of products/sites for GMP inspections.*

**. . .**

## Risk Questions and Tools Change With the Level of Analysis

- “***Low” level:***  	       Risk questions focus on identifying and characterizing risks to drug quality for specific drug products or within a specific products classes.
- Quantitative and qualitative tools available.
- Analysis-driven.

***“******High” level:***                 Risk questions focus on how risks within different drug/product classes compare with each other.
- Risk analysis tools are essentially customized for each application.
- Principle-driven.

## Low-Level Example

## Low Level Modeling (Fault Tree Analysis) 

- Low Level Modeling (Fault Tree Analysis)

- Example:

## Faults/Pathways Magnified N-fold for a Simple Manufacturing Process!

## Why Use High-Level Systems Methods in  Risk Management?

- Low-level approaches are elegant and capture details, but may *miss* interactions and relevance across systems.
- Complex quantitative models may convey a level of precision and understanding about the system that is unjustified.
- Different levels of understanding and quantification may exist for each sub-component of the system.  High-level methods seek optimal use of diverse kinds of information to inform risk decisions.

## High-Level Models for Risk Management

- Systems approaches/thinking
- Risk management of complex systems is
- Multi-objective
- Multi-decision maker
- Hierarchical (overlapped)
- Sometimes conflicted/confounded
- Complex systems exceed human capacity to capture everything in a simple model.

## High-Level Risk Management Begins With Brainstorming (HHM)

- ...

- ...

*Which risk endpoints** **are potentially of interest for risk management?*

*SAMPLE CHART*

- ...

## A High-Level Approach

## Drilling Down to Sources of Risk for Model Building

- ...

*SAMPLE CHART ONLY*

## Systematically Developing the Low-Level Details

- ...

- Example *processes* for which defects might affect product sterility.

*Low-level risk analysis can be quantitative, relying on FMEA, Fault Trees, or other risk analytical approaches.  Alternatively, data gaps may be filled with estimates from expert elicitation   *

*SAMPLE CHART ONLY*

## Sometimes, Only Qualitative Information is Available for a Specific Product or Process

| Health

Severity Scale  | Probability of Occurrence |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  | Very Low | Low | Medium | High | Very High |
| Death | Medium | Medium | High | High | High |
| Chronic Illness | Low | Medium | Medium | High | High |
| Acute Illness | Low | Medium | Medium | High  | High |
| Worry | Low | Low | Low | Medium | Medium |

*Risk Estimate Based on Probability and Severity Scoring:*

## High-Level Combinations of Severity and Probability

- Increasing Severity of Harm/Consequence

- Increasing Probability of Occurrence

***Low Risk***

***Medium Risk***

***High Risk***

## Risk Ranking & Filtering (e.g., Haimes, 1998)

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |
|  |  |  |  |  |  |

**Product **

**Process **

**“****Other”**

**(Risk Ranking and Filtering) **

*SAMPLE CHART*

## Filtering:  Policy Meets Risk Management 

## The “Filtering” in RRF

- Once risks/hazards are ranked a “filter” may be used to reflect resources limitations and/or programmatic goals.
- Filters are policy-derived. For example,
- Selecting worst *N (*or* X%) *of risks across all organizational units; versus
- Selecting worst *M *(or* Y%*) of risks for the entire organization.
- Filters may have a risk, resource, or other bases, each possibly imparting differential effects on the final ranking of risks for mitigation.
*Example:  Next slide*

## Using RRF Results: Filtering 

- Using RRF Results: Filtering

## RRF in the Risk Analysis Cycle

**Multi-Factorial Risk Model **

**Risk Ranking and Filtering**

**Work Planning**

**cGMP/Compliance Inspections**

**Assessments (Data Bases)**

***Start***

**Other Factors**

## “Risk management and decision-making are all about [confronting probabilities] and where the balance between measurement and gut becomes the focal point of the whole story.”  

(P.L. Bernstein, 1996, Against the Gods: 
The Remarkable Story of Risk p. 56)

- (P.L. Bernstein, 1996, *Against the Gods: *
*The Remarkable Story of Risk* p. 56)