# Presentation: 380465

## Validation of Rapid Microbiology Systems

- Based upon
- PDA Technical Report No. 33

- (c) 2002 Jeanne Moldenhauer

**Notes:**
I was asked by Russ Madsen to provide a presentation on the PDA Technical Report No. 33, Evaluation, Validation and Implementation of New Microbiological Testing Methods.  This document was generated by a large group of individuals representing industry, equipment vendors, and regulatory/compendial/standards groups.  The group was chaired by Dr. Tony Cundell of Wyeth Laboratories./

## Task Force Members

- Brian Bauer, Ph.D., Merck & Co.
- Mark Claerbout, Lilly Research Laboratories
- Warren Casey, Ph.D., GlaxoWellcome R&D
- Anthony M. Cundell, Ph.D., Wyeth-Ayerst Pharmaceuticals, Pearl River, New York (Chair)
- Martin Easter, Ph.D., Celsis Ltd.,
- Edward Fitzgerald, Ph.D. Consultant, (USP Microbiology Subcommittee)
- Carol Gravens, BioMerieux, Inc.
- David Hussong, Ph.D., CDER, FDA
- Michael Korcynzski, Ph.D., PDA Training Institute (USP Microbiology Subcommittee)
- Robin Lerchen, American Pharmaceutical Partners
- Frederic J. Marsik, Ph.D. CDER, FDA
- Amy Meszaros, StatProbe Inc
- Jeanne Moldenhauer, Ph.D., Jordan Pharmaceuticals, Inc.
- Manju Sethi, Qualicon
- Scott Sutton, Ph.D., Alcon Laboratories (USP Microbiology Subcommitee)
- Martin Tricarico, Chemunex (USA)
- Amanda Turton, Millipore Corp
- Christine Vojt, Johnson & Johnson Diagnostics Inc.
- Kirsty Wills, Celsis Ltd.
- Jon Wuannlund, Becton Dickinson Microbiology Systems

- (c) 2002 Jeanne Moldenhauer

**Notes:**
Although this slide may be a bit hard to read, I included it to show the diversity of the group that generated the document.  We were very fortunate to have representation by FDA and USP to aid in this task.  The group also included members of the international community.

## Disclaimer

- Presentation represents the views published in the PDA Technical Report
- May not in all ways be consistent with my personal views regarding the validation methods

- (c) 2002 Jeanne Moldenhauer

**Notes:**
If I have a differing view, I will try to identify it.  Like all documents generated by PDA, this report is a living document and as one gains more experience in a specific field, the document may require update or revision to include the broader base of knowledge.

## Document Contents 

- Part I : Selection of New Microbiological Methods
- 1.0  Introduction
- 2.0  Technology Overview
- Generic Description of Types of Methods
- Technology Review
- Growth-based technologies
- Viability-based technologies
- Artifact based technologies
- Nucleic Acid-based technologies
- 3.0  Regulatory Review

- (c) 2002 Jeanne Moldenhauer

## Document Contents 

- Part II: How to Validate and Implement New Microbiological Methods
- 4.0  The Validation Process
- Equipment Model
- Validation Criteria
- Special Considerations for Microbiology
- 5.0  Glossary
- 6.0  References

- (c) 2002 Jeanne Moldenhauer

**Notes:**
One of the key items in this listing is the recognition that there are special considerations and issues with the validation of microbiological test methodology.

## Implementation Requires Validation

- May require regulatory prior approval
- Many are not equivalent to traditional methods
- May be little or no guidance available for validation of microbial methods
- Many have associated computer-related systems and 21 CFR Part 11 issues
- New technology frequently requires new learning curves
- Reluctance to change

- (c) 2002 Jeanne Moldenhauer

**Notes:**
Since the time when this document was issued, Pharmacopeal Forum, has proposed monographs on the validation of microbiological methods and the acceptance criteria for validation of microbiological methods, I.e., redefining the validation terms used for chemistry into microbiological terms.

## What Validation is Not

- Method/technology selection
- Evaluation of feasibility of the method
- Proof of concept testing
- Vendor selection
- Doing it differently for economic considerations

- (c) 2002 Jeanne Moldenhauer

**Notes:**
All of the above mentioned steps are important, and should be done, but they are all pre-validation type of tasks.

## Typical Flow for Selection,Evaluation & Validation for a Rapid Method

**Is current compendial or industry**
**standard method meeting**
**all of your company’s needs?**

**Determine****what test**
**requirements****and ****specifications ****are (faster,****less labor ****intensive, etc.)**

**Look at****alternate****methods**
**and see if the****method can**
**meet the****specified****requirements**

**Perform****sufficient**
**feasibility****proof of**
**concept****testing**

**Does**** testing yield**
**acceptable****results****?**

**Plan and****execute**
**validation****protocol.**
**Were results**** acceptable****?**

**Submit regulatory**
**supplement if required**

**Implement test (after approval)**

**Evaluate other alternate methods,****rejecting the unacceptable method**

**Continue using existing method**

- No

- No

- No

- No

- Yes

- Yes

- Yes

- (c) 2002 Jeanne Moldenhauer

**Notes:**
If for example, the other product testing required always takes a minimum of 40 days, there may not be the same economic benefit to go to a rapid method, just because you could do it faster. 

Also, not all “rapid” methods are created equally.  If a colilert will do what you want, you don’t necessarily need a hunderds of thousands of dollars piece of rapid micro equipment.

## Validation Definition

- Established documented evidencewhich provides a high degree ofassurance that a specific process will consistently produce a product meeting pre-determined specificationsand quality attributes

- (c) 2002 Jeanne Moldenhauer

**Notes:**
Used the typical definition from the existing regulatory documents.

## The Equipment Validation Model

**Method/Vendor Requirements **(Defined)
**Validation Plan Design **
- I**nstallation Qualification (IQ)**
**Operational Qualification (OQ)**
**Performance Qualification (PQ)**
**Verify and document the system performs as specified using selected compendial organisms and environmental isolates for routine testing of batches of product.**

- (c) 2002 Jeanne Moldenhauer

**Notes:**

Vendor requirements lead to vendor specifications.  A special issue to us today is 21 CFR Part 11 compliance.

The Plan :  Design and document plan objectives, methods and acceptance criteria.

From a personal perspective, I differ in the area of the PQ.  I find it easier and more productive to treat the PQ of the rapid microbiology system to be conducted using a standard solution, e.g., water, and comparing it to vendor data similar to the LAL validation methods and then testing individual solutions separately as part of the product validations.  The reason for this is the elimination of product inhibition or other adverse effects when trying to qualify the equipment.

## Intentions of Validation

- Demonstration data which shows:
-  Accuracy of method   (appropriateness)
-  Reproducibility of method    (continuity)

- (c) 2002 Jeanne Moldenhauer

## Unique Concerns for Developing ValidationRequirements for Rapid Microbiological Methods

- What type of validation support does the vendor have? (e.g., DMF, sample protocols)
- Is the new method “equivalent” to existing method
- If side-by-side studies are used, what is the impact of the rapid method being “out-of-specification”, when the traditional method is within limits?
- Is the new technology/method subject to different variabilities than traditional methods?
- Can the method handle the expected laboratory throughput?

- (c) 2002 Jeanne Moldenhauer

## Unique Concerns for Developing ValidationRequirements for Rapid Microbiological Methods (continued)

- USP definitions for validation terms(e.g., accuracy, precision, ruggedness . . . ) are written specifically for chemistry applications while automated microbiological methods probably should take these definitions into account  {USP has drafted  a chapter to resolve this}
- Are there other human interface concerns that are not present in the compendial methods?
- Difficulties in setting limits for the “unknown”

- (c) 2002 Jeanne Moldenhauer

**Notes:**
USP <1223> Validation of Alternative Methods

Was discussed in the USP / PDA Open Conference in Sanibel Island, May 2002.

## Establishing Validation Criteria

- USP Chapter <1225> Validation of Compendial Methods gives criteria, but definitions are in terms of chemical methods
- Draft revision in *Pharmacopeial Forum *with criteria for Alternative Methods.
- PDA Technical Report 33 has issued microbiological definitions of these same criteria
- Since some tests are quantitative and others are qualitative (e.g., limits tests), not all criteria are applicable to each test type

- (c) 2002 Jeanne Moldenhauer

**Notes:**
USP revision  is similar to the PDA TR 33.  You still have time to send in your comments.  It isn’t official yet.  Sadly, very few responded (positively or negatively) to the proposed change.

## Validation Criteria

- (c) 2002 Jeanne Moldenhauer

## Accuracy

- Definition: A measure of the exactness of the microbiological method that is true for all practical purposes
- Determination: Analyze at least five suspensions across the range of test. Calculate each suspension as a percentage dilution of the original.
- Acceptance criteria: Should be equivalent or better than existing methods (±30% are considered equivalent)

- (c) 2002 Jeanne Moldenhauer

## Precision

- Definition: Precision may be a measure of either the degree of reproducibility of the microbiological method under normal operating conditions
- 	Repeatability refers to use of the method within the same laboratory over a short period of time using the same analysts and same equipment
- 	Reproducibility uses different analysts with the same equipment

- (c) 2002 Jeanne Moldenhauer

## Precision (continued) 

- Determination: at least five suspensions across the range of the test should be analyzed. Each suspension should be analyzed for at least ten replicates. Standard deviation or coefficient of variation should be calculated.
- Acceptance criteria: generally, a coefficient of variation in the 15-30% range is acceptable

- (c) 2002 Jeanne Moldenhauer

## Precision (continued)

- For comparing precision of two methods,the F-test is recommended. If the critical value exceeds the statistical table, a significant difference exists between the methods.

- (c) 2002 Jeanne Moldenhauer

## Specificity

- Definition: the ability of a method to detect a range of microorganisms which demonstrate that the method is fit for use
- Determination: screen against representative range of microorganisms and sample types
- Acceptance criteria: all selected organisms are isolated and enumerated from sample matrices

- (c) 2002 Jeanne Moldenhauer

## Limit of Detection

- Definition: a parameter of a limit test which identifies the lowest number of microbes which can be detected, but not necessarily quantified in the test conditions
- Determination: multiple replicates, greater than or equal to five, should be tested, since one cannot reliably obtain samples with a count of one

- (c) 2002 Jeanne Moldenhauer

## Limit of Detection (continued)

- Acceptance criteria: ideally, if one organism is tested, it will be detected in the time frame of the assay (use of CHI-square test)

- (c) 2002 Jeanne Moldenhauer

## Limit of Quantification

- Definition: the lowest number of microbes which can be determined with acceptable precision and accuracy under the slated experimental conditions
- Determination: since there are no reliable methods to get a sample count of one, greater than or equal to five replicates should be used
- Acceptance criteria: if a single organism is present, it will be quantified in the time frame of the assay

- (c) 2002 Jeanne Moldenhauer

## Linearity

- Definition: the ability to elicit results thatare proportional to the concentration of microbes present in the sample within a given range
- Determination: at least five replicates across range of assay
- Acceptance criteria: correlation coefficient r2 = 0.9, or better with a slope not diverging more than 20% from 1.0, i.e., r2 = 0.8 - 1.2

- (c) 2002 Jeanne Moldenhauer

## Range

- Definition: the interval between the upper and lower levels of microbes that have demonstrated to be determined with precision, accuracy and linearity, using the method as written
- Determination: the range of the method is validated by verifying that the analytical method provides acceptable precision, accuracy and linearity, when applied to samples at the extreme of the range and within the range
- Acceptance criteria: depends upon performance characteristics of the method

- (c) 2002 Jeanne Moldenhauer

## Ruggedness

- Definition: degree of reproducibility of test results obtained by samples under variety of normal test conditions, e.g., different analysts, instruments, lots of reagents
- Determination: prepare a suspension of microbes and test at least ten replicates against each assay variable. Calculate standard deviation or coefficient of variation.
- Acceptance criteria: generally, a coefficient of variation in the 10-15% range is acceptable

- (c) 2002 Jeanne Moldenhauer

## Robustness

- Definition: a measure of its capacity to remain unaffected by small but deliberate variations in method parameters and provides an indication of its reliability during normal usage
- Determination: the manufacturer investigates changes up to 20% on the critical reagent concentrations, instrument operation parameters and incubation temperatures
- Acceptance criteria: review results needto be reviewed against the manufacturer’s quality requirements and instructions for use

- (c) 2002 Jeanne Moldenhauer

## Equivalence

- Definition: a measure of how similar the test results are to the method it is intended to replace
- Determination: should first be demonstrated in pure culture work. Requires the two methods be run in parallel for comparison. It’s important to have randomized experimental design. A large number of replicates shouldbe used.
- Acceptance criteria: generally, a statistical test is used

- (c) 2002 Jeanne Moldenhauer

## Microbiological Reference Standards

- Issues associated with identification and determination of material to be used as a reference standard for this type of testing.
- TR33 highlights the need to show equivalence to existing methods, but the technology for many of these systems is superior to the old method and may not show equivalence.
- Provides some guidance on preparing in-house standards.

- (c) 2002 Jeanne Moldenhauer

## Method Validation

**Variability of Microbiological Methods**
**Sample Distribution Error**
**Cell Morphology**
**Metabolic Activity**

- (c) 2002 Jeanne Moldenhauer

**Notes:**
Variability

An important point to consider during the validation of a new microbiological test method is the inherent variability in microbiology.  There are three sources of variation: sample distribution error, cell morphology and metabolic activity.

For any given test procedure, the relative importance and contribution from these sources will depend on the principle of the test method and must be carefully considered.

Sample Distribution Error
Distribution error is the biggest source of error contributing to the variation in microbiological test results.  The natural distribution of microorganisms is heterogeneous and rarely follows normal Gausian distribution even after log transformation.  The distribution tends towards a negative binomial or Poisson distribution and is extremely difficult to assess and predict particularly at the low contamination levels experienced in clean room environmental samples and purified water systems.  These facts are frequently overlooked in our assumptions and interpretation of the results from microbiological tests.
The level of assurance provided by results from a microbiological test is questionable and a function of both the efficiency of the adopted sampling plan and the test method.  Extrapolation of the results to the whole batch of a product requires assurance that the product was produced under homogeneous conditions.  Care should be exercised in the design of validation experiments to account for such distribution error.

Cell Morphology
Traditional culture methods detect and /or enumerate microorganisms by monitoring changes in turbidity or by counting colonies visible to the naked eye.  For enumeration, the assumption is that one colony forming unit is derived from a single organism that was uniformly distributed within the test aliquot.  Yet microorganisms have a variety of morphologies and can occur singly and in pairs, tetrads or irregular clusters.  Microorganisms also have a tendency to colonize surfaces and form biofilms, which may also affect results.  The number of colony forming units (cell density) in a plated sample directly affects the colonial morphology and the accuracy of the viable count estimate.

Consequently, the coefficient of variance for microbiological methods is large.  To validate any alternative method, the variability must be determined and compared with that of an equivalent traditional method or test procedures recommended by the Pharmacopoeiae.

Metabolic Activity
Successful detection and/or enumeration of microorganisms is influenced by their metabolic activity, genotype and readiness for  growth.  In any microbiological test method the presence of interfering factors must also be considered.  Microorganisms may be stressed due to exposure to processing, environmental and experimental conditions or inhibitory components in the product itself.  Stressed cells may require a period of resuscitation and repair before they can be detected by cultural methods.  Inactivating agents added to the media can neutralize some products which stress cells and inhibit their growth.  Conversely, some products may contain nutrients sufficient to support microbial survival or even growth.

The validation of alternative methods should include, therefore, an evaluation of real samples taken from a variety of environmental and process and product types relevant to the test application.  The results must be compared to the recommended method.  For non-cultural methods where the detection principle is based on metabolic activity, the effect of these stresses on the detection principle and test results should be evaluated in terms of the performance characteristics and assessed against the recommended method.

## Other Issues in Validation

- Use of multiple identical equipment within the laboratory and company
- Whether or not you are the first or fiftieth company to submit this test method
- Company’s regulatory compliance status
- Unique issues with microbiology equipment
- Regulatory requirements, e.g., LAL
- Impact on other system, e.g., sterility testing
- Etc.

- (c) 2002 Jeanne Moldenhauer