2011 UDM/OECD Technical Workshop 

# Reassessment of the UDM 

# Performance Standard

1 For Official Use Only/Do not Cite or Distribute Stephen F. Tomasino, Ph.D. U.S. Environmental Protection Agency Biological and Economic Analysis Division Office of Pesticide Programs Microbiology Laboratory2011 UDM/OECD Technical Workshop



# Presentation Overview

2 

## • Overall goals of the reassessment 

## • 2009 collaborative study and associated data 

## • Basic concepts of the statistical analysis • Proposal/recommendations • Path forward For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop 

# Overall Goals of the Reassessment • Address concerns with the UDM while the Agency evaluates and implements a new strategy/methodology • Use the best data available and apply the best available statistics • Increase our confidence in the outcome of the product performance testing while maintaining adequate stringency • Use a bundled, multi-faceted approach • Incorporate UDM revisions • Align the post registration testing with registration requirements as much as possible • Seek independent technical assessment, e.g., AOAC • If approved, provide detailed communication and instructions to end-users • Additional data generation by EPA on the UDM is highly unlikely 3 For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop 

# The Collaborative Study – key aspects • EPA works with the Efficacy Working Group • October 2008 – technical survey submitted to labs

– Collect information on how labs currently conduct the UDM

– Identify potential sources of variability • Method modifications identified – targeting the preparation of test cultures • Goal to reduce variability and improve the performance of the method • Perform equivalency testing to demonstrate what effect, if any, the modifications have on the outcome of the efficacy test. 4 For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop


### • Input from AOAC provided • January-March 2009 – EPA leads effort to develop the study protocol • Montana State University contract used to develop the data analysis section of the protocol 

### • Industry provided input to the test design and statistical methodology • August 2009 – collaborative study launched 5 For Official Use Only/Do not Cite or Distribute 

# The Collaborative Study – key aspects2011 UDM/OECD Technical Workshop

• Use of MLB’s SOP MB - 05 as the current method

– The modified version included all proposed modifications (i.e., each modification was not evaluated separately) • The modifications (MLB SOP MB - 05 and the following):

– Use of frozen stock cultures instead of stock cultures grown and stored on agar slants and stabs

– Reduce the number of 24 hr subculture periods from 3 to 1

– Use of an agar-based procedure to replace the current liquid culture protocol.
6 For Official Use Only/Do not Cite or Distribute 

# The Collaborative Study – key aspects2011 UDM/OECD Technical Workshop



### • 5 representative laboratories • Two microbes - S. aureus and P. aeruginosa • One quaternary ammonium product • Hard water and organic burden included • A mean minimum level of 6 logs per carrier – test cultures for the modified version were adjusted (diluted) based on optical density of the broth from the current method • One subculture tube (neutralizer) used for recovery per 

### carrier 7 For Official Use Only/Do not Cite or Distribute 

# The Collaborative Study – key aspects2011 UDM/OECD Technical Workshop


## • The study design called for testing the product at two levels of efficacy (high and low) with 

## the current and modified UDM in a side-byside fashion. • Three replications (test days) • 24 60- carrier tests per lab • Order of microbe randomized
8 For Official Use Only/Do not Cite or Distribute 

# The Collaborative Study – key aspects2011 UDM/OECD Technical Workshop 

# UDM Test Cultures

Pseudomonas Staphylococcus Pseudomonas pellicle For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop

10

Microbe Method Mean LD S r S R P. aeruginosa Current 6.71 0.25 0.38 P. aeruginosa Modified 6.68 0.36 0.37 S. aureus Current 6.64 0.16 0.30 S. aureus Modified 6.73 0.14 0.24 Statistical summary of the control counts

For Official Use Only/Do not Cite or Distribute 

# The Collaborative Study – data2011 UDM/OECD Technical Workshop

Control carrier log densities (mean +/- 1 SEM)

For P.a. , SEM = 0.14; for S.a. , SEM = 0.12. 

# The Collaborative Study – data

For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop

12

Laboratory Replication Current Method Modified Method 110 1 0 (Pass) 10 (Fail)

2 0 (Pass) 11(Fail) 3 0 (Pass) 3 (Fail) 112 1 1 (Pass) 1 (Pass)

2 1 (Pass) 2 (Fail) 3 1 (Pass) 1 (Pass) 114 1 3 (Fail) 1 (Pass)

2 2 (Fail) 2 (Fail) 3 1 (Pass) 0 (Pass) 116 1 1 (Pass) 0 (Pass)

2 2 (Fail) 2 (Fail) 3 1 (Pass ) 1 (Pass) 119 1 5 (Fail) 4 (Fail)

2 1 (Pass) 5 (Fail) 3 1 (Pass) 3 (Fail)

11/15 Pass 6/15 Pass Number of positive carriers and Pass/Fail outcome for the P. aeruginosa/high QAC efficacy treatment – the label claim

For Official Use Only/Do not Cite or Distribute 

# The Collaborative Study – data2011 UDM/OECD Technical Workshop 

# The Collaborative Study – data

13 P. aeruginosa2011 UDM/OECD Technical Workshop

14 Responsiveness for P. aeruginosa where each point represents the number of positive carriers from a single test per method.

For Official Use Only/Do not Cite or Distribute 

# The Collaborative Study – data2011 UDM/OECD Technical Workshop 

# The Collaborative Study – conclusions • Based on the S r and S R, the control Test LDs for P. aeruginosa were slightly more variable compared to S. aureus ; however, both methods exhibited acceptable and small S r and S R . • Across microbes and methods, the highest degree of variability was noted at the low efficacy level – this was not an unexpected result. • For P. aeruginosa , the mean number of positive carriers for the low efficacy treatment was 18.8 and 26.3 for the current and modified methods, respectively. • In contrast, for P. aeruginosa , the high efficacy treatment yielded a mean of 1.3 positives for the current method and 3.1 positives for the modified method. • With the exception of laboratory 116 for P. aeruginosa , both methods exhibited the ability to detect the presumed efficacy-response relationship. 152011 UDM/OECD Technical Workshop 

# The Collaborative Study – conclusions • The S r 

### and S R 

### were less for S. aureus tests than for the corresponding P. aeruginosa tests. • For each combination of microbe and efficacy level, both methods produced acceptably small S r 

### and S R 

### values for LR. • Across the combinations of microbe and efficacy level

– the S r values ranged from 0.15 to 0.33 for the current method and from 0.32 to 0.50 for the modified method

– the S R values ranged from 0.32 to 0.55 for the current method and from 0.39 to 1.10 for the modified method. 

### • Overall, the method modifications did not reduce 

### method variability. 162011 UDM/OECD Technical Workshop

For Official Use Only/Do not Cite or Distribute 

# The Collaborative Study – further analysis High Efficacy Treatment2011 UDM/OECD Technical Workshop Carrier counts in combination with the number of positives log reduction (LR) 

### For reassessment of the performance standard, conversion of number of positives to log reduction values – use of the MPN formula
Use of the MPN formula
For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop



# Use of the MPN formula • Widely used by FDA – see Blodgett, R.J., (2006) Most 

### Probable Number Determination from Serial Dilutions. FDA’s Bacteriological Analytical Manual • Used by EPA in the past – see Tomasino and Hamilton 

### (2006). Modification to the AOAC Sporicidal Activity of Disinfectants Test: Collaborative Study. JAOAC Int. 89:1373 - 1397 

### • See MSU’s “Knowledge Sharing Article” KSA- SM-08 for complete details on the P/N formula at http://www.biofilm.montana.edu/content/ksa- sm-08 19 For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop within-lab variability 

# Identifying sources of variability

For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop Among-lab variability 

# Identifying sources of variability

For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop Within-lab variability Among -lab variability Reproducibility SD = 0.5348 

# Identifying sources of variability

Using an assessment of variability, we can calculate the probability of events through modeling For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop 

# Important Questions to Ask • How often does a marginally ineffective 

### product pass?

– using the current performance standard

– using a new performance standard

– in combination with the number of tests conducted 

### • How often does a highly effective product fail?

– using the current performance standard

– using a new performance standard

– in combination with the number of tests conducted 23 For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop



# Proposal! • S. aureus

– 0- 3/60 positives is a pass

– > 3/60 positives is a failure

– Must conduct 3 independent tests; each must pass

– Must have a minimum of 6 logs of bacteria per carrier 

### • P. aeruginosa

– 0- 6/60 positives is a pass

– > 6/60 positives is a failure

– Must conduct 3 independent tests; each must pass

– Must have a minimum of 6 logs of bacteria per carrier 24 Performance standard for Salmonella enterica will not change. For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop 

# Overall Recommendations

Increased confidence in the outcome Revise performance standards Require multiple

testing of the same product Employ other

enhancements to the UDM such as the min/max control counts

25 For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop 

# AOAC approval and outreach • Two manuscripts will be prepared:

– First paper: Collaborative data and method performance • Submit to JAOAC by 01/31/2012

– Second paper: Statistical model and the reassessment of the performance standard • Submit to JAOAC by 02/29/2012

– Independent panel of experts, likely from AOAC Committee M, will be assigned for technical review

– Follow the official AOAC method revision process 

### • Concurrently, seek comment from stakeholders and other federal agencies (e.g., CDC) 26 For Official Use Only/Do not Cite or Distribute2011 UDM/OECD Technical Workshop 

# QUESTIONS/COMMENTS

27 For Official Use Only/Do not Cite or Distribute