Page 1 of 4 12/1/14



### Summary of EPA’s Responses to the June 25

- 28, 2013 FIFRA SAP’s Recommendations : Tier 2 Non

- Mammalian Tests

SAP comment Agency Position EPA Action SUMMARY OF SAP COMMENTS ON STATISTICS* General comments on statistics relevant to all assays: Statistical methods and analyses need more detailed description

. Agreed

. Statistics workshop held to assist in preparing more statistical guidance for each of the test methods. Additional details for randomization in all studies especially Japanese quail and medaka studies (as in mysid and LAGDA)

. Agreed

. Language in the test guidelines adjusted to help clarify.

Avoid data

- driven statistics (e.g.
, different datasets lead to different analyses)

. Agreed

. Statistical guidance improved to avoid data

- driven approach. Specific recommendations:

• Protocols should include a statistical guidance manual for each Tier 2 test including a discussion of randomization, power analyses (sufficient replicates), and blinded treatments. A statistical guidance manual is unnecessary as OECD ’s 2007 document, Current Approaches in the Statistical Analysis of Ecotoxicity Data: A Guidance to Application (No. 54)
, is referenced and more guidance added to the statistical analysis section of each test guideline.

• Recommended that the s election of probability distributions be appropriate to the data (e.g.
, Continuous ; discrete random; ordinal categorical variables) rather than transform data to be normal and homoscedastic. Failure to do so may result in error due to the tests appli ed by individual labs and conclusions drawn. Agreed

. Statistical analysis guidance adjusted accordingly.

• Statistical models used should be based on a priori knowledge of the types of endpoints measured (use ph

– typically normally distributed, versus H+ c oncentration

– usually long

- normally distributed as an example). Agreed

. Statistical analysis guidance adjusted accordingly.

• Recommend ed using Shapiro

- Wilks test for normality, but recognize the influence of sample size on power of the test (i.e. Small sa mples sizes are likely to fail to reject null), suggest using a visual review (e.g.
, Quantile

- quantile plot of residuals) in addition to test, and recommend review of normality should be on fitted model residuals not raw data. Agreed

. Statistical analysis g uidance adjusted accordingly.

• Recommended using Leven’s test of heterogeneous variances. When variances are unequal, the panel cautions against transforming data (as one might to achieve normality) as statistical packages can analyze data with unequal variance. Agreed. Revised guidance.

• Caution should be taken when using non

- parametric tests (e.g.
, Kruskal

- Wallis alternative to ANOVA) due to low power, potential Agreed. A caution added to the guidance.Page 2 of 4 12/1/14



### SAP comment Agency Position EPA Action failure to recognize assumptions of non

- parametric analysis (i.e.
, Equal variances and comparable data distributions among effect levels), and limitations testing interactions between effects that can be especially important in multigenerational tests. Using rank

- based non

- parametric analyses will be further complicated when a nalyzing ordinal data (e.g. Histopathology severity grades) with frequent tied ranks.

• ANOVA should not be performed on ranked transformed data as the Type I error increases. This is particularly true when the pre and post

- test scores are correlated, as is the case in repeated measures designs. Agreed. Statistical analysis guidance adjusted accordingly.

• Recommended using confidence intervals for the mean difference between control and each dose level to characteriz e the data rather than post

- hoc power analyses and cvs which are less informative. Agreed. The guidance adjusted to clarify reporting confidence intervals instead of CVs.

• Commented that the current experimental designs are under

powered for testing inte nded hypotheses. If sample sizes cannot be increased, alterations to the statistical methods and/or experimental design may be appropriate, based on the variability observed in data from earlier experiments. Agreed. Sample sizes increased in all tests to improve power.

• Commented that the Fisher’s LDS approach to comparing treatment levels for main effects (when F

- test rejects the H0) should be av oided, as should any test (e.g., Jonckheere

- Terpstra) that is based on observed results unless the test is desig ned for sequential review. Agreed. Fisher’s LDS not recommended.

Note: F or histopathology, panel strongly recommended use of quantitative analysis

. Acknowledged, but not accepted. A histopathology expert workshop was held to discuss quantitative analysis. Comments on Japanese Quail Two Generation Test (JQQTT)

• Select a "Standard Japanese quail strain" and take efforts to reduce genetic drift due to continued colonization

. Generally agreed. A specific “standard” strain is not available, but a defined size range is recommended.

• Reduce differences in humidity, incubation temperature, and egg storage temperature among labs Agreed

. Recommended ranges for these conditions are provided.

• Standard ize the time of day of sampling. Acknowledged, but not accepted. The sampling is logistically challenging and occurs throughout the day. Guidance to ensure consistent sampling across treatments is provided.

• Standardize li ght intensity and photoperiod

. Agreed

.

• Include blood corticosterone in response to c apture and handling stress to evaluate potential biases due to differential response
across groups

. Acknowledged, but not accepted. Research and development are needed to standardize and demonstrate the utility of these measures.Page 3 of 4 12/1/14



### SAP comment Agency Position EPA Action

• Provide more specific gui dance and standardization of tissue collection (e.g., Tissue section selected and histological analysis used)

. Agreed

. A histopathology guidance document was completed to provide more detailed guidance.

• Provide more detailed guidance and standardized meth ods of collecting behavioral data

. Agreed. Additional guidance provided.

• Increase in sample sizes for hormone analyses, tissue samples, and behavioral endpoints

. Agreed. Sample sizes increased slightly to improve power, but not make logistically problematic.

• Increasing the number of replicates to improve statistical powe

. r Agreed. The minimum number of pairs was increased to improve overall power.

• Maintaining the F2 gen eration

. Agreed. The F2 generation is retained.

• Continuing to analyze histopathology samples from all organs currently identified in the JQTT with "blind" treatments

. Acknowledged, but not accepted. The histopathology experts did not feel this was

necessa ry, but secondary review of histopath ology readings is recommended. Comments on Medaka Test (MMT/MRT)

• Describe randomization of fish and provide more det ailed experimental design (e.g.
, How to handle the tanks at each generation) to improve reproducibility

. Agreed. Additional guidance provided.

• MRT protocol is an improvement of MMT statistical power at the cost of reduced assessment of F0 and F2 generations

. Agreed. The medaka extended one

- generation reproduction test ( MEOGRT ) was developed to provide assessment of F 0 and an option for F2
, when indicated.

• Other comments covered in “general comments concerning statistical issues

. ” Acknowledged. Additional statistical guidance provided based on recommendations from the statistics expert worksh op. Comments on Larval Amphibian Growth and Development Assay (LAGDA)

• Suggested statistical methods should be updated and described for each endpoint

. Agreed. Additional statistical guidance provided based on recommendations from the statistics expert workshop

• Correct treatment of categorical or ordinal variables

. Agreed.

• Include appropriate random or repeated measures effects in models

. Acknowledged, but not accepted. The LAGDA endpoints are not conducive to repeated measure analysis as collected at a single point.

• Suggested examining interlab oratory data to identify problematic aspects of testing methods and endpoints

. Agreed. The interlab oratory data were examined to determine what aspects need additional guidance

• Other comments include in “general” section

. n/a Comments on Mysid Two Generation Test (MTTT)

• A threshold (minimum) number of replicates are needed to continue with F1 exposures

. Agreed. The recommended number of re plicates increased to 4, with performance criteria that 3 replicates be available in controls at end of treatment.

• The number of replicates may be increased by reducing the number of individuals/tank and increasing the number of tank s. Agreed. The recommended number of re plicates increased to 4, with performance criteria that 3 replicates be available inPage 4 of 4 12/1/14



### SAP comment Agency Position EPA Action controls at end of treatment.

• Examine effect of sample size on power using random generated data sets. Acknowledged, but not accepted. A power analysis was completed for the empirical data sets and replication already increased to practical limitations. SUMMARY OF SAP COMMENTS ON HISTOLOGY General comments on statistics relevant to all assays:

• The Panel strongly recommended use of a quantitative analysis as a means to enhance analytical power and consistency amon g laboratories. Acknowledged, but not accepted. The histopathology guidance for each test was improved and expanded, based on recommendations provided by the histopathology experts from the workshop

.

• Where possible, histopathology studies should be conducted using a blind approach coupled with morphometric analysis. Acknowledged, but not accepted. The histopathology guidance for each test was improved and expanded, based on recommendations provided by the histopathology experts.

• Use a systematic ser ies of sections for each tissue conducted using analysis of sections either in a systematic or random basis.

o In the systematic approach, every 3rd or 5th (or some specific number) section is reviewed.

o In the random approach, randomization is used to sele ct which sections are reviewed. Accepted. The histopathology guidance for each test was improved and expanded.

• Image analysis can be readily automated. Acknowledged, but not accepted. The implication is that image analysis should be recommended, but the histopathology experts dismissed this recommendation as not needed.

• For the Larval Amphibian Development and Growth Assay (LAGDA), the Panel recommended objective measures such as thyroid area, thyroid follicular cell height, and thyroid follicular cell number using computer

- assisted methods. Acknowledged, but not accepted. The implication is that image analysis should be recommended, but the histopathology experts dismissed this recommendation as not needed.

• Clear guidance on the selection of animals, tissue, and sections for histological and subsequent statistical analysis should be developed. Agreed. Additional guidance provided based on recommendations from the histopathology experts.

• Define a “representative” section

. Agreed. Additional guidance provided to clarify.

• Increase the number of replicates to at least n=8 (quail)

. Agreed. Revised guideline to increase the replicates. * It is important to note that several of the SAP’s criticisms and recommendations of tier 2 statistics were specific to the statistical analyses of interlab oratory data . The goal of the workshop was to develop consistent, rigorous, and appropriate guidan ce for statistical analyses of data for a single Tier 2 EDSP test submission (i.e.
, a single study from a specific lab).