United States Environmental Protection Agency Office of Environmental Information Washington, DC 20460 EPA/240/B-06/003 February 2006 

# Data Quality Assessment: Statistical Methods for Practitioners EPA QA/G-9 S

### EPA QA/G-9S February 2006 iii 

## FOREWORD

This document, Data Quality Assessment: Statistical Methods for Practitioners, provides general guidance to organizations on assess ing data quality criteria and performance specifications. The Environmental Protection Agency (EPA) has developed the Data Quality Assessment (DQA) Process for project managers and planners to determ ine whether the type, quantity, and quality of data need ed to support Agency decision making has been achieved. This guidance is the culmination of e xperiences in the design and statis tical analyses of environmental data in different Program Offices at the EPA. Many elements of prior gui dance, statistics, and scientific planning have been in corporated into this document.

This document is one of a series of quali ty management guidance documents that the EPA Quality Staff has prepared to assist us ers in implementing th e Agency-wide Quality System. Other related documents include:

EPA QA/G-4 Guidance on Systematic Planning Using the Data Quality Objectives Process

EPA QA/G-5S Guidance on Choosing a Sampling Design for Environmental Data Collection

EPA QA/G-9R Data Quality Asse ssment: A Reviewer’s Guide

This document provides guidan ce to EPA program managers and planning teams as well as to the general public as appropriate. It doe s not impose legally binding requirements and may not apply to a particular situ ation based on the circumstances. EPA retains the discretion to adopt approaches on a case-by-case basis that differ from this guidance where appropriate.

This guidance is one of the U.S Environmental Protection Agency Quality System Series documents. These documents describe the EPA policies and procedures for planning, implementing, and assessing the effectiveness of the Quality System. These documents are updated periodically to incorporate new topics and revision or refinements to existing procedures. Comments received on this, the 200 6 version, will be considered for inclusion in subsequent versions. Please send your comments to:

Quality Staff (2811R) U.S. Environmental Protection Agency 1200 Pennsylvania Avenue, NW Washington, DC 20460 Phone: (202) 564-6830 Fax: (202) 565-2441 E-mail: quality@epa.gov

Copies of the EPA Quality System documents ma y be downloaded from the Quality Staff Home Page: www.epa.gov.quality.

### EPA QA/G-9S February 2006 iv 

## PREFACE

Data Quality Assessment: Statistical Methods for Practitioners desc ribes the statistical methods used in Data Quality Assessment (DQA) in evaluating environmental data sets. DQA is the scientific and statistical ev aluation of environmental data to determine if they meet the planning objectives of the projec t, and thus are of the right ty pe, quality, and quantity to support their intended use. This guidance applie s DQA to environmental decision-making (e.g., compliance determinations) and also addre sses DQA in environmental estimation (e.g., monitoring programs).

This document is distinctly different from other guidance documents in that it is not intended to be read in a linear or continuous fashion. Instead, it is intended to be used as a "tool-box" of useful techniques in assessing the quality of data . The overall structure of the document will enable the analyst to investig ate many problems using a systematic methodology. Each statistical technique examined in the text is demonstrated separately in the form of a series of steps to be taken. The technique is then illustrated with a practic al example following the steps described.

This document is intended for all EPA and ex tramural organizations that have quality systems based on EPA policies and specifications, a nd that may periodically assess these quality systems (or have them assessed by EPA) for comp liance to the specificati ons. In addition, this guidance may be used by other organizations th at assess quality systems applied to specific environmental programs.

The guidance provided herein is non-mandatory and is intended to help personnel who have minimal experience with statistical te rminology to understand how a technique works and how it may be applied to a problem. An explan ation of DQA in plain English may be found in the companion guidance document, Data Quality Assessment: A Reviewer’s Guide (EPA QA/G-9R) (U.S. EPA, 2006b).

### EPA QA/G-9S February 2006 v 

## TABLE OF CONTENTS

Page INTRODUCTION................................................................................................................... ...... 1 STEP 1: REVIEW DQOs AND THE SAMPLING DESIGN.................................................. 5 1.1 OVERVIEW AND ACTIVITIES............................................................................6 STEP 2: CONDUCT A PREL IMINARY DATA REVIEW.................................................... 9 2.1 OVERVIEW AND ACTIVITIES..........................................................................12 2.2 STATISTICAL QUANTITIES .............................................................................12 2.2.1 Measures of Relative Standing ..................................................................12 2.2.2 Measures of Central Tendency ..................................................................13 2.2.3 Measures of Dispersion..............................................................................15 2.2.4 Measures of Association............................................................................16 2.2.4.1 Pearson’s Correlation Coefficient................................................16 2.2.4.2 Spearman’s Rank Correlation......................................................18 2.2.4.3 Serial Correlation.........................................................................18 2.3 GRAPHICAL REPRESENTATIONS...................................................................20 2.3.1 Histogram...................................................................................................20 2.3.2 Stem-and-Leaf Plot ....................................................................................21 2.3.3 Box-and-Whiskers Plot..............................................................................23 2.3.4 Quantile Plot and Ranked Data Plots.........................................................25 2.3.5 Quantile-Quantile Plots and Probability Plots ...........................................26 2.3.6 Plots for Two or More Variables ...............................................................27 2.3.6.1 Scatterplot ....................................................................................29 2.3.6.2 Extensions of the Scatterplot .......................................................30 2.3.6.3 Empirical Quantile-Quantile Plot ................................................31 2.3.7 Plots for Temporal Data.............................................................................32 2.3.7.1 Time Plot......................................................................................33 2.3.7.2 Lag Plot........................................................................................34 2.3.7.3 Plot of the Autocorrelation Function (Correlogram) ...................34 2.3.7.4 Multiple Observations in a Time Period......................................35 2.3.7.5 Four-Plot ......................................................................................36 2.3.8 Plots for Spatial Data .................................................................................37 2.3.8.1 Posting Plots ................................................................................38 2.3.8.2 Symbol Plots and Bubble Plots....................................................39 2.3.8.3 Other Spatial Graphical Representations.....................................39 2.4 PROBABILITY DISTRIBUTIONS......................................................................40 2.4.1 The Normal Distribution............................................................................40 2.4.2 The t -Distribution.......................................................................................41 2.4.3 The Lognormal Distribution ......................................................................41 2.4.4 Central Limit Theorem ..............................................................................41 STEP 3: SELECT THE STATISTICAL METHOD.............................................................. 43 3.1 OVERVIEW AND ACTIVITIES..........................................................................46 3.2 METHODS FOR A SINGLE POPULATION ......................................................46 3.2.1 Parametric Methods ...................................................................................48 3.2.1.1 The One-Sample t -test and Confidence Interval or Limit ...........48

### EPA QA/G-9S February 2006 vi 

## Page 3.2.1.2 The One-Sample Tolerance Interval or Limit..............................48 3.2.1.3 Stratified Random Sampling........................................................51 3.2.1.4 The Chen Test..............................................................................52 3.2.1.5 Land’s Method for Lognormally Distributed Data......................57 3.2.1.6 The One-Sample Proportion Test and Confidence Interval.........58 3.2.2 Nonparametric Methods.............................................................................60 3.2.2.1 The Sign Test ...............................................................................60 3.2.2.2 The Wilcoxon Signed Rank Test .................................................61 3.3 COMPARING TWO POPULATIONS .................................................................63 3.3.1 Parametric Methods ...................................................................................65 3.3.1.1 Independent Samples ...................................................................66 3.3.1.2 Paired Samples.............................................................................71 3.3.2 Nonparametric Methods.............................................................................75 3.3.2.1 Independent Samples ...................................................................75 3.3.2.2 Paired Samples.............................................................................81 3.4 COMPARING SEVERAL POPULATIONS SIMULTANEOUSLY...................86 3.4.1 Parametric Methods ...................................................................................88 3.4.1.1 Dunnett’s Test..............................................................................88 3.4.2 Nonparametric Methods.............................................................................89 3.4.2.1 The Fligner-Wolfe Test................................................................89 STEP 4: VERIFY THE ASSUMPTION S OF THE STATISTICAL METHOD ................ 93 4.1 OVERVIEW AND ACTIVITIES..........................................................................96 4.2 TESTS FOR DISTRIBU TIONAL ASSUMPTIONS............................................96 4.2.1 Graphical Methods.....................................................................................98 4.2.2 Normal Probability Plot Tests....................................................................98 4.2.3 Coefficient of Skewness/Coe fficient of Kurtosis Tests.............................98 4.2.4 Range Tests................................................................................................99 4.2.4.1 The Studentized Range Test ........................................................99 4.2.4.2 Geary’s Test...............................................................................100 4.2.5 Goodness-of-Fit Tests..............................................................................100 4.2.5.1 Chi-Square Test .........................................................................100 4.2.5.2 Tests Based on the Empirical Distribution Function .................102 4.2.6 Recommendations....................................................................................102 4.3 TESTS FOR TRENDS ........................................................................................102 4.3.1 Introduction..............................................................................................102 4.3.2 Regression-Based Methods for Es timating and Testing for Trends ........103 4.3.2.1 Estimating a Trend Using the Slope of the Regression Line.....103 4.3.2.2 Testing for Trends Us ing Regression Methods .........................104 4.3.3 General Trend Estimation Methods.........................................................104 4.3.3.1 Sen's Slope Estimator.................................................................104 4.3.3.2 Seasonal Kendall Slope Estimator.............................................104 4.3.4 Hypothesis Tests for Detecting Trends....................................................104 4.3.4.1 One Observation per Time Period for One Sampling Location 106

### EPA QA/G-9S February 2006 vii 

## Page 4.3.4.2 Multiple Observations per Time Period for One Sampling Location .....................................................................................109 4.3.4.3 Multiple Sampling Locations with Multiple Observations........110 4.3.4.4 One Observation for One Station with Multiple Seasons..........113 4.3.5 A Discussion on Tests for Trends............................................................113 4.3.6 Testing for Trends in Sequences of Data.................................................114 4.4 OUTLIERS ..........................................................................................................115 4.4.1 Background..............................................................................................115 4.4.2 Selection of a Statistical Test for Outliers ...............................................116 4.4.3 Extreme Value Test (Dixon's Test)..........................................................117 4.4.4 Discordance Test......................................................................................117 4.4.5 Rosner's Test ............................................................................................119 4.4.6 Walsh's Test .............................................................................................120 4.4.7 Multivariate Outliers................................................................................120 4.5 TESTS FOR DISPERSIONS...............................................................................122 4.5.1 Confidence Intervals for a Single Variance.............................................122 4.5.2 The F -Test for the Equality of Two Variances........................................123 4.5.3 Bartlett's Test for the Equali ty of Two or More Variances......................123 4.5.4 Levene's Test for the Equality of Two or More Variances......................123 4.6 TRANSFORMATIONS ......................................................................................126 4.6.1 Types of Data Transformations ...............................................................127 4.6.2 Reasons for Transforming Data...............................................................128 4.7 VALUES BELOW DETECTION LIMITS.........................................................130 4.7.1 Approximately less than 15% Non-detects - Substitution Methods ........131 4.7.1.1 Aitchison’s Method....................................................................131 4.7.2 Between Approximately 15% - 50% Non-detects...................................132 4.7.2.1 Cohen's Method .........................................................................132 4.7.2.2 Selecting Between Aitchison’s Method and Cohen’s Method ..132 4.7.3 Greater than Approximately 50% Non-detects - Test of Proportions......134 4.7.4 Greater than Approximately 90% Non-detects........................................135 4.7.5 Recommendations....................................................................................135 4.8 INDEPENDENCE ...............................................................................................136 STEP 5: DRAW CONCLUSIONS FROM THE DATA ...................................................... 139 5.1 OVERVIEW AND ACTIVITIES........................................................................141 5.2 PERFORM THE STATISTICAL METHOD......................................................141 5.3 DRAW STUDY CONCLUSIONS......................................................................141 5.3.1 Hypothesis Tests ......................................................................................141 5.3.2 Confidence Intervals or Limits ................................................................142 5.3.3 Tolerance Intervals or Limits...................................................................143 5.4 EVALUATE PERFORMANCE OF THE SAMPLING DESIGN......................143 5.5 INTERPRET AND COMMUNICATE THE RESULTS....................................144 APPENDIX A: STATISTICAL TABLES............................................................................. 145 APPENDIX B: REFERENCES.............................................................................................. 183 If applicable, a list of tables, figures, and boxe s is listed at the beginning of each chapter.

### EPA QA/G-9S February 2006 viii

### EPA QA/G-9S February 2006 1 

## INTRODUCTION

Data Quality Assessment (DQA) is the scie ntific and statistical evaluation of environmental data to determine if they meet the planning objectives of th e project, and thus are of the right type, quality, and quantity to s upport their intended use. DQA is built on a fundamental premise: data quality is meaningful only wh en it relates to the intended use of the data. This guidance describes the technical aspects of DQA in evaluating environmental data sets. A conceptual presentation of the DQA process is contained in Data Quality Assessment: A Reviewer’s Guide (EPA QA/G-9R) (U.S. EPA 2004).

By using DQA, a reviewer can an swer four important questions:

1. Can a decision (or estimate) be made with the desired level of certainty, given the quality of the data?

2. How well did the sampling design perform?

3. If the same sampling design strategy is used again for a similar study, would the data be expected to support the same intended use with the desired level of certainty?

4. Is it likely that sufficient samples were ta ken to enable the reviewer to see an effect if it was really present?

The first question addresses the reviewer’s imme diate needs. For example, if the data are being used for decision-making and provide eviden ce strongly in favor of one course of action over another, then the decision maker can proceed knowing that the decision will be supported by unambiguous data. However, if the data do not show sufficiently strong evidence to favor one alternative, then the data analysis alerts th e decision maker to this uncertainty. The decision maker now is in a position to make an informed choice about how to pr oceed (such as collect more or different data before making the deci sion, or proceed with the decision despite the relatively high, but tolerable, chance of drawing an erroneous conclusion).

The second question addresses how robust this sa mpling design is with respect to slightly changing conditions. If the design is very sensitive to potentially disturbing influences, then interpretation of the results may be difficult. By addressing the second question, the reviewer guards against the possibility of a spurious resu lt arising from a unique set of circumstances.

The third question addresses the problem of wh ether this could be considered a unique situation where the results of this DQA only apply only to this situation and cannot be extrapolated to other situ ations. It also addresses the suitab ility of using this data collection design again for future projects. For example, if it is intended to use a certain sampling design at a different location from where the design was first used, it shoul d be determined how well the design can be expected to perform given that th e outcomes and environmenta l conditions of this sampling event will be different from those of th e original event. As environmental conditions

### EPA QA/G-9S February 2006 2 

## will vary from one location or one time to anot her, the adequacy of the sampling design should be evaluated over a broad range of possible outcomes and conditions.

The final question addresses th e issue of whether sufficient resources were used in the study. For example, in an epidemiological investig ation, was it likely the e ffect of interest could be reliably observed given the limite d number of samples actually obtained.

The data life cycle comprises three steps: planning, implementation, and assessment. During the planning phase, the Data Quality Objec tives (DQO) Process (or any other systematic planning procedure) is used to define criteria for determining the number, location, and timing of samples to be collected in order to produce a re sult with the desired level of certainty. This, along with the sampling methods, analytical proce dures, and appropriate qua lity assurance (QA) and quality control (QC) procedures, is documen ted in the QA Project Plan. The implementation phase consists of collecting data following the QA Project Plan specifications. At the outset of the assessment phase, the data are validated and verified to ensure that the sampling and analysis protocols specified in the QA Project Plan we re followed, and that the measurement systems performed in accordance with the criteria specified in the QA Project Plan. Then, DQA completes the data life cycle through determin ing if the performan ce and acceptance criterion from the DQO planning objectives were achieve d and to draw conclusions from the data.

DQA involves five steps that begin with a re view of the planning documentation and end with an answer to the question posed during the planning phase of the study. These steps roughly parallel the actions of an environmental st atistician when analyzing a set of data. The five steps, which are described in more detail in the following chapters of this guidance, are:

1. Review the project objectives and sampling design. 2. Conduct a preliminary data review. 3. Select the statistical method. 4. Verify the assumptions of the statistical method

. 5. Draw conclusions from the data.

These five steps are presented in a linear sequence, but DQA is actually an iterative process. For example, if the preliminary data re view reveals patterns or anomalies in the data set that are inconsistent with th e project objectives, then some aspects of the study planning may have to be reconsidered in St ep 1. Likewise, if the underlying assumptions of the statistical method are not supported by the data, then previous steps of the DQA may have to be revisited.

This guidance is written for a broad audience of potential data users, data analysts, and data generators. Data users (such as project ma nagers or risk assessors who are responsible for making decisions or producing estimates regard ing environmental characteristics) should find this guidance useful for understanding and direc ting the technical work of others who produce and analyze data. Data analysts should find this guidance to be a convenient compendium of basic assessment tools. Data generators (such as analytical chemists or field sampling specialists responsible for collecting and analyzing environm ental samples and reporting the resulting data values) should find this guidance useful for unde rstanding how their work will be used and for

### EPA QA/G-9S February 2006 3 

## providing a foundation for improving the efficiency and effectiveness of the data generation
process.
This guidance presents background information and statistical tools for performing DQA; a non-statistical discussion of DQA is to be found in the companion document Data Quality Assessment: A Reviewer’s Guide (EPA QA/G-9R) (U.S. EPA 2004). Each chapter corresponds to a step in the DQA and begins with an overview of the activities to be performed for that step. Following the overviews in Chapters 1, 2, 3, and 4, specific graphical or statistical tools are described and step-by-step procedures are provi ded along with examples. Chapter 5 gives some advice on the interpretation of statistical tests. Appendix A contains statistical tables, and Appendix B provides references a nd useful publications for in-dep th statistical analyses.

### EPA QA/G-9S February 2006 4

### EPA QA/G-9S February 2006 5 

## CHAPTER 1

STEP 1: REVIEW DQOs AND THE SAMPLING DESIGN

Step 1: Review DQOs and Sampling Design

• Review the objectives of the study.

 If DQOs were developed, then review the outputs from the DQO Process.

 If DQOs have not been developed, then as certain what these objectives were.

• Translate the data user's objectives into a st atement of the primary statistical hypothesis.

 If DQOs were developed, translate them in to a statement of t he primary hypothesis.

 If DQOs have not been developed, then ascert ain what hypotheses or estimates were developed.

• Translate the data user's objectives into limits on Type I or Type II decision errors.

 If DQOs have not been developed, document t he data user's probable tolerable limits on decision errors, width of gray region, and estimated preliminary values.

 If DQOs were developed, confirm the limits on decision errors.

• Review the sampling design and note any special features or potential problems.

 Review the sampling design for any potentially serious deviations. 

## THE DATA QUALITY ASSESSMENT PROCESS

Review DQOs and Sampling Design

Conduct Preliminary Data Review

Select the Statistical Method

Verify the Assumptions

Draw Conclusions from the Data REVIEW DQOs AND SAMPLING DESIGN

Purpose
Review the DQO outputs, the sampling design, and any data collection documentation for consistency. If DQOs have not been developed, define the statistical method and specify tolerable limits on decision errors.

A ctivities

Review Study Objectives Translate Objectives into Statistical Hypothesis Develop Limits on Decision Errors Review Sampling Design

Tools

Statements of hypotheses Sampling design concepts

### EPA QA/G-9S February 2006 6 

## CHAPTER 1

STEP 1: REVIEW DQOs AND THE SAMPLING DESIGN

1.1 OVERVIEW AND ACTIVITIES
DQA begins by reviewing the key outputs from the planning phase of the data life cycle such as the Data Quality Objectives, the QA Project Plan, and any associated documents. The study objectives provide the context for understandi ng the purpose of the da ta collection effort and establish the qualitative and quantitative basis for assessing the quality of the data set for the intended use. The sampling design (documente d in the QA Project Pl an) provides important information about how to interpret the data. By studying the sampling design, the analyst can gain an understanding of the assumptions under which the design was developed, as well as the relationship between these assumptions and the objectives.

In the unfortunate in stances when project objectives have not been developed during the planning phase of the study, it is necessary to develop statements of the da ta user’s objectives prior to conducting the DQA. The primary purpose of stating the data user’s obj ectives prior to analyzing the data is to establish appropriate criteria for evaluating the quality of the data with respect to their intended use. Analysts who are not familiar with the DQO Process should refer to the Guidance for the Data Quality Objectives Process (EPA QA/G-4) (U.S. EPA 2000), a book on statistical planning and analysis, or a consulting stat istician. The seven steps of the DQO Process are illustrated in Figure 1-1.

If the project has been framed as a hypothesis test, then the uncertainty limits can be expressed as the data user's tolerance for committing false rejection (Type I or a false positive) or false acceptance (Type II or a false negative) decision errors. A false rejection error occurs when the null hypothesis is rejected when it is in fact true. A false acceptance error occurs when the null hypothesis is not rejected when it is in fact false. Other related phrases in common use include "level of significance" which is equal to the probability of a Type I error, and "power" which is equal to 1 minus probability of a Type II error. Statistical power really is a functio n that describes a “power curve” ove r a range of Type II errors. The characteristics of the power curve can be a great importance in choosing the appropriate statistical test. For detailed information on how to develop false rejec tion and false acceptance Step 1. State the Problem Define the problem that motivates the Identify the planning team; examine budget, schedule.

Step 2. Identify the Goal of the Study State how environmental data will be used in solving the problem; identify study questions; define alternative outcomes.

Step 3. Identify Information Inputs

Step 4. Define the Boundaries of the Study Specify the target population and characteristics of interest; define spatial and temporal limits, scale of inference.

Step 5. Develop the Analytic Approach Define the parameter of interest; specify the type of inference and develop logic for drawing conclusions from the findings.

Develop performance criteria for new data being collected,

Select the most resource

Statistical Estimation and other analytical approaches Step 1. State the Problem Define the problem that motivates the study; Identify the planning team; examine budget, s

Step 2. Identify the Goal of the Study State how environmental data will be used in solving the

Step 3. Identify Information Inputs
Identify data and information needed to answer study questions.

Step 4. Define the Boundaries of the Study Specify the target population and characteristics of interest; define spatial and temporal limits, scale of inference.

Step 5. Develop the Analytic Approach Define the parameter of interest and develop logic for drawing conclusions from the findings.

Step 6. Specify Performance or Acceptance Criteria
Develop performance criteria for new data being collected, acceptance criteria for data already collected.

Step 7. Develop the Detailed Plan for Obtaining Data Select the most resource effective sampling and analysis plan

- that satisfies the performance or acceptance criteria.

Statistical Hypothesis Testing Estimation and other Step 1. State the Problem Define the problem that motivates the Identify the planning team; examine budget, schedule.

Step 2. Identify the Goal of the Study State how environmental data will be used in solving the problem; identify study questions; define alternative outcomes.

Step 3. Identify Information Inputs

Step 4. Define the Boundaries of the Study Specify the target population and characteristics of interest; define spatial and temporal limits, scale of inference.

Step 5. Develop the Analytic Approach Define the parameter of interest; specify the type of inference and develop logic for drawing conclusions from the findings.

Develop performance criteria for new data being collected,

Select the most resource

Statistical Estimation and other analytical approaches Step 1. State the Problem Define the problem that motivates the study; Identify the planning team; examine budget, s

Step 2. Identify the Goal of the Study State how environmental data will be used in solving the

Step 3. Identify Information Inputs
Identify data and information needed to answer study questions.

Step 4. Define the Boundaries of the Study Specify the target population and characteristics of interest; define spatial and temporal limits, scale of inference.

Step 5. Develop the Analytic Approach Define the parameter of interest and develop logic for drawing conclusions from the findings.

Step 6. Specify Performance or Acceptance Criteria
Develop performance criteria for new data being collected, acceptance criteria for data already collected.

Step 7. Develop the Detailed Plan for Obtaining Data Select the most resource effective sampling and analysis plan

- that satisfies the performance or acceptance criteria.

Statistical Hypothesis Testing Estimation and other



## Figure 1-1. The DQO Process

### EPA QA/G-9S February 2006 7 

## decision error rates, see Chapter 6 of Guidance on Data Quality Objectives EPA QA/G-4 (U.S. EPA 2000).

If the project has been framed in terms of confidence intervals, then uncertainty is expressed as a combination of two interrelated terms: the width of the confidence interval (smaller intervals co rrespond to a smaller degree of uncertain ty) or the confidence level that the true value of the parameter of interest lies within the interval (a higher confidence level represents a smaller degree of uncertainty).

For the review of sampling design, recall that the key distinction in sampling design is between authoritative or judgmental sampling (in which sample numbers and locations are selected based on expert opini on) and probability sampling (in which sample numbers and locations are selected based on randomization a nd each member of the target population has a known probability of being included in the sample ). Judgmental sampling should be considered only when the objectives of the investigation are not of a statistical nature (for example, when the objective of a study is to iden tify specific locations of leaks, or when the study is focused solely on the sampling locations themselves). Generally, conclusions drawn from authoritative samples apply only to the individua l samples; aggregation may result in severe bias due to lack of representativeness and lead to highly erro neous conclusions. Judgmental sampling also precludes the use of the sample for any purpose other than the original one. If judgmental sample data are used, great care should be ta ken in interpreting any statistical statements concerning the conclusions to be drawn. Us ing some probabilistic statement with and judgmental sample is incorrect and should be av oided as it gives an illu sion of correctness when there is none. The further the judgmental samp le is from a truly random sample, the more questionable the conclusions. Guidance on Choosing a Sampling Design for Environmental Data Collection (EPA QA/G-5S) (U.S. EPA 2002) provi des extensive information on sampling design issues and their implicati ons for data interpretation.

The analyst should review the sampling de sign documentation with the data user's objectives in mind. Look for desi gn features that support or cont radict those objectives. For example, if the data user is interested in making a decision about the mean level of contamination in an effluent stream over time, then composite samples may be an appropriate sampling approach. On the other hand, if the data user is looking for hot spots of contamination at a hazardous waste site, compositing should be used with caution, to avoid "averaging away" hot spots. Also, look for potential problems in the implementation of the sampling design. For example, if simple random sampling has been used verify that each point in space (or time) had an equal probability of being selected for a simple random sampling design. Small deviations from a sampling plan may have minimal effect on the conclusions drawn from the data set. Significant or substantial devia tions should be flagged and thei r potential eff ect carefully considered throughout the entire DQA. The most im portant point is to veri fy that the collected data are consistent with how the QA Project Plan, Sampling and Analys is Plan, or overall objectives of the study stated them to be.

### EPA QA/G-9S February 2006 8

### EPA QA/G-9S February 2006 9 

## CHAPTER 2

STEP 2: CONDUCT A PR ELIMINARY DATA REVIEW

Step 2: Conduct a Preliminary Data Review

• Review quality assurance reports.

 Look for problems or anomalies in the implem entation of the sample collection and analysis procedures.

 Examine QC data for information to verify assumptions underlying the Data Quality Objectives, the Sampling and Analysis Plan, and the QA Project Plans.

• Calculate the statistical quantities.

 Consider calculating appropria te percentiles (Section 2.2.1).

 Select measures of central tendency (Sec tion 2.2.2) and dispersion (Section 2.2.3).

 If the data involve two variables, calculat e the correlation coefficient (Section 2.2.4).

• Display the data using graphical representations.

 Select graphical representations (Section 2.3) that illuminate the stru cture of the data set and highlight assumptions underlying the Data Q uality Objectives, the Sampling and Analysis Plan, and the QA Project Plans.

 Use a variety of graphical representations th at examine different features of the set. 

## THE DATA QUALITY ASSESSMENT PROCESS

Review DQOs and Sampling Design

Conduct Preliminary Data Review

Select the Statistical Method

Verify the Assumptions

Draw Conclusions from the Data CONDUCT PRELIMINARY DATA REVIEW

Purpose

Generate statistical quantities and graphical representations that describe the data. Use this information to learn about the structure of the data and identify any patterns or relationships.

A ctivities

Review Quality Assurance Reports Calculate Basic Statistical Quantities Graph the Data

Tools

Statistical quantities Graphical representations

### EPA QA/G-9S February 2006 10 

## List of Boxes

Page Box 2-1: Directions for Calculati ng Percentiles with an Example.............................................. 13 Box 2-2: Directions for Calculating the Measures of Central Tendency..................................... 14 Box 2-3: Example Calculations of the Measures of Central Tendency....................................... 14 Box 2-4: Directions for Calcula ting the Measures of Dispersion................................................ 15 Box 2-5: Example Calculations of the Measures of Dispersion.................................................. 15 Box 2-6: Directions for Calc ulating Pearson’s Correlation Coe fficient with an Example.......... 17 Box 2-7: Directions for Calc ulating Spearman’s Correlation Coe fficient with an Example ...... 19 Box 2-8: Directions for Estimating the Seri al Correlation Coefficient with a Example ............. 19 Box 2-9: Directions fo r Generating a Histogram......................................................................... 21 Box 2-10: Example of Generating a Histogram .......................................................................... 21 Box 2-11: Directions for Gene rating a Stem and Leaf Plot......................................................... 22 Box 2-12: Example of Genera ting a Stem and Leaf Plot............................................................. 23 Box 2-13: Directions for Generating a Box and Whiskers Plot................................................... 24 Box 2-14: Example of a Box and Whiskers Plot ......................................................................... 24 Box 2-15: Directions for Generating a Quantile Plot and a Ranked Data Plot............................ 26 Box 2-16: Example of Generating a Quantile Plot ...................................................................... 26 Box 2-17: Directions for Construc ting a Normal Probability Plot .............................................. 28 Box 2-18: Example of Normal Probability Plot .......................................................................... 28 Box 2-19: Directions for Generatin g a Scatterplot and an Example ........................................... 29 Box 2-20: Directions for Constructing an Empirical Q-Q Plot with an Example ....................... 32 Box 2-21: Directions for Generatin g a Time Plot and an Example............................................. 34 Box 2-22: Directions for Constructing a Correlogram ................................................................ 35 Box 2-23: Example Calculations for Generating a Correlogram................................................. 36 Box 2-24: Directions for Generating a Posting Plot and a Symbol Plot with an Example.......... 38

### EPA QA/G-9S February 2006 11 

## List of Figures

Page Figure 2-1. A Histogram of C oncentration Frequencies. ............................................................ 20 Figure 2-2. A Histogram of Concentration Densities.................................................................. 20 Figure 2-3. Example of a Box-Plot.............................................................................................. 23 Figure 2-4. Example of a Quantile Plot of Right-Skewed Data. ................................................. 25 Figure 2-5. Example of a Scatterplot........................................................................................... 29 Figure 2-6. Example of a Coded Scatterplot. .............................................................................. 30 Figure 2-7. Example of a Pa rallel-Coordinate Plot. .................................................................... 30 Figure 2-8. Example of a Scatterplot Matrix............................................................................... 31 Figure 2-9. Example of an Empirical Q-Q Plot........................................................................... 31 Figure 2-10. Example of a Time Plot. ......................................................................................... 33 Figure 2-11. Example of a Lag Plot............................................................................................. 34 Figure 2-12. Example of a Correlogram...................................................................................... 35 Figure 2-13. Example of a Four-Plot........................................................................................... 3 7 Figure 2-14. Example of a Posting Plot....................................................................................... 38 Figure 2-15. Example of a Symbol Plot ...................................................................................... 39 Figure 2-16. Example of a Bubble Plot ....................................................................................... 39 Figure 2-17. Two Normal Curves, Common Mean, Different Variances................................... 40 Figure 2-18. The Standard Normal Curve, Centered on Zero. .................................................... 40 Figure 2-19. Three Different Lognormal Distributions............................................................... 41

### EPA QA/G-9S February 2006 12 

## CHAPTER 2

STEP 2: CONDUCT A PREL IMINARY DATA REVIEW

2.1 OVERVIEW AND ACTIVITIES
In this step of DQA, the analyst conducts a preliminary evaluation of the data set, calculates some basic statistical quantities, and ex amines the data using gr aphical representations The first activity in conducting a preliminary data review is to review any relevant QA reports giving particular attentio n should be paid to information that can be used to check assumptions made in the DQO Process. Of great importance ar e apparent anomalies in recorded data, missing values, deviations from standard operating proc edures, and the use of nonstandard data collection methodologies.

Graphs can be used to identify patterns a nd trends, to quickly confirm or disprove hypotheses, to discover new phenomena, to identif y potential problems, and to suggest corrective measures. Since no single graphical representation will provide a complete picture of the data set, the analyst should choose different graphical techniques to illuminate different features of the data. Section 2.3 provides descriptions a nd examples of common gr aphical representations.

For a more extensive discussion of the ove rview and activities of this step, see Data Quality Assessment: A Reviewer’s Guide (EPA QA/G-9R) (U.S. EPA 2004).

2.2 STATISTICAL QUANTITIES

2.2.1 Measures of Relative Standing
Sometimes the analyst is interested in know ing the relative position of one or several observations in relation to all of the observati ons. Percentiles or qua ntiles are measures of relative standing that are useful for summarizing data. A percenti le is the data value that is greater than or equal to a given pe rcentage of the data values. Stated in mathematical terms, the
p th 

## percentile is a data value that is greater than or equal to p % of the data values and is less than or equal to (1- p )% of the data values. Therefore, if ' x ' is the p th 

## percentile, then p % of the values in the data set are less than or equal to x , and (100- p )% of the values are greater than x . A sample percentile may fall between a pair of observations. For example, the 75 th 

## percentile of a data set of 10 observations is not uniquely defi ned. Therefore, there are several methods for computing sample percentiles, the most co mmon of which is described in Box 2-1.

Important percentiles usually reviewed are the quartiles of the data, the 25 th 

## , 50 th 

## , and 75 th 

## percentiles. The 50 th 

## percentile is also called the sa mple median (Section 2.2.2), and the 25 th 

## and 75 th 

## percentile are used to estimat e the dispersion of a data set (Section 2.2.3). Also important for environmental data are the 90 th 

## , 95 th 

## , and 99 th 

## percentiles where a decision maker would like to be sure that 90%, 95%, or 99% of the cont amination levels are below a fixed risk level.

### EPA QA/G-9S February 2006 13 Box 2-1: Directions for Calculat ing Percentiles with an Example

Let X 1 , X 2 , ..., X n represent the n data points. To compute the p th percentile, y ( p ), first rank the data from smallest to largest and label these points X (1) , X (2) , . . ., X ( n ) . The p th percentile is: 

## ( ) ( ) () ( ) 1 1 + ⋅ + ⋅ − = i i X f X f p y

where r = ( n -1) p + 1, i = floor( r ), f = r – i , and 

### ( ) () n n X X = + 1 . Note that floor( r ) means calculate r and then
discard all decimals.

Example : The 90 th and 95 th percentile will be computed for the following 10 data points (ordered from smallest to largest) : 4, 4, 4, 5, 5, 6, 7, 7, 8, and 10 ppb.

For the 95th percentile, r = (10-1) × 0.95 + 1 = 9.55, i = floor(9.55) = 9, and f = 9.55 – 9 = 0.55. Therefore, the 95 th percentile is y (0.95) = 0.45 × 8 + 0.55 × 10 = 9.1.

For the 90 th percentile, r = (10-1) × 0.9 + 1 = 9.1, i = floor(9.1) = 9, and f = 9.1 – 9 = 0.1. Therefore, the 90 th percentile is y (0.9) = 0.9 × 8 + 0.1 × 10 = 8.2.



## 2.2.2 Measures of Central Tendency
Measures of central tendency characterize th e center of a data set. The three most common estimates are the mean, median, and th e mode. Directions for calculating these quantities are contained in Box 2-2; examples are provided in Box 2-3.

The most commonly used measure of the center of a data set is the sample mean, denoted by X . The sample mean can be thought of as the " center of gravity" of the data set. The sample mean is an arithmetic average for simple sampling designs; however, for complex sampling designs, such as stratification, the sample mean is a weighted arithmetic average. The sample mean is influenced by extreme values (large or small) and the treatment of non-detects (see Section 4.7).

The sample median, is the second most popular measure of the center of the data. This value falls directly in the middle of the ordered data set. This means that 1⁄2 of the data are smaller than the sample median and 1⁄2 of the data are larger than the sample median. The median is another name for the 50 th 

## percentile (Section 2.2.1). The median is not influenced by extreme values and can easily be used if non-detects are present.

Another method of measuring th e center of the data is the sample mode. The sample mode is the value that occurs with the greatest frequency. Since the sample mode may not exist or be unique, it is the least commonly used measure of center. However, the mode is useful for qualitative data.

### EPA QA/G-9S February 2006 14 Box 2-2: Directions for Calculating the Measures of Central Tendency

Let X 1 , X 2 , ..., X n represent the n data points.

Sample Mean : The sample mean, X 

## , is the sum of the data points divided by the sample size, n :

i n

i X n = X 

# ∑

= 1 1

Sample Median : The sample median, ~ X , is the center of the ordered data set. To compute the sample median, sort the data from smallest to largest and label these points X ( 1 ) , X ( 2 ) , . . ., X ( n ) . Then,



### () ( ) 

# [ ]

( ) ( ) ⎪ ⎪ ⎪ ⎩ ⎪ ⎪ ⎪ ⎨ ⎧ +

=

+ +

odd is if even is if 2 1

~

2 / 1 1 2 / 2 /

n X n X X

X

n n n

Sample Mode : The sample mode is the value in the sample that occurs with the greatest frequency. The sample mode may not exist or be unique. Count the number of times each value occurs. The sample mode is the value that occurs most frequently.

Box 2-3: Example Calculations of the Measures of Central Tendency
The following is an example of computing the sample mean, median, and mode for the 10 data points: 4, 4, 7, 7, 4, 10, 4, 3, 7, and 8.

Sample mean:



## 8

. 5 10 58
10 8 7 3 4 10 4 7 7 4 4

= = + + + + + + + + +

= X

Sample median: The ordered data are: 3, 4, 4, 4, 4, 7, 7, 7, 8, and 10. Since n = 10 is even, the sample median is:

median 

### () ( ) 

# [ ] () () 

# [ ] [ ]

. = + = X + X = X + X = / 

## 5 5 7 4 2 1
2 1
2 1

6 5 1 2 / 10 2 10 +

Sample mode: Computing the number of times each value occurs yields:

4 appears 4 times; 5 appears 0 times; 6 appears 1 time; 7 appears 3 times; 8 appears 1 time; and 10 appears 1 time.

As the value of 4 occurs most often, it is the sample mode of this data set.

### EPA QA/G-9S February 2006 15 

## 2.2.3 Measures of Dispersion


## Measures of central tendency are more m eaningful if accompanied by a measure of the spread of values about the center. Measures of dispersion in a data set include the range, variance, sample standard deviation, coefficien t of variation, and the interquartile range. Directions for computing these measures are give n in Box 2-4; examples are given in Box 2-5.

Box 2-4: Directions for Calcul ating the Measures of Dispersion

Let X 1 , X 2 , ..., X n represent the n data points.

Sample Range : The sample range, R, is the difference between the largest and smallest values of the data set, i.e., R = max( X i ) - min( X i ).

Sample Variance : To compute the sample variance, s 2 , compute: 1 1 2

1 1 2

2

− ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

−

= 

# ∑

∑

=

=
n X n X

s n

i i n

i i

Sample Standard Deviation : The sample standard deviation, s , is the square root of the sample variance,
i.e., 2 s s =

Coefficient of Variation : The coefficient of variation (CV) is the sample standard deviation divided by the sample mean (Section 2.2.2), i.e., CV = s X / . The CV is often expressed as a percentage.

Interquartile Range : The interquartile range (IQR) is the difference between the 75 th and the 25 th percentiles, i.e., IQR = y (75) - y (25).

Box 2-5: Example Calculations of the Measures of Dispersion
The directions in Box 2-4 and the following 10 data points (in ppm): 4, 5, 6, 7, 4, 10, 4, 5, 7, and 8, are used to calculate the measures of dispersion. From Box 2-3, X = 6 ppm .

Sample Range : R = max( X i ) - min( X i ) = 10 - 4 = 6 ppm

Sample Variance : 

# ( ) ( )
2 2 2 2 2 2
2 ppm 4 9 60 10 1 396

1 10 8 5 4 10 1 8 5 4

= ⋅ −

=

− + + +

− + + +

= L L s

Sample Standard Deviation : ppm 2 4 2

=

= = s s

Coefficient of Variation : % 33 33

. 0 ppm 6 / ppm 2 / = = = = X

s CV

Interquartile Range : Using the directions in Section 2.2.1 to compute the 25 th and 75 th percentiles of the data are: y (25) = X (3) = 4.25 ppm and y (75) = X (8) = 7 ppm. The interquartile range (IQR) is the difference between these values: IQR = y (75) - y (25) = 7 – 4.25 = 2.75 ppm



## The easiest measure of dispersion to compute is the sample range. For small samples, the range is easy to interpret and may adequately re present the dispersion of the data. For large

### EPA QA/G-9S February 2006 16 

## samples, the range is not very informative b ecause it only considers extreme values and is therefore greatly influenced by outliers.

Generally speaking, the sample variance measures the average squared distance of data points from the sample mean. A large sample vari ance implies the data are not clustered close to the mean. A small sample variance (relative to th e mean) implies most of the data are near the mean. The sample variance is affected by extrem e values and by a large number of non-detects. The sample standard deviation is the square root of the sample variance and has the same unit of measure as the data.

The coefficient of variation (CV) is a meas ure having no units that allows the comparison of dispersion across several sets of data. The CV (also known as the relative standard deviation) is often used in environmental applications because variability (when expressed as a standard deviation) is often pr oportional to the mean.

When extreme values are present, the interqua rtile range may be more representative of the dispersion of the data than the standard deviat ion. This statistical quantity is the difference of the 75 th 

## and 25 th 

## percentiles and therefore, is not influenced by extreme values.

2.2.4 Measures of Association



## Data sets often include measurements of several characteristics (variables) for each sampling point. There may be interest in underst anding the relationship or level of association between two or more of these variables. One of the most common measures of association is the correlation coefficient. The correlation coeffi cient measures the re lationship between two variables, such as a linear relationship between two sets of measurements. Note that the correlation coefficient does not imply cause a nd effect. The analyst may say the correlation between two variables is high a nd the relationship is strong, but may not say an increase or decrease in one variable cause s the other variable to increas e or decrease without further evidence and strong statistical controls.

2.2.4.1 Pearson’s Correlation Coefficient



## The Pearson correlation coefficient measures the strength of the linear relationship between two variables. A linear association imp lies that as one variable increases, the other increases or decreases linearly. Values of th e correlation coefficient close to +1 (positive correlation) imply that as one vari able increases, the other increases nearly linearly. On the other hand, a correlation coefficient close to –1 implies that as one variable increases, the other decreases nearly linearly. Values close to 0 imply little linear co rrelation between the variables. When data are truly independent, the correlation between data points is zero (note, however, that a correlation of 0 does not necessarily imply i ndependence). Directions and an example for calculating Pearson’s correlation coeffi cient are contained in Box 2-6.

### EPA QA/G-9S February 2006 17 Box 2-6: Directions for Calculating Pears on’s Correlation Coefficient with an Example

Let X 1 , X 2 , ..., X n represent one variable of the n data points and let Y 1 , Y 2 , ..., Y n represent a second variable of the n data points. The Pearson correlation coefficient, r , between X and Y is:

⎥ ⎥ ⎥

⎦ ⎤

⎢ ⎢ ⎢

⎣ ⎡ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

− ⋅ ⎥ ⎥ ⎥

⎦ ⎤

⎢ ⎢ ⎢

⎣ ⎡ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

− ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

−

= 

# ∑ ∑ ∑ ∑ ∑ ∑ ∑

= = = = = = =
2

1 1 2 2

1 1 2 1 1 1

1 1 1

n

i i n

i i n

i i n

i i n

i i n

i i n

i i i

Y n Y X n X Y X n Y X

r

Example : Consider the following data set (in ppb):

Sample No. 1 2 3 4 Arsenic 8.0 6.0 2.0 1.0 Lead 8.0 7.0 7.0 6.0

126 6 1 7 2 7 6 8 8

, 198

, 105

, 28

, 17 4

1 4

1 2 4

1 2 4

1 4

1 = ⋅ + ⋅ + ⋅ + ⋅ = = = = = 

# ∑ ∑ ∑ ∑ ∑

= = = = = i i i i i i i i i i i Y X Y X Y X

and 865 0

4 28 28 198 4 17 17 105 4 28 17 126

. =

- - - r = ⎟ ⎠ ⎞ ⎜ ⎝ ⎛

⋅ ⎟ ⎠ ⎞ ⎜ ⎝ ⎛

⋅ ⋅

Since r is close to 1, there is a strong linear relationship between these two contaminants.



## The correlation coefficient does not detect nonl inear relationships so it should be used only in conjunction with a scatter pl ot (Section 2.3.7.2). A scatter plot can be used to determine if the correlation coefficient is meaningful or if some measure of nonlinear relationships should be used. The correlation coefficient can be si gnificantly influenced by extreme values so a scatter plot should be used first to identify such values.

An important property of the correlation coeffi cient is that it is unaffected by changes in location of the data (adding or subt racting a constant from all of the X measurements or all the Y measurements) or by changes in scale of the data (multiplying or dividing all X or all Y values by a positive constant). Thus, linear transformations on the X s and Y s do not affect the correlation of the measurements. This is reasonable si nce the correlation reflects the degree to which linearity between X and Y measurements occur and the degr ee of linearity is unaffected by changes in location or scale. For example, if a variable was temperature in Celsius, then the correlation would not change if Cels ius was converted to Fahrenheit.

On the other hand, if nonlinear transformations of the X and/or Y measurements are made, then the Pearson correlation between the transf ormed values will differ from the correlation of the original measuremen ts. For example, if X and Y , respectively, represent PCB and dioxin concentrations in soil, and x = log( X

) and y = log( Y ), then the Pearson correlations between X and

### EPA QA/G-9S February 2006 18 

## Y

, X and y , x and Y , and x and y , will all be different, in general, since the logarithmic transformation is a nonlinear transformation.

Pearson’s correlation may be se nsitive to the presence of one or two extreme values, especially when sample sizes are small. Such values may result in a high correlation, suggesting a strong linear trend, when only moderate trend is present. This may happen, for instance, if a single ( X

, Y

) pair has very high values for both X and Y while the remaining data values are uncorrelated. Extreme values may also lead to low correlations between X and Y , thus tending to mask a strong linear trend. This may happen if all the ( X

, Y

) pairs except one (or two) tend to cluster tightly about a stra ight line, and the exceptiona l point has a very large X value paired with a moderate or small Y value (or vice versa). As influences of extreme values can be important, it is wise to use a scatter plot (Section 2.3.7.2) in conjunction wi th a correlation coefficient.

2.2.4.2 Spearman’s Rank Correlation


## An alternative to the Pearson correlation is Sp earman’s rank correlation coefficient. It is calculated by first replacing each X value by its rank (i.e., 1 for the smallest X value, 2 for the second smallest X value, etc.) and each Y value by its rank. These pairs of ranks are then treated as the ( X

, Y

) data and Spearman’s rank correlation is calculated using the same formulae as for Pearson’s correlation (Box 2-6). Directions a nd an example for calculating the Spearman’s Rank correlation coefficient are contained in Box 2-7.

Since meaningful (i.e., monotoni c increasing) transformations of the data will not alter the ranks of the respective vari ables (e.g., the ranks for log ( X

) will be the same for the ranks for X

), Spearman’s correlation will not be altered by nonlinear increasing transformations of the X s or the Y s. For instance, the Spearman correlation between PCB and dioxin concentrations ( X

and Y

) in soil will be the same as the co rrelations between their logarithms ( x and y ). This desirable property, and the fact that Spearman’s correlation is less sensitive to extreme values, makes Spearman’s correlation a good alternative or compleme nt to Pearson’s correlation coefficient.

2.2.4.3 Serial Correlation


## For a sequence of data points taken serially in time, or one-by-one in a row, the serial correlation coefficient can be calculated by repl acing the sequencing variable by the numbers 1 through n and calculating Pearson’s correlation coefficient with x being the actual data values, and y being the numbers 1 through n . For example, for a sequence of data collected at a waste site along a straight transit line, the distances on the transit line of the data points are replaced by the numbers 1 through n , e.g., first 10-foot sample point = 1, the 20-foot sample point = 2, the 30-foot sample point = 3, etc., for samples taken at 10-foot intervals. Directions for the Serial correlation coefficient, along with an example, are given in Box 2-8.

### EPA QA/G-9S February 2006 19 Box 2-7: Directions for Calculating Spearma n’s Correlation Coefficient with an Example

Let X 1 , X 2 , ..., X n represent a set of ranks of the n data points of one data set and let Y 1 , Y 2 , ..., Y n represent a set of ranks of a second variable of the n data points. The Spearman Rank correlation coefficient, r , between X and Y is computed by:

⎥ ⎥ ⎥

⎦ ⎤

⎢ ⎢ ⎢

⎣ ⎡ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

− ⋅ ⎥ ⎥ ⎥

⎦ ⎤

⎢ ⎢ ⎢

⎣ ⎡ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

− ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

−

= 

# ∑ ∑ ∑ ∑ ∑ ∑ ∑

= = = = = = =
2

1 1 2 2

1 1 2 1 1 1

1 1 1

n

i i n

i i n

i i n

i i n

i i n

i i n

i i i

Y n Y X n X Y X n Y X

r

Example : Concentrations of arsenic and lead are taken at 4 sample locations. The following table gives the data set (in ppb) ranked according to the arsenic values. Ranks are given in parentheses. Note that any tied values are assigned average rank.

Sample No. 4 3 2 1 Arsenic 1.0 (1) 2.0 (2) 6.0 (3) 8.0 (4) Lead 6.0 (1) 7.0 (2.5) 7.0 (2.5) 8.0 (4)

Now, 5

. 29 4 4 5

. 2 3 5

. 2 2 1 1

, 5

. 29

, 30

, 10

, 10 4

1 4

1 2 4

1 2 4

1 4

1 = ⋅ + ⋅ + ⋅ + ⋅ = = = = = 

# ∑ ∑ ∑ ∑ ∑

= = = = = i i i i i i i i i i i Y X Y X Y X

and 9487

. 0

4 10 10 5

. 29 4 10 10 30 4 10 10 5

. 29

=

- - - r = ⎟ ⎠ ⎞ ⎜ ⎝ ⎛

⋅ ⎟ ⎠ ⎞ ⎜ ⎝ ⎛

⋅ ⋅

Since r is close to 1, there is a strong linear relati onship between these two contaminants when ranked.

Box 2-8: Directions for Estimating the Se rial Correlation Coefficient with a Example

Directions: Let X 1 , X 2 , . . . , X n represent the data values collected in sequence over equally spaced periods of time. Label the periods of time 1, 2, ..., n to match the data values. Use the directions in Box 2-6 to calculate the Pearson’s Correlation Coefficient between the data X and the time periods Y

. Example: The following are hourly readings from a discharge monitor.

Time 12:00 13:00 14:00 15:00 16:00 17:00 18:0 0 19:00 20:00 21:00 22:00 23:00 24:00

Reading 6.5 6.6 6.7 6.4 6.3 6.4 6.2 6.2 6.3 6.6 6.8 6.9 7.0

Time Periods 1 2 3 4 5 6 7 8 9 10 11 12 13

Using Box 2-6, with the readings being the X values and the Time Periods being the Y values gives a serial correlation coefficient of 0.4318.

### EPA QA/G-9S February 2006 20 

## 2.3 GRAPHICAL REPRESENTATIONS

2.3.1 Histogram


## A histogram is a visual representation of the data collected into groups. This graphical technique provides a visual method of identifying the underlying distribution of the data. The data range is divided into severa l bins or classes and the data is sorted into the bins. A histogram is a bar graph conveying the bins and the frequency of data points in each bin. Othe r forms of the histogram use a normalization of the bin frequencies for the heights of the bars. The two most common normalizations are relative frequencies (frequencies divided by sample size) and densities (relative frequency divi ded by the bin width). Figure 2-1 is an exampl e of a histogram using frequencies and Figure 2- 2 is a histogram of densities.

Histograms provide a visual method of accessing location, shape and spread of the data. Also, extreme values and multiple modes can be identified. The details of the data are lost, but anoverall picture of the data is obtained. The stemand-leaf plot described in the next section offers the same insights into the data as a histogram, but the data values are retained. Therefore, stem-and-leaf plots can be more informative than histograms for smaller data sets.

The visual impression of a histogram is sensitive to the choice of bins. A large number of bins will increase data detail while fewer bins will increase the smoothness of the histogram. A good starting point when choosing the nu mber of bins is the square-root of the sample size. Another factor in choosing bins is the choice of endpoint s. Using simple bin endpoints can improve the readability of the histogram. Simple bin endpoints include multiples of 5k units for some integer k . For example, 0 to <5, 5 to <10, etc. (Figure 2- 1), or 1 to <1.5, 1.5 to <2, etc. Finally, when plotting a histogram for a continuou s variable, e.g., concentration, it is necessary to decide on an endpoint convention; that is, what to do with data points that fall on the boundary of a bin. With discrete variables, (e.g., family size) the interval s can be centered in between the variables. For the family size data, the intervals can span between 1.5 and 2.5, 2.5 and 3.5, and so on. Then the whole numbers that relate to the family size can be centered within the box. Directions for generating a histogram are contained in Box 2- 9 and an example is contained in Box 2-10. 0 5 10 15 20 25 30 35 4 0 0 2 4 6 8 10

Concentration (ppm)Number of Observations



## Figure 2-1. A Histogram of Concentration Fre q uencies

0 5 10 15 20 25 30 35 40 0 2 4 6 8

Concentration (ppm)Percentage of Observations (per ppm)



## Figure 2-2. A Histogram of Concentration Densities

### EPA QA/G-9S February 2006 21 Box 2-9: Directions for Generating a Histogram
STEP 1: Partition the data range into several bins. Partitioning means the bins should be connected, but not overlapping. For example, 0 to <5, 5 to <10, etc. In almost all cases, the bin widths should be the same. When selecting bins, first choose a number of bins. A good rule of thumb is the square root of the sample size. Next, select bin endpoints that are simple (for example, multiples of 5k for some integer k ) and will produce approximately the number of desired bins.

STEP 2: Place each data point into the proper bin. This creates a summary of the data called a frequency table. If desired, compute one or both types of normalizations of the frequencies. The relative frequency is the frequency divided by the sample size. The density is the relative frequency divided by the bin width.

STEP 3: Determine the horizontal axis based on the range of the frequencies or normalized frequencies.

STEP 4: A histogram is a bar graph of the frequencie s or normalized frequencies. For each bin, draw a bar using the bin endpoints on the x-axis as the width and the frequency or normalized frequency on the y-axis as the height.

Box 2-10: Example of Generating a Histogram
Consider the following 22 measurements of a contamin ant concentration (in ppm): 17.7, 17.4, 22.8, 35.5, 28.6, 17.2, 19.1, <4, 7.2, <4, 15.2, 14.7, 14.9, 10.9, 12.4, 12.4, 11.6, 14.7, 10.2, 5.2, 16.5, and 8.9.

STEP 1: With 22 data points, a rough estimate of the number of bins is 69

. 4 22 = or 5. Since the data ranges from 0 to 40, the suggested bins are 0 to <8, 8 to <16, etc. This is a little awkward and using multiples of 5 to create simpler bins leads to 0 to <5, 5 to <10, etc. This choice leads to 8 bins which is close to suggested number and are the bins that will be used.

STEP 2: Column 2 of the frequency table below shows the frequency or number of observations within each bin defined in Step 1. The third column shows the relative frequencies which is the frequency divided by the sample size. The final column of the table gives the densities or the relative frequencies divided by the bin widths.

STEP 3: The horizontal axis for the data is from 0 to 40 ppm. The vertical axis for the histogram of frequencies is from 0 - 10 and the vertical axis for the histogram of densities is from 0% - 10%.

STEP 4: The histogram of frequencies is shown in Figure 2-1 and the histogram of densities is shown in Figure 2-2.



## 2.3.2 Stem-and-Leaf Plot


## The stem-and-leaf plot is used to show bot h the data values and visual information about the distribution of the data. Like a histogram, a stem-and-leaf plot is visual representation Relative Bin Frequency Frequency Density 0 - 5 ppm 2 9.09 1.8 5 - 10 ppm 3 13.64 2.7 10 - 15 ppm 8 36.36 7.3 15 - 20 ppm 6 27.27 5.5 20 - 25 ppm 1 4.55 0.9 25 - 30 ppm 1 4.55 0.9 30 - 35 ppm 0 0.00 0.0 35 - 40 ppm 1 4.55 0.9

### EPA QA/G-9S February 2006 22 

## of the data collected into groups. However, data detail is retained in a stem-and-leaf. As with histograms, stem-and-leaf plots provide an understanding of the shape of the data (and the underlying distribution), i. e., location, shape, spread, numbe r of modes and identification of potential outliers. Stem-a nd-leaf is also sensitive to bin selection. A stem-and-leaf plot can be more useful than a histogram in analyzing sm all data sets because it not only allows a visualization of the data dist ribution, but enables the data to be reconstructed.

First, bins that divide the data range are se lected in a similar manner as for a histogram; these are the stems. Since the number of data poi nts is typically small th e number of bins or stems should be approximately 4 to 8. Data point s are then sorted into the proper stem. Each observation in the stem-and-leaf plot consists of tw o parts: the stem and the leaf. The stem is generally made up of the leading dig it or digits of the data values while the leaf is made up of the trailing digit or digits. The stem s are displayed on the vertical axis and the trailing digits of the data points make up the leaves. Changing the stem can be accomplished by increasing or decreasing the leading digits that are used, divi ding the groupings of one stem (i.e., all numbers which start with the numeral 6 can be divided in to smaller groupings), or multiplying the data by a constant factor (i.e., multiply the data by 10 or 100). Non-detects can be placed at the detection limit with the leaf indicating the observation was actually a nondetect. Directions for constructing a stem-and-leaf pl ot are given in Box 2-11 and an example is contained in Box 2-12.

Box 2-11: Directions for Generating a Stem and Leaf Plot

Let X 1 , X 2 , ..., X n represent the n data points. To develop a stem-and-leaf plot, complete the following steps:

STEP 1: Arrange the observations in ascending order. The ordered data is labeled (from smallest to largest) X

(1) , X (2) , ..., X ( n )

. STEP 2: Choose either one or more of the leading digits to be the stem values. For example, if data ranges from 0 to 30, then the best choice for stems would be the values of the tens column, 0, 1, 2, 3.

STEP 3: List the stem values from smallest to largest along the vertical axis. Enter the trailing digits for each data point as the leaf. The leaves extend to the right of the stems. Continuing the example from above, the values of the ones column (and possible one decimal place) would be used as the leaf.

### EPA QA/G-9S February 2006 23 Box 2-12: Example of Generating a Stem and Leaf Plot
Consider the following 22 samples of trifluorine (in ppm): 17.7, 17.4, 22.8, 35.5, 28.6, 17.2, 19.1, <4, 7.2, <4, 15.2, 14.7, 14.9, 10.9, 12.4, 12.4, 11.6, 14.7, 10.2, 5.2, 16.5, and 8.9.

STEP 1: Arrange the observations in ascending order: <4, <4, 5.2, 7.7, 8.9, 10.2, 10.9, 11.6, 12.4, 12.4, 14.7, 14.7, 14.9, 15.2, 16.5, 17.2, 17.4, 17.7, 19.1, 22.8, 28.6, 35.5.

STEP 2: The data ranges from 0 to 40 so the best choice for the stems is the tens column. Initially, this gives 4 stems. We can divide the stems in half to give stems of 0 to <5, 5 to <10, etc. This gives 8 stems.

STEP 3: List the stem values along the vertical axis from smallest to largest. Enter the leaf (the remaining digits) values in order from lowest to highest to the right of the stem.

0 | <4 <4 0 | 5.2 7.7 8.9 1 | 0.2 0.9 1.6 2.4 2.4 4.7 4.7 4.9 1 | 5.2 6.5 7.2 7.4 7.7 9.1 2 | 2.8 2 | 8.6 3 | 3 | 5.5

Inspection of the stem-and-leaf indicates the data values are centered in the low teens with a slightly rightskewed distribution. Summary statistics can be calculated to strengthen these visual conclusions.



## 2.3.3 Box-and-Whiskers Plot


## Box and Whisker plots, also know n as box-plots, are useful in situations where a picture of the dist ribution is desired, but it is not necessary or feasible to portray all the details of the data. A box-plot (see Figure 2-3) displays several percentiles of the data set. It is a simple plot, yet provides insight in to the location, shape, and spread of the data and underlying distribut ion. A simple box-plot contains only the 0th (minimum data value), 25th, 50th, 75th and 100th (maximum data value) percentiles. A more complex version includes identification of the mean with a plus-sign and potential outliers(identified by stars). Since the box-plot is compact (essentially onedimensional), several can be placed on to a single graph. This allows a simple method to compare the lo cations, spreads and shapes of several data sets or diffe rent groups within a single data set. In this situation, the width of the box can be proportiona l to the sample size of each data set.

A box-plot divides the data into 4 sections, each containing 25% of the data. The length of the central box indicates the sp read of the data (the centra l 50%) while the length of the whiskers shows the breadth of th e tails of the distribution. Th e box-plot also demonstrates the shape of the data in the following manner. If the upper box and whisker are approximately the same length as the lower box and whisker, then th e data are distributed symmetrically. If the upper box and whisker are longer than the lower box and whisker, then the data are rightFigure 2-3. Example of a Box-Plot

### EPA QA/G-9S February 2006 24 

## skewed. If the upper box and whis ker are shorter than the lower box and whisker, then the data are left-skewed. Directions for generating a bo x and whiskers plot are contained in Box 2-13, and an example is contained in Box 2-14.

If the mean is added to the box-plot, then r ecall comparing the mean and median provides another method of identifying the shape of the data . If the mean is approximately equal to the median, then the data are distributed symmetrically. If the mean is greater than the median, then the data are right-skewed; if the mean is less than the median, then the data are left-skewed.

Box 2-13: Directions for Gene rating a Box and Whiskers Plot
STEP 1: Compute the 0 th (minimum value), 25th, 50 th (median), 75th and 100 th (maximum value) percentiles.

STEP 2: Plot these points either vertically or horizontally. If more than one box-plot is drawn, then the other axis can be used to identify the box-plots of the different groups or data sets. Draw a box around the 25th and 75th percentiles and add a line through the box at the 50th percentile. Optionally, make the width of the box proportional to the sample size.

STEP 3: If desired, compute the mean and indicate this value on the box-plot with a plus-sign. Also, identify potential outliers if desired. A potential outlier is a value at a distance greater than 1.5 × IQR from the closest end of the box.

STEP 4: Draw the whiskers from each end of the box to the furthest data point that has not been identified as an outlier. Plot the potentia l outliers using asterisks.

Box 2-14: Example of a Box and Whiskers Plot
Consider the following 22 samples of trifluorine (in ppm) listed in order from smallest to largest: 4.0, 6.1, 9.8, 10.7, 10.8, 11.5, 11.6, 12.4, 12.4, 14.6, 14.7, 14.7, 16.5, 17.0, 17.5, 20.6, 20.8, 25.7, 25.9, 26.5, 32.0, and 35.5.

STEP 1: The minimum value is 4.0, y (25) = 11.5, the median is 14.7, y (75) = 20.8, and the maximum value is 35.5.

STEP 2: The simple box-plot is drawn using the summary statistics calculated in step 1

STEP 3: The mean of the data is 16.9. The only potential outlier is 35.5 because 35.5 – 20.8 = 14.7 which is greater than 1.5 × IQR = 13.95.

STEP 4: The lower whisker extends from 11.5 to 4.0 and the upper whisker extends from 20.8 to 32.0

Examining the box-plot, it is easy to see that the data are centered at 15 ppm and 50% of the data lie between approximately 12 ppm and 21 ppm. Also, since the upper box and whisker are longer than the lower box and whisker, the distribution of the data is right-skewed.

### EPA QA/G-9S February 2006 25 

## 2.3.4 Quantile Plot and Ranked Data Plots


## The quantile plot and the ranke d data plot are two further methods of visualizing the location, spread and shape of the data. Both plot s are displays of the ranked data and differ only in the horizontal axis labels. No subjective decisi ons such as bin size are necessary and all of the data is plotted rather than summaries.

A ranked data plot is a display of the data from smallest to largest at evenly spaced intervals. A quantile plot is a graph of the ranked data versus the fr action of data points it exceeds. Therefore, a quantile plot can be used to read the quantile information such as the median, quartiles, and the inter quartile range. This additional information can aid in the interpretation of the shape of the data.

Using either the quantile plot or the ranked data plot, the spread of the data may be visualized by examining the slope of the graph. The closer the general slope is to 0, the lower the variability of the data set.

Also using either plot, the sh ape of the data maybe determined by inspecting the tails of the graph. If the left and right tails have approximately the same curvature, then the data are distributed symmetrically. If the curvature of the right-tail is greater than the curvature of the left-tail, then the data are right-ske wed. This is the case of the data plotted in Figure 2-4. If the curvature of the left-tail isgreater than the curvature of the righttail, then the data are left-skewed. Finally, the degree of curvature in the tails of either plot is proportional to the length of the tail of the data. In Figure 2-4, the plot rises slowly to a point, then the slope increases dramatically. This implies there is a large concentration of small data values and relatively few large data values. In this case, the left-tail of the data is very short and the right-tail is relatively long.

Directions for developing a qua ntile plot are given in Box 2-15 and an example is given in Box 2-16. 0 0.2 0.4 0.6 0.8 1 0 1 2 3 4

Fraction of Data (f-values)Data Values Upper Quartile Lower Quartile Interquartile Range

Median



## Figure 2-4. Example of a Quantile Plot of RightSkewed Data

### EPA QA/G-9S February 2006 26 Box 2-15: Directions for Generating a Qu antile Plot and a Ranked Data Plot

Let X 1 , X 2 , ..., X n represent the n data points. and X (1) , X (2) , ..., X ( n ) be the ordered data from smallest to largest . Compute the fractions f i = ( i - 0.5)/ n for i = 1,

... , n . The quantile plot is a plot of the pairs ( f i , X ( i

) ), with straight lines connecting consecutive points. If desired, add vertical lines indication the quartiles, median, or other quantiles of interest.

Alternatively, a ranked data plot can be made by simply plotting the ordered X values at equally spaced intervals along the horizontal axis.

Box 2-16: Example of Generating a Quantile Plot
Consider the following 10 data points (in ppm): 4, 5, 6, 7, 4, 10, 4, 5, 7, and 8. The data ordered from smallest to largest, X ( i ) , are shown in the second row of the table below and the third row displays the values f i for each i where f i = ( i - 0.5)/ n

.

i 1 2 3 4 5 6 7 8 9 10
X

( i

) 4 4 4 5 5 6 7 7 8 10
f i 0.05 0.15 0.25 0.35 0.45 0.55 0.65 0.75 0.85 0.95

The pairs ( f i , X ( i

) ) are then plotted to yield the following quantile plot:

0 0.2 0.4 0.6 0.8 1 4 6 8 10

Fraction of Data (f-values)Data (ppm)

Note that the graph curves upward. This indicates that the data are skewed to the right.



## 2.3.5 Quantile-Quantile Plots and Probability Plots


## There are two types of quantile-quantile plots or q-q plots. One is an empirical q-q plot which involves plotting the quantiles of two data sets against each ot her. This is a technique to determine if the data sets were generated by th e same underlying distributi on and is discussed in Section 2.3.6.3. The other type of q-q plot i nvolves graphing the quantile s of a data set against the quantiles of a specific probability distribution. This is a technique to determine if the data set was generated by the theoretical distribution. The following s ection will focus on the most common of these plots for environmental data, th e normal probability plot (normal q-q plot). However, the discussion holds for other probability distributions. The normal probability plot is

### EPA QA/G-9S February 2006 27 

## a visual method to roughly determ ine how well the data set is m odeled by a normal distribution. Formal tests are contained in Chapter 4, S ection 2. Directions for developing a normal probability plot are given in Box 2-17 and an exam ple is given in Box 2-18. A discussion of the normal distribution is cont ained in Section 2.4.

A normal probability plot is the graph of the quantiles of a data set against the quantiles of the standard normal distribution. This can be accomplished by using a software package, plotting the sample quantiles against standard no rmal quantiles, or plotting the sample quantiles on normal probability paper. If the graph is approximately linear (the correlation coefficient being fairly high excluding outli ers), then this is an indica tion that the data are normally distributed and a formal test should be performed. If the graph is not line ar, then the departures from linearity give important information about ho w the data distribution deviates from a normal distribution.

A nonlinear normal probability plot may be used to interpret the shape and tail behavior of the data. First, add the quart ile line, the line through the first and third quartiles, to the plot. Next, examine the relationship of the tails of th e normal probability plot to the quartile line.

Relationship of the q-q plot to the quartile line Distribution of the data in relation to the normal distribution upper-tail lower-tail shape tail behavior 

## above below symmetric heavy below above symmetric light above above right-skewed - below below left-skewed -

A normal probability plot can also be used to identify potential outlier s. A data value (or a few data values) much larger or much smaller will appear distant from the other values, which will be concentrated in the center of the graph.

As a final note, using a simple natural log transformation of the data, the normal probability plot can be used to determine if the data are well modeled by a lognormal distribution. The lognormal is an important probability distribution when analyzing environmental data where normality cannot be assumed.

2.3.6 Plots for Two or More Variables



## Data often consist of measurements of severa l characteristics (variables) for each sample point in the data set. For example, a data set may consist of measurements of lead, mercury, and chromium for each soil sample or may consist of daily concentration readings for several monitoring sites. In this case, gr aphs may be used to compare and contrast different variables.

### EPA QA/G-9S February 2006 28 Box 2-17: Directions for Constructing a Normal Probability Plot

Let X 1 , X 2 , ..., X n represent the n data points. Many software packages will produce normal probability plots and this is the recommended method. If one of these packages is unavailable, then use the following procedure.

STEP 1: Compute the fractions f i = ( i - 0.5)/ n for i = 1,

... , n

.
STEP 2: Compute the normal quantiles for the f i . These are 

## ( ) i i f Y 1

− Φ = , where 

## ( )

⋅ Φ − 1 is the inverse of the standard normal cumulative density function. The Y i can be found using Table A-1: locate an f i in the body of the table, then the Y i is the found on the edges. For example, if f i = 0.975, then Y i is 1.96.

STEP 3: Plot the pairs ( Y i , X i ), for i = 1,

... , n

. STEP 4: If desired, then add the quartile line through the first and third quartiles. This acts as a guide to determine if the points follow a straight line, which may be fitted to the data.

If the graph of these pairs approximately form a straight line, then the data may be well modeled by a normally distribution. Otherwise, use the table above to determine the shape and tail behavior.

Box 2-18: Example of Normal Probability Plot
Consider the following 15 data points: 5, 5, 6, 6, 8, 8, 9, 10, 10, 10, 10, 10, 12, 14, and 15.

STEP 1: The computed fractions, f i , are: 0.0333, 0.1000, 0.1667, 0.2333,

... , 0.9667.

STEP 2: Working inside-out in Table A-1, the normal quantiles for the f i are: -1.83, - 1.28, -0.97, -0.73, -0.52, -0.34, -0.17, 0.00, 0.17, 0.34, 0.52, 0.73, 0.97, 1.28, 1.83.

STEP 3: The data and normal quantile pairs are plotted in the graph below.

STEP 4: Using Box 2-1, the first and third quartiles are 7 and 10, respectively. The first and third quartiles of the standard normal distribution are –0.67 and 0.67, respectively. The quartile line is the line through the points (-0.67, 7) and (0.67, 10), and has been added to the plot.

The plot looks approximately linear, but deviates from the quartile line in the upper-tail. However, no definite conclusion should be drawn from a plot of 15 points. A formal test of normality (section 4.2) should be performed.

### EPA QA/G-9S February 2006 29 

## To compare and contrast several variables, collections of the single variable displays described in previous sections are useful. For example, the analyst may generate box-plots or histograms for each variable using the same axis fo r all of the variables. Separate plots for each variable may be overlaid on one graph, such as overlaying quantile plots for each variable. Another useful technique for comparing two vari ables is to place histograms (sometimes called bi-histograms) or stem-and-leaf plots back to back. Additional plots to display two or more variables are described in Sections 2.3.6.1 thr ough 2.3.6.3. In many software packages, three dimensional scatter plots can be rotated and comp ared in order to find hidden relationships and anomalies.

2.3.6.1 Scatterplot


## For data sets consisting of multiple observations per sampling point, a scatterplot is one of the most powerful gra phical tools for analyzing the relationship between tw o or more variables. Scatterplots are easy to construct for two variables (Figure 2-5) and many software packages can construct 3-dimensional scat terplots. Directions for constructing a 2-dimensional scatterplot are given in Box 2-19 along with an example.

Box 2-19: Directions for Generating a Scatterplot and an Example

Let X 1 , X 2 , ..., X n represent one variable of the n data points and let Y 1 , Y 2 , ..., Y n represent a second variable of the n data points. The paired data can be written as ( X i , Y i ) for i = 1, ..., n . To construct a scatterplot, plot one variable along the horizontal axis and the other variable along the vertical axis.

Example: PCE values are displayed on the vertical axis and Chromium VI values are displayed on the horizontal axis of Figure 2-5.

Obs PCE (ppb) Chromium VI (ppb) Obs PCE (ppb) Chromium VI (ppb) Obs PCE (ppb) Chromium VI (ppb) 1 14.49 3.76 9 2.23 0.77 17 4.14 2.36 2 37.21 6.92 10 3.51 1.24 18 3.26 0.68 3 10.78 1.05 11 6.42 3.48 19 5.22 0.65 4 18.62 6.30 12 2.98 1.02 20 4.02 0.68 5 7.44 1.43 13 3.04 1.15 21 6.30 1.93 6 37.84 6.38 14 12.60 5.44 22 8.22 3.48 7 13.59 5.07 15 3.56 2.49 23 1.32 2.73 8 4.31 3.56 16 7.72 3.01 24 7.73 1.61 25 5.88 1.42



## A scatterplot can clearly show the relationship between two variables if the data range is sufficiently large. Truly linear re lationships can always be identified in scatter plots, but truly nonlinear relationships may appear linear (or some other form) if the data range is relatively small. Scatterplots of linearly correlated variable s cluster about a straight line. As an example of a nonlinear relationship, consider two variables where one variable is approximately equal to the square of the other. With an adequate range in the data, a scatterplot of this data would display a 02468 0 10 20 30 40

Chromium VI (ppb)PCE (ppb)



## Figure 2-5. Example of a Scatterplot

### EPA QA/G-9S February 2006 30 

## partial parabolic curve. Other important modelin g relationships that may appear are exponential or logarithmic. Two additional uses of scatterp lots are the identification of potential outliers for a single variable or for the paired variables a nd the identification of clustering in the data.

2.3.6.2 Extensions of the Scatterplot


## It is easy to construct a 2-dimensional sc atterplot by hand and ma ny software packages can construct a useful 3-dimensi onal scatterplot. However, with more than 3 variables, it is difficult to construct and interpret a scatterplot. Therefore, several graphical representations have been developed that extend the idea of a sc atterplot for data consisting of more than 2 variables.

The simplest of these graphical techniques is a code d scatterplot. All possible two-way combinations are given a code and the pairs of data are plotted on one scatterplot. The coded scatterplotdoes not provide information on threeway or higher intera ctions between the variables. If the data ranges for the variables are comparable, then a single set of axes may suffice, if greater data ranges, different scales will be required. As an example, consider a data set of 3 variables, A
, B , and C and assume the all of the data ranges are similar. An analyst may choose to display the pairs ( A i 

## , B i 

## ) using a small circle, the pairs ( A i 

## , C i 

## ) using a plus-sign, and the pairs ( B i 

## , C i 

## ) using a square on one scatterplot. The completed coded scatterplot is given in Figure 2-6.

The parallel-coordinates method employs a scheme where coordinate axes are drawn in parallel (instead of perpendicular). Figure 2-7 is an example of a parallel-coordinate plot. Consider a sample point X consisting of values X 1 

## for measurement 1, X 2 

## for measurement 2, through X p 

## for
measurement p . A parallelcoordinate plot is constructed by placing an axis for each of the p measurements in parallel and plotting X 1 

## on axis 1, X 2 

## on axis 2, and so on. The points are then 0 10 20 0 10 20 30 40

(ppb)(ppb) Chromium vs. PCE

Atrazine vs. PCE

Atrazine vs. Chromium IV



## Figure 2-6. Example of a Coded Scatterplot

Data Values for Variable 1

Data Values for Variable 2

012345678 0 10 20 30 40

Data Values for Variable 3 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14



## Figure 2-7. Example of a Parallel-Coordinate Plot

### EPA QA/G-9S February 2006 31 

## joined with a broken line. This graph contains al l of the information of a scatterplot in addition to information concerning 3-way and hi gher interactions. However, with p measurements, onemust construct several parallelcoordinate plots in order to display all possible pairs.

A scatterplot matrix is another useful method of extending scatterplots to higher dimensions. In this case, a scatterplot is created for all possible pairs of measurements which are then displayed in a matrix format. This method is easy to implement and provides a concise method of displaying the individual scatterplots. Interpretation proceeds as with a simple or coded scatterplot. As in those cases, this method does not provide information about 3-way or higher interactions. An example of a scatterplot matrix is contained in Figure 2-8.

2.3.6.3 Empirical Quantile-Quantile Plot


## An empirical quantile-quantile (q-q) plot is a plot of the quantiles (Section 2.2.1) of two data sets agai nst each other and is similar to the normal probability plot discussed in Section 2.3.5. This plot (Figure 2-9) is used to compare the distributions of two measurements; for example, the distributions of lead and iron concentrations in a drinking water well. This technique is used to determine if the data sets were generated from the same underlying distribution. If the graph is approximately linear, then the distri butions are roughly the same. If the graph is not linear, then the departures from linearity give important informati on about how the two data distributions differ. The interpretation is similar to that of a norma l probability plot. Directions for constructing an empirical q-q plot with an ex ample are given in Box 2-20. 0 10 20 30 40 0 10 20 30 40

Chromium IV (ppb)Chromium IV (ppb)

0 2 4 6 8 10 12 14 0 10 20 30 40

Atrazine (ppb)Chromium VI (ppb) 0 10 20 30 40 0 2 4 6 8 10 12 14

Chromium IV (ppb)Atrazine (ppb)

0 2 4 6 8 10 12 14 0 2 4 6 8 10 12 14

Atrazine (ppb)Atrazine (ppb)



## Figure 2-8. Example of a Scatterplot Matrix

Figure 2-9. Example of an Empirical Q-Q Plot

### EPA QA/G-9S February 2006 32 Box 2-20: Directions for Constructing an Empirical Q-Q Plot with an Example

Let X 1 , X 2 ,..., X n represent n data points of one variable and let Y 1 , Y 2 ,..., Y m represent a second variable of

m data points. Let X (1) , X (2) ,..., X ( n ) and Y (1) , Y (2) ,..., Y ( m ) be the ordered data sets.

If n = m , then an empirical q-q plot of the two variables is simply a plot of the ordered values of the variables, i.e., a scatterplot of the pairs ( X (1) , Y (1) ), ( X (2) , Y (2) ), ..., ( X ( n ) , Y ( n ) ).

Suppose n > m . Then the empirical quantile-quantile plot will consist of m pairs of points with the ordered Y

values plotted against the m evenly spaced quantiles of X . Using Box 2-1, compute the quantiles that correspond to the fractions 

## ( ) ( ) 1 1

− − = m j q j for j = 1,

... , m

.
Example : Consider contaminant readings from two separate drinking water wells at the same site.

well 1: 1.32, 3.26, 3.56, 4.02, 4.14, 5.22, 6.30, 7.72, 7.73, 8.22 well 2: 0.65, 0.68, 0.68, 1.42, 1.61, 1.93, 2.36, 2.49, 2.73, 3.01, 3.48, 5.44.

Since the sample sizes are not equal, we need to compute the 10 evenly space quantiles for well 2.

q 1 = 0, so the first quantile is y ( q 1 ) = 0.65.
q 2 = 1/9. So 

## ( ) 9 / 20 1 9 / 1 11 1 1 2 = + ⋅ = + ⋅ − = q n r , 

## ( ) 2 = = r i floor , and 9 / 2 = − = i

r f

. Therefore, the second quantile is 

## ( ) ( ) ( ) 

## ( ) ( ) 68

. 0 9 / 2 9 / 7 3 2 2 = ⋅ + ⋅ = X X q y

.
Continuing this process for j = 2,

... ,10 yields the following 10 quantiles:

0.650, 0.680, 1.009, 1.547, 1.894, 2.374, 2.570, 2.886, 3.376, 5.440.

These, paired with the well 1 data, are plotted in Figure 2-9.



## 2.3.7 Plots for Temporal Data


## Data collected over specific time intervals (e .g., monthly, biweekly , or hourly) have a temporal component. For example, air monitoring measurements of a pollutant may be collected once a minute or once a day; water quality monito ring measurements of a contaminant level may be collected weekly or monthly. An analyst ex amining temporal data may be interested in the trends over time, correlation among time periods, or cyclical patterns. Some graphical techniques specific to temporal data are the tim e plot, lag plot, correlogram, and variogram.

A data sequence collected at regular time interv als is called a time series. Time series data analysis is beyond the scope of this guidance. It is recommended that the interested reader consult a statistician. The graphical representa tions presented in this section are recommended for any data set that includes a temporal compone nt regardless of the d ecision to perform a time series analysis. The graphical techniques described below will help identify temporal patterns that need to be accounted for in any analysis of the data.

The analyst examining temporal environmental data may be interested in seasonal trends, directional trends, serial correlation, or stationarity. Seasonal trends are patterns in the data that repeat over time, i.e., the data rise and fall re gularly over one or more time periods. Seasonal trends may be large scale, such as a yearly cycl e where the data show th e same pattern of rising and falling from year to year, or the trends may be small scale, such as a daily cycle. Directional

### EPA QA/G-9S February 2006 33 

## trends are positive or negative trends in the data which is of importance to environmental applications where contaminant levels may be incr easing or decreasing. Serial correlation is a measure of the strength of the linear relations hip of successive observa tions. If successive observations are related, statistical quantitie s calculated without accounting for the serial correlation may be biased. Finall y, another item of interest for temporal data is stationarity (cyclical patterns). Stationary data look the sa me over all time periods. Directional trends or a change in the variability in the data imply non-stationarity.

2.3.7.1 Time Plot


## A time plot (also known as a time series plot) is simply a plot of the data over time. This plot makes it easy to identify lack of randomness, changes in location, change in scale, smallscale trends, or large-scale trends over time. Sm all-scale trends are displayed as fluctuations over smaller time periods. For example, ozone le vels over the course of one day typically rise until the afternoon, then decrease, and this process is repeated every day. Larger scale trends, such as seasonal fluctuations, appe ar as regular rises and drops in the graph. For example, ozone levels tend to be higher in the summer than in th e winter so ozone data te nd to show both a daily trend and a seasonal trend. A time plot can also show directional trends or changing variability over time.

A time plot (Figure 2-10) is constructed by plotting the measurements on the vertical axis versus the actual time of observation or the order of observation on the horizontal axis. The points plotted may be connected by lines, but this may create an unfounded sense of continuity. It is important to use the actual time or number at which the observation was made. This can create discontinuities in the plot but are need ed as the data that should have been collected now appear as “missing values” bu t do not disturb the inte grity of the plot. Plotting the data at equally spaced intervals when in reality there were different time peri ods between observations is not advised.

The scaling of the vertical axis of a time plot is of some importance. A wider scale tends to emphasize large-scale trends, whereas a narrowe r scale tends to emphasize small-scale trends. Using the ozone example above, a wide scale would emphasize the seasonal component of the data, whereas a smaller scale would tend to emph asize the daily fluctuations. Directions for constructing a time plot are contained in Box 2-21 along with an example. 0 5 10 15 20 25 30 35 40 45 50 0 5 10 15 20

TimeData Values



## Figure 2-10. Example of a Time Plot

### EPA QA/G-9S February 2006 34 Box 2-21: Directions for Generating a Time Plot and an Example

Let X 1 , X 2 , ..., X n represent n data points listed in order by time, i.e., the subscript represents the ordered time interval. A plot of the pairs ( i

, X i ) is a time plot of the data.

Example: Consider the following 50 daily observations listed in order by day:

10.05, 11.22, 15.90, 11.15, 10.53, 13.33, 11.81, 14.78, 10.93, 10.31,
7.95, 10.11, 10.27, 14.25, 8.60, 9.18, 12.20, 9.52, 7.59, 10.33,
12.13, 11.31, 10.13, 7.11, 6.72, 8.97, 10.11, 7.72, 9.57, 6.23,
7.25, 8.89, 9.14, 12.34, 9.99, 11.26, 5.57, 9.55, 8.91, 7.11,
6.04, 8.67, 5.62, 5.99, 5.78, 8.66, 5.80, 6.90, 7.70, 8.87. A time plot is constructed by plotting the pairs ( i

, X i ) where i represents the number of the day and X i represents the concentration level. The plot is shown in Figure 2-10. Note the slight negative trend.



## 2.3.7.2 Lag Plot


## A lag plot is another method to determine if the data set or time series is random. Nonrandom structure in the lag plot implie s nonrandomness in the data. Examples of nonrandom structure are linear patter ns or elliptical patterns. A linear pattern implies the data contain a directional trend while an elliptical pattern implies the data contain a seasonal component.

If we have data points X 1 

## , X 2 

## , ... 

## , X n 

## , then a lag plot is a scatterplot of the points ( X

t 

## , X t

- k



## ) for some integer lag k , the most common being lags 1, 2, or 3. Figure 2-11 is a 1-lag plot for the data in the example from Box 2-21. Notice that there is a light linear struct ure suggestion a possible directional trend in the data. See Section 2.3.7.3 for higher lags.

2.3.7.3 Plot of the Autocorrelation Function (Correlogram)



## Serial correlation is a measure of the strength of the relationship between successive observations. If successive observations are related, then the data mu st be transformed or the relationship must be accounted for in the data analysis. The correlogram is a plot that is used to display serial correlations when the data are collected at equally spaced time intervals. The autocorrelatio n function is a summary of the serial correlations of data. The 1 st 

## sample autocorrelation coefficient, r 1 

## , is the correlation between points at lag 1 (points that are 1 time unit apart); r 2 

## is the correlation between points at lag 2; etc. A correlogram (Figure 2-12) is a plot of the sample autocorrelation coefficients, r k 

## , versus k

. Figure 2-11. Example of a Lag Plot

### EPA QA/G-9S February 2006 35 

## For a large independe nt data sequence of n time points, the autocorrelations are approximately normally distributed with mean zero and variance 1/ n . Therefore, to determine if the time points are independent, first plot the approximate 95% confidence lines n / 2 ± 

## (shown as dashed lines in Figure 2-12) on the correlogram. If any of the autocorrelations lie outside the confidence lines, then there is evidence of serial correlation and we conclude that the time points are not independent.

In examining Figure 2-12, there are 4 time poi nts that lie outside the 95% confidence bounds. These are at lags 1, 2, 12 and 24. This demonstrates strong evidence that the sequence is serially correlated (most likely containing a positive trend along with an annual component).

In application, the correlogram is only useful for data at equally spaced intervals and for irregular intervals a text on the geostatistical use of a variogram is recomm ended. Directions for constructing a correlogram are contained in Box 2-22; example calculations are contained in Box 2-23. For large sample sizes, a correlogram is tedious to construc t by hand; therefore, statistical software should be used.

Box 2-22: Directions for Constructing a Correlogram

Let X 1 , X 2 , ..., X n represent the data points ordered by time for equally spaced time points, i.e., X 1 was collected at time 1, X 2 was collected at time 2, and so on. To construct a correlogram, first compute the sample autocorrelation coefficients, r k . So for k = 0, 1, ..., compute

k k r g
g = 0 and 

# ( ) ( ) X X X X n = g k t t n

k t k − −

−

+ = 

# ∑

1 1

Once the r k have been computed, a correlogram is the graph ( k

, r k ) for k = 0, 1, . . . , and so on. As an approximation, compute up to k = n /6. Also, note that r 0 = 1. Finally, place horizontal lines at n / 2 ±

.



## 2.3.7.4 Multiple Observations in a Time Period


## Many times in environmental data collection, multiple samples are taken at each time point. For example, the data collection design may specify collecting 5 samples from a drinking well every Wednesday for three months. In this cas e, the time plot described in Section 2.3.7.1 may be used to display the complete data set, display the mean weekly level, display a confidence interval for each mean, or display a confidence interval fo r each mean with the individual data values. A time plot of all the data will allow the analyst to determine if the variability for the different coll ection periods changes. A time plot of the means will allow the 0 5 10 15 20 25 30 -0.5 -0.25 0 0.25

0.5 0.75 1 1.25

kr(k)



## Figure 2-12. Example of a Correlogram

### EPA QA/G-9S February 2006 36 

## analyst to determine if the means are possibl y changing between the co llection periods. In addition, each collection period may be treated as a distinct variable and the methods described in Section 2.3.6 may be applied.

Box 2-23: Example Calculations for Generating a Correlogram
A correlogram will be constructed using the following four hourly data points: hour 1: 4.5 ppm, hour 2: 3.5 ppm, hour 3: 2.5 ppm, and hour 4: 1.5 ppm. Only four data points are used so all computations may be shown for illustrative purposes. Therefore, t he guideline of computing no more than n /6 autocorrelation coefficients will be ignored. The first step to constructing a correlogram is to compute the sample mean (Box 2-2), which is 3. Then, 

# ( ) ( ) ( ) ( )( )( )( ) 

# [ ] 25

. 1 3 5

. 1 3 5

. 2 3 5

. 3 3 5

. 4 4 1 4 1 4 1 2 2 2 2 2 4

1 0 4

1 0 = − +

− +

− +

− = − = − − 

# ∑ ∑

= −

= X X X X X X = g t
t t t
t 

# ( ) ( ) ( )( ) ( )( ) ( )( ) 

## [ ] 3125

. 0 3 5

. 2 3 5

. 1 3 5

. 3 3 5

. 2 3 5

. 4 3 5

. 3 4 1 4 1 1 4

2 1 = − − +

− − +

− − = − −

−

= 

# ∑ X X X X = g t t
t
Similarly, g 2 = -0.375 and g 3 = -0.5625.

So 25

. 0 25

. 1 3125

. 0

0 1 1 = = = g

g r 

## , 3

. 0 25

. 1 375

. 0

0 2 2 − −
= = g

g r and 45

. 0 25

. 1 5625

. 0

0 3 2 − −
= = g

g r



## .

Thus, the correlogram of these data is a plot of (0, 1) (1, 0.25), (2, -0.3) and (3, -0.45) with two confidence lines

at n / 2 ± = ±1. This graph is shown below.

01234 -1 -0.8 -0.6 -0.4 -0.2

0 0.2 0.4 0.6 0.8

1

kr(k)

-1.0 +1.0

(Hours)



## 2.3.7.5 Four-Plot


## A four-plot is a collection of 4 specific plot s that provide differen t visual methods for illustrating a measurement process. There are 4 ba sic assumptions that underlie all measurement processes, there should be: random data, a fixed di stribution, with a fixe d location and variance. If all of these assumptions hold, then the measur ement process or data set is considered to be stable. The fixed distribution we discuss here is a normal di stribution, but any probability distribution of interest could be used. A four-plot consists of a time plot, a lag plot, a histogram and a normal probability plot. The data are random if the lag plot is structureless. The data are

### EPA QA/G-9S February 2006 37 

## from a normal distribution if the histogram is symmetric and bell-shaped and the normal probability plot is linear. If th e time plot is flat and non-drifti ng, then the data have a fixed location. If the time plot has a constant spread , the data have a fixed va riance. Figure 2-13 is a four-plot for the data contained in the example of Box 2-21. In this partic ular example, note that the data are not quite normal (deviations from the straight line on the plot), does not have a fixed location ( a downward trend in the time plot), and possibly has se rial correlation present (the tendency of the lag plot to be increasing from left to right).

Figure 2-13. Example of a Four-Plot

2.3.8 Plots for Spatial Data


## The graphical representations of the precedi ng sections may be useful for exploring spatial data. However, an analyst examining spa tial data may be interested in the location of extreme values, overall spatial trends, and the degree of continuity among neighboring locations. Graphical representations for spatial data includ e postings, symbol plots, correlograms, h-scatter plots, and cont our plots.

### EPA QA/G-9S February 2006 38 

## The graphical representations presented in th is section are recommended for all spatial data regardless of whether or not geostatistical methods will be used to analyze the data. The graphical representations described below will help identify spatial patterns that need to be accounted for in the analysis of the data.

2.3.8.1 Posting Plots


## A posting plot (Figure 2-14) is a map of data locations along with the corresponding data values. Data posting may reveal obvious errors in data location and identify data values that may be in error. The graph of the sampling locations gives the analyst an idea of where the data were collected (i .e., the sampling design), areas that may have been inaccessible, and areas of special interest to the decision maker which may have been heavily sampled. It is often useful to mark the highest and lowest values of the data to see if there are any obvious trends. If all of the hi ghest concentrations fall in one region of the plot, the analyst may consider some method su ch as post-stratifying the data (stratification after the data are collected and analyzed) to account for this fact in the analysis. Directions for generating a posting plot are contained in Box 2-24.

Box 2-24: Directions for Generating a Posti ng Plot and a Symbol Plot with an Example
On a map of the site, plot the location of each sample. At each location, either indicate the value of the data point (a posting plot) or indicate the data value by a symbol (a symbol plot) or circle (a bubble plot). The circles on a bubble plot are equal to the square roots of the data values.

Example: The spatial data displayed in the table below contains both a location (Northing and Easting) and a concentration level ([c]). The chosen symbols are floor (concentration/5) and range from ‘0’ to ‘7.’. The data values with corresponding symbols are:

N E [c] Sym N E [c] Sym N E [c] Sym 15 0 4.0 0 10 10 15.2 3 5 15 12.4 2 15 5 11.6 2 10 15 35.5 7 5 20 22.8 4 15 10 14.9 2 10 20 14.7 2 5 25 19.1 3 15 15 17.4 3 10 25 16.5 3 0 10 10.2 2 15 20 17.7 3 5 0 8.9 1 0 15 5.2 1 15 25 12.4 2 5 5 14.7 2 0 20 4.9 0 10 0 28.6 5 5 10 10.9 2 0 25 17.2 3 10 5 7.7 1

The posting plot of this data is displayed in Figure 2-14, the symbol plot is displayed in Figure 2-15, and the bubble plot is displayed in Figure 2-16. 4.0 11.6 14.9 17.4 17.7 12.4

28.6 7.7 15.2 35.5 14.7 16.5

8.9 14.7 10.9 12.4 22.8 19.1

10.2 5.2 4.9 17.2 C r e e k

R o a d



## Figure 2-14. Example of a Posting Plot

### EPA QA/G-9S February 2006 39 

## 2.3.8.2 Symbol Plots and Bubble Plots


## For large amounts of data, a posting plot may be visual unappealing. In this case, a symbol plot (Figure 2-15) or bubbl e plot (Figure 2-16) can be us ed. Rather than plotting the actual data values, symbols or bubbles that repres ent the data values are displayed. A symbol plot breaks the data range into bins and plots a bin label corresponding to the data value. A bubble plot consists of circles, w ith radii equal to the square root s of the data values. So unlike posting and symbol plots, a bubble pl ot gives a visual impression of the size of the data values. Directions for generating a symbol or bubbl e plot are contained in Box 2-24.

0 2 2 3 3 2

5 1 3 7 2 3

1 2 2 2 4 3

2 1 0 3 C r e e k

R o a d



## Figure 2-15. Example of a Symbol Plot Figure 2-16. Example of a Bubble Plot

2.3.8.3 Other Spatial Graphical Representations


## The three plots described in Sections 2.3.8. 1 and 2.3.8.2 give information on the location of extreme values and spatial trends. The graphs below provide another it em of interest to the data analyst, continuity of the spatial data. The graphical representations are not described in detail because they are used more for prelim inary geostatistical analysis. These graphical representations can be difficult to develop a nd interpret. For more information on these representations, consult a statistician.

An h-scatterplot is a plot of all possible pair s of data whose locatio ns are separated by a fixed distance in a fixed directi on (indexed by h). For example, an h-scatterplot could be based on all the pairs whose locations are 1 meter apart in a southerly direction. An h-scatterplot is similar in appearance to a scatterp lot (Section 2.3.6.1). The shape of the spread of the data in an h-scatterplot indicates the degree of continuity among data values a cer tain distance apart in particular direction. If all the plotted values fall close to a fixed line, then the data values at locations separated by a fixed distance in a fixed location are very similar. As data values become less and less similar, the spread of the data around the fi xed line increases outward. The data analyst may construct several h-scatterplots with different distances to evaluate the change in continuity in a fixed direction.

### EPA QA/G-9S February 2006 40 

## A spatial correlogram is a plot of the co rrelations of the h-scatterplots. As the h-scatterplot only displays the correlation be tween the pairs of data whose locations are separated by a fixed distance in a fixed direction, it is useful to have a graphical representation of how these correlations change for different separati on distances in a fixed direction. The spatial correlogram is such a plot which allows the anal yst to evaluate the change in continuity in a fixed direction as a function of the distance between two points. A spatial correlogram is similar in appearance to a temporal correlogram (Section 2.3.8.2).

Contour plots are used to reve al overall spatial trends in the data by interpolating data values between sample locations. Most contou r procedures depend on the density of the grid covering the sampling area (higher density grids usually provide more information than lower densities). A contour plot gives one of the best overall pictur es of the important spatial features. However, contouring often requires that the actual fluctuations in the data values are smoothed so that many spatial features of the data may not be visible. The cont our map should be used with other graphical representati ons of the data and requires e xpert judgment to interpret.

2.4 PROBABILITY DISTRIBUTIONS

2.4.1 The Normal Distribution


## Data, especially measurements, often occur in natural patterns that can be considered to come from a distribution of values. In most instances, the data values will be grouped around some measure of central tendency such as the mean or median. The spread of the data is called the variance (the square root of this is called the standard deviation). A distribution with a large variance will be more spread out than one with a small variance (Figure 2-17). If a histogram of the data has a bell-shape (symmetric pattern about the mean

with rapidly tapering ta ils), then the underlying distribution is often a normal distribution.

If it is known or assumed that the underlying distribution is normal, then this is usually written as ‘distributed N( μ 

## , σ

2 

## )’ where μ 

## is the mean and σ 2 

## is the variance. By subtracting μ 

## and dividing by σ 

## , any normal distribution can be transformed to a standard normal distribution, N(0,1). A standard normal random variable is most often denoted by Z . A plot of the standard normal is given in Figure 2-18. It is frequently necessary to refer to the percentile s of a standard normal and Figure 2-17. Two Normal Curves, Common Mean, Different Variances

Figure 2-18. The Standard Normal Curve, Centered on Zero

### EPA QA/G-9S February 2006 41 

## in this guidance document, the subscript to a quoted Z -value will denote the percentile (or area under the curve, cumulative from the left), see Figure 2-18 (show ing the area, 0.8413 corresponding to a Z-value of 1.00, and written as Z 0.8413 

## =1.00). Although common practice to use this subscript notation, some text tables a nd software programs use a different nomenclature; the user is advised to verify the meaning of any statistic encountered.

2.4.2 The t -Distribution



## The standard normal curve is used when ex act information on the mean and variance are available, but when only estimates from a sample are available, a different type of sampling distribution applies. When only information fr om a random sample on sample mean and sample variance is known for decision making purposes, a Student’s- t distribution is a ppropriate. It resembles a standard normal but is lower in the center and fatter in th e tails. The degree of fatness in the tails is a function of the degrees of freedom available, which in turn is related to sample size. As the sample size increases, the es timates of the mean and variance improve. As a result, the Student’s- t distribution more closely resembles the standard normal distribution.

2.4.3 The Lognormal Distribution


## Another commonly used distribution in environmental work is th e lognormal distribution. The lognormal distribution is bounded on the left by 0, has a fatter right-tail than the normal distribution, and has a right-skewed shape. These characteristics are shown in Figure 2-19. The lognormal and normal distributions are related by a simple transformation: if X is distributed lognormally, then Y

= ln( X

) is distributed normally. However, logtransforming lognormal data to perform statistical procedures requiring normal data is a practice that should be done with care

2.4.4 Central Limit Theorem


## In many testing and estimation situations in environmental work, the focus of the investigation centers on the mean of a population. It is rare that true normality of the observations can be assumed. In most cases, th e normally-based statistical tests are not overly affected by the lack of normality since tests are very robust and perform tolerably well unless gross non-normality is present. In addition, many tests become increasingly tolerant of deviations from normality as the number of obser vations increase. In simple terms, the underlying distribution of the samp le mean more closely resemble s a normal distribution as the number of observations increases. This occurs no matter what the underly ing distribution of an individual observation. This phenomenon is calle d the Central Limit Theorem and is the basis of many statistical procedures. Figure 2-19. Three Different Lognormal Distributions

### EPA QA/G-9S February 2006 42

### EPA QA/G-9S February 2006 43 

## CHAPTER 3

STEP 3: SELECT THE STATISTICAL METHOD

Step 3: Select the Statistical Method

• Select the statistical method based on the data user's objectives and the results of the preliminary data review.

 If the problem involves comparing study results to a fixed threshold, such as a regulatory standard, consider the methods in Section 3.2.

 If the problem involves comparing two populations, such as comparing data from two different locations or processes, then consider the hypothesis tests in Section 3.3.

• Identify the assumptions underlying the statistical method.

 List the key underlying assumptions of the statistical procedure, such as distributional form, dispersion, independence, etc.

 Note any sensitive assumptions where relatively small deviations could jeopardize the validity of the results. 

## THE DATA QUALITY ASSESSMENT PROCESS

Review DQOs and Sampling Design

Conduct Preliminary Data Review

Select the Statistical Method

Verify the Assumptions

Draw Conclusions from the Data SELECT THE STATISTICAL METHOD

Purpose

Select an appropriate procedure for analyzing data based on the preliminary data review.

A ctivities

Select Statistical Method Identify Assumptions Underlying Test

Tools

Hypothesis tests for a single population

H y pothesis tests for comparin g two populations Confidence Intervals for a single population

Confidence Intervals for comparin g two populations

### EPA QA/G-9S February 2006 44 

## List of Boxes

Page 

## Box 3-1: Directions for the One-Sample t -Test........................................................................... 49 Box 3-2: Directions fo r Computing a One-Sample t Confidence Interval or Limit .................... 49 Box 3-3: A One-Sample t -Test Example..................................................................................... 50 Box 3-4: An Example of a One-Sample t Upper Confidence Limit for a Population Mean....... 50 Box 3-5: Directions for Co mputing a One-Sample Tolerance Interval or Limit......................... 51 Box 3-6: An Example of a One-Sample Upper Tolerance Limit ................................................ 51 Box 3-7: Directions for a One-Sample t -Test for a Stratified Random Sample .......................... 54 Box 3-8: An Example of a One-Sample t -Test for a Stratified Random Sample........................ 55 Box 3-9: Directions for the Chen Test......................................................................................... 56 Box 3-10: Example of the Chen Test........................................................................................... 5 7 Box 3-11: Directions for Computing Confidence Limits for the Population Mean of a Lognormal Distribu tion Using Land’s Method ......................................... 58 Box 3-12: An Example Using Land’s Method ............................................................................ 58 Box 3-13: Directions for the On e-Sample Test for Proportions.................................................. 59 Box 3-14: Directions for Computing a Conf idence Interval for a Population Proportion........... 59 Box 3-15: An Example of the One-Sample Test for Proportions................................................ 60 Box 3-16: Directions for th e Sign Test (One-Sample) ................................................................ 62 Box 3-17: An Example of the Sign Test (One-Sample) .............................................................. 63 Box 3-18: Directions for the Wilc oxon Signed Rank Test (One-Sample) .................................. 64 Box 3-19: An Example of the Wilc oxon Signed Rank Test (One-Sample) ................................ 65 Box 3-20: Directions for the Two-Sample t -Test (Equal Variances) .......................................... 67 Box 3-21: Directions for a Two-Sample t Confidence Interval (Equal Variances)..................... 68 Box 3-22: An Example of a Two-Sample t -Test (Equal Variances) ........................................... 68 Box 3-23: Directions for the Two-Sample t -Test (Unequal Variances)...................................... 69 Box 3-24: Directions for a Two-Sample t Confidence Interval (Unequal Variances) ................ 70 Box 3-25: An Example of the Two-Sample t -Test (Unequal Variances).................................... 70 Box 3-26: Directions for a Two- Sample Test for Proportions .................................................... 72 Box 3-27: Directions for Computing a Conf idence Interval for the Difference Between Population Proportions................................................................................................ 73 Box 3-28: An Example of a Two-Sample Test for Proportions .................................................. 73 Box 3-29: Directions for the Paired t -Test................................................................................... 74 Box 3-30: Directions for Computing the Paired t Confidence Interval....................................... 74 Box 3-31: An Example of the Paired t -Test................................................................................. 75 Box 3-32: Directions for the Wilcoxon Rank Sum Test.............................................................. 77 Box 3-33: An Example of the Wilcoxon Rank Sum Test............................................................ 78 Box 3-34: A Large Sample Example of the Wilcoxon Rank Sum Test ...................................... 79 Box 3-35: Directions for the Quantile Test.................................................................................. 80 Box 3-36: A Example of the Quantile Test ................................................................................. 80 Box 3-37: Directions for the Slippage Test ................................................................................. 81 Box 3-38: A Example of the Slippage Test ................................................................................. 82 Box 3-39: Directions for the Sign Test (Paired Samples)............................................................ 83

### EPA QA/G-9S February 2006 45 

## Page 

## Box 3-40: An Example of the Sign Test (Paired Samples) ......................................................... 84 Box 3-41: Directions for the Wilcoxon Signed Rank Test (Paired Samples).............................. 85 Box 3-42: An Example of the Wilcoxon Signed Rank Test (Paired Samples) ........................... 86 Box 3-43: A Large Sample Example of the Wilcoxon Signed Rank Test (Paired Samples) ...... 87 Box 3-44: Directions for Dunnett’s Test ..................................................................................... 88 Box 3-45: An Example of Dunnett’s Test ................................................................................... 89 Box 3-46: Directions of the Fligner-Wolfe Test.......................................................................... 91 Box 3-47: An Example of the Fligner-Wolfe Test ...................................................................... 92

### EPA QA/G-9S February 2006 46 

## CHAPTER 3

STEP 3: SELECT THE STATISTICAL METHOD

3.1 OVERVIEW AND ACTIVITIES


## This chapter provides an overview of issues associated with selecting an appropriate statistical method that will be used to draw conc lusions from the data. There are two important outputs from this step: (1) the c hosen method, and (2) the assump tions underlying the method. If a particular statistical procedur e has been specified either in the DQO Process, the QA Project Plan, or the particular program or study, the analyst should use the results of the preliminary data review to determine if it is appropriate for the da ta collected. If a part icular procedure has not been specified, then th e analyst should select one based upon the data user's objectives and the preliminary data review.

One division in the methods of this sect ion is between parametric and nonparametric hypothesis tests. Parametric te sts typically concer n the population mean or quantile, use the actual data values, and assume data values follow a specific probability distribution. Nonparametric tests typically concern the popula tion mean or median, use data ranks, and don’t assume a specific probability distribution. Parametric tests will have more power than a nonparametric counterpart if the assumptions are met. However, the distributional assumptions are often strict or undesirable for the parametric tests and deviations can lead to misleading results.

3.2 METHODS FOR A SINGLE POPULATION



## The methods of this section concern co mparing a single population parameter to a regulatory value (i.e. a fixed number) or the es timation of the population parameter. If the regulatory or action-value was estimated, then a one-sample method is not appropriate and a two-sample test should be selected.

An example of a one-sample test would be to determine if 95% of all companies emitting sulfur dioxide into the air are below a fixed discharge level. For this example, the population parameter is a proportion and the threshold value is 95% (0.95). Comparing the mean contaminant concentration of a hazardous site to the mean concentration of a background area would be a considered a two-sample test.

The hypothesis tests discussed in this section may be used to determine if there is evidence that θ < 

## θ 0 

## , θ > θ 0 

## , or θ ≠ 

## θ 0 

## where θ represents the popul ation mean, median, proportion, or quantile, and θ 0 

## represents the threshold value. There are also confidence/tolerance interval procedures to estimate θ . Section 3.2.1 discusses parametric hypothesis tests and confidence /tolerance intervals for a popu lation mean or a population proportion. Section 3.2.2 disc usses nonparametric hypothesis test s for the population median or population mean.EPA QA/G-9S 47 DRAFT FINALSeptember 2004Decision Tree for Selecting the Specific MethodTestPopulationParameterDistributionalAssumptionSectiont-Test and CIMeanNormal 3.2.1.1Stratifiedt-TestMeanNormal 3.2.1.3Chen TestMeanRight-Skewed 3.2.1.4Land’s CI MethodMeanLognormal 3.2.1.5Test for a Proportion and CIProportion3.2.1.6Sign TestMedianNone 3.2.2.1Wilcoxon Signed Ranks TestMedian /MeanSymmetric 3.2.2.2t-Test and CI (equal variances)Diff in MeansNormal 3.3.1.1.1t-Test and CI (unequal variances)Diff in MeansNormal 3.3.1.1.2Test for Proportions and CIDiff in Props3.3.1.1.3Pairedt-TestDiff in MeansNormal 3.3.1.2.1Wilcoxon Rank Sum TestDiff in MeansSame Variance 3.3.2.1.1Quantile Test Right-Tail 3.3.2.1.2Slippage Test Right-Tail 3.3.2.1.3Sign TestMedianNone 3.3.2.2.1Wilcoxon Signed Ranks TestMedian /MeanSymmetric 3.3.2.2.2Dunnett’s TestMean3.4.1.1Kruskal-Wallis TestMean3.4.1.2Pairedk-SampleNonparametricOne-SampleNonparametricPairedIndependentNonparametricIndependentParametricParametricTwo-SampleParametric

### EPA QA/G-9S 48 February 2006 

## 3.2.1 Parametric Methods


## These methods rely on the knowing the specific distribution of the population or of the statistic of interest.

3.2.1.1 The One-Sample t -test and Confidence Interval or Limit



## Purpose: Test for a difference between a population m ean and a fixed threshold or to estimate a population mean.

Data : A simple or systematic random sample, x 1 

## , ... 

## , x n 

## , from the population of interest.

Assumptions : The data are independent and come from an approximately normal distribution or the sample size is large ( n ≥ 

## 30).

Limitations and Robustness: One-sample t methods are robust agains t the population distribution deviating moderately from normality. However, they are not robust agai nst outliers and have difficulty dealing with non-de tects. The nonparametric meth ods of section 3.2.2 are an alternative. The substitution or adjustment procedures of chapter 4 can be used for non-detects.

Directions for the one-sample t-te st are contained in Box 3-1, with an example in Box 3–3. Directions for the one-sample confidence interval are contained in Box 3-2, with an example in Box 3-4.

3.2.1.2 The One-Sample Tolerance Interval or Limit



## Purpose: A tolerance interval specif ies a region that contains a certain proportion of the population with a certain confidence . For example, “the 99% tole rance interval for 90% of the population is 7.5 to 9.9”, is interpreted as, “I ca n be 99% certain that the interval 7.5 to 9.9 captures 90% of the population.”

Data: A simple or systematic random sample, x 1 

## , ... 

## , x n 

## , from the population of interest. The sample may or may not contain compositing.

Assumptions: The data are independent and approximately normally distributed.

Limitations and Robustness: Tolerance intervals are robust ag ainst the population distribution deviating moderately from normality. Howeve r, they are not robus t against outliers.

Directions for the one-sample tole rance interval are contained in Box 3-5, with an example in Box 3-6.

### EPA QA/G-9S 49 February 2006 Box 3-1: Directions for the One-Sample t -Test

COMPUTATIONS: Compute the mean, X , and standard deviation, s , of the data set.

STEP 1. Null Hypothesis: C : H 0 = μ

STEP 2. Alternative Hypothesis: i) C : H A > μ (upper-tail test) ii) C : H A < μ (lower-tail test) iii) C : H A ≠ μ (two-tail test)

STEP 3. Test Statistic:

n s X t C

−
= 0

STEP 4. a) Critical Value: Use Table A-2 to find: i) α

− − 1

1
, n t ii) α

− − − 1 1
, n t iii) 2 1 1 α

− − , n t

STEP 4. b) p-value: Use Table A-2 to find: i) 

## ( ) 0 1 t t n >

− P ii) 

## ( ) 0 1 t t n <

− P iii) 

# ( ) 0 1 P 2 t t n > ⋅

− , where 0 t is the absolute value of t 0
.
STEP 5. a) Conclusion: i) If t 0 > α

− − 1

1
, n t , then reject the null hypothesis that the true population mean is equal to the threshold C. ii) If t 0 < α

− − − 1 1
, n t , then reject the null hypothesis. iii) If 2 1 1 0 /

, α

− − > n

t t , then reject the null hypothesis.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the true population mean is equal to the threshold C.
STEP 6. If the null hypothesis was not rejected, there is only one false acceptance error rate ( β at μ 1 ), and if 

# ( )

( ) 2 C 2 1 2 1 2 1 1 2

α β α
μ ′

− − ′

− +

−

+ ≥ z z z s n , then the sample size was probably large enough to achieve the DQOs.

The value of α ′ is α for a one-sided test and 2 / α for a two-sided test.

Box 3-2: Directions fo r Computing a One-Sample t Confidence Interval or Limit

COMPUTATIONS: Compute the sample mean, X , and sample standard deviation, s , of the data set.

A 100(1 - α )% confidence interval for μ is n s t X n ⋅

±

− − 2

1

, 1 α , where Table A-2 is used to find 2 1

, 1 α

− − n t

.
A 100(1 - α )% upper confidence limit (UCL) for μ is n s t X n ⋅

+

− − α 1

, 1 .
A 100(1 - α )% lower confidence limit (LCL) for μ is n s t X n ⋅

−

− − α 1

, 1 .

### EPA QA/G-9S 50 February 2006 Box 3-3: A One-Sample t -Test Example
Consider the following 9 random data values (in ppm):

82.39, 103.46, 104.93, 105.52, 98.37, 113.23, 86.62, 91.72, 108.21.

This data will be used to test: H 0 : μ ≤ 95 ppm vs. H A : μ > 95 ppm. The decision maker has specified a 5% false rejection error rate ( α ) at 95 ppm (C), and a 20% false acceptance error rate ( β ) at 105 ppm ( μ 1 ).

COMPUTATIONS: The mean is ppm 38 99

. = X and the standard deviation is s = 10.41 ppm.

STEP 1. Null Hypothesis: H 0 : μ ≤ 95

STEP 2. Alternative Hypothesis: H A : μ > 95 (upper-tail test)

STEP 3. Test Statistic: 26 1

9 41

. 10 95

38 99 0 . = . =

n s C X = t − −

STEP 4. a) Critical Value: Using Table A-2, 86

. 1 95

. 0

, 8 1

, 1 = =

− − t t n α
STEP 4. b) p-value Using Table A-2, 0.10 < p-value < 0.15. Using statistical software, 

## ( ) ( ) 1216

. 0 26

. 1 P value

- p 8 0 1 = > = > =

− t P t t n .
STEP 5. a) Conclusion: Since 1.26 < 1.86, we fail to reject the null hypothesis that the true population mean is at most 95 ppm.

STEP 5. b) Conclusion: Since p-value = 0.1216 > 0.05 = significance level, we fail to reject the null hypothesis that the true population mean is at most 95 ppm.

STEP 6. Since the null hypothesis was not rejected and there is only one false acceptance error rate, it is possible to use the sample size formula to determine if the error rate has been satisfied. Since, 

# ( )

( ) ( ) ( ) 049 8 2 645 1
105 95 842 0 645 1 41

. 10 2 C 2

2 2 2 2 1 2 1 2 1 1 2

. = . +

- . +

. = z z + z s n α β α
μ ′

− − ′

− +

− ≥
,

the false acceptance error rate has probably been satisfied.

Box 3-4: An Example of a One-Sample t Upper Confidence Limit for a Population Mean
The effluent from a discharge point in a plating manufacturing plant was sampled 7 times over the course of 4 days for the presence of Arsenic with the following results (in ppm): 8.1, 7.9, 7.9, 8.2, 8.2, 8.0, 7.9. A 95% upper confidence limit for the population mean will be computed.

COMPUTATIONS: The sample mean is X = 8.03 ppm and the sample standard deviation is s = 0.138 ppm.

A 95% upper confidence limit for μ is: 

## 7 138

. 0 943

. 1 03

. 8

. . 1
, 1 

## ⋅ ±

⋅ +

− − 

## e i
n s t X

n α or 8.131 ppm.

### EPA QA/G-9S 51 February 2006 Box 3-5: Directions for Computing a One-Sample Tolerance Interval or Limit

COMPUTATIONS: Compute the sample mean, X , and sample standard deviation, s , of the data set.

A 100(1 – α )% tolerance interval for (1 – p )% of the population is s k X

⋅

± 2 , where 2

, 1 2

2 / 1 2 1

α χ

− −

⋅ −

⋅ =

n p n n z k

.

Table A-1 is used to find 2 / 1 p z

− and Table A-9 is used to find 2

, 1 α χ

−

n .

A 100(1 – α )% upper tolerance limit for (1 – p )% of the population is s k X

⋅

+ 1 and a 100(1 – α )% lower tolerance limit for (1 – p )% of the population is s k X

⋅

− 1 , where

a ab z z k p p −

+

= − − 2 1 1 1 , 

## ( ) 1 2 1 2 1

− ⋅ − = − n z a α , and n z z b p 2 1 2 1 α

−

− − =

.

Box 3-6: An Example of a On e-Sample Upper Tolerance Limit
The effluent from a discharge point in a plating manufacturing plant was sampled 7 times over the course of 4 days for the presence of Arsenic with the following results (in ppm): 8.1, 7.9, 7.9, 8.2, 8.2, 8.0, 7.9. A 95% upper tolerance limit for 90% of the population will be computed.

COMPUTATIONS: The sample mean is X = 8.03 ppm and the sample standard deviation is s = 0.138 ppm.

Also, 7731

. 0 6 2 65

. 1 1 2

=

⋅ − = a , 2495

. 1 7 65

. 1 28

. 1 2 2

=

− = b , and 716

. 2 7731

. 0 2495

. 1 7731

. 0 28

. 1 28

. 1 2

1 = ⋅ − +

= k

.

A 95% upper tolerance limit for 90% of the population is: 

## 138

. 0 716

. 2 03

. 8

. . 1 

## ⋅ + ⋅ + 

## e i s k X or 8.405
ppm. So we are 95% confident that at least 90% of the population is less than 8.405 ppm.



## 3.2.1.3 Stratified Random Sampling


## This section provides a brief introduction of stratified random sampling and directions for a one-sample t -test with stratified data. For a more in-depth discussion of stratified random sampling, see Guidance on Choosing a Sampling Design for Environmental Data Collection
, EPA QA/G-5S (U.S. EPA 2002).

A stratified random sample occurs when a population is split into subpopulations (called strata) and simple random samples are ta ken from the subpopulations. Reasons for implementing stratified random sampling include administrative convenience and a gain in the precision in the estimates if a heterogeneous population can be split into homogeneous subpopulations.

### EPA QA/G-9S 52 February 2006 

## Suppose the cost of taking a sample is C = c 0 

## + Σ 

## c h 

## n h 

## , where c 0 

## is the overhead cost, c h 

## is the cost of sampling in stratum h , and n h 

## is the sample size taken from stratum h . Then the variance of the sample mean is minimized for given cost C and the cost is minimized for a specified variance if n h 

## is proportional to h h h 

## c W / σ 

## , where W h 

## is the stratum weight (the size of the subpopulation in relation to the entire population) and σ h 

## is the population standard deviation for stratum h . Therefore, a larger sample is taken in a given stratum when the stratum is large, the stratum variance is large, or sampli ng is cheaper in the stratum. If the cost of sampling is equal across all strata, then the vari ance of the sample mean is minimized for a fixed total sample size n when 

# ∑ ⋅ =

h h h h h 

## W W n n σ σ



## .

This type of sample size sele ction is called Neyman allocation. In practice, the unknown σ h 

## is replaced by the observed sample standard deviation.

Purpose: Test for a difference between a population mean and a fixed threshold or to estimate the population mean.

Data: A stratified random sample,

h hn h 

## x x , , 1 

## K , for h = 1,

... 

## , L , where the n h 

## are not necessarily equal. Also, let the total sample size be L 

## n n n + + = L 1 

## .
Assumptions: The data in each stratum comes from an approximately normal distribution.

Limitations and Robustness: Defining the strata requires pr evious knowledge of the population. Further, determining samples for each stratu m requires knowledge or assumptions about the variance in each stratum. Finally , if strata are chosen poorly, then observations may appear to be outliers when they are actually obser vations from a different stratum.

Directions for a one-sample t-test for a stratifi ed sample are contained in Box 3-7, with an example in Box 3-8.

3.2.1.4 The Chen Test



## Purpose: Test for a difference between a population mean and a fixed threshold when the underlying distribution is right-skewe d, i.e., the right-ta il of the dist ribution is longer than the left-tail. The standard one-sample t -test works best when the underl ying distribution is normal or at least symmetric. Often times though, environmen tal data has an underlyi ng distribution that is right-skewed. Therefore, the Chen test may be more appropriate when most of the data values are relatively small, but there are al so a few relatively large values.

Data: A simple or systematic random sample, x 1 

## , ... 

## , x n 

## , from the population of interest.

### EPA QA/G-9S 53 February 2006 

## Assumptions: A simple or systematic random sample, x 1 

## , ... 

## , x n 

## , from the population of interest which is believed to be right-skewed. This can be verified by inspecti on of a histogram of the data or a sample skewness that is greater than one.

### EPA QA/G-9S 54 February 2006 Box 3-7: Directions for a One-Sample t -Test for a Stratified Random Sample

COMPUTATIONS: Calculate the stratum weights, 

# ∑

= L

h h h h V V = W 1 , where V h is the surface area of stratum h
multiplied by the depth of sampling in stratum h . For each stratum, calculate the sample stratum mean and the sample stratum standard deviation,

h h i n
hi X n X h

=

= 

# ∑ 1

1 and 

# ( )

. X X n = s h hi n

i h h h 2

1 2 1 1

−

− 

# ∑

=
Compute the overall sample mean and the sample variance of the mean, 

# ∑

= = L

h h h st X W X

1 and 

# ∑

= = L

h h n h st n s W s

1 2

2

.

Finally, calculate the approximate degrees of freedom, 

## ( ) 

# ∑

=

− = L

h h h h

h st

n n s

W s df

1 2 4 4 4

1 (round up to the next integer).

STEP 1. Null Hypothesis: C : H 0 = μ

STEP 2. Alternative Hypothesis: i) C : H A > μ (upper-tail test) ii) C : H A < μ (lower-tail test) iii) C : H A ≠ μ (two-tail test)

STEP 3. Test Statistic:
st st
s X t C 0 −

=

STEP 4. a) Critical Value: Use Table A-2 t find: i) α

− 1

, df t ii) α

− − 1

, df t iii) 2 1

, α

− df t

STEP 4. b) p-value: Use Table A-2 to find: i) 

## ( ) 0 P t

t df > ii) 

## ( ) 0 P t

t df < iii) 

# ( ) 0 P 2 t t df > ⋅

STEP 5. a) Conclusion: i) If t 0 > α

− 1

, df t , then reject the null hypothesis that the true population mean is equal to the threshold C. ii) If t 0 < α

− − 1

, df t , then reject the null hypothesis. iii) If 2 / 1

, 0 α

− > df

t t , then reject the null hypothesis.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the true population mean is equal to the threshold C.

STEP 6. If the null hypothesis was not rejected, there is only one false acceptance error rate ( β at μ 1 ), and if 

# ( )

( ) 2 C 2 1 2 1 2 1 1 2

α β α
μ ′

− − ′

− +

− + ≥ z z z s n st , then the sample size was probably large enough to achieve the DQOs.

The value of α ′ is α for a one-sided test and 2 / α for a two-sided test.

### EPA QA/G-9S 55 February 2006 Box 3-8: An Example of a One-Sample t -Test for a Stratified Random Sample
Consider a stratified random sample consisting of two strata where stratum 1 comprises 25% of the total site surface area and stratum 2 comprises the other 75%. Suppose 40 samples were collected from stratum 1, and 60 samples were collected from stratum 2. This information will be used to test the null hypothesis that the overall site mean is 40 ppm versus the lower-tail alternative. The decision maker has specified a 1% false rejection decision limit ( α ) at 40 ppm and a 20% false acceptance decision error limit ( β ) at 35 ppm ( μ 1 ).

COMPUTATIONS: The stratum weights are W 1 = 0.25, W 2 = 0.75. For stratum 1, the sample mean is 31 ppm and the sample standard deviation is 18.2 ppm. For stratum 2, the sample mean is 35 ppm, and the sample standard deviation is 20.5 ppm.

The sample overall mean concentration is 34 35 75

. 0 31 25

. 0

1 = ⋅ + ⋅ = = 

# ∑

= L

h h h st X W X and the sample variance

of the mean is

46

. 4 60 5

. 20 75

. 0 40 2

. 18 25

. 0 2 2 2 2

1 2 2 2

= ⋅ + ⋅

= = 

# ∑

= L

h h h h st n s W s

.

The approximate degrees of freedom is:



## ( ) 74 7

. 73

59 60 5

. 20 75

. 0
39 40 2

. 18 25

. 0 46

. 4

1 2 4 4

2 4 4 2

1 2 4 4 4
→ =

⋅ ⋅ +

⋅ ⋅ =

− = 

# ∑

= L

h h h h

h st

n n s

W s df

.

STEP 1. Null Hypothesis: 0 4 : H 0 = μ

STEP 2. Alternative Hypothesis: 0 4 : H A < μ (lower-tail test)

STEP 3. Test Statistic: 841

. 2 46

. 4 40 34 C 0 − = −
= −

= st st
s X t

STEP 4. a) Critical Value: Using Table A-2, 378

. 2 99

. 0

, 74 1

, − = − = −

− t

t df α .
STEP 4. b) p-value: Using Table A-2, p-value < 0.005. (the exact value is 

## ( ) ( ) 0029

. 0 841

. 2 P P 74 0 = − < = < t

t t df ).

STEP 5. a) Conclusion: Since test statistic = -2.841 < -2.378 = critical value, we reject the null hypothesis that the true population mean is equal to 40.

STEP 5. b) Conclusion: Since p-value = 0.0029 < 0.01 = significance level, we reject the null hypothesis that the true population mean is equal to 40.



## Limitations and Robustness: Chen’s test is a gene ralization of the one-sample t -test. Like the t

test, this Chen’s test can have some difficulties in dealing with non-detects, especially if there are a large number of them. For a moderate amount of non-detects, a substitution method (e.g.,1⁄2 of the detection limit) will suffice if the threshold level C is much larger than the detection limit, otherwise refer to the methods discussed in Chapter 4.

### EPA QA/G-9S 56 February 2006 

## Directions for the Chen test are contained in Box 3-9, with an example in Box 3-10.

Box 3-9: Directions for the Chen Test
COMPUTATIONS: Visually check the assumption of right-sk ewness by inspecting a histogram. If at most 15% of the data points are below the detection limit (DL) and C is much larger than the DL, then replace values below the DL with DL/2. Compute the mean, X , and the standard deviation, s of the data set. Then compute



## ( ) 

## ( ) 

# ( ) b n

n ns X X i
i n

=

− − −

= 

# ∑
1 2 3 3

1 (sample skewness), a b

n = 6 , and t X C

s n 0 = −

.

NOTE: The sample skewness should be greater than 1 to indicate the underlying distribution is right-skewed.

STEP 1. Null Hypothesis: C : H 0 = μ

STEP 2. Alternative Hypothesis: i) C : H A > μ (upper-tail test) ii) C : H A < μ (lower-tail test) iii) C : H A ≠ μ (two-tail test)

STEP 3. Test Statistic: 

# ( ) 

# ( ) T t a t at t

=+ + + + 0 0 2 2 0 0 3 12 4 2

STEP 4. a) Critical Value: Use Table A-1to find: i) α

− 1 z ii) α z iii) 2 1 α

− z

STEP 4. b) p-value: Use Table A-1 to find: i) 

## ( ) T Z > P ii) 

## ( ) T Z < P iii) 

# ( ) T Z > ⋅ P 2

STEP 5. a) Conclusion: i) If T > α

− 1 z , then reject the null hypothesis that the true population mean is equal to the threshold value C. ii) If T < α

z , then reject the null hypothesis. iii) If 2 1 /

α

− > z T , then reject the null hypothesis.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the true population mean is equal to the threshold value C.

### EPA QA/G-9S 57 February 2006 Box 3-10: Example of the Chen Test
Consider the following sample of a contam inant concentration measurements (in ppm):

2.0, 2.0, 5.0, 5.2, 5.9, 6.6, 7.4, 7.4, 9.7, 9.7, 10.2, 11.5, 12.4, 12.7, 14.1, 15.2, 17.7, 18.9, 22.8, 28.6, 30.5, 35.5.

We want to test the null hypothesis that the mean μ is less than 10 ppm versus the alternative that it exceeds 10 ppm. A significance level of 0.05 is to be used.

COMPUTATIONS: A histogram of the 22 data points indicates a right-skewed distribution. It is found that
227

. 13 = X ppm and s = 9.173 ppm. Also,



## ( ) 078

. 1 227

. 13 173

. 9 20 21 22 22

1 3 3 = −

⋅ ⋅ = 

# ∑

= i i X b

, 0383

. 0 22 6 078

. 1

= = a , and 650

. 1

22 173

. 9 10

227

. 13 0 = −
= −
=

n s C X t

.

A skewness value of 1.078 indicates the data are right-skewed.

STEP 1. Null Hypothesis: H 0 : μ ≤ 10

STEP 2. Alternative Hypothesis: H A : μ > 10 (upper-tail test)

STEP 3. Test Statistic: 

# ( ) ( ) 96

. 1 2 4 2 1 3 0 0 2 2 0 0 = + ⋅ + + ⋅ + = t t a t a t T

.
STEP 4. a) Critical Value: Using Table A-1, α

− 1 z = 95

. 0 z = 1.645.

STEP 4. b) p-value: Using Table A-1, 

## ( ) 0250

. 0 9750

. 0 1 96

. 1 P = − = > Z

. STEP 5. a) Conclusion: Since the test statistic = 1.96 > 1.645 = the critical value, we reject the null hypothesis.

STEP 5. b) Conclusion: Since the p-value = 0.0250 < 0.05 = the significance level, we reject the null hypothesis.



## 3.2.1.5 Land’s Method for Lognormally Distributed Data



## Purpose: Estimate the mean of a lognormally dist ributed population using confidence limits. Another alternative is to log-tr ansform the data to make it approximately normal, then use the t confidence interval described in Box 3-2, and fi nally transforming the c onfidence bounds back to the original scale. This method produces a bi ased interval estimate and should be avoided, unless a median is being estimated in which case this method produces an unbiased estimate.

Data: A simple or systematic random sample, x 1 

## , ... 

## , x n 

## , from the population of interest. The sample may or may not contain compositing.

Assumptions: The underlying distribution of the data is approximately lognormal.

Limitations and Robustness: Land’s method is extremely sensitive to outliers since the mean and standard deviation are not robus t against outliers unless a preven tative limit on variance is used.

### EPA QA/G-9S 58 February 2006 

## Directions for Land’s Method ar e contained in Box 3-11, with an example in Box 3-12.

Box 3-11: Directions for Computing Confidence Limits for the Population Mean of a Lognormal Distribution Using Land’s Method

COMPUTATIONS: Transform the data: y i = ln x i , i

= 1,

... , n . Next, compute the sample mean, y , and sample
variance, 2 y s , of the transformed data. The values for α

− 1 H and α H come from Table A-17.

An upper one-sided 100(1 - α )% confidence limit for the population mean is ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

− + + − 1 2 exp 1 2

n H s s y y y α

.

A lower one-sided 100 α % confidence limit for population mean is ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

− + + 1 2 exp 2

nH s s y y y α

.

Box 3-12: An Example Using Land’s Method
A random sample of 15 concentrations from a monito ring process (assumed to be lognormal) is reported:

8.12, 7.32, 4.82, 6.52, 7.80, 11.89, 12.94, 7.51, 18.14, 4.09, 5.70, 15.57, 6.68, 8.15, 5.56.

Compute an upper one-sided 95% confidence limit for the population mean of the process.

COMPUTATIONS: The log-transformed data set is:

2.09, 1.99, 1.57, 1.87, 2.05, 2.48, 2.56, 2.02, 2.90, 1.41, 1.74, 2.75, 1.90, 2.10, 1.72

The sample mean and sample standard deviation of the transformed data are 0767

. 2 = y and 4272

. 0 = y s

. The value of H 0.95 is found by interpolation in Table A-17.

An upper one-sided 95% confidence limit for the population mean is

97

. 10 14 9939

. 1 4272

. 0 2 4272

. 0 0767

. 2 exp 1 2 exp 2 95

. 0 2

= ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

⋅ + + = ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

− + + n H s s y y y



## 3.2.1.6 The One-Sample Proportion Test and Confidence Interval



## Purpose: Test for a difference between a population proportion, P , and a fixed threshold ( P 0 

## ) or to estimate the population proportion. If P 0 

## = 0.5, this test is e quivalent to the Sign test.

Data: A simple or systematic random sample, x 1 

## , ... 

## , x n 

## , from the population of interest.

Assumptions: The data constitutes an independent random sample from the population.

Limitations and Robustness: Both nP 0 

## and n (1- P 0 

## ) must be at least 5 to apply the normal approximation. Otherwise, exact tests must be us ed and a statistician should be consulted.

### EPA QA/G-9S 59 February 2006 

## Directions for the one-sample test for proportions are contained in Box 3-13, with an example in Box 3-15. Directions for computing a confid ence interval for a proportion are contained in Box 3-14.

Box 3-13: Directions for the On e-Sample Test for Proportions

COMPUTATIONS: Compute p , the sample proportion of data values that fit the desired characteristic.

STEP 1. Null Hypothesis: 0 0 : H P P =

STEP 2. Alternative Hypothesis: i) 0 A : H P P > (upper-tail test) ii) 0 A : H P P < (lower-tail test) iii) 0 A : H P P ≠ (two-tail test)

STEP 3. Test Statistic: )/n P ( P P c p z
0 0 0 0 1

− − +

= , where ⎩ ⎨ ⎧

< >

−

= 0 0 if / 5

. 0 if / 5

. 0
P p n P p n c

STEP 4. a) Critical Value: Use Table A-1 to find: i) α

− 1 z ii) α z iii) 2 1 α

− z

STEP 4. b) p-value: Use Table A-1 to find: i) 

## ( ) 0 P z

Z > ii) 

## ( ) 0 P z

Z < iii) 

# ( ) 0 P 2 z Z > ⋅

STEP 5. a) Conclusion: i) If z 0 > α

− 1 z , then reject the null hypothesis that the true proportion is equal to the threshold value P 0
. ii) If z 0 < α z , then reject H 0
. iii) If 2 / 1 0 α

− > z z , then reject H 0
.
STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the true proportion is equal to the threshold value P 0
. STEP 6. If the null hypothesis was not rejected, there is only one false acceptance error rate ( β at P 1 ), and if

⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎣ ⎡

− − − ≥ ′ P P ) P ( P z + ) P ( P z n - -
0 1 1 1 1 0 0 1 2 1 1 β α , then the sample size was large enough to achieve the

DQOs. The value of α ′ is α for a one-sided test and 2 / α for a two-sided test.

Box 3-14: Directions for Computing a Conf idence Interval for a Population Proportion
COMPUTATIONS: Compute p , the sample proportion of data values that fit the desired characteristic.

A 100(1 - α )% confidence interval for P is 

## ( )

n p p z p −

⋅ ±

− 1 2 1 α , where Table A-1 is used to find 2 1 α

− z

.

### EPA QA/G-9S 60 February 2006 Box 3-15: An Example of the On e-Sample Test for Proportions
Consider 85 concentration samples of which 11 have ar e greater than the clean-up standard. Test the null hypothesis that the population proportion of concentrations greater than the clean-up standard is 0.2 versus the lower-tailed alternative. The decision maker has specified a 5% false rejection rate ( α ) for P 0 = 0.2, and a false acceptance rate ( β ) of 20% for P 1 = 0.15.

COMPUTATIONS: Since both nP 0 = 85

⋅ 0.2 = 17 and n (1- P 0 ) = 85

⋅ (1-0.2) = 68 are at least 5, the normal approximation can be applied. From the data, the sample proportion is p = 11/85 = 0.1294

STEP 1. Null Hypothesis: H 0 : P ≥ 0.20

STEP 2. Alternative Hypothesis: H A : P < 0.20

STEP 3. Test Statistic: 49 1 85 2 0 1 2 0 2 0 85 5 0 1294 0
1 5 0

0 0 0 0 . )/

. (

. . /

. . = )/n P ( P P /n

. p z

− =

− − +

− − +

=

STEP 4. a) Critical Value: Using Table A-1, α z = 05

. 0 z = -1.645 .

STEP 4. b) p-value: Using Tables A-1, 

## ( ) 0681

. 0 9319

. 0 1 49

. 1 P = − = − < Z

. STEP 5. a) Conclusion: Since the test statistic = -1.49 > -1.645, we fail to reject the null hypothesis that the true population proportion is at least 0.20.

STEP 5. b) Conclusion: Since p-value = 0.0681 > 0.05 = significance level, we fail to reject the null hypothesis that the true population proportion is at least 0.20.

STEP 6. Since the null hypothesis was not rejected and there is only one false acceptance error rate, it is possible to use the sample size formula to determine if the error rate has been satisfied. Since,


## ( ) ( ) ( ) ( ) 53

. 365 2

. 0 15 0 15 0 1 15 0 84

. 0 2 0 1 2 0 64 1 1 1 2

0 1 1 1 1 0 0 1 2

=

. . . . . . P P P P z + P P z n - - ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎣ ⎡

− − +

−

= ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎣ ⎡

− − − < ′ β α

,

the sample size was not large enough to achieve the DQOs. So the null hypothesis was not rejected, but the false acceptance error rate was not satisfied. Therefore, there is insufficient evidence that the proportion is less than 0.20, but this conclusion is uncertain because the sample size was too small.



## 3.2.2 Nonparametric Methods


## These methods rely on the relative rankings of data values. Knowledge of the precise form of the population dist ribution is not necessary.

3.2.2.1 The Sign Test



## Purpose: Test for a difference between the population median, and a fixed threshold.

Data: A simple or systematic random sample, x 1 

## , ... 

## , x n 

## , from the population of interest. The sample may or may not contain compositing.

### EPA QA/G-9S 61 February 2006 

## Assumptions: The Sign test can be used for any underlying population distribution.

Limitations and Robustness : The Sign test has less power than the one-sample t -test or the Wilcoxon Signed Rank test. However, the Sign test makes no distributional assumptions like the other two tests and it can handl e non-detects if the detection limit is below the threshold.

Directions for conducting a sign te st are contained in Box 3-16, with an example in Box 3-17.

3.2.2.2 The Wilcoxon Signed Rank Test



## Purpose: Test for a difference between the true lo cation (mean or median) of a population and a fixed threshold. If the underl ying population distribut ion is approximately normal, then the onesample t -test will have more power (chance of rejec ting the null hypothesis when it is false) than the Wilcoxon Signed Rank test. For symmetric distributions, the Wilcoxon Signed Rank test will have more power than the Sign test. If the sample size is small and the data is not approximately symmetric nor normally distribu ted, then the Sign test should be used.

Data: A simple or systematic random sample, x 1 

## , ... 

## , x n 

## , from the population of interest.

Assumptions : The data set comes from an a pproximately symmetric distribution.

Limitations and Robustness For large sample sizes ( n > 50), the one-sample t -test is more robust against violations of its a ssumptions than the Wilcoxon Signed Rank test. The Wilcoxon signed rank test may produce misleading results if ther e are many tied data values. If possible, measurements should be recorded with sufficient accuracy so that a large number of tied values do not occur. Estimated concentrations should be reported for data below the detection limit, even if these estimates are negative, as their rela tive magnitude is of importance. If this is not possible, substitute the value DL/2 for each value below the detection limit providing all the data have the same detection limit. When different detection limits are present, all data could be censored at the highest detection li mit but this will substantially weaken the test. A statistician should be consulted on the poten tial use of Gehan ranking.

Directions for the Wilcoxon signed rank test are contained in Box 3-18, with an example in Box 3-19.

### EPA QA/G-9S 62 February 2006 Box 3-16: Directions for the Sign Test (One-Sample)

COMPUTATIONS: Let C be the threshold. Compute the deviations C

− = i i x d . If any of the deviations are zero delete them and correspondingly reduce the sample size. Finally, compute B , the number of times d i > 0.
STEP 1: Null Hypothesis: H 0 : median = C
STEP 2: Alternative Hypothesis: i) H A : median > C (upper-tail test) ii) H A : median < C (lower-tail test) iii) H A : median ≠ C (two-tail test)

STEP 3: Test Statistic: If n ≤ 20, then the test statistic is B

. If n > 20, then the test statistic is 4 / 2

/ 0 n n B z −
=

. STEP 4 a): Critical Value: If n ≤ 20, then use Table A-18 to find i) B upper ( n , 2 α ) ii) B lower ( n , 2 α ) - 1 iii) B lower ( n , α ) – 1 and B upper ( n , α )

If n > 20, then use Table A-1 to find i) α

− 1 z ii) α z iii) 2 / 1 α

− z
STEP 4 b): p-value: If n ≤ 20, then let 

## ( )

n i n i p 2 1 ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

= and the p-value is

i) 

## ( ) 

# ∑

= n

B i i p

ii) 

## ( ) 

# ∑

= B

i i p 0
iii) 

## () () ⎭ ⎬ ⎫ ⎩ ⎨ ⎧

⋅ 

# ∑ ∑

= = n

B i B

i i p i p , min 2 0 If n > 20, then use Table A-1 to find i) 

## ( ) 0 P z

Z > ii) 

## ( ) 0 P z

Z < iii) 

# ( ) 0 P 2 z Z > ⋅ STEP 5 a): Conclusion: If n ≤ 20, then i) If B ≥ B upper ( n , 2 α ), then reject the null hypothesis that the true population median is equal to the threshold value C. ii) If B ≤ B lower ( n , 2 α ) - 1, then reject the null hypothesis. iii) If B ≥ B upper ( n , α ) or B ≤ B lower ( n , α ) - 1, then reject the null hypothesis.

If n > 20, then i) If z 0 > α

− 1 z , then reject the null hypothesis that the true population median is equal to the threshold value C. ii) If z 0 < α z , then reject the null hypothesis. iii) If | z 0 | > 2 / 1 α

− z , then reject the null hypothesis.

STEP 5 b): Conclusion: If p-value < α , then reject the null hypothesis that the true population median is equal to the threshold value C.

### EPA QA/G-9S 63 February 2006 Box 3-17: An Example of the Sign Test (One-Sample)
The following 10 data points (in ppb) will be used to test the null hypothesis that the population median is at least 1000 ppb versus the lower-tail alternative. The decision maker has specified a 10% false rejection error rate ( α ) at 1000 ppb (C), and a 20% false acceptance error rate ( β ) at 900 ppb ( 1 ~ μ ).

COMPUTATIONS: The table below displays the data values and the deviations:

x

i 974 1044 1093 897 879 1161 839 824 796 <750 (DL) d

i -26 44 93 -103 -121 161 -161 -176 -204 -625

Therefore, B = number of d i > 0 = 3.

STEP 1: Null Hypothesis: H 0 : median ≥ 1000 ppb

STEP 2: Alternative Hypothesis: H A : median < 1000 ppb

STEP 3: Test Statistic: Since n ≤ 20, the test statistic is B = 3.

STEP 4 a): Critical Value: Since n ≤ 20, Table A-18 is used to find B lower ( n , 2 α ) - 1 = 2.

STEP 4 b): p-value: Since n ≤ 20, p-value = 1719

. 0 2 1 10 3

0 10 = ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ 

# ∑

= i i .

STEP 5 a): Conclusion: Since test statistic = 3 > 2 = critical value, we fail to reject the null hypothesis that the true median is at least 1000 ppb.

STEP 5 b): Conclusion: Since p-value = 0.1719 > 0.10 = significance level, we fail to reject the null hypothesis that the true median is at least 1000 ppb.



## 3.3 COMPARING TWO POPULATIONS



## The two-sample methods of this secti on concern the comparison of two population parameters (means, medians, or proportions). During Step 1: Revi ew DQOs and Sampling Design, the population parameters were specifie d. In environmental applications, the two populations to be compared may be a potentially contaminated area with a background area or concentration levels from an upgradient and a downgradient well. Another example is comparing site concentration leve ls before and after clean-up to test for significant improvement.

The populations of interest can be one of two types, independen t or paired. With independent populations, measurem ents are taken separately from the two groups of sampling units; for example, concentrations from a c ontaminated site and a background site. The observations from paired populat ions are correlated. Here, two measurements are taken upon one set of sampling units at separate instances ; for example, measurements before and after clean-up or two labs making separate measurements on a single set of samples. Parametric and nonparametric methods are described for bot h independent and paired populations.

### EPA QA/G-9S 64 February 2006 Box 3-18: Directions for the Wilcox on Signed Rank Test (One-Sample)
COMPUTATIONS: Let C be the threshold. Compute the deviations d i = x i - C. If any of the deviations are zero, then delete them and correspondingly reduce the sample size. Rank the absolute deviations, | d i |, from smallest to largest. If there are tied obser vations, then assign the average rank. Let R i be the signed rank of | d

i |, where the sign of R i is determined by the sign of d i

. STEP 1. Null Hypothesis: H 0 : location = C
STEP 2. Alternative Hypothesis: i) H A : location > C (upper-tail test) ii) H A : location < C (lower-tail test) iii) H A : location ≠ C (two-tail test)
STEP 3. Test Statistic: If n ≤ 20, then 

### { } 

# ∑

> +
=
0 : i R i i R T , the sum of the positive signed ranks.

If n > 20, then 

## ( ) 

# ( ) + + +

−
= T n n T z
var 4 / 1 0 ,

where 

# ( ) ( )( ) 

# ( ) ∑

= +

− − + +

= g

j j j t t n n n T

1 2 1 48 1 24 1

2 1 var 

## , g is the number of

tied d i groups and t j is the size of group j

. STEP 4. a) Critical Value: If n ≤ 20, then use Table A-7 to find: If n > 20, then use Table A-1 to find: i) n ( n +1)/2 - w α i) z 1- α ii) w α ii) z α iii) w α /2 iii) z 1- α /2

STEP 4. b) p-value: If n ≤ 20, then use Table A-7 to find: If n > 20, then use Table A-1 to find: i) 

## ( ) 

# ( ) +

− + ≤ T n n W 2 / 1 P i) 

# ( ) 0 

## P z Z >
ii) 

# ( ) + ≤ T W P ii) 

# ( ) 0 

## P z Z <
iii) 

## ( ) 

# ( ) ( ) { } + + ≤

− + ≤

⋅ T W T n n W P

, 2 / 1 P min 2 iii) 2 P(Z>| 0 

## z |)

STEP 5. a) Conclusion: If n ≤ 20, then: If n > 20, then:

i) If + T ≥ n ( n +1)/2 - w α

, then reject the null i) If z 0 > z 1- α , then reject the null ii) If + T ≤ w α

, then reject the null hypothesis. ii) If z 0 < z α

, then reject the null iii) If + T ≥ n ( n +1)/2 - w α /2 or + T ≤ w α

, then reject the null iii) If | z 0 | > z 1- α /2 , then reject the null

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the true population location is equal to the threshold C.

STEP 6. If the null hypothesis was not rejected and n>20, then the sample size necessary to achieve the DQOs assuming only one false acceptance error rate ( β

at μ 1 ) has been specified is 

# ( ) ( ) ( ) ⎪ ⎭ ⎪ ⎬ ⎫
⎪ ⎩ ⎪ ⎨ ⎧

+

− +

⋅ ≥ ′

− − ′

− +

2 C var 16

. 1 2 1 2 1 2 1 1 α β α
μ z z z T n

.

If this is true then the false acceptance error rate has probably been satisfied (the value of α ′ is α for a one-sided test and 2 / α for a two-sided test).

### EPA QA/G-9S 65 February 2006 Box 3-19: An Example of the Wilcox on Signed Rank Test (One-Sample)
The following 10 data points (in ppb) will be used to test the null hypothesis that the population median is at least 1000 ppb versus the lower-tail alternative. The decision maker has specified a 10% false rejection error rate ( α ) at 1000 ppb (C), and a 20% false acceptance error rate ( β ) at 900 ppb).

974, 1044, 1093, 897, 879, 1161, 839, 824, 796, <750 (detection limit)

COMPUTATIONS: For this example, the only option is to assign the value 375 ppb (DL/2) to the nondetect. The table below displays the remaining computations.

x

i 974 1044 1093 897 879 1161 839 824 796 375 d

i -26 44 93 -103 -121 161 -161 -176 -204 -625 | d

i | 26 44 93 103 121 161 161 176 204 625 rank 1 2 3 4 5 6.5 6.5 8 9 10

R

i -1 2 3 -4 -5 6.5 -6.5 -8 -9 -10

STEP 1. Null Hypothesis: H 0 : median ≥ 1000 ppb

STEP 2. Alternative Hypothesis: H A : median < 1000 ppb

STEP 3. Test Statistic: Since n ≤ 20, compute 

### { } 5

. 11 5

. 6 3 2

0 : = + + = = 

# ∑

> +

i R i i R T

STEP 4. a) Critical Value: Since n ≤ 20, Table A-7 is used to find w 0.10 = 14.

STEP 4. b) p-value: Since n ≤ 20, Table A-7 is used to find the p-value is between 0.05 and 0.075. Using software, the p-value is found to be 0.0527.

STEP 5. a) Conclusion: Since test statistic = + T = 11.5 ≤ 14 = critical value, we reject the null hypothesis that the true population median is at least 1000.

STEP 5. b) Conclusion: Since p-value = 0.0527 < 0.10 = significance level, we reject the null hypothesis that the true population median is at least 1000.

NOTE: The Sign test failed to reject the null hypothesis for this example. Recall that the Wilcoxon Signed Rank test has more power than the Sign test if the distribution of the population is symmetric.



## The hypothesis tests in this sec tion may be used to determin e if there is evidence that θ 1 

## – θ 2 

## < 

## δ 0 

## , θ 1 

## – θ 2 

## > 

## δ 0 

## , or θ 1 

## – θ 2 

## ≠ 

## δ 0 

## , where θ 1 

## and θ 2 

## represent population means, medians, or proportions and δ 0 

## represents the threshold value. Also, there are confidence interval procedures to estimate θ 1 

## – θ 2 . 

## Section 3.3.1.1 covers parametric methods for comparing two independent populations, while Section 3.3.1.2 covers para metric methods for comparing two paired populations. Section 3.3.2 describes nonparametr ic counterparts to the parametric methods.

3.3.1 Parametric Methods


## These methods rely on the knowing the specific distribution of the populations or of the statistics of interest.

### EPA QA/G-9S 66 February 2006 

## 3.3.1.1 Independent Samples

3.3.1.1.1 The Two-Sample t -Test and Confidence Interval (Equal Variances)



## Purpose: Test for a difference or estimate the difference between two population means when it can be assumed the population variances are approximately equal.

Data: A simple or systematic random sample x 1 

## , x 2 

## , . . . , x m 

## from one population, and an independent simple or systematic random sample y 1 

## , y 2 

## , . . . , y n 

## from the second population.

Assumptions : The two populations are independent. If not, then it is possible that a paired method could be used. Both are approximately norm ally distributed or the sample sizes are large ( m and n both at least 30). If th is is not the case, then a nonparametric procedure is an alternative. Finally, the varian ces of both populations are approxima tely equal. If the population variances are not equal (tests ar e available in Section 4.5), then use the methods of the next section.

Limitations and Robustness: The two-sample t -test with equal variance s is robust to moderate violations of the assumption of normality, but not to large inequalities of variances. An alternative is the parametric methods for unequal variances described in th e next section. The t

test is not robust against outliers because sample means and standard deviations are sensitive to outliers.

Directions for the two-sample t-test with equa l variances are contained in Box 3-20, with an example in Box 3-22. Directions for a two-samp le confidence interval w ith equal variances are contained in Box 3-21.

3.3.1.1.2 The Two-Sample t -Test and Confidence Interval (Unequal Variances)



## Purpose: Test for a difference or estimate the difference between two population means when it is suspected the population variances are not equal.

Data: A simple or systematic random sample x 1 

## , x 2 

## , . . . , x m 

## from the one population, and an independent simple or systematic random sample y 1 

## , y 2 

## , . . . , y n 

## from the second population.

Assumptions: The two populations are independent. If not, then it is possible that a paired method could be used. Both are approximately norm ally distributed or the sample sizes are large ( m and n both at least 30). If th is is not the case, then a nonparametric procedure is an alternative.

Limitations and Robustness: The two-sample t -test with unequal variance s is robust to moderate violations of the assumption of normality. The t -test is also not robust to outliers because sample means and standard deviations are sensitive to outliers.

### EPA QA/G-9S 67 February 2006 

## Directions for the two-sample t-test with unequa l variances are contained in Box 3-23, with an example in Box 3-25. Directions for a two-samp le confidence interval with unequal variances are contained in Box 3-24.

Box 3-20: Directions for the Two-Sample t -Test (Equal Variances)

COMPUTATIONS: Calculate the sample means, X and Y , and the sample variances, s X 2 and s Y 2 of the two

populations. Also compute the pooled standard deviation estimate, 

## ( ) ()

2 1 1 2 2

− + ⋅ − + ⋅ −

= n m s n s m s y X p

STEP 1. Null Hypothesis: 0 0 : H δ μ μ = − Y X

STEP 2. Alternative Hypothesis: i) 0 A : H δ μ μ >

− Y X (upper-tail test) ii) 0 A : H δ μ μ <

− Y X (lower-tail test) iii) 0 A : H δ μ μ ≠

− Y X (two-tail test)

STEP 3. Test Statistic: 

# ( )

n m s Y X t

p 1 1 0 0

+ − −

= δ

STEP 4. a) Critical Value: Use Table A-2 to find: i) α

− − + 1

, 2 n m t ii) α

− − + − 1

, 2 n m t iii) 2 1

, 2 α

− − + n m t

STEP 4. b) p-value: Use Table A-2 to find: i) 

## ( ) 0 2 P t t n m >

− + ii) 

## ( ) 0 2 P t t n m <

− + iii) 

# ( ) 0 2 P 2 t t n m > ⋅

− +

STEP 5. a) Conclusion: i) If t 0 > α

− − + 1

, 2 n m t , then reject the null hypothesis that the true difference between population means is equal to the threshold δ 0

. ii) If t 0 < α

− − + − 1

, 2 n m t , then reject the null hypothesis. iii) If 2 / 1

, 2 0 α

− − + > n m t t , then reject the null hypothesis.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the true difference between population means is equal to the threshold δ 0

. STEP 6. If the null hypothesis was not rejected, there is only one false acceptance error rate ( β at δ 1 ), and both m

and n are at least 

# ( ) ( ) 4 2 2 1 2 0 1 2 1 1 2

α β α δ δ ′

− − ′

− +

− + z z z s p , then the sample sizes were probably large enough to

achieve the DQOs. The value of α ′ is α for a one-sided test and 2 / α for a two-sided test.

### EPA QA/G-9S 68 February 2006 Box 3-21: Directions for a Two-Sample t Confidence Interval (Equal Variances)

COMPUTATIONS: Calculate the sample means, X and Y , and the sample variances, s X 2 and s Y 2 of the two

populations. Also compute the pooled standard deviation estimate, 

## ( ) ()

2 1 1 2 2

− + ⋅ − + ⋅ −

= n m s n s m s y X p .

A 100(1 - α )% confidence interval for Y X μ μ

− is 

# ( )

n m s t Y X p n m 1 1 2 1

, 2 + ⋅ ±

−

− − + α , where Table A-2 is used to

find 2 1

, 2 α

− − + n m t

.

Box 3-22: An Example of a Two-Sample t -Test (Equal Variances)
At a hazardous waste site, an area cleaned using an in-situ methodology (area 1) was compared with a similar, but relatively uncontaminated reference area (area 2). If t he in-situ methodology worked, then the average contaminant levels at the two sites should be approximately equal. If the methodology did not work, then area 1 should have a higher average than the reference area. Suppose 7 rand om samples were taken from area 1, and 8 were taken from area 2. Methods described in Section 4.5 were used to determine that the variances were essentially equal. The false rejection error rate was set at 5% and the false acceptance error rate was set at 20% ( β ) if the difference between the areas is 2.5 ppm ( δ 1 ).

COMPUTATIONS: The sample me ans and sample variances are 8

. 7 1 = X

, 6

. 6 2 = X

, 1

. 2 2 1 = s , and 2

. 2 2 2 = s

.
The pooled standard deviation is: 

## ( ) ( ) 4676 1 2 8 7 2 2 1 8 1 2 1 7

. . +

. s p =

− + ⋅ − ⋅ −
=

STEP 1. Null Hypothesis: 0 : H 2 1 0 = − μ μ

STEP 2. Alternative Hypothesis: 0 : H 2 1 A >

− μ μ

STEP 3. Test Statistic: 

# ( ) 5799 1

8 1
7 1 4676 1 0 6 6 8 7
1 1 0

2 1 0 .

. .

. =

n m s X X t

p =

+ − −

+ − −

= δ

STEP 4. a) Critical Value: Using Table A-2, 771

. 1 95

. 0

, 13 1

, 2 = =

− − + t t n m α STEP 4. b) p-value: Using Table A-2, 0.05 < p-value < 0.10. (The exact 

## ( ) ( ) 0.0691 5799

. 1 P P value

- p 13 0 2 = > = > =

− + t t t n m )

STEP 5. a) Conclusion: Since test statistic = 1.5799 < 1.771 = critical value, we fail to reject the null hypothesis of no difference between population means.
STEP 5. b) Conclusion: Since p-value = 0.0691 > 0.05 = significance level, we fail to reject the null hypothesis of no difference between population means.

STEP 6. Since the null hypothesis was not rejected and there is only one false acceptance error rate, we can compute the sample sizes necessary to achieve the DQOs. Each sample size must be at least 

## ( ) ( ) 94

. 4 645

. 1 25

. 0 0 5

. 2 842

. 0 645

. 1 4676

. 1 2 2 2 2 2

= ⋅ +

− + ⋅

.

Since m and n are both greater than 4.94, the false acceptance error rate has probably been satisfied.

### EPA QA/G-9S 69 February 2006 Box 3-23: Directions for the Two-Sample t -Test (Unequal Variances)

COMPUTATIONS: Calculate the sample means, X and Y , and the sample variances, s X 2 and s Y 2 of the two populations. Also, compute the degrees of freedom (Satterthwaite’s approximation) for the test:



## ( ) ( ) 1 1 2 2

2 2 2 2 2

− +

− ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

+

=

n n s

m m s n s

m s

df

Y X Y

X
, rounded down to the next integer.

STEP 1. Null Hypothesis: 0 0 : H δ μ μ = − Y X

STEP 2. Alternative Hypothesis: i) 0 A : H δ μ μ >

− Y X (upper-tail test) ii) 0 A : H δ μ μ <

− Y X (lower-tail test) iii) 0 A : H δ μ μ ≠

− Y X (two-tail test)

STEP 3. Test Statistic: 

# ( )

n s

m s Y X t

Y X 2 2 0 0

+ − −
= δ

STEP 4. a) Critical Value: Use Table A-2 to find: i) α

− 1

, df t ii) α

− − 1

, df t iii) 2 1

, α

− df t

STEP 4. b) p-value: Use Table A-2 to find: i) 

## ( ) 0 P t t df > ii) 

## ( ) 0 P t t df < iii) 

# ( ) 0 P 2 t t df > ⋅

STEP 5. a) Conclusion: i) If t 0 > α

− 1

, df t , then reject the null hypothesis that the difference between the population means is equal to δ 0

. ii) If t 0 < α

− − 1

, df t , then reject H 0

. iii) If 2 / 1

, 0 α

− > df

t t , then reject H 0

.
STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the difference between the population means is equal to δ 0

. STEP 6. If the null hypothesis was not rejected, then there is no simple method to determine if the sample sizes were large enough to achieve the DQOs.

### EPA QA/G-9S 70 February 2006 Box 3-24: Directions for a Two-Sample t Confidence Interval (Unequal Variances)
COMPUTATIONS: Calculate the sample means, X and Y , and the sample variances, s X 2 and s Y 2 of the two populations. Also, compute the degrees of freedom (Satte rthwaite’s approximation) for the confidence interval:



## ( ) ( ) 1 1 2 4

2 4 2 2 2

− +

− ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

+

=

n n s

m m s n s

m s

df

Y X Y

X
, rounded down to the next integer.

A 100(1 - α )% confidence interval for Y X μ μ

− is 

# ( )

n s

m s t Y X Y X df 2 2

2 1

, + ⋅ ±

−

− α , where Table A-2 is used to find

2 1

, 2 α

− − + n m t

.

Box 3-25: An Example of the Two-Sample t -Test (Unequal Variances)
At a hazardous waste site, an area cleaned using a new methodology (area 1) was compared with a similar area cleaned with the standard technology (area 2). If the new methodology worked, then the two sites should be approximately equal in average contaminant levels. If the new methodology did not work, then area 1 should have a higher average than the reference area. Suppos e 7 random samples were taken from area 1 and 8 were taken from area 2. As the contaminant concentrations in the two areas are supposedly equal, we will test no difference in population means versus the upper-tail alternative. Using Section 4.5, it was determined that the variances of the two populations were not equal, therefore using Satterthwaite's method is appropriate. The false rejection error rate was set at 5% and the false acceptance error rate was set at 20% ( β ) if the difference between the areas is 2.5 ppm.

COMPUTATIONS: The sample me ans and sample variances are 2

. 9 1 = X

, 1

. 6 2 = X

, 3

. 1 2 1 = s , and 7

. 5 2 2 = s

. Satterthwaite’s approximation of the degrees of freedom for the test is:



## ( ) ( ) ( ) ( ) 10 3

. 10

1 8 8 7

. 5
1 7 7 3

. 1 8 7

. 5
7 3

. 1

1 1 2 2

2 2 2

2 4

2 4 2 2 2

→ =

− +

− ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ +

=

− +

− ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

+

=

n n s

m m s n s

m s

df

Y X Y

X

STEP 1. Null Hypothesis: 0 : H 2 1 0 = − μ μ
STEP 2. Alternative Hypothesis: 0 : H 2 1 A >

− μ μ

STEP 3. Test Statistic: 

# ( ) 271

. 3

8 7

. 5
7 3

. 1 0 1 6 2

. 9

2 2 2 1 0 2 1 0 =

+ − −

+ − −

= . =

n s

m s X X t δ

STEP 4. a) Critical Value: Using Table A-2, 812

. 1 95

. 0

, 10 1

, = =

− t t df α STEP 4. b) p-value: Using Table A-2, p-value < 0.005. (The exact 

## ( ) ( ) 0.0042 271

. 3 P P value

- p 10 0 = > = > = t t t df )

STEP 5. a) Conclusion: Since test statistic = 3.271 > 1.812 = critical value, we reject the null hypothesis that there is no difference between the population means.
STEP 5. b) Conclusion: Since p-value = 0.0042 < 0.05 = significance level, we reject the null hypothesis that there is no difference between the population means.

### EPA QA/G-9S 71 February 2006 

## 3.3.1.1.3 Two-Sample Test for Population Proportions



## Purpose : Test for a difference between two population proportions, P 1 

## and P 2 

## . A simple or systematic random sample, x 1 

## , ... 

## , x n 

## , from one population and an independent simple or systematic random sample, y 1 

## , y 2 

## , . . . , y n 

## , from second population of interest.

Assumptions: The data constitutes independent simple or systematic random samples from the populations.

Limitations and Robustness: To ensure the normal approx imation is appropriate, compute mp 1 

## , m (1- p 1 

## ), np 2 

## , and n (1- p 2 

## ), where m and n are the sample sizes and p 1 

## and p 2 

## are the sample proportions. If all of the products are at least 5, then the norma l approximation may be used. Otherwise, seek the assistance from a statistici an as exact tests must be used. Since data positioning is used rather than actual data va lues, the procedures are robust to outliers.

Directions for the two-sample te st for proportions are contained in Box 3-26, with an example in Box 3-28. Directions for a two-sample c onfidence interval for the difference between proportions are contained in Box 3-27.

3.3.1.2 Paired Samples



## Observations from paired populations are corre lated. The general set up for this teat involves taken two measurements upon one group of sampling units at separate instances; for example, measurements before and after clean-up , or two labs making separate measurements on a single set of objects.

3.3.1.2.1 The Paired t -Test and Confidence Interval



## Purpose: Test for or estimate the differen ce between two paired population means.

Data: Two paired data sets x 1 

## , ... 

## , x n 

## and y 1 

## , ... 

## , y n 

## .
Assumptions: The two data sets come from approximately normal distributions or n ≥ 

## 30.

Limitations and Robustness : Since there is really only one sample, the limitations for the paired
t -test are the same as those for the one-sample t -test. These methods are robust against the population distribution deviating moderately fr om normality. However, they are not robust against outliers. In either case, the nonparametric methods of section 3.3.2.2 offer an alternative. Finally, these methods have diffi culty dealing with non-detects. The substitution or adjustment procedures of chapter 4 are an alternative, but it is best to use a nonparametric method.

Directions for the paired t-test are given in Box 3-29, with an example in Box 3-31. Directions for a paired t confidence inte rval are given in Box 3-30.

### EPA QA/G-9S 72 February 2006 Box 3-26: Directions for a Tw o-Sample Test for Proportions

COMPUTATIONS: Let p i , i = 1,2, denote the sample proportion of data values from population i that fit the characteristic of interest. Also, calculate the pooled sample proportion p , which is the total number of data values that fit the characteristic divided by the total sample size, n m +

. To ensure the normal approximation is appropriate, compute mp 1 , m (1- p 1 ), np 2 , n (1- p 2 ). If all of these values are at least 5, then the normal approximation may be used. Otherwise, seek assistance from a statistician.

STEP 1. Null Hypothesis: 0 2 1 0 : H δ = − P P

STEP 2. Alternative Hypothesis: i) 0 2 1 A : H δ >

− P P (upper-tail test) ii) 0 2 1 A : H δ <

− P P (lower-tail test) iii) 0 2 1 A : H δ ≠

− P P (two-tail test)

STEP 3. Test Statistic: 

## ( ) ( ) . 1 1 1 0 2 1 0 ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ + ⋅ − −

−

=

n m p p p

p z δ

STEP 4. a) Critical Value: Use Table A-1 to find: i) α

− 1 z ii) α z iii) 2 1 α

− z

STEP 4. b) p-value: Use Table A-1 to find: i) 

## ( ) 0 P z

Z > ii) 

## ( ) 0 P z

Z < iii) 

# ( ) 0 P 2 z Z > ⋅

STEP 5. a) Conclusion: i) If z 0 > α

− 1 z , then reject the null hypothesis that the difference in population proportions is equal to δ 0

. ii) If z 0 < α z , then reject the null hypothesis. iii) If 2 / 1 0 α

− > z z , then reject the null hypothesis.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the difference in population proportions is equal to δ 0

. STEP 6. If the null hypothesis was not rejected, there is only one false acceptance error rate ( β at P 1 - P 2 ), and both m and n are at least 

# ( ) ( ) 2 1 2 2 1 1 1 2

P P ) p ( p z z

-

-

− − ⋅ + ′ β α where 2 2 1 p p p +

= ,

then the sample sizes were probably large enough to achieve the DQOs. The value of α ′ is α for a one-sided test and 2 / α for a two-sided test. If only one of m or n meets the sample size criterion, then use statistical software to calculate the power of the test, assuming that the true values for the proportions P 1 and P 2 are those obtained in the sample. If the estimated power is below 1- β , the false acceptance error rate ha s not been satisfied.

### EPA QA/G-9S 73 February 2006 Box 3-27: Directions for Comput ing a Confidence Interval for the Difference Between Population Proportions
COMPUTATIONS: Let p i , i = 1,2, denote the sample proportion of data values from population i that fit the characteristic of interest. Also, calculate the pooled sample proportion p , which is the total number of data values that fit the characteristic divided by the total sample size, n m +

. To ensure the normal approximation is appropriate, compute mp 1 , m (1- p 1 ), np 2 , n (1- p 2 ). If all of these values are at least 5, then the normal approximation may be used. Otherwise, seek the assistance from a statistician as exact tests must be used.
A 100(1 - α )% confidence interval for the difference between population proportions, P 1 – P 2 , is 

## ( ) ( ) ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ +

− ⋅ ±

−

− n m p p z p p 1 1 1 2 1 2 1 α , where Table A-1 is used to find 2 1 α

− z

Box 3-28: An Example of a Two-Sample Test for Proportions
At a hazardous waste site, investigators must determine whether an area suspected to be contaminated with dioxin needs to be remediated. The possibly contaminat ed area (area 1) will be compared to a reference area (area 2) to see if dioxin levels in area 1 are greater than dioxin levels in the reference area. An inexpensive surrogate probe was used to determine if each individual sample is either "contaminated," i.e., over the health standard of 1 ppb, or "clean." The decision maker is willing to accept a false rejection decision error rate of 10% ( α ) and a false-negative decision error rate of 5% ( β ) when the difference in proportions between areas exceeds 0.1. A team collected 92 readings from area 1 (of which 12 were contaminated) and 80 from area 2, the reference area, (of which 10 were contaminated).
COMPUTATIONS: The sample proportion for area 1 is p 1 = 12/92 = 0.130, the sample proportion for area 2 is p 2 = 10/80 = 0.125, and the pooled sample proportion is p = (12 + 10) / (92 + 80 ) = 0.128. Since mp 1 = 12,

m (1- p 1 ) = 80, np 2 = 10, n (1- p 2 ) = 70 are all at least 5, the normal approximation is appropriate.
STEP 1. Null Hypothesis: 0 : H 2 1 0 = − P P
STEP 2. Alternative Hypothesis: 0 : H 2 1 A >

− P P (upper-tail test)

STEP 3. Test Statistic: 

## ( ) ( ) . 10 0

80 1 92 1 128

. 0 1 128

. 0 125

. 0 13

. 0

1 1 1 2 1 0
.

n m p p p p z = ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ +

− −
= ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ +

− −
=

STEP 4. a) Critical Value: Using Table A-1, 282

. 1 90

. 0 1 = =

− z z α STEP 4. b) p-value: Using Table A-1, 

## ( ) ( ) 4602

. 0 5398

. 0 1 10

. 0 P P 0 = − = > = > Z

z Z
STEP 5. a) Conclusion: Since test statistic = 0.10 < 1.282 = critical value, we fail to reject the null hypothesis of no difference between population proportions.
STEP 5. b) Conclusion: Since p-value = 0.4602 > 0.10 = significance level, we fail to reject the null hypothesis of no difference between population proportions. STEP 6. Since the null hypothesis was not rejected and there is only one false acceptance error rate ( β

= 0.05 at a difference of P 1 - P 2 = 0.1) has been specified, it is possible to calculate the sample sizes that achieve the DQOs. So m and n must be of size at least 

## ( ) ( ) 6

. 190 1

. 0 1275

. 0 1 1275

. 0 645

. 1 282

. 1 2
2 2

= − +

,

since 1275

. 0 2 125

. 0 13

. 0

= +

= P . As both m and n are less than 190.6, the false acceptance error
rate has not been satisfied. Therefore, the null hypothesis was not rejected, but the samples sizes were not large enough to ensure adequate power in the test.

### EPA QA/G-9S 74 February 2006 Box 3-29: Directions for the Paired t -Test

COMPUTATIONS: Compute the differences d i = y i - x i for i = 1,

... , n and the sample mean, D , and sample standard deviation, s D , of the differences.

STEP 1. Null Hypothesis: 0 0 : H δ μ =

D

STEP 2. Alternative Hypothesis: i) 0 A : H δ μ >

D (upper-tail test) ii) 0 A : H δ μ <

D (lower-tail test) iii) 0 A : H δ μ ≠

D (two-tail test)

STEP 3. Test Statistic:

n s D t D 0 0 δ

−

=

STEP 4. a) Critical Value: Use Table A-2 to find: i) α

− − 1

1
, n t ii) α

− − − 1 1
, n t iii) 2 1 1 α

− − , n t

STEP 4. b) p-value: Use Table A-2 to find: i) 

## ( ) 0 1 P t t n >

− ii) 

## ( ) 0 1 P t

t n <

− iii) 

# ( ) 0 1 P 2 t t n > ⋅

−
STEP 5. a) Conclusion: i) If t 0 > α

− − 1

1
, n t , then reject the null hypothesis that there is no difference between the population means. ii) If t 0 < α

− − − 1 1
, n t , then reject the null hypothesis. iii) If 2 / 1

, 1 0 α

− − > n

t t , then reject the null hypothesis.

STEP 5. b) Conclusion: If p-value < α , then reject that the difference between the population means is δ 0

. STEP 6. If the null hypothesis was not rejected, there is only one false acceptance error rate ( β at μ 1 ), and 

# ( )

( ) 2 C 2 1 2 1 2 1 1 2

α β α
μ ′

− − ′

− +

− + ≥ z z z s n D , then the sample size was probably large enough to achieve the DQOs.

The value of α ′ is α for a one-sided test and 2 / α for a two-sided test.

Box 3-30: Directions fo r Computing the Paired t Confidence Interval

COMPUTATIONS: Compute the differences d i = y i - x i for i = 1,

... , n and the sample mean, D , and sample standard deviation, s D , of the differences.

A 100(1 - α )% confidence interval for μ D is n s t D D n ⋅ ±

− − 2

1

, 1 α , where Table A-2 is used to find 2 1

, 1 α

− − n t



## .

### EPA QA/G-9S 75 February 2006 Box 3-31: An Example of the Paired t -Test
Consider the following 9 pairs of data points (in ppb):

x

i 178 52 161 245 164 184 157 308 130

y

i 92 67 62 206 106 126 108 314 126

This data will be used to test the null hypothesis that there is no difference in means versus the alternative that the mean of the first population is greater than the mean of the second population. The decision maker has specified a 5% false rejection error rate and a 10% false acceptance error rate at a difference of 50 ppb.

COMPUTATIONS: The table below displays the computations.

d

i -86 15 -99 -39 -58 -58 -49 6 -4

The sample mean of the differences is -41.33 and the sample standard deviation is 39.92.

STEP 1. Null Hypothesis: 0 : H 0 = D μ

STEP 2. Alternative Hypothesis: 0 : H A < D μ (lower-tail test)

STEP 3. Test Statistic: 11

. 3

9 92

. 39 0 33

. 41 0 0 − = − −
= −

=

n

s D t D δ

.

STEP 4. a) Critical Value: Using Table A-2, 860

. 1 95

. 0

, 8 1

, 1 − = − = −

− − t t n α .
STEP 4. b) p-value: Using Table A-2, 0.005 < p-value < 0.01. (The exact 

## ( ) ( ) 0072

. 0 11

. 3 P P value

- p 8 0 1 = − < = < =

− t t t n ).

STEP 5. a) Conclusion: Since test statistic = 3.11 < 1.860 = critical value, we reject the null hypothesis of no difference between population means.

STEP 5. b) Conclusion: Since p-value = 0.0072 < 0.05 = significance level, we reject the null hypothesis of no difference between population means.



## 3.3.2 Nonparametric Methods


## These methods rely on the relative rankings of data values. Knowledge of the precise form of the population distri butions is not necessary.

3.3.2.1 Independent Samples

3.3.2.1.1 The Wilcoxon Rank Sum Test



## Purpose: Test for a difference between two popul ation means. The Wilcoxon Rank Sum test applied with the Quantile test, provides a power ful combination for detecting true differences between two population distributions.

### EPA QA/G-9S 76 February 2006 

## Data: A random sample x 1 

## , x 2 

## , . . . , x m 

## from one population, and an independent random sample y 1 

## , y 2 

## , . . . , y n 

## from the second population.

Assumptions: The validity of the random sampling and independence assumptions should be verified by review of the pro cedures used to select the sa mpling points. The two underlying distributions are assumed to have approximately the same shape (variance) and that the only difference between them is a shift in location. A qualitative test of this assumption can be done by comparing histograms.

Limitations and Robustness: The Wilcoxon signed rank test may produce misleading results if there are many tied data values. When many ties ar e present, their relati ve ranks are the same, and this has the effect of diluti ng the statistical power of the Wilcoxon test. If possible, results should be recorded with sufficient accuracy so that a large number of tied values do not occur. Estimated concentrations should be reported for data below the detection limit, even if these estimates are negative, as their relative magnitude to the rest of the data is of importance. If this is not possible, substitute the value DL/2 fo r each value below the detection limit providing all the data have the same detection limit. When di fferent detection limits are present, all data could be censored at the highest detection limit but this will substantially weaken the test. A statistician should be consulted on the potential use of Gehan ranking.

Directions for the Wilcoxon Rank Sum test are give n in Box 3-32, with an example in Box 3-33. Directions for the large sample approximati on for the Wilcoxon Ranked Sum test are given in the example in Box 3-34.

3.3.2.1.2 The Quantile Test



## Purpose: Test for a shift to the right in the right -tail of population 1 vers us population 2 This may be regarded as being equivalent to detectin g if the values in the right-tail of population 1 distribution are generally larger th an the values in the right-tail of the population 2 distribution.

Data: A simple or systematic random sample, x 1 

## , x 2 

## , . . ., x n 

## , from the site population and an independent simple or systematic random sample, y 1 

## , y 2 

## , . . ., y m 

## , from the background population.

Assumptions: The validity of the random sampling and independence assumptions is assured by using proper randomization procedures, which can be verified by reviewing the procedures used to select the sampling points.

Limitations and Robustness : Since the Quantile test focuses on the right-tail, large outliers will bias results. Also, the Quantile test says not hing about the center of the two distributions. Therefore, this test should be used in combination with a location test like the t -test (if the data are normally distributed) or the Wilcoxon Rank Sum test.

### EPA QA/G-9S 77 February 2006 Box 3-32: Directions for th e Wilcoxon Rank Sum Test
COMPUTATIONS: Rank the pooled data from smallest to largest assigning average rank to ties. Sum the ranks of the first population and denote this by R 1 . Then compute 

## ( )

2 1 1 0 +

− = m m R W
STEP 1. Null Hypothesis: 0 : H 0 = − Y X μ μ (no difference between population means)

STEP 2. Alternative Hypothesis: i) 0 : H A >

− Y X μ μ (upper-tail test) ii) 0 : H A <

− Y X μ μ (lower-tail test) iii) 0 : H A ≠

− Y X μ μ (two-tail test)

STEP 3. Test Statistic: If m and n are at most 20, then the test statistic is W 0

. If m and n are greater than 20, then compute 

## ( ) 0 0 0 var 2 /
W mn W z −

= ,

where 

## ( ) ( ) ( )( ) 

# ( ) ⎪ ⎭ ⎪ ⎬ ⎫
⎪ ⎩ ⎪ ⎨ ⎧

−

− + + − + +

= 

# ∑

= g

j j j t t n m n m mn n m mn W

1 2

0 1 1 12 12 1 var 

## ,
g is the number of tied groups, and t j is the number of ties in the j th group. STEP 4. a) Critical Value: If m and n< = 20, use Table A-8 to find: If m and n >20 use Table A-1 to find: i) mn - w α i) z 1- α ii) w α ii) z α iii) w α /2 iii) z 1- α /2 STEP 4. b) p-value: If m and n <=20, use Table A-8 to find: If m and n > 20 use Table A-1 to find: i) 

## ( ) 0 P W mn W

rs −

< i) 

## ( ) 0 P z

Z > ii) 

## ( ) 0 P W W

rs < ii) 

## ( ) 0 P z

Z < iii) 

## ( ) ( ) { } 0 0 P

, P min 2 W W W mn W rs

rs <

− < ⋅ iii) 

# ( ) 0 P 2 z Z > ⋅
STEP 5. a) Conclusion: If m and n are at most 20, then i) If W 0 ≥ mn - w α , then reject the null hypothesis. ii) If W 0 ≤ w α , then reject the null hypothesis. iii) If W 0 ≥ mn - w α /2 or W 0 ≤ w α /2 , then reject the null hypothesis.

If m and n are greater than 20, then i) If z 0 > z 1- α , then reject the null hypothesis of no difference between population means. ii) If z 0 < z α

, then reject the null hypothesis. iii) If 2 / 1 0 α

− > z z , then reject the null hypothesis.

STEP 5. b) Conclusion: If p-value < significance level, then reject the null hypothesis
STEP 6. If the null hypothesis was not rejected, then the sample sizes necessary to achieve the DQOs should be computed. If the sample sizes are large, only one false acceptance error rate ( β

at δ 1 ) has been specified, then the false acceptance error rate has probably been satisfied if both m and n are at least 

## ( ) 

# ( )
⎪ ⎭ ⎪ ⎬ ⎫
⎪ ⎩ ⎪ ⎨ ⎧

+ + ⋅ ⋅

⋅ ′

− − ′

− 4 var 2 16

. 1 2 1 2 1 2 1 1 0 α β α δ z z z W

.

NOTE: The value of α ′ is α for a one-sided test and 2 / α for a two-sided test. The large sample normal approximation is adequate as long as min( m , n ) > 10.

### EPA QA/G-9S 78 February 2006 Box 3-33: An Example of th e Wilcoxon Rank Sum Test
At a hazardous waste site, an area cleaned (area 1) was compared with a relatively uncontaminated reference area (area 2). If the methodology worked, then the two sites should be approximately equal in average contaminant levels. If the methodology did not work, then area 1 should have a higher average than the reference area. The false rejection error rate was set at 10% ( α ) and the false acceptance error rate was set at 20% ( β ) if the difference between the areas is 2.5 ppb. Seven random samples were taken from area 1 and 8 samples were taken from area 2:

Area 1: 17, 23, 26, 5, 13, 13, 12 Area 2: 16, 20, 5, 4, 8, 10, 7, 3

COMPUTATIONS: The ordered pooled data and their ranks are (Area 1 denoted by *):

Pooled

data 3 4 5* 5 7 8 10 12* 13* 13* 16 17* 20 23* 26*

Rank 1 2 3.5* 3.5 5 6 7 8* 9.5* 9.5* 11 12* 13 14* 15*

The sum of the ranks of area 1 is R 1 = 3.5 + 8 + 9.5 + 9.5 + 12 + 14 + 15 = 71.5 and W 0 = 71.5 - 7(7 + 1)/2 = 43.5

STEP 1. Null Hypothesis: 0 : H 0 = − Y X μ μ

STEP 2. Alternative Hypothesis: 0 : H A >

− Y X μ μ

STEP 3. Test Statistic: Since m and n are less than 20, the test statistic is W 0

. STEP 4. a) Critical Value: Using Table A-8, mn - w 0.10 = 56 – 16 = 40

STEP 4. b) p-value: Using Table A-8, mn - w 0.05 < W 0 < mn - w 0.025 , which implies 0.05 < p-value < 0.025. Using statistical software, p-value = 0.0410.

STEP 5. a) Conclusion: Since Test Statistic = 43.5 > 40 = Critical Value, we reject the null hypothesis of no difference between population means.

STEP 5. b) Conclusion: Since p-value = 0.0410 < 0.10 (significance level), we reject the null hypothesis



## Directions for the Quantile te st are contained in Box 3-35, with an example in Box 3-36.

3.3.2.1.3 The Slippage Test



## Purpose: Test for a shift to the right in the extr eme right-tail of population 1 (site) versus population 2 (background). This is e quivalent to asking if a set of the largest values of the site distribution are larger than the maximu m value of the background distribution.

Data: A simple or systematic random sample, x 1 

## , x 2 

## , . . ., x m 

## , from the site population and an independent simple or systematic random sample, y 1 

## , y 2 

## , . . ., y n 

## , from the background population.

### EPA QA/G-9S 79 February 2006 Box 3-34: A Large Sample Example of the Wilcoxon Rank Sum Test
Arsenic concentrations (in ppm) from a site are to be compar ed to a reference area. The null hypothesis is that the means of the two areas are equal versus the upper-tail alternative. The false rejection error rate was set at 5% ( α ) and the false acceptance error rate was set at 20% ( β ) if the difference between the areas is 2.5 ppm.

Site concentrations ( m = 22): 11.2, 11.3, 12.2, 13.2, 14.2, 15.9, 16.3, 17.1, 18.6, 19.2, 21.5, 22.3, 22.4, 22.7, 22.8, 23.3, 24.1, 25.8, 30.2, 30.7, 31.4, 37.1

background concentrations ( n = 21): 6.1, 8.5, 11.1, 11.3, 12.6, 12.8, 13.6, 15.0, 15.2, 15.3, 16.1, 16.2, 17.0, 17.1, 17.6, 19.2, 19.2, 19.6, 21.1, 22.2, 25.0
COMPUTATIONS: The ordered pooled data and their rank s are (site concentrations are denoted by *):

Value Rank Value Rank Value Rank Value Rank

6.1 1 14.2* 12* 17.6 23 22.7* 34* 8.5 2 15.0 13 18.6* 24* 22.8* 35* 11.1 3 15.2 14 19.2 26 23.3* 36* 11.2* 4* 15.3 15 19.2 26 24.1* 37* 11.3 5.5 15.9* 16* 19.2* 26* 25.0 38 11.3* 5.5* 16.1 17 19.6 28 25.8* 39* 12.2* 7* 16.2 18 21.1 29 30.2* 40* 12.6 8 16.3* 19* 21. 5* 30* 30.7* 41* 12.8 9 17.0 20 22.2 31 31.4* 42* 13.2* 10* 17.1 21.5 22.3* 32* 37.1* 43* 13.6 11 17.1* 21.5* 22.4* 33*

The sum of the site ranks is

R 1 = 4 + 5.5 + 7 + 10 + 12 + 16 + 19 + 21.5 + 24 + 26 + 30 + 32 + 33 + 34 + 35 + 36 + 37 + 39 + 40 + 41 + 42 + 43 = 587

and W 0 = 587 – 22(22 + 1)/2 = 334.

STEP 1. Null Hypothesis: 0 : H 0 = − Y X μ μ

STEP 2. Alternative Hypothesis: 0 : H A >

− Y X μ μ

STEP 3. Test Statistic: Since m and n are greater than 20, the test statistic is 

## ( ) 0 0 0 var 2 /
W mn W z −

= . To

compute var( W 0 ), we need to determine the number of tied groups and the number of values in each group. There are 3 tied groups, so g = 3. The number of tied values in the groups are 2 (at 11.3), 2 (at 17.1), and 3 (at 19.2). Therefore,


## ( ) ( ) ( )( ) 

# ( ) ( )( ) 

# [ ] 23

. 1693 1 3 3 1 2 2 1 2 2 1 21 22 21 22 12 21 22 12 1 21 22 21 22 var 2 2 2 0 = ⎭ ⎬ ⎫ ⎩ ⎨ ⎧

− ⋅ +

− ⋅ +

− ⋅

− + + ⋅ ⋅

− + + ⋅ ⋅

= W



## .

Finally, 50

. 2 23

. 1693 2 / 21 22 334 0 = ⋅ −
= z

.

STEP 4. a) Critical Value: Using Table A-1, z 0.95 = 1.645.

STEP 4. b) p-value: Using Table A-1, 

## ( ) 0062

. 0 9938

. 0 1 50

. 2 P = − = > Z

. STEP 5. a) Conclusion: Since Test Statistic = 2.50 > 1.645 = Critical Value, we reject the null hypothesis of no difference between population means.

STEP 5. b) Conclusion: Since p-value = 0.0062 < 0.05 = significance level, we reject the null hypothesis of no difference between population means.

### EPA QA/G-9S 80 February 2006 Box 3-35: Directions for the Quantile Test COMPUTATIONS: Select a quantile, 0.5 ≤ b 1 < 1. Rank the pooled data from smallest to largest. Find the number of pooled data points larger than the th 1 b quantile, 

## ( ) { } 1 1 floor 1 − ⋅ − +

− + = b n m n m c , where floor means to calculate the value and discard all decimals.
STEP 1. Null Hypothesis: H 0 : The right-tails of the two population distributions are the same. STEP 2. Alternative Hypothesis: H A : The right-tail of the distribution of Population 1 (site) is shifted to the right, i.e., the values in the right-tail of the site distribution are larger than the values in the right-tail of the background distribution.

STEP 3. Test Statistic: s = number of site samples greater than the th 1 b quantile.
STEP 4. a) Critical Value: If m and n are both at most 20, then use Table A-19 to find q α . STEP 4. b) p-value: If m and n are both at least 15, then use Table A-1 to find

⎟ ⎠ ⎞ ⎜ ⎝ ⎛

− − <

− σ μ 5

. 0 P 1 s Z , where n m c n +

⋅ = μ and 1 1

− + ⋅ ⎟ ⎠ ⎞ ⎜ ⎝ ⎛

+

− ⋅ + ⋅ = n m m

n m c

n m c n σ

STEP 5. a) Conclusion: If s ≥ q α , then reject the null hypothesis that the right tails of the two population distributions are the same. STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the right tails of the two population distributions are the same.

Box 3-36: A Example of the Quantile Test At a hazardous waste site a new, cheaper, in-site methodology was compared against an existing methodology by remediating separate areas of the site using each method. It will be assumed that the new methodology works as well as the old and we will test for evidence that the new method leaves higher concentrations. A Quantile Test with a significance level of 0.05 will be used to make this determination based on 12 samples from the area remediated using the new methodology and 7 samples from the area remediated using the standard methodology. New Methodology: 7, 18, 2, 4, 6, 11, 5, 9, 10, 2, 3, 3 Standard Methodology: 17, 8, 20, 4, 6, 5, 4 COMPUTATIONS: The 0.75 quantile is selected. The ranked pooled data set is:
2*, 2*, 3*, 3*, 4, 4, 4*, 5*, 5, 6, 6*, 7*, 8, 9*, 10*, 11*, 17, 18*, 20, where * denoted samples from the new methodology portion of the site. The number of values larger than the 0.75 quantile is: 

## ( ) { } ( ) { } 5 1 75

. 0 1 12 7 floor 12 7 1 1 floor 1 = − ⋅ − +

− + = − ⋅ − +

− + = b n m n m c
STEP 1. Null Hypothesis: H 0 : The right-tails of the two population distributions are the same. STEP 2. Alternative Hypothesis: H A : The right-tail of the distribution of Population 1 (site) is shifted to the right, i.e., the values in the right-tail of the site distribution are larger than the values in the right-tail of the background distribution.

STEP 3. Test Statistic: s = number of site samples greater than the 0.75 quantile = 3 STEP 4. a) Critical Value: Since m and n are both less than 20, we use Table A-19 to find q 0.10 = 5 STEP 4. b) p-value: Since m and n are both less than 15, we can’t use the normal approximation for the p-value. STEP 5. a) Conclusion: Since s = 3 < 5 = q 0.10 , we fail to reject the null hypothesis that the right-tails of the two-population distributions are the same.

### EPA QA/G-9S 81 February 2006 

## Assumptions: The validity of the random sampling and independence assumptions is assured by using proper randomization procedures, which can be verified by reviewing the procedures used to select the sampling points.

Limitations and Robustness: Since the Slippage test focuses on the right-tail, large outliers will bias results. Also, the Slippa ge test says nothing about the ce nter of the two distributions. Therefore, this test should be used in combination with a location test like the t -test (if the data are normally distributed) or the Wilcoxon Rank Sum test.

Directions for the slippage te st are contained in Box 3-37, with an example in Box 3-38.

Box 3-37: Directions for the Slippage Test

COMPUTATIONS: Let x 1 ,..., x m be the site data and y 1 ,..., y n be the background data. Order the samples separately.

STEP 1. Null Hypothesis: H 0 : The right-tails of the two population distributions are the same.

STEP 2. Alternative Hypothesis: H A : The extreme right-tail of the distribution of Population 1 (site) is shifted to the right, i.e., the largest values of the site distribution are larger than the largest values of the background distribution.

STEP 3. Test Statistic: s = number of site values greater than the maximum background value.

STEP 4. a) Critical Value: Use Table A-14 to find S α . Note that α can be 0.01, 0.05, or 0.10.

STEP 4. b) p-value: 

## ( ) ( ) ( ) ( ) ( ) ( ) 

# ∑ ∑ −

= = − − − +

⋅ + ⋅

− =

− − − +

⋅ + ⋅ 1

0 ! ! 1 ! ! 1 ! !

1 ! ! s

i m

s i i m i n m

n m m n i m i n m

n m m n

Where “n!’ = nx(n-1)x(n-2)x(n-3)x...3x2x1

STEP 5. a) Conclusion: If s ≥ S α , then reject the null hypothesis that the right-tails of the two population distributions are the same.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the right-tails of the two population distributions are the same.



## 3.3.2.2 Paired Samples



## Recall that the observations from paired populat ions are correlated. The general setting involves taken two measurements upon one group of sampling units at separate instances; for example, measurements before and after clean-up or two labs making separate measurements on a single set of objects.

3.3.2.2.1 The Sign Test



## Purpose : Test for a difference between the medians of two paired populations. This test is very similar to the one sample versi on presented in section 3.2.2.1.

### EPA QA/G-9S 82 February 2006 Box 3-38: A Example of the Slippage Test
At a hazardous waste site a new, cheaper, in-site methodology was compared against an existing methodology by remediating separate areas of the site using each method. It will be assumed that the new methodology works as well as the old and we will test for evidence that the new method leaves higher concentrations. The Slippage Test with a significance level of 0.05 will be used to make this determination based on 12 samples from the area remediated using the new methodology and 7 samples from the area remediated using the standard methodology.

New Methodology: 7, 18, 2, 4, 6, 11, 5, 9, 10, 2, 3, 3 Standard Methodology: 17, 8, 20, 4, 6, 5, 4

COMPUTATIONS: The ordered samples are:

New Methodology: 2, 2, 3, 3, 4, 5, 6, 7, 9, 10, 11, 18 Standard Methodology: 4, 4, 5, 6, 8, 17, 20

STEP 1. Null Hypothesis: H 0 : The right-tails of the two population distributions are the same.

STEP 2. Alternative Hypothesis: H A : The extreme right-tail of the distribution of Population 1 (site) is shifted to the right, i.e., the largest values of the site distribution are larger than the largest values of the background distribution.

STEP 3. Test Statistic: s = number of site values greater than the maximum background value = 0.

STEP 4. a) Critical Value: Using Table A-14, S 0.05 = 6.

STEP 4. b) p-value: 

## ( ) ( ) ( ) 1 0 1 ! !

1 ! ! 1 1 0

0 = − =

− − − +

⋅ + ⋅

− 

# ∑ −

= i i m i n m

n m m n

STEP 5. a) Conclusion: Since test statistic = 0 < 6 = S 0.05 , we fail to reject the null hypothesis that the right-tails of the two population distributions are the same.

STEP 5. b) Conclusion: Since p-value = 1 > 0.05 = significance level, we fail to reject the null hypothesis that the right-tails of the two population distributions are the

same.



## Data: Two paired data sets x 1 

## , ... 

## , x n 

## and y 1 

## , ... 

## , y n 

## selected randomly or systematically.

Assumptions : The Sign test can be used no matter wh at the underlying distributions may be.

Limitations and Robustness: The Sign test has less power than the two-sample t -test or the Wilcoxon Signed Rank Test. However, the Sign te st makes no distributional assumptions like the other two tests and it can ha ndle non-detects (if the detection limit is below the threshold).

Directions for the paired populat ions sign test are contained in Box 3-39, with an example in Box 3-40.

3.3.2.2.2 The Wilcoxon Signed Rank Test



## Purpose: Test for a difference between the locat ions (means or medians) of two paired populations. This test is very similar to the one sample version presented in Section 3.2.2.2.

### EPA QA/G-9S 83 February 2006 Box 3-39: Directions for th e Sign Test (Paired Samples)
COMPUTATIONS: Compute the deviations d i = x i - y i . If any of the deviations are zero delete them and correspondingly reduce the samp le size. Finally, compute B = number of differences greater than zero.

STEP 1. Null Hypothesis: H 0 : median 1 = median 2

STEP 2. Alternative Hypothesis: i) H A : median 1 > median 2 (upper-tail test) ii) H A : median1 < median 2 (lower-tail test) iii) H A : median 1 

## ≠ median 2 (two-tail test)

STEP 3. Test Statistic: If n ≤ 20, then the test statistic is B

.

If n > 20, then the test statistic is 4 / 2

/ 0 n n B z −
=

.

STEP 4. a) Critical Value: If n ≤ 20, then use Table A-18 to find: If n > 20, then use Table A-1 to find: i) B upper ( n , 2 α ) i) z 1- α ii) B lower ( n , 2 α ) – 1 ii) z α iii) B lower ( n , α ) and B lower ( n , α ) iii) z 1- α /2

STEP 4. b) p-value: If n ≤ 20, then let 

# ( )

n 

## i n i p 2 1 ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

= where 

## ! )! ( !
i i n n i n

− = ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

i) 

## ( ) 

# ∑

= n

B i i p

ii) 

## ( ) 

# ∑

= B

o i i p

iii) 

## () () ⎪ ⎭ ⎪ ⎬ ⎫
⎪ ⎩ ⎪ ⎨ ⎧

⋅ 

# ∑ ∑

= = n

B i B

i i p i p , min 2

0

If n > 20, then use Table A-1 to find: i) 

## ( ) 0 P z

Z > ii) 

## ( ) 0 P z

Z < iii) 

# ( ) 0 P 2 z Z > ⋅

STEP 5. a) Conclusion: If n ≤ 20, then i) If B ≥ B upper ( n , 2 α ), then reject the null hypothesis that the two population medians are equal. ii) If B ≤ B lower ( n , 2 α ) - 1, then reject the null hypothesis. iii) If B ≥ B upper ( n , α ) or B ≤ B lower ( n , α ) - 1, then reject the null hypothesis.

If n > 20, then i) If z 0 > z 1- α , then reject the null hypothesis that the two population medians are equal. ii) If z 0 < z α

, then reject the null hypothesis. iii) If | z 0 | > z 1- α /2 , then reject the null hypothesis.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the two population medians are equal.

### EPA QA/G-9S 84 February 2006 Box 3-40: An Example of the Sign Test (Paired Samples)
Consider the following 9 pairs of data points (in ppb):

x

i 178 52 161 245 164 184 157 308 130 y

i 92 67 62 206 106 126 108 314 126

This data will be used to test the null hypothesis that there is no difference in medians versus the alternative that the median of the first population is greater than the median of the second population. The decision maker has specified a 5% false rejection error rate and a 10% false acceptance error rate at a difference of 50 ppb.

COMPUTATIONS: The table below displays the computations.

d

i 86 -15 99 39 58 58 49 -6 4

Therefore, B = the number of differences greater than zero = 7.

STEP 1. Null Hypothesis: H 0 : 2 1 ~ ~ μ μ =

STEP 2. Alternative Hypothesis: H A : 2 1 ~ ~ μ μ >

STEP 3. Test Statistic: Since n ≤ 20, the test statistic is B = 7.

STEP 4. a) Critical Value: Since n ≤ 20, Table A-18 is used to find B upper ( n , 2 α ) = 8.

STEP 4. b) p-value: Since n ≤ 20, p-value = 0898

. 0 2 1 9 9

7 9 = ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ 

# ∑

= i i .

STEP 5. a) Conclusion: Since test statistic = 7 < 8 = critical value, we fail to reject H 0

. STEP 5. b) Conclusion: Since p-value = 0.0898 > 0.05 = significance level, we fail to reject H 0

.



## Data : Two paired data sets x 1 

## , ... 

## , x n 

## and y 1 

## , ... 

## , y n 

## selected randomly or systematically.

Assumptions: The data sets come from a a pproximately symmetric distributions.

Limitations and Robustness: For large sample sizes ( n > 50), the paired t -test is more robust to violations of its assumptions th an the Wilcoxon signed rank test. Fo r small sample sizes, if the data are not approximately symmetric or norma lly distributed, the sign test should be used.

The Wilcoxon signed rank test may produce misleading results if there are many tied data values. Ties have the effect of diluti ng the statistical power of the Wilcoxon test. If possible, results should be recorded with sufficient accuracy so that a large number of tied values do not occur. Estimated concentrations should be reported for data below the detection limit, even if these estimates are negative, as their relative magnitude to the rest of the data is of importance. If this is not possible, substitute the value DL/2 fo r each value below the detection limit providing all the data have the same detection limit. When di fferent detection limits are present, all data could be censored at the highest detection limit but this will substantially weaken the test. A statistician should be consulted on the potential use of Gehan ranking.

### EPA QA/G-9S 85 February 2006 

## Directions for the paired sample Wilcoxon si gned rank test are contained in Box 3-41, with an example in Box 3-42. Directions for the large sample version of the test are contained in the example in Box 3-43.

Box 3-41: Directions for the Wilcoxon Signed Rank Test (Paired Samples)
COMPUTATIONS: Compute the deviations d i = x i - y i . If any of the deviations are zero delete them and correspondingly reduce th e sample size. Rank the absolute deviations, | d i |, from smallest to largest. If there are tied observations, then assign the average rank. Let R i be the signed rank of | d i |, where the sign of R i is determined by the sign of d i

. STEP 1. Null Hypothesis: H 0 : location 1 = location 2 STEP 2. Alternative Hypothesis: i) H A : location 1 > location 2 (upper-tail test) ii) H A : location 1 < location 2 (lower-tail test) iii) H A : location 1 ≠ location 2 (two-tail test)

STEP 3. Test Statistic: If n ≤ 20, then 

### { } 

# ∑

> +
=
0 : i R i i R T , the sum of the positive signed ranks.

If n > 20, then 

## ( ) 

# ( ) + + +

−
= T n n T z
var 4 / 1 0 ,

where 

# ( ) ( )( ) 

# ( ) ∑

= +

− − + +

= g

j j j t t n n n T

1 2 1 48 1 24 1

2 1 var 

## , g is the number of

tied d i groups and t j is the size of group j

. STEP 4. a) Critical Value: If n ≤ 20, use Table A-7 to find: If n > 20, use Table A-1 to find: i) n ( n +1)/2 - w α i) z 1- α ii) w α ii) z α iii) w α /2 iii) z 1- α /2 STEP 4. b) p-value: If n ≤ 20,use Table A-7 to find: If n > 20, use Table A-1 to find: i) 

## ( ) 

# ( ) +

− + ≤ T n n W 2 / 1 P i) 

## ( ) 0 P z

Z > ii) 

# ( ) + ≤ T W P ii) 

## ( ) 0 P z

Z < iii) 

## ( ) 

# ( ) ( ) { } + + ≤

− + ≤

⋅ T W T n n W P

, 2 / 1 P min 2 iii) 

# ( ) 0 P 2 z Z > ⋅
STEP 5. a) Conclusion: If n ≤ 20, If n > 20, i) If + T ≥ n ( n +1)/2 - w α

, then reject the null i) If z 0 > z 1- α , then reject the null ii) If + T ≤ w α

, then reject the null hypothesis. ii) If z 0 < z α

, then reject the null iii) If + T ≥ n ( n +1)/2 - w α /2 or + T ≤ w α

, then reject the null iii) If | z 0 | > z 1- α /2 , then reject the null
STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the two population locations are equal.
STEP 6. If the null hypothesis was not rejected, then the sample sizes necessary to achieve the DQOs should be computed. If the sample size is large, only one false acceptance error rate ( β

at δ 1 ) has been specified, and 

# ( ) ( )
⎪ ⎭ ⎪ ⎬ ⎫
⎪ ⎩ ⎪ ⎨ ⎧

+ +

⋅ ≥ − − − +

2 var 16

. 1 2 1 2 1 2 1 1 α β α δ z z z T n ,
then the false acceptance error rate has probably been satisfied. The value of α ′ is αfor a onesided test and 2 / α for a two-sided test.

### EPA QA/G-9S 86 February 2006 Box 3-42: An Example of the Wilcoxon Signed Rank Test (Paired Samples)
Consider the following 9 pairs of data points (in ppb):

x

i 178 52 161 245 164 184 157 308 130 y

i 92 67 62 206 106 126 108 314 126
This data will be used to test the null hypothesis that there is no difference in medians versus the alternative that the median of the first population is greater than the median of the second population. The decision maker has specified a 5% false rejection error rate and a 10% false acceptance error rate at a difference of 50 ppb.
COMPUTATIONS: The table below displays the computations.

d

i 86 -15 99 39 58 58 49 -6 4 | d

i | 86 15 99 39 58 58 49 6 4 rank 8 3 9 4 6.5 6.5 5 2 1

R i 8 -3 9 4 6.5 6.5 5 -2 1
STEP 1. Null Hypothesis: H 0 : median1 = median 2
STEP 2. Alternative Hypothesis: H A : median 1< median 2 (lower-tail test)

STEP 3. Test Statistic: Since n ≤ 20, compute 

### { } 40 9 8 5

. 6 5

. 6 5 4 1

0 : = + + + + + + = = 

# ∑

> +

i R i i R T

STEP 4. a) Critical Value: Since n ≤ 20, Table A-7 is used to find n ( n +1)/2 - w 0.05 = 45 – 8 = 37.
STEP 4. b) p-value: Since n ≤ 20, Table A-7 is used to find the approximate p-value of 0.025. Using software, the p-value is found to be 0.0195.
STEP 5. a) Conclusion: Since test statistic = 40 ≥ 37 = critical value, we reject the null hypothesis of equal population medians.
STEP 5. b) Conclusion: Since p-value = 0.0195 < 0.05 = significance level, we reject the null hypothesis of equal population medians.
NOTE: The Sign test failed to reject the null hypothesis for this example. The Wilcoxon Signed Rank test has more power than the Sign test if the distribution of the population is symmetric.



## 3.4 COMPARING SEVERAL POPU LATIONS SIMULTANEOUSLY


## This section describes procedures for comparing several population means simultaneously. The comparison is made betwee n several treatment populations versus a single control population. For example, we can simu ltaneously test for a difference between the concentrations at several site s versus a single background area.

The methods in section compare the several populations while cont rolling the overall significance level. If individual two-sample t -tests are performed at significance level α 

## , then the overall significance level is higher than α 

## . Possible much higher if there are many experimental groups. For example, comparing tw o experimental groups to a control using two two-sample t -tests at significance level 0.05 results in an overall signific ance level of 1-(1- 0.05)(1-0.05) = 0.0975. The tests in this sect ion are more powerful in detecting differences between the experimental groups and the contro l than other multiple comparison methods that compare all possible pairs of population means, e.g., the ANOVA F -test and the Kruskal-Wallis
test.

### EPA QA/G-9S 87 February 2006 Box 3-43: A Large Sample Example of the W ilcoxon Signed Rank Te st (Paired Samples)
A hazardous waste site has recently gone through remediation. To determine if the remediation method was effective, 24 paired samples (before and after clean-up) will be compared. The following data will be used to test the null hypothesis that there is no difference in medians versus the alternative that the median concentration before is greater than the median concentration after clean-up. The decision maker has specified a 5% false rejection error rate and a 10% false acceptance error rate at a difference of 30 ppb.

before: 331, 351, 259, 323, 305, 336, 196, 233, 336, 349, 352, 341, 172, 253, 275, 285, 212, 349, 301, 343, 368, 332, 374, 311 after: 246, 270, 229, 326, 295, 238, 278, 302, 331, 264, 267, 249, 288, 272, 270, 313, 337, 284, 271, 253, 295, 271, 289, 281 COMPUTATIONS: The table below displays the computation of the signed ranks.

d i | d i | rank R i d i | d i | rank R i d i | d i | rank R i 85 85 17.5 17.5 5 5 2.5 2.5 -125 125 24 -24 81 81 14 14 85 85 17.5 17.5 65 65 11 11 30 30 8 8 85 85 17.5 17.5 30 30 8 8 -3 3 1 -1 93 93 21 21 90 90 20 20 10 10 4 4 -117 117 23 -23 73 73 13 13 98 98 22 22 -19 19 5 -5 60 60 10 10 -82 82 15 -15 5 5 2.5 2.5 85 85 17.5 17.5 -69 69 12 -12 -28 28 6 -6 30 30 8 8
STEP 1. Null Hypothesis: H 0 : median 1 = median 2

STEP 2. Alternative Hypothesis: H A : median 1 > median 2 (upper-tail test)

STEP 3. Test Statistic: Since n > 20, compute 

## ( ) 

# ( ) + + +

−
= T n n T z
var 4 / 1 0 . First,



### { } 214 8 14 5

. 17

0 : = + + + = = 

# ∑

> +

L

i R i i R T . To compute 

# ( ) + T var , we need

to identify the number of tied groups and the number of values in each of the tied groups. There are 3 tied groups so g = 3. The number of values in the tied groups are 2 (at 2.5), 3 (at 30), and 4 (at 85). Therefore, 

# ( ) ( )( ) 

# ( ) ( ) ( ) { } 125

. 1223 1 4 4 1 3 3 1 2 2 48 1 24 1 24 2 1 24 24 var 2 2 2

= − ⋅ +

− ⋅ +

− ⋅ − + ⋅ + ⋅

= + T



## .

Finally, 

## ( ) 83

. 1 125

. 1223 4 / 1 24 24 214 0 = + ⋅ −
= z

STEP 4. a) Critical Value: Since n > 20, Table A-1 is used to find z 0.95 = 1.645.

STEP 4. b) p-value: Since n > 20, Table A-1 is used to find 

## ( ) 0336

. 0 9664

. 0 1 P 0 = − = > z Z

.
STEP 5. a) Conclusion: Since test statistic = 1.83 > 1.645 = critical value, we reject the null hypothesis of equal population medians.

STEP 5. b) Conclusion: Since p-value = 0.0336 < 0.05 = significance level, we reject the null hypothesis of equal population medians.



## The Dunnett test of Section 3.4.1.1 is a para metric test that comp ares several treatment means to a control mean. Section 3.4.2.1 pres ents a nonparametric alte rnative to the Dunnett
test.

### EPA QA/G-9S 88 February 2006 

## 3.4.1 Parametric Methods


## These methods rely on the knowing the sp ecific distribution of the populations.

3.4.1.1 Dunnett’s Test



## Purpose: Test simultaneously for a difference betw een several population means and the mean of a control population. A typi cal application would involve comparing different potentially cleaned areas of a hazardous waste site to an uncontaminated reference area.

Data: A set of k -1 independent experimental random samples,

i in i 

## x x , , 1 

## K , i = 1,

... 

## , k -1 and an independent random sample from the control population,

c cn c 

## x x , , 1 

## K . Assumptions: The Dunnett test is si milar to the two-sample t -test so the populations need to be approximately normal or the sample sizes need to be large ( ≥ 

## 30). If this is not the case, then the nonparametric Fligner-Wolfe test can be used. Ho wever, that test simple detects a difference between all population means. Both tests assume that the k populations have equal variances.

Limitations and Robustness: The Dunnett critical values (Tab le A-14) are for the case of equal number of samples in the control and experi mental groups, but are approximately correct provided the number of samples from the investig ated groups are more than half but less than double the size of the control group. Also, Table A-14 is for one-tailed tests only.

Directions for Dunnett’s test are contained in Box 3-44, with an example in Box 3-45.

Box 3-44: Directions for Dunnett’s Test
COMPUTATIONS: Calculate the sample mean, X i , and the sample variance, s i 2 for each of the k -1 populations

along with c X and 2 c s . Also compute 

## ( ) () 

# ∑

−

= − +

− = 1

1 2 2 1 1 SSE k

i i i c c s n s n

STEP 1. Null Hypothesis: H 0 : μ I = μ c , for i = 1,

... , k -1
STEP 2. Alternative Hypotheses: i) H A : At least one μ I > μ c , for i = 1,

... , k -1. ii) H A : At least one μ I < μ c , for i = 1,

... , k -1.

STEP 3. Test Statistic: For each of the k -1 experimental populations, compute:

⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛
+ −

=

c i c i i

n n N-k X

X t 1 1 SSE , where N is the total sample size.

STEP 4. a) Critical Value: Use Table A-15 to determine the critical value T D (

α ) where the degrees of freedom are N

- k

. STEP 4. b) p-value: Too complex for this guidance.
STEP 5. a) Conclusion: For each i , reject the correspondi ng null hypothesis if t i > T D (

α ).
STEP 5. b) Conclusion: Use the critical value approach.

### EPA QA/G-9S 89 February 2006 Box 3-45: An Example of Dunnett’s Test
At a hazardous work site, 6 designated areas previously identified as ‘problems’ have been cleaned. In order for these areas to be admitted to the overall work site abatem ent program these areas should be sown to be the same as the reference area. The means of these areas will be compared to mean of a reference area located on the site using Dunnett’s test. The null hypothesis is no difference between the means of the ‘problem’ areas and the mean of the reference area and the alternative hypotheses are that the ‘problem’ means are greater than the reference area mean. The significance level is 0.05. Summary statistics for the data are given in the table below.

Reference IAK3 ZBF6 3BG5 4GH2 5FF3 6GW4

n i 7 6 5 6 7 8 7

X

i 10.3 11.4 12.2 10.2 11.4 11.9 12.1

s i

2 2.5 2.6 3.3 3.0 3.2 2.6 2.8

n c

/ n i 1.16 1.4 1.16 1 0.875 1

t i 1.18 1.93 0.11 1.22 1.84 2.00

Since all of the sample size ratios fall between 0.5 and 2.0, Dunnett’s test may be used.

COMPUTATIONS: The sample means and sample variances are displayed in rows 2 and 3 above. The t i row is the collection of test statistics that are generated in Step 3 below. The final computation is 

## () () () 4

. 110 8

. 2 1 7 6

. 2 1 6 5

. 2 1 7 MSE = ⋅ − + + ⋅ − + ⋅ − = L .
STEP 1. Null Hypothesis: H 0 : μ I = μ c , for i = 1,

... ,6

STEP 2. Alternative Hypotheses: H A : μ I > μ c , for i = 1,

... ,6

STEP 3. Test Statistic: For each of the 6 ‘problem’ areas, t i was computed. For

example, 18

. 1

7 1 6 1
7 46 4

. 110 3

. 10 4

. 11
1 = ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ + ⋅

− −

= t . The results are displayed in

the 5 th row of the table.

STEP 4. a) Critical Value: Using Table A-15 of Appendix A with N

- k = 46-7 = 39 degrees of freedom, the critical value T D (0.95) = 2.37.

STEP 5. a) Conclusion: Since t i < 2.37 = T D (0.95) for all i , we fail to reject the null hypothesis. We conclude that none of the ‘problem’ areas have contamination levels significantly higher than the reference area. Therefore, these areas may be admitted to the work site abatement program.



## 3.4.2 Nonparametric Methods


## These methods rely on the relative rankings of data values. Knowledge of the precise form of the population distri butions is not necessary.

3.4.2.1 The Fligner-Wolfe Test



## Purpose: Test simultaneously for a difference betw een several population locations (means or medians) and the location of a control popula tion. A typical app lication would involve

### EPA QA/G-9S 90 February 2006 

## comparing different potentially cleaned areas of a hazardous waste site to an uncontaminated reference area. This test is si milar to the Wilcoxon Rank Sum test.

Data: A set of k -1 independent experimental simple or systematic random samples,

i in i 

## x x , , 1 

## K , i = 1,

... 

## , k -1 and an independent simple or systematic random sample from the control population,

c cn c 

## x x , , 1 

## K . Let 

# ∑

−

= 

## = 1

1 * k

i i 

## n N 

## and N = N * 

## + n c 

## .

Assumptions: All data sets come from distributions with similar shapes.

Limitations and Robustness: The alternative hypothesis is quite restrictive as it demands all of the experimental groups have population means that are either at least as large as the control mean (or at most the control mean). They ar e not appropriate tests wh en some experimental group means may be higher than the contro l mean and some might be lower.

Directions for the Fligner-Wolfe test are contai ned in Box 3-46, with an example in Box 3-47.

### EPA QA/G-9S 91 February 2006 Box 3-46: Directions of the Fligner-Wolfe Test

COMPUTATIONS: Rank the N pooled data points from smallest to largest assigning average rank to ties. Let r ij denote the rank of data point X ij . Compute 

# ( )

2 1 * * 1

1 1 0 +

− = 

# ∑∑ −

= = N N r FW k

j n

i ij

i

.

Note that the first term of FW is the sum of the ranks from all experimental groups.

STEP 1. Null Hypothesis: 1

, , 1 for

, location location : H 0 − = = k i c i K
STEP 2. Alternative Hypothesis: i) 1

, , 1 for

, location location : H A − = ≥ k i c i K (at least one strict inequality) ii) 1

, , 1 for

, location location : H 0 − = ≤ k i c i K (at least one strict inequality)

STEP 3. Test Statistic: If min( n c , N * ) ≤ 20, then the test statistic is FW 0

. If min( n c , N * ) > 20, then the test statistic is 

## ( ) 0 * 0 0 var 2 /
FW

N n FW z c −

= , where



## ( ) ( ) ( ) 

# ( ) ∑

= −

− − +

= g

j j j c c t t N N N n N N n FW

1 2 * *

0 1 1 12 12 1 var , g is the number of tied

groups and t j is the number of tied values in the j th group.

STEP 4. a) Critical Value: If min( n c , N * ) ≤ 20, then use Table A-8 to find w α . If min( n c , N * ) > 20, then use Table A-1 to find: i) z 1- α ii) z α STEP 4. b) p-value: If min( n c , N * ) ≤ 20, then use Table A-8 to find: i) 

# ( ) 0 * P FW N n W c

rs −

< ii) 

## ( ) 0 P FW W

rs <

If min( n c , N * ) > 20, then use Table A-1to find: i) 

## ( ) 0 P z

Z > ii) 

## ( ) 0 P z

Z <

STEP 5. a) Conclusion: If min( n c , N * ) ≤ 20, then i) If FW 0 ≥ n c N *
- w α , then reject the null hypothesis of no difference between population locations. ii) If FW 0 ≤ w α , then reject the null hypothesis.

If min( n c , N * ) > 20, then i) If z 0 > z 1- α , then reject the null hypothesis of no difference between population locations. ii) If z 0 < z α

, then reject the null hypothesis.

STEP 5. b) Conclusion: If p-value < significance level, then reject the null hypothesis of no difference between population locations.

NOTE: The large sample normal approximation is adequate as long as min( n c , N * ) > 10.

### EPA QA/G-9S 92 February 2006 Box 3-47: An Example of the Fligner-Wolfe Test
4 contaminated ponds are to be combined with a fifth (reference) pond before further work can commence. If the ponds are approximately equal, the proposed remediation method will be acceptable. The assumption of normality cannot be made but all ponds were produced by the same waste process, they should exhibit similar characteristics. The significance level is 0.05. Data values with ranks in parentheses are given in the table below.

Reference Pond North East South West

10 (1) 15 (3) 20 (8 ) 12 (2) 19 (5.5)
19 (5.5) 25 (14) 26 (15) 16 (4) 20 (8)
20 (8) 32 (19.5) 34 (21.5) 22 (10) 28 (16)

23 (11.5) 39 (23) 43 (24.5) 23 (11.5) 31 (18)

24 (13) 43 (24.5) 52 (28) 34 (21.5) 49 (27)
30 (17)
32 (19.5)
46 (26)

COMPUTATIONS: Note that n c = 8 and N * = 20 since n i = 5 for i = 1,...,4. Compute 

# ( ) 5

. 94 210 5

. 304 2 1 * * 1

1 1 0 = − = +

− = 

# ∑∑ −

= = N N r FW k

j n

i ij

i

.

STEP 1. Null Hypothesis: 

## 4
, , 1 for
, pond pond : H 0 

## K = = 

## i

c i

STEP 2. Alternative Hypothesis: 

## 4
, , 1 for
, pond pond : H A 

## K = ≥ 

## i

c i (at least one strict inequality)

STEP 3. Test Statistic: Since min( n c , N * ) = 8 ≤ 20, then the test statistic is FW 0 = 94.5.

STEP 4. a) Critical Value: Since min( n c , N * ) ≤ 20, Table A-8 is used to find w 0.05 = 47.

STEP 4. b) p-value: Since min( n c , N * ) ≤ 20, using Table A-8 gives a p-value of 

## ( ) ( ) 10

. 0 5

. 65 P 5

. 94 20 8 P > < = − ⋅ < rs rs W W (the exact p-value is 0.2380.)

STEP 5. a) Conclusion: Since min( n c , N * ) ≤ 20, and α w N n FW c − = − ⋅ = < = * 0 47 20 8 113 5

. 94 , we fail to reject the null hypothesis of no difference between ponds.

STEP 5. b) Conclusion: Since p-value = 0.2380 > 0.05 = significance level, we fail to reject the null hypothesis of no difference between population ponds.

### EPA QA/G-9S 93 February 2006 

## CHAPTER 4

STEP 4: VERIFY THE ASSUMPTI ONS OF THE STATISTICAL METHOD

Step 4: Verify the Assump tions of the Statistical Test

• 

### Determine approach for verifying assumptions.

 

### Identify any strong graphical evidence from the preliminary data review.

 

### Review (or develop) the statistical model for the data.

 

### Select the methods for verifying assumptions.

• 

### Perform tests of assumptions.

 

### Adjust for distributional assumption if warranted.

 

### Perform the calculations required for the tests.

• 

### If necessary, determine corrective actions.

 

### Determine whether data transfo rmations will correct the problem.

 

### If data are missing, explore the feasibility of using theoretical justification or of collecting new data.

 

### Consider robust procedures or nonparametric hypothesis tests. 

## THE DATA QUALITY ASSESSMENT PROCESS

Review DQOs and Sampling Design

Conduct Preliminary Data Review

Select the Statistical Method

Verify the Assumptions

Draw Conclusions from the Data VERIFY THE ASSUMPTIONS OF THE STATISTICAL TEST

Purpose

Examine the underlying assumptions of the statistical method in light of the environmental data.

A ctivities

Determine Approach for Verifying Assumptions Perform Tests of Assumptions Determine Corrective Actions

Tools

Tests of distributional assumptions Tests for independence and trends Tests for dispersion assumptions

### EPA QA/G-9S 94 February 2006 

## List of Boxes Page 

## Box 4-1: Directions for Studentized Range Test ......................................................................... 99 Box 4-2: Example of the Studentized Range Test..................................................................... 100 Box 4-3: Directions for Geary’s Test......................................................................................... 1 01 Box 4-4: Example of Geary's Test............................................................................................. 1 01 Box 4-5: Directions for the Test for a Correlation Coefficient and an Example....................... 105 Box 4-6: Upper Triangular Co mputations for Basic Mann-Ke ndall Trend Test with a Single Measurement at Each Time Point............................................................................... 106 Box 4-7: Directions for the Mann-Kenda ll Trend Test for Small Sample Sizes....................... 106 Box 4-8: An Example of Mann-Kendall Trend Test for Small Sample Sizes........................... 106 Box 4-9: Directions for the Mann-Kenda ll Test Using a Normal Approximation.................... 108 Box 4-10: An Example of Mann-Kendall Trend Test by Normal Approximation.................... 109 Box 4-11: Data for Multiple Times and Multiple Stations........................................................ 111 Box 4-12: Testing for Comparability of St ations and an Overall Monotonic Trend................. 112 Box 4-13: Directions for the Wald-Wolfowitz Runs Test......................................................... 114 Box 4-14: An Example of the Wald-Wolfowitz Runs Test....................................................... 115 Box 4-15: Directions for the Extr eme Value Test (Dixon's Test) ............................................. 118 Box 4-16: An Example of the Extr eme Value Test (Dixon's Test) ........................................... 118 Box 4-17: Directions for the Discordance Test ......................................................................... 119 Box 4-18: An Example of the Discordance Test ....................................................................... 119 Box 4-19: Directions for Rosner's Test for Outliers.................................................................. 120 Box 4-20: An Example of Rosner's Test for Outliers................................................................ 121 Box 4-21: Directions for Walsh's Test for Large Sample Sizes ................................................ 122 Box 4-22: Directions for Constructing Confid ence Intervals and Confidence Limits for the Sample Variance and Standard De viation with an Example .................................... 123 Box 4-23: Directions for an F -Test to Compare Two Population Variances with an Example 124 Box 4-24: Directions for Bartlett's Test..................................................................................... 1 25 Box 4-25: An Example of Bartlett's Test................................................................................... 125 Box 4-26: Directions for Levene's Test ..................................................................................... 126 Box 4-27: An Example of Levene's Test ................................................................................... 127 Box 4-28: Directions for Transf orming Data and an Example.................................................. 129 Box 4-29: Directions for Aitchison’s Method to Adjust Means and Variances........................ 131 Box 4-30: An Example of Aitchison’s Method ......................................................................... 132 Box 4-31: Directions for Cohen's Method................................................................................. 133 Box 4-32: An Example of Cohen's Method............................................................................... 133 Box 4-33: Example of Doubl e Linear Interpolation.................................................................. 134 Box 4-34: Directions for Se lecting Between Cohen’s Method or Aitchison’s Method ............ 134 Box 4-35: Example of Determining Between Cohen’s Method and Aitchison’s Method ........ 135 Box 4-36: Directions for the Rank von Neumann Test ............................................................. 137 Box 4-37: An Example of the Rank von Neumann Test ........................................................... 138

### EPA QA/G-9S 95 February 2006 

## List of Tables Page 

## Table 4-1. Data for Examples in Section 4.2............................................................................... 96 Table 4-2. Tests for Normality................................................................................................ ..... 97 Table 4-3. Recommendations for Select ing a Statistical Test for Outliers................................ 117 Table 4-4. Guidelines for Anal yzing Data with Non-Detects.................................................... 130 Table 4-5. Guidelines for Recommended Parame ters for Different Coefficient of Variations and Censoring ........................................................................................................... 136

### EPA QA/G-9S 96 February 2006 

## CHAPTER 4

STEP 4: VERIFY THE ASSUMPTION S OF THE STATISTICAL METHOD

4.1 OVERVIEW AND ACTIVITIES


## In this step, the analyst should assess the va lidity of the statistical method chosen in step 3 by examining its underlying assumptions or determine that the data support the underlying assumptions necessary for the sele cted method, or if a different statistical method should be used.

If it is determined that one or more of the assumptions is not met, then an alternative action is required. Typically, this means the sele ction of a different statistical method. Each method of Chapter 3 provides a deta iled list of alternative methods.

Parametric tests also have di fficulty dealing with outliers an d non-detects. If either is found in the data, then a corre ctive action would be to use the corresponding nonparametric method. In general, nonparametric methods handle outliers and non- detects better than parametric methods. If a trend in the data is detected or the da ta are found to be dependent, then the methods of Chapter 3 should not be applied. Time series or geostati stical methods may be required and a statistician should be consulted. For a more exte nsive discussion of the overview and activities of this step, see Data Quality Assessment: A Reviewer’s Guide (EPA QA/G-9R) (U.S.EPA 2004).

4.2 TESTS FOR DISTRIBUTIONAL ASSUMPTIONS


## Many statistical tests and mode ls are only appropriate for da ta that follow a particular distribution. This section will aid in determining if a distributional assumption of a statistical test is satisfied, in particular, the assumption of nor mality. Two of the most important distributions for tests involving environmental data are the normal distributi on and the lognormal distribution, both of which are discussed in this section. To te st if the data follow a distribution other than the normal distribution or the lognor mal distribution, apply the ch i-square test discussed in Section 4.2.5 or cons ult a statistician.

There are many methods available for veri fying the assumption of normality ranging from simple to complex. This section disc usses methods based on graphs, sample moments (kurtosis and skewness), sample ranges, the Shap iro-Wilk test and clos ely related tests, and goodness-of-fit tests. Discussions for the simplest tests contain step-by-step directions and examples based on the data in Table 4-1. These te sts are summarized in Table 4-2. This section ends with a comparison of the tests to help the analyst select a test for normality.

Table 4-1. Data for Examples in Section 4.2


## These are 10 observations of dust fr om commercial air filters in ppm

15.63 11.00 11.75 10.45 13.18 10.37 10.54 11.55 11.01 10.23 X 

### =11.57

s = 1.677

n = 10

### EPA QA/G-9S 97 February 2006 

## Table 4-2. Tests for Normality

Test Section Sample Size Recommended Use

Shapiro Wilk W Test 4.2.2 ≤ 5000 Highly recommended.

Filliben's Statistic 4.2.2 ≤ 100 Highly recommended, especially when used in conjunction with a normal probability plot. Skewness and Kurtosis Tests 4.2.3 > 50 Useful for large sample sizes. Studentized Range Test 4.2.4 ≤ 1000 Highly recommended (with some conditions).

Geary's Test 4.2.4 > 50 Useful when tables for other tests are not available.

Chi-Square Test 4.2.5 Large a 

### Useful for grouped data and when the comparison distribution is known.
Lilliefors Kolmogorov- Smirnoff Test 4.2.5 > 50 Useful when tables for other tests are not available.

a 

### The necessary sample size depends on the number of groups formed when implementing this test. Each group

should contain at least 5 observations.



## The normal distribution is one of the most common probability distributions in the analysis of environmental data. It can often be used to approximate other probability distributions and in some instances more comp lex distributions can be transformed to an appropriate normal distribution. Additionally, as the sample size becomes larger, the sample mean has an approximate normal distribution hence the common assumption associated with parametric tests is that the data follows a normal distribution.

The graph of a normally distributed random variable is bell-shaped (see Figure 4-1) with the highest point located at the mean which is equal to the median. A normal curve is symmetric about the mean, hence the part to the left of the mean is a mirror image of the part to the ri ght. In environmental data, random errors occurring during the measurement process may be normally distributed.

Environmental data commonly exhibit frequency distributions that are non-negative and skewed with heavy or lo ng right-tails. Several standard parametric probability models have th ese properties, including the Weibull, gamma, and lognormal distributions. The lognormal distribution (Figure 4-1) is a commonly used distribution for modeling environmen tal contaminant data. The advantage to this distribution is that a simple (logarithmic) tr ansformation will transform a lognor mal distribution into a normal distribution. Therefore, the me thods for testing for normality described in this section can be used to test for lognormality if a logarithmic tr ansformation has been used. It should be noted that as the shape parameter of the lognormal approaches zero, the lognormal approaches a normal. When the shape parameter is large, the lognormal becomes more positively skewed. 0 5 10 15 20 25 0 0.1 0.2 0.3

Lognormal Distribution Normal Distribution



### Figure 4-1. Density Plots for the Normal and Lognormal Distributions

### EPA QA/G-9S 98 February 2006 

## 4.2.1 Graphical Methods


## Graphical methods (Section 2.3) present detailed information about data sets that may not be apparent from a test statistic. Histograms, stem-and-leaf plots, and normal probability plots are some graphical methods that are useful for determining whet her or not data follow a normal curve. Both the histogram and stem-and-leaf pl ot of a normal distributi on are bell-shaped. The normal probability plot of a normal distributi on follows a straight line. For non-normally distributed data, there will be larg e deviations in the tails or middl e of a normal probability plot.

Using a plot to decide if the data are normally distributed is a quali tative judgment. For extremely non-normal data, it is easy to make th is determination; however, in many cases the decision is not straightforward. Therefore, formal test procedures are us ually necessary to test the assumption of normality.

4.2.2 Normal Probability Plot Tests


## One of the most powerful tests for normality is the Shapiro-Wilk W test. This test is similar to computing a correlation between the quantiles of the st andard normal distribution and the ordered values of a data set. If the normal probability plot is approximately linear (i.e., the data follow a normal curve), the test statistic will be relatively high. If the normal probability plot is nonlinear, then the test statistic will be relatively low.

The W test is recommended in several EPA gui dance documents and in many statistical texts. Tables of critical values for sample si zes up to 50 have been developed for determining the significance of the test statistic. Howe ver, many software packages can perform the W test for data sets with sample sizes as large as 5000. This test is difficult to compute by hand since it requires two different sets of tabled valu es and a large number of summations and multiplications. Therefore, directions for implemen ting this test are not given in this document.

Several tests related to the Sh apiro-Wilk test have been pr oposed. D’Agostino’s test for sample sizes between 50 and 1000 and Royston’s te st for sample sizes up to 2000 are two such tests that approximate some of the key quantities or parameters of the W test.

Another related test is the Filliben statistic , also called the probabi lity plot correlation coefficient. This test measures the linearity of the points on the normal pr obability plot. Similar to the W test, if the normal probability plot is approximately linear, then the correlation coefficient will be relatively high. Although easier to compute that the W test, the Filliben statistic is still difficult to compute by hand. Ther efore, directions for implementing this test are not given in this guidance.

4.2.3 Coefficient of Skewness/Coef ficient of Kurtosis Tests


## The degree of symmetry (or asymmetry) di splayed by a data set is measured by the coefficient of skewness ( g 3 

## ). The coefficient of kurtosis, g 4 

## , measures the degree of flatness of a probability distribution near its center. Several test methods have been proposed using these coefficients to test for normality. One method te sts for normality by adjusting the coefficients of

### EPA QA/G-9S 99 February 2006 

## skewness and kurtosis to approximate a standard normal distribution for sample sizes greater than 50.

Two other tests based on these coefficients include a combined test based on a chisquared ( χ

2 

## ) distribution and Fisher’s cumu lant test. Fisher’s cumulant test computes the exact sampling distribution of g 3 

## and g 4 

## ; therefore, it is more powerfu l than previous methods which assume that the distributions of the two coefficients are normal. Fisher’s cumulant test requires a table of critical values, and thes e tests require a sample size of greater than 50. Tests based on skewness and kurtosis are rarely used as th ey are less powerful than many alternatives.

4.2.4 Range Tests


## Nearly 100% of the area of a normal curve lie s within ±5 standard deviations from the mean. Tests for normality have been developed based on this fact. Two such tests are the Studentized Range test a nd Geary’s test. Both of these tests use a ratio of an estimate of the sample range to the sample standard deviation. Very large and very sm all values of the ratio imply that the data are not well modeled by a normal distribution.

4.2.4.1 The Studentized Range Test


## This test compares the sample range to the samp le standard deviation. Tables of critical values for sample sizes up to 1000 (Table A-3 of Appendix A) are available for determining whether the absolute value of this ratio is significantly large. Directions for implementing this method are given in Box 4-1 along with an exam ple. The studentized range test does not perform well if the data are asymmetric or if th e tails of the data are heavier than the normal distribution. In addition, this test may be sensitive to extreme values. Many environmental data sets are positively skewed (have a long tail of high values) and are si milar to a lognormal distribution.. If the data appear to be lognormally distributed, then this test should not be used. In most cases, the studentized range test performs as well as the Shapiro-Wilk test and is easier to apply.

Box 4-1: Directions fo r Studentized Range Test

COMPUTATIONS: Using Section 2.2.3, calculate sample range, w , and sample standard deviation, s

. STEP 1. Null Hypothesis: H 0 : The underlying distribution of the data is a normal distribution.

STEP 2. Null Hypothesis: H A : The underlying distribution of the data is not a normal distribution.

STEP 3. Test Statistic: R = w / s
STEP 4. a) Critical Value: Use Table A-3 to find the critical values a and b

. STEP 5. a) Conclusion: If R falls outside the two critical values, then reject the null hypothesis that the underlying distribution is normal.



## Directions for the studentized range test are contained in Box 4-1, with an example in Box 4-2.

### EPA QA/G-9S 100 February 2006 Box 4-2: Example of th e Studentized Range Test
Using a significance level of 0.01, determine if the underlying distribution of the data from Table 4-1 can be modeled using a normal distribution.

COMPUTATIONS: w = X (10) – X ( 1 ) = 15.63 – 10.23 = 5.40 and s = 1.677.

STEP 1. Null Hypothesis: H 0 : The underlying distribution of the data is a normal distribution.

STEP 2. Null Hypothesis: H A : The underlying distribution of the data is not a normal distribution.

STEP 3. Test Statistic: R = w / s

= 5.4 / 1.677 = 3.22.

STEP 4. a) Critical Value: Using Table A-3, the critical values a and b are 2.51 and 3.88.

STEP 5. a) Conclusion: Since R falls between the two critical values, we fail to reject the null hypothesis that the underlying distribution is normal.



## 4.2.4.2 Geary’s Test


## Geary’s test compares the sum of the absolute deviations from the m ean to the sum of the squares. If the ratio is too la rge or too small, then the underlyi ng distribution of the data should not be modeled as a normal distribution. Dire ctions for implementing this method are given in Box 4-3 and an example is given in Box 4-4. Th is test does not perform as well as the ShapiroWilk test or the Studentized Range test.

Directions for Geary’s test are given in Box 4-3, with an example in Box 4-4.

4.2.5 Goodness-of-Fit Tests



## Goodness-of-fit tests are used to test whether data follow a sp ecific distribution, i.e., how "good" a specified distribution fits the data. In verifying assumptions of normality, one would compare the data to a normal distribution with a specified mean and variance.

4.2.5.1 Chi-Square Test


## One classic goodness-of-fit test is the chi-square test whic h involves breaking the data into groups and comparing these groups to the expected groups from the known distribution. There are no fixed methods for selecting these gro ups and this test also requires a large sample size since at least 5 observations per group is neededto implement this test. In addition, the chisquare test does not have the power of the Shapir o-Wilk test or some of the other tests mentioned above. However, it is more flexible since the da ta can be compared to probability distributions other than the normal but the application of goodness-of-fit test s to non-normal data is beyond the scope of this guidance.

### EPA QA/G-9S 101 February 2006 Box 4-3: Directions for Geary’s Test

COMPUTATIONS: Calculate the sample mean, X , the sample sum of squa res, SSS, and the sum of absolute deviations, SAD:

X n X

i= n

i = 

# ∑

1

1 , SSS X n X ,

i= n

i i= n

i = − ⎛

⎝ ⎜ ⎜ ⎞

⎠ ⎟ ⎟ 

# ∑ ∑

1 2

1 2 1 and SAD X X

i= n

i = −



# ∑

1 .

STEP 1. Null Hypothesis: H 0 : The underlying distribution of the data is a normal distribution.

STEP 2. Null Hypothesis: H A : The underlying distribution of the data is not a normal distribution.

STEP 3. Test Statistic:

n . . a z 2123 0 7979 0 0 −

= , where a SAD
n SSS =

⋅ .

STEP 4. a) Critical Value: Use Table A-1 to find z 1- α /2

.
STEP 4. b) p-value: Use Table A-1 to find 

# ( ) 0 P 2 z Z > ⋅ .

STEP 5. a) Conclusion: If 2 / 1 0 α

− > z z , then reject the null hypothesis that the underlying
distribution is normal.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the underlying distribution is normal.

Box 4-4: Example of Geary's Test
Using a significance level of 0.05, determine if the underlying distribution of the data from Table 4-1 can be modeled using a normal distribution.

COMPUTATIONS:

, 571

. 11 = X 694

. 11 = SAD , and 298

. 25 = SSS

. STEP 1. Null Hypothesis: H 0 : The underlying distribution of the data is a normal distribution.

STEP 2. Null Hypothesis: H A : The underlying distribution of the data is not a normal distribution.

STEP 3. Test Statistic: 9339

. 0

10 2123

. 0 7979 0 7352 0 0 − = −
= . . z , since 7352 0 298 25 10 694

11

.

. . a =

⋅ =

.

STEP 4. a) Critical Value: Using Table A-1, z 0.975 = 1.96.

STEP 4. b) p-value: Using Table A-1, 2

⋅ P( Z > 0.9339) = 2

⋅ (1 – 0.8238) = 0.3524.

STEP 5. a) Conclusion: Since α

− = < = 1 0 96

. 1 9339

. 0 z z , we fail to reject the null hypothesis that the underlying distribution is normal.

STEP 5. b) Conclusion: Since p-value = 0.3524 > 0.05 = α , we fail to reject the null hypothesis that the underlying distribution is normal.

### EPA QA/G-9S 102 February 2006 

## 4.2.5.2 Tests Based on the Empirical Distribution Function



## Various methods have been used to meas ure the discrepancy between the empirical distribution function and the theore tical cumulative distribution f unction (cdf). These measures are referred to as empirical distribution functi on statistics. The best known is the KolmogorovSmirnov (K-S) statistic. The K-S approach is appr opriate if the sample size exceeds 50 or if the theoretical cumulative density function is a sp ecific distribution with known parameters. A modification to the test, called th e Lilliefors K-S test, can be used to test that the data ( n > 50) comes from a normal distributi on with mean and variance equal to the sample values.

Unlike the K-S type statistics, most empirical distribution function st atistics are based on integrated or average values between the empirical distribution function and hypothesized cumulative distribution functions. The two most powerful are the Cramer-von Mises and Anderson-Darling statistics. Extensive simulatio ns show that the Ande rson-Darling empirical distribution function statis tic is as good as any te st, including the Shapiro-Wilk test, when testing for normality. However, the Shapiro-Wilk test is applicable only for the normal distribution, while the Anderson-Darling method applies to other distributions.

4.2.6 Recommendations


## Tests for normality with small samples have ve ry little statistical power. Therefore, for small sample sizes it is common for a nonparametric st atistical test be select ed during Step 3 of the DQA in order to avoid incorrectly assuming the data are normally distributed when there is simply not enough information.

This guidance recommends using the Shapiro-Wilk W test, wherever practicable. The Shapiro-Wilk W test is one of most powerful tests for normality. This test is difficult to implement by hand but can be applied easily using a statistical software package. If the ShapiroWilk W test is not feasible, then this guidance recommends using either Filliben's statistic together with inspection of the nor mal probability plot, or the studentized range test. Filliben's statistic performs similarly to the Shapiro-Wilk te st. The studentized range is a simple test to perform, but, it is not ap plicable for non-symmetric data with large tails. If the data are not highly skewed and the tails are not significantly large compared to a normal distribution, then the studentized range provides a si mple and powerful test that can be calculated by hand. The Lilliefors Kolmogorov-Smirnoff test is statistically powerful but is also more difficult to apply and uses specialized tables not readily available.

4.3 TESTS FOR TRENDS

4.3.1 Introduction


## This section presents statisti cal tools for detecting and esti mating trends in environmental data. The detection and estimation of tempor al or spatial trends are important for many environmental studies or monitori ng programs. In cases where temporal or spatial patterns are strong, simple procedures such as time plots or linear regression over time can reveal trends. In more complex situations, sophistic ated statistical models and pr ocedures may be needed. For

### EPA QA/G-9S 103 February 2006 

## example, the detection of trends may be compli cated by the overlaying of long- and short-term trends, cyclical effects (e.g., seasonal or weekly systematic variations), autocorrelations, or impulses or jumps (e.g., due to interv entions or procedural changes).

The graphical representations of Section 2. 3.7 are recommended as the first step to identify possible trends. A Time Plot and/or Lag Pl ot is recommended for temporal data as it may reveal long-term trends, seasonal behavior , or impulses. A pos ting or bubble plot is recommended for spatial data to reveal spatial tr ends such as areas of high concentration or inaccessible areas.

For most of the statistical tools presented below, the fo cus is on monotonic long-term trends (i.e., a trend that is ex clusively increasing or decreasing) , as well as other sources of systematic variation, such as seasonality. The investigations of trend in this section are limited to one-dimensional domains, e.g., trends in a po llutant concentration over time. The current edition of this document does not address spatia l trends or trends over space and time, which may involve sophisticated geostatis tical techniques such as krigi ng and require the assistance of a statistician. Section 4.3.2 discusses estima ting and testing for trends using regression techniques. Section 4.3.3 di scusses nonparametric trend esti mation procedures, and Section 4.3.4 discusses hypothesis tests for detecting tr ends under several types of situations.

4.3.2 Regression-Based Methods for Es timating and Testing for Trends

4.3.2.1 Estimating a Trend Using the Slope of the Regression Line


## The classic procedures for assess ing linear trends involve regre ssion. Linear regression is a commonly used procedure in which calculations ar e performed on a data se t containing pairs of observations ( X

i 

## , Y i 

## ), so as to obtain the slope and intercept of a line that best fits the data. For temporal data, the X i 

## values represent time and the Y i 

## values represent the observations. An estimate of the magnitude of trend can be obtaine d by performing a regression of the data versus time and using the slope of the re gression line as the measure of th e strength of the trend.

Regression procedures are eas y to apply. All statisti cal software packages and spreadsheet programs will calculate the slope and inte rcept of the best fitting line, as well as the correlation coefficient r (see Section 2.2.4). However, regr ession entails several limitations and assumptions. First of all, simple linear regressi on (the most commonly used method) is designed to detect linear relationships between two variables; other types of regression models are generally needed to detect non- linear relationships such as cy clical or non-monotonic trends. Regression is very sensitive to outliers and presents difficult ies in handling data below the detection limit, which are commonly encountered in environmental studies. Hypothesis testing for linear regression also relies on two key assump tions: normally distributed errors, and constant variance. It may be difficult or burdensome to verify these assumptions in practice, so the accuracy of the slope estimate may be suspect. Mo reover, the analyst must ensure that time plots of the data show no cyclical patterns, outlier tests show no extreme data values, and data validation reports indicate that nearly all the measurem ents were above detection limits. Due to these drawbacks, linear regression is not r ecommended as a general tool for estimating and

### EPA QA/G-9S 104 February 2006 

## detecting trends, although it may be useful as an informal and quick screening tool for identifying strong linear trends.

4.3.2.2 Testing for Trends Using Regression Methods


## For simple linear regression, the statistical test of whether the sl ope is significantly different from zero is equivalent to testing if the correlation coe fficient is significantly different from zero. This test assumes a linear relation between Y and X with independent normally distributed errors and cons tant variance across all the X values. Non-detect s and outliers may, however, invalidate the test.

Directions for this test are given in Box 4-5 along with an example.

4.3.3 General Trend Estimation Methods

4.3.3.1 Sen's Slope Estimator


## Sen’s Slope Estimate is a nonparametric alternat ive for estimating a slop e. This approach involves computing slopes for a ll the pairs of time points and th en using the median of these slopes as an estimate of the overall slope. As su ch, it is insensitive to outliers and can handle a moderate number of values below the detection limit and missing values. Assume that there are

n time points, and let X i 

## denote the data value for the i th 

## time point. If there are no missing data, there will be n ( n -1)/2 possible pairs of time points ( i
, j ) in which i < j . The slope for such a pair is b ij 

## = ( X

j 

## – X i 

## ) / ( j - i ). Sen's slope estimator is then the median of the n ( n -1)/2 pairwise slopes. If there is no underlying trend, there would be an approximately equal number of positive and negative slopes, and thus the median would be near zero.

4.3.3.2 Seasonal Kendall Slope Estimator


## If the data exhibit cyclic trends, then Sen's slope estimator can be modified to account for the cycles. For example, if data are available fo r each month for a number of years, 12 separate sets of slopes would be determined; similarly, if daily observations exhibit weekly cycles, seven sets of slopes would be determined. In these esti mates, the above pairwise slope is calculated for each time period and the median of all of the slop es is an estimator of the slope for a long-term trend. This is known as the seasonal Kendall slope estimator.

4.3.4 Hypothesis Tests for Detecting Trends



## Most of the trend tests treated in this section involve the Ma nn-Kendall test or extensions of it. The Mann-Kendall test does not assu me any particular di stributional form and accommodates values below the detection limit by assigning them a common value. The test can also be modified to deal with multiple observations, multiple sampling locations, and seasonality.

### EPA QA/G-9S 105 February 2006 Box 4-5: Directions for the Test for a Correlation Co efficient and an Example

COMPUTATIONS: Calculate the correlation coefficient, r (Section 2.2.4).

STEP 1. Null Hypothesis: H 0 : The correlation coefficient is zero.

STEP 2. Alternative Hypothesis: H A : The correlation coefficient is different from zero.

STEP 3. Test Statistic: t r

r

n 0 2 1
2 =

−

− .

STEP 4. a) Critical Value: Use Table A-2 to find 2 / 1

, 2 α

− − n t

.

STEP 4. b) p-value: Use Table A-2 to find 

# ( ) 0 2 P 2 t t n > ⋅

− .

STEP 5. a) Conclusion: If 2 / 1

, 2 0 α

− − > n

t t , then reject the null hypothesis that the
correlation coefficient is zero.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that the correlation coefficient is zero.

Example : Using a significance level of 0.10, determine if the following data set (in ppb) has significant correlation between it’s two variables:

Sample Number 1 2 3 4 Arsenic 8.0 6.0 2.0 1.0

Lead 8.0 7.0 7.0 6.0

COMPUTATIONS: In Section 2.2.4, the correlation coefficient r for this data was calculated to be 0.865.

STEP 1. Null Hypothesis: H 0 : The correlation coefficient is zero.

STEP 2. Alternative Hypothesis: H A : The correlation coefficient is different from zero.

STEP 3. Test Statistic: 438

. 2

2 4 865

. 0 1 865

0

2 0 =

− − = . t

.

STEP 4. a) Critical Value: Using Table A-2, 303

. 4 975

. 0

, 2 = t

.
STEP 4. b) p-value: Using Table A-2, 0.10< p-value < 0.20 (the exact p-value = 0.1350)

STEP 5. a) Conclusion: Since 975

. 0

, 2 0 t t < , we fail to reject the null hypothesis that the
correlation coefficient is zero.

STEP 5. b) Conclusion: Since p-value = 0.1350 > 0.10 ( α ) we fail to reject the null hypothesis that the correlation coefficient is zero.

### EPA QA/G-9S 106 February 2006 

## 4.3.4.1 One Observation per Time Peri od for One Sampling Location



## The Mann-Kendall test involve s computing a statistic S , which is the difference between the number of pairwise differences that are pos itive minus the number that are negative. If S is a large positive value, then there is evidence of an increasing trend in the data. If S is a large negative value, then there is ev idence of a decreasing trend in the data. The null hypothesis or baseline condition for this test is that there is no temporal trend in the data values. The alternative hypothesis is th at of either an upward tr end or a downward trend.

Box 4-6: Upper Triangular Computations for Basic Mann-Kendall Trend Test with a Single Measu rement at Each Time Point

Original Time Measurement t 1 X

1 t 2 X

2 t 3 X

3 t 4 X

4 . .

.

. .

. t n -1 X

n -1 t n X

n

X

1 Y 12 Y 13 Y 32 . .

. Y 1 , n -1 Y 1 n #(plusses) #(minuses)

X

2 Y 23 Y 32 . .

. Y 2, n -1 Y 2 n #(plusses) #(minuses)

.

.

. .

.

. .

.

. .

.

. .

.

. X

n-2 Y n -2, n -1 Y n -2, n #(plusses) #(minuses)

X

n-1 Y n -1, n #(plusses) #(minuses)

Total #(plusses) Total #(minuses) where Y ij = sign ( X j - X i ) = + if X j > X i = 0 if X j = X i = - if X j < X i

Box 4-7: Directions for the Mann-Kenda ll Trend Test for Small Sample Sizes
COMPUTATIONS: Create the upper triangular table of pairwise differences as described in Box 4-6.
STEP 1. Null Hypothesis: H 0 : There is no trend.
STEP 2. Null Hypothesis: i) H A : There is a downward trend. ii) H A : There is an upward trend.

STEP 3. Test Statistic: S = Total #(plusses) – Total #(minuses)
STEP 4. a) Critical Value: Use Table A-12a to find the critical value.
STEP 4. b) p-value: Use Table A-12b to find the p-value.
STEP 5. a) Conclusion: If | S | ≥ the critical value, then reject the null hypothesis of no trend.
STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis of no trend.

### EPA QA/G-9S 107 February 2006 

## The computations for the basic Mann-Kendall trend test are de picted in Box 4-6. Assign a value of DL/2 to all non-detects. The test statistic is the difference between the number of strictly positive differences and the number of st rictly negative differences. Differences of zero are not included in the test st atistic (and should be avoided, if possible, by recording data to sufficient accuracy). The steps for conducting the Mann-Kendall test for small sample sizes ( n < 10) are contained in Box 4-7 and an example is contained in Box 4-8. For sample sizes greater than 10, there is a norma l approximation for the Mann-Kendall test. Directions for this approximation are contained in Box 4-9 with an exam ple given in Box 4-10.

Box 4-8: An Example of Mann-Kendall Trend Test for Small Sample Sizes
Consider 5 measurements ordered by the time of their collection: 5, 6, 11, 8, and 10. This data will be used to test for an upward trend at a significance level of 0.05.

COMPUTATIONS: A triangular table was used to construct the pairwise differences.

STEP 1. Null Hypothesis: H 0 : There is no trend.

STEP 2. Null Hypothesis: H A : There is an upward trend.

STEP 3. Test Statistic: S = Total #(plusses) – Total #(minuses) = 8 - 2 = 6

STEP 4. a) Critical Value: Using Table A-12a, the critical value is 8.

STEP 4. b) p-value: Using Table A-12b, the p-value is 0.117.

STEP 5. a) Conclusion: As S = 6 < 8 (critical value), fail to reje ct the null hypothesis of no trend

STEP 5. b) Conclusion: Since p-value = 0.117 > 0.05(significance level), we fail to reject the null 

## hypothesis of no trend. Time Data 1 5 2 6 3 11 4 8 5 10 No. of + Signs No. of - Signs

5 6 11 8 + +
+ +
+

- +
+

- + 4 3 0 1 8 0 0 2 0 2

### EPA QA/G-9S 108 February 2006 Box 4-9: Directions for the Mann-Kenda ll Test Using a Normal Approximation
COMPUTATIONS: If the sample size is 10 or more, a normal approximation to the Mann-Kendall procedure may be used. Compute S as described in Box 4-7 above.

STEP 1. Null Hypothesis: H 0 : There is no trend.

STEP 2. Null Hypothesis: i) H A : There is a downward trend. ii) H A : There is an upward trend.

STEP 3. Test Statistic: 

## ( ) ( ) S S S z V sign 0 −

= , where 

## ( ) ( )( ) 

# ( )( ) ⎥ ⎥ ⎥ ⎦ ⎤

⎢ ⎢ ⎢ ⎣ ⎡

+

− − +

− = 

# ∑

= 5 2 1 5 2 1 18 1 V

1 j j j g

j t t t n n n S

,

g

is the number of tied groups, and t j is the number of points in the j th group. Note that sign(S) = 1 if S > 0, 0 if S = 0 and –1 if S < 0.

STEP 4. a) Critical Value: Use Table A-1 to find z 1- α .
STEP 4. b) p-value: Use Table A-1 to find 

# ( ) 0 P z

Z >

.

STEP 5. a) Conclusion: If α

− > 1 0 z z , then reject the null hypothesis of no trend.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis of no trend.

### EPA QA/G-9S 109 February 2006 Box 4-10: An Example of Mann-Kendall Trend Test by Normal Approximation
A test for an upward trend with α = 0.05 will be based on the 11 weekly measurements is shown below.

COMPUTATIONS: Using Box 4-6, a triangular table was constructed of the pairwise slopes. “0” indicates a tie.

Data

10 10 10

5 10 20 18 17 15 24 1 10 2 10 0 3 10 0 0 4 5

-

-

- 5 10

0 0 0

+ 6 20

+
+
+
+
+ 7 18

+
+
+
+
+

- 8 17

+
+
+
+
+

-

- 9 15

+
+
+
+
+

-

-

- 10 24

+
+
+
+
+
+
+
+
+ 11 15

+
+
+
+
+

-

-

- 0

- No. of + Signs 6 6 6 7 6 1 1 1 1 0 35 No. of - Signs 1 1 1 0 0 4 3 2 0 1 13

S

= (sum of + signs) - (sum of - signs) = 35 - 13 = 22.

STEP 1. Null Hypothesis: H 0 : There is no trend.

STEP 2. Null Hypothesis: H A : There is an upward trend.

STEP 3. Test Statistic: There are several observations tied at 10 and 15. Thus, the formula for tied values will be used. In this formula, g = 2, t 1 = 4 for tied values of 10, and t 2 = 2 for tied values of 15. Therefore,


## ( ) ( )( ) ( )( ) ( )( ) 

## [ ] { } 33 155 5 2 2 1 2 2 5 4 2 1 4 4 5 11 2 1 11 11 18 1 V

.

S = + ⋅ − + + ⋅ − − + ⋅ − =

and 

## ( ) ( ) 685

. 1 33

. 155 1 22
V sign 0 = −
= −
= S S

S z

.

STEP 4. a) Critical Value: Using Table A-1, z 0.95 = 1.645.

STEP 4. b) p-value: Using Table A-1, p-value = 

## ( ) 685

. 1 P >

Z = 0.046.

STEP 5. a) Conclusion: Since test statistic = 1.685 > 1.645 = critical value, we reject the null hypothesis of no trend.

STEP 5. b) Conclusion: Since p-value = 0.046 < 0.05 = α , we reject the null hypothesis of no trend.



## 4.3.4.2 Multiple Observations per Time Period for One Sampling Location


## Often, more than one sample is collected for each time period. There are two ways to deal with multiple observations per time period. One method is to compute a summary statistic, such as the median, for each time period and to apply one of the Mann- Kendall trend tests of Section 4.3.4.1 to the summary statistic. Therefore, instead of using the i ndividual data points in the triangular table, the summary statistic would be used. Then the steps given in Box 4-7 or Box 4-9 could be applied to the summary statistics.

### EPA QA/G-9S 110 February 2006 

## An alternative approach is to consider all the multiple observations within a given time period as being essentially taken at th e same time within that period. The S statistic is computed as before with n being the total of all obser vations. The variance of the S statistic (calculated in step 2 of Box 4-9) is changed to:



## ( )( ) 

# ( )( ) ( )( ) ⎥ ⎥ ⎥ ⎦ ⎤

⎢ ⎢ ⎢ ⎣ ⎡

+

− − +

− − +

− = 

# ∑ ∑

=

= 5 2 1 5 2 1 5 2 1 18 1 VAR

1 1 k k k h

k j j j g

j u u u t t t n n n (S)



# ( )( ) ( )( )
( )( ) 

# ( ) ( )
( ) 1 2 1

1

2 1 9 2 1 2 1

1 1 1 1

− − ⋅ −

+

− − − − ⋅ − −

+ 

# ∑ ∑ ∑ ∑

= = = =
n n u u t t

n n n u u u t t t h

k k k g

j j j h

k k k k g

j j j j



## where g represents the number of tied groups, t j 

## represents the number of data points in the j th



## group, h is the number of time periods which contain multiple data, and u k 

## is the sample size in the k th 

## time period.

The preceding variance formula assumes that the data are not correlated. If correlation within single time periods is suspected, it is pr eferable to use a summary statistic (e.g., the median) for each time period and then apply either Box 4-7 or Box 4-9 to the summary statistics.

4.3.4.3 Multiple Sampling Locations with Multiple Observations


## The preceding methods involve a single sampling location (station). However, environmental data often consist of sets of data collected at several sampling locations (see Box 4-11). For example, data are often systematica lly collected at several fi xed sites on a lake or river, or within a region or basi n. The data collection plan (o r experimental design) must be systematic in the sense that approximately th e same sampling times should be used at all locations. In this situation, it is desirable to express the results by an overall regional summary statement across all sampling locations. However, there must be consistency in behavioral characteristics across sites over time in order fo r a single summary statement to be valid across all sampling locations. A useful plot to assess the consistency requirement is a single time plot (Section 2.3.7.1) of the measurements from all stations where a different symbol is used to represent each station.

If the stations exhibit approximate trends in the same direction with comparable slopes, then a single summary statement across stations is valid and this implies two relevant sets of hypotheses should be investigated:

Comparability of stations. H 0 

## : Similar dynamics affect all K stations vs. H A 

## : At least two stations exhibit different dynamics.

Testing for overall monotonic trend. H 0 * 

## : Contaminant levels do not change over time vs. H A * 

## : There is an increasing (or decreasing) trend consistent acros s all stations.

### EPA QA/G-9S 111 February 2006 

## Therefore, the analyst must first test fo r homogeneity of stations, and then, if homogeneity is confirmed, test for an overall monotonic trend. Directions for the test are contained in Box 4-11 and ideally, the stations in Box 4-11 should have equal sample sizes. However, the numbers of observations at the stat ions can differ slightly, because of isolated missing values, but the overall time periods spanned must be similar. This guidance recommends that for less than 3 time periods , an equal number of obs ervations (a balanced design) are required. For 4 or more time pe riods, up to 1 missing value per sampling location may be tolerated.

Box 4-11: Data for Multiple Times and Multiple Stations

Let i = 1, 2, ..., n represent time, k = 1, 2, ..., K represent sampling locations, and X ik

represent the measurement at time i for location k . This data can be summarized in matrix form, as shown below.

Stations

1 2

. . . 

## K

Time 1 2

.

.

. n X 11 X

21

.

.

. X

n 1 X 12 X

22

.

.

. X

n 2 . . .

. . .

. . .

. . .

. . .

. . . X 1 K X

2 K

.

.

. X

nK

S 1 V( S 1 )

Z 1 S 2 V( S 2 )

Z

2 . .

.

. .

.

. .

. S K V( S K ) Z K

where S k = Mann-Kendall statistic for station k (see STEP 3, Box 4-7), V( S k ) = variance for S statistic for station k (see STEP 3, Box 4-9), and


## ( )

k k k k S S S z V ) ( sign

−
=



## a. One Observation per Time Period. 

## When only one measurement is taken for each time period for each station, a generalizati on of the Mann-Kendall statistic can be used to test the above hypotheses. This pr ocedure is described in Box 4-12.

b. Multiple Observations per Time Period. 

## If multiple measurements are taken at some times and stations, then the previous approaches are still applicable. However, the variance of the statistic S k 

## must be calculated using the equation for calculating V( S ) given in Section 4.3.4.2. Note that S k 

## is computed for each station, so n , t j 

## , g , h , and u k

## are all stationspecific.

### EPA QA/G-9S 112 February 2006 Box 4-12: Testing for Comparability of Stations and an Overall Monotonic Trend

Let i = 1, 2, ..., n represent time, k = 1, 2, ..., K represent sampling locations, and X ik

represent the measurement at time i for location k . Let α represent the significance level for testing homogeneity and α * represent the significance level for testing for an overall trend.

COMPUTATIONS: Calculate the Mann-Kendall statistic S k and its variance V( S k ) for each of the K stations using the methods of Box 4-9. Now, calculate 

## ( ) ( )

k k k k S S S Z V sign

−
= , for k = 1,

... , K and Z K Z k k K

=

= 

# ∑ 1

1 .

Test of Homogeneity

STEP 1. Null Hypothesis: H 0 : Similar dynamics affect all K stations.

STEP 2. Null Hypothesis: H A : At least two stations exhibit different dynamics.

STEP 3. Test Statistic: χ h k k K Z KZ 2 2

1 2

= −

= 

# ∑

.

STEP 4. a) Critical Value: Use Table A-9 to find 2 1

, 1 α χ

− − K .

STEP 4. b) p-value: Use Table A-9 to find 

# ( ) 2 2 1 P h K χ χ >

− .

STEP 5. a) Conclusion: If 2 1

, 1 2

α χ χ

− − > K h , then reject the null hypothesis that similar dynamics affect
all K stations.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis that similar dynamics affect all K

stations.

If H 0 is not rejected, then proceed to test of overall trend. Otherwise, individual α *-level Mann-Kendall tests should be conducted using the methods presented in Section 4.3.4.1.

Test of Overall Trend

STEP 1. Null Hypothesis: H 0 *: Contaminant levels do not change over time.

STEP 2. Null Hypothesis: H A *: There is an increasing (or decreasing) trend consistently exhibited across all stations.

STEP 3. Test Statistic: 2 2 Z K o ⋅

= χ

.

STEP 4. a) Critical Value: Use Table A-9 to find 2

* 1

, 1 α χ

− .

STEP 4. b) p-value: Use Table A-9 to find 

# ( ) 2 2 1 P o χ χ >

.

STEP 5. a) Conclusion: If 2

* 1

, 1 2

α χ χ

− > h , then reject the null hypothesis that contaminant levels do not change over time.

STEP 5. b) Conclusion: If p-value < α , then reject the null hypothesis .

### EPA QA/G-9S 113 February 2006 

## 4.3.4.4 One Observation for One Station with Multiple Seasons



## Temporal data are often collected over ex tended periods of time. Within the time variable, data may exhibit periodic cycles, which ar e patterns in the data that repeat over time. For example, temperature and humidity may change with the season or month, and may affect environmental measurements. (For more information on seasonal cycles, see Section 2.3.7). In the following discussion, the term season represents one time point in the pe riodic cycle, such as a month within a year or an hour within a day.

If seasonal cycles are anticipated, then tw o approaches for testing for trends are the seasonal Kendall test and Sen's test for trends. The seasonal Kendall test may be used for large sample sizes, and Sen's test for trends may be us ed for small sample sizes. If different seasons manifest similar slopes (rates of change) but pos sibly different intercep ts, then the Mann-Kendall technique of Section 4.3.4.3 is applicable, replacing time by year and replacing station by season.

The seasonal Kendall test, which is an ex tension of the Mann- Kendall test, involves calculating the Mann-Kendall test statistic, S , and its variance separately for each season. The sum of the S 's and the sum of their variances are then used to form an overall te st statistic that is assumed to be approximately normally di stributed for larger size samples.

For data at a single site, collected at mu ltiple seasons within multiple years, the techniques of Section 4.3.4.3 can be applied to test for homogeneity of time trends across seasons. The methodology follows Boxes 4-11 and 4- 12 exactly except that ‘station’ is replaced by ‘season’ and the inferences refer to seasons.

4.3.5 A Discussion on Tests for Trends



## This section discusses some further considerations for choosing among the many tests for trend. Mann-Kendall type nonpara metric trend tests and estimate s use ordinal time (ranks) rather than cardinal time (actual time values) and this re stricts the interpretation of measured trends. All of the Mann-Kendall Trend Tests presented ar e based on certain pairwise differences in measurements at different time points. The only in formation about these differences that is used in the Mann-Kendall calculations is the sign, and can be regarded as generalizations of the signed rank test. However, since information about the magnitudes of the differences is not used, this can adversely affect the statis tical power when only limited amounts of data are available.

There are nonparametric methods based on ranks that take such ma gnitudes into account and still retain the bene fit of robustness to out liers. These procedures can be thought of as replacing the data with their ranks and then co nducting parametric analyses. These include the Wilcoxon Rank Sum test and its many generalizati ons. These methods are more resistant to outliers than parametric methods. Rank-based me thods which make fuller use of the information in the data than the Mann-Kendall methods are not as robust with respect to outliers as the signed rank test and the Mann-Kendall tests, but they have more statistic al power. This kind of tradeoff

### EPA QA/G-9S 114 February 2006 

## between power and robustness shows the need for an evaluation process leading to the selection of the best statistical procedure for the situation.

4.3.6 Testing for Trends in Sequences of Data



## There are cases where it is desirable to see if a sequence of data (f or example, readings from a monitoring station) could be considered ra ndom variation or correlat ed in some way. One test to make this determination is the Wald-Wolfowitz test. This test can only be used if the data are binary, i.e., there are only tw o potential values. For example, the data could be either ‘Yes/No’ or an inves tigation of persistent violation of a permitted limit by a pollution control process where a violation equals 1, with 0 fo r not in violation. Directions for the WaldWolfowitz test are given in Box 4- 13 and an example in Box 4-14.

Box 4-13: Directions for th e Wald-Wolfowitz Runs Test
Consider a sequence of binary values. Let m and n denote the number of observations for the two values with

n < m . This test is used to test the null hypothesis that the sequence is random against the alternative hypothesis that the values in the sequence are correlated or may come from different populations.

COMPUTATIONS: List the data in the order collected and compute T , the number of runs in the sequence. For example, if the data sequence is AAABAABBBBBBBABB, then T = 6.

STEP 1. Null Hypothesis: H 0 : data sequence is random.

STEP 2. Alternative Hypothesis: H A : data sequence is non-random.

STEP 3. Test Statistic: If either m or n is less than 10, then T

.

If both m and n are at least 10, then

r r T z σ μ

−
= 0 , where



## 1 2 +

+ = n m mn

r 

## μ and 

## ( ) ( )( ) 1 2 2
2

− + + − −
= n m n m n m mn mn

r σ

STEP 4. a) Critical Value: If either m or n is less than 10, then use Table A-13 to find w α /2

. If both m and n are both at least 10, then use Table A-1 to find z 1- α /2

.
STEP 4. a) p-value: If both m and n are both at least 10, then use Table A-1 to find 

# ( ) 0 z Z P >

.

STEP 5. a) Conclusion: If either m or n is less than 10 and if T < w α /2 or T > 2
r 

## μ - w α /2 , then reject the null hypothesis that the data sequence is random. If both m and n are at least 10 and if | z 0 | > z 1- α /2 , then reject the null hypothesis that the data sequence is random.

STEP 5. b) Conclusion: If p-value < α , the reject the null hypothesis that the data sequence is random.

### EPA QA/G-9S 115 February 2006 Box 4-14: An Example of th e Wald-Wolfowitz Runs Test
The main discharge station at a chemical manufacturing plant is under a monitoring program. The permit states that the discharge should have a pH of 7.0 and should never be less than 5.0. So the plant manager has decided to use a pH of 6.0 to an indicate potential problems. In a four-week period the following values were recorded:

6.5 6.6 6.4 6.2 5.9 5.8 5.9 6.2 6.2 6.3 6.6 6.6 6.7 6.4 6.2 6.3 6.2 5.8 5.9 5.8 6.1 5.9 6.0 6.2 6.3 6.2

Since the plant manager has decided that a pH of 6.0 will indicate trouble the data have been replaced with a binary indicator. If the value is greater than 6.0, the value will be replaced by a 1; otherwise the value will be replaced by a 0. So the binary sequence is:

1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 1 0 0 1 1 1

As there are 8 values of ‘0’and 19 values of ‘1’, n = 8 and m = 19 and the number of runs is T = 7. Test at a significance level of 0.10 whether the data sequence is random.

STEP 1. Null Hypothesis: H 0 : data sequence is random.

STEP 2. Alternative Hypothesis: H A : data sequence is non-random.

STEP 3. Test Statistic: Since n is less than 10, the test statistic is T = 7.

STEP 4. a) Critical Value: Since n is less than 10, Table A-13 is used to find w 0.05 = 9.

STEP 5. a) Conclusion: Since n is less than 10 and T =7 < 9 = w 0.05 , we reject H 0

.



## 4.4 OUTLIERS

4.4.1 Background


## Potential outliers are measurements that are extremely large or small relative to the rest of the data and, therefore, are suspected of misr epresenting the population from which they were collected. Potential outliers may result from transcription errors, data-coding errors, or measurement system problems. However, outliers may also represent true extreme values of a distribution (for instance, hot spots) and indica te more variability in the population than was expected. Failure to remove true outliers or the removal of false outliers both lead to a distortion of estimates of population parameters and if it is recommended that the QA Project Plan or Sampling and Analysis Plan be revi ewed for anomalies that could account for the potential outlier.

Statistical outlier tests give the analyst proba bilistic evidence that an extreme value does not "fit" with the distribution of the remainder of the data and is therefore a statistical outlier. These tests should only be used to identify data points that require further investigation. The tests alone cannot determine whethe r a statistical outlier should be discarded or corrected within a data set. This decision should be ba sed on judgmental or scientific grounds.

### EPA QA/G-9S 116 February 2006 

## There are 5 steps involved in treating extreme values or outliers:

1. Identify extreme values that may be potential outliers; 2. Apply statistical test; 3. Scientifically review statistical outl iers and decide on their disposition; 4. Conduct data analyses with and wi thout statistical outliers; and 5. Document the entire process.

Potential outliers may be identified through the graphical representations of Chapter 2 (step 1 above). Graphs such as the box and whisker plot , ranked data plot, normal probability plot, and time plot can all be used to identify observations that are much larger or smaller than the rest of the data. If potential outliers are identified, the next step is to apply one of the statistical tests described in the following sections. Secti on 4.4.2 provides recommendations on selecting a statistical test for outliers.

If a data point is found to be an outlier, the analyst may either: 1) correct the data point; 2) discard the data point from analysis; or 3) us e the data point in all analyses. This decision should be based on scientific reasoning in addition to the results of the statistical test. For instance, data points containing transcription errors should be corrected, whereas data points collected while an instrument was malfunctioning may be discarded. Dis carding an outlier from a data set should be done with extreme caution, pa rticularly for environmental data sets, which often contain legitimate extreme values. If an outlier is discarded from the data set, all statistical analysis of the data should be a pplied to both the full and truncated data set so that the effect of discarding observations may be assessed. If scie ntific reasoning does not explain the outlier, it should not be discarded from the data set.

If any data points are found to be statistical outliers through the use of a statistical test, this information will need to be documented along w ith the analysis of the data set, regardless of whether any data points are discarded. If no data points are discarded, document the identification of any statistical outliers by documenti ng the statistical test performed and the possible scientific reasons invest igated. If any data points are discarded, document each data point, the statistical test performed, the scien tific reason for discarding each data point, and the effect on the analysis of deleting the data points. This informati on is critical for effective peer review.

4.4.2 Selection of a Statisti cal Test for Outliers



## There are several statistical tests for determining whether or not one or more observations are statistical outlie rs. Step by step directions for implementing some of these tests are described in Sections 4.4.3 through 4.4.6. Section 4.4. 7 describes statistical tests for multivariate outliers.

If the data are approximately normally dist ributed, this guidance recommends Rosner's test when the sample size is greater than 25 and the Extreme Value test when the sample size is less than 25. If only one outlier is suspected, then the Discordance test may be substituted for either of these tests. If the data are not normally distributed, or if the da ta cannot be transformed

### EPA QA/G-9S 117 February 2006 

## so that the transformed data are normally distri buted, then the analyst should either apply a nonparametric test (such as Walsh's test) or consult a statistician. A summary of these recommendations is contained in Table 4-3.

Table 4-3. Recommendations for Select ing a Statistical Test for Outliers

Sample

Size Test Section Assumes Normality Multiple Outliers

n ≤ 25 Extreme Value Test 4.4.3 Yes No/Yes

n ≤ 50 Discordance Test 4.4.4 Yes No

n ≥ 25 Rosner's Test 4.4.5 Yes Yes

n ≥ 50 Walsh's Test 4.4.6 No Yes



## 4.4.3 Extreme Value Test (Dixon's Test)



## Dixon's Extreme Value test can be used to te st for statistical outliers when the sample size is less than or equal to 25. This test c onsiders both extreme values that are much smaller than the rest of the data (case 1) and extreme values that are much larger than the rest of the data (case 2). This test assumes that the data wit hout the suspected outlier are normally distributed; therefore, it is necessary to pe rform a test for normality on the data without the suspected outlier before applying this test. If th e data are not normally distributed, then either transform the data, apply a different test, or consult a statistician. Directi ons for the Extreme Value test are contained in Box 4-15; an example of this test is contained in Box 4-16.

This guidance recommends using this test when only one outlier is suspected in the data. If more than one outlier is su spected, the Extreme Value test ma y lead to masking where two or more outliers close in value "hid e" one another. Therefore, if the analyst decides to use the Extreme Value test for multiple outliers, apply the test to the least extreme value first.

4.4.4 Discordance Test



## The Discordance test can be used to test if one extreme value is an outlier. The Discordance test assumes that th e data without the suspected ou tlier are approximately normally distributed. Therefore, it is necessary to ch eck for normality before applying this test. Directions and an example of the Discordance test are contained in Box 4-17 and Box 4-18.

### EPA QA/G-9S 118 February 2006 Box 4-15: Directions for the Ex treme Value Test (Dixon's Test)

Let X (1) , X (2) , . . . , X ( n ) represent the data ordered from smallest to largest. Check that the data without the suspect outlier are normally distributed, using one of the methods of Section 4.2. If normality fails, either transform the data or apply a different outlier test.

STEP 1. Null Hypothesis: H 0 : There are no outliers in the data.

STEP 2. Alternative Hypothesis: i) H A : X (1) is an outlier. ii) H A : X ( n ) is an outlier.

STEP 3. Test Statistic: i) Compute the test statistic C , where

3 ≤ n ≤ 7 8 ≤ n ≤ 10 11 ≤ n ≤ 13 14 ≤ n ≤ 25


### ( ) () ( ) () 1 1 2 X X X X C

n − −

= 

### ( ) () ( ) () 1 1 1 2 X X X X C

n − −

=

− 

### ( ) () ( ) () 1 1 1 3 X X X X C

n − −

=

− 

### ( ) () ( ) () 1 2 1 3 X X X

X C

n − −

=

−

ii) Compute the test statistic C , where

3 ≤ n ≤ 7 8 ≤ n ≤ 10 11 ≤ n ≤ 13 14 ≤ n ≤ 25



### () ( )

( ) () 1 1 X X X X C

n n n

−

−

= − 

### () ( ) () () 2 1 X X X X C

n n n

−

−

= − 

### () ( ) () () 2 2 X X X X C

n n n

−

−

= − 

### () ( ) () () 3 2 X X X X C

n n n

−

−

= −

STEP 4. a) Critical Value: Use Table A-4 to find d α

. STEP 5. a) Conclusion: If C > d α

, then reject the null hypothesis that there are no outliers in the data.

Box 4-16: An Example of the Extreme Value Test (Dixon's Test)
The data in order of magnitude from smallest to largest are (in ppm):

82.39, 86.62, 91.72, 98.37, 103.46, 104.93, 105.52, 108.21, 113.23, 150.55.

As the value 150.55 is much larger than the other values, it is suspected that this data point might be an outlier. The Studentized Range test (Section 4.2.6) shows that there is no reason to suspect that the data are not normally distributed.

STEP 1. Null Hypothesis: H 0 : There are no outliers in the data.

STEP 2. Alternative Hypothesis: H A : X (n) is an outlier.

STEP 3. Test Statistic: Since n = 10, 584 0 62 86 55 150 23 113 55 150

) 2 ( ) ( ) 1 ( ) (

.

. . . . X - X X - X C

n n n

=

− −
= = −

.

STEP 4. a) Critical Value: Using Table A-4, d 0.05 = 0.477.

STEP 5. a) Conclusion: Since C = 0.584 > 0.477 = d 0.05 , we reject the null hypothesis of no outliers in the data.

### EPA QA/G-9S 119 February 2006 Box 4-17: Directions fo r the Discordance Test

Let X (1) , X (2) , . . . , X ( n ) represent the data ordered from smallest to largest. Check that the data without the suspect outlier are normally distributed, using one of the methods of Section 4.2. If normality fails, either transform the data or apply a different outlier test.

COMPUTATIONS: Compute the sample mean, X , and the sample standard deviation, s

. STEP 1. Null Hypothesis: H 0 : There are no outliers in the data.

STEP 2. Alternative Hypothesis: i) H A : X (1) is an outlier. ii) H A : X ( n ) is an outlier.

STEP 3. Test Statistic: i) 

### ( )

s X X D 1 −

= ii) 

### ( )

s X X D n −

=

STEP 4. a) Critical Value: Use Table A-5 to find d α

. STEP 5. a) Conclusion: If D > d α

, then reject the null hypothesis that there are no outliers in the data.

Box 4-18: An Example of the Discordance Test
The data in order of magnitude from smallest to largest are (in ppm):

82.39, 86.62, 91.72, 98.37, 103.46, 104.93, 105.52, 108.21, 113.23, 150.55.

It is suspected that the data point 150.55 might be an outlier. The Studentized Range test (Section 4.2.6) shows that there is no reason to suspect that the data are not normally distributed.

COMPUTATIONS: X = 104.5 ppm and s = 18.922 ppm.

STEP 1. Null Hypothesis: H 0 : There are no outliers in the data.

STEP 2. Alternative Hypothesis: H A : X (n) is an outlier.

STEP 3. Test Statistic: 

### ( ) 43

. 2 922

. 18 5

. 104 55

. 150

= −
= −

= s X

X D n

STEP 4. a) Critical Value: Using Table A-5, d 0.05 = 2.176.

STEP 5. a) Conclusion: Since D = 2.43 > 2.176 = d α

, we reject the null hypothesis of no outliers.



## 4.4.5 Rosner's Test


## A parametric test developed by Rosner can be used to detect up to 10 outliers for sample sizes of 25 or more. This test assumes that th e data without the suspected outliers are normally distributed. Therefore, it is necessary to perform a test for normality before applying this test. If the data are not normally distributed, then either transform the data, appl y a different test, or consult a statistician. Directions for Rosner's test are contained in Box 4-19 and an example is contained in Box 4-20.

### EPA QA/G-9S 120 February 2006 Box 4-19: Directions for Ro sner's Test for Outliers

Let X 1 , X 2 , . . . , X n represent the ordered data points. By ins pection, identify the maximum number of possible outliers, 1 ≤ r 0 ≤ 10. Check that the data are normally distributed without the suspected outlier(s) using one of the methods of Section 4.2.

COMPUTATIONS: Using the entire data set, compute the sample mean, ) 0 ( X , and the sample standard deviation, ) 0 ( s . Determine the observation farthest from X ( ) 0 and label it y ( ) 0 . Delete y ( ) 0 from the data and compute the sample mean, X ( ) 1 , and the sample standard deviation, s ( ) 1 . Now determine the observation
farthest from X ( ) 1 and label it y ( ) 1 . Delete ) 1 ( y from the data and compute the sample mean, ) 2 ( X , and the sample standard deviation, ) 2 ( s . Continue this process until r 0 extreme values have been eliminated. After this process the analyst should have: 

# { } { } 

# { } ) 1 ( ) 1 ( ) 1 ( ) 1 ( ) 1 ( ) 1 ( ) 0 ( ) 0 ( ) 0 ( 0 0 0
, , , , , , , , , − − − r r r y s X y s X y s X

K STEP 1. Null Hypothesis: H 0 : There are no outliers in the data.

STEP 2. Alternative Hypothesis: H A : There are as many as r 0 outliers in the data.

Steps 3 through 5 of this test are iterative. First, test if there are r 0 outliers. If not, then test if there are r 0 - 1 outliers. Continue, until it is determined that either there are a certain number of outliers or that there are no outliers.

STEP 3. Test Statistic: r r r

r R y X

s = − − −

− () ()

( ) 1 1

1 , where r starts at r 0 and runs through 1.

STEP 4. a) Critical Value: Use Table A-6 to find λ r . STEP 5. a) Conclusion: If R r > λ r , then conclude that there are r outliers. Otherwise, return to step 3 to test for r -1 outliers.



## 4.4.6 Walsh's Test


## A nonparametric test was developed by Walsh to detect multiple outliers in a data set. This test requires a large sample size: n > 220 for a significance level of α = 0.05, and n > 60 for a significance level of α = 0.10. However, since the test is a nonparametric test, it may be used when the data is not normally distributed. It sh ould be noted that this test is used infrequently with environmental data. Directions for the test for large sample sizes are given in Box 4-21.

4.4.7 Multivariate Outliers


## Multivariate analysis, such as factor analys is and principal components analysis, involves the analysis of several variables simultaneously. Outliers in multivariate analysis are then values that are extreme in relationship to either one or more variables. As the number of variables increases, identifying potential outliers using gr aphical representations b ecomes more difficult. In addition, special procedures are required to te st for multivariate outliers and details of these procedures are beyond the scope of this guidance.

### EPA QA/G-9S 121 February 2006 Box 4-20: An Example of Ro sner's Test for Outliers
Consider the following 32 ordered data points (in ppm): 2.07, 40.55, 84.15, 88.41, 98.84, 100.54, 115.37, 121.19, 122.08, 125.84, 129.47, 131.90, 149.06, 163.89, 166.77, 171.91, 178.23, 181.64, 185.47, 187.64, 193.73, 199.74, 209.43, 213.29, 223.14, 225.12, 232.72, 233.21, 239.97, 251.12, 275.36, 395.67.

A normal probability plot excluding the suspected outliers shows that there is no reason to suspect that the data is not normally distributed. In addition, this graph identified four potential outliers: 2.07, 40.55, 275.36, and 395.67. Rosner's test at a significance level of 0.05 will be applied to see if there are r 0 = 4 or fewer outliers.

COMPUTATIONS: The summary statistics and suspected outliers are listed in the following table:

i X i ( ) s i ( ) y i ( )

0 169.923 75.133 395.67 1 162.640 63.872 2.07 2 167.993 57.460 40.55 3 172.387 53.099 275.36

STEP 1. Null Hypothesis: H 0 : There are no outliers in the data.

STEP 2. Alternative Hypothesis: H A : There are as many as 4 outliers in the data.

STEP 3. Test Statistic: 939

. 1 099

. 53 387

. 172 36

. 275 ) 3 ( ) 3 ( ) 3 (

4 = −

= −

= s X y R

STEP 4. a) Critical Value: Using Table A-6, λ 4 = 2.89.

STEP 5. a) Conclusion: Since R 4 = 1.939 < 2.89 = λ 4 , there aren’t 4 outliers. Therefore, test if there are 3 outliers by computing

STEP 3. Test Statistic: 218

. 2 46

. 57 993

. 167 55

. 40 ) 2 ( ) 2 ( ) 2 (

3 = −

= −

= s X y R

STEP 4. a) Critical Value: Using Table A-6, λ 3 = 2.91.

STEP 5. a) Conclusion: Since R 3 = 2.218 < 2.91 = λ 3 , there aren’t 3 outliers. Therefore, test if there are 2 outliers by computing

STEP 3. Test Statistic: 514

. 2 872

. 63 64

. 162 07

. 2 ) 1 ( ) 1 ( ) 1 (

2 = −

= −

= s X y R

STEP 4. a) Critical Value: Using Table A-6, λ 2 = 2.92.

STEP 5. a) Conclusion: Since R 2 = 2.514 < 2.92 = λ 2 , there aren’t 2 outliers. Therefore, test if there are 1 outlier by computing

STEP 3. Test Statistic: 005

. 3 133

. 75 923

. 169 67

. 395 ) 0 ( ) 0 ( ) 0 (

1 = −

= −

= s X y R

STEP 4. a) Critical Value: Using Table A-6, λ 1 = 2.94.

STEP 5. a) Conclusion: Since R 1 = 3.005 > 2.94 = λ 1 , we conclude there is one outlier, 395.67.

### EPA QA/G-9S 122 February 2006 Box 4-21: Directions for Walsh' s Test for Large Sample Sizes

Let X (1) , X (2) , . . ., X ( n ) represent the ordered data. If n ≤ 60, do not apply this test. If 60 < n ≤ 220, then α = 0.10. If n > 220, then α = 0.05.

COMPUTATIONS: Identify the number of possible outliers, r . Compute 

# ( ) n c 2 ceiling = , c r k +

= , α 1 2

= b , and 

# ( ) ( )

1 1 1

2 2

− − −

− +

= b c c b c b a

where ceiling( ) indicates rounding the value to the next largest integer.

STEP 1. The r smallest points are outliers (with an α level of significance) if 

## ( ) 0 1 ) ( ) 1 ( ) ( < + +

− + k r r aX X a X

STEP 2. The r largest points are outliers (with an α level of significance) if 

## ( ) 0 1 ) 1 ( ) ( ) 1 ( > + +

−

− +

− − + k n r n r n aX X a X

STEP 3. If both of the inequalities are true, then both small and large outliers are indicated.



## 4.5 TESTS FOR DISPERSIONS


## Many statistical tests make assumptions about the dispersion (variance) of data. This section considers some of the most commonly us ed statistical tests for comparing variances. Section 4.5.1 constructs a confid ence interval for a population vari ance. Section 4.5.2 provides a test for comparing two population variances. Se ction 4.5.3 (Bartlett’s test) and Section 4.5.4 (Levene’s test) describe tests that compare two or more population variances. The analyst should be aware that many statistic al tests only require the approxi mate equality of variances and that many of the tests remain va lid unless there is gro ss inequality in the population variances.

4.5.1 Confidence Intervals for a Single Variance



## This section discusses confidence intervals fo r a single variance or st andard deviation. The method described in Box 4-22 can be used to find a two-sided 100(1- α )% confidence interval. The upper end poi nt of a two-sided 100(1- α )% confidence interval is also a 100(1- α /2)% upper confidence limit , and the lower end point is also a 100(1- α /2)% lower confidence limit . Since the standard deviation is the s quare root of the variance, a confidence interval for the variance can be converted to a confidence interval for the standard deviation by taking the square roots of the endpoints of the interval. The confiden ce interval procedure assumes the data are a random sample from a normally distributed populat ion and can be highly sensitive to outliers or to departures from normality.

### EPA QA/G-9S 123 February 2006 Box 4-22: Directions for Constructing Confidence Intervals and Confidence Limits for the Sample Variance and Standard Deviation with an Example

Let X 1 , X 2 , . . . , X n represent the n data points.

COMPUTATIONS: Calculate the sample variance s 2 (Section 2.2.3).

A 100(1- α )% confidence interval for the population variance, σ 2 , is 

## ( ) ( ) ⎪ ⎭ ⎪ ⎬ ⎫
⎪ ⎩ ⎪ ⎨ ⎧

− −

−

− − 2 2 /

, 1 2

2 2 / 1

, 1 2 1

, 1

α

α χ χ n

n s n s n , where Table A-9 is

used to find 2 2 /

, 1 α χ

−

n and 2 2 / 1

, 1 α χ

− − n . A 100(1- α )% confidence interval for the population standard deviation is the square root of the interval above.

Example : Ten samples were analyzed for lead (in ppb): 46.4, 46.1, 45.8, 47, 46.1, 45.9, 45.8, 46.9, 45.2, 46.

COMPUTATIONS: The sample variance is s 2 = 0.286.

A 95% confidence interval for the population variance, σ 2 , is


## ( ) ( ) { } 953

. 0

, 135

. 0 70

. 2 286

. 0 9

, 02

. 19 286

. 0 9 286

. 0 1 10

, 286

. 0 1 10
2 025

. 0

, 9 2 975

. 0

, 9 = ⎭ ⎬ ⎫ ⎩ ⎨ ⎧

⋅ ⋅

= ⎪ ⎭ ⎪ ⎬ ⎫
⎪ ⎩ ⎪ ⎨ ⎧

⋅ − ⋅ −
χ χ



## 4.5.2 The F -Test for the Equality of Two Variances



## An F -test may be used to compare two population variances. The assumptions underlying the F -test are that the two samples are inde pendent random samples from two normal populations. The F -test for equality of variances is highly sensitive to departures from normality. Directions for implementing an F -test with an example are given in Box 4-23.

4.5.3 Bartlett's Test for the Equa lity of Two or More Variances


## Bartlett's test is a means of testing whethe r two or more population variances of normal distributions are equal. In the case of only two variances, Bartlett 's test is equivalent to the F

test. Often in practice unequal variances and non-normality occur together and Bartlett's test is itself sensitive to departures from norma lity. With long-tailed distribu tions, the test too often rejects equality of the variances. Rejecting the equality of variances does not mean that one or more is significantly different from the ot hers. It simply implies the va riances are unequal as a group. Directions for Bartlett's test are given in Box 4-24 and an example is given in Box 4-25.

4.5.4 Levene's Test for the Equality of Two or More Variances



## Levene's test provides an alternative to Bart lett's test for comparing population variances. Levene's test is less sensitive to departures from normality than Bartlett's test and has greater power than Bartlett's for non-normal data. In additi on, Levene's test has power nearly as great as Bartlett's test for normally distributed data. Di rections and an example of Levene's test are contained in Box 4-26 and Box 4-27, respectively.

### EPA QA/G-9S 124 February 2006 Box 4-23: Directions for an F -Test to Compare Two Populati on Variances with an Example

Let X 1 , X 2 , . . . , X m represent the m data points from population 1 and Y 1 , Y 2 , . . . , Y n represent the n data points from population 2.

COMPUTATIONS: Calculate the sample variances s X 2 and s Y 2 (Section 2.2.3).

STEP 1. Null Hypothesis: H 0 : 2 2 Y X σ σ =

.

STEP 2. Alternative Hypothesis: H A : 2 2 Y X σ σ ≠

.

STEP 3. Test Statistic: 

## ( ) ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

= = 2 2

2 2

0 , max , max
X Y

Y X Y X s s

s s F F F . If F 0 = F X , then let k = m – 1 and

q = n – 1. If F 0 = F Y , then let k = n – 1 and q = m – 1.

STEP 4. a) Critical Value: Use Table A-10 to find F k

, q , 1- α /2

. STEP 4. b) p-value: Use Table A-10 to find 2

⋅ P( F k

, q > F 0 ).

STEP 5. a) Conclusion: If F 0 > F k

, q , 1- α /2 , then reject the null hypothesis of equal population variances.

STEP 5. a) Conclusion: If p-value < α , then reject the null hypothesis of equal population variances.

Example : Manganese concentrations (in ppm) were collected from 2 wells. A 0.05-level F -test will be used to test if the population variances are equal.

well X : 50, 73, 244, 202 well Y : 272, 171, 32, 250, 53

COMPUTATIONS: The sample variances are s X 2 = 9076 and s Y 2 = 12125.

STEP 1. Null Hypothesis: H 0 : 2 2 Y X σ σ =

.

STEP 2. Alternative Hypothesis: H A : 2 2 Y X σ σ ≠

.

STEP 3. Test Statistic: 

## ( ) 336

. 1 9076 12125

, 12125 9076 max , max , max 2 2

2 2

0 = ⎟ ⎠ ⎞ ⎜ ⎝ ⎛

= ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

= = X Y

Y X Y X s s

s s F F F

.

Also, k = 5 – 1 = 4 and q = 4 – 1 = 3.

STEP 4. a) Critical Value: Using Table A-10, F 4, 3, 0.975 = 15.1.

STEP 4. b) p-value: Using Table A-10, p-value = 2

⋅ P( F 4, 3 > 1.336) which exceeds 0.20. (Using software, the exact p-value is 0.8454)

STEP 5. a) Conclusion: Since F 0 = 1.336 < 15.1 = F 4, 3, 0.975 , we fail to reject the null hypothesis of equal population variances.

STEP 5. a) Conclusion: Since p-value = 0.8454 > 0.05 = α , we fail to reject the null hypothesis of equal population variances.

### EPA QA/G-9S 125 February 2006 Box 4-24: Directions for Bartlett's Test

Consider k independent random samples with a sample size of n i for the i th group and let k n n N + + = L 1 .
COMPUTATIONS: For each of the k groups, calculate the sample variance, 2 i s (Section 2.2.3). Also, compute

the pooled variance: 

## ( ) 

# ∑

= −

− = k

i i i p s n k N s 1 2

2 1 1

.
STEP 1. Null Hypothesis: H 0 : 2 2 1 k σ σ = = L (The variances are equal)
STEP 2. Alternative Hypothesis: H A : The variances are not equal.

STEP 3. Test Statistic: 

## ( ) 

# ( ) ( ) 

# ( ) s n s k N B i i k

i p 2

1 2 0 ln 1 ln

⋅ − − ⋅ − = 

# ∑

= .

STEP 4. a) Critical Value: Use Table A-9 to find 2 1

, 1 α χ

− − k . STEP 4. b) p-value: Use Table A-9 to find 

# ( )
0 2 1 B P k >

− χ

. STEP 5. a) Conclusion: If B 0 > 2 1

, 1 α χ

− − k , then reject the null hypothesis of equal variances.
STEP 5. a) Conclusion: If p-value < α , then reject the null hypothesis of equal variances.

Box 4-25: An Example of Bartlett's Test
Manganese concentrations were collected from 6 wells over a 4 month period. It is important to determine if the variances of the six wells are equal. Bartlett's test at a significance level of 0.05 will be used.

Sampling Date Well 1 Well 2 Well 3 Well 4 Well 5 Well 6 January 1 50 272 February 1 73 171 68 March 1 244 46 32 34 48 991 April 1 202 77 53 3940 54 54

n i

( N =17) 4 2 4 2 2 3 X

i 142.25 61.50 132 1987 51 371

s i

2 9076.25 480.5 12454 7628418 18 288349

COMPUTATIONS: The pooled variance is 

## () () 

## [ ] 84

. 751836 288349 1 3 25

. 9076 1 4 6 17 1 2
= ⋅ − + + ⋅ −

− = L s p

STEP 1. Null Hypothesis: H 0 : 2 2 1 k σ σ = = L
. STEP 2. Alternative Hypothesis: H A : The variances are not equal.

STEP 3. Test Statistic: 

## ( ) ( ) ( ) ( ) ( )( ) 

## [ ] 288349 ln 1 3 25

. 9076 ln 1 4 84

. 751836 ln 6 17 0 ⋅ − + + ⋅ − − ⋅ − = L B 15

. 43 = STEP 4. a) Critical Value: Using Table A-9, 07

. 11 2 95

. 0

, 5 = χ

.
STEP 4. b) p-value: Using Table A-9, p-value = 

# ( ) 15

. 43 2 5 > χ P < 0.005. (Using statistical software, the exact p-value is almost 0).

STEP 5. a) Conclusion: Since B 0 = 43.15 > 11.07 = 2 1

, 1 α χ

− − k , we reject the null hypothesis of equal population variances. STEP 5. a) Conclusion: Since p-value = almost 0 < 0.05 = α , we reject the null hypothesis of equal population variances.

### EPA QA/G-9S 126 February 2006 Box 4-26: Directions for Levene's Test

Consider k independent random samples with a sample size of n i for the i th group and let k n n N + + = L 1 .

COMPUTATIONS: For each of the k groups, calculate the group mean, X i . Then compute the absolute
residuals, ij ij i z X X = − , where X ij represents the j th value of the i th group. For each group, calculate the mean

absolute residual, 

# ∑

= = i n j ij i i z n z 1 1 . Next, calculate the overall mean absolute residual,

i i k

i ij n

j k

i z n N z N 1 z i 

# ∑ ∑ ∑

= = = = =
1 1 1 1

.

Finally, compute the following sums of squares for the absolute residuals:

2

1 1 2 z N z SS k

i n

j ij TOTAL i

⋅ − = 

# ∑∑

= = , 2

1 2 z N z n SS k

i i i GROUPS ⋅ − = 

# ∑

= , and GROUPS TOTAL ERROR SS SS SS

− =

.

STEP 1. Null Hypothesis: H 0 : 2 2 1 k σ σ = = L
.
STEP 2. Alternative Hypothesis: H A : The variances are not equal.

STEP 3. Test Statistic: 

## ( ) ( ) k N / SS k / SS F
ERROR GROUPS

− −

= 1 0 .

STEP 4. a) Critical Value: Use Table A-10 to find F k -1, N

- k , 1- α . STEP 4. b) p-value: Use Table A-10 to find P( F k -1, N

- k > F 0 ).

STEP 5. a) Conclusion: If F 0 > F k -1, N

- k , 1- α , then reject the null hypothesis of equal population variances.

STEP 5. a) Conclusion: If p-value < α , then reject the null hypothesis of equal population variances.



## 4.6 TRANSFORMATIONS



## Most statistical tests and pro cedures contain assumptions ab out the data. For example, some common assumptions are that the data is normally distributed; va riance components of a statistical model are additive; two independent data sets have equal variance; and a data set has no trends over time or space. If the data do not satisfy such assumptions, then the results of a statistical procedure or test may be biased or incorrect. Fortunately, data that do not satisfy statistical assumptions may often be converted or transformed mathematically into a form that allows standard statistical tests to perform adequately.

It is not recommended to transform data for estimation purpos es. Transforming, estimating, and then transforming the estimate back to the original domain will, in general, lead to biased estimates. A better approach to estim ation here is to use a nonparametric procedure.

### EPA QA/G-9S 127 February 2006 Box 4-27: An Example of Levene's Test
Four months of data on arsenic concentration (ppm) were collected from six wells at a Superfund site. This data set is shown in the table below. Before analyzing this da ta, it is important to determine if the variances of the six wells are equal. Levene's test at a significance level of 0.05 will be used to make this determination.

COMPUTATIONS: The table below contains the data values, the absolute residuals (labeled res), the sample means and the absolute residual means.

well 1 well 2 well 3 well 4 well 5 well 6 Month value res value res value res value res value res value res

1 22.90 6.43 2.00 13.76 2.0 27.6 7.84 3.42 24.90 11.41 0.34 1.95

2 3.09 13.38 1.25 14.51 109.4 79.8 9.30 1.96 1.30 12.19 4.78 2.49

3 35.70 19.23 7.80 7.96 4.5 25.1 25.90 14.64 0.75 12.74 2.85 0.56

4 4.18 12.29 52.00 36.24 2.5 27.1 2.00 9.26 27.00 13.51 1.20 1.09

47

. 16 1 = X 76

. 15 2 = X 60

. 29 3 = X 26

. 11 4 = X 49

. 13 5 = X 29

. 2 6 = X

83

. 12 1 = z 12

. 18 2 = z 90

. 39 3 = z 32

. 7 4 = z 46

. 12 5 = z 52

. 1 6 = z

The overall absolute residual mean is z = (12.83 + 18.12 + 39.9 + 7.32 + 12.46 + 1.52)/6 = 15.36.

The sum of squares are: SS TOTAL = 630089
. , SS WELLS = 352290 . , and SS ERROR = 277799

. .
STEP 1. Null Hypothesis: H 0 : 2 6 2 1 σ σ = = L
. STEP 2. Alternative Hypothesis: H A : The variances are not equal.

STEP 3. Test Statistic: 

## ( ) ( ) ( ) ( ) 56

. 4 6 24 99

. 2777 1 6 9

. 3522 1 0 =

− −
=

− −
= k N / SS k / SS F
ERROR GROUPS

.

STEP 4. a) Critical Value: Using Table A-10, F 5, 18, 0.95 = 2.77.

STEP 4. b) p-value: Using Table A-10, 0.01 < p-value < 0.001. Using software, P( F 5, 18 > 4.56) = 0.0073.

STEP 5. a) Conclusion: Since F 0 = 4.56 > 2.77 = F 5, 18, 0.95 , we reject the null hypothesis of equal population variances.

STEP 5. a) Conclusion: Since p-value = 0.0073 < 0.05 = α , we reject the null hypothesis of equal population variances.



## 4.6.1 Types of Data Transformations


## Any mathematical function that is applied to every point in a data set is called a transformation. Some commonly used transformations include:

Logarithmic ( Log X or Ln X): This transformation is best suited for data that is rightskewed which may occur when the measurem ent follows a lognormal distribution. The transformation is also helpful when the variance at each level of the data is proportional to the square of the mean of the data points at that leve l. For example, if the variance of data collected around 50 ppm is approximately 250, but the va riance of data collected around 100 ppm is approximately 1000, then a logarithmic transforma tion may be useful. Th is situation is often

### EPA QA/G-9S 128 February 2006 

## characterized by having a constant coefficient of variation (sta ndard deviation divided by the mean) over all possible data values.

If some of the original values are zero or negative, it is customary to add a small quantity to make the data value non-zero since the logarith m of zero or a negative number does not exist. The size of the small quantity depends on the magnitude of the non-zero data. As a working point, a value of one tenth the smallest non-zero value could be selected. It does not matter whether a natural (ln) or base 10 (log) transforma tion is used because the two transformations are related by the expression ln( X

) = 2.303 log( X

). Directions for applying a logarithmic transformation with an example are given in Box 4-28.

Square Root ( X 

## ): This transformation may be used when dealing with small whole numbers, such as bacteriological counts, or the o ccurrence of rare events, su ch as violations of a standard over the course of a year. The underlyi ng assumption is that the original data follow a Poisson-like distribution in which case the mean a nd variance of the data ar e equal. It should be noted that the square root tran sformation overcorrects when very small values and zeros appear in the original data. In these cases, 1 + X 

## is often used as a transformation.

Inverse Sine ( Arcsine X): This transformation may be used for binomial proportions based on count data to achieve stability in variance. The resulting transformed data are expressed in radians (angular degr ees). Special tables must be used to transform the proportions into degrees.

Box-Cox Transformations: This transformation is a complex power transformation that takes the original data and raises each data observation to the power lambda ( λ ). A logarithmic transformation is a special case of the Box-Cox transformation. The rationale is to find λ such that the transformed data have the best possibl e additive model for the variance structure, the errors are normally distributed, and the varian ce is as constant as possible over all possible concentration values. The Maximum Li kelihood technique is used to find λ such that the residual error from fitting the theorized model is minimized. In practice, the exact value of λ is often rounded to a convenient value for ease in interpretati on (for example, λ = 0.45 would be rounded to 0.5 as it would then ha ve the interpretation of a square root transform). One of the drawbacks of the Box-Cox transformation is the difficulty in physic ally interpreting the transformed data.

4.6.2 Reasons for Transforming Data


## By transforming the data, assumptions that are not satisfied in the original data can be satisfied by the transformed data. For instance, a right-skewed distribution can be transformed to be approximately Gaussian (norma l) by using a logarithmic or squa re-root transformation. Then the normal-theory procedures can be applied to the transformed data. If data are lognormally distributed, then apply procedures to logarithms of the data. However, selecting the correct transformation may be difficult. If standard transf ormations do not apply, it is suggested that the data user consult a statistician.

### EPA QA/G-9S 129 February 2006 Box 4-28: Directions for Tran sforming Data and an Example

Let X 1 , X 2 , . . . , X n represent the n data points. To apply a transformation, simply apply the transforming function to each data point. When a transformation is implemented to make the data satisfy some statistical assumption, it will need to be verified that the transformed data satisfy this assumption.

Example: Transforming Lognormal Data

A logarithmic transformation is particularly useful for pollution data. Pollution data are often right-skewed, thus the log-transformed data will tend to be symmetric. Consider the data set shown below with 15 data points. A histogram of this data below shows that the data are possibly lognormally distributed. The transformed data are shown in column 2. A histogram of the transformed data below shows that the transformed data appear to be normally distributed.

Observed Transformed Observed Transformed X → ln( X ) X → ln( X ) 0.22 → -1.51 0.47 → -0.76 3.48 → 1.25 0.67 → -0.40 6.67 → 1.90 0.75 → -0.29 2.53 → 0.93 0.60 → -0.51 1.11 → 0.10 0.99 → -0.01 0.33 → -1.11 0.90 → -0.11 1.64 → 0.50 0.26 → -1.35 1.37 → 0.31

-1 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

Observed ValuesNumber of Observations

-4 -3 -2 -1 0 1 2 3 4 0 1 2 3 4 5 6 7

Transformed ValuesNumber of Observations



## While transformations are useful for dealing with data that do not satisfy statistical assumptions, they can also be used for various other purposes. For example, transformations are useful for consolidating data that may be spread out or that have several extreme values. In addition, transformations can be used to derive a linear relationship betw een two variables on the newly transformed data, so that lin ear regression analysis can be appl ied. They can also be used to efficiently estimate quantities such as the m ean and variance of a l ognormal distribution.

### EPA QA/G-9S 130 February 2006 

## Transformations may also make the analysis of da ta easier by changing the scale into one that is more familiar or easier to work with.

Once the data have been transformed, all st atistical analysis should be performed on the transformed data. Rarely should an attempt made to transform the da ta back to the original form because this can lead to biased estimates. Fo r example, estimating quantities such as means, variances, confidence limits, and re gression coefficients in the tr ansformed scale typically leads to biased estimates when transformed back into or iginal scale. However, it may be difficult to understand or apply results of statis tical analysis expressed in the transformed scale. Therefore, if the transformed data do not give noticeable bene fits to the analysis, it is better to use the original data. There is no point in working w ith transformed data unle ss it adds value to the analysis.

4.7 VALUES BELOW DE TECTION LIMITS


## Data generated from chemical analysis may fall below the detection limit (DL) of the analytical procedure. These measurement data are generally described as non-detects rather than as zero or not present and the appropriate limit of detection is usually reported. In cases where measurement data are described as non-det ects, the concentration of the chemical is unknown although it lies somewhere between zero and the detection limit. Data that includes both detected and non-detected re sults are called censored data in the statistical literature.

There are a variety of ways to evaluate da ta that includes values below the detection limit. However, there are no general procedures th at are applicable in all cases. Some general guidelines are presented in Table 4-4. Alt hough these guidelines are usually adequate, they should be implemented cautiously.

Table 4-4. Guidelines for Analyzing Data with Non-Detects

Approximate Percentage of Non-Detects Section Statistical Analysis Method

< 15% 4.7.1 Replace non-detects with 0, DL/2, DL., Cohen’s Method.

15% - 50% 4.7.2 Trimmed mean, Cohen's Method, Winsorized mean and standard deviation. > 50% - 90% 4.7.3 Tests for proportions (Section 3.2.1.5)



## All of the suggested procedures for anal yzing data with non-detects depend upon the amount of data below the detec tion limit. For relatively small amounts below detection limit values, replacing the non-detects with a small number and proceeding with the usual analysis may be satisfactory depending on the purpose of th e analysis. For moderate amounts of data below the detection limit, a more detailed adjust ment is appropriate. In situations where relatively large amounts of data fall below the de tection limit, one may need only to consider whether or not the chemical was detected above some level. Table 4-4 provides percentages to

### EPA QA/G-9S 131 February 2006 

## assist the user in evaluating thei r particular situation. However, it should be recognized that these percentages are not hard and fast rules.

In addition, sample size influences which proc edures should be used to evaluate the data. For example, the case where 1 sample out of 4 is not detected should be treated differently from the case where 25 samples out of 100 are non-detects. Therefore, this guidance suggests that the data analyst consult a statistician for the most a ppropriate way to evaluate data containing values below the detection level.

4.7.1 Approximately less than 15% Non- detects - Substitution Methods



## If a small proportion of the observations are non- detects, then these may be replaced with a small number, usually DL/2, and the usual analysis performed. Alternative substitution values are 0 (see Aitchison’s Method below) or the dete ction limit. It should be noted that Cohen’s Method (section 4.7.2.1) will also work with small amounts of non-detects.

4.7.1.1 Aitchison’s Method


## Later adjustments to the mean and variance a ssume that the data values really were present but could not be recorded since they were below the detection limit. However, there are cases where the data values are below the detect ion limit because they are actually zero, i.e., the contaminant or chemical of concern being entirely absent. Such data sets typically contain a mixture of zero values and present, but nondete cted values. Aitchison’s Method is simply adjustment formulas for the mean and variance if 0 values are substituted for non-detects. Directions for Aitchison’s method are containe d in Box 4-29 with an example in Box 4-30.

Box 4-29: Directions for Aitchison’ s Method to Adjust Means and Variances

Let X 1 , X 2 , . ., X m , X m +1 ,. . . , X n represent the data points where the first m values are above the detection limit and the remaining n
- m data points are below the detection limit.

COMPUTATIONS: Using the data above the detection level, compute the sample mean and sample variance: 

# ∑

= = m

i i d X X 1 and ⎪ ⎭ ⎪ ⎬ ⎫
⎪ ⎩ ⎪ ⎨ ⎧ ⎟ ⎠ ⎞ ⎜ ⎝ ⎛

−

− = 

# ∑ ∑

= = 2

1 1 2 2 1 1 1 m

i i m

i i d X m X m s

.

Compute the adjusted sample mean and sample variance, d X n m X = and 

## ( ) ( ) 2 2 2 1 1 1 d d X n n m

n m s n m s

− − +

− −
=

.

### EPA QA/G-9S 132 February 2006 Box 4-30: An Example of Aitchison’s Method
The following data consist of 10 Methylene Chloride samples: 1.9,1.3, <1, 2.0, 1.9, <1, <1, <1, 1.6, 1.7. There are 6 values above the detection limit and 4 below, so m = 6 and n
- m = 4. Aitchison’s method will be used to estimate the mean and sample variance of this data.

COMPUTATIONS: Compute the mean and variance for the 6 values above the detection limit

733

. 1 = d X and 0667

. 0 2

= d s

.

The adjusted sample mean and sample variance are: 04

. 1 733

. 1 10 6

= ⋅ = X and



## ( ) 8382

. 0 7333

. 1 1 10 10 4 6 0667

. 0 1 10 1 6 2 2

= ⋅

− ⋅ ⋅ + ⋅

− −
= s

.



## 4.7.2 Between Approximately 15% - 50% Non-detects

4.7.2.1 Cohen's Method


## Cohen's method provides adjusted estimates of the sample me an and standard deviation that accounts for data below th e detection level. The adjust ed estimates are based on the statistical technique of maximum likelihood estima tion of the mean and va riance so that the nondetects are accounted for. Care has to be take n when using the adjusted mean and variance in statistical tests. If the percentage of data be low the detection limit is relatively small (e.g., less than 20%), the significance level and power of the statistical test are approximately correct. As the proportion of data below detection increases, power declines and the true significance level increases dramatically. This is mainly attrib utable to the lack of independence between the adjusted mean and adjusted variance. If more than 50% of the observations are not detected, Cohen's method should not be used. In addition, this method requi res that the data without the non-detects be normally distributed and that the detection limit is always the same. Directions for Cohen's method are contained in B ox 4-31 with an example in Box 4-32.

4.7.2.2 Selecting Between Aitchison’ s Method and Cohen’s Method


## Cohen’s underlying model is that the population contains a no rmal distribution, but we cannot see the values below the censoring point . Aitchison’s underlying model is that the population consists of a proportion following a norm al distribution together with a proportion of values at zero. The difference in concepts becomes relevant depending on the types of inferences made. For example, in estimati ng upper quantiles, the analyst may use only the normal portion for the statistics, adjusting the quan tile to account for the estimated proportion at zero. If a confidence interval for the mean was required a simple substitution of zero for all data below detection would suffice. To determine if a data set is better adjusted by Cohen’s method or Aitchison’s method, a simple graphical procedure using a Norm al Probability Plot (Section 2.3.5) can be used. Dire ctions for this procedure are gi ven in Box 4-34 with an example in Box 4-35.

### EPA QA/G-9S 133 February 2006 Box 4-31: Directions for Cohen's Method

Let X 1 , X 2 , . ., X m , X m +1 ,. . . , X n represent the data points where the first m values are above the detection limit (DL) and the remaining n
- m data points are below the detection limit.

COMPUTATIONS: Using the data above the detection level, compute the sample mean and sample variance:


# ∑

= 

## = m

i i d 

## X
m X

1 

## 1 and 

## ⎪ ⎭ ⎪ ⎬ ⎫ ⎪ ⎩ ⎪ ⎨ ⎧ ⎟ ⎠ ⎞ ⎜ ⎝ ⎛

−

− = 

# ∑ ∑

=

= 2

1 1 2 2 

## 1
1 1 m

i i m

i i d 

## X
m X
m s

.

Compute h n m

n = − and 

# ( ) 2 DL X s

d 2 d

− = γ . Use h

, γ , and Table A-11 to determine $ λ . If the exact values of h and

γ do not appear in the table, use double linear interpolation (Box 4-33) to estimate $ λ

.
Estimate the corrected sample mean, X , and sample variance, s 2 : 

# ( ) DL X X X d d −

− = λ ˆ 

# ( ) 2 2 2 

## ˆ DL X s s d d 

## −

+ = 

## λ

.

Box 4-32: An Example of Cohen's Method
Sulfate concentrations (mg/L) were measured for 24 data po ints with 3 values falling below the detection limit of 1450 mg/L. The 24 values are:

1850, 1760, <1450, 1710, 1575, 1475, 1780, 1790, 1780, <1450, 1790, 1800, <1450, 1800, 1840, 1820, 1860, 1780, 1760, 1800, 1900, 1770, 1790, 1780.

Cohen's Method will be used to adjust the sample mean and sample variance for use in a t -test to determine if the mean is greater than 1600 mg/L.

COMPUTATIONS: The sample mean and sample variance of the m = 21 values above the detection level are

9

. 1771 = d X and 69

. 8593 2

= d s

.

The values of h and γ are: 125

. 0 24 21 24

= −
= h and 

## ( ) 083

. 0 1450 9

. 1771 69

. 8593

2 =

− = γ . Table A-11 does not contain

the exact entries for h and γ , double linear interpolation was used to estimate 149839

. 0 ˆ

= λ (see Box 4-33).

The adjusted sample mean and sample variance are: 

# ( ) ( ) X

. . . . DL X X X d d 

## = = − ⋅ − = − − = 67 1723 1450 9 1771 149839 0 9 1771 ˆ λ 

# ( ) ( ) s

. . . DL X s s d d 2 2 2 2 2 

## 95 24119 1450 9

. 1771 149839 0 69 8593 ˆ

= = − ⋅ + = − + = 

## λ

### EPA QA/G-9S 134 February 2006 Box 4-33: Example of Double Linear Interpolation
The details of the double linear interpolation are provided to assist in the use of Table A-11. The desired
value for $ λ corresponds to γ = 0.083 and h = 0.125 from Box 4-32. The values from Table A-11 used for interpolation are:

h γ c 1 = 0.10 c 2 = 0.15

r 1 = 0.05 x 11 = 0.11431 x 12 = 0.17925

r 2 = 0.10 x 21 = 0.11804 x 22 = 0.18479
We first interpolate between columns: 

## ( ) ( ) 14678

. 0 11431

. 0 17925

. 0 10

. 0 15

. 0 10

. 0 125

. 0 11431

. 0 11 12 1 2 1 11 1 = − ⋅

− − + = − ⋅

− − + = x x c c c c x y



## ( ) ( ) 151415

. 0 11804

. 0 18479

. 0 10

. 0 15

. 0 10

. 0 125

. 0 11804

. 0 21 22 1 2 1 21 2 = − ⋅

− − + = − ⋅

− − + = x x c c c c x y

Now we interpolate between the rows: 

## ( ) ( ) 149839

. 0 14678

. 0 151415

. 0 05

. 0 10

. 0 05

. 0 083

. 0 14678

. 0 ˆ 1 2 1 2 1 1 = − ⋅

− − + = − ⋅

− − + = y y r r r r y λ

Box 4-34: Directions for Selecting Between Cohen’s Method or Aitchison’s Method

Let X 1 , X 2 , . ., X m , . . . , X n represent the data points with the first m values are above the detection limit (DL) and the remaining n
- m data points are below the DL.

STEP 1: Use Box 2-17 to construct a Normal Probability Plot of all the data but only plot the values belonging to those above the detection level. This is called the Censored Plot.

STEP 2: Use Box 2-17 to construct a Normal Probability Plot of only those values above the detection level. This called the Detects only Plot.

STEP 3: If the Censored Plot is more linear than the Detects Only Plot, use Cohen’s Method to estimate the sample mean and variance. If the Detects Only Plot is more linear than the Censored Plot, then use Aitchison’s Method to estimate the sample mean and variance.



## 4.7.3 Greater than Approximately 50% No n-detects - Test of Proportions



## If more than 50% of the data are below the detection limit but at least 10% of the observations are quantified, then the best option is a test of pr oportions. Thus, if the parameter of interest is a mean, consider switching the para meter of interest to some percentile greater than the percent of data below the detection limit. For example, if 67% of the data are below the DL, consider switching the parameter of interest to the 75 th 

## percentile. Then the method described in 3.2.1.5 can be applied to test a hypothesis concerning the 75 th 

## percentile. It is important to note that the tests of proportions may not be applicable for composite samples. In this case, the data analyst should consult a statistician before proceeding with analysis.

### EPA QA/G-9S 135 February 2006 Box 4-35: Example of Determining Betw een Cohen’s Method and Aitchison’s Method
Readings of Chlorobenzene were obtained from a monitoring well:

<1, <1, <1, 1.2, 1.25, 1.3, 1.45, 1.35, 1.55, 1.6, 1.85, 2.1

Step 1: Using the directions in Box 2-17 the following is the Censored Plot:

STEP 2: Using the directions in Box 2-17 the following is the Detects only Plot:

STEP 3: Since the Censored Plots is more linear than the Detects Only Plot, Cohen’s Method should be used to estimate the sample mean and variance.



## 4.7.4 Greater than Approximately 90% Non-detects


## If very few quantified values are found, a method based on the Poisson distribution may be used as an alternative approach. However, with a large proportion of non-detects in the data, the data analyst should consult with a sta tistician before proceed ing with analysis.

4.7.5 Recommendations



## If the number of sample observations is small ( n < 20), Cohen’s and other maximum likelihood methods can produce biased results since it is difficult to assure that the underlying distribution is appropriate and th e solutions to the likelihood equati on are statistically consistent only if the number of samples is large. A dditionally, most methods will yield estimated parameters with large estimation variance, which reduces the power to detect import differences from standards or between populations. While thes e methods can be applied to small data sets, the user should be cautioned that they will only be effective in detecting large departures from the null hypothesis.

If the degree of censoring is relatively low, reasonably good estimates of means, variances and upper percentiles can be obtained. However, if the rate of censoring is very high (greater than 50%) then little can be done statistically except to focus on some upper quantile of

### EPA QA/G-9S 136 February 2006 

## the contaminant distribution, or on some proporti on of measurements above a certain critical level that is at or above the censoring limit.

When the numerical standard is at or belo w one of the censoring levels and a one-sample test is used, the most useful statistical method is to test whether the proportion of a population that is above (below) the standa rd is too large, or to test whether and upper quantile of the population distribution is above th e numerical standard. Table 4-5 gives some recommendation on which statistical parameter to use when censoring is present in data sets for different sizes of the coefficient of variation.

Table 4-5. Guidelines for Reco mmended Parameters for Different Coefficient of Variations and Censoring

Approximate Proportion of Data Below the Detection Limit Approximate Coefficient of Variation (CV) Low (<50%) High (>50%)

Large: CV > 1.5 Mean or Upper Percentile Upper Percentile

Medium: 0.5 < CV < 1.5 Mean or Upper Percentile Upper Percentile

Small: CV < 0.5 Mean or Median Median



## When comparing two data sets with different censoring leve ls (i.e., different detection limits), it is recommended that all data be cens ored at the highest censoring value present and a nonparametric test such as the Wilcoxon Rank Sum Te st (Section 3.3.2.1.1) used to compare the two data sets. There is a corres ponding loss of sta tistical power but to a certain extent this can be minimized through the use of large sample sizes.

4.8 INDEPENDENCE


## When data are truly independent, the correlati on between data points is by definition zero and the selected statistical tests attains the desired decision error rates (given the appropriate assumptions have been satisfied). When correlati on exists, the effectiveness of statistical tests is diminished. Environmental data are particularly susceptible to correlation problems due to the fact that such environmental data are collected under a spatial pattern or sequentially over time.

If observations are positively correlated over ti me or space, then the effective sample size for a test tends to be smaller than the actua l sample size—i.e., each additional observation does not provide as much ‘new’ information because it s value is partially determined by the value of adjacent observations. This smaller effective samp le size means that the degrees of freedom for the test statistic is smaller, or equivalently, the test is not as po werful as originally thought. In addition to affecting the false acceptance error rate, applying the usual tests to correlated data tends to result in a test whose actual significance level is larger than the nominal error rate.

### EPA QA/G-9S 137 February 2006 

## When observations are correlated, the estimate of the variance in the test statistic formula is often understated. For example, consider the mean of a series of n temporally-ordered observations. If these observations are inde pendent, then the variance of the mean is σ 2 

## / n , where

σ

2 

## is the variance of an individua l observation. However, if th e observations are not independent and the correlation between successive observations is ρ , then the variance of the mean is 

# ( ) ( ) q X + = 1 var 2 

## σ where 

# ( ) 

# ∑

−

= 

## − = 1

1 

## 2 n

k k 

## k n n q ρ , which will tend to be larger than σ 2 

## / n if the correlations are positive. If one conducts a t -test at a certain significance level using the usual formula for the estimated variance, then the actual significance level can be as mu ch as double what was expected even for low values of ρ

. One of the most effective ways to determine statistical independence is through use of the Rank von Neumann Test. Directions for this test are given in Box 4-36 with an example in Box 4-37. Compared to other tests of statis tical independence, the Rank von Neumann test has been shown to be more powerful over a wide variet y of cases. It is also a reasonable test when the data follow a normal distribution.

Box 4-36: Directions for the Rank von Neumann Test

Let X 1 , X 2 , . . . , X n represent the data values collected in sequence over equally spaced periods of time.

COMPUTATIONS: Order the data measurements from smallest to largest and assign rank r i to measurement X i

. If measurements are tied, then assign the average rank.

STEP 1. Null Hypothesis: H 0 : The data are independent.

STEP 2. Alternative Hypothesis: H A : The data are not independent.

STEP 3. Test Statistic: 

# ( ) ( ) 

# ∑

= − −

− = n

i i i r r n n v

2 2 1 2 0 1 12

STEP 4. a) Critical Value: Use Table A-16 to find v n , α . STEP 5. a) Conclusion: If v 0 < v n , α , then reject the null hypothesis that the data are independent.

NOTE: If the Rank von Neumann ratio test indicates significant evidence of dependence in the data, then a statistician should be consulted before further analysis is pe rformed. If ranks are tied, the power of the statistical test is diminished

### EPA QA/G-9S 138 February 2006 Box 4-37: An Example of the Rank von Neumann Test
The following are hourly readings from a discharge monitor: hourly readings from a discharge monitor.

COMPUTATIONS:

Time 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 24:00 Reading 6.5 6.6 6.7 6.4 6.3 6.4 6.2 6.2 6.3 6.6 6.8 6.9 7.0 Rank 7 8.5 10 5.5 3.5 5.5 1.5 1.5 3.5 8.5 11 12 13

STEP 1. Null Hypothesis: H 0 : The data are independent.

STEP 2. Alternative Hypothesis: H A : The data are not independent.

STEP 3. Test Statistic: 

# ( ) ( )( ) ( ) 

# { } 473

. 0 12 13 5

. 8 10 7 5

. 8 1 13 13 12 2 2 2 2 0 = − + +

− +

−

− ⋅ = L v

STEP 4. a) Critical Value: Using Table A-16, v 13, 0.05 = 1.14.

STEP 5. a) Conclusion: Since v 0 = 0.473 < 1.14 = v 13, 0.05 , we reject the null hypothesis that the data are independent.

### EPA QA/G-9S 139 February 2006 

## CHAPTER 5

STEP 5: DRAW CONCLUSIONS FROM THE DATA

Step 5: Draw Conclusions from the Data

• 

### Perform the calculations for the statistical method.

 

### Perform the calculations and document them clearly.

 

### If anomalies or outliers are present in the data set, perform the calculations with and without the questionable data.

• 

### Evaluate the results and draw conclusions.

 

### If the null hypothesis is rejected, then draw the conclusions and document the analysis.

 

### If the null hypothesis is not rejected, veri fy whether the tolerable limits on false acceptance decision errors have been satisfied. If so, draw conclusions and document the analysis; if not, determine corrective actions, if any.

 

### Interpret the results of the test or confidence interval.

• 

### Evaluate the performance of the sampling design if the design is to be used again.

 

### Evaluate the statistical power of the design over the full range of parameter values; consult a statistician as necessary. 

## THE DATA QUALITY ASSESSMENT PROCESS

Review DQOs and Sampling Design

Conduct Preliminary Data Review

Select the Statistical Method

Verify the Assumptions

Draw Conclusions from the Data DRAW CONCLUSIONS FROM THE DATA

Purpose

Perform the statistical procedure and interpret the results in the context of the data user's objectives.

A ctivities

Pe r form the Statistical Procedure Draw Study Conclusions. Evaluate Performance of the Sampling Design

Tools

Issues in hypothesis testing related to communicating the results of the DQA

### EPA QA/G-9S 140 February 2006 

## List of Boxes

Page 

## Box 5-1: Checking Adequacy of Sample Size for a One-Sample t -Test for Simple Random Sampling....................................................................................................... 142 Box 5-2: Example of Power Calculations for the One-Sample Test of a Single Proportion..... 144

### EPA QA/G-9S 141 February 2006 

## CHAPTER 5

STEP 5: DRAW CONCLUSIONS FROM THE DATA

5.1 OVERVIEW AND ACTIVITIES


## In this final step of the DQA, the analyst performs the statistical hypothesis test or computes the confidence interval and draws conclu sions that address the data user's objectives.

5.2 PERFORM THE STATISTICAL METHOD


## The goal of this activity is to conduct the statistical hypot hesis test or compute the confidence interval procedure chosen in Chapter 3. The calculations for the procedure should be clearly documented and easily veri fiable. In addition, document ation of the results should be understandable so they can be communicated eff ectively to those who may hold a stake in the resulting decision. If computer software is used to perform the calculations, then the procedures should be adequately documented.

5.3 DRAW STUDY CONCLUSIONS

5.3.1 Hypothesis Tests


## The goal of this activity is to translate the resu lts of the statistical hypo thesis test so that the data user may draw a conclusion fr om the data; the results being either:

(a) reject the null hypothesis , in which case there is significant evidence in favor of the alternative hypothesis. The decision can be made with sufficient confidence and without further analysis. This is becau se the statistical test s described in this document inherently control the false rejec tion error rate within the data user's tolerable limits when the unde rlying assumptions are valid.

(b) fail to reject the null hypothesis , in which case there is not significant evidence for the alternative hypothesis. The analys t is concerned about a possible false acceptance error. The most thorough pro cedure for verifying whether the false acceptance error limits have been satisfied is to compute the estimated power of the statistical test.

Alternatively the sample size required to sa tisfy the data user's objectives can be calculated retrospectivel y using an estimate of the varian ce or an upper confidence limit on variance obtained from the actual data . If this theoretical sample si ze is less than or equal to the number of samples actually taken, then the test is probably sufficiently powerful. The equations required to perform these calculations have been provided in the instructions for many of the hypothesis test procedures in Chapter 3. An exam ple of this method is c ontained in Box 5-1, but it is emphasized that this only gives an estimat e of power, not an absolute determination.

### EPA QA/G-9S 142 February 2006 Box 5-1: Checking Adequacy of Sample Size for a One-Sample t -Test for Simple Random Sampling

In Box 3-3, the one-sample t -test was used to test the hypothesis H 0 : μ ≤ 95 ppm vs. H A : μ > 95 ppm. DQOs specified that the test should limit the false rejection error rate to 5% and the false acceptance error rate to 20% if the true mean was 105 ppm.

A random sample of size n = 9 had sample mean X = 99.38 ppm and sample standard deviation s = 10.41 ppm. The null hypothesis was not rejected. Assuming that the true value of the standard deviation was equal to the sample estimate of 10.41 ppm, it was found that a sample size of 9 would be required. This validated the sample size of 9 which had actually been used.

In such a case it makes sense to build in some conservatism, for example, by using an upper 90% confidence limit for σ in the sample size calculation of Box 3-3. Using Box 4-22, it is found that an upper 90% confidence limit for the true standard deviation is

76 15 49

. 3 8 41 10 1

. . U n s = = −

.

Using this value for s in the sample size calculation of Box 3-3 leads to the sample size estimate of 17. Hence, a sample size of at least 17 should be used to be 90% sure of achieving the DQOs.

Since it is generally desirable to avoid the need for additi onal sampling, it is advisable to conservatively estimate sample size in the first place. In cases where DQOs depend on a variance estimate, this conservatism is achieved by intentionally overestimating the variance.



## 5.3.2 Confidence Intervals or Limits


## A confidence interval is simply an interv al estimate for the p opulation parameter of interest. The interval’s width is dependent upon the variance of the point estimate, the sample size, and the confidence level. Mo re specifically, the width is larg e if the variance is large, the sample size is small, or the confidence level is large.

The interpretation of a confidence interval makes use of probability in an intuitive sense. When a confidence interval has been constructed using the data, there is still a chance that the interval does not include the true value of the pa rameter estimated. For example, consider this confidence interval statement: “the 95% confid ence interval for the unknown population mean is 43.5 to 48.9”. It is interpreted as , “I can be 95% certain that the in terval 43.5 to 48.9 captures the unknown mean.” Notice how there is a 5% chance that the interval does not capture the mean.

The confidence level is the ‘confidence’ we have that the popul ation parameter lies within the interval. This concep t is analogous to the false rejecti on error rate. The width of the interval is related to st atistical power, or the false acceptance e rror rate. Rather than specifying a desired false acceptance error rate, the desi red interval width can be specified.

A confidence interval can be used to make to decisions and in some situations a test of hypothesis is set up as a confidence interval. Confidence intervals are analogous to two-sided hypothesis tests. If the threshold value lies outside of the interval, then there is evidence that the population parameter differs from the threshold va lue. In a similar manner, confidence limits can also be related to one-sided hypothesis test s. If the threshold value lies above (below) an

### EPA QA/G-9S 143 February 2006 

## upper (lower) confidence bound, then there is ev idence that the population parameter is less (greater) than the threshold.

5.3.3 Tolerance Intervals or Limits



## A tolerance interval is an in terval estimate for a certain pr oportion of the population. The interval’s width is dependent upon the variance of the population, the sample size, the desired proportion of the population, and the confidence leve l. More specifically, the width is large if the variance is large, the sample size is small, the proportion is larg e, or the confidence level is large.

When a tolerance interval has been constructed using the data , there is still a chance that the interval does not include the desired proportion of the population. For example, consider this tolerance interval statement: “the 99% tolerance interval for 90% of the population is 7.5 to 9.9”. It is interpreted as, “I can be 99% certain that the interval 7.5 to 9.9 captures 90% of the population.” Notice how there is a 1% chance that the interval does not capture at least the desired proportion.

The confidence level is the ‘confidence’ we have that the desired proportion of the population lies within the interval . This concept is analogous to the false rejection error rate. The width of the interval is related to statistica l power, or the false acceptance error rate. Rather than specifying a desired false accep tance error rate, the desired interval width can be specified.

A tolerance interval can be used to make to decisions, and in some situations a test of hypothesis can be set up as a tole rance interval. Tolerance inte rvals are analogous to two-sided hypothesis tests. If the threshold value lies outside of the interval, then there is evidence that the desired proportion of the populati on differs from the threshold va lue. In a similar manner, tolerance limits can also be re lated to one-sided hypothesis tests. If the threshold value lies above (below) an upper (lower) tolerance limit, th en there is evidence that the desired proportion of the population is less (gre ater) than the threshold.

5.4 EVALUATE PERFORMANCE OF THE SAMPLING DESIGN


## If the sampling design is to be used again, eith er in a later phase of the current study or in a similar study, the analyst will be interested in evaluating the overall performance of the design. To evaluate the sampling design, the analyst perf orms a statistical power analysis that describes the estimated power of the statistical test over the range of possible parameter values. The estimated power is computed for all parameter va lues under the alternative hypothesis to create a power curve. A power analysis helps the analyst evaluate the adequacy of the sampling design when the true parameter value lies in the vicinity of the action level. In this manner, the analyst may determine how well a statistical test performed and compare this performance with that of other tests. Box 5-2 illustra tes power calculations for a test of a single proportion.

### EPA QA/G-9S 144 February 2006 Box 5-2: Example of Power Calculations fo r the One-Sample Test of a Single Proportion

This box illustrates power calculations for the test of H 0 : P ≥ 0.20 vs. H A : P < 0.20, with a false rejection error rate of 5% when P = 0.20 presented in Box 3-13. The power of the test will be calculated assuming P 1 = 0.15 and before data is available. Since nP 0 and n (1- P 0 ) both exceed 5, the normal approximation may be used.

STEP 1: Determine the general conditions for rejection of the null hypothesis. In this case, the null hypothesis is rejected if the sample proportion is sufficiently smaller than P 0 . Using Box 3-11, H 0 is rejected if



## ( ) z n P P P n p

α

− − <

− − +
1 0 0 0 / 1 5

. 0

or n Q P z P n p 0 0 1 0 5

. 0
α

− − < + ,

where p is the sample proportion and – 

### z 1

− α is the standard normal critical value.

STEP 2: Determine the specific conditions for rejection of the null hypothesis if P 1 is the true value of the proportion P . Using the equations above, rejection occurs if



## ( ) ( ) ( ) 55 0 85 / 85

. 0 15

. 0 85 / 8

. 0 2

. 0 645 1 15 0 20 0
/ 1 / 1
/ 1 5

. 0

1 1 0 0 1 1 0

1 1 1

. . . .
n P P n P P z P P

n P P P n p

− =

⋅ ⋅ − −
=

− − − − <

− − +

− α

STEP 3: Find the probability of rejection if P 1 is the true proportion. The quantity on the left-hand side of the above inequality is a standard normal variable. Hence the power at P 1 = 0.15 is the probability that a standard normal variable is less than -0.55. Using Table A-1, this probability is approximately 0.3, which is fairly small.



## 5.5 INTERPRET AND COMMUNICATE THE RESULTS


## At this point, the analyst has performed the applicable statistical procedure and has drawn conclusions. In many cases, the conclusions are st raightforward and convincing so they lead to an unambiguous path forward for the project. In other cases, however, it is advantageous to consider these conclusions in a broader context in order to determine a course of action, see Data Quality Assessment: A Reviewer’s Guide (EPA QA/G-9R) (U.S. EPA 2004).

### EPA QA/G-9S 145 February 2006 

## APPENDIX A:

STATISTICAL TABLES

### EPA QA/G-9S 146 February 2006 

## List of Tables

Page 

## TABLE A-1. STANDARD NORMAL DISTRIBUTION ........................................................ 147 TABLE A-2. CRITICAL VALUES OF STUDENT'S- t DISTRIBUTION .............................. 149 TABLE A-3. CRITICAL VALUES FOR THE STUDENT IZED RANGE TEST................... 150 TABLE A-4. CRITICA L VALUES FOR THE EXTREME VALUE TEST (DIXON'S TEST) ................................................................................................ 151 TABLE A-5. CRITI CAL VALUES FOR DISC ORDANCE TEST ......................................... 152 TABLE A-6. APPROXIMATE CRITICAL VALUES λ r 

## FOR ROSNER'S TEST.................. 153 TABLE A-7. QUANTILES OF THE WILCOXON SIGNED RANKS TEST......................... 155 TABLE A-8. CRITICAL VALUES FOR THE WILCOXON RANK-SUM TEST................. 156 TABLE A-9. PERCENTILES OF THE CHI-SQUARE DISTRIBUTION .............................. 159 TABLE A-10. PERCENTILES OF THE F -DISTRIBUTION.................................................. 160 TABLE A-11. VALUES OF THE PARAMETER $ 

## λ FOR COHEN'S ESTIMATES ADJUSTING FOR NOND ETECTED VALUES.............................................. 164 TABLE A-12a: CRITICAL VALUES FOR THE MANN-KENDALL TEST FOR TREND.. 165 TABLE A-12b: PROBABILITIES FO R THE SMALL-SAMPLE MANN-KENDALL TEST FOR TREND......................................................................................... 165 TABLE A-13. QUANTILES FOR THE WALD-WOLFOWITZ TEST FOR RUNS.............. 166 TABLE A-14. CRITICAL VALUES FOR THE SLIPPAGE TEST ........................................ 170 TABLE A-15. DUNNETT’S TEST (ONE TAILED) ............................................................... 173 TABLE A-16. APPROXIMATE α -LEVEL CRITICAL POINTS FOR RANK VON NEUMANN RATIO TEST ............................................................................... 175 TABLE A-17. VALUES FOR COMPUTING A ONE-SIDED CONFIDENCE LIMIT ON A LOGNORMAL MEAN ................................................................................. 176 TABLE A-18. CRITICAL VALUES FOR THE SIGN TEST.................................................. 178 TABLE A-19. CRITICAL VALUES FOR THE QUANTILE TEST....................................... 179

### EPA QA/G-9S 147 February 2006 

## TABLE A-1. STANDARD NO RMAL DISTRIBUTION

Table values are P( Z ≤ z p 

### ) = p

.

z p 

### .00 .01 .02 .03 .04 .05 .06 .07 .08 .09
-0.0 .5000 .4960 .4920 .4880 .4840 .4801 .4761 .4721 .4681 .4641 -0.1 .4602 .4562 .4522 .4483 .4443 .4404 .4364 .4325 .4286 .4247 -0.2 .4207 .4168 .4129 .4090 .4052 .4013 .3974 .3936 .3897 .3859 -0.3 .3821 .3783 .3745 .3707 .3669 .3632 .3594 .3557 .3520 .3483 -0.4 .3446 .3409 .3372 .3336 .3300 .3264 .3228 .3192 .3156 .3121 -0.5 .3085 .3050 .3015 .2981 .2946 .2912 .2877 .2843 .2810 .2776 -0.6 .2743 .2709 .2676 .2643 .2611 .2578 .2546 .2514 .2483 .2451 -0.7 .2420 .2389 .2358 .2327 .2296 .2266 .2236 .2206 .2177 .2148 -0.8 .2119 .2090 .2061 .2033 .2005 .1977 .1949 .1922 .1894 .1867 -0.9 .1841 .1814 .1788 .1762 .1736 .1711 .1685 .1660 .1635 .1611 -1.0 .1587 .1562 .1539 .1515 .1492 .1469 .1446 .1423 .1401 .1379 -1.1 .1357 .1335 .1314 .1292 .1271 .1251 .1230 .1210 .1190 .1170 -1.2 .1151 .1131 .1112 .1093 .1075 .1056 .1038 .1020 .1003 .0985 -1.3 .0968 .0951 .0934 .0918 .0901 .0885 .0869 .0853 .0838 .0823 -1.4 .0808 .0793 .0778 .0764 .0749 .0735 .0721 .0708 .0694 .0681 -1.5 .0668 .0655 .0643 .0630 .0618 .0606 .0594 .0582 .0571 .0559 -1.6 .0548 .0537 .0526 .0516 .0505 .0495 .0485 .0475 .0465 .0455 -1.7 .0446 .0436 .0427 .0418 .0409 .0401 .0392 .0384 .0375 .0367 -1.8 .0359 .0351 .0344 .0336 .0329 .0322 .0314 .0307 .0301 .0294 -1.9 .0287 .0281 .0274 .0268 .0262 .0256 .0250 .0244 .0239 .0233 -2.0 .0228 .0222 .0217 .0212 .0207 .0202 .0197 .0192 .0188 .0183 -2.1 .0179 .0174 .0170 .0166 .0162 .0158 .0154 .0150 .0146 .0143 -2.2 .0139 .0136 .0132 .0129 .0125 .0122 .0119 .0116 .0113 .0110 -2.3 .0107 .0104 .0102 .0099 .0096 .0094 .0091 .0089 .0087 .0084 -2.4 .0082 .0080 .0078 .0075 .0073 .0071 .0069 .0068 .0066 .0064 -2.5 .0062 .0060 .0059 .0057 .0055 .0054 .0052 .0051 .0049 .0048 -2.6 .0047 .0045 .0044 .0043 .0041 .0040 .0039 .0038 .0037 .0036 -2.7 .0035 .0034 .0033 .0032 .0031 .0030 .0029 .0028 .0027 .0026 -2.8 .0026 .0025 .0024 .0023 .0023 .0022 .0021 .0021 .0020 .0019 -2.9 .0019 .0018 .0018 .0017 .0016 .0016 .0015 .0015 .0014 .0014 -3.0 .0013 .0013 .0013 .0012 .0012 .0011 .0011 .0011 .0010 .0010 -3.1 .0010 .0009 .0009 .0009 .0008 .0008 .0008 .0008 .0007 .0007 -3.2 .0007 .0007 .0006 .0006 .0006 .0006 .0006 .0005 .0005 .0005 -3.3 .0005 .0005 .0005 .0004 .0004 .0004 .0004 .0004 .0004 .0003 -3.4 .0003 .0003 .0003 .0003 .0003 .0003 .0003 .0003 .0003 .0002

### EPA QA/G-9S 148 February 2006 

## TABLE A-1. STANDARD NORMAL DISTRIBUTION (CONT.)

Table values are P( Z ≤ z p 

### ) = p

.

z p 

### .00 .01 .02 .03 .04 .05 .06 .07 .08 .09
0.0 .5000 .5040 .5080 .5120 .5160 .5199 .5239 .5279 .5319 .5359 0.1 .5398 .5438 .5478 .5517 .5557 .5596 .5636 .5675 .5714 .5753 0.2 .5793 .5832 .5871 .5910 .5948 .5987 .6026 .6064 .6103 .6141 0.3 .6179 .6247 .6255 .6293 .6331 .6368 .6406 .6443 .6480 .6517 0.4 .6554 .6591 .6628 .6664 .6700 .6736 .6772 .6808 .6844 .6879 0.5 .6915 .6950 .6985 .7019 .7054 .7088 .7123 .7157 .7190 .7224 0.6 .7257 .7291 .7324 .7357 .7389 .7422 .7454 .7486 .7517 .7549 0.7 .7580 .7611 .7642 .7673 .7704 .7734 .7764 .7794 .7823 .7852 0.8 .7881 .7910 .7939 .7967 .7995 .8023 .8051 .8078 .8106 .8133 0.9 .8159 .8186 .8212 .8238 .8264 .8289 .8315 .8340 .8365 .8389 1.0 .8413 .8438 .8461 .8485 .8508 .8531 .8554 .8577 .8599 .8621 1.1 .8643 .8665 .8686 .8708 .8729 .8749 .8770 .8790 .8810 .8830 1.2 .8849 .8869 .8888 .8907 .8925 .8944 .8962 .8980 .8897 .9015 1.3 .9032 .9049 .9066 .9082 .9099 .9115 .9131 .9147 .9162 .9177 1.4 .9192 .9207 .9222 .9236 .9251 .9265 .9279 .9292 .9306 .9319 1.5 .9332 .9345 .9357 .9370 .9382 .9394 .9406 .9418 .9429 .9441 1.6 .9452 .9463 .9474 .9484 .9495 .9505 .9515 .9525 .9535 .9545 1.7 .9554 .9564 .9573 .9582 .9591 .9599 .9608 .9616 .9625 .9633 1.8 .9641 .9649 .9656 .9664 .9671 .9678 .9686 .9693 .9699 .9706 1.9 .9713 .9719 .9726 .9732 .9738 .9744 .9750 .9756 .9761 .9767 2.0 .9772 .9778 .9783 .9788 .9793 .9798 .9803 .9808 .9812 .9817 2.1 .9821 .9826 .9830 .9834 .9838 .9842 .9846 .9850 .9854 .9857 2.2 .9861 .9864 .9868 .9871 .9875 .9878 .9881 .9884 .9889 .9890 2.3 .9893 .9896 .9898 .9901 .9904 .9906 .9909 .9911 .9137 .9916 2.4 .9918 .9920 .9922 .9925 .9927 .9929 .9931 .9932 .9934 .9936 2.5 .9938 .9940 .9941 .9943 .9945 .9946 .9948 .9949 .9951 .9952 2.6 .9953 .9955 .9956 .9957 .9959 .9960 .9961 .9962 .9963 .9964 2.7 .9965 .9966 .9967 .9968 .9969 .9970 .9971 .9972 .9973 .9974 2.8 .9974 .9975 .9976 .9977 .9977 .9978 .9979 .9979 .9980 .9981 2.9 .9981 .9982 .9982 .9983 .9984 .9984 .9985 .9985 .9986 .9986 3.0 .9987 .9987 .9987 .9988 .9988 .9989 .9989 .9989 .9990 .9990 3.1 .9990 .9991 .9991 .9991 .9992 .9992 .9992 .9992 .9993 .9993 3.2 .9993 .9993 .9994 .9994 .9994 .9994 .9994 .9995 .9995 .9995 3.3 .9995 .9995 .9995 .9996 .9996 .9996 .9996 .9996 .9996 .9997 3.4 .9997 .9997 .9997 .9997 .9997 .9997 .9997 .9997 .9997 .9998

### EPA QA/G-9S 149 February 2006 t 1- α 

## TABLE A-2. CRITICAL VALUES OF STUDENT'S- t DISTRIBUTION

1 - α Degrees of Freedom 0.70 0.75 0.80 0.85 0.90 0.95 0.975 0.99 0.995

1 2 3 4 5 0.727 0.617 0.584 0.569 0.559 1.000 0.816 0.765 0.741 0.727 1.376 1.061 0.978 0.941 0.920 1.963 1.386 1.250 1.190 1.156 3.078 1.886 1.638 1.533 1.476 6.314 2.920 2.353 2.132 2.015 12.706 4.303 3.182 2.776 2.571 31.821 6.965 4.541 3.747 3.365 63.657 9.925 5.841 4.604 4.032

6 7 8 9 10 0.553 0.549 0.546 0.543 0.542 0.718 0.711 0.706 0.703 0.700 0.906 0.896 0.889 0.883 0.879 1.134 1.119 1.108 1.100 1.093 1.440 1.415 1.397 1.383 1.372 1.943 1.895 1.860 1.833 1.812 2.447 2.365 2.306 2.262 2.228 3.143 2.998 2.896 2.821 2.764 3.707 3.499 3.355 3.250 3.169

11 12 13 14 15 0.540 0.539 0.538 0.537 0.536 0.697 0.695 0.694 0.692 0.691 0.876 0.873 0.870 0.868 0.866 1.088 1.083 1.079 1.076 1.074 1.363 1.356 1.350 1.345 1.34 1.796 1.782 1.771 1.761 1.753 2.201 2.179 2.160 2.145 2.131 2.718 2.681 2.650 2.624 2.602 3.106 3.055 3.012 2.977 2.947

16 17 18 19 20 0.535 0.534 0.534 0.533 0.533 0.690 0.689 0.688 0.688 0.687 0.865 0.863 0.862 0.861 0.860 1.071 1.069 1.067 1.066 1.064 1.337 1.333 1.330 1.328 1.325 1.746 1.740 1.734 1.729 1.725 2.120 2.110 2.101 2.093 2.086 2.583 2.567 2.552 2.539 2.528 2.921 2.898 2.878 2.861 2.845

21 22 23 24 25 0.532 0.532 0.532 0.531 0.531 0.686 0.686 0.685 0.685 0.684 0.859 0.858 0.858 0.857 0.856 1.063 1.061 1.060 1.059 1.058 1.323 1.321 1.319 1.318 1.316 1.721 1.717 1.714 1.711 1.708 2.080 2.074 2.069 2.064 2.060 2.518 2.508 2.500 2.492 2.485 2.831 2.819 2.807 2.797 2.787

26 27 28 29 30 0.531 0.531 0.530 0.530 0.530 0.684 0.684 0.683 0.683 0.683 0.856 0.855 0.855 0.854 0.854 1.058 1.057 1.056 1.055 1.055 1.315 1.314 1.313 1.311 1.310 1.706 1.703 1.701 1.699 1.697 2.056 2.052 2.048 2.045 2.042 2.479 2.473 2.467 2.462 2.457 2.779 2.771 2.763 2.756 2.750

40 60 120

∞ 0.529 0.527 0.526 0.524 0.681 0.679 0.677 0.674 0.851 0.848 0.845 0.842 1.050 1.046 1.041 1.036 1.303 1.296 1.289 1.282 1.684 1.671 1.658 1.645 2.021 2.000 1.980 1.960 2.423 2.390 2.358 2.326 2.704 2.660 2.617 2.576
Note: The last row of the table ( ∞ degrees of freedom) gives the critical va lues for a standard normal distribution ( Z ), e.g., t ∞ , 0.95 = z 0.95 = 1.645.

### EPA QA/G-9S 150 February 2006 

## TABLE A-3. CRITICAL VALUES FO R THE STUDENTIZED RANGE TEST

Level of Significance α

0.10 0.05 0.01

n a b a b a b

3 4 5

6 7 8 9 10

11 12 13 14 15

16 17 18 19 20

25 30 35 40 45

50 55 60 65 70

75 80 85 90 95

100 150 200 500 1000 1.78 2.04 2.22

2.37 2.49 2.59 2.68 2.76

2.84 2.90 2.96 3.02 3.07

3.12 3.17 3.21 3.25 3.29

3.45 3.59 3.70 3.79 3.88

3.95 4.02 4.08 4.14 4.19

4.24 4.28 4.33 4.36 4.40

4.44 4.72 4.90 5.49 5.92 2.00 2.41 2.71

2.95 3.14 3.31 3.45 3.57

3.68 3.78 3.87 3.95 4.02

4.09 4.15 4.21 4.27 4.32

4.53 4.70 4.84 4.96 5.06

5.14 5.22 5.29 5.35 5.41

5.46 5.51 5.56 5.60 5.64

5.68 5.96 6.15 6.72 7.11 1.76 1.98 2.15

2.28 2.40 2.50 2.59 2.67

2.74 2.80 2.86 2.92 2.97

3.01 3.06 3.10 3.14 3.18

3.34 3.47 3.58 3.67 3.75

3.83 3.90 3.96 4.01 4.06

4.11 4.16 4.20 4.24 4.27

4.31 4.59 4.78 5.47 5.79 2.00 2.43 2.75

3.01 3.22 3.40 3.55 3.69

3.80 3.91 4.00 4.09 4.17

4.24 4.31 4.37 4.43 4.49

4.71 4.89 5.04 5.16 5.26

5.35 5.43 5.51 5.57 5.63

5.68 5.73 5.78 5.82 5.86

5.90 6.18 6.39 6.94 7.33 1.74 1.87 2.02

2.15 2.26 2.35 2.44 2.51

2.58 2.64 2.70 2.75 2.80

2.84 2.88 2.92 2.96 2.99

3.15 3.27 3.38 3.47 3.55

3.62 3.69 3.75 3.80 3.85

3.90 3.94 3.99 4.02 4.06

4.10 4.38 4.59 5.13 5.57 2.00 2.45 2.80

3.10 3.34 3.54 3.72 3.88

4.01 4.13 4.24 4.34 4.44

4.52 4.60 4.67 4.74 4.80

5.06 5.26 5.42 5.56 5.67

5.77 5.86 5.94 6.01 6.07

6.13 6.18 6.23 6.27 6.32

6.36 6.64 6.84 7.42 7.80

### EPA QA/G-9S 151 February 2006 

## TABLE A-4. CRITICAL VALUES FOR THE EXTREME VALUE TEST (DIXON'S TEST)

Level of Significance α

n 0.10 0.05 0.01

3 4 5

6 7 8 9 10

11 12 13 14 15

16 17 18 19 20

21 22 23 24 25 0.886 0.679 0.557

0.482 0.434 0.479 0.441 0.409

0.517 0.490 0.467 0.492 0.472

0.454 0.438 0.424 0.412 0.401

0.391 0.382 0.374 0.367 0.360 0.941 0.765 0.642

0.560 0.507 0.554 0.512 0.477

0.576 0.546 0.521 0.546 0.525

0.507 0.490 0.475 0.462 0.450

0.440 0.430 0.421 0.413 0.406 0.988 0.889 0.780

0.698 0.637 0.683 0.635 0.597

0.679 0.642 0.615 0.641 0.616

0.595 0.577 0.561 0.547 0.535

0.524 0.514 0.505 0.497 0.489

### EPA QA/G-9S 152 February 2006 

## TABLE A-5. CRITICAL VALUES FOR DISCORDANCE TEST

Level of Significance α Level of Significance α

N 0.01 0.05 n 0.01 0.05

3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20

21 22 23 24 25 26 27 28 29 30 1.155 1.492 1.749 1.944 2.097 2.221 2.323 2.410

2.485 2.550 2.607 2.659 2.705 2.747 2.785 2.821 2.854 2.884

2.912 2.939 2.963 2.987 3.009 3.029 3.049 3.068 3.085 3.103 1.153 1.463 1.672 1.822 1.938 2.032 2.110 2.176

2.234 2.285 2.331 2.371 2.409 2.443 2.475 2.504 2.532 2.557

2.580 2.603 2.624 2.644 2.663 2.681 2.698 2.714 2.730 2.745 31 32 33 34 35 36 37 38 39 40

41 42 43 44 45 46 47 48 49 50 3.119 3.135 3.150 3.164 3.178 3.191 3.204 3.216 3.228 3.240

3.251 3.261 3.271 3.282 3.292 3.302 3.310 3.319 3.329 3.336 2.759 2.773 2.786 2.799 2.811 2.823 2.835 2.846 2.857 2.866

2.877 2.887 2.896 2.905 2.914 2.923 2.931 2.940 2.948 2.956

### EPA QA/G-9S 153 February 2006 

## TABLE A-6. APPROXIMATE CRITICAL VALUES λ r 

## FOR ROSNER'S TEST

α α α

n r 0.05 0.01 n r 0.05 0.01 n r 0.05 0.01

25 1 2 3 4 5 10 2.82 2.80 2.78 2.76 2.73 2.59 3.14 3.11 3.09 3.06 3.03 2.85 32 1 2 3 4 5 10 2.94 2.92 2.91 2.89 2.88 2.78 3.27 3.25 3.24 3.22 3.20 3.09 39 1 2 3 4 5 10 3.03 3.01 3.00 2.99 2.98 2.91 3.37 3.36 3.34 3.33 3.32 3.24

26 1 2 3 4 5 10 2.84 2.82 2.80 2.78 2.76 2.62 3.16 3.14 3.11 3.09 3.06 2.89 33 1 2 3 4 5 10 2.95 2.94 2.92 2.91 2.89 2.80 3.29 3.27 3.25 3.24 3.22 3.11 40 1 2 3 4 5 10 3.04 3.03 3.01 3.00 2.99 2.92 3.38 3.37 3.36 3.34 3.33 3.25

27 1 2 3 4 5 10 2.86 2.84 2.82 2.80 2.78 2.65 3.18 3.16 3.14 3.11 3.09 2.93 34 1 2 3 4 5 10 2.97 2.95 2.94 2.92 2.91 2.82 3.30 3.29 3.27 3.25 3.24 3.14 41 1 2 3 4 5 10 3.05 3.04 3.03 3.01 3.00 2.94 3.39 3.38 3.37 3.36 3.34 3.27

28 1 2 3 4 5 10 2.88 2.86 2.84 2.82 2.80 2.68 3.20 3.18 3.16 3.14 3.11 2.97 35 1 2 3 4 5 10 2.98 2.97 2.95 2.94 2.92 2.84 3.32 3.30 3.29 3.27 3.25 3.16 42 1 2 3 4 5 10 3.06 3.05 3.04 3.03 3.01 2.95 3.40 3.39 3.38 3.37 3.36 3.29

29 1 2 3 4 5 10 2.89 2.88 2.86 2.84 2.82 2.71 3.22 3.20 3.18 3.16 3.14 3.00 36 1 2 3 4 5 10 2.99 2.98 2.97 2.95 2.94 2.86 3.33 3.32 3.30 3.29 3.27 3.18 43 1 2 3 4 5 10 3.07 3.06 3.05 3.04 3.03 2.97 3.41 3.40 3.39 3.38 3.37 3.30

30 1 2 3 4 5 10 2.91 2.89 2.88 2.86 2.84 2.73 3.24 3.22 3.20 3.18 3.16 3.03 37 1 2 3 4 5 10 3.00 2.99 2.98 2.97 2.95 2.88 3.34 3.33 3.32 3.30 3.29 3.20 44 1 2

3 4 5 10 3.08 3.07 3.06 3.05 3.04 2.98 3.43 3.41 3.40 3.39 3.38 3.32

31 1 2 3 4 5 10 2.92 2.91 2.89 2.88 2.86 2.76 3.25 3.24 3.22 3.20 3.18 3.06 38 1 2 3 4 5 10 3.01 3.00 2.99 2.98 2.97 2.91 3.36 3.34 3.33 3.32 3.30 3.22 45 1 2 3 4 5 10 3.09 3.08 3.07 3.06 3.05 2.99 3.44 3.43 3.41 3.40 3.39 3.33

### EPA QA/G-9S 154 February 2006 

## TABLE A-6. APPROXIMATE CRITICAL VALUES λ r 

## FOR ROSNER'S TEST (CONT.)

α α α

n r 0.05 0.01 n r 0.05 0.01 n r 0.05 0.01

46 1 2 3 4 5 10 3.09 3.09 3.08 3.07 3.06 3.00 3.45 3.44 3.43 3.41 3.40 3.34 70 1 2 3 4 5 10 3.26 3.25 3.25 3.24 3.24 3.21 3.62 3.62 3.61 3.60 3.60 3.57 250 1 5 10 3.67 3.67 3.66 4.04 4.04 4.03

47 1 2 3 4 5 10 3.10 3.09 3.09 3.08 3.07 3.01 3.46 3.45 3.44 3.43 3.41 3.36 80 1 2 3 4 5 10 3.31 3.30 3.30 3.29 3.29 3.26 3.67 3.67 3.66 3.66 3.65 3.63 300 1 5 10 3.72 3.72 3.71 4.09 4.09 4.09

48 1 2 3 4 5 10 3.11 3.10 3.09 3.09 3.08 3.03 3.46 3.46 3.45 3.44 3.43 3.37 90 1 2 3 4 5 10 3.35 3.34 3.34 3.34 3.33 3.31 3.72 3.71 3.71 3.70 3.70 3.68 350 1 5 10 3.77 3.76 3.76 4.14 4.13 4.13

49 1 2 3 4 5 10 3.12 3.11 3.10 3.09 3.09 3.04 3.47 3.46 3.46 3.45 3.44 3.38 100 1 2 3 4 5 10 3.38 3.38 3.38 3.37 3.37 3.35 3.75 3.75 3.75 3.74 3.74 3.72 400 1 5 10 3.80 3.80 3.80 4.17 4.17 4.16

50 1 2 3 4 5 10 3.13 3.12 3.11 3.10 3.09 3.05 3.48 3.47 3.46 3.46 3.45 3.39 150 1 2 3 4 5 10 3.52 3.51 3.51 3.51 3.51 3.50 3.89 3.89 3.89 3.88 3.88 3.87 450 1 5 10 3.84 3.83 3.83 4.20 4.20 4.20

60 1 2 3 4 5 10 3.20 3.19 3.19 3.18 3.17 3.14 3.56 3.55 3.55 3.54 3.53 3.49 200 1 2 3 4 5 10 3.61 3.60 3.60 3.60 3.60 3.59 3.98 3.98 3.97 3.97 3.97 3.96 500 1 5 10 3.86 3.86 3.86 4.23 4.23 4.22

### EPA QA/G-9S 155 February 2006 

## TABLE A-7. QUANTILES OF THE WILCOXON SIGNED RANKS TEST



## Values in the table are such that P( T + 

## ≤ w α 

## ) is approximately equal to, but less than α . For example, if n = 12, then P( T + 

## ≤ 17) = 0.0461, which is slightly less than 0.05. Note the exact probability was computed using the statistical software package R.

N w 0.005 w 0.01 w 0.025 w 0.05 w 0.075 w 0.10 w 0.15 w 0.20 4 - - - - 0 0 1 2 5 - - - 0 1 2 2 3 6 - - 0 2 2 3 4 5 7 - 0 2 3 4 5 7 8 8 0 1 3 5 7 8 9 11 9 1 3 5 8 9 10 12 14 10 3 5 8 10 12 14 16 18 11 5 7 10 13 16 17 20 22 12 7 9 13 17 19 21 24 27 13 9 12 17 21 24 26 29 32 14 12 15 21 25 28 31 35 38 15 15 19 25 30 33 36 40 44 16 19 23 29 35 39 42 47 50 17 23 27 34 41 45 48 53 57 18 27 32 40 47 51 55 60 65 19 32 37 46 53 58 62 68 73 20 37 43 52 60 65 69 76 81

### EPA QA/G-9S 156 February 2006 

## TABLE A-8. CRITICAL VALUES FO R THE WILCOXON RANK-SUM TEST



## Table values are the largest x values such that P( W rs 

## ≤ x ) ≤ α . Therefore, significance levels, α , are approximate. If there are ties , then the test is approximate.

max( m , n ) min( m , n ) α 2 3 4 5 6 7 8 9 10 0.010 - - - - - - - - - 0.025 - - - - - - 0 0 0 0.050 - - - 0 0 0 1 1 1 2

0.100 - 0 0 1 1 1 2 2 3 0.010 - - - 0 0 0 1 1 0.025 - - 0 1 1 2 2 3 0.050 0 0 1 2 2 3 4 4 3

0.100 1 1 2 3 4 5 5 6 0.010 - 0 1 1 2 3 3 0.025 0 0 2 3 4 4 5 0.050 1 2 3 4 5 6 7 4

0.100 3 4 5 6 7 9 10 0.010 1 2 3 4 5 6 0.025 2 3 5 6 7 8 0.050 4 5 6 8 9 11 5

0.100 5 7 8 10 12 13 0.010 3 4 6 7 8 0.025 5 6 8 10 11 0.050 7 8 10 12 14 6

0.100 9 11 13 15 17 0.010 6 7 9 11 0.025 8 10 12 14 0.050 11 13 15 17 7

0.100 13 16 18 21 0.010 9 11 13 0.025 13 15 17 0.050 15 18 20 8

0.100 19 22 25 0.010 14 16 0.025 17 20 0.050 21 24 9

0.100 25 28 0.010 13 0.025 23 0.050 27 10

0.100 32

### EPA QA/G-9S 157 February 2006 

## TABLE A-8. CRITICAL VALUES FOR TH E WILCOXON RANK-SUM TEST (CONT.)



## Table values are the largest x values such that P( W rs 

## ≤ x ) ≤ α . Therefore, significance levels, α , are approximate. If there are ties , then the test is approximate.

max( m , n ) min( m , n ) α 11 12 13 14 15 16 17 18 19 20
0.010 - - 0 0 0 0 0 0 1 1 0.025 0 1 1 1 1 1 2 2 2 2 0.050 1 2 2 3 3 3 3 4 4 4 2

0.100 3 4 4 4 5 5 6 6 7 7 0.010 1 2 2 2 3 3 4 4 4 5 0.025 3 4 4 5 5 6 6 7 7 8 0.050 5 5 6 7 7 8 9 9 10 11 3

0.100 7 8 9 10 10 11 12 13 14 15 0.010 4 5 5 6 7 7 8 9 9 10 0.025 6 7 8 9 10 11 11 12 13 14 0.050 8 9 10 11 12 14 15 16 17 18 4

0.100 11 12 13 15 16 17 18 20 21 22 0.010 7 8 9 10 11 12 13 14 15 16 0.025 9 11 12 13 14 15 16 18 19 20 0.050 12 13 15 16 18 19 20 22 23 25 5

0.100 15 17 18 20 22 23 25 27 28 30 0.010 9 11 12 13 15 16 18 19 20 22 0.025 13 14 16 17 19 21 22 24 25 27 0.050 16 17 19 21 23 25 26 28 30 32 6

0.100 19 21 23 25 27 29 31 34 36 38 0.010 12 14 16 17 19 21 23 24 26 28 0.025 16 18 20 22 24 26 28 30 32 33 0.050 19 21 24 26 28 30 33 35 37 39 7

0.100 23 26 28 31 33 36 38 41 43 46 0.010 15 17 20 22 24 26 28 30 32 34 0.025 19 22 24 26 28 31 34 36 38 41 0.050 23 26 28 31 33 36 39 41 44 47 8

0.100 27 30 33 36 39 42 45 48 51 54 0.010 18 21 23 26 28 31 33 36 38 40 0.025 23 26 28 31 24 37 39 42 45 48 0.050 27 30 33 36 39 42 45 48 51 54 9

0.100 31 35 38 41 45 48 52 55 58 62 0.010 22 24 27 30 33 36 38 41 44 47 0.025 26 29 33 36 39 42 45 48 52 55 0.050 31 34 37 41 44 48 51 55 58 62 10

0.100 36 39 43 47 51 54 58 62 66 70

### EPA QA/G-9S 158 February 2006 

## TABLE A-8. CRITICAL VALUES FOR TH E WILCOXON RANK-SUM TEST (CONT.)



## Table values are the largest x values such that P( W rs 

## ≤ x ) ≤ α . Therefore, significance levels, α , are approximate. If there are ties , then the test is approximate.

max( m , n ) min( m , n ) α 11 12 13 14 15 16 17 18 19 20 0.010 25 28 31 34 37 41 44 47 50 53 0.025 30 33 37 40 44 47 51 55 58 62 0.050 34 38 42 46 50 54 57 61 65 69 11

0.100 40 44 48 52 57 61 65 69 73 78 0.010 31 35 38 52 46 48 53 56 60 0.025 37 41 45 49 53 57 61 65 69 0.050 42 47 51 55 60 64 68 72 77 12

0.100 49 53 58 63 68 72 77 81 86 0.010 39 43 47 51 55 59 63 67 0.025 45 50 54 59 63 67 72 76 0.050 51 56 61 65 70 75 80 84 13

0.100 58 63 68 74 79 84 89 94 0.010 47 51 56 60 65 69 73 0.025 55 59 64 69 74 78 83 0.050 61 66 71 77 82 87 92 14

0.100 69 74 80 85 91 97 102 0.010 56 61 66 70 75 80 0.025 64 70 75 80 85 90 0.050 72 77 83 88 94 100 15

0.100 80 86 92 98 104 110 0.010 66 71 76 82 87 0.025 75 81 86 92 98 0.050 83 89 95 101 107 16

0.100 93 99 106 112 119 0.010 77 82 88 93 0.025 87 93 99 105 0.050 96 102 109 115 17

0.100 106 113 120 127 0.010 88 94 100 0.025 99 106 112 0.050 109 116 123 18

0.100 120 128 135 0.010 101 107 0.025 113 119 0.050 123 130 19

0.100 135 143 0.010 114 0.025 127 0.050 138 20

0.100 151

### EPA QA/G-9S 159 February 2006 2 1 α χ

− 

## TABLE A-9. PERCENTILES OF THE CHI-SQUARE DISTRIBUTION

1 - α
df 0.005 0.010 0.025 0.050 0.100 0.900 0.950 0.975 0.990 0.995

1 2 3 4 5

6 7 8 9 10

11 12 13 14 15

16 17 18 19 20

21 22 23 24 25

26 27 28 29 30

40 50 60 70 80 90 100 0.0 4 393 0.0100

0.072 0.207 0.412

0.676 0.989

1.34 1.73 2.16

2.60 3.07 3.57 4.07 4.60

5.14 5.70 6.26 6.84 7.43

8.03 8.64 9.26 9.89 10.52

11.16 11.81 12.46 13.12 13.79

20.71 27.99 35.53 43.28 51.17 59.20 67.33 0.0 3 157 0.0201

0.115 0.297 0.554

0.872

1.24 1.65 2.09 2.56

3.05 3.57 4.11 4.66 5.23

5.81 6.41 7.01 7.63 8.26

8.90 9.54 10.20 10.86 11.52

12.20 12.88 13.56 14.26 14.95

22.16 29.71 37.48 45.44 53.54 61.75 70.06 0.0 3 982 0.0506

0.216 0.484 0.831

1.24 1.69 2.18 2.70 3.25

3.82 4.40 5.01 5.63 6.26

6.91 7.56 8.23 8.91 9.59

10.28 10.98 11.69 12.40 13.12

13.84 14.57 15.31 16.05 16.79

24.43 32.36 40.48 48.76 57.15 65.65 74.22 0.0 2 393 0.103 0.352 0.711 1.145

1.64 2.17 2.73 3.33 3.94

3.57 5.23 5.89 6.57 7.26

7.96 8.67 9.39 10.12 10.85

11.59 12.34 13.09 13.85 14.61

15.38 16.15 16.93 17.71 18.49

26.51 34.76 43.19 51.74 60.39 69.13 77.93 0.0158 0.211 0.584 1.064

1.61

2.20 2.83 3.49 4.17 4.87

5.58 6.30 7.04 7.79 8.55

9.31 10.09 10.86 11.65 12.44

13.24 14.04 14.85 15.66 16.47

17.29 18.11 18.94 19.77 20.60

29.05 37.69 46.46 53.33 64.28 73.29 82.36 2.71 4.61 6.25 7.78 9.24

10.64 12.02 13.36 14.68 15.99

17.28 18.55 19.81 21.06 22.31

23.54 24.77 25.99 27.20 28.41

29.62 30.81 32.01 33.20 34.38

35.56 36.74 37.92 39.09 40.26

51.81 63.17 74.40 85.53 96.58 107.6 118.5 3.84 5.99 7.81 9.49 11.07

12.59 14.07 15.51 16.92 18.31

19.68 21.03 22.36 23.68 25.00

26.30 27.59 28.87 30.14 31.41

32.67 33.92 35.17 36.42 37.65

38.89 40.11 41.34 42.56 43.77

55.76 67.50 79.08 90.53 101.9 113.1 124.3 5.02 7.38 9.35 11.14 12.83

14.45 16.01 17.53 19.02 20.48

21.92 23.34 24.74 26.12 27.49

28.85 30.19 31.53 32.85 34.17

35.48 36.78 38.08 39.36 40.65

41.92 43.19 44.46 45.72 46.98

59.34 71.42 83.30 95.02 106.6 118.1 129.6 6.63 9.21 11.34 13.28 15.09

16.81 18.48 20.09 21.67 23.21

24.73 26.22 27.69 29.14 30.58

32.00 33.41 34.81 36.19 37.57

38.93 40.29 41.64 42.98 44.31

45.64 46.96 48.28 49.59 50.89

63.69 76.15 88.38 100.4 112.3 124.1 135.8 7.88 10.60 12.84 14.86 16.75

18.55 20.28 21.96 23.59 25.19

26.76 28.30 29.82 31.32 32.80

34.27 35.72 37.16 38.58 40.00

41.40 42.80 44.18 45.56 46.93

48.29 49.64 50.99 52.34 53.67

66.77 79.49 91.95 104.2 116.3 128.3 140.2EPA QA/G-9S 160 February 2006F1- αTABLE A-10. PERCENTILES OF THEF-DISTRIBUTIONDegrees of Freedom for NumeratorDegreesFreedom forDenominator1 2 3 4 5 6 7 8 9 10 12 15 20 24 30 60 120∞1 .90.95.975.9939.9161648405249.5200800500053.6216864540355.8225900562557.2230922576458.2234937585958.9237948592859.4239957598159.9241963602260.2242969605660.7244977610661.2246985615761.7248993620962.0249997623562.32501001626162.82521010631363.12531014633963.3254101863662 .90.95.975.998.5318.538.598.59.0019.039.099.09.1619.239.299.29.2419.239.299.29.2919.339.399.39.3319.339.399.39.3519.439.499.49.3719.439.499.49.3819.439.499.49.3919.439.499.49.4119.439.499.49.4219.439.499.49.4419.439.499.49.4519.539.599.59.4619.539.599.59.4719.539.599.59.4819.539.599.59.4919.539.599.53 .90.95.975.995.5410.117.434.15.469.5516.030.85.399.2815.429.55.349.1215.128.75.319.0114.928.25.288.9414.727.95.278.8914.627.75.258.8514.527.55.248.8114.527.35.238.7914.427.25.228.7414.327.15.208.7014.326.95.188.6614.226.75.188.6414.126.65.178.6214.126.55.158.5714.026.35.148.5513.926.25.138.5313.926.14 .90.95.975.99.9994.547.7112.221.274.14.326.9410.618.061.24.196.599.9816.756.24.116.399.6016.053.44.056.269.3615.551.74.016.169.2015.250.53.986.099.0715.049.73.956.048.9814.849.03.946.008.9014.748.53.925.968.8414.548.13.905.918.7514.447.43.875.868.6614.246.83.845.808.5614.046.13.835.778.5113.945.83.825.758.4613.845.43.795.698.3613.744.73.785.668.3113.644.43.765.638.2613.544.15 .90.95.975.99.9994.066.6110.016.347.23.785.798.4313.337.13.625.417.7612.133.23.525.197.3911.431.13.455.057.1511.029.83.404.956.9810.728.83.374.886.8510.528.23.344.826.7610.327.63.324.776.6810.227.23.394.746.6210.126.93.274.686.529.8926.43.244.626.439.7225.93.214.566.339.5525.43.194.536.289.4725.13.174.506.239.3824.93.144.436.129.2024.33.124.406.079.1124.13.114.376.029.0223.86 .90.95.975.99.9993.785.998.8122.835.53.465.147.2610.927.03.294.766.609.7823.73.184.536.239.1521.93.114.395.998.7520.83.054.285.828.4720.03.014.215.708.2619.52.984.155.608.1019.02.964.105.527.9818.72.944.065.467.8718.42.904.005.377.7218.02.873.945.277.5617.62.843.875.177.4017.12.823.845.127.3116.92.803.815.077.2316.72.763.744.967.0616.22.743.704.906.9716.02.723.674.856.8815.7EPA QA/G-9S 161 February 2006F1- αTABLE A-10. PERCENTILES OF THEF-DISTRIBUTION (CONT.)Degrees of Freedom for NumeratorDegreesFreedom forDenominator1 2 3 4 5 6 7 8 9 10 12 15 20 24 30 60 120∞7 .90.95.975.99.9993.595.598.0712.229.23.264.746.549.5521.73.074.355.898.4518.82.964.125.527.8517.22.883.975.297.4616.22.833.875.127.1915.52.783.794.996.9915.02.753.734.906.8414.62.723.684.826.7214.52.703.644.766.6214.12.673.574.676.4713.72.633.514.576.3113.32.593.444.476.1612.92.583.414.426.0712.72.563.384.365.9912.52.513.304.255.8212.12.493.274.205.7411.92.473.234.145.6511.78 .90.95.975.99.9993.465.327.5711.325.43.114.466.068.6518.52.924.075.427.5915.82.813.845.057.0114.42.733.694.826.6313.52.673.584.656.3712.92.623.504.536.1812.42.593.444.436.0312.02.563.394.365.9111.82.543.354.305.8111.52.503.284.205.6711.22.463.224.105.5210.82.423.154.005.3610.52.403.123.955.2810.32.383.083.895.2010.12.343.013.785.039.732.322.973.734.959.532.292.933.674.869.339 .90.95.975.99.9993.365.127.2110.622.93.014.265.718.0216.42.813.865.086.9913.92.693.634.726.4212.62.613.484.486.0611.72.553.374.325.8011.12.513.294.205.6110.72.473.234.105.4710.42.443.184.035.3510.12.423.143.965.269.892.383.073.875.119.572.343.013.774.969.242.302.943.674.818.902.282.903.614.738.722.252.863.564.658.552.212.793.454.488.192.182.753.394.408.002.162.713.334.317.8110 .90.95.975.99.9993.294.966.9410.021.02.924.105.467.5614.92.733.714.836.5512.62.613.484.475.9911.32.523.334.245.6410.52.463.224.075.399.932.413.143.955.209.522.383.073.855.069.202.353.023.784.948.962.322.983.724.858.752.282.913.624.718.452.242.843.524.568.132.202.773.424.417.802.182.743.374.337.642.162.703.314.257.472.112.623.204.087.122.082.583.144.006.942.062.543.083.916.7612 .90.95.975.99.9993.184.756.559.3318.62.813.895.106.9313.02.613.494.475.9510.82.483.264.125.419.632.393.113.895.068.892.333.003.734.828.382.282.913.614.648.002.242.853.514.507.712.212.803.444.397.482.192.753.374.307.292.152.693.284.167.002.102.623.184.016.712.062.543.073.866.402.042.513.023.786.252.012.472.963.706.091.962.382.853.545.761.932.342.793.455.591.902.302.723.365.42EPA QA/G-9S 162 February 2006F1- αTABLE A-10. PERCENTILES OF THEF-DISTRIBUTION (CONT.)Degrees of Freedom for NumeratorDegreesFreedom forDenominator1 2 3 4 5 6 7 8 9 10 12 15 20 24 30 60 120∞15 .90.95.975.99.9993.074.546.208.6816.62.703.684.776.3611.32.493.294.155.429.342.363.063.804.898.252.272.903.584.567.572.212.793.414.327.092.162.713.294.146.742.122.643.204.006.472.092.593.123.896.262.062.543.063.806.082.022.482.963.675.811.972.402.863.525.541.922.332.763.375.251.902.292.703.295.101.872.252.643.214.951.822.162.523.054.641.792.112.462.964.481.762.072.402.874.3120 .90.95.975.99.9992.974.355.878.1014.82.593.494.465.859.952.383.103.864.948.102.252.873.514.437.102.162.713.294.106.462.092.603.133.876.022.042.513.013.705.692.002.452.913.565.441.962.392.843.465.241.942.352.773.375.081.892.282.683.234.821.842.202.573.094.561.792.122.462.944.291.772.082.412.864.151.742.042.352.784.001.681.952.222.613.701.641.902.162.523.541.611.842.092.423.3824 .90.95.975.99.9992.934.265.727.8214.02.543.404.326.669.342.333.013.724.727.552.192.783.384.226.592.102.623.153.905.982.042.512.993.675.551.982.422.873.505.231.942.362.783.364.991.912.302.703.264.801.882.252.643.174.641.832.182.543.034.391.782.112.442.894.141.732.032.332.743.871.701.982.272.663.741.671.942.212.583.591.611.842.082.403.291.571.792.012.313.141.531.731.942.212.9730 .90.95.975.99.9992.884.175.577.5613.32.493.324.185.398.772.282.923.594.517.052.142.693.254.026.122.052.533.033.705.531.982.422.873.475.121.932.332.753.304.821.882.272.653.174.581.852.212.573.074.391.822.162.512.984.241.772.092.412.844.001.722.012.312.703.751.621.932.202.553.491.641.892.142.473.361.611.842.072.393.221.541.741.942.212.921.501.681.872.112.761.461.621.792.012.5960 .90.95.975.99.9992.794.005.297.0812.02.393.153.934.987.772.182.763.344.136.172.042.533.013.655.311.952.372.793.344.761.872.252.633.124.371.822.172.512.954.091.772.102.412.823.861.742.042.332.723.691.711.992.272.633.541.661.922.172.503.321.601.842.062.353.081.541.751.942.202.831.511.701.882.122.691.481.651.822.032.551.401.531.671.842.251.351.471.581.732.081.291.391.481.601.89EPA QA/G-9S 163 February 2006F1- αTABLE A-10. PERCENTILES OF THEF-DISTRIBUTION (CONT.)Degrees of Freedom for NumeratorDegreesFreedom forDenominator1 2 3 4 5 6 7 8 9 10 12 15 20 24 30 60 120∞120 .90.95.975.99.9992.753.925.156.8511.42.353.073.804.797.322.132.683.233.955.781.992.452.893.484.951.902.292.673.174.421.822.182.522.964.041.772.092.392.793.771.722.022.302.663.551.681.962.222.563.381.651.912.162.473.241.601.832.052.343.021.551.751.952.192.781.481.661.822.032.531.451.611.761.952.401.411.551.691.862.261.321.431.531.661.951.261.351.431.531.771.191.251.311.381.54∞.90.95.975.99.9992.713.845.026.6310.82.303.003.694.616.912.082.603.123.785.421.942.372.793.324.621.852.212.573.024.101.772.1022.412.803.741.722.012.292.643.471.671.942.192.513.271.631.882.112.413.101.601.832.052.322.961.551.751.942.182.741.491.671.832.042.511.421.571.711.882.271.381.521.641.792.131.341.461.571.701.991.241.321.391.471.661.171.221.271.321.451.001.001.001.001.00

### EPA QA/G-9S 164 February 2006 

## TABLE A-11. VALUES OF THE PARAMETER $ 

## λ FOR COHEN'S ESTIMATES ADJUSTING FOR NONDETECTED VALUES

h

γ .01 .02 .03 .04 .05 .06 .07 .08 .09 .10 .15 .20

.00 .05 .10 .15 .20

.25 .30 .35 .40 .45

.50 .55 .60 .65 .70

.75 .80 .85 .90 .95 1.00 .010100 .010551 .010950 .011310 .011642

.011952 .012243 .012520 .012784 .013036

.013279 .013513 .013739 .013958 .014171

.014378 .014579 .014773 .014967 .015154 .015338 .020400 .021294 .022082 .022798 .023459

.024076 .024658 .025211 .025738 .026243

.026728 .027196 .027849 .028087 .028513

.029927 .029330 .029723 .030107 .030483 .030850 .030902 .032225 .033398 .034466 .035453

.036377 .037249 .038077 .038866 .039624

.040352 .041054 .041733 .042391 .043030

.043652 .044258 .044848 .045425 .045989 .046540 .041583 .043350 .044902 .046318 .047829

.048858 .050018 .051120 .052173 .053182

.054153 .055089 .055995 .056874 .057726

.058556 .059364 .060153 .060923 .061676 .062413 .052507 .054670 .056596 .058356 .059990

.061522 .062969 .064345 .065660 .066921

.068135 .069306 .070439 .071538 .072505

.073643 .074655 .075642 .075606 .077549 .078471 .063625 .066159 .068483 .070586 .072539

.074372 .076106 .077736 .079332 .080845

.082301 .083708 .085068 .086388 .087670

.088917 .090133 .091319 .092477 .093611 .094720 .074953 .077909 .080563 .083009 .085280

.087413 .089433 .091355 .093193 .094958

.096657 .098298 .099887 .10143 .10292

.10438 .10580 .10719 .10854 .10987 .11116 .08649 .08983 .09285 .09563 .09822

.10065 .10295 .10515 .10725 .10926

.11121 .11208 .11490 .11666 .11837

.12004 .12167 .12225 .12480 .12632 .12780 .09824 .10197 .10534 .10845 .11135

.11408 .11667 .11914 .12150 .12377

.12595 .12806 .13011 .13209 .13402

.13590 .13775 .13952 .14126 .14297 .14465 .11020 .11431 .11804 .12148 .12469

.12772 .13059 .13333 .13595 .13847

.14090 .14325 .14552 .14773 .14987

.15196 .15400 .15599 .15793 .15983 .16170 .17342 .17925 .18479 .18985 .19460

.19910 .20338 .20747 .21129 .21517

.21882 .22225 .22578 .22910 .23234

.23550 .23858 .24158 .24452 .24740 .25022 .24268 .25033 .25741 .26405 .27031

.27626 .28193 .28737 .29250 .29765

.30253 .30725 .31184 .31630 .32065

.32489 .32903 .33307 .33703 .34091 .34471

h

γ .25 .30 .35 .40 .45 .50 .55 .60 .65 .70 .80 .90

.00 .05 .10 .15 .20

.25 .30 .35 .40 .45

.50 .55 .60 .65 .70

.75 .80 .85 .90 .95 1.00 .31862 .32793 .33662 .34480 .35255

.35993 .36700 .37379 .38033 .38665

.39276 .39679 .40447 .41008 .41555

.42090 .42612 .43122 .43622 .44112 .44592 .4021 .4130 .4233 .4330 .4422

.4510 .4595 .4676 .4735 .4831

.4904 .4976 .5045 .5114 .5180

.5245 .5308 .5370 .5430 .5490 .5548 .4941 .5066 .5184 .5296 .5403

.5506 .5604 .5699 .5791 .5880

.5967 .6061 .6133 .6213 .6291

.6367 .6441 .6515 .6586 .6656 .6724 .5961 .6101 .6234 .6361 .6483

.6600 .6713 .6821 .6927 .7029

.7129 .7225 .7320 .7412 .7502

.7590 .7676 .7781 .7844 .7925 .8005 .7096 .7252 .7400 .7542 .7673

.7810 .7937 .8060 .8179 .8295

.8408 .8517 .8625 .8729 .8832

.8932 .9031 .9127 .9222 .9314 .9406 .8388 .8540 .8703 .8860 .9012

.9158 .9300 .9437 .9570 .9700

.9826 .9950 1.007 1.019 1.030

1.042 1.053 1.064 1.074 1.085 1.095 .9808 .9994 1.017 1.035 1.051

1.067 1.083 1.098 1.113 1.127

1.141 1.155 1.169 1.182 1.195

1.207 1.220 1.232 1.244 1.255 1.287 1.145 1.166 1.185 1.204 1.222

1.240 1.257 1.274 1.290 1.306

1.321 1.337 1.351 1.368 1.380

1.394 1.408 1.422 1.435 1.448 1.461 1.336 1.358 1.379 1.400 1.419

1.439 1.457 1.475 1.494 1.511

1.528 1.545 1.561 1.577 1.593

1.608 1.624 1.639 1.653 1.668 1.882 1.561 1.585 1.608 1.630 1.651

1.672 1.693 1.713 1.732 1.751

1.770 1.788 1.806 1.824 1.841

1.851 1.875 1.892 1.908 1.924 1.940 2.176 2.203 2.229 2.255 2.280

2.305 2.329 2.353 2.376 2.399

2.421 2.443 2.465 2.486 2.507

2.528 2.548 2.568 2.588 2.607 2.626 3.283 3.314 3.345 3.376 3.405

3.435 3.464 3.492 3.520 3.547

3.575 3.601 3.628 3.654 3.679

3.705 3.730 3.754 3.779 3.803 3.827

### EPA QA/G-9S 165 February 2006 

## TABLE A-12a: CRITI CAL VALUES FOR THE MANN-KENDALL TEST FOR TREND

Sample size n Significance Level

α 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

0.20 4 6 7 7 8 10 11 13 14 16 17 19 20 22 25 27 28
0.10 6 8 9 11 12 14 17 19 20 24 25 29 30 34 35 39 42
0.05 6 8 11 13 16 18 21 23 26 28 31 35 38 42 45 49 52
0.01 - 10 13 17 20 24 27 31 34 40 43 47 52 58 63 67 72



## TABLE A-12b: PROBABILITIES FOR THE SMALL-SAMPLE MANN-KENDALL TEST FOR TREND

n n S 4 5 8 9 S 6 7 10 0 0.625 0.592 0.548 0.540 1 0.500 0.500 0.500 2 0.375 0.408 0.452 0.460 3 0.360 0.386 0.431 4 0.167 0.242 0.360 0.381 5 0.235 0.281 0.364 6 0.042 0.117 0.274 0.306 7 0.136 0.191 0.300 8 0.042 0.199 0.238 9 0.068 0.119 0.242 10 0.0083 0.138 0.179 11 0.028 0.068 0.190 12 0.089 0.130 13 0.0083 0.035 0.146 14 0.054 0.090 15 0.0014 0.015 0.108 16 0.031 0.060 17 0.0054 0.078 18 0.016 0.038 19 0.0014 0.054 20 0.0071 0.022 21 0.00020 0.036 22 0.0028 0.012 23 0.023 24 0.00087 0.0063 25 0.014 26 0.00019 0.0029 27 0.0083 28 0.000025 0.0012 29 0.0046 30 0.00043 31 0.0023 32 0.00012 33 0.0011 34 0.000025 35 0.00047 36 0.0000028 37 0.00018 39 0.000058 41 0.000015 43 0.0000028 45 0.00000028EPA QA/G-9S 166 February 2006TABLE A-13. QUANTILES FOR THEWALD-WOLFOWITZ TEST FOR RUNSnmW0.01W0.05W0.10nmW0.01W0.05W0.10nmW0.01W0.05W0.10nmW0.01W0.05W0.104--35344634575--36344745684566-347345845694567-3483459456104678-34934510467115679-3410445115671256710-451145612567135771134512456135781457812345134561457815678133451456615578166781435615567166781768815356165671767818789164561756718678197891745618567196787207891845619567620678419456520567EPA QA/G-9S 167 February 2006TABLE A-13. QUANTILES FOR THE WALD-WOLFOWITZ TEST FOR RUNS (CONT.)nmW0.01W0.05W0.10nmW0.01W0.05W0.10nmW0.01W0.05W0.10nmW0.01W0.05W0.10204568856699567101078911117899567106781178912789105771167812789137910116781267813791014891012678137891489101589101367814789158910168101114688157991689111791011156891679101781011189101116789178910189101119911121778918891019911122091112187910198101120911121979102081011207910EPA QA/G-9S 168 February 2006TABLE A-13. QUANTILES FOR THE WALD-WOLFOWITZ TEST FOR RUNS (CONT.)nmW0.01W0.05W0.10nmW0.01W0.05W0.10nmW0.01W0.05W0.10nmW0.01W0.05W0.10121279101313910111414911121515810111381010149111215911121691112148101015101112161011131710121315810111610111217101213181012141691112171011121810121319111315179111218101113191113142012141618911121910121320111314191012132011121320101213161610121317171113141818131416191914161717111314181113151913141620141617EPA QA/G-9S 169 February 2006TABLE A-13. QUANTILES FOR THE WALD-WOLFOWITZ TEST FOR RUNS (CONT.)nmW0.01W0.05W0.10nmW0.01W0.05W0.10nmW0.01W0.05W0.10nmW0.01W0.05W0.101811 13 141912 14 162014 15 161912141520121416201314162020141617Whennormis greater than 20 theWpquantile is given by:()()()Wmnm nzmn mn m nmn mnp p= +++− −+ +−122 212wherezpis the appropriate quantile from the standard normal (see Table A-1).

### EPA QA/G-9S 170 February 2006 

## TABLE A-14. CRITICAL VALUES FOR THE SLIPPAGE TEST LEVEL OF SIGNIFICANCE ( 

## α ) APPROXIMATELY 0.01

n = size of population 1 (site)

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

5 5 6 6 7 8 8 9 9 10 11 11 12 12 13 14 14
6 5 5 6 6 7 8 8 9 9 10 10 11 11 12 12 13

7 5 5 6 6 7 7 8 8 9 9 10 10 10 11 11 12
8 4 5 5 6 6 7 7 8 8 8 9 9 10 10 11 11
9 4 5 5 5 6 6 7 7 8 8 8 9 9 10 10 10
10 4 4 5 5 6 6 6 7 7 7 8 8 9 9 9 10

11 4 4 5 5 5 6 6 6 7 7 7 8 8 9 9 9

12 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9
13 4 4 4 5 5 5 6 6 6 7 7 7 7 8 8 8

14 4 4 4 4 5 5 5 6 6 6 7 7 7 7 8 8
15 3 4 4 4 5 5 5 5 6 6 6 7 7 7 7 8
16 3 4 4 4 4 5 5 5 6 6 6 6 7 7 7 7
17 3 4 4 4 4 5 5 5 5 6 6 6 6 7 7 7
18 3 3 4 4 4 5 5 5 5 5 6 6 6 6 7 7
19 3 3 4 4 4 4 5 5 5 5 6 6 6 6 6 7m= size of population 2 (background)

20 3 3 4 4 4 4 5 5 5 5 5 6 6 6 6 7

n = size of population 1 (site)

25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
25 7 8 8 9 10 11 12 13 13 14 15 16 17 18 19 19
30 6 7 7 8 9 10 10 11 12 13 13 14 15 14 16 17
35 5 6 7 7 8 9 9 10 11 11 12 12 13 14 14 15
40 5 6 6 7 7 8 9 9 10 10 11 11 12 12 13 13
45 5 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12
50 5 5 5 6 6 7 7 8 8 9 9 10 10 10 11 11

55 4 5 5 6 6 6 7 7 8 8 9 9 9 10 10 11
60 4 5 5 5 6 6 7 7 7 8 8 8 9 9 10 10
65 4 4 5 5 5 6 6 7 7 7 8 8 8 9 9 9
70 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9 9

75 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9
80 4 4 4 5 5 5 6 6 6 6 7 7 7 8 8 8
85 4 4 4 4 5 5 5 6 6 6 6 7 7 7 8 8
90 3 4 4 4 5 5 5 5 6 6 6 6 7 7 7 8
95 3 4 4 4 4 4 5 5 6 6 6 6 7 7 7 7m= size of population 2 (background)

100 3 4 4 4 4 5 5 5 5 6 6 6 6 7 7 7

### EPA QA/G-9S 171 February 2006 

## TABLE A-14. CRITICAL VALUES FOR THE SLIPPAGE TEST (CONT.) LEVEL OF SIGNIFICANCE ( 

## α ) APPROXIMATELY 0.05

n = size of population 1 (site)

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

5 4 4 5 5 6 6 7 7 8 8 9 9 9 10 10 11
6 4 4 4 5 5 6 6 6 7 7 8 8 8 9 9 10

7 3 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9
8 3 4 4 4 5 5 5 6 6 6 6 7 7 7 8 8
9 3 3 4 4 4 5 5 5 5 6 6 6 7 7 7 7
10 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7 7

11 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7

12 3 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6
13 3 3 3 3 4 4 4 4 4 5 5 5 5 6 6 6

14 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 6
15 3 3 3 3 3 4 4 4 4 4 4 5 5 5 5 5
16 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5
17 2 3 3 3 3 3 4 4 4 4 4 4 5 5 5 5
18 2 3 3 3 3 3 3 4 4 4 4 4 4 5 5 5
19 2 3 3 3 3 3 3 3 4 4 4 4 4 4 5 5m= size of population 2 (background)

20 2 2 3 3 3 3 3 3 4 4 4 4 4 4 4 5

n = size of population 1 (site)

25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
25 5 5 6 6 7 8 8 9 9 10 10 11 12 12 13 13
30 4 5 5 6 6 7 7 8 8 9 9 9 10 10 11 11
35 4 4 5 5 6 6 6 7 7 8 8 8 9 9 10 10
40 4 4 4 5 5 5 6 6 7 7 7 8 8 8 9 9
45 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8
50 3 3 4 4 4 5 5 5 6 6 5 7 7 7 7 8

55 3 3 4 4 4 4 5 5 5 6 6 6 6 7 7 7
60 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7
65 3 3 3 4 4 4 4 5 5 5 5 5 6 6 6 6
70 3 3 3 3 4 4 4 4 5 5 5 5 5 6 6 6

75 3 3 3 3 4 4 4 4 4 5 5 5 5 5 6 6
80 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 6
85 3 3 3 3 3 3 3 4 4 4 4 5 5 5 5 5
90 2 3 3 3 3 3 4 4 4 4 4 4 5 5 5 5
95 2 3 3 3 3 3 3 4 4 4 4 4 4 5 5 5m= size of population 2 (background)

100 2 3 3 3 3 3 3 4 4 4 4 4 4 4 5 5

### EPA QA/G-9S 172 February 2006 

## TABLE A-14. CRITICAL VALUES FOR THE SLIPPAGE TEST (CONT.) LEVEL OF SIGNIFICANCE ( 

## α ) APPROXIMATELY 0.10

n = size of population 1 (site)

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

5 3 4 4 4 5 5 6 6 6 7 7 7 8 8 9 9
6 3 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8

7 3 3 3 4 4 4 5 5 5 5 6 6 6 7 7 7
8 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7
9 3 3 3 3 4 4 4 4 4 5 5 5 5 6 6 6
10 2 3 3 3 3 4 4 4 4 4 5 5 5 5 5 6

11 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5

12 2 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5
13 2 2 3 3 3 3 3 3 4 4 4 4 4 4 5 5

14 2 2 2 3 3 3 3 3 3 4 4 4 4 4 4 5
15 2 2 2 3 3 3 3 3 3 3 4 4 4 4 4 4
16 2 2 2 3 3 3 3 3 3 3 4 4 4 4 4 4
17 2 2 2 2 3 3 3 3 3 3 3 4 4 4 4 4
18 2 2 2 2 3 3 3 3 3 3 3 3 4 4 4 4
19 2 2 2 2 2 3 3 3 3 3 3 3 3 4 4 4m= size of population 2 (background)

20 2 2 2 2 2 3 3 3 3 3 3 3 3 3 4 4

n = size of population 1 (site)

25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
25 4 4 5 5 6 6 6 7 7 8 8 9 9 9 10 10
30 3 4 4 4 5 5 6 6 6 7 7 7 8 8 9 9
35 3 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8
40 3 3 3 4 4 4 5 5 5 5 6 6 6 7 7 7
45 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7
50 3 3 3 3 4 4 4 4 4 5 5 5 5 6 6 6

55 2 3 3 3 3 4 4 4 4 4 5 5 5 5 5 6
60 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5
65 2 2 3 3 3 3 3 4 4 4 4 4 4 5 5 5
70 2 2 3 3 3 3 3 3 4 4 4 4 4 4 5 5

75 2 2 2 3 3 3 3 2 3 4 4 4 4 4 4 5
80 2 2 2 3 3 3 3 3 3 3 4 4 4 4 4 4
85 2 2 2 3 3 3 3 3 3 3 4 4 4 4 4 4
90 2 2 2 2 3 3 3 3 3 3 3 4 4 4 4 4
95 2 2 2 2 3 3 3 3 3 3 3 3 4 4 4 4m= size of population 2 (background)

100 2 2 2 2 2 3 3 3 3 3 3 3 3 4 4 4EPA QA/G-9S 173 February 2006TABLE A-15. DUNNETT’S TEST (ONE TAILED)Total Number of Investigated Groups (k– 1)DegreesofFreedomα23456789101214162.05.103.802.544.342.924.713.205.083.405.243.575.433.715.603.835.753.945.884.036.114.196.294.326.454.443.05.102.942.133.282.413.522.613.702.763.852.873.972.974.083.064.173.134.253.204.393.314.513.414.613.494.05.102.611.962.882.203.082.373.222.503.342.603.442.683.522.753.592.823.662.873.772.973.863.053.943.115.05.102.441.872.682.092.852.242.982.363.082.453.162.533.242.593.302.653.362.703.452.783.532.863.602.926.05.102.341.822.562.022.712.172.832.272.922.363.002.433.062.493.122.543.172.593.262.673.332.743.482.797.05.102.271.782.481.982.822.112.732.222.812.302.892.372.952.423.002.473.052.523.132.593.202.663.262.718.05.102.221.752.421.942.552.082.662.172.742.252.812.322.872.382.922.422.962.473.042.543.112.603.162.659.05.102.181.732.371.922.502.052.602.142.682.222.752.282.812.342.862.392.902.432.972.503.042.563.092.6110.05.102.151.712.341.902.472.022.562.122.642.192.702.262.762.312.812.352.852.402.922.462.982.523.032.5712.05.102.111.692.291.872.411.992.502.082.582.162.642.222.692.272.742.312.782.352.842.422.902.472.952.5216.05.102.061.662.231.832.341.952.432.042.502.112.562.172.612.222.652.262.692.302.752.362.812.412.852.46EPA QA/G-9S 174 February 2006TABLE A-15. DUNNETT’S TEST (ONE TAILED) (CONT.)Total Number of Investigated Groups (k– 1)DegreesofFreedomα2 3 4 5 6 78 9 10 12 14 1620.05.102.031.642.191.812.301.932.392.012.462.082.512.142.562.192.602.232.642.262.702.332.752.382.802.4224.05.102.011.632.172.802.281.912.362.002.432.062.482.122.532.172.572.212.602.242.662.302.722.352.762.4030.05.101.991.622.151.792.251.902.341.982.402.052.452.102.502.152.542.192.572.222.632.282.682.332.722.3740.05.101.971.612.131.772.231.882.311.962.372.032.422.082.472.132.512.172.542.202.602.262.652.312.692.3550.05.101.961.612.111.772.221.882.291.962.322.022.412.072.452.122.492.162.522.192.582.252.632.302.672.3460.05.101.951.602.101.762.211.872.281.952.342.012.402.062.442.112.482.152.512.182.572.242.612.292.652.3370.05.101.951.602.101.762.211.872.281.952.342.012.402.062.442.112.482.152.512.182.562.242.612.292.652.3380.05.101.941.602.101.762.201.872.281.952.342.012.392.062.432.102.472.152.502.182.552.232.602.282.642.3290.05.101.941.602.091.762.201.862.271.942.332.002.392.062.432.102.472.142.502.172.552.232.602.282.632.31100.05.101.931.592.081.752.181.852.271.932.331.992.382.052.422.092.462.142.492.172.542.222.592.272.632.31120.05.101.931.592.081.752.181.852.261.932.321.992.372.052.412.092.452.132.482.162.532.222.582.272.622.31∞.05.101.921.582.061.732.161.842.231.922.291.982.342.032.382.072.422.112.452.142.502.202.552.242.582.28

### EPA QA/G-9S 175 February 2006 

## TABLE A-16. APPROXIMATE α -LEVEL CRITICAL POINTS FOR RANK VON NEUMANN RATIO TEST

α

n 0.05 0.10

4 5 6 7 8 9 10 0.70 0.80 0.86 0.93 0.98 1.04 0.60 0.97 1.11 1.14 1.18 1.23

11 12 13 14 15 16 17 18 19 20 1.08 1.11 1.14 1.17 1.19 1.21 1.24 1.26 1.27 1.29 1.26 1.29 1.32 1.34 1.36 1.38 1.40 1.41 1.43 1.44

21 22 23 24 25 26 27 28 29 30 1.31 1.32 1.33 1.35 1.36 1.37 1.38 1.39 1.40 1.41 1.45 1.46 1.48 1.49 1.50 1.51 1.51 1.52 1.53 1.54

32 34 36 38 40 42 44 46 48 50 1.43 1.45 1.46 1.48 1.49 1.50 1.51 1.52 1.53 1.54 1.55 1.57 1.58 1.59 1.60 1.61 1.62 1.63 1.63 1.64

55 60 65 70 75 80 85 90 95 100 1.56 1.58 1.60 1.61 1.62 1.64 1.65 1.66 1.66 1.67 1.66 1.67 1.68 1.70 1.71 1.71 1.72 1.73 1.74 1.74

### EPA QA/G-9S 176 February 2006 

## TABLE A-17. VALUES OF H 1- α 

## = H 0.90 

## FOR COMPUTING A ONE-SIDED UPPER 90% CONFIDENCE LIMIT ON A LOGNORMAL MEAN

n s y 3 5 7 10 12 15 21 31 51 101
0.10 1.686 1.438 1.381 1.349 1.338 1.328 1.317 1.308 1.301 1.295 0.20 1.885 1.522 1.442 1.396 1.380 1.365 1.348 1.335 1.324 1.314 0.30 2.156 1.627 1.517 1.453 1.432 1.411 1.388 1.370 1.354 1.339 0.40 2.521 1.755 1.607 1.523 1.494 1.467 1.437 1.412 1.390 1.371 0.50 2.990 1.907 1.712 1.604 1.567 1.532 1.494 1.462 1.434 1.409 0.60 3.542 2.084 1.834 1.696 1.650 1.606 1.558 1.519 1.485 1.454 0.70 4.136 2.284 1.970 1.800 1.743 1.690 1.631 1.583 1.541 1.504 0.80 4.742 2.503 2.119 1.914 1.845 1.781 1.710 1.654 1.604 1.560 0.90 5.349 2.736 2.280 2.036 1.955 1.880 1.797 1.731 1.672 1.621 1.00 5.955 2.980 2.450 2.167 2.073 1.985 1.889 1.812 1.745 1.686 1.25 7.466 3.617 2.904 2.518 2.391 2.271 2.141 2.036 1.946 1.866 1.50 8.973 4.276 3.383 2.896 2.733 2.581 2.415 2.282 2.166 2.066 1.75 10.48 4.944 3.877 3.289 3.092 2.907 2.705 2.543 2.402 2.279 2.00 11.98 5.619 4.380 3.693 3.461 3.244 3.005 2.814 2.648 2.503 2.50 14.99 6.979 5.401 4.518 4.220 3.938 3.629 3.380 3.163 2.974 3.00 18.00 8.346 6.434 5.359 4.994 4.650 4.270 3.964 3.697 3.463 3.50 21.00 9.717 7.473 6.208 5.778 5.370 4.921 4.559 4.242 3.965 4.00 24.00 11.09 8.516 7.062 6.566 6.097 5.580 5.161 4.796 4.474 4.50 27.01 12.47 9.562 7.919 7.360 6.829 6.243 5.769 5.354 4.989 5.00 30.01 13.84 10.61 8.779 8.155 7.563 6.909 6.379 5.916 5.508 6.00 36.02 16.60 12.71 10.50 9.751 9.037 8.248 7.607 7.048 6.555 7.00 42.02 19.35 14.81 12.23 11.35 10.52 9.592 8.842 8.186 7.607 8.00 48.03 22.11 16.91 13.96 12.96 12.00 10.94 10.08 9.329 8.665 9.00 54.03 24.87 19.02 15.70 14.56 13.48 12.29 11.32 10.48 9.725 10.00 60.04 27.63 21.12 17.43 16.17 14.97 13.64 12.56 11.62 10.79 ALUES FOR COMPUTING A ONE-SIDED CONFIDENCE LIMIT ON A LOGNORMAL MEAN 

## TABLE A-17. VALUES OF H α



## = H 0.10 

## FOR COMPUTING A ONE-SIDED LOWER 10% CONFIDENCE LIMIT ON A LOGNORMAL MEAN

n s y 3 5 7 10 12 15 21 31 51 101
0.10 -1.431 -1.320 -1.296 -1.285 -1.281 -1.279 -1.277 -1.277 -1.278 -1.279 0.20 -1.350 -1.281 -1.268 -1.266 -1.266 -1.266 -1.268 -1.272 -1.275 -1.280 0.30 -1.289 -1.252 -1.250 -1.254 -1.257 -1.260 -1.266 -1.272 -1.280 -1.287 0.40 -1.245 -1.233 -1.239 -1.249 -1.254 -1.261 -1.270 -1.279 -1.289 -1.301 0.50 -1.213 -1.221 -1.234 -1.250 -1.257 -1.266 -1.279 -1.291 -1.304 -1.319 0.60 -1.190 -1.215 -1.235 -1.256 -1.266 -1.277 -1.292 -1.307 -1.324 -1.342 0.70 -1.176 -1.215 -1.241 -1.266 -1.278 -1.292 -1.310 -1.329 -1.349 -1.370 0.80 -1.168 -1.219 -1.251 -1.280 -1.294 -1.311 -1.332 -1.354 -1.377 -1.403 0.90 -1.165 -1.227 -1.264 -1.298 -1.314 -1.333 -1.358 -1.383 -1.409 -1.439 1.00 -1.166 -1.239 -1.281 -1.320 -1.337 -1.358 -1.387 -1.414 -1.445 -1.478 1.25 -1.184 -1.280 -1.334 -1.384 -1.407 -1.434 -1.470 -1.507 -1.547 -1.589 1.50 -1.217 -1.334 -1.400 -1.462 -1.491 -1.523 -1.568 -1.613 -1.663 -1.716 1.75 -1.260 -1.398 -1.477 -1.551 -1.585 -1.624 -1.677 -1.732 -1.790 -1.855 2.00 -1.310 -1.470 -1.562 -1.647 -1.688 -1.733 -1.795 -1.859 -1.928 -2.003 2.50 -1.426 -1.634 -1.751 -1.862 -1.913 -1.971 -2.051 -2.133 -2.223 -2.321 3.00 -1.560 -1.817 -1.960 -2.095 -2.157 -2.229 -2.326 -2.427 -2.536 -2.657 3.50 -1.710 -2.014 -2.183 -2.341 -2.415 -2.499 -2.615 -2.733 -2.864 -3.007 4.00 -1.871 -2.221 -2.415 -2.596 -2.681 -2.778 -2.913 -3.050 -3.200 -3.366 4.50 -2.041 -2.435 -2.653 -2.858 -2.955 -3.064 -3.217 -3.372 -3.542 -3.731 5.00 -2.217 -2.654 -2.897 -3.126 -3.233 -3.356 -3.525 -3.698 -3.889 -4.100 6.00 -2.581 -3.104 -3.396 -3.671 -3.800 -3.949 -4.153 -4.363 -4.594 -4.849 7.00 -2.955 -3.564 -3.904 -4.226 -4.377 -4.549 -4.790 -5.037 -5.307 -5.607 8.00 -3.336 -4.030 -4.418 -4.787 -4.960 -5.159 -5.433 -5.715 -6.026 -6.370 9.00 -3.721 -4.500 -4.937 -5.352 -5.547 -5.771 -6.080 -6.399 -6.748 -7.136 10.00 -4.109 -4.973 -5.459 -5.920 -6.137 -6.386 -6.730 -7.085 -7.474 -7.906

### EPA QA/G-9S 177 February 2006 

## TABLE A-17. VALUES OF H 1- α 

## = H 0.95 

## FOR COMPUTING A ONE-SIDED UPPER 95% CONFIDENCE LIMIT ON A LOGNORMAL MEAN

n s y 3 5 7 10 12 15 21 31 51 101
0.10 2.750 2.035 1.886 1.802 1.775 1.749 1.722 1.701 1.684 1.670 0.20 3.295 2.198 1.992 1.881 1.843 1.809 1.771 1.742 1.718 1.697 0.30 4.109 2.402 2.125 1.977 1.927 1.882 1.833 1.793 1.761 1.733 0.40 5.220 2.651 2.282 2.089 2.026 1.968 1.905 1.856 1.813 1.777 0.50 6.495 2.947 2.465 2.220 2.141 2.068 1.989 1.928 1.876 1.830 0.60 7.807 3.287 2.673 2.368 2.271 2.181 2.085 2.010 1.946 1.891 0.70 9.120 3.662 2.904 2.532 2.414 2.306 2.191 2.102 2.025 1.960 0.80 10.43 4.062 3.155 2.710 2.570 2.443 2.307 2.202 2.112 2.035 0.90 11.74 4.478 3.420 2.902 2.738 2.589 2.432 2.310 2.206 2.117 1.00 13.05 4.905 3.698 3.103 2.915 2.744 2.564 2.423 2.306 2.205 1.25 16.33 6.001 4.426 3.639 3.389 3.163 2.923 2.737 2.580 2.447 1.50 19.60 7.120 5.184 4.207 3.896 3.612 3.311 3.077 2.881 2.713 1.75 22.87 8.250 5.960 4.795 4.422 4.081 3.719 3.437 3.200 2.997 2.00 26.14 9.387 6.747 5.396 4.962 4.564 4.141 3.812 3.533 3.295 2.50 32.69 11.67 8.339 6.621 6.067 5.557 5.013 4.588 4.228 3.920 3.00 39.23 13.97 9.945 7.864 7.191 6.570 5.907 5.388 4.947 4.569 3.50 45.77 16.27 11.56 9.118 8.326 7.596 6.815 6.201 5.681 5.233 4.00 52.31 18.58 13.18 10.38 9.469 8.630 7.731 7.024 6.424 5.908 4.50 58.85 20.88 14.80 11.64 10.62 9.669 8.652 7.854 7.174 6.590 5.00 65.39 23.19 16.43 12.91 11.77 10.71 9.579 8.688 7.929 7.277 6.00 78.47 27.81 19.68 15.45 14.08 12.81 11.44 10.36 9.449 8.661 7.00 91.55 32.43 22.94 18.00 16.39 14.90 13.31 12.05 10.98 10.05 8.00 104.6 37.06 26.20 20.55 18.71 17.01 15.18 13.74 12.51 11.45 9.00 117.7 41.68 29.64 23.10 21.03 19.11 17.05 15.43 14.05 12.85 10.00 130.8 46.31 32.73 25.66 23.35 21.22 18.93 17.13 15.59 14.26



## TABLE A-17. VALUES OF H α



## = H 0.05 

## FOR COMPUTING A ONE-SIDED LOWER 5% CONFIDENCE LIMIT ON A LOGNORMAL MEAN

n s y 3 5 7 10 12 15 21 31 51 101
0.10 -2.130 -1.806 -1.731 -1.690 -1.677 -1.666 -1.655 -1.648 -1.644 -1.642 0.20 -1.949 -1.729 -1.678 -1.653 -1.646 -1.640 -1.636 -1.636 -1.637 -1.641 0.30 -1.816 -1.669 -1.639 -1.627 -1.625 -1.625 -1.627 -1.632 -1.638 -1.648 0.40 -1.717 -1.625 -1.611 -1.611 -1.613 -1.617 -1.625 -1.635 -1.647 -1.662 0.50 -1.644 -1.594 -1.594 -1.603 -1.609 -1.618 -1.631 -1.646 -1.663 -1.683 0.60 -1.589 -1.573 -1.584 -1.602 -1.612 -1.625 -1.643 -1.662 -1.685 -1.711 0.70 -1.549 -1.560 -1.582 -1.608 -1.622 -1.638 -1.661 -1.686 -1.713 -1.744 0.80 -1.521 -1.555 -1.586 -1.620 -1.636 -1.656 -1.685 -1.714 -1.747 -1.783 0.90 -1.502 -1.556 -1.595 -1.637 -1.656 -1.680 -1.713 -1.747 -1.785 -1.826 1.00 -1.490 -1.562 -1.610 -1.658 -1.681 -1.707 -1.745 -1.784 -1.827 -1.874 1.25 -1.486 -1.596 -1.662 -1.727 -1.758 -1.793 -1.842 -1.893 -1.949 -2.012 1.50 -1.508 -1.650 -1.733 -1.814 -1.853 -1.896 -1.958 -2.020 -2.091 -2.169 1.75 -1.547 -1.719 -1.819 -1.916 -1.962 -2.015 -2.088 -2.164 -2.247 -2.341 2.00 -1.598 -1.799 -1.917 -2.029 -2.083 -2.144 -2.230 -2.318 -2.416 -2.526 2.50 -1.727 -1.986 -2.138 -2.283 -2.351 -2.430 -2.540 -2.654 -2.780 -2.921 3.00 -1.880 -2.199 -2.384 -2.560 -2.644 -2.740 -2.874 -3.014 -3.169 -3.342 3.50 -2.051 -2.429 -2.647 -2.855 -2.953 -3.067 -3.226 -3.391 -3.574 -3.780 4.00 -2.237 -2.672 -2.922 -3.161 -3.275 -3.406 -3.589 -3.779 -3.990 -4.228 4.50 -2.434 -2.924 -3.206 -3.476 -3.605 -3.753 -3.960 -4.176 -4.416 -4.685 5.00 -2.638 -3.183 -3.497 -3.798 -3.941 -4.107 -4.338 -4.579 -4.847 -5.148 6.00 -3.062 -3.715 -4.092 -4.455 -4.627 -4.827 -5.106 -5.397 -5.721 -6.086 7.00 -3.499 -4.260 -4.699 -5.123 -5.325 -5.559 -5.886 -6.227 -6.608 -7.036 8.00 -3.945 -4.812 -5.315 -5.800 -6.031 -6.300 -6.674 -7.066 -7.502 -7.992 9.00 -4.397 -5.371 -5.936 -6.482 -6.742 -7.045 -7.468 -7.909 -8.401 -8.953 10.00 -4.852 -5.933 -6.560 -7.168 -7.458 -7.794 -8.264 -8.755 -9.302 -9.918

### EPA QA/G-9S 178 February 2006 

## TABLE A-18. CRITICAL VAL UES FOR THE SIGN TEST

The table values are such that for order statistics x [ lower ] 

### and x [ upper ] 

### , [] [] 

# ( ) α

− ≥ < < 1 median true P upper lower 

### x x . Note that the significance levels are for tw o-sided tests. For one-sided test, divide the significance level in half.

α

0.20 0.10 0.05 0.02 0.01 n lower upper lower upper lower upper lower upper lower upper

5 1 5 1 5 - - - - - -
6 1 6 1 6 1 6 - - - -
7 2 6 1 7 1 7 1 7 - -
8 2 7 2 7 1 8 1 8 1 8
9 3 7 2 8 2 8 1 9 1 9
10 3 8 2 9 2 9 1 10 1 10
11 3 9 3 9 2 10 2 10 1 11
12 4 9 3 10 3 10 2 11 2 11
13 4 10 4 10 3 11 2 12 2 12
14 5 10 4 11 3 12 3 12 2 13
15 5 11 4 12 4 12 3 13 3 13
16 5 12 5 12 4 13 3 14 3 14
17 6 12 5 13 5 13 4 14 3 15
18 6 13 6 13 5 14 4 15 4 15
19 7 13 6 14 5 15 5 15 4 16
20 7 14 6 15 6 15 5 16 4 17
21 8 14 7 15 6 16 5 17 5 17
22 8 15 7 16 6 17 6 17 5 18
23 8 16 8 16 7 17 6 18 5 19
24 9 16 8 17 7 18 6 19 6 19
25 9 17 8 18 8 18 7 19 6 20
26 10 17 9 18 8 19 7 20 7 20
27 10 18 9 19 8 20 8 20 7 21
28 11 18 10 19 9 20 8 21 7 22
29 11 19 10 20 9 21 8 22 8 22
30 11 20 11 20 10 21 9 22 8 23
31 12 20 11 21 10 22 9 23 8 24
32 12 21 11 22 10 23 9 24 9 24
33 13 21 12 22 11 23 10 24 9 25 34 13 22 12 23 11 24 10 25 10 25
35 14 22 13 23 12 24 11 25 10 26

36 14 23 13 24 12 25 11 26 10 27
37 15 23 14 24 13 25 11 27 11 27
38 15 24 14 25 13 26 12 27 11 28
39 16 24 14 26 13 27 12 28 12 28 40 16 25 15 26 14 27 13 28 12 29EPA QA/G-9S 179 February 2006TABLE A-19. CRITICAL VALUES FOR THE QUANTILE TESTmis the number of background samples,nis the number of site samples, andcis the number values larger than thethb1quantile.Ifsis greater than or equal to the table value,q α, then reject the null hypothesis of no difference at the given significance level.N4 5 6 7 8 9 10 11 12

0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90c4 2 1 4 2 1 5 3 1 5 3 1 6 3 2 6 3 2 7 4 2 7 4 2 8 4 20.1070.054 4 5 5 6 6 7 840.01c4 2 1 5 3 1 5 3 1 6 3 2 6 3 2 7 4 2 7 4 2 8 4 2 8 4 20.1030.055 6 7 7 8 850.014 5 6c5 3 1 5 3 1 6 3 2 6 3 2 7 4 2 7 4 2 8 4 2 8 4 2 9 5 20.104 3 3 5 6 4 6 4 7 7 8 50.054 3 560.015 6 6 7 7 8 8 9c5 3 1 6 3 2 6 3 2 7 4 2 7 4 2 8 4 2 8 4 2 9 5 2 9 5 20.103 4 4 4 5 50.054 3 3 5 6 4 6 7 7 8 870.015 6 7 7 8 8 9 9c6 3 2 6 3 2 7 4 2 7 4 2 8 4 2 8 4 2 9 5 2 9 5 2 10 5 20.102 4 5 6 4 7 8 50.054 3 3 4 4 4 7 5 8 580.015 6 6 7 8 8 9 9c6 3 2 7 4 2 7 4 2 8 4 2 8 4 2 9 5 2 9 5 2 10 5 2 10 5 20.102 3 7 80.054 3 5 5 4 6 4 6 4 7 5 5 8 5 590.014 6 7 7 8 8 9 9c7 4 2 7 4 2 8 4 2 8 4 2 9 5 2 9 5 2 10 5 2 10 5 2 11 6 30.102 3 2 5 6 4 7 80.054 3 6 4 7 5 8 5 8 5 9 6100.014 5 4 6 4 7 7 5 8 9 9 10c7 4 2 8 4 2 8 4 2 9 5 2 9 5 2 10 5 2 10 5 2 11 6 3 11 6 30.102 3 2 3 6 4 4 7 5 8 50.054 3 5 5 6 4 7 5 8110.014 4 6 4 7 5 7 5 8 5 8 9 6 9 6c8 4 2 8 4 2 9 5 2 9 5 2 10 5 2 10 5 2 11 6 3 11 6 3 12 6 30.103 2 5 2 6 4 4 7 3 5 3 50.054 3 2 4 6 4 7 8 5 8 9m120.014 5 4 6 5 7 5 7 5 8 5 9 6 9 6 10 6EPA QA/G-9S 180 February 2006TABLE A-19. CRITICAL VALUES FOR THE QUANTILE TEST (CONT.)mis the number of background samples,nis the number of site samples, andcis the number values larger than thethb1quantile.Ifsis greater than or equal to the table value,q α, then reject the null hypothesis of no difference at the given significance level.n4 5 6 7 8 9 10 11 12

0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90c8 4 2 9 5 2 9 5 2 10 5 2 10 5 2 11 6 3 11 6 3 12 6 3 12 6 30.103 2 2 6 3 7 3 3 8 30.054 3 2 5 5 4 6 4 4 7 5 8 5 8 5 9 5130.014 4 6 5 7 5 7 5 8 6 9 6 9 6 10 6c9 5 2 9 5 2 10 5 2 10 5 2 11 6 3 11 6 3 12 6 3 12 6 3 13 7 30.103 2 5 2 2 6 4 3 3 30.054 3 2 5 4 6 4 7 5 3 7 5 3 8 5 8 5 9 6140.014 4 6 5 7 5 8 6 8 6 9 6 9 6 10 7c9 5 2 10 5 2 10 5 2 11 6 3 11 6 3 12 6 3 12 6 3 13 7 3 13 7 30.103 2 5 2 4 6 4 7 3 8 5 3 8 30.054 3 2 5 4 6 3 3 7 5 3 8 5 6 9 6150.014 4 6 5 7 5 7 5 8 6 9 6 9 7 10 7c10 5 2 10 5 2 11 6 3 11 6 3 12 6 3 12 6 3 13 7 3 13 7 3 14 7 30.103 5 6 4 4 5 5 3 5 30.054 3 2 5 2 4 3 6 4 3 7 3 7 5 3 8 3 8 9 6160.014 4 6 5 7 5 8 5 8 6 9 6 9 6 10 7c10 5 2 11 6 3 11 6 3 12 6 3 12 6 3 13 7 3 13 7 3 14 7 3 14 7 30.105 6 4 8 5 8 5 30.054 3 2 5 4 3 6 4 3 3 7 5 3 8 5 3 3 9 6170.014 4 3 6 5 7 5 7 5 8 6 9 6 9 6 10 7c11 6 3 11 6 3 12 6 3 12 6 3 13 7 3 13 7 3 14 7 3 14 7 3 15 8 30.102 3 3 5 5 30.054 3 5 4 6 4 3 7 5 3 7 5 3 8 5 3 8 3 9 6180.014 3 4 6 5 3 7 5 8 6 8 6 9 6 9 6 10 7c11 6 3 12 6 3 12 6 3 13 7 3 13 7 3 14 7 3 14 7 3 15 8 3 15 8 30.102 3 2 5 4 6 4 7 7 8 50.054 3 5 4 6 3 7 5 3 5 3 8 5 3 9 6 3 9 6 3190.014 3 4 3 6 5 3 7 5 8 6 8 6 9 6 10 7 10 7c12 6 3 12 6 3 13 7 3 13 7 3 14 7 3 14 7 3 15 8 3 15 8 3 16 8 40.102 3 2 5 4 8 50.054 3 5 4 6 4 3 7 3 7 5 3 8 5 3 6 3 9 6 4m200.014 3 4 3 6 5 3 7 5 8 5 8 6 9 6 9 7 10 7EPA QA/G-9S 181 February 2006TABLE A-19. CRITICAL VALUES FOR THE QUANTILE TEST (CONT.)mis the number of background samples,nis the number of site samples, andcis the number values larger than thethb1quantile.Ifsis greater than or equal to the table value,q α, then reject the null hypothesis of no difference at the given significance level.n13 14 15 16 17 18 19 20

0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90c8 4 2 9 5 2 9 5 2 10 5 2 10 5 2 11 6 3 11 6 3 12 6 30.108 9 10 110.059 10 11 1240.01c9 5 2 9 5 2 10 5 2 10 5 2 11 6 3 11 6 3 12 6 3 12 6 30.100.059 9 10 10 11 11 12 1250.01c9 5 2 10 5 2 10 5 2 11 6 3 11 6 3 12 6 3 12 6 3 13 7 30.108 9 9 10 10 11 11 120.0560.019 10 10 11 11 12 12 13c10 5 2 10 5 2 11 6 3 11 6 3 12 6 3 12 6 3 13 7 3 13 7 30.105 5 6 6 6 7 70.059 9 10 10 11 11 12 1270.0110 10 11 11 12 12 13 13c10 5 2 11 6 3 11 6 3 12 6 3 12 6 3 13 7 3 13 7 3 14 7 30.105 9 10 6 6 7 70.059 10 6 10 6 11 11 12 7 12 1380.0110 11 11 12 12 13 13 14c11 6 3 11 6 3 12 6 3 12 6 3 13 7 3 13 7 3 14 7 3 14 7 30.109 10 11 12 120.059 6 6 10 6 6 11 7 7 7 790.0110 10 11 11 12 12 13 13c11 6 3 12 6 3 12 6 3 13 7 3 13 7 3 14 7 3 14 7 3 15 8 30.100.059 6 10 6 10 6 11 7 11 7 12 7 12 7 13 8100.0110 11 11 12 12 13 13 14c12 6 3 12 6 3 13 7 3 13 7 3 14 7 3 14 7 3 15 8 3 15 8 30.109 10 6 10 11 11 12 120.059 6 10 6 11 7 7 12 7 13 8 13 8110.0110 11 11 7 12 12 13 14 14c12 6 3 13 7 3 13 7 3 14 7 3 14 7 3 15 8 3 15 8 3 16 8 40.105 6 6 6 7 70.059 10 10 11 11 7 12 12 13 8m120.0110 6 11 7 11 7 12 7 12 13 8 13 8 14EPA QA/G-9S 182 February 2006TABLE A-19. CRITICAL VALUES FOR THE QUANTILE TEST (CONT.)mis the number of background samples,nis the number of site samples, andcis the number values larger than thethb1quantile.Ifsis greater than or equal to the table value,q α, then reject the null hypothesis of no difference at the given significance level.n13 14 15 16 17 18 19 20

0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90 0.50 0.75 0.90c13 7 3 13 7 3 14 7 3 14 7 3 15 8 3 15 8 3 16 8 4 16 8 40.109 9 10 6 10 6 11 11 7 12 7 12 70.056 10 6 11 11 12 7 12 13 13130.0110 7 11 7 12 7 12 7 13 8 13 8 14 8 14 8c13 7 3 14 7 3 14 7 3 15 8 3 15 8 3 16 8 4 16 8 4 17 9 40.103 4 12 7 40.059 6 10 6 10 6 11 7 11 7 12 7 13 8140.0110 7 11 7 11 7 12 8 12 8 13 8 13 8 14 9c14 7 3 14 7 3 15 8 3 15 8 3 16 8 4 16 8 4 17 9 4 17 9 40.109 3 9 3 10 11 4 4 12 4 40.0510 6 10 6 11 7 11 7 12 7 12 7 13 8 13 8150.0111 7 11 7 12 8 12 8 13 8 13 8 14 9 14 9c14 7 3 15 8 3 15 8 3 16 8 4 16 8 4 17 9 4 17 9 4 18 9 40.103 6 3 10 6 4 11 4 7 4 12 4 40.059 6 10 7 11 7 12 7 12 8 13 8 13 8160.0110 7 11 7 11 8 12 8 13 8 13 9 14 9 14 9c15 8 3 15 8 3 16 8 4 16 8 4 17 9 4 17 9 4 18 9 4 18 9 40.109 3 6 3 10 6 6 11 7 4 7 4 12 7 4 40.0510 6 10 11 7 4 11 7 4 12 12 8 13 8 13 8170.0111 7 11 7 12 8 12 8 13 8 13 9 14 9 14 9c15 8 3 16 8 4 16 8 4 17 9 4 17 9 4 18 9 4 18 9 4 19 10 40.109 3 10 6 11 7 4 12 7 4 13 40.056 10 6 4 11 4 11 7 4 12 7 4 12 13 8 14 8180.0110 7 11 7 12 7 12 8 13 8 13 8 14 9 15 9c16 8 4 16 8 4 17 9 4 17 9 4 18 9 4 18 9 4 19 10 4 19 10 40.109 10 11 7 12 4 40.0510 6 4 10 6 4 11 7 4 11 7 4 12 7 4 12 4 13 8 13 8190.0111 7 11 7 12 8 12 8 13 8 13 8 14 9 14 9c16 8 4 17 9 4 17 9 4 18 9 4 18 9 4 19 10 4 19 10 4 20 10 40.109 6 10 11 12 7 12 13 40.0510 6 4 10 7 4 11 7 4 11 7 4 12 7 4 13 8 4 13 8 4 14 8m200.0111 7 11 8 12 8 12 8 13 8 14 9 14 9 15 9

### EPA QA/G-9S 183 February 2006 

## APPENDIX B:

REFERENCES

### EPA QA/G-9S 184 February 2006 

## APPENDIX B: REFERENCES



## This appendix provides references for the topics and procedures described in this document. The references are broken into thr ee groups: Primary, Basic Statistics Textbooks, and Secondary. This classification do es not refer in any way to the subject matter content but to the relevance to the intended audien ce for this document, ease in unde rstanding statistical concepts and methodologies, and accessibility to the non-sta tistical community. Primary references are those thought to be of particular benefit as ha nds-on material, where the degree of sophistication demanded by the writer seldom re quires extensive training in statistics; most of these references should be on an environmental statistician’s books helf. Users of this document are encouraged to send recommendations on additional references to the address listed in the Foreword.

Some sections within the ch apters reference materials found in most introductory statistics books. This document uses Walpol e and Myers (1985), Freedman, Pisani, Purves, and Adhakari (1991), Mendenhall (1987), and Dixon and Massey (1983). Table B-1 (at the end of this appendix) lists specific chapters in these books where topics contained in this guidance may be found. This list could be extended much further by use of other basic textbooks; this is acknowledged by the simple statement that furthe r information is available from introductory text books.

Some important books specific to the analysis of environmental da ta include: Gilbert (1987), an excellent all-round ha ndbook having strength in sampling, estimation, and hot-spot detection; Gibbons (1994), a book specifically concen trating on the application of statistics to groundwater problems with emphasis on method detection limits, censored data, and the detection of outliers; and Madansky (1988), a slig htly more theoretical volume with important chapters on the testing for Normality, transformations, and testing for independence. In addition, Ott (1995) describes modeling, probabilistic processes, and the Lognormal distribution of contaminants, and Berthouex and Brown (1994) pr ovide an engineering approach to problems including estimation, experimental design and the fitting of models. Millard and Neerchal (2001) has excellent discussions of applied stat istical methods using the software package SPlus. Ginevan and Splitstone ( 2004) contains applied examples of many of the statistical tests discussed in this guidance and in cludes some more sophisticated tests for the more statistically minded. Gibbons and Coleman (2001) apply sophis ticated statistic al theory to environmental issues, but the text is not intended for the casual reader.

B.1 CHAPTER 1



## Chapter 1 establishes the framework of qualita tive and quantitative criteria against which the data that has been collected will be assessed . The most important feature of this chapter is the concept of the test of hypot heses framework which is descri bed in any introductory textbook. A non-technical exposition of hypothe sis testing is also to be found in U.S. EPA (2000, 1994b) which provides guidance on planning for environmen tal data collection. An application of the DQO Process to geostatistical error mana gement may be found in Myers (1997).

### EPA QA/G-9S 185 February 2006 

## A full discussion of sampling methods with the attendant theory are to be found in Gilbert (1987) and a shorter di scussion may be found in U.S. EPA (1989). Cochran (1966) and Kish (1965) also provide more a dvanced theoretical c oncepts but may require the assistance of a statistician for full comprehension. More s ophisticated sampling designs such as composite sampling, adaptive sampling, and ranked set samp ling, will be discussed in future Agency guidance.

B.2 CHAPTER 2



## Standard statistical quantities and graphica l representations are discussed in most introductory statistics books. In addition, Berthouex & Brown (1994) and Madansky (1988) both contain thorough discussions on the subject . There are also several textbooks devoted exclusively to graphical representations, in cluding Cleveland (1993), which may contain the most applicable methods for environmental data, Tufte (1983), and Chambers, Cleveland, Kleiner and Tukey (1983).

Two EPA sources for temporal data that keep theoretical discussions to a minimum are U.S. EPA (1992a) and U.S. EPA (1992b). For a more complete discussion on temporal data, specifically time series analysis, see Box and Jenkins (1970), Wei (1990) , or Ostrum (1978). These more complete references provide both th eory and practice; however, the assistance of a statistician may be needed to adapt the me thodologies for immediate use. Theoretical discussions of spatial data may be found in J ournel and Huijbregts (1978) , Cressie (1993), and Ripley (1981).

B.3 CHAPTER 3



## The hypothesis tests covered in this edition of the guidance are well known and straightforward; basic statistics texts cover these s ubjects. Besides basic statistical text books, Berthouex & Brown (1994), Hardin and Gilber t (1993), and U.S. EPA (1989, 1994) may be useful to the reader. In addition, there are some statistics books devoted specifically to hypothesis testing, for example, see Lehmann (1 991). These books may be too theoretical for most practitioners, and their application to en vironmental situations may not be obvious.

The statement in this document that the si gn test requires approximately 1.225 times as many observations as the Wilcoxon rank sum test to achieve a give n power at a given significance level is attribut able to Lehmann (1975).

B.4 CHAPTER 4



## This chapter is essentially a compendium of statistical tests drawn mostly from the primary references and basic st atistics textbooks. Gilbert (19 87) and Madansky (1988) have an excellent collection of technique s and U.S. EPA (1992a) contains techniques specific to water problems.

### EPA QA/G-9S 186 February 2006 

## For Normality (Section 4.2), Madansky (1988) has an excellent discussion on tests as does Shapiro (1986). For trend testing (Section 4. 3), Gilbert (1987) has an excellent discussion on statistical tests and U.S. EPA (1992) provides adjustments for trends and seasonality in the calculation of descriptive statistics.

There are several very good textbooks devoted to the treatment of outliers (Section 4.4). Two authoritative texts are Barnett and Lewis (1978) and Hawkins (1980). Additional information is also to be found in Beckman and Cook (1983) and Tietjen and Moore (1972).

Tests for dispersion (Section 4. 5) are described in the basi c textbooks and examples are to be found in U.S. EPA (1992a). Transformation of data (Section 4.6) is a sensitive topic and thorough discussions may be found in Gilbert (1987), and Dixon and Massey (1983). Equally sensitive is the analysis of da ta where some values are record ed as non-detected (Section 4.7); Gibbons (1994) and U.S. EPA (1992a) have relevant discussions and examples.

B.5 CHAPTER 5



## Chapter 5 discusses some of the philosophica l issues related to hypothesis testing which may help in understanding and communicating the te st results. Although there are no specific references for this chapter, many topics (e.g., th e use of p-values) are di scussed in introductory textbooks. Future editions of this guidance will be expanded by incorporating practical experiences from the environmenta l community into this chapter.

B.6 LIST OF REFERENCES

B.6.1 Primary References



## Berthouex, P.M., and L.C. Brown, 1994. Statistics for Environmental Engineers . Lewis, Boca Raton, FL.

Gibbons, R.D. and D. E. Coleman, 2001. Statistical Methods for Detection and Quantification of Environmental Contamination . John Wiley, New York, NY.

Gilbert, R.O., 1987. Statistical Methods for Envir onmental Pollution Monitoring. John Wiley, New York, NY.

Ginevan, M.E. and D.E. Splitstone, 2004. Statistical Tools for Environmental Quality Measurement . Chapman and Hall, Boca Raton, FL.

Myers, J.C., 1997. Geostatistical Error Management , John Wiley, New York, NY.

Ott, W.R., 1995. Environmental Statistics and Data Analysis. Lewis, Boca Raton, FL.

### EPA QA/G-9S 187 February 2006 

## U.S. Environmental Protection Agency , 1992a. Addendum to Interim Final Guidance Document on the Statistical Analysis of Ground-Water Monitoring Data at RCRA Facilities. EPA/530/R-93/003. Office of Solid Waste. (NTIS: PB89-151026)

U.S. Environmental Protection Agency, 2000. Guidance for the Data Quality Objectives Process (EPA QA/G4). EPA/600/R-96/055. Of fice of Research and Development.

U.S. Environmental Protection Agency, 2002. Choosing a Sampling Design for Environmental Data Collection (EPA QA/G5S). Office of Environmental Information.

U.S. Environmental Protection Agency, 2004. Data Quality Assessment: A Reviewer’s Guide (Final Draft) (EPA QA/G9R). Office of Environmental Information.

B.6.2 Basic Statistics Textbooks



## Dixon, W.J., and F.J. Massey, Jr., 1983. Introduction to Statistical Analysis (Fourth Edition). McGraw-Hill, New York, NY.

Freedman, D. R. Pisani, R. Purves, and A. Adhikari, 1991. Statistics. W.W. Norton & Co., New York, NY.

Mendenhall, W., 1987. Introduction to Probability and Statistics (Seventh Edition). PWS-Kent, Boston, MA.

Walpole, R., and R. Myers, 1985. Probability and Statistics for Engineers and Scientists (Third Ed.). MacMillan, New York, NY.

B.6.3 Secondary References



## Aitchison, J., 1955. On the distribution of a positive random variable having a discrete probability mass at the origin. Journal of American Statistical Association 50(272):901-8

Barnett, V., and T. Lewis, 1978. Outliers in Statistical Data. John Wiley, New York, NY.

Beckman, R.J., and R.D. Cook, 1983. Outlier......s, Technometrics 25:119-149.

Box, G.E.P., and G.M. Jenkins, 1970. Time Series Analysis, Fo recasting, and Control. HoldenDay, San Francisco, CA.

Chambers, J.M, W.S. Cleveland, B. Kleiner, and P.A. Tukey, 1983. Graphical Methods for Data Analysis. Wadsworth & Brooks/Cole Pub lishing Co., Pacific Grove, CA.

Chen, L., 1995. Testing the mean of skewed distributions, Journal of the American Statistical Association 90:767-772.

### EPA QA/G-9S 188 February 2006 

## Cleveland, W.S., 1993. Visualizing Data. Hobart Press, Summit, NJ. Cochran, W.G., 1966. Sampling Techniques (Third Edition). J ohn Wiley, New York, NY.

Cohen, A.C., Jr. 1959. Simplified estimators for the normal distribution when samples are singly censored or truncated, Technometrics 1:217-237.

Conover, W.J., 1980. Practical Nonparam etric Statistics (Second Edition). John Wiley, New York, NY.

Cressie, N., 1993. Statistics for Spatial Data. John Wiley, New York, NY.

D'Agostino, R.B., 1971. An omnibus test of nor mality for moderate and large size samples, Biometrika 58:341-348.

David, H.A., H.O. Hartley, and E.S. Pearson, 1954. The distribution of the ratio, in a single normal sample, of range to standard deviation, Biometrika 48:41-55.

Dixon, W.J., 1953. Processing data for outliers, Biometrika 9:74-79.

Filliben, J.J., 1975. The probability plot correlation coefficient test for normality, Technometrics 17:111-117.

Geary, R.C., 1947. Testing for normality, Biometrika 34:209-242.

Geary, R.C., 1935. The ratio of the mean devia tion to the standard deviation as a test of normality, Biometrika 27:310-32.

Gibbons, R. D., 1994. Statistical Methods for Groundwater Monitoring . John Wiley, New York, NY.

Grubbs, F.E., 1969. Procedures for detec ting outlying observations in samples, Technometrics 11:1-21.

Hardin, J.W., and R.O. Gilbert, 1993. Comparing Statistical Tests for Detecting Soil Contamination Greater than Background, Re port to U.S. Department of Energy, PNL8989, UC-630, Pacific Northwest Laboratory, Richland, WA. Hawkins, D.M., 1980. Identification of Outliers. Chapman and Hall, New York, NY.

Hochberg, Y., and A. Tamhane, 1987. Multiple Comparison Procedures . John Wiley, New York, NY.

Journel, A.G., and C.J. Huijbregts, 1978. Mining Geostatistics. Academic Press, London.

Kish, L., 1965. Survey Sampling . John Wiley, New York, NY.

### EPA QA/G-9S 189 February 2006 

## Kleiner, B., and J.A. Hartigan, 1981. Repr esenting points in many dimensions by trees and castles, Journal of the American Statistical Association 76:260.

Lehmann, E.L., 1991. Testing Statistical Hypotheses . Wadsworth & Brooks/Cole Publishing Co., Pacific Grove, CA.

Lehmann, E.L., 1975. Nonparametrics: Statistical Methods Based on Ranks . Holden-Day, Inc., San Francisco, CA.

Lilliefors, H.W., 1969. Correction to the pa per "On the Kolmogorov-Smirnov test for normality with mean and variance unknown," Journal of the American Statistical Association 64:1702.

Lilliefors, H.W., 1967. On the Kolmogorov-Smirnov test for normality with mean and variance unknown, Journal of the American Statistical Association 64:399-402.

Madansky, A., 1988. Prescriptions for Working Statisticians. Springer-Verlag, New York, NY.

Millard, S.P. and N.K. Neerchal, 2001. Envi ronmental Statistics with S-Plus. CRC Press, Boca Raton, FL.

Ostrum, C.W., 1978. Time Series Analysis (Second Edition). Sage University Papers Series, Vol 9. Beverly Hills and London.

Ripley, B.D., 1981. Spatial Statistics. John Wiley and Sons, Somerset, NJ.

Rosner, B., 1975. On the detection of many outliers, Technometrics 17:221-227.

Royston, J.P., 1982. An extension of Shapiro an d Wilk's W test for norma lity to large samples, Applied Statistics 31:161-165.

Sen, P.K., 1968a. Estimates of the regr ession coefficient based on Kendall's tau, Journal of the American Statistical Association 63:1379-1389.

Sen, P.K., 1968b. On a class of alig ned rank order tests in two-way layouts, Annals of Mathematical Statistics 39:1115-1124. Shapiro, S., 1986. Volume 3: How to Test Normality and Other Distributional Assumptions. American Society for Quality Control, Milwaukee, WI.

Shapiro, S., and M.B. Wilk, 1965. An anal ysis of variance test for normality (complete samples), Biometrika 52:591-611.

Stefansky, W., 1972. Rejecting outliers in factorial designs, Technometrics 14:469-478.

### EPA QA/G-9S 190 February 2006 

## Stephens, M.A., 1974. EDF statistics for goodness-of-fit and some comparisons, Journal of the American Statistical Association 90:730-737.

Tietjen, G.L., and R.M. Moore, 1972. Some Gr ubbs-type statistics for th e detection of several outliers, Technometrics 14:583-597.

Tufte, E.R., 1983. The Visual Display of Quantitative Information. Graphics Press, Cheshire, CN.

Tufte, E.R., 1990. Envisioning Information. Graphics Press, Cheshire, CN.

U. S. Environmental Protection Agency, 1989. Methods for Evaluating the Attainments of Cleanup Standards: Volume 1: Soils and Solid Media. EPA/230/02-89-042. Office of Policy, Planning, and Evaluation. (NTIS: PB89-234959 )

U. S. Environmental Protection Agency, 1992b. Methods for Evaluating the Attainments of Cleanup Standards: Volume 2: Ground Water. EPA/230/R-92/014. Office of Policy, Planning, and Evaluation. (NTIS: PB94-138815)

U. S. Environmental Protection Agency, 1994. Methods for Evaluating the Attainments of Cleanup Standards: Volume 3: Reference-Based Standards . EPA/230/R-94-004. Office of Policy, Planning, and Ev aluation. (NTIS: PB94-176831)

Walsh, J.E., 1958. Large sample nonparametr ic rejection of outlying observations, Annals of the Institute of Statistical Mathematics 10:223-232.

Walsh, J.E., 1953. Correction to "Some nonparametr ic tests of whether th e largest observations of a set are too large or too small," Annals of Mathematical Statistics 24:134-135.

Walsh, J.E., 1950. Some nonparametric tests of wh ether the largest observa tions of a set are too large or too small, Annals of Mathematical Statistics 21:583-592.

Wang, Peter C.C., 1978. Graphical Representation of Multivariate Data. Academic Press, New York, NY.

Wegman, Edward J., 1990. Hyperdimensional da ta analysis using pa rallel coordinates, Journal of the American Statistical Association 85: 664.

Wei, W.S., 1990. Time Series Analysis (Second Edition) . Addison Wesley, Menlo Park, CA.