Cente

AN ASME REPORT CRTD Vol. 60



### Final Report

________________________________________________________________


# Reference Method Accuracy and Precision (ReMAP): Phase 1

Precision of Manual Stack Emission

Measurements

__ ___________________________________________________________________________



## Prepared by:

W. Steven Lanier GE Energy and Environmental Research Corporation

Charles D. Hendrix

Statistical Consultant

under the auspices of: American Society of Mechanical Engineers Research Committee on Industrial and Municipal Waste



# February 2001Reference Method Accuracy and Precision (ReMAP Phase I) An Assessment of the
Precision of EP A Manual Stack Emission Measurements

Executive Summary

This report documents re sults from the first phase of a study co

- sponsored by the American Society of Mechanical Engineers (ASME) to assess the accuracy and precision of manual test methods adopted by the US Environmental Protection Agency (EP A) for determining the stack

concent ration of selected air pollutants. The program is entitled Reference Method Accuracy and Precision and is referred to by the acronym ReMAP. The Phase I effort addresses the precision of the selected measurement methods. The formal Purpose Statement for the program is:

"To determine the precision of pollutant emission measurements based on analysis of

available simultaneous

- sample test data which were generated using EPA Manual Reference Test Methods 5 and 5i (Particulate Matter), 23 (Dioxin and Furan), 2 6 (HCl),

29 (multi

- metals), 101a and 101b (mercury) and 108 (arsenic) at a number of stationary

air sources. "

As used in the ReMAP program, precision is defined as random error that inadvertently enters the measurement process. This error may enter at a ny stage of the measurement process including sample collection, sample recovery, or sample analysis. The impact of such errors is that

measurement results deviate from the true stack concentration. Because these errors occur randomly, data from repeated a pplication of any manual test method (to a hypothetical stack with a chosen average concentration of the selected pollutant) should result in a bell shaped frequency

distribution. This distribution is centered about the chosen stack concentration but data are expected indicating measured concentrations both above and below the chosen stack value.

Precision of a measurement method is indicated by the horizontal spread of the bell curve. One common way to characterize the bell curve shape is to determine th e estimated standard deviation (s) of the distribution. Alternately, the bell distribution also describes the range of measurement results anticipated from repeated application of the measurement method at the given concentration. Based upon the best estim ate of the standard deviation, it is a simple matter to calculate other indicators of data quality that have direct practical significance. The ReMAP program selected two additional, directly derived parameters to characterize precision of manual

methods:

1. The anticipated range for 99 out of 100 future single measurements. If the measurement method is repeatedly applied to a stack with a given concentration, this precision metric defines the upper and lower concentration bounds for 99% of individual me asurements.

2. The anticipated range for 99 out of 100 future triplicate measurements. Since most environmental regulations define the reportable stack concentration as the average of three repeated test runs, this metric defines the anticipated range of results in triplicate (3 single time series) measurements due to random error in the measurement process.

For each of these precision metrics it is important to note the inherent assumption that the measurement method is being applied to a hypothetical stack with an average pollutant concentration that does not vary with time. In real

- world applications, single and triplicate measurement results will indicate variation from both random errors and systematic errors (bias) in the measurement process as wel l as temporal variation in facility operation.As noted, the primary objective of the ReMAP program is to characterize the bell shaped curve

for each measurement method by first estimating the standard deviation s. The other two precision metrics are directly calculated from s. Specifically 99 out of 1 00 single measurements will fall within ± 2.54 s of the true concentration. The anticipated range for the average of repeated measurements comes progressively closer to the true concentration. More precisely, the anticipated range varies inversely with the square root of the number of measurements. Thus, if s is the standard deviation of anticipated single measurements, the anticipated range for 99 out of 100 future triplicate measurements will fall within ± 2.54 s /v 3 of the true concentration.

A centra l tenet of the ReMAP program is to determine how the precision of manual measurement methods varies with the concentration of the pollutant being measured. A constant measurement standard deviation might occur if the primary source of random error is an an alytical process. A simple example might be the repeatability with which a technician can measure the weight gain on a particulate filter. The capabilities of the weighing scale do not significantly vary with the particulate weight gain. However, the magni tude of random errors associated with extraction and recovery of the sample from the stack might be expected to vary in proportion to stack

concentration.

The technical effort of the ReMAP program is to estimate the standard deviation of selected EPA

Me asurement Reference Methods as a function of average stack concentration. ReMAP is not intended to validate any of the Reference Methods addressed herein nor is it to be used as a

substitute for Method 301.

The concentration of pollutants released from i ndustrial facilities must be assumed to vary with time. Accordingly, results from typical, single

- train stack tests are of limited value for determination of the precision of a measurement method. Instead, estimation of method precision

must be based on da ta from special tests where multiple sampling trains are used to simultaneously determine the stack pollutant concentration. Such multi

- train tests minimize the impact of temporal and spatial (where probes are co

- located) variations on the data and the res ults can be used to estimate the standard deviation of the particular measurement at the specific stack

concentration.

The ReMAP program performed a careful assessment of the statistical analysis procedures required to estimate the precision of Manual Re ference Methods using multi

- train sampling data (see Appendix). To assure the quality of data used in the statistical analysis, an extensive effort was expended in gathering data from the original sources and carefully evaluating them to assure

that consis tent data reduction procedures were used.

Conceptually, the ReMAP statistical analysis procedure is straightforward. First, data from a

multi

- train test run are averaged to provide an estimate of the average concentration for the run (C i ). The standard d

eviation for the test run (S i ) is also calculated. Clearly, a calculated standard deviation from a single test using a dual sampling probe provides a relatively poor estimate of the true standard deviation of the method ( s) at the true concentration (μ). However, after accounting for various biases, a significant array of data from multi

- train tests should provide a reasonable basis for estimating the true standard deviation as a function of concentration. The ReMAP proced ure is to assume that the standard deviation varies with concentration according to a power function relationship and then to fit the data to that equation using regression analysis.

Results from the regression analysis represent the best estimate availa ble on the standard deviation of the measurement method at any given concentration. However, the ReMAP analysis procedure also provides for calculation of confidence intervals on the regression. Theseconfidence intervals define the upper and lower bounds for the regression line at the 95%

confidence level. Based on the regression line and the confidence intervals, the various precision

metrics can be determined for each Method as a function of concentration. Results from the analyses are summarized below.

EP A Method 5 and 5i for Particulate Matter

- Front Half Only Method 5 was one of the first EP A Reference Methods developed for stack sampling. There is a relatively large body of multi

- train data indicating that the relative standard deviation (RSD) h as minimal variation with concentration. Over a broad concentration range, RSD is predicted to remain between 5 and 11 %. There are certain data comparability concerns associated with the Method 5 precision analysis. However, assuming that the Method is ap plied to a stack with a particulate matter concentration less than 150 mg/dscm, the best available estimate is that RSD will be below 10%. Moreover, the influence of random error in the measurement process should result in the average of triplicate measur ements deviating from the true average concentration by no more than about ±14.7% (±2.54*10%/ v3).

Method 5i was specifically developed for application to stacks with particulate concentrations below 50 mg/dscm. The Method itself requires dual

- train sampli

ng and provides an upper limit

on the allowable deviation between the simultaneous measurements. Thus the available data had been pre screened to eliminate test results with large standard deviation (i.e., > about 14% RSD). This prescreening, coupled with the relatively small concentration range for the data, resulted in the ReMAP analysis finding no statistically significant variation of standard deviation with

concentration.

Based on a pooled analysis, the characteristic standard deviation for Method 5i was found to be 1.43 mg/dscm. Based on this best estimate of standard deviation. the ReMAP analysis indicates
that 99 out of l00

Method Si single measurements should deviate from the true concentration by no more than ± 3.68 mg/dscm. For triplicate measur ements 99 out of l00 Method 5i data results should deviate from the true concentration by no more than ± 2.12 mg/dscm.

Method 23 for Dioxin and Furan

Data collected with Method 23 are used to report dioxin and furan emissions as either the total

mass o f tetra through octa chlorinated dioxin plus furan or as the toxic equivalent emission, adjusting the mass of each congener according to its toxicity relative to 2,3,7,8 TCDD. The individual 2,3,7,8 substituted dioxin and furan congeners in tetra through o cta homologues are weighted by specific factors ranging from zero to 1.0 to determine the toxic equivalent of 2,3,7,8

TCDD.

The precision of Method 23 was assessed when data were reported in both forms. Even though

the same data were used for both assess ments, the ReMAP results suggest that the measurement precision varies according to how the data are reported. This implies that the random error associated with the sum of all the congener masses is different than random errors associated with

the sum of weighted masses.

The ReMAP assessment of Method 23 was performed using a limited database of multi

- train emissions data. For application of Method 23 for determination of total PCDD/PCDF mass, the ReMAP analysis found that RSD varied between about 6.3% a nd 20% for stack concentrations in the range of 2 to 27 ng/dscm. The following table presents the anticipated upper and lower

bounds for 99 out of l00 Method 23 measurements as a function of the true stack concentration.Table ES

- 1. Anticipated Range of M easurement Results Due to Random Error in Application of

Method 23 for Total PCDD/PCDF Determination.

As noted, indicated measurement precision is different when Method 23 is used to determine concentration on a toxic equivalence basis. Specifically, the regression analysis found no statistically significant variation of standard deviation with concentration. Pooled analysis indicates that the best estimate of standard deviation is 0.027 ng ITEQ/dscm when the emission concentration is in the range of 0.02 to 9.9 ng ITEQ/dscm. This further indicates that 99 out of l00 future single measurements should fall with ±0.069 ng ITEQ/dscm of the true concentration and 99 out of l00 triplicate measurements should fall within ±0.04 ng ITEQ/dscm of the true

concen tration.

The absolute value of anticipated range for future Method 23 measurements (as ITEQ) are quite small in absolute terms but they are on the same order as regulatory emission limits being considered in some regions. As indicated above, the best est imate of standard deviation is 0.027 ng ITEQ/dscm. However, at 95% confidence, the standard deviation may be as large as 0.037 ng ITEQ/dscm and the potential range for 99 out of l00 future measurements might deviate from the true concentration by as much a s ±0.095 ng ITEQ/dscm. Relying upon a single measurement has the potential to create problematic findings. If emission limits were set at 0.095 ng ITEQ/dscm, to be assured of compliance at the 95% confidence level, measurement results could not exceed

zero . Similarly, measurement results must be above 0.19 ng ITEQ/dscm to establish, with 95%

confidence that the true stack concentration exceeded the emission limit.

Most regulations and permit limits establish compliance based on averaging results from trip licate measurements. The anticipated range for 99 out of l00 future triplicate measurements is reduced, relative to single measurements, by v3

. Thus, compliance with an emission limit of 0.095 ng ITEQ/dscm is assured (at the 95% confidence level) when the triplicate average is at or below 0.04 ng ITEQ/dscm. Similarly, at 95% confidence, exceedence of the 0.095 ng ITEQ/dscm limit is assured when the three run average is above 0.15 ng ITEQ/dscm.

Method 26 for Hydrochloric Acid

ReMAP analysis of available data for Method 26 for HCI indicated that RSD is typically in the range of 5% to 10%. RSD does increase when the method is applied to stacks with very lowconcentration. Table ES

- 2 summarized the anticipated upper and lower bounds for 99 out of 100

Method 26 measurements as a function of true stack HCI concentration.

Table ES

- 2. Anticipated Range of HCI Measurement Res ults Due to Random Error in Application of Method 26.

Metho ds 29, l0 la and l 0 lb for Total Mercury

Several measurement methods hav e been devel

oped for me asurement of total em ission concentration and for mercury speciation. The ReMAP analysis took all available multi

- train mercury data collected using Methods 29, 101a and 101b but only used the data for total mercury

concentration. T he data analysis indicates that over the conc entration range of 50 to 783 μ g/dscm, the measurement method RSD varied from 9.6 to 12.4%. As concentration drops from 50 to 5

μ g/dscm, the RSD is expected to rise from 12.4% to 15.4%. Table ES

- 3 summarizes the anticipated upper an

d lower bounds for 99 out of l00 mercury measurements using Methods 29 and 101 as a function of true stack total mercury concentration.

Table ES

- 3. Anticipated Range of Total Hg Measurement Results Due to Random

Error in Applicati on of Methods 29, 101a and 101b.

Method 29 for Multi

-

Metals

Method 29 is also used for measurement of several other metal emissions. Precision analysis was completed for six other metals including antimo ny, arsenic, beryllium, cadmium, chromium, andlead. With the exception of cadmium, the analysis indicates that these metals behave similarly

with respect to measurement method precision. A composite analysis was performed for the five similarly behaving metals and the results

, 0 indicate that use of Method 29 provides an RSD that

varies between 13 and 18% when the individual metal concentrations are between about 20 and

100 μ g/dscm. Table ES

- 4 summarizes the anticipated upper and lower bounds for 99 out of 100 Sb, As, Be, Cr, and Pb measurements using Methods 29 as a function of true stack total metal

concentration

.

As regards cadmium measurements using Method 29, the analysis indicates that standard

deviation is a weaker function of concentration, at least at higher concentration ranges. The best

estimate of RSD is 9.1 % when cadmium concentration is 80 μ g/dscm and 18.7 % when the concentration drops to 20 μ g/dscm. However, at 5 μ g/dscm, predicted RSD is 38.6% and at 1.4

μ g/dsc m RSD is predicted to exceed 75%.

ES

- 6



### Table ES

- 4. Anticipated Range of Sb, As, Be, Cr, and Pb

Measurement Results Due to Random Error in Application of Methods 29 (Composite Analysis).