Cente

AN ASME REPORT CRTD Vol. 60

Final Report

________________________________________________________________

Reference Method Accuracy and 

Precision (ReMAP): Phase 1

Precision of Manual Stack Emission 

Measurements

_____________________________________________________________________________

Prepared by:

W. Steven Lanier

GE Energy and Environmental Research Corporation

Charles D. Hendrix

Statistical Consultant

under the auspices of:

American Society of Mechanical Engineers

Research Committee on Industrial and Municipal Waste

February 2001
Reference Method Accuracy and Precision (ReMAP Phase I) An Assessment of the 

Precision of EP A Manual Stack Emission Measurements

Executive Summary

This report documents results from the first phase of a study co-sponsored by the American 

Society of Mechanical Engineers (ASME) to assess the accuracy and precision of manual test 

methods adopted by the US Environmental Protection Agency (EP A) for determining the stack 

concentration of selected air pollutants. The program is entitled Reference Method Accuracy and 

Precision and is referred to by the acronym ReMAP. The Phase I effort addresses the precision of 

the selected measurement methods. The formal Purpose Statement for the program is: 

"To determine the precision of pollutant emission measurements based on analysis of 

available simultaneous -sample test data which were generated using EPA Manual

Reference Test Methods 5 and 5i (Particulate Matter), 23 (Dioxin and Furan), 26 (HCl), 

29 (multi-metals), 101a and 101b (mercury) and 108 (arsenic) at a number of stationary 

air sources. " 

As used in the ReMAP program, precision is defined as random error that inadvertently enters the 

measurement process. This error may enter at any stage of the measurement process including 

sample collection, sample recovery, or sample analysis. The impact of such errors is that 

measurement results deviate from the true stack concentration. Because these errors occur

randomly, data from repeated application of any manual test method (to a hypothetical stack with 

a chosen average concentration of the selected pollutant) should result in a bell shaped frequency 

distribution. This distribution is centered about the chosen stack concentration but data are 

expected indicating measured concentrations both above and below the chosen stack value. 

Precision of a measurement method is indicated by the horizontal spread of the bell curve. One 

common way to characterize the bell curve shape is to determine the estimated standard deviation 

(s) of the distribution. Alternately, the bell distribution also describes the range of measurement 

results anticipated from repeated application of the measurement method at the given

concentration. Based upon the best estimate of the standard deviation, it is a simple matter to 

calculate other indicators of data quality that have direct practical significance. The ReMAP 

program selected two additional, directly derived parameters to characterize precision of manual 

methods: 

1. The anticipated range for 99 out of 100 future single measurements. If the measurement 

method is repeatedly applied to a stack with a given concentration, this precision metric defines 

the upper and lower concentration bounds for 99% of individual measurements. 

2. The anticipated range for 99 out of 100 future triplicate measurements. Since most

environmental regulations define the reportable stack concentration as the average of three

repeated test runs, this metric defines the anticipated range of results in triplicate (3 single time 

series) measurements due to random error in the measurement process. 

For each of these precision metrics it is important to note the inherent assumption that the 

measurement method is being applied to a hypothetical stack with an average pollutant 

concentration that does not vary with time. In real-world applications, single and triplicate

measurement results will indicate variation from both random errors and systematic errors (bias) 

in the measurement process as well as temporal variation in facility operation. 
As noted, the primary objective of the ReMAP program is to characterize the bell shaped curve 

for each measurement method by first estimating the standard deviation s. The other two

precision metrics are directly calculated from s. Specifically 99 out of 100 single measurements 

will fall within ± 2.54 s of the true concentration. The anticipated range for the average of 

repeated measurements comes progressively closer to the true concentration. More precisely, the 

anticipated range varies inversely with the square root of the number of measurements. Thus, if s 

is the standard deviation of anticipated single measurements, the anticipated range for 99 out of 

100 future triplicate measurements will fall within ± 2.54 s /v 3 of the true concentration. 

A central tenet of the ReMAP program is to determine how the precision of manual measurement 

methods varies with the concentration of the pollutant being measured. A constant measurement 

standard deviation might occur if the primary source of random error is an analytical process. A 

simple example might be the repeatability with which a technician can measure the weight gain 

on a particulate filter. The capabilities of the weighing scale do not significantly vary with the 

particulate weight gain. However, the magnitude of random errors associated with extraction and 

recovery of the sample from the stack might be expected to vary in proportion to stack

concentration. 

The technical effort of the ReMAP program is to estimate the standard deviation of selected EPA 

Measurement Reference Methods as a function of average stack concentration. ReMAP is not 

intended to validate any of the Reference Methods addressed herein nor is it to be used as a 

substitute for Method 301.

The concentration of pollutants released from industrial facilities must be assumed to vary with 

time. Accordingly, results from typical, single -train stack tests are of limited value for

determination of the precision of a measurement method. Instead, estimation of method precision 

must be based on data from special tests where multiple sampling trains are used to

simultaneously determine the stack pollutant concentration. Such multi-train tests minimize the 

impact of temporal and spatial (where probes are co-located) variations on the data and the results 

can be used to estimate the standard deviation of the particular measurement at the specific stack 

concentration. 

The ReMAP program performed a careful assessment of the statistical analysis procedures 

required to estimate the precision of Manual Reference Methods using multi-train sampling data 

(see Appendix). To assure the quality of data used in the statistical analysis, an extensive effort 

was expended in gathering data from the original sources and carefully evaluating them to assure 

that consistent data reduction procedures were used. 

Conceptually, the ReMAP statistical analysis procedure is straightforward. First, data from a 

multi-train test run are averaged to provide an estimate of the average concentration for the run 

(Ci). The standard deviation for the test run (Si) is also calculated. Clearly, a calculated standard 

deviation from a single test using a dual sampling probe provides a relatively poor estimate of the 

true standard deviation of the method (s) at the true concentration (μ). However, after accounting 

for various biases, a significant array of data from multi-train tests should provide a reasonable 

basis for estimating the true standard deviation as a function of concentration. The ReMAP 

procedure is to assume that the standard deviation varies with concentration according to a power 

function relationship and then to fit the data to that equation using regression analysis. 

Results from the regression analysis represent the best estimate available on the standard

deviation of the measurement method at any given concentration. However, the ReMAP analysis 

procedure also provides for calculation of confidence intervals on the regression. These
confidence intervals define the upper and lower bounds for the regression line at the 95% 

confidence level. Based on the regression line and the confidence intervals, the various precision 

metrics can be determined for each Method as a function of concentration. Results from the 

analyses are summarized below. 

EP A Method 5 and 5i for Particulate Matter - Front Half Only 

Method 5 was one of the first EP A Reference Methods developed for stack sampling. There is a 

relatively large body of multi-train data indicating that the relative standard deviation (RSD) has 

minimal variation with concentration. Over a broad concentration range, RSD is predicted to 

remain between 5 and 11 %. There are certain data comparability concerns associated with the 

Method 5 precision analysis. However, assuming that the Method is applied to a stack with a 

particulate matter concentration less than 150 mg/dscm, the best available estimate is that RSD 

will be below 10%. Moreover, the influence of random error in the measurement process should 

result in the average of triplicate measurements deviating from the true average concentration by 

no more than about ±14.7% (±2.54*10%/v3).

Method 5i was specifically developed for application to stacks with particulate concentrations 

below 50 mg/dscm. The Method itself requires dual-train sampling and provides an upper limit 

on the allowable deviation between the simultaneous measurements. Thus the available data had 

been pre screened to eliminate test results with large standard deviation (i.e., > about 14% RSD). 

This prescreening, coupled with the relatively small concentration range for the data, resulted in 

the ReMAP analysis finding no statistically significant variation of standard deviation with 

concentration. 

Based on a pooled analysis, the characteristic standard deviation for Method 5i was found to be 

1.43 mg/dscm. Based on this best estimate of standard deviation. the ReMAP analysis indicates 

that 99 out of l00 Method Si single measurements should deviate from the true concentration by 

no more than ± 3.68 mg/dscm. For triplicate measurements 99 out of l00 Method 5i data results 

should deviate from the true concentration by no more than ± 2.12 mg/dscm. 

Method 23 for Dioxin and Furan 

Data collected with Method 23 are used to report dioxin and furan emissions as either the total 

mass of tetra through octa chlorinated dioxin plus furan or as the toxic equivalent emission, 

adjusting the mass of each congener according to its toxicity relative to 2,3,7,8 TCDD. The 

individual 2,3,7,8 substituted dioxin and furan congeners in tetra through octa homologues are 

weighted by specific factors ranging from zero to 1.0 to determine the toxic equivalent of 2,3,7,8 

TCDD. 

The precision of Method 23 was assessed when data were reported in both forms. Even though 

the same data were used for both assessments, the ReMAP results suggest that the measurement 

precision varies according to how the data are reported. This implies that the random error 

associated with the sum of all the congener masses is different than random errors associated with 

the sum of weighted masses. 

The ReMAP assessment of Method 23 was performed using a limited database of multi-train 

emissions data. For application of Method 23 for determination of total PCDD/PCDF mass, the 

ReMAP analysis found that RSD varied between about 6.3% and 20% for stack concentrations in 

the range of 2 to 27 ng/dscm. The following table presents the anticipated upper and lower 

bounds for 99 out of l00 Method 23 measurements as a function of the true stack concentration. 
Table ES-1. Anticipated Range of Measurement Results Due to Random Error in Application of 

Method 23 for Total PCDD/PCDF Determination. 

As noted, indicated measurement precision is different when Method 23 is used to determine 

concentration on a toxic equivalence basis. Specifically, the regression analysis found no

statistically significant variation of standard deviation with concentration. Pooled analysis

indicates that the best estimate of standard deviation is 0.027 ng ITEQ/dscm when the emission 

concentration is in the range of 0.02 to 9.9 ng ITEQ/dscm. This further indicates that 99 out of 

l00 future single measurements should fall with ±0.069 ng ITEQ/dscm of the true concentration 

and 99 out of l00 triplicate measurements should fall within ±0.04 ng ITEQ/dscm of the true 

concentration. 

The absolute value of anticipated range for future Method 23 measurements (as ITEQ) are quite 

small in absolute terms but they are on the same order as regulatory emission limits being 

considered in some regions. As indicated above, the best estimate of standard deviation is 0.027 

ng ITEQ/dscm. However, at 95% confidence, the standard deviation may be as large as 0.037 ng 

ITEQ/dscm and the potential range for 99 out of l00 future measurements might deviate from the 

true concentration by as much as ±0.095 ng ITEQ/dscm. Relying upon a single measurement has 

the potential to create problematic findings. If emission limits were set at 0.095 ng ITEQ/dscm, to 

be assured of compliance at the 95% confidence level, measurement results could not exceed 

zero. Similarly, measurement results must be above 0.19 ng ITEQ/dscm to establish, with 95% 

confidence that the true stack concentration exceeded the emission limit. 

Most regulations and permit limits establish compliance based on averaging results from triplicate 

measurements. The anticipated range for 99 out of l00 future triplicate measurements is reduced, 

relative to single measurements, by v3. Thus, compliance with an emission limit of 0.095 ng 

ITEQ/dscm is assured (at the 95% confidence level) when the triplicate average is at or below 

0.04 ng ITEQ/dscm. Similarly, at 95% confidence, exceedence of the 0.095 ng ITEQ/dscm limit 

is assured when the three run average is above 0.15 ng ITEQ/dscm. 

Method 26 for Hydrochloric Acid 

ReMAP analysis of available data for Method 26 for HCI indicated that RSD is typically in the 

range of 5% to 10%. RSD does increase when the method is applied to stacks with very low 
concentration. Table ES-2 summarized the anticipated upper and lower bounds for 99 out of 100 

Method 26 measurements as a function of true stack HCI concentration. 

Table ES-2. Anticipated Range of HCI Measurement Results Due to Random Error in

Application of Method 26. 

Methods 29, l0la and l0lb for Total Mercury 

Several measurement methods have been developed for measurement of total emission 

concentration and for mercury speciation. The ReMAP analysis took all available multi-train 

mercury data collected using Methods 29, 101a and 101b but only used the data for total mercury 

concentration. The data analysis indicates that over the concentration range of 50 to 783 μg/dscm, 

the measurement method RSD varied from 9.6 to 12.4%. As concentration drops from 50 to 5 

μg/dscm, the RSD is expected to rise from 12.4% to 15.4%. Table ES-3 summarizes the 

anticipated upper and lower bounds for 99 out of l00 mercury measurements using Methods 29 

and 101 as a function of true stack total mercury concentration. 

Table ES-3. Anticipated Range of Total Hg Measurement Results Due to Random 

Error in Application of Methods 29, 101a and 101b.

Method 29 for Multi-Metals 

Method 29 is also used for measurement of several other metal emissions. Precision analysis was 

completed for six other metals including antimony, arsenic, beryllium, cadmium, chromium, and 
lead. With the exception of cadmium, the analysis indicates that these metals behave similarly 

with respect to measurement method precision. A composite analysis was performed for the five 

similarly behaving metals and the results, 0 indicate that use of Method 29 provides an RSD that 

varies between 13 and 18% when the individual metal concentrations are between about 20 and 

100 μg/dscm. Table ES-4 summarizes the anticipated upper and lower bounds for 99 out of 100 

Sb, As, Be, Cr, and Pb measurements using Methods 29 as a function of true stack total metal 

concentration. 

As regards cadmium measurements using Method 29, the analysis indicates that standard

deviation is a weaker function of concentration, at least at higher concentration ranges. The best 

estimate of RSD is 9.1 % when cadmium concentration is 80 μg/dscm and 18.7 % when the 

concentration drops to 20 μg/dscm. However, at 5 μg/dscm, predicted RSD is 38.6% and at 1.4 

μg/dscm RSD is predicted to exceed 75%. 

ES-6 

Table ES-4. Anticipated Range of Sb, As, Be, Cr, and Pb 

Measurement Results Due to Random Error in Application of 

Methods 29 (Composite Analysis).