### STATISTICAL APPROACHES USED TO ASSESS THE VARIABILITY OF DATA

AUGUST 6, 2010

### In assessing the variability of the best perfor mers, we use a statistical formula designed to

estimate a MACT floor level that is equivalent to the average of the best performing sources

based on future compliance tests (or calculated in puts in the case of mercury). Specifically,

the MACT floor limit is an upper prediction limit (UPL) calculated with the Student’s t-test

using the TINV function in Microsoft Excel®. Th e Student’s t-test has also been used in

other EPA rulemakings (e.g., NSPS for Hospita l/Medical/Infectious Waste Incinerators,

NESHAP for Industrial, Commercial, and Institutional Bo ilers and Process Heaters) in

accounting for variability. A prediction interv al for a future observation (or average of

several observations) is an interv al that will, with a specified degree of confidence, contain

the next (or some other pre-specified) randomly selected observation (o r average of several

observations) from a population. In other words, the prediction interval estimates what the

upper bound of future values (or averages) will be, based upon present or past background

samples taken. The critical element is thus the representativeness of that data set. The UPL

consequently represents the value which we can expect the mean of future observations to

fall below within a specified le vel of confidence, based upon th e results of an independent

sample from the same population. In other words, if we were to randomly select a future test

condition from any of these sources (i.e., averag e of 3 runs or 30-day average) we can be 99

percent confident that the reported level will fall at or below the UPL value.

An upper tolerance limit is ordinarily utilized for large data sets and is intended to assure that

observed values are lower than a single highest observation. This methodology is intended to

produce values that do not underestimate variability but for this reason tends to produce

conservative estimates when applied to data se ts containing multiple observations, which is

the case for the MACT pools for each HAP in this rulemaking. Statistical tolerance limits are

limits within which we expect a stated proportion of the population to lie, with a stated level

of confidence. For example, an upper tolerance limit (UTL) can be defined such that we can

expect 99% of the population to fall below 99% of the time in repeated sampling.

The following table defines some of the basic te rms used by the various statistical procedures

for establishing MACT floors.

Symbol Term Definition/comments

μ Population mean The population mean refers to the "expected" value (i.e., first

moment) of the underlying population distribution. For example,

the population mean of a Normal(0,1) distribution is 0. It cannot

be calculated from the data and is typically unknown.

σ 2 Population variance The population variance refers to the second central moment of

the underlying population distribution. For example, the

population variance of a Normal(0,1) distribution is 1. It cannot

be calculated from the data and is typically unknown.

σ Population standard

deviation The square root of the population variance.

z p , t p Population percentile The p- th population percentile refers to the value below which

p % of the population falls. For example, the 95 th population

percentile of a Normal(0,1) distribution is z .95 = 1.645. It cannot

be calculated from the data and is typically unknown.Symbol Term Definition/comments

n / N Number of observations

or values in the sample The number of values or emissions data points for the specified

type of source; in the UCL and UTL methods, represents the

number of run-by-run data points; in the UPL method, n i represents the number of test runs for source i . In the UPL

method, N represents the number of sources in the MACT floor

analysis.



### x Sample mean The arithmetic mean of the sample values (observations or

emissions data points) for the specified type of source;

determined as:

Where n is the number of data points in the sample, and x i is the

value of data point i . When data from more than one source is

available, the arithmetic mean for the j-th source, is calculated as

1 

### 1 j n

j ij

i j 

### x x

n

= 

### = 

# ∑

Where

j 

### n represents the number of test runs for source j.

The sample mean is an unbiased estimator of the population

mean (i.e., the expected value of the sample mean equals the

population mean), and as n gets large, the sample mean will

approximate the population mean more closely.

x i The value of observation

or data point number i Used to represent individual values or observations (emissions

test run values); for example, x 3 is the third value in a sequence

of n x - values.

df Degrees of freedom Used in conjunction with t statistic; in the UCL and UTL

methods, df = n-1; in the UPL method, is the sum of n i -1 for

each of the N sources included in the analysis; see Section 2.4.1

for additional information.

t(p,df) p-th (population)

percentile of the tdistribution with df

degrees of freedom Used to test hypotheses about the population mean and construct

confidence intervals for the population mean, and prediction

intervals for future observations. In Excel, this can be obtained

as "PINV(2*(1-p/100), df)".

m Number of future values In the UPL method, represents the number of future test values

that will be used to determine compliance with the emission

limits; typically, the number of compliance test runs required by

the rule.

p Proportion of population

contained in an interval Expressed as a percentage and used in conjunction with the UTL

to represent the proportion of the population that is below the

upper tolerance limit (see Section 2.4.3).

Confidence level 

### Statistical measure of the numb er of times out of 100 that

test results (e.g., the populati on mean) can be expected to

be within a specified range. When used in the t statistic,

represents the level of confidence that the population mean will

be contained within the confidence limits (see Section 2.4.5).

s Sample standard

deviation The standard deviation of the sample values; determined as:



## ( ) 2

1 

### 1

1 n

i

i 

### s x x

n

= 

### = −

− 

# ∑

Where n is number of values in the sample.Symbol Term Definition/comments

s 2 Sample variance The square of the sample standard deviation.

The sample variance is an unbiased estimator of the population

variance (i.e., the expected value of the sample variance equals

the population variance), and as n gets large, the sample variance

will approximate the population variance more closely.

UPL Upper prediction limit Represents the value which we can expect a single future

observation, or the mean of a specified number of future

observations (e.g., 3-run average) to fall below for the specified

level of confidence, based upon the results of an independent

sample from the same population. (See Section 2.4.1 for

additional information and equation.)

UCL Upper confidence limit Represents the value which we can expect the mean of the

population to fall below in repeated sampling for the specified

level of confidence. (See S ection 2.4.2 for additional

information.)

UTL Upper tolerance limit Represents the value which we can expect a specified percentage

of the population to fall below in repeated sampling for the

specified level of confidence (e.g., 99% of the population falls

below the UTL 99% of the time). (See Section 2.4.3 for

additional information.)

UL Upper limit 

### The upper limit is roughly equivalent to the sample

percentile. The formulation assumes that the data (1)

represent the population rather than a random sample from

that population, and (2) are normally distributed. Note that



### the UL does not appear to have a robust statistical

foundation. (See Section 2.4.4 for additional information and

equation.)

x th

percentile Sample percentile The value below which (roughly) x% of the sample

measurements lie; for example, the 99 th sample percentile of a

given data set means that (roughly) 99% of the data lie below the

specified value. Different methods exist to estimate the sample

percentile, which may or may not be unbiased estimators of the

corresponding population percentile. (See Section 2.4.5 for

additional information and equation.)ApproachFormulaInterpretationUse/applicationUCL=Upper confidencelimit of a 99%confidenceinterval for themean( )20.99,sUCL x t dfn=+ ×1df n= −n= the number of topNsourcesdft= the 99thpercentile of the tdistribution withdfdegrees offreedom2s= variancex= mean•The upper confidence limit(UCL) represents the valuewhich we can expect thepopulation mean μ to fallbelow 99% of the time inrepeated sampling.•In other words, if we were torepeat the sample 100 times,then we would expect μ to fallbelow 99 of the 100 UCLscalculated from the 100different samples.•This formulation of the UCL should only be appliedto (reasonably) normally distributed data. If thedistribution of the data is far from normallydistributed, then the data should be appropriatelytransformed, or a different formulation of the UCL(e.g., nonparametric) should be used instead.99% UTL with99% confidence= 99 percentupper tolerancelimit with 99%confidence( )( )saabnptnptxUTL×−−−+−−+=21,11,1wherex=mean( )12)1,1(12−−γ−−=nnta( )nntnptb22)1,1(1,1−γ−−−−=n=number of observations

γ=99% (confidence level)p= proportion of population contained in interval (99%)s=standard deviation•We can concludewith 99%confidence that atleast 99% of thepopulation liesbelow the UpperTolerance Limit(UTL)•Statistical tolerancelimits are limitswithin which weexpect a statedproportion of thepopulation to lie.•Compare the 99thsample percentile(usually themaximum value)with the UTL•Only apply tonormally distributeddata (see UCLcomment)Upper predictionlimit (UPL) with99 confidenceUPL =Total Number of sources =n =Total # test runs =Mean =1 11inNiji jxn= ==∑∑xPooled Variance =( )221 111inNiji js xxn= == −−∑∑dftm= number of future runs= the 99thpercentile of the tdistribution withdfdegrees offreedom•The upper prediction limit (UPL) represents the value which we canexpect a future observation (or mean of several future observations)to fall below 99% of the time in repeated sampling.•In other words, if we were torepeat the present sample (from whichthe UPL is derived) 100 times then we would expect a futureobservation (or mean of several future observations) generated fromthe same population distribution to fall below 99 of the 100 derivedUPLs. Note that this takes intoaccount the fact that since futureobservations (or means of several of them) are random variables,the values may be different for each of the 100 repetitions.•The UPL allows us to use information from present (or past) data tomake statistical inferences about future observations.•To infer to futurevalues of meanof new data•Contrast thefuture measures(with the UPL)•Only apply tonormallydistributed data(see UCLcomment)11Niidf n=⎛ ⎞= −⎜ ⎟⎝ ⎠∑1Niin=∑N2,1 1df pXt sn m⎛ ⎞+ +⎜ ⎟⎝ ⎠99thsamplepercentileThe p-th sample percentile of Nordered values is obtained by firstcalculating the rank11100Nnp+, rounding tothe nearest integer, and thentaking the value that correspondsto that rank (n)−= ×•The 99thsample percentile is an estimator of the 99thpopulationpercentile, below which 99% of the population values lie. If thesample size is small, then sample percentiles may be biasedestimators of corresponding population percentiles, particularlywhen p is close to 0 or 100. For example, if N=3 then the 99thsample percentile will be the largest value in the sample, but theexpected value of this observationwill be substantially smaller thanthe 99thpopulation percentile.•Creates a cut-off belowwhich the 99thsamplepercentile (and all othersmaller observations inthe sample) will fall.