STATISTICAL APPROACHES USED TO ASSESS THE VARIABILITY OF DATA 

AUGUST 6, 2010 
In assessing the variability of the best performers, we use a statistical formula designed to 

estimate a MACT floor level that is equivalent to the average of the best performing sources 

based on future compliance tests (or calculated inputs in the case of mercury). Specifically, 

the MACT floor limit is an upper prediction limit (UPL) calculated with the Student’s t-test 

using the TINV function in Microsoft Excel®. The Student’s t-test has also been used in 

other EPA rulemakings (e.g., NSPS for Hospital/Medical/Infectious Waste Incinerators, 

NESHAP for Industrial, Commercial, and Institutional Boilers and Process Heaters) in 

accounting for variability. A prediction interval for a future observation (or average of 

several observations) is an interval that will, with a specified degree of confidence, contain 

the next (or some other pre-specified) randomly selected observation (or average of several 

observations) from a population. In other words, the prediction interval estimates what the 

upper bound of future values (or averages) will be, based upon present or past background 

samples taken. The critical element is thus the representativeness of that data set. The UPL 

consequently represents the value which we can expect the mean of future observations to 

fall below within a specified level of confidence, based upon the results of an independent 

sample from the same population. In other words, if we were to randomly select a future test 

condition from any of these sources (i.e., average of 3 runs or 30-day average) we can be 99 

percent confident that the reported level will fall at or below the UPL value. 

An upper tolerance limit is ordinarily utilized for large data sets and is intended to assure that 

observed values are lower than a single highest observation. This methodology is intended to 

produce values that do not underestimate variability but for this reason tends to produce 

conservative estimates when applied to data sets containing multiple observations, which is 

the case for the MACT pools for each HAP in this rulemaking. Statistical tolerance limits are 

limits within which we expect a stated proportion of the population to lie, with a stated level 

of confidence. For example, an upper tolerance limit (UTL) can be defined such that we can 

expect 99% of the population to fall below 99% of the time in repeated sampling. 

The following table defines some of the basic terms used by the various statistical procedures 

for establishing MACT floors. 

Symbol Term Definition/comments

μ Population mean The population mean refers to the "expected" value (i.e., first 

moment) of the underlying population distribution. For example, 

the population mean of a Normal(0,1) distribution is 0. It cannot 

be calculated from the data and is typically unknown. 

σ

2 Population variance The population variance refers to the second central moment of 

the underlying population distribution. For example, the 

population variance of a Normal(0,1) distribution is 1. It cannot 

be calculated from the data and is typically unknown. 

σ Population standard 

deviation

The square root of the population variance. 

zp, tp Population percentile The p-th population percentile refers to the value below which 

p% of the population falls. For example, the 95th population 

percentile of a Normal(0,1) distribution is z.95 = 1.645. It cannot 

be calculated from the data and is typically unknown. 
Symbol Term Definition/comments

n / N Number of observations 

or values in the sample

The number of values or emissions data points for the specified 

type of source; in the UCL and UTL methods, represents the 

number of run-by-run data points; in the UPL method, ni

represents the number of test runs for source i. In the UPL 

method, N represents the number of sources in the MACT floor 

analysis.

x Sample mean The arithmetic mean of the sample values (observations or 

emissions data points) for the specified type of source; 

determined as:

Where n is the number of data points in the sample, and xi is the 

value of data point i. When data from more than one source is 

available, the arithmetic mean for the j-th source, is calculated as

1

1 j n

j ij

i j

x x

n =

= ∑

Where j n represents the number of test runs for source j. 

The sample mean is an unbiased estimator of the population 

mean (i.e., the expected value of the sample mean equals the 

population mean), and as n gets large, the sample mean will 

approximate the population mean more closely.

xi The value of observation 

or data point number i

Used to represent individual values or observations (emissions 

test run values); for example, x3 is the third value in a sequence 

of n x- values.

df Degrees of freedom Used in conjunction with t statistic; in the UCL and UTL 

methods, df = n-1; in the UPL method, is the sum of ni-1 for 

each of the N sources included in the analysis; see Section 2.4.1 

for additional information.

t(p,df) p-th (population) 

percentile of the tdistribution with df

degrees of freedom

Used to test hypotheses about the population mean and construct 

confidence intervals for the population mean, and prediction 

intervals for future observations. In Excel, this can be obtained 

as "PINV(2*(1-p/100), df)".

m Number of future values In the UPL method, represents the number of future test values 

that will be used to determine compliance with the emission 

limits; typically, the number of compliance test runs required by 

the rule.

p Proportion of population 

contained in an interval

Expressed as a percentage and used in conjunction with the UTL 

to represent the proportion of the population that is below the 

upper tolerance limit (see Section 2.4.3).

Confidence level Statistical measure of the number of times out of 100 that 

test results (e.g., the population mean) can be expected to 

be within a specified range. When used in the t statistic, 

represents the level of confidence that the population mean will 

be contained within the confidence limits (see Section 2.4.5).

s Sample standard 

deviation

The standard deviation of the sample values; determined as: 

( )

2

1

1

1

n

i

i

s x x

n =

= − − ∑

Where n is number of values in the sample.
Symbol Term Definition/comments

s

2 Sample variance The square of the sample standard deviation. 

The sample variance is an unbiased estimator of the population 

variance (i.e., the expected value of the sample variance equals 

the population variance), and as n gets large, the sample variance 

will approximate the population variance more closely.

UPL Upper prediction limit Represents the value which we can expect a single future 

observation, or the mean of a specified number of future 

observations (e.g., 3-run average) to fall below for the specified 

level of confidence, based upon the results of an independent 

sample from the same population. (See Section 2.4.1 for 

additional information and equation.)

UCL Upper confidence limit Represents the value which we can expect the mean of the 

population to fall below in repeated sampling for the specified 

level of confidence. (See Section 2.4.2 for additional 

information.)

UTL Upper tolerance limit Represents the value which we can expect a specified percentage 

of the population to fall below in repeated sampling for the 

specified level of confidence (e.g., 99% of the population falls 

below the UTL 99% of the time). (See Section 2.4.3 for 

additional information.)

UL Upper limit The upper limit is roughly equivalent to the sample 

percentile. The formulation assumes that the data (1) 

represent the population rather than a random sample from 

that population, and (2) are normally distributed. Note that 

the UL does not appear to have a robust statistical 

foundation. (See Section 2.4.4 for additional information and 

equation.)

x

th

percentile

Sample percentile The value below which (roughly) x% of the sample 

measurements lie; for example, the 99th sample percentile of a 

given data set means that (roughly) 99% of the data lie below the 

specified value. Different methods exist to estimate the sample 

percentile, which may or may not be unbiased estimators of the 

corresponding population percentile. (See Section 2.4.5 for 

additional information and equation.)
Approach Formula Interpretation Use/application 

UCL= Upper confidence limit of a 99% confidence interval for the mean 

( ) 2 0.99, s UCL x t df

n

=+ ×

df n = −1

n= the number of top 

N sources 

df t = the 99th percentile of the tdistribution with df degrees of 

freedom 

2

s = variance 

x = mean 

• The upper confidence limit 

(UCL) represents the value which we can expect the population mean μ to fall below 99% of the time in repeated sampling. 

• In other words, if we were to 

repeat the sample 100 times, then we would expect μ to fall below 99 of the 100 UCLs calculated from the 100 different samples. 

• This formulation of the 

to (reasonably) normally distribution of the data idistributed, then the data transformed, or a differe(e.g., nonparametric) sh

99% UTL with 99% confidence = 99 percent upper tolerance limit with 99% confidence 

( )( ) s a abnptnpt xUTL

×

−−−+−−

+=

2 1,11,1

where 

x =mean 

( ) 12

)1,1( 1

2

−

−γ−

−=

n

nt

a

( )

n

nt nptb

2 2 )1,1( 1,1 −γ− −−−=

n=number of observations 

γ =99% (confidence level) 

p= proportion of population contained in interval (99%) s=standard deviation 

• We can conclude 

with 99% confidence that at least 99% of the population lies below the Upper Tolerance Limit (UTL) 
Upper prediction 

limit (UPL) with 99 confidence UPL = 

Total Number of sources = 

n =Total # test runs = Mean = 

1 1

1 i N n

ij

i j

x

n = =

x = ∑∑

Pooled Variance = 

( )2 2

1 1

1

1

i N n

ij

i j

s xx

n = =

= −

−

∑∑

df t

m = number of future runs 

= the 99th percentile of the tdistribution with df degrees of 

freedom 

• The upper prediction limit (UPL) represents the value which w

expect a future observation (or mean of several future observatto fall below 99% of the time in repeated sampling. 

• In other words, if we were to repeat the present sample (from w

the UPL is derived) 100 times then we would expect a future observation (or mean of several future observations) generated the same population distribution to fall below 99 of the 100 derUPLs. Note that this takes into account the fact that since futurobservations (or means of several of them) are random variablethe values may be different for each of the 100 repetitions. 

• The UPL allows us to use information from present (or past) da

make statistical inferences about future observations. 

1

1

N

i

i

df n

=

⎛ ⎞

= − ⎜ ⎟ ⎝ ⎠ ∑

1

N

i

i

n

=

∑

N

2

,

1 1 Xt s df p n m ⎛ ⎞ + + ⎜ ⎟ ⎝ ⎠
99th sample 

percentile 

The p-th sample percentile of N ordered values is obtained by first calculating the rank 

1 1

100

N

n

p

+ , rounding to 

the nearest integer, and then taking the value that corresponds to that rank (n) 

−

= ×

• The 99th sample percentile is an estimator of the 99th population 

percentile, below which 99% of the population values lie. If the sample size is small, then sample percentiles may be biased estimators of corresponding population percentiles, particularly when p is close to 0 or 100. For example, if N=3 then the 99th

sample percentile will be the largest value in the sample, but the expected value of this observation will be substantially smaller than the 99th population percentile. 

•