Document ID: EPA-HQ-OW-2003-0002-0089
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2003-03-12T05:00Z

Twentieth
Annual
EPA
Conference
May
1997
8
­
27
MR.
TELLIARD:
Our
next
speaker
is
Henry
Kahn.
Henry
is
Chief
of
the
Statistical
Branch
Group
with
the
Office
of
Science
and
Technology.
Henry
is
my
counterpart
in
statistics.
Henry
is
going
to
talk
to
you
today
about
alternative
estimates
of
detection
and
quantitation.

ALTERNATIVE
ESTIMATES
OF
DETECTION
Henry
D.
Kahn
and
Charles
E.
White
U.
S.
Environmental
Protection
Agency
Kathleen
Stralka
and
Raphael
Kuznetsovski
Science
Applications
International
Corporation
ABSTRACT
This
paper
considers
alternative
procedures
for
estimation
of
detection
levels
based
on
some
modifications
of
procedures
contained
in
the
EPA
Method
Detection
Limit
(
MDL)
described
in
40
CFR,
Part
136,
Appendix
B.
Also
discussed
is
measurement
precision
and
its
central
role
in
determining
detection
levels.
Difficulties
associated
with
precision
estimates
based
on
a
calibration
relationship
are
illustrated.
The
MDL
procedure
uses
estimates
of
measurement
precision
determined
by
replicate
measurements
in
the
low
concentration
range.
The
modifications
of
the
MDL
procedure
considered
here
are
specified
in
the
procedure
as
optional
steps
and
result
in
enhanced
estimates
of
measurement
precision
used
in
determining
detection
levels.
These
optional
steps
are
rarely,
if
ever,
used
in
current
practice.
The
procedures
are
illustrated
using
data
generated
by
EPA
Method
1638
for
the
determination
of
trace
elements
by
inductively
coupled
plasma­
mass
spectrometry.
These
data
demonstrate
the
pattern
typical
of
much
chemical
analytical
data
where
the
error
structure
has
approximately
constant
variance
in
the
low
concentration
range
and
proportional
variance
(
i.
e.,
variance
increasing
with
concentration)
at
higher
concentrations.
The
Rocke­
Lorenzato
model
for
this
relationship
will
be
used
in
evaluating
the
results
of
implementing
the
MDL
optional
steps
which
demonstrate
that
the
procedures
yield
MDL
values
in
the
low
concentration
range
where
error
variance
is
effectively
constant.

___________________________________________________________________________________
Presented
at
the
20th
Annual
Conference
on
Analysis
of
Pollutants
in
the
Environment,
May
7­
8,
1997,
Norfolk,
Virginia
Twentieth
Annual
EPA
Conference
8
­
28
May
1997
ALTERNATIVE
ESTIMATES
OF
DETECTION
Henry
D.
Kahn
and
Charles
E.
White
U.
S.
Environmental
Protection
Agency
Kathleen
Stralka
and
Raphael
Kuznetsovski
Science
Applications
International
Corporation
INTRODUCTION
A
number
of
different
procedures
are
available
for
determining
detection
levels
for
chemical
analytical
measurements.
The
US
Environmental
Protection
Agency
(
EPA)
has
promulgated
a
procedure
for
determining
detection
limits,
referred
to
as
a
Method
Detection
Limit
(
MDL)
and
described
in
40
CFR,
Part
136,
Appendix
B,
that
has
become
widely
used.
The
key
to
determining
detection
level
values
is
the
precision
of
measurement
(
defined
as
the
standard
deviation
of
replicate
measurements
at
a
given
concentration)
in
the
low
concentration
range,
i.
e.,
near
and
at
zero.
One
of
the
advantages
of
the
EPA
procedure
is
that
it
provides
step
by
step
process
for
determining
precision.
In
this
paper,
we
will
address
the
following:
measurement
precision
as
an
important
characteristic
of
an
analytical
method;
statistical
models
of
precision;
the
relationship
between
precision
and
alternative
methods
for
determining
detection
levels;
illustration
of
the
estimation
of
detection
levels
with
some
inductively
coupled
plasma
mass
spectrometric
(
ICPMS)
metals
data
using
the
EPA
MDL
procedure
and
the
Rocke­
Lorenzato
model.

MEASUREMENT
PRECISION
Measurement
precision
is
defined
as
the
degree
of
agreement
among
replicate
analyses
of
a
sample.
This
degree
of
agreement
is
expressed
as
the
standard
deviation
of
the
replicates.
An
understanding
of
the
precision
associated
with
an
analytical
procedure
is
fundamental
to
the
evaluation
of
the
capabilities
of
the
procedure.
This
is
especially
true
with
regard
to
establishing
detection
levels.
The
manner
in
which
precision
varies
with
concentration
being
measured
is
sometimes
referred
to
as
"
error
structure"
of
the
test
method.
This
is
discussed
by
Caulcutt
and
Boddy
(
1983)
who
distinguish
between
a
constant
precision
method
where
the
standard
deviation
of
measurement
is
constant
throughout
the
range
of
measurement
and
an
decreasing
precision
method
where
the
standard
deviation
is
directly
proportional
to
the
concentration
being
measured
and
thus
increases
or
decreases
as
the
concentration
increases
or
decreases,
respectively.

Unfortunately,
the
description
of
error
structure
by
Caulcutt
and
Boddy
(
1983)
does
not
consider
what
has
become
recognized
as
a
common
type
of
error
structure,
i.
e.,
one
in
which
precision
is
effectively
constant
in
the
low
concentration
region
near
zero
and
then
becomes
proportional
to
concentration
(
i.
e.,
measurement
variability
increases)
as
concentration
increases.
Most
EPA
chemical
analysis
methods
are
expected
to
exhibit
such
a
combined
error
structure.
Therefore,
the
estimated
precision
in
the
range
near
zero
is
critical
to
establishing
detection
and
quantification
levels.
The
adoption
of
the
proportional
error
model
without
proper
consideration
of
the
behavior
near
zero
implies,
incorrectly,
that
error
decreases
to
zero
as
concentration
decreases.
Failure
to
recognize
the
combined
error
structure
can
lead
to
difficulties
as
the
following
example
from
Gibbons
(
1994),
and
repeated
in
Gibbons
(
1995),
illustrates.
Twentieth
Annual
EPA
Conference
May
1997
8
­
29
Figure
1:
Standard
Deviations
of
Replicates
Versus
Benzene
Spike
Concentrations,
Observed
Standard
Deviations
and
Estimated
Standard
Deviations
from
Weighted
Regression
by
Gibbons
(
1994,
1995).
Gibbons
used
a
weighted
regression
analysis
to
estimate
a
calibration
relationship
for
benzene.
He
then
used
the
residual
variance
from
the
regression
relationship
to
estimate
the
error
variance
over
the
calibration
range,
extrapolated
the
variance
to
the
low
concentration
range
[
more
than
two
orders
of
magnitude
below
the
range
of
the
observed
data]
and
took
the
square
root
to
estimate
the
standard
deviation.
The
results
of
this
are
shown
in
Figure
1
which
displays
the
observed
standard
deviations
and
the
estimated
standard
deviation
relationship
as
reported
by
Gibbons.
In
the
Figure,
circles
indicate
standard
deviations
modeled
(
i.
e.,
not
observed)
while
the
dots
indicate
standard
deviations
estimated
from
actual
replicate
sample
measurements
made
at
spike
concentrations.
The
Figure
is
rather
striking;
it
shows
clearly
that
the
observed
data
and
the
values
determined
by
the
model
do
not
match.
The
observed
data
suggest
a
concave
curve,
bending
toward
an
intersection
with
the
vertical
axis
at
a
positive
value
at
zero
concentration,
which
is
consistent
with
the
combined
error
structure
model.
The
modeled
data,
in
contrast,
have
a
convex
shape
with
the
extrapolated
values
curving
down
to
zero.
Indeed,
the
curvature
becomes
quite
extreme
in
the
low
concentration
region
where
the
values
were
extrapolated
outside
the
range
of
the
observed
data.

Since
the
MDL
is
determined
as
the
product
of
the
standard
deviation
and
a
constant
that
depends
only
on
the
number
of
replicates,
the
modeled
results
shown
in
Figure
1
were
used
as
the
basis
for
criticism
of
EPA's
MDL
procedure.
The
criticism
alleges
that
the
MDL
facilitates
the
development
of
arbitrarily
low
detection
levels.
Noting
the
decrease
in
the
standard
deviations
extrapolated
to
low
concentrations,
Gibbons
(
1994)
states
that
"
low
published
MDLs
can
be
achieved
only
by
spiking
at
lower
and
lower
concentrations,
regardless
of
their
relationship
to
the
true
MDL.".
The
same
data
and
estimated
relationship
were
used
to
support
similar
assertions
in
Gibbons
(
1995).
Inspection
of
Figure
1,
however,
demonstrates
that
the
data,
unlike
the
extrapolated
model,
do
not
support
these
assertions.
Twentieth
Annual
EPA
Conference
8
­
30
May
1997
We
emphasize
that
the
data
shown
in
Figure
1
were
obtained
directly
from
tables
in
the
two
sources
cited;
i.
e.,
the
reported
values
were
not
manipulated
or
adjusted
to
obtain
the
results
shown
in
the
plot.
An
appropriate
model
for
precision
of
measurement
might
have
avoided
the
difficulty
demonstrated
by
the
modeled
results
shown
in
the
plot.
This
is
discussed
in
the
next
section.

Statistical
Models
of
Precision
Statistical
models
of
precision
of
measurement
must
represent
a
range
of
observed
behavior.
Traditional,
uncomplicated
models
of
precision,
such
as
the
constant
precision
model
throughout
the
range
of
measurement
or
precision
as
a
simple
linear
relationship
with
concentration
are
rarely,
if
ever,
realized
empirically
over
a
wide
range
of
concentration.
This
is
illustrated
by
the
problems
cited
in
Gibbons
(
1994
and
1995)
A
new
approach
proposed
by
Rocke
and
Lorenzato
(
1995)
models
precision
of
measurement
as
comprised
of
two
error
components,
one
additive
and
one
multiplicative.
The
Rocke­
Lorenzato
(
R­
L)
model
has
a
number
of
appealing
features:
1)
it
allows
data
near
zero
to
determine
the
estimated
standard
deviation
near
zero
and
the
data
at
large
concentrations
determine
the
standard
deviation
at
large
concentrations;
2)
it
allows
for
data
at
large
concentrations
to
be
lognormal
which
is
often
observed;
and
3)
it
has
a
credible
physical
interpretation.
The
R­
L
model
is
y
=
x
exp0
+
,

where:
y
=
measured
concentration
x
=
true
concentration
0
=
multiplicative
error
~
N(
0,
F
0
2)
,
=
additive
error
~
N
(
0,
F,
2
).

The
standard
deviation
of
the
model
is
given
by
S(
x)
=
[
F,
2
+
x2exp(
F
0
2)
{
exp(
F
0
2)
­
1}]
½
.

The
standard
deviation
is
easier
to
work
with
if
we
define
A0
=
F,
2
and
A1
=
exp(
F
0
2)
{
exp(
F
0
2)
­
1}

so
that
S(
x)
=
[
A0
+
A1
x2
]
½
.

The
use
of
this
model
to
illustrate
the
determination
of
detection
levels
will
be
described
below.
Twentieth
Annual
EPA
Conference
May
1997
8
­
31
Methods
for
Determination
of
Detection
This
section
will
present
a
brief
summary
of
four
alternative
methods
for
determining
detection
levels.
All
are
similar
functionally
and
all
require
estimates
of
precision
at
zero
or
low,
near
zero
concentrations.

Currie
(
1995)
IUPAC
Pure
and
Applied
Chemistry:

"
The
decision
`
detected'
or
`
not
detected'
is
made
by
comparison
of
the
estimated
quantity
with
the
Critical
Value
LC
of
the
respective
distribution,
such
that
the
probability
of
exceeding
LC
is
no
greater
than
"
if
the
analyte
is
absent
(
L=
0,
null
hypothesis)"

LC
=
t1­",<
s0
In
this
definition,
t1­",<
is
the
value
of
a
t­
statistic
with
<
degrees
of
freedom
and
s0
is
an
estimate
of
the
standard
deviation
of
measurement
at
zero
concentration.

Rocke­
Lorenzato
(
1995)
Technometrics:

"
3.1
Detection
Limits:
...
observations
at
true
concentration
..
zero
are
normally
distributed
with
standard
deviation
F,.
If
r
replicates
are
used,
then
any
average
of
measured
values
greater
than
D
is
unlikely
to
have
come
from
a
zero
concentration
sample."

D
=
3
F,
/
or
American
Chemical
Society
[
ACS]
(
1983)
Analytical
Chemistry:

"
The
limit
of
detection
(
LOD)
is
defined
as
the
lowest
concentration
level
that
can
be
determined
to
be
statistically
different
from
a
blank."
The
recommended
value
of
LOD
is
3
F
where
F
is
the
standard
deviation
of
blank
measurements.

US
EPA
(
1984):

The
US
EPA
method
detection
limit
(
MDL),
described
in
40
CFR
Part
136,
Appendix
B,
was
first
promulgated
as
an
EPA
procedure
in
the
Federal
Register
in
1984.
The
MDL
is
based
on
the
procedure
described
in
Glaser
et
al
(
1981)
which
defines
the
MDL
as
MDL
=
t0.99,<
s
where
t0.99,<
is
the
99th
percentile
point
of
the
t
distribution
with
<
degrees
of
freedom
and
s
is
an
estimate
of
the
standard
deviation
of
measurement
at
a
low
concentration
level.
The
MDL
procedure
provides
a
step
by
step
procedure
for
selecting
concentration
levels
at
which
replicate
measurements
are
made
that
are
the
basis
for
the
estimated
standard
deviation
and
the
number
of
degrees
of
freedom
for
the
t
statistic.
For
the
recommended
7
replicates,
<
=
6
and
t0.99,6
=
3.14.
Twentieth
Annual
EPA
Conference
8
­
32
May
1997
These
four
methods
for
determining
detection
levels
are
similar.
Operationally,
the
objective
is
to
provide
some
level
above
background
at
which
it
is
possible
to
decide
that
an
analyte
is
present
in
a
sample.
This
is
implemented
by
specifying
a
multiple
of
the
standard
deviation
of
measurement
at
or
near
zero
concentration
and
determining
an
estimate
of
that
level
using
chemical
measurements
generated
for
that
purpose.
The
Currie,
R­
L,
and
ACS
estimators
require
estimates
of
the
standard
deviation
at
zero
while
the
EPA
procedure
allows
for
the
use
of
a
standard
deviation
estimated
at
a
level
that
is
usually
greater
than
zero.

The
EPA
procedure,
in
fact,
has
a
number
of
advantages
over
the
other
procedures
that
include
the
following:
the
EPA
procedure
is
stated
in
a
straight­
forward,
step­
by­
step
manner
that
many
chemists
find
easy
to
follow;
objective
procedures
are
provided
for
selecting
initial
spike
concentrations
at
which
standard
deviation
estimates
are
generated
and
optional
iterative
steps
are
provided
that
result
in
improved
estimates
of
standard
deviations
and
detection
levels.
The
advantages
associated
with
the
EPA
procedure
can
be
quite
significant
especially
in
cases
where
it
is
not
feasible
to
obtain
measurements
directly
on
blank
or
zero
concentration
samples.
Under
the
assumption
that
the
R­
L
model,
or
a
similar
model,
is
an
appropriate
representation
of
measurement
error,
the
EPA
procedure
provides
a
conservative
estimate
of
detection
capabilities.
That
is,
if
measurement
error
increases
with
concentration,
the
EPA
procedure
should
tend
to
result
in
somewhat
larger
estimates
of
detection
levels
in
comparison
with
procedures
that
utilize
an
estimate
of
the
standard
deviation
at
zero.

MEASUREMENT
VARIATION
FOR
METHOD
1638
The
MDL
procedures
will
be
illustrated
using
replicate
measurement
data
at
low
spike
concentrations
using
draft
EPA
Method
1638:
Inductively
coupled
plasma
mass
spectrometry
for
metals.
Method
1638
was
used
to
generate
replicate
measurements
for
a
number
of
metals
at
low
spike
concentrations.
For
each
metal,
seven
replicate
measurements
were
made
at
12
concentrations,
from
zero
to
25,000
parts
per
trillion.
As
shown
in
Figure
2,
the
variation
among
replicate
measurements
does
not
increase
in
a
linear
relationship
to
the
spike
concentration.
The
use
of
the
semi­
log
scale
in
Figure
2
provides
a
plot
of
the
relationship
that
allows
for
a
clear
picture
of
the
relationship
between
standard
deviation
and
concentration.
As
hypothesized
by
Glaser
et
al.
(
1981)
in
the
development
of
the
MDL,
measurement
variation
appears
to
be
approximately
constant
for
spike
concentrations
in
a
neighborhood
near
zero.
This
apparent
relationship
between
measurement
and
variation
is
consistent
conceptually
with
the
Rocke
and
Lorenzato
(
1995)
two­
component
model
described
above.

MDL
PROCEDURE
Key
features
of
the
EPA
MDL
procedure,
of
direct
relevance
to
the
discussion
in
this
paper,
are
described
in
summary
form
in
this
section.
Complete
descriptions
of
the
procedure
are
given
in
EPA
(
1995)
and
Glaser
et
al.
(
1981).

Selection
of
Concentrations
Selection
of
analyte
concentrations
at
which
to
determine
initial
estimates
of
the
detection
level
are
critically
important.
The
initial
estimate
of
the
detection
level
is
used
as
the
concentration
at
which
samples
are
spiked
and
replicate
measurements
made
to
estimate
the
precision
used
in
the
determination
of
the
MDL.
The
rules
in
the
MDL
Procedure
for
selecting
initial
spiking
concentration
are
as
follows:
Twentieth
Annual
EPA
Conference
May
1997
8
­
33
a.
The
concentration
value
that
corresponds
to
an
instrument
signal/
noise
in
the
range
of
2.5
to
5.

b.
The
concentration
equivalent
of
three
times
the
standard
deviation
of
replicate
instrumental
measurements
of
the
analyte
in
reagent
water.

c.
The
region
of
the
standard
curve
where
there
is
a
significant
change
in
sensitivity,
i.
e.,
a
break
in
the
slope
of
the
standard
curve.

d.
Instrument
limitations.

In
our
opinion,
the
most
objective
of
these
is
Rule
b.
(
which
corresponds
to
1.
b.
in
EPA
[
1995]).
Replicate
measures
at
zero
concentration
were
available
for
the
ICPMS
data
so
the
standard
deviation
of
these
measurements
was
used
as
the
basis
for
the
initial
estimates
of
the
MDL.
Subsequent
estimates
of
the
standard
deviation
required
by
the
MDL
procedure
were
determined
by
the
Rocke­
Lorenzato
model
for
standard
deviation
as
a
function
of
concentration
fit
to
the
ICPMS
data.
The
procedure
is
described
briefly
below.

MDL
Iteration
Procedure
The
MDL
procedure
uses
an
iteration
routine
to
proceed
to
a
final
value
for
the
MDL.
The
iterative
steps
are
summarized
below
in
order:

1.
S0
=
standard
deviation
of
replicates
at
0
(
rule
b.,
above);

2.
MDL0
=
3
S0
=
initial
MDL
estimate;

3.
S1
=
estimate
of
standard
deviation
based
on
7
replicates
at
MDL0
;

4.
MDL1
=
3.14
S1
=
MDL,
where
3.14
is
the
99th
percentile
value
of
the
t
distribution
with
6
degrees
of
freedom;

5.
If
1<
MDL1
/
MDL0
<
5,
then
go
to
step
6.
If
this
inequality
is
not
satisfied,
then
repeat
initial
MDL
concentration
selection
process,
i.
e.,
assess
S0
and
other
rules
for
concentration
selection
described
above
and
obtain
new
replicate
measurements;

6.
S2
=
estimate
of
standard
deviation
based
on
7
replicates
at
MDL1;

7.
If
S2
2
/
S1
2
<
3.05
(
i.
e.,
an
F
statistic
comparison
of
S2
2
and
S1
2
),
then
S
=
/{[
S1
2
+
S2
2
]/
2}
and
MDL2
=
2.68
S
,
where
2.68
is
the
99th
percentile
value
of
the
t
distribution
with
12
degrees
of
freedom;
and
MDL
=
MDL2.

8.
If
S2
2/
S1
2
$
3.05
then
MDL1
=
3.14
S2
.
Twentieth
Annual
EPA
Conference
8
­
34
May
1997
9.
S2
2
is
redesignated
as
S1
2
and
reiterate
steps
beginning
at
Step
6.

Currently,
steps
6,
7,
8
and
9,
are
designated
in
the
MDL
procedure
as
optional
steps
and
are
rarely
implemented.
Typically,
the
MDL
is
determined
by
Step
4
if
the
condition
in
Step
5
is
satisfied.
In
actual
practice,
implementation
of
the
MDL
procedure
would
require
generation
of
replicate
measurements
at
concentrations
determined
by
MDL0
and
MDL1
in
order
to
generate
estimates
S1
and
S2,
respectively.
In
this
study,
the
required
estimates,
S1
and
S2,
were
determined
by
a
Rocke­
Lorenzato
model
fit
to
the
ICPMS
data.
That
is,
the
parameters
of
the
Rocke­
Lorenzato
model
(
A0
and
A1
,
defined
above)
for
standard
deviation
of
measurement
as
a
function
of
concentration
were
estimated
from
the
ICPMS
data
and
the
estimates
S1
and
S2
required
for
the
MDL
procedure
were
calculated
from
the
estimated
model
and
the
concentrations
generated
in
Steps
2
and
4.
This
requires
that
the
Rocke­
Lorenzato
model
provides
an
adequate
fit
to
the
observed
data.
For
the
ICPMS
data,
we
were
able
to
obtain
satisfactory
fits
to
the
data
using
weighted
least
squares
to
estimate
the
parameters.
In
general,
the
fit
of
the
estimated
models
was
satisfactory,
especially
in
the
critical
low
concentration
range.
This
is
illustrated
in
Figure
3
which
shows
the
estimated
parameters
and
plots
of
the
fits
for
weighted
least
squares,
non­
linear
least
squares
and
maximum
likelihood
estimates
of
the
parameters
for
Pb206.
The
use
of
the
MDL
procedure
with
standard
deviations
estimated
from
the
model
is
illustrated
in
Figure
4
for
the
Pb206
data
and
the
model
parameters
estimated
with
weighted
least
squares.
A
summary
of
similar
results
for
all
the
ICPMS
data
and
plots
for
all
the
metals
are
contained
in
SAIC
(
1997)
"
Summary
of
statistical
evaluations
of
data
on
the
measurement
of
selected
metals
by
inductively
coupled
plasma
mass
spectroscopy
using
EPA
Method
1638".
Twentieth
Annual
EPA
Conference
May
1997
8
­
35
Figure
2:
Standard
Deviations
of
Replicate
Measurements
at
Low
Concentrations,
EPA's
ICP/
MS
Data.
Twentieth
Annual
EPA
Conference
8
­
36
May
1997
Measurement
Precision
at
the
MDL
Components
of
measurement
precision
at
a
given
concentration
may
be
evaluated
using
the
estimated
Rocke­
Lorenzato
model.
That
is,
assuming
the
model
provides
a
satisfactory
fit
to
data
on
precision
as
a
function
of
concentration,
the
fitted
model
can
be
used
to
evaluate
the
relative
contributions
of
the
constant
error
term
and
the
proportional
error
term
to
the
overall
error
at
any
given
concentration.
Of
particular
interest
are
the
components
of
error
at
the
MDL..
Any
value
selected
as
a
detection
criterion
should
be
in
a
concentration
range
where
error
is
effectively
constant.
The
MDL
procedure
is
intended
to
result
in
such
a
value.
This
is
implied
as
a
property
of
the
MDL
in
the
discussion
in
Glaser
et
al.
(
1981).
Also,
this
corresponds
to
the
discussion
of
detection
in
Rocke­
Lorenzato
(
1995)
in
which
a
detection
level
is
defined
only
in
terms
of
,
the
precision
of
measurement
at
zero.
Using
the
estimated
Rocke­
Lorenzato
model,
it
is
possible
to
evaluate
the
components
of
the
error
at
the
MDL
values
obtained
through
application
of
the
EPA
procedure.
This
may
be
done
as
follows:

Let
a0
and
a1
denote
estimated
values
of
A0
and
A1,
respectively,
and
let
denote
the
estimated
standard
deviation
of
the
Rocke­
Lorenzato
model
for
a
given
spike
concentration
.
Then,
for
x
=
MDL,
define
and
which
measure
the
relative
proportions
of
the
contribution
of
and
through
their
representations
as
functions
of
and
,
respectively,
at
the
MDL.
In
effect,
T
0
=
percent
of
s(
MDL)
due
to
proportional
error
and
T,
=
percent
of
s(
MDL)
due
to
constant
error.

DISCUSSION
The
statistics
and
were
evaluated
for
each
of
the
metals
in
the
ICPMS
data
for
each
method
of
estimating
and
,
i.
e.,
weighted
least
squares
(
WLS),
nonlinear
least
squares
(
NLS)
and
maximum
likelihood
estimates
(
MLE).
The
results
are
summarized
in
the
following
table.
In
general,
for
all
of
the
ICPMS
metals,
weighted
least
squares
estimates
provided
the
best
fit
of
the
data
to
the
R­
L
model
in
comparison
to
nonlinear
least
squares
and
maximum
likelihood.
Weighted
least
squares
was
also
the
least
complicated
to
implement
computationally.
Maximum
likelihood
did
not
perform
well
generally
due
to
numerical
problems
with
the
computer
routines
used
to
solve
for
the
maximum
likelihood
estimates
in
some
cases.
These
results
are
discussed
in
SAIC
(
1997).
As
discussed
by
Rocke
and
Lorenzato
(
1995),
computation
of
the
maximum
likelihood
estimates
requires
nonlinear
minimization
over
several
parameters
for
each
calculation
of
the
integrand
in
the
product
of
n
improper
integrals.
As
a
consequence,
it
is
not
surprising
that
the
maximum
likelihood
estimates
would
be
more
sensitive
to
computational
difficulties.
This
does
not
detract
from
the
theoretical
validity
and
importance
of
the
maximum
likelihood
approach.
In
fact,
this
indicates
that
additional
investigation
of
maximum
likelihood
estimation
for
this
problem
should
be
pursued.
Twentieth
Annual
EPA
Conference
May
1997
8
­
37
The
results
in
the
Table
provide
an
indication
of
the
degree
to
which
measurement
precision
at
the
MDL
is
determined
by
the
constant
and
proportional
error
components.
For
the
WLS
and
NLS
estimates,
the
results
show
a
consistent
pattern
of
dominance
of
the
constant
error
term
at
the
MDL.
That
is,
on
average,
roughly
97%
of
the
precision
at
the
MDL
is
determined
by
the
constant
term.
This
is
expected
because
the
MDL
procedure
is
intended
to
result
in
a
detection
level
at
a
low
concentration
where
precision
is
effectively
constant.
This
is
an
important
property
for
a
detection
criterion
to
possess
to
be
useful
as
an
indicator
of
the
presence
of
an
analyte.
The
fact
that
this
is
the
case
for
these
data
provides
support
for
the
use
of
the
MDL
procedure.
The
method
described
here,
using
and
,
is
just
one
approach
for
evaluating
components
of
measurement
precision.
There
may
be
other
criteria
for
evaluation
of
measurement
precision
that
would
lead
to
a
different
numerical
representation
of
the
results.
Nevertheless,
we
believe
that,
in
general,
any
evaluation
based
on
an
appropriate
mathematical
criterion
would
lead
to
similar
conclusions.
We
base
this
judgment
on
the
observed
pattern
of
the
data
on
precision
as
a
function
of
concentration
and
the
performance
of
the
MDL
procedure
which
is
illustrated
in
Figure
4.

The
results
presented
here
demonstrate
the
implementation
of
the
MDL
procedure
with
the
additional
optional
steps.
The
procedure
was
illustrated
using
the
Rocke­
Lorenzato
model
to
determine
S1
and
S2
and
pooling
the
results
to
determine
the
final
MDL.
The
use
of
the
Rocke­
Lorenzato
model
and
the
requirement
that
the
ratio
of
successive
MDL
values
be
between
1
and
5,
insures
that
the
final
MDL
value
is
between
MDL0
and
MDL1
because
of
the
strictly
increasing
nature
of
the
model.
The
strictly
increasing
property
is
due
to
the
multiplicative
component
of
the
model
which
is
additive
with
a
coefficient,
a1,
which
is
always
positive.
In
actual
practice,
however,
S1
and
S2,
would
be
estimated
using
the
seven
replicate
measurements
at
the
concentrations
determined
at
each
step.
This,
of
course,
would
yield
less
predictable
results
than
those
presented
here.
However,
replicate
measurements
at
an
additional
concentration,
selected
according
to
the
rules
provided
in
the
procedure
and
pooling
the
results,
will
add
statistical
precision
to
the
estimate
of
the
standard
deviation
and
a
check
on
the
initial
MDL
determination.
In
spite
of
the
strictly
increasing
nature
of
the
standard
deviation
estimated
by
the
Rocke­
Lorenzato
model,
the
degree
of
increase
in
the
low
concentration
range
is
expected
to
be
small
so
that
there
is
a
range
where
the
standard
deviation
is
effectively
constant.
As
long
as
the
MDL
is
based
on
a
standard
deviation
in
this
concentration
range,
the
procedure
should
provide
a
satisfactory
result.
The
results
obtained
here
suggest
that
this
is
the
case
for
the
ICPMS
data
considered.
Twentieth
Annual
EPA
Conference
8
­
38
May
1997
WLS
NLS
MLE
Analyte
Ag
107
2.87
97.13
2.93
97.07
10.06
89.94
Cd
111
2.11
97.89
2.27
97.73
9.82
90.18
Cu
63
1.63
98.37
1.83
98.17
10.92
89.08
Ni
60
3.04
96.96
3.10
96.90
86.23
13.77
Pb
206
2.63
97.37
2.74
97.26
12.64
87.36
Pb
207
2.35
97.65
2.48
97.52
12.06
87.94
Pb
208
3.18
96.82
3.30
96.70
8.09
91.91
Sb
123
2.11
97.89
2.21
97.79
7.56
92.44
Se
82
6.43
93.57
6.41
93.59
70.29
29.71
Tl
205
3.41
96.59
3.59
96.41
14.53
85.47
Zn
66
3.13
96.87
3.14
96.86
NA
NA
Average
2.99
97.01
3.09
96.91
24.22
75.78
TABLE:
Percent
of
estimated
measurement
precision
due
to
constant
and
proportional
error
at
estimated
analyte
specific
MDL
T,
(%)
=
percent
of
s(
MDL)
due
to
constant
error
T
0
(%)
=
percent
of
s(
MDL)
due
to
proportional
error
WLS
=
weighted
least
squares
estimates
NLS
=
nonlinear
least
squares
estimates
MLE
=
maximum
likelihood
estimates
Twentieth
Annual
EPA
Conference
May
1997
8
­
39
REFERENCES
American
Chemical
Society
Committee
on
Environmental
Improvement.
(
1983)
Principles
of
Environmental
Analysis,
Analytical
Chemistry,
V.
55,
2210­
2218.

Caulcutt,
R.
and
Boddy,
R.
(
1983)
Statistics
for
Analytical
Chemists.
Chapman
and
Hall,
New
York.

Currie,
L.
A.
(
1995)
Nomenclature
in
evaluation
of
analytical
methods
including
detection
and
quantification
capabilities,
Pure
&
Applied
Chemistry,
V.
67,
1699­
1723.

EPA
(
1995)
Appendix
B
to
Part
136
­
Definition
and
Procedure
for
the
Determination
of
the
Method
Detection
Limit
­
Revision
1.11,
40
Combined
Federal
Register,
U.
S.
Government
Printing
Office,
Washington.

EPA
(
1996)
Method
1638:
Determination
of
Trace
Elements
in
Ambient
Waters
by
Inductively
Coupled
Plasma
 
Mass
Spectrometry,
Engineering
and
Analysis
Division
(
4303),
Washington.

Gibbons,
R.
D.
(
1994)
Statistical
Methods
for
Groundwater
Monitoring.
Wiley,
New
York.

Gibbons,
R.
D.
(
1995)
Some
statistical
and
conceptual
issues
in
the
detection
of
low
level
environmental
pollutants,
Environmental
and
Ecological
Statistics,
Vol.
2,
Num.
2.

Glaser,
J.
A.,
Foerst,
D.
L.,
McKee,
G.
D.,
Quane,
S.
A.
and
Budde,
W.
L.
(
1981)
Trace
Analyses
for
wastewaters.
Environmental
Science
and
Technology,
Vol.
15,
1426­
35.

Rocke,
D.
and
Lorenzato,
S.
(
1995)
A
Two­
Component
Model
for
Measurement
Error
in
Analytical
Chemistry,
Technometrics,
Vol.
37,
No.
2.
Twentieth
Annual
EPA
Conference
8
­
40
May
1997
R­
L
Model
a0:
1.06706321
a1:
0.00009617
s(
eps):
1.03298752
s(
nu):
0.00980581
F­
Test:
18.35938648
p­
value:
0.00159929
log
RSS:
7.44184892
Weighted
Least
Squares
Spike
Concentration
(
log­
scaled)
Fitted
Curve
0
10
100
1000
25000
1
108
214
°
°°
°
°
°
°
°
°
°
°
°

°°°
°
°
°
°
°

°

°
°

Fitted
Values
(
log­
scaled)
Residuals
1
5
10
50
­
30
­
10
0
10
20
°°°
°
°
°
°
°

°

°
°

°
°
°
°
°
°
°
°
°

°

°
°

Quantiles
of
Standard
Normal
Residuals
­
1
0
1
­
30
­
10
0
10
20
°
°
°
°
°
°
°
°
°
°
°
°
a0:
6.40406438
a1:
0.00010445
s(
eps):
2.53062529
s(
nu):
0.01021944
error:
12.51783357
log
RSS:
7.35689372
Nonlinear
Least
Squares
Spike
Concentration
(
log­
scaled)
0
10
100
1000
25000
1
108
214
°
°°
°
°
°
°
°
°
°
°
°

°°°°
°
°
°
°

°

°
°

Fitted
Values
(
log­
scaled)
5
10
50
­
30
­
20
­
10
0
10
°°°°
°
°
°
°

°

°
°

°
°
°
°
°
°
°
°
°

°

°
°

Quantiles
of
Standard
Normal
­
1
0
1
­
30
­
20
­
10
0
10
°
°
°
°
°
°
°
°
°
°
°
°
a0:
28.53929844
a1:
0.00223586
s(
eps):
5.34221849
s(
nu):
0.04720596
iter:
32.00000000
log
RSS:
13.85397717
Maximum
Likelihood
Spike
Concentration
(
log­
scaled)
0
10
100
1000
25000
1
473
946
°
°°
°
°
°
°
°
°
°
°
°

°°
°
°
°
°
°

°

°

°

Fitted
Values
(
log­
scaled)
5
10
50
500
­
800
­
400
0
°°
°
°
°
°
°
°

°

°

°
°
°
°
°
°
°
°
°

°

°

°

Quantiles
of
Standard
Normal
­
1
0
1
­
800
­
400
0
°
°
°
°
°
°
°
°
°
°
°
°
Rocke­
Lorenzato
Model
for
Standard
Deviation
vs
Concentration
Comparison
of
WLS,
NLS,
and
MLE
Fits
ICPMS
Metals
Data
Pb
206
with
Spike
Concentration
<=
25000
Figure
3
Estimation
results
using
Weighted
Least
Squares
(
WLS),
Non
Linear
Least
Squares
(
NLS)
and
Maximum
Likelihood
Estimation
(
MLE)
to
fit
the
Rocke­
Lorenzato
Model
for
standard
deviation
vs.
concentration
using
data
on
Pb206
via
ICPMS.
Twentieth
Annual
EPA
Conference
May
1997
8
­
41
Standard
Deviation
MDL
*
S0
(
0,0.58)
°
°
°
°
°
°
°
°
°
°
°

*
S1
(
1.74,
1.03)

MDL0
*
S2
(
3.24,
1.03)

MDL1
0
1
10
20
50
100
200
500
1000
2000
5000
10000
25000
Spike
Concentration
(
parts
per
trillion)
0.1
1
10
100
The
R­
L
Parameters:
a0=
1.067063
a1=
0.000096
MDL=
2.7693
Procedure
for
the
Determination
of
the
Method
Detection
Limit
adjusted
with
respect
to
the
Rocke­
Lorenzato
Model
Method
of
Estimation:
Weighted
Least
Squares
Analyte:
Pb
206
(
Note:
Both
axes
are
on
a
log10
scale.)

AIC
(
1997)
"
Summary
of
statistical
evaluations
of
data
on
the
measurement
of
selected
metals
by
Inductively
coupled
plasma
mass
spectroscopy
using
EPA
Method
1638".
Figure
4:
Illustration
of
the
MDL
procedure
including
optional
iterative
step
with
data
on
Pb206
via
ICPMS,
standard
deviation
determined
by
Rocke­
Lorenzato
model
fit
using
Weighted
Least
Squares,
final
MDL
value
(
at
vertical
solid
line)
calculated
with
S
determined
by
pooling
S1
2
and
S2
2.

QUESTION
AND
ANSWER
SESSION
MR.
TELLIARD:
Questions?

MR.
PHILLIPS:
John
Phillips,
Ford
Motor
Company.
Can
you
tell
me
what
are
the
criteria
you
use
for
MDL
acceptance?
Is
it
a
one
to
five
times,
the
spike
concentration
must
be
one
to
five
times
the
MDL
or
one
to
10?
I
see
a
lot
of
difference
from
lab
to
lab.

MR.
KAHN:
It's
one
to
five.
It's
not
necessarily
acceptance,
it's
part
of
the
procedure,
it's
in
that
slide
on
the
iterative
steps.

MR.
TELLIARD:
If
the
results
aren't
in
the
right
range,
you
repeat.
Twentieth
Annual
EPA
Conference
8
­
42
May
1997
MR.
PHILLIPS:
So,
if
it's
not
in
one
to
five
range,
you
have
to
repeat
it?

MR.
TELLIARD:
Yes.

MR.
PHILLIPS:
Correct?

MR.
KAHN:
Yes.

MR.
TELLIARD:
Right.

MR.
PHILLIPS:
Can
you
tell
me,
you
did
12
different
concentrations
in
the
study
that
you
did
at
the
ICPMS?

MR.
KAHN:
Right.

MR.
PHILLIPS:
Did
you
try
and
compute
MDLs
at
each
of
those
different
concentrations,
and
if
so,
can
you
tell
me
what
the
range
of
MDLs
was?

MR.
KAHN:
No,
we
didn't
do
that.
But
actually,
if
you
can
get
the
slides
up
that
have
the
data
with
the
plots
of
the
standard
deviations
versus
concentration,
either
the
small
multiples
or
the
thallium
example.
What
you
see
there
is
a
fairly
large
range
of
where
the
standard
deviation
is
constant,
or
effectively
constant.

MR.
PHILLIPS:
Yes.

MR.
KAHN:
No,
not
that
one.

MR.
TELLIARD:
The
one
with
all
the
little
graphs
on
it.

MR.
KAHN:
All
the
little
plots
there.
So,
what
that
means
is,
if
you're
in
this
range,
in
that
low
concentration
range,
the
range
where
measurement
variability
is
effectively
constant,
you're
going
to
get
roughly
the
same
MDL.

MR.
PHILLIPS:
Now,
this
works
for
metals,
but
how
about
chromatographic
data,
where
you
don't
have
random
positive
and
negative
noise?

MR.
TELLIARD:
We
didn't...

MR.
KAHN:
We
haven't
done
that,
although
that's
something
that
we
should
do,
it's
something
I
want
to
do.

MR.
TELLIARD:
He
keeps
telling
me
we're
going
to
do
more.

MR.
KAHN:
I
keep
telling
Bill
we've
got
to
generate
more
data.
But
seriously,
this
is
a
very
serious
point.
If
you
want
to
know
what
the
error
structure
is
like,
and
that's
the
term
often
used
to
characterize
these
kinds
of
data,
error
structure.
If
you
want
to
know
what
it's
like,
you've
got
to
take
data
in
that
range.
You
don't
extrapolate
models
out
of
that
range.
Spend
the
money,
go
through
the
effort...
Twentieth
Annual
EPA
Conference
May
1997
8
­
43
MR.
PHILLIPS:
Yes,
I'd
probably
generated
over
1,000
of
these
types
of
curves
in
the
last
year.
Much
of
it
looking
at
metals
data.
I
would
say
that
it
usually
follows
the
Rocke­
Lorenzato
or
exponential
fit
to
the
data.
But
if
you
determine
MDLs
at
each
of
those
points,
you
get
widely
varying
results,
many
of
which
are
acceptable
for
the
MDL
protocol.
As
a
matter
of
fact,
it's
sometimes
many
orders
of
magnitude
difference
in
an
MDL
determination,
at
each
of
those
different
points,
with
the
seven
replicates.
That's,
I
think,
why
it
was
referenced
as
an
unanchored
statistic,
because
you
can
get
almost
any
number
you
want,
depending
on
which
data
point
you
choose
there.

MR.
TELLIARD:
Not
if
you
follow
the
spiking
level
that
you're
supposed
to
start
with.

MR.
PHILLIPS:
I'm
saying
that's
all
within
the
spiking
level
that's
in
the
protocol.

MR.
KAHN:
Yes,
but
you're
supposed
to
use,
I
mean
that's
the
point
of
specifying,
those
initial
spike
level
determinations.
I'm
not
saying
that
there's
no
variation,
that
you're
not
going
to
get
any
variation,
you
will
get
some
variation
and
it's
also
possible
that
that
should
be
accounted
for,
as
we
consider
modifications
to
the
MDL
procedure.
The
point
of
what
we
were
doing
here
was
to
evaluate
the
current
MDL
procedure
and
the
optional
iteration
scheme
that's
in
the
current
method
with
an
objective
measure
of
the
intial
spike
level.

MR.
TELLIARD:
Andy?

MR.
EATON:
Henry,
Andy
Eaton,
Montgomery
Watson
Labs.
Two
comments
pertaining
to
the
data
that's
up
there.
If
you
look
at
that
data
and
the
fact
that
you
do
have
a
relatively
flat
standard
deviation,
the
whole
argument
for
iterative
and
going
down
until
you
get
to
two
to
five
times
the
expected
signal
to
noise
ratio,
maybe
irrelevant
because
you've
got
cases
there
where
presumably
you're
already
flat
at
well
above
the
two
to
five
times,
and
therefore
it
calculates
out
fine,
although
I
concur
that
you
should
keep
going
to
look.
Second
thing,
the
one
thing
that's
not
in
the
EPA
MDL
protocol,
and
was
something
that
we
did
build
into
the
requirements
for
the
information
collection
rule,
is
to
gather
that
data
at
multiple
concentration...
or
at
multiple
runs
over
multiple
days.
So,
you're
really
looking
at
your
overall
precision
and
not
the
precision
that
you
get
when
you
do
seven
replicates
right
in
a
row.

MR.
TELLIARD:
It
hasn't
been
the
function
to
do
that.
You
know,
we're
going
to
put
this
thing
back
out
there
in
the
late
summer
and
we'll
make
some
changes,
but
I
don't
think
that
will
be
one
of
them.
It
doesn't
prohibit
it,
is
one
of
our
answers
on
that.
I
don't
think
it
buys
a
lot
for
you
at
the
bench,
when
you're
trying
to
do
this
thing,
to
do
it
that
way.

MR.
KAHN:
I
mean,
if
you
have
control
over,
if
you
have
good
control
over
your
measurement
process,
the
fact
that
you
do
it
at
different
days,
shouldn't
make
that
much
of
a
difference.
If
it
does,
I
would
say
somehow
you've
lost
control
of
the
process.

MR.
TELLIARD:
We're
losing
control
of
the
time.
One
quick
question.

MR.
MADDALONE:
Could
we...
this
is
Ray
Maddalone,
TRW.
Would
you
move
that
slide
up
a
little
bit?
I
didn't
see
it
from
the
back.
What's
the
scale
on
the
standard
deviation
there?
It's
zero
to
100
to
200.
I
wondered,
if
you'd
move
that
up
a
bit
and
saw
what
the
standard
of
deviation
really
looks
like
at
those
low
levels,
what
sort
of
variability
we
would
actually
be
seeing
and
what
would
be
the
numbers
for
the
MDL.
Also,
I'd
like
to
see
that
first
chart
that
was
referenced
to
Gibbons,
where
they
showed
the
real
data
versus
the
Gibbons
model.
Twentieth
Annual
EPA
Conference
8
­
44
May
1997
MR.
TELLIARD:
Can
we
do
this
later?
We're
running
out
of
time.

MR.
MADDALONE:
I
just
wanted
to
make,
just
ask
Henry
a
question
about
that.
Did
you
actually
calculate
with
the
real
data,
not
with
the
Gibbons
model
data,
which
you
objected
to.
Did
you
take
those
data
points
that
were
the
real
data
points
and
calculate
the
range
of
MDLs
from
them?

MR.
KAHN:
No,
we
did
not.

MR.
MADDALONE:
Thank
you.

MR.
KAHN:
The
question
there
was
whether
the
data
fit
the
model.
Clearly
the
data
do
not
fit
the
model
Gibbons
was
using.
If
the
data
don't
fit
the
model,
conclusions
based
on
the
model
are
suspect.

MR.
TELLIARD:
Harry?

MR.
MCCARTY:
Real
briefly,
Bill,
I'll
amplify
what
Andy
Eaton
said.
I'm
the
person
who
looked
at
the
MDL
studies
from
over
300
ICR
labs
for
up
to
17
analytes.
Three
days
makes
a
difference.
The
other
comment
I
didn't
make,
Bill
Draper's
presentation
yesterday,
most
of
the
labs
submitting
data
can't
calculate
an
MDL,
because
they're
using
a
spread
sheet
that
uses
the
wrong
sigma
value.
The
sigma
sub
N,
not
sigma
sub
N­
1.
If
you
take
that
variation
out,
you're
accounting
for
not
the
orders
of
magnitude,
that
Bill
was
talking
about,
but
you're
talking,
you
know,
in
some
cases,
20
to
25
percent,
just
in
the
variability
you'll
see
across
the
lab.

But
Henry,
I
would
suggest
you
get
in
touch
with
the
ICR
brethren
you
have
there
at
the
agency.
All
that
data
is
available,
much
of
it
is
in
fact
in
electronic
format
and
you
could
pull
together
a
fair
number
of
MDL
studies
in
a
limited
range
of
concentrations,
because
the
ICR
did
say
spike
somewhere
in
this
range.
There
are
good
data,
if
you
will,
done
over
three
days,
the
way
it
was
requested.
There
are
bad
data
done
on
a
single
day
or
two
days,
you
may
be
able
to
make
some
of
those
comparisons.
But
for
all
the
effort
that
was
put
into
looking
at
this,
there's
a
hell
of
a
lot
of
data
out
there
and
it
would
be
nice
to
be
able
to
use
it.

MR.
TELLIARD:
Thank
you,
Henry.