Document ID: EPA-HQ-OAR-2005-0078-0005
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2005-02-25T05:00Z

APPENDIX
2
Survey
Sample
Size
and
Expected
Response
Rate
1
Survey
Sample
Size
and
Expected
Response
Rate
Required
Number
of
Completed
Forms
The
required
number
of
completed
survey
forms
(
n)
is
derived
from
the
level
of
precision
required
for
the
estimates
and
the
desired
level
of
confidence.
A
typical
estimated
quantity
is
the
percentage
(
P)
of
the
population
which
has
a
particular
characteristic.
The
uncertainty
in
this
estimate
is
denoted
by

P,
the
half­
width
of
the
confidence
interval
for
P.
When
estimating
a
fixed
percentage
P,
the
uncertainty
at
the
specified
level
of
confidence
is
determined
by
the
number
of
completed
questionnaire
forms
obtained
in
the
survey.
Conversely,
for
a
fixed
level
of
confidence,
the
required
number
of
completed
forms
may
be
estimated
by
specifying
a
desired
value
for
the
half­
width
of
the
confidence
interval

P.

Table
1
shows
the
required
sample
sizes
for
two
levels
of
confidence,
90
and
95%
(
Levy
and
Lemeshow).
If
the
desired

P
is
fixed
at
2
percentage
points,
there
is
a
different
sample
size
required
for
each
value
of
P,
ranging
from
n
=
502
at
P
=
5%
to
n
=
2,641
at
P
=
50%
for
a
95%

confidence
interval.
Since
many
characteristics
are
being
estimated
in
the
survey,
the
safest
approach
is
to
use
the
highest
number
of
required
survey
forms.
This
occurs
when
P
=
50%.
If
n
=
2,641
forms
are
completed,
then
characteristics
which
occur
in
less
than
half
of
the
population
will
have
a

P
less
than
2
percentage
points.
(
Note,
however,
that
a

P
of
2
percentage
points
may
be
a
very
large
relative
error
for
small
values
of
P;
i.
e.,
if
P
=
2%
and

P
=
2%,
then
the
relative
error
is
100%).
2
Table
1
a.
Number
of
responses
(
n)
required
for
a
95%
confidence
interval
of
P
±

P.

P
1%
2%
3%
4%
5%
6%
7%

______________________________________________________________­­­­­­­_
P
=
5%
2,007
502
223
125
80
56
41
10%
3,803
951
423
238
152
106
78
15%
5,388
1,347
599
337
216
150
110
20%
6,761
1,690
751
423
270
188
138
25%
7,923
1,981
880
495
317
220
162
30%
8,874
2,219
986
555
355
247
181
35%
9,614
2,403
1,068
601
385
267
196
40%
10,142
2,535
1,127
634
406
282
207
45%
10,459
2,615
1,162
654
418
291
213
50%
10,564
2,641
1,174
660
423
293
216
b.
Number
of
responses
(
n)
required
for
a
90%
confidence
interval
of
P
±

P.

P
1%
2%
3%
4%
5%
6%
7%

______________________________________________________________­­­­­­­_
P
=
5%
1,414
353
157
88
57
39
29
10%
2,679
670
298
167
107
74
55
15%
3,795
949
422
237
152
105
77
20%
4,763
1,191
529
298
191
132
97
25%
5,581
1,395
620
349
223
155
114
30%
6,251
1,563
695
391
250
174
128
35%
6,772
1,693
752
423
271
188
138
40%
7,144
1,786
794
446
286
198
146
45%
7,367
1,842
819
460
295
205
150
50%
7,442
1,860
827
465
298
207
152
3
Required
Number
of
Telephone
Contact
Attempts
Given
an
estimate
of
n,
the
required
number
of
completed
surveys,
the
number
of
telephone
contact
attempts
(
N)
that
will
be
required
is
dependent
on
several
additional
factors
that
can
only
be
estimated
imprecisely.
These
factors
include:

r1
­
the
Phase
1
response
rate
for
agreeing
to
complete
the
initial
screening,
expressed
as
a
percentage
of
initial
telephone
contacts;

e1
­
the
eligibility
rate,
which
is
the
percentage
of
Phase
1
respondents
who
complete
the
screening
that
are
eligible
to
complete
the
survey;

r2
­
the
Phase
1
response
rate
for
agreeing
to
be
called
back
to
complete
the
survey,
expressed
as
a
percentage
of
the
eligible
Phase
1
respondents;
and
r3
­
the
Phase
2
response
rate
for
completing
the
entire
survey
form,
expressed
as
a
percentage
of
the
eligible
Phase
1
respondents
who
agree
to
be
called
back
to
complete
the
survey.

Of
the
four
factors
above,
the
eligibility
rate
(
e1)
will
have
the
greatest
impact
on
the
required
number
of
telephone
contacts.
The
large
impact
of
this
factor
is
due
to
the
relatively
low
rate
of
incidence
(
approximately
10
percent)
of
asthma
in
the
general
population.
The
survey
design
has
no
effect
on
this
factor.

The
remaining
factors
are
three
different
types
of
response
rates.
Each
of
these
three
factors
may
be
influenced
by
the
survey
design.
Due
to
the
salesman's
"
foot­
in­
the­
door"
effect
(
Hornik,
et
al.),
it
is
expected
that
r3
>
r2
>
r1.
The
foot­
in­
the­
door
effect
states
that
compliance
with
a
small
initial
request
significantly
enhances
the
likelihood
of
compliance
with
a
subsequent
"
target"
request.
Hence,
the
smallest
response
with
the
largest
negative
effect
is
expected
to
be
r1,
the
response
rate
for
completing
the
screening.
The
reasons
for
nonresponse
in
Phase
1
include
no
answer
after
repeated
tries
or
an
outright
refusal
to
participate
after
initial
contact
is
made.

In
their
1988
analysis
of
survey
results
for
Canadian
smokers,
Bull,
et
al.,
report
telephone
response
rates
by
the
cumulative
number
of
call
attempts
made:

1
39%
4
2
67%

3
82%

4
88%

5
92%

6
96%

The
dramatic
increase
in
response
that
occurs
with
an
increase
in
calling
effort
is
confirmed
in
other
U.
S.
studies.
However,
the
very
high
level
of
response
seen
in
Canada
does
not
appear
to
be
reflective
of
current
conditions
in
the
United
States.

In
the
early
1990'
s,
Mishra,
et
al.
report
completion
rates
for
telephone
surveys
with
a
minimum
of
four
call
attempts
in
Orange
County,
CA,
that
range
from
46
percent
to
59
percent,
with
a
midpoint
of
52.5
percent.
The
authors
note
that
ownership
of
telephone
answering
machines
was
over
70
percent
during
this
time
period.
Kristal,
et
al.
report
similar
results
in
their
telephone
health
survey
in
the
State
of
Washington.
This
study
was
conducted
in
two
stages:
the
first
stage
included
up
to
11
initial
call
attempts
including
callbacks;
and
the
second
stage
included
an
additional
11
initial
call
attempts.
Response
rates
in
the
first
stage
were
65
percent
for
women
and
53
percent
for
men,
with
an
average
of
approximately
59
percent.
Only
a
few
percentage
points
of
improvement
were
reported
to
result
from
the
second­
stage
effort.

In
summary,
the
early
Canadian
response
rate
appears
to
be
significantly
higher
than
those
reported
in
later
U.
S.
studies.
The
Canadian
study
represents
a
perhaps
unobtainable
goal
for
U.
S.
telephone
surveys.
Surveys
taken
in
the
Pacific
coast
area
of
the
United
States
five
years
later
show
a
marked
decrease
in
response
rates,
despite
a
relatively
high
number
of
call
attempts.

The
average
of
the
two
midpoints
for
these
studies
is
approximately
56
percent.
This
effect
is
partially
due
to
the
increased
reliance
on
telephone­
answering
machines
in
the
United
States
in
the
1990'
s.
The
results
also
imply
that
residents
along
the
Pacific
coast
simply
were
not
home
as
often
as
Canadian
residents
in
the
late
1980'
s.
However,
the
lower
response
may
also
be
due
to
the
larger
numbers
of
two­
income
and
single­
occupant
residences
on
the
Pacific
coast.
The
currently
planned
survey
will
use
random­
digit
dialing
with
a
minimum
of
seven
call
attempts
for
initial
contact.
The
response
rate
for
completing
the
screening
is
estimated
to
be
r1
=
60%

(
0.60),
reflecting
approximately
the
Washington
State
study
midpoint.
This
estimate
may
be
compared
to
the
midpoint
for
the
Orange
County
study,
augmented
by
eight
percent
given
the
increase
in
the
number
of
contact
attempts
from
four
to
seven.
This
increase
is
the
same
as
that
obtained
in
the
Canadian
study
when
contact
attempts
increase
from
four
to
six.
5
Successful
completion
of
the
screening
phase
will
identify
the
actual
number
of
eligible
respondents.
At
this
time,
EPA
estimates
that
approximately
10
percent
of
U.
S.
households
have
asthmatics
(
Mannino).
Hence,
the
eligibility
rate
is
estimated
to
be
e1
=
0.10.

Respondents
determined
to
be
eligible
in
the
first
phase
will
be
asked
to
participate
in
the
main
portion
of
the
survey.
Not
all
eligible
households
will
agree
to
participate.
However,
households
are
more
likely
to
agree
to
participate
once
the
screening
is
completed.
Hornik,
et
al.
report
that
having
a
"
foot
in
the
door"
increased
response
rates
from
48
percent
to
59
percent
in
Israel,

amounting
to
a
relative
increase
in
response
of
approximately
23
percent.
Hence,
the
estimated
response
rate
for
eligible
respondents
agreeing
to
participate
in
the
second
phase
of
the
survey
is
r2
=
0.74.

In
Phase
2,
eligible
respondents
who
have
completed
the
screening
and
have
agreed
to
participate
in
the
full
survey
effort
will
be
contacted
to
complete
the
full
survey.
At
this
stage,

the
primary
obstacle
will
be
reaching
the
respondent
when
they
are
at
home
and
at
a
convenient
time.
Information
about
the
best
time
for
future
contacts
will
be
obtained
when
respondents
agree
to
participate
in
the
full
survey.
This
will
reduce
the
chance
that
a
respondent
who
has
agreed
to
participate
cannot
be
contacted.
However,
it
is
likely
that
passive
refusals
and
terminations
will
be
encountered
at
this
phase.
Passive
refusal
occurs
when
a
respondent
who
has
agreed
to
participate
requests
that
the
interviewer
"
call
back
later."
After
several
attempts,

with
the
same
result,
it
becomes
obvious
that
the
respondent
does
not
really
intend
to
participate
but
has
not
given
a
direct
refusal.
Passive
refusals
may
also
occur
when
telephone­
answering
machines
are
used
to
"
screen"
calls.
Terminations
occur
when
the
respondent
refuses
to
complete
the
entire
interview.
Longer
interviews
such
as
the
currently
planned
survey
incur
the
risk
of
higher
termination
rates.

A
passive
refusal
rate
of
9.6
percent
for
high­
effort
surveys
is
reported
in
Mishra,
et
al.

Termination
rates
of
approximately
five
percent
are
indicated
in
Hornik,
et
al.
Hence,
the
estimate
for
the
phase
2
response
rate
is
r3
=
0.85.

The
total
number
of
required
telephone
contacts
is
estimated
as:

N
=
n
/
(
e1
r1
r2
r3
).
6
Using
the
hypothetical
goal
of
n
=
2,641
completed
forms
as
derived
above,
the
required
number
of
initial
telephone
contact
attempts
is
estimated
to
be:

N
=
n
/
(
0.10)(
0.60)(
0.74)(
0.85)
=
2,641
/
0.03774
=
69,979.

If
other
precision
goals
are
desired,
the
required
number
of
completed
forms
shown
in
the
bottom
row
of
Table
1a
or
1b
should
be
divided
by
0.03774
to
estimate
the
required
number
of
contact
attempts.

For
example,
EPA
has
set
a
goal
of
three
percentage
points
at
the
90%
confidence
interval
for
each
of
its
sample
subsets
(
i.
e.,
children
with
asthma
and
low­
income
adults
with
asthma).
For
a
goal
of
three
percentage
points,
the
sample
size
required
at
P
=
50%
is
n
=
827.
To
achieve
this
goal
for
low­
income
populations,
EPA
intends
to
over
sample
in
communities
known
to
have
a
high
percentage
of
low­
income
households.
However,
as
information
identifying
the
nation's
population
of
child
asthmatics
does
not
exist,
EPA
must
increase
the
size
of
its
overall
sample
to
achieve
its
goal
for
children.
EPA
estimates
that
one
in
four
individuals
who
suffer
from
asthma
are
children
(
Mannino).
Therefore,
in
order
to
achieve
a
precision
rate
of
+/­
3
percent
at
the
90%
confidence
interval
the
total
number
of
required
initial
telephone
contact
attempts
is
estimated
as
N
=
n
/
(
0.25)(
0.10)(
0.60)(
0.74)(
0.85)
=
827/
0.009435
=
87,652.

Note:
Appendix
4
provides
a
bibliography
of
cited
references.