Darkness Made Visible:

Field Management and Nonresponse in the 2004 SCF

Arthur B. Kennickell 

Senior Economist and Project Director 

Survey of Consumer Finances 

Board of Governors of the Federal Reserve System 

Mail Stop 153 

Washington, DC 20551 

Voice: 202-452-2247 

Fax: 202-452-5295 

Email: akennickell@frb.gov 

August 12, 2005 

Paper prepared for the 2005 Annual Joint Statistical Meetings, Minneapolis, MN 

The author wishes to thank Leslie Athey, Steven Pedlow, Fritz Scheuren, other NORC central office 

staff, and interviewers and other field staff at NORC for their help in testing and implementing the 

phased sample management model described here. Thanks also to Barry Johnson, Thomas Petska 

and Michael Strudler at SOI and to Brian Bucks, Gerhard Fries, Kevin Moore and Michael Neal at 

the Federal Reserve, without whom this work would not have been possible. The author is 

particularly grateful to the interviewers and field staff who have worked on past years of the SCF 

for the education they have given him in the way that surveys operate in the field. The opinions 

expressed in this paper are the responsibility of the author alone and do not necessarily reflect the 

views of the Board of Governors of the Federal Reserve System. 
Darkness Made Visible: Field Management and Nonresponse in the 2004 SCF 

Abstract 

Nonresponse in field surveys is the joint outcome of the decision of survey staff to apply effort to 

inform and persuade respondents, and the evaluation of such inputs by respondents. In most such 

surveys, the field staff are under great pressure to produce completed interviews. Thus, as discussed 

in Kennickell [2004], they have an incentive to apply effort to cases that are most likely, in their 

view, to be completed with least effort. To the extent that interviewers’ perceptions are unbiased, 

such behavior would tend to amplify latent patterns of nonresponse. When the characteristics of 

respondents that affect the likelihood of participation are correlated with variables of analytical 

interest in the survey, bias results, unless a means can be found of discovering and adjusting for the 

underlying behavioral structures. But, absent constraints on the behavior of interviewers, the 

observed outcomes are contaminated by the endogeneity of effort, and only strong a priori 

assumptions could disentangle the interviewer effects from the respondent effects. To address the 

problem of endogenous effort, the 2004 Survey of Consumer Finances introduced a phased plan of 

sample management to make effort more nearly exogenous through the first two of three phases of 

field work. Thus, nonresponse in these controlled stages should largely reflect respondent 

characteristics, not a mixture of respondent and interviewer characteristics. The dual frame design 

of the SCF offers two classes of sample cases for modeling nonresponse. For the area-probability 

sample, tract-level data are available from the 2000 Census of Population. For the list sample, frame 

case-specific data based on statistical records derived from tax returns are available. For both set 

of cases, some interviewer observations are also available. This paper presents estimates of 

nonresponse models based on these data. 
Nonresponse in field surveys is the joint outcome of the decision of survey staff to apply 

effort to inform and persuade respondents, and the evaluation of such inputs by respondents. In 

most such surveys, the field staff are under great pressure to produce completed interviews. Thus, 

as discussed in Kennickell [2004], they have an incentive to apply effort to cases that are most 

likely, in their view, to be completed with least effort. To the extent that interviewers’ perceptions 

are unbiased, such behavior would tend to amplify latent patterns of nonresponse. When the 

characteristics of respondents that affect the likelihood of participation are correlated with variables 

of analytical interest in the survey, bias results, unless a means can be found of discovering and 

adjusting for the underlying behavioral structures. But, absent constraints on the behavior of 

interviewers, the observed outcomes are contaminated by the endogeneity of effort, and only strong 

a priori assumptions could disentangle the interviewer effects from the respondent effects. 

To address the problem of endogenous effort, the 2004 Survey of Consumer Finances (SCF) 

introduced a phased plan of sample management intended to make effort more nearly exogenous 

through the first two of three phases of field work. As described in more detail later in this paper, 

interviewers were given a flexible protocol intended to ensure that all sample cases were exposed 

to a certain level of effort and that two specific points in the course of the application of effort were 

marked for cases that had not been completed earlier. Those points classify the field operations on 

the cases into at most three phases, the last of which is limited only by the close of field work. If 

the framework holds, then response within the first two phases may be taken as independent of 

behavioral variations in the level of effort applied. 

The first section of the paper provides an overview of the SCF and the approach to 

management of field resources developed for the 2004 survey and describes how this protocol 

worked in practice. The second section presents models of nonresponse in different phases of the 

field effort, conditional on both tract-level and case-level data. The final section summarizes the 

key findings and points toward the next step for the SCF. 
2

I. Background on the SCF and the phased sample management protocol

The SCF is a triennial survey of household finances sponsored by the Federal Reserve Board 

in cooperation with the Statistics of Income Division (SOI) of the Internal Revenue Service.1

 Data 

for the 2004 survey, the basis the analysis presented here, were collected by NORC, a national 

organization for research and computing at the University of Chicago. For this survey, about 47 

percent of the interviews were completed by telephone, though in-person work was a large part of 

the evolution of the great majority of cases over the field period, which ran from June 2004 to 

January 2005. 

The survey employs a dual-frame sample design to select households from across the United 

States. A national area-probability sample (see O’Muircheartaigh et al. [2003]) is intended to give 

sufficiently robust coverage to describe characteristics that are widely distributed in the population, 

such as home ownership and use of credit cards. Using statistical records derived from tax return 

as a frame, a list sample employs stratification by a “wealth index” (see Kennickell [2001]) to oversample wealthy households, who hold a disproportionate share of total wealth and who also tend to 

hold assets less likely to be held by more “typical” households. In the 2004 survey, about a third 

of the final interviews derived from the list sample. All respondents, except those in the wealthiest 

strata of the list sample, were initially offered $20 as thanks for participation in the survey.2 

With the exception of households that did not file a tax return, the area-probability and list 

samples in principle cover the same population.3

 The important differences in the samples are in 

terms of stratification and clustering. The area-probability sample is a multi-stage equal-probability 

design with clustering at the last stage, where the cluster is generally an area equivalent to a census 

1

See Aizcorbe, Kennickell and Moore [2003] for an overview of the survey and the 2001 

data. 

2

The wealthiest list sample respondents were not initially offered money, because it was 

thought that the amount was small relative to their circumstances and that it might trivialize 

participation in their minds. However, if they requested money, a procedure was in place to 

allow them to receive the same amount as other respondents. Later in the field period, 

interviewers were allowed to negotiate payment of a higher amount of money. The effects of 

such payments are ignored here, largely because there is no information on offers made to 

sample nonrespondents. 

3

In 2001, an estimated 12 percent of households did not file a federal individual income 

tax return. 
3

tract. The median number of cases selected into a cluster is five. The list sample is selected using 

the same broad localities selected at the first stage of the area-probability selection, but the units are 

selected without control on location within those areas. Thus, the list sample cases are more likely 

to be dispersed across a broader area than the area cases. 

The survey combines a number of features that one might think would tend to exacerbate 

problems of nonresponse. First, the subject matter is one that is often considered highly private, 

particularly in light of continuing revelations of attempted and actual “identity theft.” Second, 

wealthy households are more likely to have either a staff or physical structures intended to keep 

everyone not explicitly invited by the household away from its private life. Third, wealthy 

households may also be more difficult to locate, because they often have multiple homes and 

because they sometimes take concrete steps to disguise their location. Finally, the survey is 

burdensome, in terms of both the recall needed to answer the questions and the time required for an 

interview. The number of questions asked of a respondent varies with the complexity of the 

household’s portfolio, but in the 2004 survey the median interview length was 82 minutes, while the 

95th percentile of interview length was 165 minutes. 

The 2004 survey contract specified a set of minimum required response rates. An overall 

minimum was specified for the area-probability sample and stratum-specific minima for the list 

sample. These rates were calibrated to be feasible and to maintain comparability with earlier 

surveys. The minimum targets are driven by practical considerations, but they may also have the 

effect of both complicating the management tasks faced by field supervisors and highlighting the 

sample structure in a way that could be deleterious to neutral application of effort. 
4

Table 1: Final case disposition codes, by sample type, Nonresponse in the SCF is a serious 

2004 SCF. 

problem (table 1).4 The overall 2004 

All AP LS 

unweighted response rate was only 51.7 

Not a housing unit 3.9 7.6 * 

Vacant unit 5.0 9.7 * percent overall—68.7 percent in the area 

Sample incorrect 0.4 0.7 0.0 probability sample and 34.7 percent in the Deceased 0.3 * 0.5 

Out of country for field period 6.4 0.0 0.1 list sample. It should be noted that the Complete, telephone 18.2 18.5 17.9 

Complete, in-person 21.2 32.2 9.6 response rate in the list sample varies Conversion, telephone 2.3 2.9 1.7 

Conversion, in person 1.6 2.5 0.6 strongly by sample stratum, with the stratum 

Complete, proxy 0.2 0.2 0.1 

Partially complete 0.1 0.0 0.1 most likely to be very wealthy having a 

Postcard refusal 6.4 * 13.1 response rate of only about 10 percent. Refusal 14.7 15.9 13.3 

Breakoff 0.3 0.1 0.4 However, these overall figures obscure some Refusal, gatekeeper 0.7 0.4 1.2 

Unlocatable 0.5 0.6 0.5 important detail. Unavailable for field period 0.2 0.2 0.1 

Language barrier (non-Spanish) 0.7 0.9 0.4 The category “stopped work” 

Too ill 0.7 0.9 0.4 

Other active nonresponse 0.7 0.8 0.6 accounts for a substantial fraction of the 

Stopped work 22.2 5.9 39.5 

sample, and particularly so for the list 

All 100.0 100.0 100.0 sample. Cases in this group are ones that 

Memo items: were considered still workable as of the end 

Out of scope 15.9 18.0 13.6 

Complete 43.5 56.3 30.0 of the field period. Although clear refusals

Active nonresponse 18.4 19.8 17.0 

Stopped work 22.2 5.9 39.5 account for a sizable fraction of the last 

Response rate 51.7 68.7 34.7 working case dispositions before this final

code, “not home” and “call back/busy” both *: not applicable 

account for a much larger fraction of such 

cases overall (table 2). Indeed, it is only for the list sample case that direct refusals seem to figure 

at all at this stage; however, it may be that the lower overall rate of stopped work cases in the area 

probability sample reflects a systematic tendency to code final refusals in that group rather than 

continuing to work such cases. The important point is that managers felt there was still “life” in all 

4

See Kennickell [2004, 2003, 2002, 1999a, 1999b] and Kennickell and McManus [1993] 

for an overview of nonresponse research on the SCF. 
5

Table 2: Last working disposition for cases with a final 

disposition of “stopped work,” by sample type, 2004 SCF. 

All 

Presumed to be no contact 0.2 

Address observations completed 0.2 

AP LS 

0.0 0.2 

0.0 0.2 

0.0 0.2 

71.1 33.4 

0.0 4.2 

0.0 6.9 

0.0 0.4 

0.0 2.1 

0.0 0.2 

17.8 18.6 

2.2 9.9 

0.0 0.6 

2.2 0.0 

0.0 0.4 

0.0 1.1 

0.0 2.8 

0.0 14.6 

0.0 0.9 

0.0 0.2 

6.7 1.3 

0.0 0.4 

0.0 1.5 

of such cases at the end of the field period, 

and the fact that they remained without a 

final completed or refused code reflects 

decisions that were made about the relative 

payoffs of working other cases. 

Examination of call records in earlier 

waves of the SCF (see Kennickell [2003]) 

indicated that there was a systematic and 

notably nonuniform application of effort by 

interviewers. Moreover, there was evidence 

that this modulation of effort appeared to 

influence the ultimate patterns of 

nonresponse. Additionally, the skewed 

distribution of effort appears at face value to 

be logistically inefficient. That paper 

developed a simple formal model of 

interviewers’ behavior in a survey where 

Deceased 

Not home/No answer 

Inaccessible 

Unocatable (home) 

Unlocatable (business) 

New lead 

New lead failed 

Call back/Unavailable/Busy 

R to call 

Special letter sent 

Language barrier (Spanish) 

Language barrier (not Spanish) 

Screener completed 

Non-respondent refusal 

Refusal 

Hostile refusal 

Express mail package refused 

Broken appointment 

Questionnaire break off 

Completed with incorrect resp. 

All stopped work cases 

0.2 

36.7 

3.9 

6.2 

0.4 

1.9 

0.2 

18.5 

9.3 

0.6 

0.2 

0.4 

1.0 

2.5 

13.3 

0.8 

0.2 

1.7 

0.4 

1.4 

100.0 100.0 100.0 

their rewards are focused only on completion of cases, without regard any other characteristics of 

the cases. The virtually obvious implication of that model is that, all other things being equal, an 

interviewer would attempt to complete cases in the order of their expected subjective likelihood of 

completion. If the characteristics that enter the calculation of interviewers’ expectations are 

correlated with the variables of interest in the survey, then latent patterns of nonresponse bias would 

be amplified. 

Another problem with disproportionate application of effort is an ethical one. If different 

respondents are given different levels of effort to explain why they should participate, then implicitly 

a decision is made that the voices of some respondents are more important than others within their 

sample stratum. This seems unacceptable. 
6

Figure 1: Simplified diagram of the phased sample management protocol, 2004 SCF. 

Initial attempts

 Phase 1 

Completed 

 Phase 1

Completed

Follow-up attempts

 Phase 2 

Completed 

Completed 

Express mailing

 Phase 3: 

Unstructured 

Refused 

To address the variations in effort, the 2004 SCF introduced a phased protocol for the 

application of field effort. Figure 1 provides a simplified schematic of the procedures, and appendix 

figure A1 provides additional schematic detail. 

Underlying the case management protocol is a threshold model of respondents’ behavior. 

Respondent i is assumed to reach an interim decision when the input received from interviewers, 

mailed material and other sources exceeds T1i. The decision may be either to agree to be 

interviewed or to decline participation. In most social science surveys, respondents who decline 

initially are re-contacted in an attempt to secure their cooperation. Upon being re-contacted, a 
7

respondent faces a second decision threshold, T2i, which yields a second decision either to 

participate or refuse. For respondents who refuse at that point, one could specify a sequence of 

possible re-contacts and decisions. The important point in this model is that the first two transitions 

are, in principle, driven by respondent characteristics, not interviewer characteristics. 

The phased contact model for the SCF was designed to have two clear “break points” that 

mark the progress of a case in the application of effort. In the first phase, interviewers were limited 

in the number of attempts they may make to contact and persuade a respondent to participate in the 

survey. Ideally, effort was applied in this phase until either a respondent agreed to participate or the 

respondent crossed a threshold that caused them to refuse. Respondents who could not be contacted 

after repeated attempts at this stage are assumed to have taken actions to isolate themselves that 

make them equivalent to those who directly refused. At this point, a specially designed package of 

materials was sent by express mail to the remaining respondents. This package was designed to 

motivate the project, the role of the respondent, the protections in place for the respondent, the role 

of NORC and the use of the data at the Federal Reserve Board. Although it summarized information 

already available to the interviewer in other materials for use with the respondent as needed, the 

express mailing was organized to present that material in a compact, integrated and attractive way. 

Sending the package by express mail was intended to heighten the sense of importance of the 

respondent and of the material presented. Because interviewers saw this mailing as a powerful 

persuasion tool, the initial fear was that they might tend to minimize the effort leading up to the 

mailing. To ensure that interviewers did not jump to this stage without sufficient initial effort, all 

requests for an express mail package were actually executed by field managers, who were 

responsible for checking interviewers’ efforts to that point. Care was taken to inform the 

interviewers when the package was transmitted and received. For respondents who had not already 

agreed to participate, the sending of this package marked the end of Phase 1.5 

The second phase was intended to be a period of limited follow-up after the express mailing. 

If the respondent declined participation within the allowed period of follow-up, the case was to be 

moved a status requiring further review by the field manager before additional effort was to be 

5

Appendix table A1 shows the distribution of case disposition codes immediately 

preceding the marking of the end of Phase 1. 
8

Table 3: Completion and refusal status of cases for the undertaken. This third and final phase was

three phases of the 2004 SCF field period, percent of 

in-scope cases active in each phase. intended to be left to the discretion of the field 

All AP LS staff, as is the case for the full duration of most 

Postcard refusal 6.3 NA 12.9 field periods. The various phases were marked 

Complete, Phase 1 28.6 42.0 15.2 using a set of case disposition codes in the 

Refused, Phase 1 3.6 2.3 4.7 electronic call records, which are maintained 

Complete, Phase 2 

Refused, Phase 2 

28.1 

39.9 

35.4 

27.2 

22.2 

50.8 for every case. Note that there is no necessary 

Complete, Phase3 

Incomplete, Phase 3 

17.5 

82.5 

27.7 

72.5 

8.0 

92.0 

correspondence in the time across cases at 

which the phase markers might be set; 

variations in staffing levels and exigencies of 

individual cases might well cause the time to vary greatly. 

Because the necessary actions of the interviewer are much too complicated to specify in 

precise detail a priori and because effort is inherently difficult to measure unambiguously, the case 

management protocol was established as a set of monitored guidelines, which managers were 

allowed to violate in light of the idiosyncracies of individual cases. At the point of interviewer 

training, both interviewers and their managers expressed enthusiasm for the protocol. During the 

field period, regular reports were generated for the field managers showing cases that appeared to 

have violated the guidelines, but the managers had the ultimate responsibility for monitoring. 

Table 3 shows the percent of cases active in each phase that terminated within the phase. 

Overall, more than a quarter of all in-scope cases were completed within Phase 1. Although cases 

coded as final refusals in Phase 1 were supposed to have been moved immediately to Phase 2, 

information from the field supports the view that some cases refused so strongly that recontact was 

not a possibility. For further analysis here, the relatively small number of such cases are taken to 

have passed two threshold points and refused in Phase 3. The much higher final refusal rate 

recorded in Phase 2 indicates a serious problem with the way the protocol was followed beyond the 

first phase. By design, all cases that refused in Phase 2 should have been marked immediately for 

inclusion in Phase 3; such cases might have been treated as final refusals once that assignment was 

made, but only the most extreme refusals should have been accepted without the assignment of a 

case disposition code signifying the end of Phase 2. 
9

Examination of the number of attempts recorded in the call records indicates that there was 

substantial variation in the way the markers of the sample phases were observed by the field 

managers.6

 As shown in table 4, although the great majority of the activity in Phase 1 was contained 

within the first ten attempts, there are still a fair number of cases with larger numbers of attempts. 

For Phase 2 (table 5), the spread in the number of attempts is much broader. Several issues appear 

to be at the root of this unanticipated deviation from the sample management plan. First, the 

perception of an “attempt” that was sufficient to count toward progress within a phase has a 

subjective component and the information stored in the call records is not always sufficient to 

recapture that judgment directly. Thus, the available measure probably overstates the number of 

attempts that managers would have counted as meaningful. Second, Phase 2 lacked the compelling 

incentive present in marking the end of Phase 1—that is, the express mailing package. Second, the 

protocol was new both for the 2004 SCF and for NORC. For this reason, it is reasonable to assume 

Table 4: Number of attempts at which cases were completed or refused before the express mailing, or number 

attempts at the time of the express mailing; both samples, area-probability sample only and list sample only; 

percent of cases in each completion status and sample type. 

x attempts All 

Comp. 

1 9.9 

2 5.8 

3 8.4 

4 10.1 

5 15.2 

6 13.8 

7 9.3 

8 8.0 

9 4.8 

10 4.4 

11 2.5 

12 1.7 

13 1.6 

14 0.7 

15 0.8 

>15 3.2 

Ref. 

3.6 

4.6 

13.7 

11.4 

10.8 

8.8 

9.5 

6.2 

8.1 

6.2 

3.6 

2.0 

1.0 

1.3 

2.0 

7.5 

Express 

10.0 

11.8 

15.5 

16.6 

14.1 

9.3 

6.4 

4.7 

3.2 

2.4 

1.8 

1.1 

0.8 

0.7 

0.4 

1.3 

AP 

Comp. 

11.7 

6.4 

8.9 

10.9 

15.5 

14.2 

9.0 

7.2 

4.3 

3.8 

2.1 

1.4 

1.3 

0.5 

0.6 

2.2 

Ref. 

4.0 

7.0 

16.0 

16.0 

14.0 

11.0 

10.0 

8.0 

6.0 

1.0 

2.0 

1.0 

0.8 

1.0 

1.0 

2.0 

Express 

7.5 

10.4 

15.8 

15.6 

13.8 

9.9 

7.3 

5.7 

3.2 

2.7 

2.3 

1.7 

2.6 

1.1 

0.5 

1.8 

LS 

Comp. 

5.0 

4.1 

6.8 

8.0 

14.3 

12.8 

10.1 

10.0 

6.2 

5.9 

3.5 

2.6 

1.5 

1.1 

1.4 

5.9 

Ref. Express 

3.4 11.8 

3.4 12.8 

12.6 15.2 

9.2 17.2 

9.2 14.4 

7.7 8.9 

9.2 5.7 

5.3 4.0 

9.2 3.2 

8.7 2.1 

4.4 1.5 

2.4 0.7 

0.8 0.0 

1.5 0.3 

2.4 0.4 

10.1 1.0 

6

A small number of cases had obvious violations of the protocol. For example, in some 

cases an express mail package was sent after an interview had already been completed. Such 

problems are ignored in the analysis presented here. 
10 

Table 5: Number of attempts beyond Phase 1 at which cases were completed or refused before the end of 

Phase 2, or number attempts beyond Phase 1 at point when the end of Phase 2 was coded; both samples, areaprobability sample only and list sample only; percent of cases in each completion status and sample type. 

# attempts All 

Comp. 

1 0.1 

2 0.5 

3 1.9 

4 1.9 

5 3.9 

6 5.0 

7 6.4 

8 8.0 

9 8.2 

10 8.5 

11 7.3 

12 7.6 

13 5.5 

14 5.7 

15 5.7 

>15 23.9 

Ref. 

5.2 

9.7 

11.0 

9.9 

8.7 

8.0 

7.8 

6.7 

5.3 

5.2 

4.0 

3.0 

2.6 

1.85 

1.8 

9.3 

Phase 2 

20.8 

13.4 

15.9 

12.9 

7.6 

6.9 

6.0 

4 

2.9 

2.4 

1.5 

1.2 

0.5 

1.0 

0.7 

2.2 

AP 

Comp. 

0.1 

0.7 

1.7 

1.8 

3.6 

4.7 

6.5 

7.6 

8.5 

8.7 

7.4 

6.8 

6.4 

5.6 

6.6 

23.3 

Ref. 

3.3 

10.6 

13.7 

12.0 

8.6 

9.2 

6.9 

5.7 

5.5 

4.1 

2.9 

2.3 

2.5 

2.1 

1.7 

9.0 

Phase 2 

20.5 

14.2 

15.9 

12.8 

6.95 

7.51 

4.3 

4.3 

2.4 

2.8 

1.8 

1.2 

0.7 

1.1 

0.7 

2.9 

LS 

Comp. 

0.0 

0.3 

2.0 

1.9 

4.3 

5.4 

6.4 

8.4 

7.9 

8.2 

7.2 

8.6 

4.5 

5.8 

4.73 

24.6 

Ref. Phase 2 

6.0 21.1 

9.3 12.5 

10.1 15.9 

9.1 13.0 

8.8 8.2 

7.5 6.4 

8.1 7.7 

7.1 3.7 

5.2 3.3 

5.6 2.0 

4.4 1.2 

3.3 1.2 

2.7 0.4 

1.6 0.9 

1.8 0.7 

9.5 1.6 

there might have been some initial confusion about how to proceed. Indeed, from debriefing the 

field managers, it appeared that some people saw the act of coding the end of Phase 2 as somehow 

limiting their ability to work further on such cases. In fact, marking the end of Phase 2 only should 

only have provided a focus for a formal reevaluation of a case for the usefulness of additional work. 

Although the available Phase 1 marker may be sufficient for the original purposes of 

partitioning cases by levels of effort, the Phase 2 marker is clearly inadequate. For further analysis, 

two alternative markers are used along with the Phase 1 indicator. The point at which the sooner 

of either the end of Phase 1 had been reached or ten attempts had been undertaken is defined as 

Phase 1A. Similarly, the point at which the sooner of either the end of Phase 2 had been reached 

or ten attempts beyond Phase 1A had been undertaken is defined as Phase 2A. Phase3A is defined 

as the period beyond Phase 2A. 
11

II. Models of nonresponse

A variety of information is available on both respondents and nonrespondents to support a 

model-based investigation of nonresponse. The census tract for the residence of every areaprobability case is known and, in principle, that for the list sample may be inferred from a nine-digit 

ZIP code that is available for all list sample cases. The tract identifier may be used to link the 

survey cases with tract-level statistics from the 2000 Census of Population. In some instances, this 

information may serve as a noisy indicator of respondents’ characteristics, but in other instances 

reflect structural characteristics of their neighborhoods. The variables extracted (see “Variable 

definitions for tables 6, 7 and 8") include indicators of population density, ethnic and racial 

composition, presence of exclusively non-English speakers, the age and income distributions, 

educational levels, work status, commuting time, housing type and vintage, typical rent and the 

prevalence of telephones. These variables are intended to span a variety of economic, cultural and 

other factors differences that might reasonably affect the propensity to respond. 

The linkage to census tracts appears very clean for the area-probability sample. The list 

sample raises more problems. Each ZIP code for the list sample cases was supposed to be that of 

the residence of the taxpayer selected for the sample. But as has been clear throughout the history 

of the SCF, some tax returns are filed from a business address or through an accountant or lawyer. 

Unfortunately, we have no systematic information to identify the cases that did not have residential 

addresses.7

 In addition, seven list sample cases were matched to tracts that had insufficient 

population to compute the variables used in the models. 

Other data are available for the list sample from the tax-based information used in the 

original sample design (see “Variables definitions for tables 7, 8 and 9"). Unlike the tract-level 

variables, this information is specific to each of the sample observations. The variables examined 

include the age of the primary taxpayer, sources and amounts of several types of income and 

information on itemized deductions. 

7

Presumably, the detailed address information maintained by interviewers would have 

contained such information, but that is not available for any purpose other than the 

administration of the survey. Moreover, such information is systematically destroyed at the end 

of the project. 
12

Interviewers were also required to collect some information on the informant who served for 

the initial screening and on characteristics of the sample address. Because the informant-level data 

are missing for about 40 percent of the 2004 sample, they are not useful for this nonresponse study. 

The address characteristics have less serious problems of missing data. Overall, 890 of the in-scope 

cases worked by the interviewers had missing information on the key variables describing physical 

limitations to contacting the respondent directly—presence of a doorman or guard, a locked gate, 

etc. Almost 60 percent of these cases were ones that received a final disposition code indicating that 

work had stopped and almost 30 percent were given the final disposition “final refusal.” All but 99 

of the worked cases with missing data were members of the list sample. In addition, because the 665 

list sample cases that returned a refusal postcard were never seen by an interviewer, information 

about the addresses is not available. Thus, results using these data for the list sample should be 

interpreted with caution. 

The area-probability and list samples are modeled separately here. Although both samples 

cover very similar populations (as noted earlier, the list sample does not include households that did 

not file a tax return), the differences in the dispersion of the samples, the stratification of the list 

sample to obtain more wealthy households, and the approach field managers and interviewers may 

have taken to the samples argue for not estimating a pooled model. In addition, there are important 

differences in the data available for the two samples beyond the common tract-level data. 

For the dependent variables shown in table 6, a series of six logit models was estimated for 

the area-probability sample (table 7). The models are structured to show the incremental effects of 

additional effort in the phases described above and to show the net effect across all phases. The first 

five models use the tract-level data along with the interviewer observations on obstructions to 

contacting the respondent. An additional model is shown for the final dependent variable, 

completion during any phase of the field period, without the interviewer observations in order to 

show the incremental effect of this information. 

The two models of overall response (columns 5 and 6 of the table) are very similar. There 

are significant regional effects–higher response in the eastern north central region and lower 

response in the mid-Atlantic region–that may signal characteristics of “typical” residents or 
13 

Table 6: Dependent variables and sample definitions for logit models shows in tables 7, 8 and 9. 

Dependent variable Sample 

Area-probability sample models 

PHASE1 

PHASE1A 

PHASE2A 

PHASE3A 

COMP 

=1 if case completed before the express mailing 

=1 if completed before the express mailing or 

mailing or 10 attempts, whichever came first 

=1 if completed before 10 attempts after express 

mailing or 20 attempts, whichever came first 

=1 if completed after 10 attempts after the express 

mailing or 20 attempts, whichever came first 

=1 if cases completed at any point 

All in-scope cases. 

All in-scope cases. 

All in-scope cases where PHASE1A=0 

All in-scope cases where PHASE2A=0 

All in-scope cases 

List sample models 

POSTCARD 

PHASE1A 

PHASE2A 

PHASE3A 

COMP 

COMPXPC 

=1 if refusal postcard not returned 

=1 if completed before the express mailing or 

mailing or 10 attempts, whichever came first 

=1 if completed before 10 attempts after express 

mailing or 20 attempts, whichever came first 

=1 if completed after 10 attempts after the express 

mailing or 20 attempts, whichever came first 

=1 if completed at any point 

=1 if completed at any point 

All in-scope cases. 

All in-scope cases where 

POSTCARD=1 

All in-scope cases where 

POSTCARD=1 and PHASE1A=0 

All in-scope cases where 

POSTCARD=1 and PHASE2A=0 

All in-scope cases 

All cases in-scope cases where 

POSTCARD=1 

differences in field management styles in these areas.8 Cases in neighborhoods with relatively high 

proportions of African-Americans, of people under the age of 18 and of workers who had relatively 

short commutes were more likely to cooperate. The association with commuting time has a natural 

economic interpretation: those with long commutes have less spare time, and thus should place a 

higher value on that time than would otherwise be the case. Response tends to be less likely in 

neighborhoods with higher proportions of people who have less than a high-school education. The 

access limitation variables included in model 5 indicate that respondents in housing units with a 

guard or doorman or where a “no trespassing” sign has been posted are less likely to be interviewed; 

simply living in a unit with a locked gate or lobby door appears to be unrelated to response. The 

effects of the tract-level variables discussed above do not change substantially when the access 

8

Because areas were reassigned, sometimes several times, over the course of the field 

period, it would be very difficult to extract a manager-specific effect from the data. Similarly, 

because cases were often reassigned to different interviewers or were worked on by other staff, 

interviewer-specific effects cannot be identified easily. 
14

limitation effects are omitted, but there are other differences. In model 6, without the effects, 

population density has a significant negative effect; in the other model the effect is still negative, 

but not significant. Model 5, with the access effects included, shows additional positive significant 

effects of residence in the south Atlantic region and the percent of occupied housing units in the 

census tract, and negative effects from the proportion of households with a telephone and the 

proportion of townhouses in the tract. Because telephone coverage is so high (in 2001, 94.5 percent 

of all household had a telephone), it may be best to think of the positive effect on response of low 

telephone coverage 

The overall response patterns are a result of outcomes in the separate phases of the field 

work. Phase 1 is the least complicated point at which to view the effects of the variables on 

Table 7: Logit models of response for various phases of the field period, area-probability sample. 

(1) (2) (3) (4) (5) (6) P_LTHS -0.006 -0.007 -0.005 -0.041+ -0.018+ -0.017+ 

0.010 0.010 0.014 0.025 0.011 0.010 

PHASE1 PHASE1A PHASE2A PHASE3A COMP COMP P_INC_LT10 -0.001 0.006 0.006 0.025 0.013 0.015 

0.010 0.010 0.015 0.029 0.012 0.012 

Intercept 2.418 0.413 -2.584 -4.281 -0.216 0.976 P_INC_75_150 -0.005 -0.009 0.013 -0.012 -0.001 0.000 

1.982 1.969 2.786 5.322 2.128 2.067 0.007 0.007 0.009 0.018 0.007 0.007 

NON_MSA -0.002 0.092 0.036 -0.210 0.032 0.041 P_INC_GE150 -0.031# -0.015 0.018 0.005 0.000 0.006 

0.058 0.058 0.081 0.148 0.063 0.061 0.010 0.009 0.012 0.024 0.010 0.009 

SM_MSA 0.004 -0.081 -0.067 -0.173 -0.112 -0.087 P_WORKERS 0.010 0.036# -0.015 -0.008 0.014 0.008 

0.075 0.075 0.109 0.204 0.081 0.080 0.011 0.011 0.016 0.032 0.012 0.012 

NEW_ENGLAND -0.054 -0.368* -0.370+ 0.735* -0.274 -0.224 P_UNEMP -0.003 0.006 -0.001 -0.053 -0.001 -0.007 

0.163 0.165 0.226 0.352 0.170 0.168 0.014 0.014 0.020 0.044 0.015 0.015 

MID_ATLANTIC -0.243* -0.030 -0.287+ -0.236 -0.258* -0.274# P_COMMUT_LT25 0.006 0.011# 0.009 -0.005 0.010* 0.008+ 

0.112 0.110 0.150 0.292 0.115 0.113 0.004 0.004 0.006 0.011 0.005 0.005 

S_ATLANTIC 0.094 0.011 0.079 0.589# 0.152+ 0.081 P_COMMUT_GE45 0.010+ 0.022# 0.006 -0.038* 0.010 0.007 

0.086 0.085 0.118 0.219 0.093 0.089 0.006 0.006 0.009 0.017 0.007 0.006 

E_S_CENTRAL -0.184 -0.333* 0.318 0.704+ 0.016 -0.039 P_OCC_HOU 0.026 0.022 0.014 0.168# 0.051* 0.035 

0.153 0.154 0.212 0.404 0.166 0.162 0.022 0.022 0.034 0.060 0.027 0.025 

W_S_CENTRAL 0.251* 0.416# 0.450# -1.229# 0.383# 0.412# P_OWNOCC -0.002 -0.004 -0.007 -0.011 -0.006 -0.008 

0.112 0.113 0.168 0.460 0.134 0.130 0.006 0.006 0.008 0.014 0.006 0.006 

E_N_CENTRAL -0.066 0.034 0.061 -0.180 0.011 0.010 P_HOU_1ATT_UNIT -0.010# -0.009# -0.001 0.000 -0.007* -0.004 

0.087 0.086 0.121 0.241 0.094 0.092 0.004 0.003 0.004 0.008 0.003 0.003 

W_N_CENTRAL -0.267* -0.254* 0.160 0.113 -0.066 -0.066 P_HOU_2_4_UNIT 0.009+ 0.008+ 0.002 -0.006 0.009 0.008 

0.128 0.127 0.170 0.306 0.134 0.132 0.005 0.005 0.008 0.014 0.006 0.006 

MOUNT_PACIFIC 0.286* 0.326# -0.070 -0.536 0.079 0.144 P_HOU_5_49_UNIT -0.005 -0.007 -0.005 0.001 -0.007 -0.007 

0.131 0.132 0.196 0.379 0.145 0.143 0.005 0.005 0.007 0.012 0.005 0.005 

POP_DENSITY 0.002 -0.003 -0.006 0.003 -0.004 -0.007* P_HOU_GE50_UNIT -0.005 0.000 0.001 0.004 0.002 -0.001 

0.004 0.004 0.005 0.009 0.004 0.003 0.005 0.005 0.007 0.015 0.006 0.005 

P_NATIVE_BORN 0.000 0.001 0.002 0.006 0.002 -0.001 P_HOU_LE1939 -0.003 -0.003 0.004 0.010 0.002 0.003 

0.009 0.009 0.013 0.024 0.010 0.010 0.004 0.003 0.005 0.009 0.004 0.004 

P_HISP 0.010 0.006 -0.016 -0.014 -0.012 -0.011 P_HOU_1940_1959 0.001 0.000 -0.003 0.004 0.000 0.000 

0.015 0.014 0.020 0.037 0.016 0.016 0.003 0.003 0.004 0.008 0.003 0.003 

P_AFAM 0.002 0.002 0.007* -0.005 0.005+ 0.004+ P_HOU_1960_1989 -0.001 -0.001 0.000 0.008 0.001 0.001 

0.002 0.002 0.003 0.007 0.003 0.003 0.003 0.003 0.004 0.007 0.003 0.003 

P_ASIA -0.024# 0.010 -0.001 -0.021 0.005 0.010 L_MED_RENT -0.424* -0.368* 0.327 -0.072 -0.025 -0.140 

0.008 0.007 0.010 0.021 0.008 0.007 0.189 0.186 0.250 0.455 0.195 0.190 

P_RACE_OTH -0.059# -0.013 0.002 -0.085 -0.013 -0.015 P_PHONE -0.038+ -0.038+ -0.004 -0.141# -0.054* -0.034 

0.022 0.020 0.028 0.093 0.022 0.022 0.021 0.021 0.032 0.055 0.026 0.025 

P_SPONLY_SPAN -0.008 -0.004 0.027 0.040 0.026 0.021 GUARD -0.072 0.457+ -1.538# -1.776+ -0.481* . 

0.018 0.018 0.026 0.047 0.020 0.020 0.251 0.242 0.493 1.050 0.243 . 

P_SPONLY_OTH 0.016 -0.007 0.006 0.024 0.000 -0.005 LOCKED_LOBBY -0.193 -0.167 -0.031 -0.661 -0.237 . 

0.011 0.011 0.015 0.029 0.012 0.011 0.165 0.161 0.219 0.435 0.166 . 

P_AGE_LT18 0.026* 0.045# -0.024 0.060+ 0.028* 0.025* LOCKED_GATE -0.311 -0.349+ 0.170 0.132 -0.111 . 

0.012 0.012 0.016 0.032 0.013 0.013 0.195 0.195 0.248 0.459 0.205 . 

P_AGE_GE65 0.001 0.018 -0.027+ 0.005 0.001 -0.005 NO_TRESSPASS -0.500+ -0.589* -0.630+ -0.526 -0.753# . 

0.011 0.011 0.016 0.032 0.012 0.012 0.262 0.263 0.347 0.605 0.249 . . 

P_GEBA 0.010 0.009 -0.014 -0.012 -0.005 -0.007 . 

0.007 0.007 0.009 0.017 0.007 0.007 N 4271 4271 2295 1459 4271 4367 

P_SOMCOLL -0.011 -0.024# 0.008 0.012 -0.011 -0.009 

0.010 0.010 0.014 0.025 0.010 0.010 #: <=1%, *: <=5%, +: <=10% 
15

nonresponse. Unlike the overall response model, it is (in principle) uncontaminated by behavioral 

variations in the application of effort. In addition, the dividing point for the period is less ambiguous 

than is the case for the later phases. In the initial phase—according to either the Phase 1 or 

Phase 1A marker—response is positively associated with residence in the west south central or 

mountain pacific region, living in neighborhoods with higher proportions of people under the age 

of 18, higher concentrations of buildings with two to four units, and higher proportions of workers 

with long commutes to work. Response is negatively associated with residence in the west north 

central region, higher proportions of townhouses, a higher coverage rate for telephones, higher 

median levels of rent and the presence of a “no trespassing” sign at the sample address. The positive 

effect of the fraction of workers with long commuting times, particularly contrasted with positive 

effect of short commuting time in the overall response model, seems strange if it is taken as 

reflecting respondent characteristics. One possibility is that neighborhoods with high fractions of 

workers with long commutes also contain other people who for some reason are more willing to 

participate, but this smaller part of the pool of eligible respondents would have been exhausted early 

on. Of the effects significant in these two models, only two positive effects (residence in the west 

south central region and the proportion of people aged less than 18) and three negative effects (the 

proportion of townhouses, the coverage rate of telephones, and presence of a “no trespassing” sign) 

are also significant and of the same sign in the overall response model. For the straightforwardly 

defined Phase 1 model, several other effects are significant and all negative in their influence on 

response propensity: residence in the mid-Atlantic region, the proportion of people in the tract who 

are Asian or “other” race and the proportion of households with incomes of $150,000 or more; of 

these only the mid-Atlantic effect carries through to the final model. 

Generally over the course of the remaining phases, fewer variables are significant in the 

models and none of the initial effects are consistently sustained. In Phase 3A, where the application 

of effort was largely determined by the judgment of the field staff about where effort would be most 

likely to yield completed interviews, a largely different pattern of significant estimates emerges, 

some of which are the reverse in sign of the corresponding estimates for the first phase. 

Overall, the break-out of response propensities over phases of the field period is hard to 

interpret, though the results do at least indicate that some sort of behaviorally-based selection 

process took place. Because the tract-level data are neighborhood characteristics, not respondent 
16

characteristics, we can only guess at whether the observed effects are driven by neighborhood 

context or by the degree to which the respondents tend to share the neighborhood characteristics–or 

both. For example, we cannot tell whether the decline from Phase 1 in the importance of living in 

a high-income neighborhood as a driver of nonresponse is a result of a filtering through all income 

levels of respondents in such neighborhoods or whether individual high-income respondents tend 

to become more cooperative after receiving more information. In essence, we do not know the 

initial correspondence between respondents and their neighborhood characteristics or how that 

correlation changes among nonrespondents remaining at various points in the field period. To 

discriminate more clearly between these alternatives, we need either reliable interviewer 

observations on a broad array of characteristics observable for all respondents or external 

information on all respondents. As noted above, most of the interviewer-collected observations for 

the sample are too riddled with missing data to be useful for this exercise. But for the list sample 

we do have a limited amount of information specific to each selected case. 

If we are to apply inferences from the list sample to the area-probability sample as well, it 

would be helpful if the pattern of association with the tract-level variables were similar in the two 

samples. A number of factors might limit this possibility. First, as noted above, the match to tractlevel data is somewhat less certain for the list sample than for the area-probability sample. If the 

likelihood of having a business address is about the same for participants and non-participants, this 

would only make the estimates noisier. Second, the list sample is differentially sampled by an 

indicator of wealth. Controlling separately for the sample stratum should diminish distortions from 

this source, but such conditioning could offset some other effects that have a latent correlation with 

wealth; excluding the wealthiest cases might be a useful robustness check, but the smaller sample 

size would diminish the power. Third, the list sample does not include households where no one 

filed a tax return. Such households tend to be those that have very little labor income—this group 

primarily comprises people receiving some type of public assistance, people whose sole income 

source is social security, and those who are avoiding or evading taxes. Fourth, the list sample cases 

were given a chance to refuse participation absolutely by returning a postcard; no follow-up 

persuasion was allowed for such cases. Clearly this allowance might well influence the phasespecific models, but unless the postcard refusal cases are unlike area-probability cases that persisted 

in refusals through Phase 3, there should be no effect on an overall model of response. Fifth, 
17

although there is some clustering in the list sample, that sample tends to be much more thinly spread 

than the area-probability sample. Consequently it may have been difficult to work with as great 

efficiency early in the field period. If there are aspects of effort not captured in the measures used 

to define the sample phases, then at least the phase-specific models could differ for this reason. 

Sixth, the contract for the survey specified a minimum number of cases that must be completed in 

each list sample stratum. These levels, which were chosen based on the experience of what had been 

seen as feasible in earlier surveys, were intended to ensure that every stratum would receive 

attention. The data suggest that list sample cases that did not return the refusal postcard were 

worked slightly harder than the area-probability cases, but it may still be that this difference reflects 

a higher average level of difficulty for the list sample.9 

To explore the comparability of the response propensities for the area-probability and list 

samples, table 8 shows estimates of models for the list sample of overall completion (columns 1 

and 3) and for overall completion beyond the postcard stage (columns 2, 4 and 5), using the tractlevel variables and with (columns 3, 4 and 5) and without (columns 1 and 2) indicators for the list 

sample strata and with (column 5) and without (columns 1, 2, 3 and 4) the access limitation 

variables. This combination of models is intended to give a sense of the robustness of the models 

to the specification. A number of relationships are common or nearly so across these models, but 

the correspondence with the overall response models for the area-probability sample is weak. In one 

of the five list sample models (column 5) population density of the census tract was associated with 

a higher response propensity, in two models (columns 1 and 2) the proportion of people in the 

census tract younger than age 18 had a positive effect, and in one model (column 5) the presence of 

a guard or doorman has a negative effect on response. If the list sample included in the estimation 

is restricted to the least wealthy bottom four sample strata (about 40 percent of that sample), the 

significant effects for the full list sample are generally sustained (not shown in the table). Thus, 

while some part of the differences in the estimates for the two samples may still be related to the 

oversampling in the list sample, other factors including differences in the ways the samples were 

9

Overall, the mean number of attempts made on a case was 11.5, the median was 10 and 

the 95th percentile of the distribution was 24. For the area-probability sample, the figures were 

11.1, 9 and 24; for the list sample, they were 12.0, 11 and 24. 
18 

Table 8: Logit models for overall response and overall response after the postcard stage, list sample. 

(1) (2) (3) (4) (5) P_INC_GE150 -0.013# -0.017# -0.006 -0.009+ -0.005 

0.005 0.005 0.005 0.005 0.006 

COMP COMPXP COMP COMPXP COMPXP P_WORKERS 0.009 0.012 0.009 0.013 0.024* 

0.009 0.009 0.009 0.009 0.010 

Intercept -2.981 -2.975 -2.866 -2.869 -2.975 P_UNEMP 0.013 0.014 0.013 0.014 0.029* 

1.869 1.918 1.890 1.949 2.113 0.012 0.012 0.012 0.012 0.014 

NON_MSA -0.037 -0.031 -0.046 -0.038 0.010 P_COMMUT_LT25 0.003 0.004 0.005 0.007 0.007 

0.097 0.100 0.098 0.101 0.107 0.004 0.004 0.004 0.005 0.005 

SM_MSA -0.019 -0.094 -0.011 -0.086 -0.113 P_COMMUT_GE45 0.006 0.007 0.007 0.009 0.013* 

0.158 0.162 0.159 0.164 0.175 0.006 0.006 0.006 0.006 0.006 

NEW_ENGLAND 0.075 0.083 0.080 0.102 0.274+ P_OCC_HOU 0.047 0.029 0.050 0.032 0.040 

0.131 0.136 0.132 0.138 0.149 0.031 0.032 0.031 0.032 0.036 

MID_ATLANTIC -0.002 -0.016 -0.009 -0.031 0.102 P_OWNOCC -0.006 -0.004 -0.007 -0.005 -0.006 

0.100 0.104 0.101 0.106 0.113 0.005 0.005 0.005 0.005 0.005 

S_ATLANTIC -0.002 0.004 -0.014 -0.012 -0.003 P_HOU_1ATT_UNIT 0.003 0.002 0.002 0.001 -0.001 

0.091 0.094 0.092 0.096 0.102 0.004 0.004 0.004 0.004 0.005 

E_S_CENTRAL 0.161 0.064 0.150 0.052 -0.107 P_HOU_2_4_UNIT 0.003 0.004 0.001 0.001 -0.003 

0.221 0.227 0.224 0.231 0.238 0.005 0.005 0.005 0.005 0.006 

W_S_CENTRAL -0.122 -0.074 -0.146 -0.119 -0.110 P_HOU_5_49_UNIT -0.012# -0.012# -0.011# -0.011# -0.011* 

0.122 0.127 0.123 0.128 0.137 0.004 0.004 0.004 0.004 0.005 

E_N_CENTRAL 0.012 0.025 0.002 0.003 -0.188+ P_HOU_GE50_UNIT -0.011# -0.011# -0.010# -0.010# -0.011# 

0.093 0.098 0.094 0.099 0.102 0.004 0.004 0.004 0.004 0.004 

W_N_CENTRAL -0.075 -0.024 -0.073 -0.003 -0.001 P_HOU_LE1939 -0.006* -0.007* -0.005+ -0.006+ -0.008# 

0.155 0.162 0.156 0.164 0.174 0.003 0.003 0.003 0.003 0.003 

MOUNT_PACIFIC 0.052 0.004 0.096 0.059 0.088 P_HOU_1940_1959 -0.006# -0.007# -0.006* -0.006* -0.009# 

0.153 0.158 0.155 0.161 0.173 0.003 0.003 0.003 0.003 0.003 

POP_DENSITY 0.004+ 0.005* 0.002 0.003 0.008# P_HOU_1960_1989 -0.002 -0.003 -0.002 -0.003 -0.005+ 

0.002 0.002 0.002 0.003 0.003 0.002 0.002 0.002 0.002 0.003 

P_NATIVE_BORN 0.000 -0.002 0.000 -0.003 -0.004 L_MED_RENT -0.012 -0.001 -0.036 -0.035 0.122 

0.010 0.010 0.010 0.011 0.011 0.128 0.131 0.129 0.133 0.142 

P_HISP 0.023 0.019 0.023 0.018 -0.008 P_PHONE -0.030 -0.010 -0.035 -0.016 -0.032 

0.014 0.015 0.015 0.015 0.017 0.031 0.031 0.031 0.032 0.036 

P_AFAM 0.006+ 0.006 0.006 0.005 0.004 GUARD . . . . -0.429# 

0.003 0.004 0.003 0.004 0.004 . . . . 0.150 

P_ASIA 0.013+ 0.015* 0.010 0.012 0.009 LOCKED_LOBBY . . . . -0.265 

0.007 0.007 0.007 0.008 0.008 . . . . 0.200 

P_RACE_OTH 0.014 0.004 0.014 0.002 0.008 LOCKED_GATE . . . . -0.520# 

0.032 0.033 0.032 0.034 0.036 . . . . 0.134 

P_SPONLY_SPAN -0.022 -0.021 -0.022 -0.020 0.013 NO_TRESSPASS . . . . 0.016 

0.018 0.019 0.018 0.019 0.021 . . . . 0.226 

P_SPONLY_OTH -0.013 -0.018 -0.011 -0.017 -0.014 LS_STRAT7 . . -1.002 -1.103 -0.932 

0.011 0.012 0.011 0.012 0.013 . . 0.135 0.137 0.145 

P_AGE_LT18 0.018+ 0.022* 0.015 0.018 0.016 LS_STRAT6 . . -0.252# -0.308 -0.202# 

0.011 0.011 0.011 0.011 0.012 . . 0.067 0.069 0.074 

P_AGE_GE65 0.022* 0.026# 0.021* 0.025# 0.029# LS_STRAT5 . . 0.035 0.008 0.003 

0.010 0.010 0.010 0.010 0.011 . . 0.071 0.073 0.078 

P_GEBA 0.011 0.010 0.012+ 0.012+ 0.007 LS_STRAT4 . . 0.212# 0.217# 0.204* 

0.007 0.007 0.007 0.007 0.008 . . 0.076 0.079 0.085 

P_SOMCOLL 0.000 -0.002 0.001 -0.001 -0.001 LS_STRAT3 . . 0.304# 0.368 0.333# 

0.011 0.011 0.011 0.012 0.013 . . 0.084 0.090 0.096 

P_LTHS -0.008 -0.006 -0.006 -0.003 -0.003 LS_STRAT2 . . 0.520 0.660 0.503 

0.012 0.013 0.012 0.013 0.014 . . 0.098 0.106 0.112 . 

P_INC_LT10 0.002 0.003 0.004 0.004 0.003 

0.011 0.011 0.011 0.011 0.013 N 4997 4336 4997 4336 3526 

P_INC_75_150 -0.010+ -0.013* -0.010+ -0.013* -0.013* 

0.006 0.006 0.006 0.006 0.007 #: p<=1%, *: p<=5%, +: p<=10% 

worked seem more likely to be part of the explanation. 

The absence of strong parallel findings in the two samples for a parallel set of variables 

limits our ability to draw general conclusions for both sample. Nonetheless, response propensities 

for the list sample are interesting in their own right and examination of the effects of including 

observation-level variables in response propensity models may still suggest latent sources of 

response bias for both samples. Table 9 provides estimates of a set of response propensity models 

for the postcard stage (column 1), the subsequent three sample phases (columns 2, 3, 4 and 5), the 

overall period beyond the postcard stage (column 6), and the full overall period including the 

postcard stage (column 7). The explanatory variables used are the tract-level variables, indicators 
19

for the list sample strata, and the case-level variables used in the sample design. Because of the 

relatively high fraction of missing information on access limitations for this sample, that information 

is not included in the models. 

The overall model (column 7) shows a variety of significant effects involving both the tractlevel variables and the case-specific variables. Among the tract-level variables, higher proportions 

of people who are African-American, Hispanic, aged younger than 18 or aged 65 and older, and 

residence in neighborhoods with higher proportions of occupied housing units are positively 

associated with response. Residence in neighborhoods with higher proportions of housing units in 

buildings with five or more units and residence in buildings built before 1960 are negatively 

associated with response.10 There are no significant geographic factors. Comparison of these model 

estimates with those shown in table 8 indicates that the tract-level characteristics are largely 

independent of the case-specific variables.11 Response propensity is quadratic in age of the primary 

taxpayer, increasing until age 59 and then declining–reaching the equivalent effect of age 40 at about 

age 80. Income characteristics of the unit have strong effects on the response propensity. The 

amounts of total income and total financial income (the sum of dividends and taxable and nontaxable 

interest incomes) are both negative in their influence as is receipt of income from rents or royalties. 

At the same time, receipt of two of the components—nontaxable interest and dividends—have a 

positive effect; for a person having all three types, the estimated coefficients imply that the positive 

effect is not offset until the level of financial income reaches $178,000. The level of itemized 

deductions for charitable contributions also has a positive effect; this result is consistent with a role 

for altruism in the decision to respond. Both the result for financial income and that for charitable 

contributions were reported in an overall nonresponse analysis of the list sample in the 1992 SCF 

by Kennickell and McManus [1993]. 

10If the access limitation variables are included in this model, the effects of housing size 

and vintage are sustained. Although the comparison is affected by the high missing data rate for 

this sample, these factors at least give no sign of being proxies for obstacles to accessing the 

respondent’s residence. 

11A set of comparable models estimated using only the case-level variables also suggests 

the independence of most of the significant effects in this model. 
20

To be approached by an interviewer, a list sample case must not have returned the refusal 

postcard. Avoiding this refusal requires one of two things: either the respondent never opens the 

mailing containing the postcard or having opened it they choose not to return the postcard. That 

choice could be based on a misunderstanding that the card should only be returned if they wanted 

to participate, a belief that return of the postcard might place them at risk, that they felt they had 

sufficient means of evading an interviewer later, or that they accepted the opportunity to receive 

more information about the survey. In the project debriefing, interviewers reported that very many 

list sample respondents did not recall ever having received the mailing; this suggests that the 

propensity to open unsolicited mail may be a key driver at the postcard stage. As shown in column 

1, only a few systematic factors emerge in the model describing the initial passive agreement to 

continue (that is, the card was not returned). Neighborhoods that have higher proportions of 

Hispanics, household incomes of $150,000 or more, and occupied housing units, and households 

with income from partnerships or s-corporations or losses from self-employment income were more 

likely not to return the refusal postcard. 

There are no characteristics in the models that have a significant effect in all phases of actual 

(or potential) contact with an interviewer. Even the two measures of Phase 1 highlight many 

different factors, though a number are consistent. In the first phase, neighborhoods with a higher 

proportion of people with a bachelor’s degree have higher response rates; this accords with reports 

from interviewers that more educated people are easier to persuade, because they are more likely 

to understand the importance of the survey. Refusals are more likely in neighborhoods with a higher 

proportion of households with at least $150,000 of income. At the same time, receipt of taxable and 

nontaxable interest incomes and age of the primary taxpayer (peaking at about age 60) are positively 

related to response propensity; and as in the overall model, total income and total financial income 

have a negative effect. 

The model for Phase 2A has no significant tract-level factors in common with the models 

for Phase 1 response; among the case-specific variables they share the key effects of age and 

financial income. The model for phase 3A has no significant factors in common other than stratum 

indicator with any of the models for the earlier phases. Indeed, the Phase 3A model reveals little 

systematic structure in terms of tract-level variables aside from a correlation of higher population 

density and higher percentages of unemployed workers with response. None of the case-specific 
21 

variables beyond the stratum indicators are significant; this suggests that continued effort may do 

less than usually hoped to alter the composition of the pool of cases remaining after the initial stages 

of field work. At the same time, it does not appear that allowing the application of effort in Phase 

3A to the discretion of the field staff added much additional systematic distortion. 
22 

Table 9: Logit models of response for various phases of the field period, list sample. 

(1) (2) (3) (4) (5) (6) (7) P_HOU_GE50_UNIT -0.002 -0.001 -0.008 -0.008 -0.010 -0.009* -0.009* 

0.005 0.006 0.005 0.006 0.011 0.004 0.004 

POST- PHSE1 PHA- PHA- PHA- COMPXPC COMP P_HOU_LE1939 0.003 0.002 -0.003 -0.010* 0.005 -0.006* -0.005+ 

CARD SE1A SE2A SE3A 0.004 0.004 0.004 0.004 0.009 0.003 0.003 

P_HOU_1940_1959 0.003 -0.003 -0.011# -0.004 0.008 -0.007# -0.006* 

0.003 0.004 0.003 0.004 0.008 0.003 0.003 

Intercept 4.963+ -6.526* -8.194# 0.171 -5.107 -2.782 -2.490 P_HOU_1960_1989 0.004 0.002 -0.003 -0.006+ 0.007 -0.004 -0.002 

2.902 2.932 2.785 3.157 6.041 2.194 2.111 0.003 0.003 0.003 0.003 0.007 0.002 0.002 

NON_MSA -0.057 -0.092 -0.087 -0.076 0.525 -0.028 -0.048 L_MED_RENT 0.020 -0.024 0.308+ -0.194 -0.380 -0.006 -0.006 

0.157 0.127 0.121 0.149 0.388 0.103 0.099 0.178 0.184 0.172 0.184 0.355 0.135 0.131 

SM_MSA 0.342 -0.038 0.148 -0.140 -0.822 -0.066 0.022 P_PHONE -0.122* -0.020 0.021 -0.060 -0.021 -0.013 -0.034 

0.270 0.206 0.194 0.246 0.718 0.167 0.161 0.055 0.042 0.042 0.044 0.090 0.032 0.031 

NEW_ENGLAND -0.120 -0.050 -0.057 0.198 0.312 0.125 0.094 LS_STRAT7 0.502 -1.195# -0.602 -0.121 -1.068 -0.425 -0.356 

0.184 0.191 0.182 0.187 0.386 0.140 0.134 0.341 0.437 0.380 0.378 0.918 0.280 0.265 

MID_ATLANTIC 0.047 -0.031 0.167 -0.130 -0.527 -0.023 -0.002 LS_STRAT6 0.246 -0.358+ -0.202 0.065 0.877* 0.008 0.054 

0.144 0.144 0.133 0.153 0.353 0.108 0.103 0.189 0.213 0.195 0.211 0.439 0.152 0.143 

S_ATLANTIC -0.004 -0.179 -0.006 -0.054 0.305 -0.013 -0.003 LS_STRAT5 0.130 -0.133 0.062 0.087 0.540+ 0.068 0.102 

0.132 0.130 0.122 0.134 0.265 0.097 0.092 0.120 0.132 0.118 0.132 0.294 0.094 0.089 

E_S_CENTRAL 0.588 0.341 0.024 0.128 -0.077 0.051 0.144 LS_STRAT4 0.005 0.231* 0.036 0.044 0.709# 0.070 0.100 

0.409 0.280 0.282 0.336 0.716 0.235 0.227 0.113 0.117 0.109 0.125 0.268 0.089 0.084 

W_S_CENTRAL -0.249 -0.608# -0.103 -0.133 0.082 -0.132 -0.152 LS_STRAT3 -0.176 0.450# 0.231 0.114 -0.521 0.118 0.081 

0.168 0.185 0.163 0.181 0.334 0.131 0.124 0.150 0.156 0.145 0.171 0.453 0.121 0.114 

E_N_CENTRAL -0.098 -0.001 0.084 0.011 -0.358 -0.015 -0.008 LS_STRAT2 -0.281 0.447* 0.271 0.192 0.459 0.277+ 0.192 

0.129 0.129 0.124 0.139 0.308 0.101 0.095 0.206 0.222 0.205 0.243 0.513 0.170 0.158 

W_N_CENTRAL -0.134 -0.029 -0.529* 0.098 0.566 -0.047 -0.117 AGE -0.031 0.051* 0.044+ 0.094# 0.010 0.067# 0.059# 

0.199 0.214 0.243 0.214 0.350 0.166 0.158 0.025 0.026 0.024 0.030 0.062 0.020 0.019 

MOUNT_PACIFIC 0.225 0.408* 0.251 0.019 -0.163 0.079 0.107 AGE_SQ 0.011 -0.042+ -0.034 -0.074# -0.011 -0.053# -0.050# 

0.241 0.198 0.198 0.228 0.462 0.163 0.157 0.022 0.025 0.023 0.029 0.059 0.019 0.018 

POP_DENSITY -0.003 -0.007 -0.006 0.005 0.017# 0.003 0.002 L_TOTINC 0.063 -0.196# -0.248# -0.131 -0.054 -0.192# -0.155# 

0.003 0.005 0.004 0.003 0.006 0.003 0.002 0.071 0.080 0.075 0.081 0.163 0.059 0.055 

P_NATIVE_BORN 0.010 0.008 0.012 -0.015 -0.002 -0.003 -0.002 D_SALARIES -0.291 -0.014 -0.189 -0.407 0.978 -0.112 -0.165 

0.015 0.015 0.014 0.015 0.028 0.011 0.011 0.326 0.376 0.353 0.375 0.642 0.270 0.256 

P_HISP 0.037+ 0.032 0.012 0.024 0.022 0.018 0.026+ L_SALARIES 0.033 0.021 0.046 0.039 -0.055 0.029 0.034 

0.020 0.020 0.020 0.022 0.038 0.016 0.015 0.028 0.033 0.030 0.031 0.053 0.023 0.022 

P_AFAM 0.005 0.008+ 0.009* 0.003 -0.003 0.005 0.006+ D_NTAX_INTEREST 0.043 0.258* 0.350# 0.299* 0.135 0.346# 0.340 

0.005 0.004 0.004 0.006 0.013 0.004 0.004 0.114 0.123 0.114 0.124 0.239 0.090 0.087 

P_ASIA -0.004 -0.002 0.004 0.020+ -0.008 0.013+ 0.010 D_TAX_INTEREST -0.303 0.462+ 0.519* 0.592+ 0.026 0.620# 0.491* 

0.010 0.011 0.010 0.011 0.024 0.008 0.007 0.329 0.267 0.260 0.324 0.715 0.222 0.209 

P_RACE_OTH 0.084 -0.064 0.021 -0.012 0.015 0.003 0.016 D_DIVIDENDS 0.094 0.439# 0.395# 0.119 0.155 0.292* 0.282* 

0.066 0.060 0.039 0.055 0.084 0.034 0.033 0.166 0.165 0.158 0.187 0.376 0.131 0.125 

P_SPONLY_SPAN -0.026 -0.029 -0.011 -0.029 -0.006 -0.018 -0.025 L_FININC -0.102# -0.063* -0.050+ -0.085# -0.061 -0.075# -0.092# 

0.025 0.026 0.026 0.027 0.045 0.019 0.019 0.032 0.032 0.030 0.034 0.065 0.024 0.023 

P_SPONLY_OTH 0.020 -0.003 0.000 -0.030+ 0.004 -0.017 -0.012 D_NET_KG_GAIN -0.151 0.184 0.134 0.033 -0.428 0.037 0.004 

0.016 0.017 0.015 0.017 0.034 0.012 0.012 0.120 0.132 0.122 0.138 0.273 0.098 0.093 

P_AGE_LT18 -0.029+ 0.025+ 0.021 0.014 0.005 0.019+ 0.014 D_NET_KG_LOSS 0.067 0.227+ 0.033 0.126 -0.384 0.038 0.040 

0.016 0.015 0.014 0.016 0.032 0.011 0.011 0.123 0.130 0.121 0.122 0.246 0.092 0.088 

P_AGE_GE65 -0.011 0.019 0.034# 0.012 0.001 0.025* 0.021* D_FARM_INC -0.132 -0.072 -0.027 -0.354 0.479 -0.111 -0.116 

0.014 0.014 0.013 0.015 0.030 0.011 0.010 0.196 0.233 0.218 0.251 0.369 0.168 0.161 

P_GEBA -0.001 0.030# 0.023* -0.006 0.018 0.011 0.011 D_RENT_ROY -0.097 0.059 -0.134 -0.196* 0.279 -0.133+ -0.146* 

0.010 0.010 0.009 0.011 0.022 0.008 0.007 0.095 0.100 0.092 0.101 0.201 0.073 0.070 

P_SOMCOLL -0.005 0.014 0.008 -0.022 0.048 0.001 0.001 D_PART_SCORP 0.221+ -0.059 -0.243* 0.144 0.241 -0.036 0.029 

0.015 0.016 0.015 0.017 0.034 0.012 0.011 0.123 0.126 0.117 0.141 0.277 0.097 0.093 

P_LTHS -0.019 0.018 0.019 -0.034+ 0.023 -0.003 -0.006 D_ESTATE_TRUST 0.019 0.018 -0.214 0.222 -0.105 0.007 0.029 

0.017 0.017 0.016 0.019 0.035 0.013 0.012 0.149 0.176 0.170 0.155 0.326 0.120 0.116 

P_INC_LT10 -0.007 0.006 -0.003 0.012 0.000 0.006 0.004 D_SCH_C_NET_GAIN 0.093 0.191+ 0.164 -0.080 0.347 0.072 0.098 

0.016 0.015 0.014 0.017 0.029 0.012 0.011 0.107 0.111 0.104 0.116 0.222 0.083 0.079 

P_INC_75_150 0.013 -0.019* -0.009 -0.017+ -0.001 -0.012* -0.009 D_SCH_C_NET_LOSS 0.444# 0.211 0.179 -0.202 0.180 0.004 0.078 

0.008 0.009 0.008 0.009 0.018 0.006 0.006 0.156 0.150 0.139 0.158 0.300 0.110 0.107 

P_INC_GE150 0.012+ -0.027# -0.017# -0.003 0.007 -0.009 -0.005 D_TOT_PENSION 0.159 -0.594+ -0.136 0.197 -0.341 0.002 0.046 

0.007 0.008 0.007 0.008 0.016 0.006 0.005 0.288 0.322 0.285 0.306 0.667 0.224 0.213 

P_WORKERS -0.022+ 0.003 0.004 0.011 0.037 0.014 0.009 L_TOT_PENSION -0.019 0.060+ 0.019 -0.022 0.024 -0.001 -0.007 

0.013 0.013 0.012 0.013 0.025 0.009 0.009 0.029 0.033 0.030 0.032 0.068 0.023 0.022 

P_UNEMP 0.004 0.009 0.019 -0.028 0.056+ 0.011 0.010 D_TOT_SOCSEC 1.092 -2.404 -1.784 -0.591 -9.112 -1.789 -1.578 

0.018 0.017 0.015 0.022 0.032 0.013 0.012 1.510 1.839 1.656 1.886 6.084 1.367 1.309 

P_COMMUT_LT25 -0.003 0.009 0.002 0.013* 0.003 0.007 0.006 L_TOT_SOCSEC -0.118 0.275 0.207 0.066 0.902 0.197 0.173 

0.006 0.006 0.006 0.007 0.012 0.005 0.004 0.155 0.189 0.170 0.193 0.611 0.140 0.134 

P_COMMUT_GE45 -0.003 0.009 0.003 0.019* -0.015 0.008 0.006 D_TOT_DEDUCT 0.171 0.090 0.320 -0.249 -0.958 -0.037 0.019 

0.008 0.008 0.007 0.008 0.018 0.006 0.006 0.315 0.301 0.284 0.376 1.109 0.248 0.234 

P_OCC_HOU 0.120* 0.030 0.017 0.064 0.007 0.027 0.047 D_MORT_DEDUCT 0.031 -0.001 -0.084 0.064 0.124 0.013 0.016 

0.056 0.043 0.043 0.045 0.090 0.033 0.032 0.109 0.124 0.112 0.121 0.242 0.087 0.084 

P_OWNOCC -0.013* 0.012+ -0.008 -0.005 0.007 -0.004 -0.007 D_CASH_CHARITY -0.294 -0.523 -0.546+ -0.315 1.139 -0.359 -0.378 

0.007 0.007 0.007 0.007 0.014 0.005 0.005 0.353 0.353 0.329 0.414 1.140 0.277 0.264 

P_HOU_1ATT_UNIT 0.002 -0.002 -0.006 0.011* -0.011 0.001 0.002 L_CASH_CHARITY 0.026 0.049 0.039 0.044 0.016 0.044* 0.045* 

0.006 0.005 0.005 0.006 0.015 0.004 0.004 0.028 0.032 0.029 0.031 0.059 0.022 0.022 

P_HOU_2_4_UNIT -0.006 0.009 0.004 0.002 -0.010 0.003 0.002 

0.007 0.007 0.007 0.008 0.018 0.006 0.005 N 4997 4336 4336 3552 2962 4336 4997 

P_HOU_5_49_UNIT -0.004 0.001 -0.011+ -0.007 -0.016 -0.010* -0.011# 

0.006 0.006 0.006 0.006 0.012 0.004 0.004 #: p<=1%, *: p<=5%, +: p<=10% 
23 

Variable definitions for tables 7, 8 and 9. 

2000 Census data available for area-probability and list 

sample cases: 

NON_MSA: =1 if sample area not an MSA.

SM_MSA: =1 is sample area not self-representing.

NEW_ENGLAND: =1 if sample area in New England.

MID_ATLANTIC: =1 if sample area in mid-Atlantic. 

S_ATLANTIC: =1 if sample area south Atlantic.

E_S_CENTRAL: =1 if sample area east south central.

W_S_CENTRAL: =1 if sample area west south central.

E_N_CENTRAL: =1 if sample area east north central.

W_N_CENTRAL: =1 if sample area west north central

MOUNT_PACIFIC: =1 if sample area mountain Pacific.

PACIFIC: omitted category: Pacific coast

POP_DENSITY: number of people in census tract divided

by area of tract in square meters.

P_NATIVE_BORN: % native born in tract.

P_HISP: %a Hispanic in tract.

P_AFAM: % African American in tract.

P_ASIA: %a Asian American in tract.

P_RACE_OTH: % other nonwhite race in tract.

P_SPONLY_SPAN: % in tract speaking only Spanish.

P_SPONLY_OTH: % in tract speaking only language other

than Spanish or English.

P_AGE_LT18: % in tract aged less than 18.

P_AGE_GE65: % in tract aged 65 or older.

P_GEBA: % of adults in tract with a bachelor’s degree or

higher education.

P_SOMCOLL: % of adults in tract with some college but

less than a bachelor’s degree.

P_LTHS: % of adults in tract with less than a high school

diploma.

P_INC_LT10: % of households in tract with income less

than 10,000.

P_INC_75_150: % of households in tract with income

75,000 to 150,000.

P_INC_GE150: % of households in tract with income of

150,000 or more.

P_WORKERS: % of people in tract in labor force.

P_UNEMP: % of people in tract unemployed.

P_COMMUT_LT25: % of workers in tract commuting to

work 25 minutes or less.

P_COMMUT_GE45: % of workers in tract commuting to

work 45 minutes of more.

P_OCC_HOU: % of occupied housing units in tract.

P_OWNOCC: % of housing units in tract owner-occupied.

P_HOU_1_ATT_UNIT: % of housing units in tract

attached single-family homes.

P_HOU_2_4_UNIT: % of housing units in tract in

buildings with 2 to 4 units.

P_HOU_5_49_UNIT: % of housing units in tract in

buildings with 5 to 49 units.

P_HOU_GE50_UNIT: % of housing units in tract in

buildings with 50 or more units.

P_HOU_LE1939: % of housing units in tract built in 1939

or earlier.

P_HOU_1940_1959: % of housing units in tract built

between 1940 and 1959.

P_HOU_1960_1989: percet of housing units in tract built

between 1960 and 1989.

L_MED_RENT: natural logarithm of median value of

residential rent in tract.

P_PHONE: % of residences with a telephone.

GUARD: =1 if doorman or guard at gate at residence of

respondent.

LOCKED_LOBBY: =1 if residence of respondent in a

building with a locked lobby.

LOCKED_GATE: =1 if resident of respondent behind a

locked gate.

NO_TRESPASS: =1 if “no trespassing” sign posted at

residence of respondent.

LS_STRAT7: =1 if list sample stratum 7.

LS_STRAT6: =1 if list sample stratum 6.

LS_STRAT5: =1 if list sample stratum 5.

LS_STRAT4: =1 if list sample stratum 4.

LS_STRAT3: =1 if list sample stratum 3.

LS_STRAT2: =1 if list sample stratum 2.

LS_STRAT1: omitted category: list sample stratum 1.

SOI data available for list sample cases only: 

AGE: age of primary taxpayer.

AGE_SQ: square of age of primary taxpayer.

L_TOTINC: ln(total 2002 income).

D_SALARIES: =1 if had 2002 wages.

L_SALARIES: ln(max(1,2002 wages)).

D_NTAX_INTEREST: =1 if had 2002 nontaxable interest

income.

D_TAX_INTEREST: =1 if had 2002 taxable interest

income.

D_DIVIDENDS: =1 if had 2002 dividends.

L_FININC: ln(max(1,2002 income from nontaxable

interest, taxable interest and dividends)).

D_NET_KG_GAIN: =1 if had 2002 positive capital gains.

D_NET_KG_LOSS: =1 if had 2002 capital losses.

D_FARM_INC: =1 if had 2002 farm income.

D_RENT_ROY: =1 if had 2002 income from rents or

royalties.

D_PART_SCORP: =1 if had 2002 income from

partnerships or subchapter s corporations.

D_ESTATE_TRUST: =1 it had 2002 income from estates

or trusts.

D_SCH_C_NET_GAIN: =1 if had 2002 positive selfemployment income.

D_SCH_C_NET_LOSS: =1 if had 2002 losses from selfemployment income.

D_TOT_PENSION: =1 if had 2002 pension income.

L_TOT_PENSION: ln(max(1,2002 pension income)).

D_TOT_SOCSEC: =1 if had 2002 social security income.

L_TOT_SOCSEC: ln(max(1,2002 social security income.)).

D_TOT_DEDUCT: =1 if had 2002 itemized deductions.

D_MORT_DEDUCT: =1 if had 2002 deduction for

mortgage interest.

D_CASH_CHARITY: =1 if had 2002 deduction for

charitable contributions.

L_CASH_CHARITY: ln(max(1,2002 deduction for

charitable contributions.
24 

III. Summary and future research

In the SCF, as in many other surveys, nonresponse is a serious problem. To maintain 

response rates in the face of growing difficulties in gaining cooperation, the survey has been obliged 

to invest increasingly in training, materials, contact management and refusal conversion. A key to 

making progress—or at least maintaining the status quo—is understanding the causes of 

nonresponse. Earlier research on the SCF (Kennickell [2004] and references cited there) has 

highlighted the importance of the choices made by interviewers in application of effort to persuade 

respondents to participate, and the possible distortions that may be introduced as a result of those 

choices. With the goals of gaining a clearer picture of response propensities uncontaminated by 

variations in interviewers’ levels of effort and of managing field resources more efficiently, field 

operations for the 2004 SCF were redesigned around a phased protocol of sample management. 

The field period was divided into three phases. The first phase was designed to cover a 

period of a limited number attempts devoted to locating and contacting the respondent and to 

persuading the respondent to participate. If contact proved impossible or the respondent balked at 

participating, the case was to have been marked to receive a special informational package by 

express mail. This mailing marked the beginning of the second phase. The second phase was 

designed to include sufficient follow-up to ensure that respondents who had been contacted 

previously did not have lingering questions after the express mailing. Where the respondent had not 

yet been reached directly by an interviewer, the goal was to make a reasonable effort to reach the 

respondent to follow up on the mailing. Again, if contact within a limited number of attempts 

proved impossible or if the respondent refused to participate, the case was to have been assigned to 

the third phase. The division of the field work across the first two phases is taken to be driven by 

a threshold decision model on the part of the respondent; the level of effort applied to reach that 

threshold should not have been driven by any feeling the interviewers or field managers had about 

the case. In the third phase, the field staff were allowed to make their own evaluation of which cases 

still had the possibility of being completed and to target effort to those cases largely as they saw fit. 

Unfortunately, the adherence to the protocol appears to have been imperfect. According to 

the call records for the cases, the number of attempts made to interview cases was somewhat more 

spread in the first phase than had been intended, but the problem was far worse in the second phase. 

Although some of this deviation may be explained by the fact that the information available in the 
25

call records is too crude to distinguish what are probably important differences in the level of effort 

applied to the cases, the spread is too large to be explained entirely by that means. Thus, analysis 

of response over the field period requires alternative measures of the sample phases, at least for the 

field effort beyond the first phase. 

The paper presents a number of logit models of response for different parts of the field 

period. For the area-probability sample, data matched by census tract and interviewer observations 

on human and physical barriers to contacting the respondent are used as independent variables. The 

data reveal a number of patterns in an overall model of nonresponse (for example, higher response 

propensity among neighborhoods where more people have relatively short commutes and lower 

propensity in neighborhoods where more people have less than a high school education). A few of 

the patterns seen in the initial sample phase are sustained to the end of the field period (for example, 

people in neighborhoods with a high proportion of people under the age of 18 are more likely to 

respond) and some initial effects are offset by the close of the field period (for example, in the first 

phase, people living in neighborhoods with higher fractions of households having $150,000 or more 

of income are less likely to respond). However, interpretation of the models is clouded by the fact 

that there is no means of distinguishing whether the significant effects in the model are describing 

important context effects, respondent characteristics or both. Moreover, the correlation between 

area characteristics and the characteristics of respondent who have not yet been interviewed at each 

point may well change over the field period. Without case-specific data, we only know that there 

are systematic selection effect and we can only guess at the underlying structure. 

The list sample offers the opportunity for bringing some case-level data to bear on 

nonresponse. The SOI data set available from the original sample design contains the age of the 

primary taxpayer, income characteristics and information on itemized deductions. To apply 

conclusions from the analysis of the list sample to the area-probability sample requires that the 

populations modeled are the same. But estimates of overall response models using only the tractlevel data for the two samples reveal a structure that seems different in many ways. 

Nonetheless, further modeling of nonresponse for the list sample is important in its own right 

and it still may give a general sense of what is missed by relying only on contextual indicators, such 

as tract-level characteristics, to characterize nonresponse. The models presented for the list sample 

do indicate that both contextual and case-level variables are significant explanatory factors. Some 
26

key tract level factors in overall response are residence in neighborhoods with higher fractions of 

older buildings (negative effect), of apartment buildings with more than five units (negative effect), 

of people aged younger than 18 or 65 and older (positive effects), of people with at least a bachelor’s 

degree (positive effect), and of households with upper-middle class incomes (negative effect). The 

case-specific variables show strong overall effects for age (positive effect), financial income 

(negative) and charitable contributions (positive), among others. As in the case of the models for 

the area-probability sample, there is variation in the importance of factors over the phases of work. 

For the future, tightening the field application of the phased sample management protocol 

is a high priority. A clearer division of the field period would offer several possibilities. First, a 

model indicating the types of cases that tend to be under-represented up through the end of Phase 

2 could be used to give guidance to field staff in the final phase of the survey that would be better 

informed by benefits and costs.12 It seems unlikely that such modeling could be done using current 

field work, if only because of the great logistical difficulties of moving cases together through the 

phases of work. But Phase 1 results might be used to calibrate results estimated from the previous 

round of a survey. Second, a better understanding of how different cases respond over the field 

period could lead to improvements in the approach taken to persuade people to participate. Third, 

clearer modeling could lead to better nonresponse adjustments at the weighting stage. Finally, the 

trajectory of effects of various factors over the field period could lead to deeper insights into the 

nonignorable nonresponse that nearly all surveys fear. 

12An alternative approach is the “responsive sampling” model (Heeringa and Groves 

[2004]), which attempts to target effort dynamically toward cases most likely to be completed. 
27 

Bibliography 

Aizcorbe, Ana M., Arthur B. Kennickell, and Kevin B. Moore [2003] “Recent Changes in U.S. 

Family Finances: Evidence from the 1998 and 2001 Survey of Consumer Finances,” Federal 

Reserve Bulletin, v. 89 (January), pp. 1-32. 

Heeringa, Steven G. and Robert M. Groves [2004] “Responsive Design for Household Surveys,” 

mimeo Institute for Social Research, University of Michigan. 

Kennickell, Arthur B. [2004] “Action at a Distance: Interviewer Effort and Nonresponse in the 

SCF,” Proceedings of the American Statistical Association Social Statistics Section. 

Kennickell, Arthur B. [2003] “Reordering the Darkness: Application of Effort and Nonresponse in 

the Survey of Consumer Finances,” Proceedings of the American Statistical Association Survey 

Methods Research Section. 

Kennickell, Arthur B. [2001] “Modeling Wealth with Multiple Observations of Income: Dedesign 

of the Sample for the 2001 Survey of Consumer Finances,” working paper, 

http://www.federalreserve.gov/pubs/oss/oss2/method.html. 

Kennickell, Arthur B. [2000] “Asymmetric Information, Interviewer Behavior, and Unit 

Nonresponse,” working paper, http://www.federalreserve.gov/pubs/oss/oss2/method.html. 

Kennickell, Arthur B. [1999b] “Analysis of Nonresponse Effects in the 1995 Survey of Consumer 

Finances,”Journal of Official Statistics, v. 15 no. 2, pp. 283-304. 

Kennickell, Arthur B. [1999a] “What Do the ‘Late’ Cases Tell Us? Evidence from the 1998 Survey 

of Consumer Finances,” http://www.federalreserve.gov/pubs/oss/oss2/method.html. 

Kennickell, Arthur B. and Douglas A. McManus [1993] “Sampling for Household Financial 

Characteristics Using Frame Information on Past Income,” Proceedings of the American Statistical 

Association Social Statistics Section. 

O’Muircheartaigh, Colm, Stephanie Eckman, and Charlene Weiss [2002] “Traditional and Enhanced 

Field Listing for Probability Sampling,” Proceedings of the American Statistical Association Social 

Statistics Section, pp. 2563-7. 
28

Appendix 

Figure A1: Sample management protocol, 2004 SCF. 
29 

Table A1: Case disposition immediately preceding request for express 

mailing, in-scope cases that received an express mailing, 2004 SCF. 

All AP LS 

Presumed to be no contact 3.7 2.7 4.4 

Screener break off 0.4 0.7 0.2 

Sample address observations completed 6.0 6.0 6.1 

No one over 18 0.1 0.2 * 

Not Home/No Answer 30.8 31.5 30.3 

Inaccessible 2.9 2.6 3.2 

Non HU 0.0 0.0 * 

Vacant 0.1 0.3 * 

Sample incorrect 0.1 0.1 0.0 

Unlocatable (home) 1.7 0.0 2.9 

Unlocatable (business) 0.4 0.0 0.6 

New lead 4.7 0.0 7.9 

New lead failed 0.2 0.0 0.3 

Call back/Unavailable/Busy 12.8 10.3 14.5 

R to call 1.3 1.1 1.4 

Special letter sent 3.1 2.1 3.8 

R too ill 0.2 0.4 0.1 

Language barrier (Spanish) 0.0 0.1 0.0 

Language barrier (not Spanish) 0.3 0.5 0.1 

Screener/Informant obs. completed 7.9 11.0 5.7 

Non-respondent refusal 1.3 0.9 1.5 

Refusal 17.7 22.9 14.2 

Hostile refusal 1.5 2.4 0.9 

Appointment/Rescheduled 0.3 0.4 0.3 

Broken appointment 1.1 2.1 0.5 

Questionnaire break off 0.3 0.4 0.1 

Conversion of break-off 0.0 0.0 0.0 

Interview completed 0.1 0.2 0.1 

Final refusal 1.0 1.2 0.9 

Final unlocatable 0.0 0.0 0.0 

Final language barrier (not Spanish) 0.0 0.1 0.0 

Final R too ill 0.0 0.0 0.0 

Memo items: % of all in-scope cases 

Completed before express mailing 26.7 39.1 14.2 

Final incomplete before express mailing 3.1 1.9 4.3 

*: not applicable