Journal of Of®cial Statistics, Vol. 13, No. 1, 1997, pp. 1±18

Who Lives Here?

Survey Undercoverage and Household Roster Questions

Roger Tourangeau,1 Gary Shapiro,2 Anne Kearney3 and Lawrence Ernst4

1. Introduction

During the fall of 1992, NORC carried out a methodological experiment for the U.S.

Bureau of the Census. The purpose of the study was to test new versions of the roster

questions used to enumerate members of sample households in many surveys in an effort

to improve coverage of household members. In addition, the new roster questions

permitted comparison of different rules for linking persons to dwellings. In 1991, a test

of differing rostering procedures had been conducted in conjunction with the American

Housing Survey (Shapiro, Diffendal, and Cantor 1993). In the same year, a small-scale

We carried out an experimental comparison of three versions of questions for enumerating the

residents of a dwelling. One version took the approach that is used in many surveys: it began

by asking respondents to name all persons living at the dwelling. The experimental versions

began by asking how many persons had spent the previous night at the dwelling and used other

probes to complete the roster. The two experimental versions differed only in that one version

did not require persons to be listed by their full names, allowing respondents to use initials or

nicknames instead. A total of 509 interviews were completed, about a third of them with each

version of the questionnaire. The results indicated that both experimental versions of the

roster questions yielded more persons per household than the standard version; however,

only the version that did not require full names yielded more persons identi®ed as usual

residents of the dwelling. Additional analyses indicated that the same types of persons

were listed on all three versions; only the number of persons listed differed.

Key words: Roster; undercoverage; usual residence.

q Statistics Sweden

1 NORC, 1350 Connecticut Ave., NW, Washington, DC 20036, U.S.A.

2 Westat, Inc., 1650 Research Boulevard, Rockville, MD 20850, U.S.A.

3 U.S. Bureau of the Census, Washington, DC 20233, U.S.A.

4 U.S. Bureau of Labor Statistics, 2 Massachusetts Ave., NE, Room 3160, Washington, DC 20212, U.S.A.

Acknowledgment: The study reported here was carried out by NORC under contract to the U.S. Bureau of the

Census (Contract No. 50-YABC-2-66023, MR092-01 SIPP Roster Research). The views expressed here are

solely those of the authors and do not represent the official views of the U.S. Census Bureau. When the project

was carried out, Gary Shapiro and Lawrence Ernst were at the U.S. Bureau of the Census.

An earlier version of this article was presented at the 1993 meeting of the American Statistical Association. We

thank Randall Parmer, Michael Roebuck, and Lynn Weidman for their comments on that version.

The design of this study grew out of a meeting held in Chicago during the fall of 1992. The authors gratefully

acknowledge the contributions of those attending this design meeting, including Richard Kulka and Kenneth

Rasinski of NORC's staff, Peter Hainer of Curry College, Seymour Sudman and Diane O'Rourke of the Survey

Research Laboratory at the University of Illinois, and Laurie Schwede and Leslie Brownrigg from the U.S.

Bureau of the Census. We also acknowledge the invaluable assistance of Hee-Choon Shin on the analysis of the

data.
experiment, funded by the Survey of Income and Program Participation (SIPP), compared

two versions of the roster questions (Cantor and Edwards 1992; Shapiro et al. 1993). This

study built on the results of these earlier efforts to improve the coverage of federal surveys.

Problems in enumerating household members are part of the larger problem of coverage

errors in surveys and the decennial census. Coverage errors encompass problems resulting

in overcounts (typically arising from counting individuals more than once) as well as those

producing undercounts, although most research attention has focused on the issue of

undercounts. The omission of members of households that are partially enumerated is

thought to be a major source of undercounts. Shapiro et al. (1993) estimated that 60%

or more of the persons missed in the Current Population Survey (CPS) are members of

partially enumerated households ± that is, both the dwelling unit and the household

were included, but individual household members were nonetheless omitted from the

survey. Similarly, using data from the 1986 Los Angeles test census, Fein and West

(1988) estimated that roughly two-thirds of the persons missed by the test census were

members of partially enumerated households. Results from the 1990 Post-Enumeration

Survey carried out to estimate coverage in the 1990 census suggest that nearly a third

of the omissions occurred within partially enumerated households.

Recently, several evaluations of coverage problems in the census (Fein 1990; Fein and

West 1988; Hogan 1993; Robinson, Ahmed, Gupta, and Woodrow 1993) and in the CPS

(Hainer, Hines, Martin, and Shapiro 1988; Fay 1989) have been reported. Much of the

work described there and in earlier research on undercoverage problems (e.g., Valentine

and Valentine 1971) suggests that within-household omissions arise through several distinct processes. For example, Brownrigg and Martin (1993) argue that undercoverage

occurs for ®ve main reasons: (1) mobility; (2) illiteracy and other language problems;

(3) deliberate concealment; (4) irregular household structure or living arrangements;

and (5) resistance as a strategy for dealing with the government and other persons from

outside the community.

These causes can be grouped into two larger categories ± those involving deliberate

omissions and those involving dif®culty in applying the residence rules underlying the

roster questions to the living situations of the persons to be enumerated. As Hainer and

his coworkers put it:

One reason people are missed is motivational. Black (and Hispanic) males are deliberately

omitted from household rosters because of the potential loss of household income if the

men were known to authorities. 1⁄4 A second cause of undercoverage is the lack of ®t

between the census de®nitions of household and residency, and people's actual living

situations.

Both processes are thought to contribute to the very high rates of undercoverage observed

for young Black men (ages 20 to 24), one-third of whom may be missed in the CPS (Hainer

et al. 1988; see also Shapiro et al. 1993). In Black households, respondents may believe

that complete reporting of household membership may lead to such consequences as

loss of welfare eligibility, eviction, or arrest. Within Hispanic households, the threat of

deportation may be an additional concern.

Compounding these motivational problems are the de®nitional dif®culties that can arise

with the complex family structures apparent in many Black households. According to

Hainer and his colleagues, Black households often comprise extended families whose

2 Journal of Of®cial Statistics
members reside in several nearby dwellings and whose composition can change quickly;

in such families, key roles may be ®lled by persons unrelated to the head of household, and

members of the household may disagree among themselves as to who is actually a member. Ethnographic research suggests that Hispanic households may also exhibit structures

that do not readily ®t the de®nitions embodied in most survey roster questions. McKay

(1992) has noted a common arrangement within Salvadoran households that have recently

immigrated to the United States, in which a core family shares a residence with one or

more roomers. The roomers are particularly likely to be omitted from household rosters

reported for surveys (Hainer et al. 1988; McKay 1992); according to McKay, they are

simply not seen as ``residing'' with the household.

The study reported here attempted to address both categories of causes. In an effort to

reduce deliberate omissions, we allowed some respondents not to report full names; in an

effort to circumvent de®nitional dif®culties, we developed an extensive set of probes.

There is evidence that undercoverage seriously biases estimates for a number of household characteristics. Moreover, a number of readily-measured household characteristics

that may indicate a bad ®t with the standard de®nitions are known to be related to undercoverage. These include household size (members of large households are more likely to

be omitted; see Fein and West 1988) and household composition (household members

who are not related to the head of household and members who are related but not members of the same nuclear family are more likely to go unreported; Fay 1989; Fein 1990;

Fein and West 1988; Hainer et al. 1988). Race and ethnic differences in undercoverage

may also result in part from differences in family structure and the resulting differences

in the ®t between respondents' living arrangements and the de®nitions underlying the

enumeration procedure. Similarly, variables that are likely to be related to the need

for concealment of household members (such as the presence of household members

who are welfare recipients or recent immigrants) have also been shown to predict

undercoverage (Fein and West 1988).

The questionnaires used in this study were modelled on those used in the Survey of

Income and Program Participation (SIPP). SIPP is a longitudinal survey of adults, conducted

in person, that measures economic and demographic characteristics. Households in the SIPP

sample are interviewed every four months for a total period of participation of two and a half

years; households that move during that period are followed to their new addresses. The

SIPP questionnaire embodies what is probably the most common approach for enumerating

members of sample households in surveys. Virtually the ®rst thing that new respondents are

asked to do is to list the names of everyone who lives or stays at the address; later probes

then attempt to capture babies, roomers, persons who are temporarily away, and others

who may have been omitted from the initial roster of residents. (In addition, the

longitudinal character of SIPP requires that the roster from one wave be updated in the

next wave and that changes in household composition be recorded.)

As is true in many surveys, the SIPP counts a person as a resident of a particular

dwelling if the person ``usually'' lives there. Under current U.S. Census Bureau de®nitions, a person's usual residence is the one where the person lives and sleeps most of

the time; if a person spends time at several dwellings, the usual residence is the one where

he or she stays most often. In principle, then, a plurality rule is to be used; in practice,

however, respondents are left to their own devices in determining who to count as a usual

Tourangeau et al.: Who Lives Here? Survey Undercoverage and Household Roster Questions 3
resident. Even if respondents were aware of the plurality rule, they may not have the

information needed to implement it correctly; the respondent may not know where the person in question stayed when he or she was not staying at the sample residence. Other situations may also make it dif®cult for respondents to decide whether someone should be

counted as a usual resident. It may be unclear, for example, whether to count a person

who is staying in the respondent's home temporarily. The respondent may need an explicit

reference period to help decide whether to include temporary residents. The study

described here attempted to capture housing situations like these by using several alternative methods of developing household rosters.

2. Method

Sample. The sample for the experiment consisted of 644 occupied dwelling units spread

across a total of 49 blocks in three sites ± the Chicago and Washington, D.C. metropolitan

areas and the city of Baltimore. The Field Division of the U.S. Census Bureau selected and

listed blocks that were predominantly Black and low income in composition; each block

included at least 40 housing units. Twenty-®ve blocks were selected in the Chicago area,

18 of them located within the city limits and seven of them from two suburbs. Fifteen

blocks were selected in the Washington metropolitan area, eight of them within the city

limits and the remaining seven from nearby suburbs. All nine blocks in the Baltimore

area were located within the city of Baltimore. Based on listings of each of the 49 blocks,

NORC staff selected 15 housing units on each block using a systematic selection

procedure; of the 735 dwellings selected in this way, 644 turned out to be occupied and

thus eligible for interview.

NORC ®eld staff completed interviews at a total of 509 of these dwellings, for a

response rate of 79.0%. Unfortunately, other than their addresses, we have no information

on the nonrespondents. Data were collected on a total of 1,949 individuals. Almost 92% of

these ± 1,791 persons ± were Black.

The experiment compared three versions of the questionnaire. The interviewers

arranged the questionnaires in a predetermined random sequence and then administered

the questionnaires in this random order at each cooperating household. A total of 38

interviewers completed one or more interviews. The interviewing took place over a

four-week period from November to December 1992.

Questionnaires. The three versions of the questionnaires embodied different approaches

to enumerating the residents of the dwelling. The ®rst, or standard, version of the

questionnaire begins with items taken from the SIPP control card and used in many other

household surveys. In Version 1, an initial roster of residents is compiled based on

responses to the standard roster questions (``What are the names of all persons living

or staying here? Have I missed any babies or small children? 1⁄4 Does (NAME) usually

live here?''), and demographic information is collected for each person on the roster.

Next, a series of probes not included in standard rostering procedures were administered.

The probes attempted to identify residents omitted from the initial roster. These additional

probes ask:

· How many people besides those you've already listed stayed here last night?

· Is there anyone else who stayed at least one night during the past month?

4 Journal of Of®cial Statistics
· Is there anyone else who usually stays here but was not here during the last month?

· Is there anyone else who ate here at least once during the last week?

· Is there anyone else who usually eats here but who did not eat here during the last week?

· Is there anyone else who you consider to be a member of this household?

· Is there anyone else who considers himself or herself to be a member of this household?

Persons identi®ed by these additional probes were added to the roster for the dwelling. The

probes cover several reference periods (the prior night, the past week, and the past month)

and, as a result, their exact yield may depend on both on time of the year and the day of the

week on which they are administered.

Once the roster was complete, Version 1 continued with a series of questions designed

to pinpoint the relation of each person listed to the sample dwelling. These questions concern a number of dimensions that respondents may consider in determining whether to

count an individual as a usual resident (Gerber 1990):

· How many nights did NAME stay here during the last month?

· On how many days did NAME eat here at least once during the last week?

· Does NAME contribute money, food, or other help to this household?

· Do you consider NAME to be a member of this household?

· Was NAME here at all during the last day?

· Does NAME consider himself/herself a member of this household?

· Does NAME usually live here?

· Does NAME have other places where he/she frequently stays?

After these follow-up items, there was a series of questions on labor force participation.

Version 1 concluded with several items regarding the respondent's reactions to some of

the earlier questions.

The other two versions of the questionnaire contain essentially the same items as

Version 1 but administered them in a different order. In Version 1, the standard roster

questions come ®rst, then the additional probes, and ®nally the follow-up items; in

Versions 2 and 3, the additional probes were administered ®rst, then the follow-up items,

and ®nally the standard roster questions. (Both the additional probes and the standard

items were reworded slightly to ®t their new context.) Like Version 1, Versions 2 and 3

concluded with the labor force questions and the items assessing reactions to some of the

earlier questions. Versions 2 and 3 differ from each other in only one respect: Version 3

required respondents to identify household members by their full names, whereas

Version 2 did not; Version 2 allowed respondents to use initials, nicknames, or other

means of identifying individuals instead. In this version, the introduction to the roster

questions read:

We're going to ask some questions about each of these people. To help us keep track of

which persons we're talking about, can you identify each one with a ®rst name, a nickname, a set of initials, or by some other means? It is not necessary to give their full names.

At the conclusion of Version 2, respondents were asked to provide full names for all

persons on the roster to permit any follow-up interviewing. In both Version 1 and

Version 3, full names had been requested from the outset.

Tourangeau et al.: Who Lives Here? Survey Undercoverage and Household Roster Questions 5
3. Results

The analysis examined ®ve main issues: (1) Did the household rosters compiled under the

three versions of the rostering questions include different numbers of persons? (2) Did

the rosters compiled under the three versions include persons with different relations to the

residence? (3) What criteria are associated with household informants' decisions on

whether to classify someone as a usual resident? (4) Do alternatives to the standard

procedures for linking persons to dwellings improve coverage? and (5) Were respondents

to Version 2 ultimately willing to provide full names for the persons they had listed on the

household roster? In most of the analyses reported here, we examine persons listed on

the roster for Version 1 prior to the administration of the additional probes. Thus, when

Version 1 is compared to the other versions, the comparison assesses the effect of the

additional probes on number of persons reported. Similarly, when we compare Version 2

to Version 3, the comparison re ̄ects the effect of requiring full names in Version 3. A major

issue for the analysis is whether asking the additional probes and relaxing the requirement

that full names be used increased the number of usual household members reported.

The data from the experiment are clustered both by segment and by interviewer.

Although neither the segments nor the interviewers constitute random selections from a

larger population, we sought to capture the clustering in the data by treating them as if

they were from a two-stage probability sample with segments as the ®rst-stage unit and

households as the second. We then reanalyzed the results, treating interviewers as the

®rst-stage units. We used SUDAAN to carry out both sets of analyses. The two sets of

results are quite similar, yielding the same conclusions in every case. This convergence

is not surprising since interviewer assignments generally encompassed only one or two

segments and each segment was worked only by one or two interviewers. As a result,

we report only the signi®cance tests from the analyses that treat the interviewers as the

®rst-stage unit by which the data are clustered. For the more exploratory analyses that

examine person-level data, we ignore the clustering of the observations.

There are several ways of comparing the number of persons rostered under the three

versions of the questionnaire. One is to compare the average number of total persons listed

per housing unit across versions. We would certainly expect to get larger averages when

the additional probe questions are included since the probes would add persons with casual

connections to the dwelling unit. Another method is to compare averages of usual residents

across versions. That is, we only count persons for whom we get an af®rmative answer to

the question ``Does NAME usually live here?'' This is a simple way of deciding who we

want to count as belonging to the unit, consistent with the practice of most federal surveys.

In Section 3.4, we examine several alternative rules for deciding who should be linked to a

given dwelling.

3.1. Persons rostered by version

Table 1 shows the average number of total persons and usual residents listed on each roster

by version of the questionnaire. For Version 1, the table displays the results only for those

persons listed in response to the standard roster questions (which were administered ®rst)

and excludes those added by the later probes.

Total persons listed. Versions 2 and 3, with their numerous experimental probes, netted

6 Journal of Of®cial Statistics
more total persons that Version 1. On the average, respondents listed about one additional

person per dwelling on these two versions (a mean of 2.90 for the standard questions

administered on Version 1 versus means of 3.94 and 3.87 for Versions 2 and 3). An

analysis of variance con®rmed that these differences by version were statistically

signi®cant ± F2; 37  15:46, p < :001. Additional analyses indicate that the differences

across versions in the total number of males F2; 37  8:09, p < :001, females

F2; 37  6:04, p < :01, Black males F2; 37  13:04, p < :001, and Black females

F2; 37  6:84, p < :01 were all statistically signi®cant as well. Post hoc comparisons

indicated that, in each case, Versions 2 and 3 differed signi®cantly from Version 1, but

not from each other.

The differences across versions in the total number of persons rostered are greatly

reduced if persons added by the later probes are included in the ®gures for Version 1.

The total number of persons per household rises to 3.67; the total number of males to

1.65 and the total number of females to 1.99 when these persons are included. Furthermore, the differences across versions in the total number of persons per household are

no longer signi®cant. This suggests that the order in which the special probes were

administered (either before the standard roster questions, as in Versions 2 and 3, or after

them, as in Version 1) had little effect on their ultimate yield.

Usual residents. It is not surprising that the probes increased the average number of

persons listed on the rosters, since the expanded rosters included dinner guests, overnight

visitors, and others with only tenuous connections to the household. The key issue is

whether the number of persons identi®ed as usual residents of the household differs by version. All three versions of the questionnaire contained an item to distinguish usual residents from others on the roster. In each version, respondents were asked ``Does NAME

usually live here?'' for each person they had listed on the roster. Responses to this item

were used to identify the usual residents. Again, with Version 1, we focus on persons identi®ed as usual residents in response to the standard roster questions.5

Overall, there is no evidence of differences among the three versions in the number of

Tourangeau et al.: Who Lives Here? Survey Undercoverage and Household Roster Questions 7

5 For the Version 1 households, only five additional usual residents were identified in response to the later probes,

yielding only .03 more usual residents per household. None of the conclusions are altered when these additional

usual residents are included in the figures for Version 1.

Table 1. Average number of total persons and usual residents rostered by version

Version 1 Version 2 Version 3

(n  173) (n  177) (n  159)

Total Usual Total Usual Total Usual

persons residents persons residents persons residents

Total 2.90 2.83 3.94 3.13 3.87 2.99

All males 1.28 1.25 1.86 1.47 1.70 1.25

All females 1.62 1.58 1.99 1.63 2.10 1.69

Black males 1.12 1.09 1.80 1.45 1.55 1.18

Black females 1.48 1.45 1.93 1.60 1.98 1.60

Note: A few persons were missing sex data; as a result, column entries may not sum to column totals. Figures for

Version 1 re ̄ect only persons listed using the standard rostering questions.
usual residents. Version 1 yielded an average of 2.83 per household if just the standard

items are considered, Version 2 an average of 3.13, and Version 3 an average of 2.99.

An analysis of variance indicated that the overall differences by version were not

signi®cant ± F2; 37  1:38 p > :25. However, when we focus on the Black males ±

the group known to have the highest rates of undercoverage ± the differences are more

striking. Version 2 produced the highest mean number of Black males listed as usual

residents ± 1.45 per household; the corresponding ®gures for Versions 1 and 3 are 1.09

and 1.18, respectively. Version 2, thus, produced an increase of 33% more Black males

reported as usual residents than did the standard rostering questions in Version 1. The

increase for Version 3 over Version 1 is only 8%. Across the three versions, the differences

in the mean number of Black male usual residents reported was signi®cant ±

F2; 37  5:01 p < :01. A post hoc comparison indicates that the results for Version

2 differed from those for Version 1.

These differences across versions in the number of usual residents reported did not

extend to Black females. The means are 1.45, 1.60, and 1.60 for the three versions (see

Table 1); an analysis of variance con®rms that these differences were not statistically

signi®cant ± F2; 37 < 1.

Additional analyses. The analyses reported so far examine the average counts of

different classes of persons by version. As is common with counts, the distribution of

the data was highly skewed, and it is possible that the analyses reported earlier gave

misleading results because of the nonnormal distribution of the observations. There are

a number of methods available for dealing with the skewness of the data. With counts,

transformations are sometimes applied. Winer (1971, p. 400), for example, recommends

taking a log transformation; .5 or 1 is often added to the raw counts ®rst to permit the transformation of cases with counts of zero. We reanalyzed the data, using the natural logarithm

of the counts (plus .5) instead of the raw counts.

There were no differences between the results from the analyses of the raw and the

transformed counts. For example, the analysis of the log number of Black males reported

as usual residents showed a signi®cant effect by version F2; 37  4:32, p < :05. The

results of post hoc comparisons were also unchanged when the log transformed counts

were analyzed in place of the raw counts.

The comparisons reported above examine total persons and usual residents. It is also

possible to compare the three versions of the questionnaire under different rules for linking

persons to dwellings; Section 3.4 reports some comparisons along these lines.

3.2. Characteristics of persons listed on the rosters

It is possible that the different versions of the roster questions netted not only different

numbers of persons but different types as well. Version 2, for instance, may capture a

higher proportion of persons with loose or unconventional ties to the sample dwelling.

We explored this possibility by comparing the characteristics of the usual residents listed

under each version, according to responses to the follow-up items. These items asked how

many nights each person stayed at the sample dwelling during the previous month, how

many days he or she ate there during the past week, whether the respondent considered

the person to be a member of the household, whether the person considered himself or

8 Journal of Of®cial Statistics
herself a household member, whether the person had other places where he or she stayed

frequently, whether he or she had been at the residence at all during the last day, and

whether he or she contributed money or other help to the household. Table 2 displays

the results of these comparisons.

The usual residents listed on the three versions do not appear to differ much. On all three

versions, the usual residents were persons who slept at the sample dwelling almost every

night, who ate there almost every day, and who were there some time during the last

day. Nearly all considered themselves household members and were in turn considered

household members by the respondent. The majority of those identi®ed as usual residents

(59.0%) also contributed money or some other form of help to the household. The usual

residents listed on the three versions of the questionnaire thus seem to have similar relationships to the sample dwellings. (Additional analyses showed that this same pattern held

when the Black males listed as usual residents were examined separately.) The same sorts

of persons were reported as usual residents on all three versions of the questionnaires; the

only difference was that on Version 2 more of them were reported.

3.3. Distinguishing usual residents from others listed

Altogether, the household rosters listed 1,949 persons, of whom 1,528 were identi®ed by

the household informant as usual residents of the dwelling. (For ten persons, the usual

residence variable was missing; these ten are dropped from Table 3 below.) What characteristics are associated with respondents' decisions to classify persons listed on the roster

as usual residents? To address this question, we examined responses to the item ``Does

NAME usually live here'' as a function of responses to the follow-up items. We ®t logistic

regression models to determine which of the follow-up items were signi®cantly related to

responses to the usual residence item. Because these analyses are somewhat more exploratory in nature, we have not tried to take into account the clustering of these person-level

data by household and by interviewer or segment.

We tested an additive model that included six of the follow-up items as predictors.6

Tourangeau et al.: Who Lives Here? Survey Undercoverage and Household Roster Questions 9

Table 2. Selected characteristics of usual residents by version

Total Version 1 Version 2 Version 3

Mean nights stayed at dwelling 29.5 (1,500) 29.3 (486) 29.5 (542) 29.5 (472)

Mean days ate at dwelling 6.78 (1,455) 6.60 (471) 6.79 (540) 6.79 (453)

Proportion contributing money,

other help 59.0% (1,498) 61.7% (483) 56.4% (546) 59.3% (469)

Proportion staying other places 3.5% (1,492) 2.7% (482) 3.1% (550) 4.8% (460)

Proportion there yesterday 98.0% (1,517) 97.6% (489) 98.6% (553) 97.7% (475)

Proportion considering self a

household member 99.7% (1,519) 99.2% (488) 100.0% (553) 100.0% (488)

Proportion considered a

household member 99.5% (1,506) 98.8% (488) 99.8% (543) 100.0% (475)

Note: Sample sizes given in parentheses; n's  ̄uctuate from item to item due to missing and out-of-range values.

Figures for Version 1 re ̄ect only persons listed using the standard rostering questions.

6 We dropped the follow-up item that asked whether the person considered himself or herself a household

member, because that item agreed more than 98% of the time with the item that asked whether the respondent

considered the person a household member.
Analyses of the data separately by version of the questionnaire yielded results similar to

those given in the combined analysis; as a result, we present only the latter here. This analysis examined all persons rostered under Version 1, including those ®rst reported in

response to the special probes administered after the standard roster questions. The additive model gave a good ®t to the data, with a likelihood ratio chi-square value of 1,580.4

(with 6 degrees of freedom).

Five of the six variables emerged from this analysis as signi®cant predictors of whether

the household respondent classi®ed a person as a usual resident ± the number of nights the

person had stayed in the last month (the logistic regression coef®cient for this variable was

0.192, with a standard error of 0.025); whether the person contributed money or other help

to the household (logistic regression coef®cient of 1.72, with a standard error of 0.501);

whether the person had other places where he or she stayed frequently (logistic regression

coef®cient of ÿ2:20, with a standard error of 0.448); whether the respondent considered

the person a member of the household (logistic regression coef®cient of 3.67 with a

standard error of 0.661); and whether the person had been at the dwelling in the last

day (logistic regression coef®cient of 1.19, with a standard error of .529). Only the number

of days the person had eaten at the sample dwelling in the past week failed to make a

10 Journal of Of®cial Statistics

Table 3. Persons listed as usual residents by selected characteristics

Variable % Identi®ed as n

usual residents

Total 78.8 1,939

Nights at dwelling in last month

Fewer than 7 6.8 368

7±29 78.2 119

30±31 99.5 1,398

Days ate at dwelling in last week

Fewer than 7 17.2 429

7 97.2 1,430

Contributes money or other help?

Yes 98.0 905

No 61.5 1,008

Considered a household member?

Yes 94.4 1,594

No 2.7 329

Considers self a household member?

Yes 94.7 1,520

No 2.7 331

There during last day?

Yes 91.4 1,634

No 11.0 301

Stays frequently at other places?

Yes 13.3 437

No 97.8 1,475

Note: Because of missing and out-of-range values, row entries sum to different totals. Responses to two of the

items (the number of nights the person stayed at the residence in the last month and the number of days the person

ate at the residence in the last week) have been grouped into categories.
signi®cant contribution to the classi®cation of persons as usual residents, despite a strong

bivariate relation. The results were very similar when we restricted this analysis to Black

males; the same ®ve variables again emerged as signi®cant predictors of whether the

Black male was classi®ed by the respondent as a usual resident of the dwelling. We stress

that these results are merely descriptive and do not necessarily represent the key variables

in the underlying cognitive process of the respondents.

Across all three versions of the questionnaire, people who spend the night at the

dwelling more often, who had no other place where they frequently stayed, who

contributed money or other support to the household, who considered themselves household members, and who had been at the dwelling within the last day were likely to be

labelled usual residents. Their counterparts who had slept there fewer nights, frequently

stayed elsewhere, did not contribute money, did not consider themselves household members, or did not visit the dwelling in the last day were much less likely to be identi®ed as

usual residents. Table 3 shows the percentage of persons identi®ed as usual residents conditional on values of the seven follow-up variables.7 All seven show a strong relationship

to whether the respondent classi®ed the person in question as a usual resident.

We did two further analyses concerning the classi®cation of persons as usual residents.

The ®rst analysis determined the number of persons who met a series of common sense

tests of residence. We determined the number of persons who spent at least 21 nights

out of the last 30 at the residence, ate at the residence every day in the last week, contributed money or other help to the household, had been at the residence some time during the

last day, considered themselves household members, and had no other place where they

stayed frequently. (We dropped the item asking whether the respondent considered the

person a household member since it was largely redundant with the one asking whether

the person in question considered himself or herself a member.) We counted how many

of these six tests each person met. Nearly 97% of those listed as usual residents ±

1,476 out of the 1,528 usual residents ± met at least four of these six criteria. In fact,

91% met at least ®ve of these criteria. By contrast, the vast majority of those who were

not classi®ed as usual residents ± 383 out of 411, or 93.2% ± met two or fewer of these

criteria. Thus, there seemed to be little room for confusion about how to classify almost

everyone who was listed on the rosters.

Our ®nal analysis attempted to provide a more direct estimate of the number of persons

in the sample whose living arrangements did not readily map on to the concept of a usual

residence. We classi®ed persons on the rosters by the number of other places they stayed

frequently and by the number of nights they had spent at the sample dwelling during the

prior month. Table 4 shows the results. The top panel displays persons classi®ed by the

household informant as usual residents of the sample dwelling. The bottom panel shows

the persons listed on the roster but not classi®ed as usual residents; persons in this bottom

Tourangeau et al.: Who Lives Here? Survey Undercoverage and Household Roster Questions 11

7 By contrast, the figures in Table 2 are conditional on a person's being classified as a usual resident. For

example, fewer than 60 of the 1,528 persons listed as usual residents of sample dwellings were reported to

stay elsewhere frequently. These persons represent only 3.5% of the usual residents (Table 2). These same

persons classified as usual residents make up 13.3% of the 437 persons listed on a roster who were reported

to stay somewhere else frequently. The majority of these 437 persons were overnight visitors, dinner guests, or

others with a limited connection to the sample dwelling who were added to the rosters in response to one of the

special probes.
group are at risk of undercoverage under the standard procedures for linking persons to

dwellings. Among the persons not counted as usual residents, at least three distinct groups

± encompassing a total of 63 persons ± seem to us potentially at risk of exclusion: (1) the

33 persons who were reported not to stay at any place frequently; (2) the 17 persons who

stayed at the sample dwelling eight or more nights in the prior month and who had one

other place they were said to stay frequently; and (3) the 13 persons said to stay at two

or more other places frequently. Members of these three groups either have no clear

link to any dwelling or links to multiple dwellings. Still, most of those not counted as usual

residents do not appear very problematic. For example, there are 234 persons who had

stayed at the sample dwelling for a week or less and who had just one other place where

they stayed frequently; it seems reasonable to assume that this other place is their home.

Among those grouped in the top panel of the table, 59 were said to stay frequently at at

least one other dwelling. It is possible that such persons also have unconventional living

arrangements; however, because these persons were counted as usual residents of the

sample dwelling they are at risk not of being excluded but of being linked to more than

one dwelling.

3.4. Other rules for linking persons to dwellings

If there are substantial numbers of persons who split their time among several dwellings,

the ``usual resident'' question may not be the best way to determine which people should

be linked to which dwelling. It is possible that the plurality rule (linking persons to the

dwelling where they stay most often) is still generally workable, but that it could be

operationalized more accurately with other questions than with the usual resident question.

For example, more speci®c questions (such as the probes we used) might bring in persons

who meet the plurality rule but who are not considered usual residents by the household

informant. More speci®c questions might also improve coverage when informants are

reluctant to acknowledge someone as a usual resident but would be willing to divulge

enough information to allow the persons to be linked to the dwelling. It is also possible

that the plurality rule simply does not cover a range of living situations and that alternative

rules need to be developed for linking persons to dwellings.

12 Journal of Of®cial Statistics

Table 4. Usual residents and other persons by other places and nights at sample dwelling

Nights at sample dwelling in last month Total

7 or fewer 8±15 16±23 24±31 missing

Usual residents

No other places 21 8 17 1,370 17 1,433

One or more other places 17 6 9 26 1 59

Number of other places missing 4 4 1 26 1 36

Not usual residents

No other places 25 1 ± 3 4 33

One other place 234 12 2 3 13 264

Two or more other places 12 1 ± ± ± 13

Number of other places missing 76 3 3 1 18 101
We examined two new methods for operationalizing the plurality rule, as well as two

alternatives to that rule. In an earlier version of this article (Kearney, Tourangeau, Shapiro,

and Ernst 1993), we also examined a more complex procedure for implementing the plurality rule. That analysis was hampered because the questionnaire did not include all the

items needed to classify each sample person under that procedure. In this article, we focus

on relatively simple methods for classifying persons that can be carried out using the items

at hand. Aside from the ``usual resident'' question, we examined two other plausible

methods for operationalizing the plurality rule. One method counted as a resident anyone

who had stayed at the sample dwelling at least 16 nights during the past month; persons

meeting this test had spent more nights at the sample dwelling than any place else during

the month. The other method liberalized the ®rst test somewhat, counting both those who

stayed at the sample dwelling for 16 or more nights and those who had no other place they

stayed frequently. In addition, we examined two rules conceived as alternatives to the

plurality rule. The ®rst counted all persons who stayed at the dwelling the previous night,

a rule sometimes referred to as the de facto rule. (The items needed to assess this rule were

included only in Versions 2 and 3 of the questionnaire.) The other alternative rule linked to

the dwelling all persons who had eaten there at least four days out of the last seven. Each of

these procedures has potential problems. For example, although the de facto rule is quite

simple, it will exclude persons who happened not to have spent the night anywhere (such

as the homeless or persons in transit), and it is poorly suited to many analytical purposes,

because it links people to dwellings to which they have only a temporary connection. Our

analysis examines only the number of persons captured by each method and the extent that

the various alternative methods give results that agree with the classi®cations under

standard method based on the ``usual resident'' item.

Table 5 shows the mean number of persons linked to each dwelling under the different

Tourangeau et al.: Who Lives Here? Survey Undercoverage and Household Roster Questions 13

Table 5. Mean number of persons per dwelling by rule and version

Version 1 Version 2 Version 3 Total

n  173 n  177 n  159 n  509

All persons

Plurality rule

Usually lives here 2.83 3.13 2.99 2.98

16 or more nights 2.83 2.99 2.93 2.95

No other place or 16 nights 3.01 3.34 3.30 3.22

Stayed last night ± 3.08 3.05 3.07

n  336

Ate 4 or more days in last week 2.86 3.16 2.98 3.01

Males only

Plurality rule

Usually lives here 1.28 1.47 1.25 1.34

16 or more nights 1.23 1.43 1.20 1.29

No other place or 16 nights 1.32 1.55 1.39 1.42

Stayed last night ± 1.47 1.25 1.37

n  336

Ate 4 days in last week 1.26 1.50 1.26 1.35
methods. The top panel displays the results for all persons and the bottom panel displays

the results for males only (who are at greater risk of undercoverage under the standard

procedure). Only the relatively liberal method which assigned to the sample dwelling

anyone who stayed there at least 16 nights during the past month or who had no place

else where they stayed frequently consistently captures more people than the standard

classi®cation; overall, the liberal method includes about 8% more people than the standard

method (an average of 3.22 per household versus 2.98). The de facto rule is the second

most inclusive, linking an average of 3.05 persons to each sample dwelling. In general,

though, the results do not differ dramatically by method and, regardless of the linking

procedure used, Version 2 consistently netted more persons than the other two versions.

In addition, the alternative procedures tend to classify individuals in the same way as

the standard method. The rate of agreement with the classi®cation under the standard

method ranged from a low of 93.9% for the rule based on the number of days the person

ate at the sample dwelling to a high 96.4% for the most inclusive procedure. The high level

of agreement among the classi®cation procedures is consistent with our earlier ®nding that

the vast majority of persons counted by the informants as usual residents met a variety of

common sense tests of residency and that most persons not counted as usual residents fail

most of these same tests.

3.5. Obtaining the full names of persons listed on Version 2

Version 2 may yield more complete reporting of Black males than the other two versions

but it does so at a price. In Version 2, residents were not necessarily identi®ed by their full

names on the roster. At the end of the interview, however, respondents were asked to provide full names for everyone listed on the roster in case it was necessary to recontact them.

Respondents provided full names for 521 of the 697 persons listed on the Version 2

rosters (74.7%) and for 448 of the 554 usual residents (80.9%).

Most respondents to the Version 2 questionnaire reported names for everyone on the

roster. Full names were provided for everyone on the roster in 72.2% of the questionnaires.

Full names were given for at least one of the persons listed in an additional 13.1% of the

cases. The corresponding ®gures for usual residents are somewhat higher; respondents

provided a full name for every usual resident listed on 79.0% of questionnaires and for

at least some of the usual residents on another 6.2%. However, 14.8% of the Version 2

respondents provided full names for none of the persons listed. In total, 114 persons (85

of them usual residents) were listed on rosters on which no full names were provided.

The omission of all names within a residence would make it dif®cult to track members

of the sample over time.

It may be possible to obtain full names for a higher proportion of the sample while still

achieving improved coverage. The Version 2 wording was: ``To help us keep track of

which persons we're talking about, can you identify each one with a ®rst name, a nickname, a set of initials, or by some other means? It is not necessary to give us full names.''

Other wording could result in more full names: ``To help us keep track of which persons

we're talking about, can you give us the full name of each person? If you would rather not

give us full names, you can identify each one with a ®rst name, a nickname, a set of initials,

or by some other means.'' This type of wording, of course, might not achieve the same

14 Journal of Of®cial Statistics
coverage gains. Another way of obtaining full names for a higher proportion of the sample

would be to offer the option of less than full name in only those geographic areas or types

of households where coverage is believed to be a particular problem.

4. Conclusions

The anonymous rostering procedure certainly showed promise. Version 2, which did not

require full names, yielded 33% more Black males listed as usual residents than the

standard questions in Version 1 did. The magnitude of this difference seems reasonable

given published estimates of the level of undercoverage in surveys and the decennial

census. For example, Hogan (1993) estimates undercoverage of approximately 5% among

Blacks of both sexes in the 1990 census; Hainer and his colleagues (Hainer et al., 1988)

estimate that an additional 17% of Black males are omitted from the Current Population

Survey. The areas used in our study were speci®cally chosen because they were likely to

exhibit high rates of undercoverage.

The apparent improvement in the coverage of Black males does not seem to result from

the extra probes for additional household members that were included in both Versions 2

and 3. These probes do indeed add persons to the rosters ± on average about 1 person per

dwelling (see Table 1). But few of the persons added by the new probes are classi®ed by

the household respondent as usual residents of the dwelling. What does make a difference

in the reported number of usual residents is an anonymous questionnaire, which was a

feature of Version 2 only. However, since Version 2 included the extra probes as well,

we cannot be sure that anonymity by itself would achieve signi®cant coverage gains.

The results involving the number of usual residents do not seem to be statistical artifacts

due to the nonnormal distribution of the data or the clustering of the observations by

segment and interviewer. It is, of course, possible that the results would not generalize

beyond this purposively selected sample; but, within a sample of areas chosen to be

representative of areas where coverage problems are most serious, rosters compiled

without requiring full names appear to improve coverage substantially.

At the outset, we distinguished two broad hypotheses about the sources of coverage

error within households. Researchers have argued that respondents may be unclear about

who should be included on the household roster and that they may deliberately conceal

household members to avoid legal or ®nancial repercussions. The confusion about whether

a given person should be included is thought to re ̄ect living arrangements ± particularly

those of minority families in inner city areas ± in which persons spend time at several

dwellings. Our results support an account based on concealment rather than confusion.

Work by Singer, Mathiowetz, and Couper (1993) suggests that the con®dentiality concerns of potential survey respondents are distinct from their privacy concerns; not requiring the use of full names would appear to address both sets of concerns (see also Kulka,

Holt, Carter, and Dowd 1991).

The results also suggest that decisions about who to count as a usual resident may be

based on relatively simple criteria. The respondents' classi®cation of the persons on the

roster as usual residents could be predicted from ®ve variables ± how often the person had spent the night during the past month, whether he or she had been at the

dwelling in the last day, whether there were other places the person stayed frequently,

Tourangeau et al.: Who Lives Here? Survey Undercoverage and Household Roster Questions 15
whether the person contributed money or other support to the household, and whether

the respondent thought of the person as a member of the household. These variables

correlated highly with each other and with the classi®cation of a person as a usual

resident.

More generally, we can identify only a few persons with unconventional living arrangements that might engender omissions from the roster. Of the more than 1,500 persons

listed as usual residents, only 59 ± less than 4% of the total ± were reported to stay

frequently at some dwelling other than the sample dwelling; of these 59 persons, 49

were said to stay often at just one other residence. This same pattern is apparent for the

persons listed on the rosters but not classi®ed as usual residents of the sample dwelling;

the vast majority of these persons were said to stay in only one other residence (see Table

4). Adding in other potentially problematic cases brings the estimated total at risk for

undercoverage to 63. It is not clear that any of these relatively ambiguous cases were

misclassi®ed by the respondents with respect to their usual residences. Even if all 63 of

these persons should have been counted as usual residents of the sample dwelling (according

to the plurality rule), it would have increased the mean number of residents per dwelling

only by 5%. The results are similar if we restrict the analysis to Black males ± counting all

the problematic cases would increase the total number of Black male residents per

dwelling by slightly more than 5%. By contrast, anonymous rostering added about 30%

more Black male residents per dwelling (see Table 1). These results lend further support to the hypothesis that concealment is a major cause of within-household undercoverage. The results in Table 5 nonetheless support the notion that new rules for

linking persons to dwellings based on more speci®c questions might also improve

coverage.

As a practical matter, it may seem silly to contemplate using anonymous rostering in

many survey applications, especially ones involving longitudinal data collection. For

several reasons, we believe that our results still have important practical implications.

First, they demonstrate that roster questions ± which have been thought to be relatively

innocuous ± can in fact be taken as threatening by some respondents. Thus, procedures

used to reduce threat with other sensitive questions may be useful with roster questions

as well. In addition, efforts to improve coverage that focus solely on the clarity of the

standard of residence may fail, because they ignore a major source of within-household

undercoverage. Second, many surveys do not in fact require the full names of the

respondents; for example, in telephone surveys, individual respondents are often selected

through methods that do not require listing every household member by name. Clearly, the

anonymous rostering procedure described here may be quite applicable in cross-sectional

face to face surveys. Third, even in the context of a longitudinal design, there may be

advantages to obtaining full names at the end of the initial interview rather than at

the beginning. It is customary to delay asking sensitive questions until the end of the

questionnaire ± after the interviewer has had a chance to develop rapport with the

respondent and to demonstrate the legitimacy of the survey. We know of no surveys

that begin the interview by asking about household income or sexual practices. For

some households, the full names of the household members may be an equally sensitive

subject.

16 Journal of Of®cial Statistics
5. References

Brownrigg, L. and Martin, E. (1993). Proposed Study Plan for Ethnographic Evaluation of

the Behavioral Causes of Undercount. Paper prepared for the Census Advisory

Committee on Population Statistics at the Joint Advisory Committee Meeting, April

13±14, Alexandria, VA.

Cantor, D. and Edwards, C. (1992). Testing Alternative Household Roster Questions for

the Survey of Income and Program Participation. Unpublished report submitted to

the U.S. Bureau of the Census.

Fay, R.E. (1989). An Analysis of Within-household Undercoverage in the Current

Population Survey. Proceedings of the U.S. Bureau of the Census Annual Research

Conference, Washington, DC: U.S. Bureau of the Census, 156±175.

Fein, D.J. (1990). Racial and Ethnic Differences in U.S. Census Omission Rates. Demography, 27, 285±302.

Fein, D.J. and West, K.K. (1988). Towards a Theory of Coverage Error: An Exploratory

Assessment of Data from the 1986 Los Angeles Test Census. Proceedings of the U.S.

Bureau of the Census Annual Research Conference, Washington, DC: U.S. Bureau of

the Census, 540±562.

Gerber, E.R. (1990). Calculating Residence: A Cognitive Approach to Household

Membership Judgments Among Low Income Blacks. Unpublished report submitted

to the U.S. Bureau of the Census.

Hainer, P., Hines, C., Martin, E., and Shapiro, G. (1988). Research on Improving Coverage

in Household Surveys. Proceedings of the U.S. Bureau of the Census Annual Research

Conference, Washington, DC: U.S. Bureau of the Census, 513±539.

Hogan, H. (1993). The 1990 Post-Enumeration Survey: Operations and Results. Journal of

the American Statistical Association, 88, 1047±1060.

Kearney, A., Tourangeau, R., Shapiro, G., and Ernst, L. (1993). Coverage Improvement

from Experimental Roster Questions. Proceedings of the Section on Survey Research

Methods, American Statistical Association, 162±167.

Kulka, R., Holt, N., Carter, W., and Dowd, K. (1991). Self Reports of Time Pressures,

Concerns for Privacy and Participation in the 1990 Mail Census. Proceedings of the

U.S. Bureau of the Census Annual Research Conference, Washington, DC: U.S. Bureau

of the Census.

Martin, E. and Polivka, A.E. (1992). The Effect of Questionnaire Redesign on Conceptual

Problems in the Current Population Survey. Proceedings of the Section on Survey

Research Methods, American Statistical Association: 655±660.

McKay, R.B. (1992). Cultural Factors Affecting Within Household Coverage and Proxy

Reporting in Hispanic (Salvadoran) Households. A Pilot Study. Proceedings of the

Section of Survey Research Methods, American Statistical Association, 614±618.

Robinson, J.G., Ahmed, B., Gupta, P.D., and Woodrow, K. (1993). Estimation of

Population Coverage in the 1990 United States Census Based on Demographic Analysis. Journal of the American Statistical Association, 88, 1061±1071.

Shapiro, G., Diffendal, G., and Cantor, D. (1993). Survey Undercoverage: Major Causes

and New Estimates of Magnitude. Proceedings of the U.S. Bureau of the Census Annual

Research Conference, Washington, DC: U.S. Bureau of the Census.

Tourangeau et al.: Who Lives Here? Survey Undercoverage and Household Roster Questions 17
Singer, E., Mathiowetz, N., and Couper, M. (1993). Privacy, Con®dentiality, and the 1990

U.S. Census. Public Opinion Quarterly, 57, 465±483.

Valentine, C.A. and Valentine, B.L. (1971). Missing Men: A Comparative Methodological

Study of Underenumeration and Related Problems. Contract No. 1510. Washington, DC:

U.S. Bureau of the Census.

Winer, B. (1971). Statistical Principles in Experimental Design. 2nd Edition. New York:

McGraw Hill.

Received April 1994

Revised October 1995

18 Journal of Of®cial Statistics