1 

Differing Person Nonresponse Rate by Interview Length and 

Respondent Characteristics: Results from the 2011 SIPP-EHC Field 

Test 

Dan Perez-Lopez and Rachael Walsh, PhD1

U.S. Census Bureau 

Daniel.Perez-Lopez@census.gov Rachael.Walsh@census.gov

This paper evaluates interview-length paradata from the 2011 test of the re-engineered Survey of Income and Program 

Participation (SIPP-EHC) to explore the incidence of person-level non-response in interviewed households. The results2 suggest 

that non-response is influenced by the length of the first interviewed person’s interview, with longer first interviews being 

associated with subsequent person non-response. Logistic regression is used to model the odds of person non-response 

considering interviewer-level, first-person interview-length, household characteristic, and non-respondent characteristic effects at 

the household- and person-levels. This analysis suggests that interview length has an influence similar in sign and magnitude on 

the incidence of person-level response in both household-level and person-level analyses. 

Introduction 

The Census Bureau is re-engineering the Survey of Income and Program Participation (SIPP). The SIPP is a 

longitudinal household survey in which data is collected from households three times a year, typically over four 

years. Beginning in 2014, the re-engineered SIPP (SIPP-EHC) will collect data from households only once a year. 

It will maintain the current four year panel length and utilize an event history calendar (EHC) to assist respondent 

recall and maintain data quality. 

In 2010 and 2011 the Census Bureau field tested two different versions of the SIPP-EHC instrument. The Bureau is 

in the process of evaluating and improving the SIPP-EHC instrument. Data collected in these two pilots is being 

compared to data collected by the current instrument, and the interviewer procedures and software used are also 

being evaluated. In the process of this evaluation, it was observed that the rate of non-response of individuals within 

households where at least some members completed the interview was higher in the 2010 field test than in a 

comparable sample of households interviewed using the current instrument (SIPP-EHC Data Evaluation Workgroup, 

2011.) This did not come as a complete surprise to the redesign team. While a single annual interview can seem 

less burdensome to respondents than thrice-yearly data collection, interviews performed with the annual SIPP-EHC 

instrument can be longer than the interviews using the current instrument. 

This paper combines interview length information from paradata, interviewer characteristics, and respondentsupplied data from the 2011 pilot to analyze the factors associated with person-level non-response. Our hypothesis 

is that the interview length of the first person interviewed has a strong effect on the odds of later household 

members’ decision to participate, with longer first person interviews being associated with higher odds of observed 

 

1

 Any views expressed are those of the authors and not necessarily those of the U.S. Census Bureau. 

2

The proportions and other statistics in the text and tables of this report describe the samples considered here for the 

purpose of evaluating the 2011 SIPP-EHC Field Test. The weighting and design effects necessary to interpret the 

reported statistics as estimates of underlying population parameters have not been incorporated into these results. 

Apparent differences may not be statistically significant, but all comparative statements in this report have 

undergone statistical testing and are significant at the 90% confidence level. 
2 

non-response in the household. In the process of our analysis, we also consider the effects of interviewer 

experience, as well as the effects of demographic and social characteristics of the respondents themselves. 

Data 

The SIPP-EHC pilot conducted in the spring of 2011 had a sample size of 4,051 households, and interviewed 

individuals using interviewers from all 12 regional offices around the country. The pilot over-sampled households 

in low-income strata with a sample design otherwise identical to the one used for the current SIPP. The data 

collected cover a wide range of demographic and socio-economic topics. 

The interview begins by asking the first respondent to list, or roster, all of the people who usually live at the housing 

unit and to give basic demographic data on all household members. This structure allows us to have age, sex, race, 

ethnicity, and educational attainment about all household members, even if they subsequently decline to participate. 

The instrument is administered in person by an interviewer, or field representative, on a laptop computer. The 

software collects data on the time it takes to complete each item in the interview, which we use in this analysis. The 

time to complete each person’s interview is calculated, and the total household interview time is also calculated. In 

addition, information about interviewers is available, including whether they were a senior interviewer, the number 

of years of Census interviewing experience they had, and the number of years of SIPP interviewing experience they 

had. 

These demographic data for all respondents and non-respondents, time of interview data, and interviewer data were 

combined with detailed data from the core section of the SIPP-EHC instrument from respondents to form the data 

set used in this paper. Since we are interested in the response behavior of adults, the persons in the analysis were 

restricted to age 15 and above. Since we are interested in the relationship between the length of the first interview 

and subsequent respondents’ behavior, we further restricted our analysis to households with at least two adults. Of 

course, to examine this relationship, we also need an adult in the household to have completed the interview, or at a 

minimum, have completed a sufficient partial interview. A sufficient partial interview is achieved when a 

respondent has rostered the household members and has completed at least the EHC portion of the survey. We 

restricted ourselves to households that had at least one sufficient partial. With these restrictions in place, we were 

ready to begin our analysis. 

Descriptive Statistics

After applying the restrictions described in the paragraph above, the 4,051 interviewed households were reduced to 

1,809 suitable for analysis. These 1,809 households contained a total of 4,528 adults. 517 of these adults were nonrespondents, and 287 households contained at least one non-respondent. 

If we consider the length of the interview of the first person interviewed, we can look at the household-level column 

of Table 1, since the household-level data is taken from the first person interviewed. We see that 27% of first person 

interviews were shorter than 30 minutes, and about three-quarters of first person interviews were shorter than 60 

minutes (74.7%). Ninety-seven percent of first-interviewed respondents completed the entire interview. A larger 

share of these first persons interviewed were 35 or older than the other adult members of the household, whose data 

is reflected in the column labeled “Potential non-respondents, first person removed.” This is not surprising, as 

children over 15 are included in this analysis, but their parents would be more likely to be the first person 

interviewed, even if all household members were present. While the adults interviewed were more likely to be 

female than male (52.6% vs 47.4%) the first interviewed people were even more likely to be female than the overall 

adult respondents (59.4%). One-quarter of the adults in this analysis were Black, 6% were Asian, and 23% reported 

some other race, or more than one race. 44% of the adults interviewed were Hispanic. First-interviewed persons 

had a higher educational attainment (defined as some college or BA or higher) than potential non-respondents, first 

person removed. However, this is not surprising, as the potential non-respondent group would tend to include adult 

children who, while 15 or older, may not have finished high school yet, let alone any college. 

If we consider Table 2, we see interview-level characteristics. Over one-quarter (27.6%) of interviewed households 

participated in some social welfare program (SSI, WIC, Food Stamps, TANF, or general assistance). Over one-third 

of residences (36%) were owner-occupied, and 40% of households were of size four or more. Turning to the 

column titled “All adults living in households with at least 2 adults” we see that senior field interviewers conducted 

about 6% of the interviews in the analysis. More than half of all interviews (56.1%) were conducted by field 
3 

representatives, or interviewers, with one year of Census interviewing experience or less. About 15% of interviews 

were conducted by interviewers with 5 years of Census interviewing experience or more. About two-fifths of all 

interviews were conducted by interviewers with previous SIPP interviewing experience. The interviews were 

conducted by all twelve regional offices, but Los Angeles accounted for about 27% of all interviews conducted, and 

Dallas for about 14%. The regional office that contributed the fewest number of interviews to this analysis was 

Denver, with about 2%. 

Household-Level Analysis

For our household-level analysis, the dependent variable was whether at least one person-level non-response was 

observed in a given household. We were interested in this type of analysis as a way of responding to reviewers’ 

comments about concerns with collinearity in our person-level analysis presented below. Essentially, in that 

analysis we consider non-first persons as if they are all independent, but of course some of these people are in the 

same households with others, so interview order and respondent fatigue may affect the relationships we’re trying to 

examine. For the household-level analysis, our dependent variable, “Was any adult in the household a nonrespondent?” allows us to examine first person and interviewer-level data and to ignore issues of interview order. 

There were 1,809 households, 287 of which had at least one non-respondent, and 1,522 of which did not. If we first 

specify a model (Model 1, Table 3) of just three measures of the length of the first person’s interview, more than 30 

minutes but less than 60, more than 60 but less than 90, and over 90 minutes, with interviews of 30 minutes or fewer 

as our reference group, we find that all three are significant, with longer first person interviews being associated 

with a higher likelihood of some person non-response at some point in the household roster. It is interesting to note, 

however, that for interviews of more than 90 minutes the odds of an observed non-response are 260% larger than for 

interviews shorter than 30 minutes. Households with a first person interview length of either 30 minutes up to 60 

minutes, or 60 minutes up to 90 minutes have 70% higher odds of any non-respondent in their household than 

households with shorter first interviews. 

If we specify a similar model, Model 1A (Table 3), but change it to include the “first person completed full 

interview” variable, we find the coefficients had similar patterns in magnitude and direction, and all remain 

significant. The completed interview variable is also significant, and has a large magnitude. 

Model 2, in Table 4, evaluates the effect of interviewer and regional office attributes and the length of the first 

person’s interview on the likelihood of observing at least one adult non-respondent in a given household. The 

effects of the length of the first person’s interview are largely similar to those in all other analyses, in both sign and 

magnitude. Senior Field Interviewers exhibit a strong association with avoiding non-response within the household, 

and years of SIPP experience do as well, although not as strongly. Larger households are associated with observing 

at least one person non-response, as is a high share of interviews in the household conducted by proxy. 

Although at the household-level we can’t look at the demographic characteristics of potential non-respondents, we 

can specify a model that takes into account the characteristics of the first person interviewed in each household. 

Model 3 in Table 5 evaluates the effect of these first person-level demographic variables and the length of the first 

person’s interview on the likelihood of observing at least one adult non-respondent in a given household. In this 

model, the effects of first person interview length are virtually unchanged from those in Model 1A, displayed on the 

left. The odds of a non-response in the household where the first person is male or Black are 31% and 40% lower

than for households where the first person is female or White, respectively. Several of the first person 

characteristics variables are not significant in predicting non-response at some point in the household interview. 

Owner-occupied housing units have 46% smaller odds of observing any non-response, at the unit level. 

Model 4 takes advantage of the previous three models and incorporates statistically significant variables from them 

to construct a model, as presented in Table 6. Almost all of the parameters are significant with similar signs and 

magnitudes as in the previous three models. 

Person-Level Analysis 

In order to further evaluate the effect of the first person’s interview on non-response by other household members, 

we also did a person-level analysis. In order to look at the likelihood of any person responding or choosing not to 

respond we constructed a dataset in which we appended first-person interview and demographic characteristics to all 
4 

household members, and then removed those first person respondents. Since we have required the first person in 

each household to have responded for entry into our analysis, we can’t consider those first people in our nonresponse analysis. After this restriction, our “no first person” dataset had 2,719 people, 2,202 of whom responded, 

and 517 of whom did not respond. 

Logistic regression was used to model, for each individual, their likelihood of non-response, where non-respondents 

were assigned a value of 1, and respondents a value of 0. We specified several models, all with this same dependent 

variable. In our first model, we look only at the effect of the length of the first person’s interview as reflected 

through four dichotomous variables for interview length in minutes. The first of these was for interviews less than 

thirty minutes, the next for interviews greater than or equal to 30 minutes, but shorter than 60 minutes, a similar one 

for interviews between 60 and 90 minutes, and finally one for interviews 90 minutes or longer. As in Tables 3-6, we 

use interviews 30 minutes or shorter as our reference category. We also construct a dummy variable for whether the 

first person’s interview was a complete interview or only a sufficient partial, with the assumption that complete 

interviews and sufficient partials affect subsequent respondents differently. As shown in Table 7, all coefficients are 

significant, and the log odds ratios show longer interviews increasing the odds of non-response and having a 

stronger effect the longer that first interview is. 

In our second model, we consider interview-level effects, such as the regional office associated with the interview, 

the interviewer’s years of interviewing experience, whether the interviewer is a senior interviewer, the size of the 

household, and the length of the first person’s interview. We also construct a variable for the share of household 

members interviewed by proxy. The SIPP-EHC allows respondents to give interviews by proxy for household 

members who are unable to participate. However, we feel there may be some relationship between non-response 

and the number of proxied interviews, perhaps as a high number of proxies would indicate a household’s lack of 

enthusiasm for participation. Alternatively, one respondent may be answering for an entire household, giving a self 

interview followed by several proxy interviews. If this first person becomes unwilling to continue providing 

responses for the every member of the household, it would result in non-response by other household members that 

is primarily caused by respondent fatigue, and not by characteristics of the non-interviewed person. When we look 

at Model 2, also in Table 7, we see that first person interview length remains significant. Household size is 

significant, and senior field representative status was significant and negative, so that respondents interviewed by an 

SFR had odds of non-response that were 80% smaller than respondents who interacted with other interviewers. 

Familiarity with the SIPP instrument was shown to be helpful in reducing non-response, as for every additional year 

of interviewer experience with the SIPP, respondents’ odds of responding to the interview were 55% higher. Larger 

household sizes were associated with higher odds of non-response, and the share of household members who were 

proxied for was also positively associated with non-response. 

Our third model evaluated the demographic characteristics collected in the household roster section at the beginning 

of the SIPP-EHC instrument. We thought it would be interesting to try to predict the likelihood of non-response by 

using the attributes of the potential non-respondents themselves, together with the length of the first person’s 

interview, first person interview complete, and share of household members proxied variables. The results are 

presented in Table 8, Model 3. The first person’s interview length is still significant and behaves the same way as in 

previous models described. Men had better odds of responding to the survey than women. Respondents who were 

of a race other than Black, Asian, or White had higher odds of response than the reference group, White 

respondents. Blacks had odds of responding that were 31% higher than Whites. When compared to high school 

graduates, those with less than a high school degree were more likely to be non-respondents. Respondents in 

households where a language other than English was spoken had higher odds of non-response. Participation in a 

social program (SSI, WIC, Food Stamps, TANF, or general assistance) was strongly associated with non-response. 

The first person interview complete variable is highly significant, and positive, such that completion of the first 

interview is highly associated with non-response later in the household. This outcome bears further investigation. 

After considering several models, we specify a model that uses significant variables from the earlier models, and 

includes geographic controls. This results of this model are presented in Table 9. The first person interview length 

is significant, with longer interviews being associated with a higher likelihood of non-response. The first person 

complete interview variable also remained significant. The share of household interviews conducted by proxy, 

number of people in the household, and participation in a social program were all associated with higher odds of 

non-response, as were educational attainment less than high school and speaking a language other than English at 

home. Meanwhile, interviews conducted by a senior field representative and by interviewers with previous SIPP 
5 

experience had lower odds of ending in non-response. Interviews with potential non-respondents who were male or 

who were of a race other than Black, Asian, or White also had lower odds of ending in non-response when 

compared to female and White respondents, respectively. 

Conclusions/Future Research 

The household- and person-level analyses support our hypothesis that non-response can be predicted by the length 

of the first person’s interview. Regional office variations were significant in the odds of non-response, in both the 

person- and household-level analyses. Over several analyses, most race and education variables did not seem to 

have a significant effect in predicting non-response. At the household- and person-levels, senior field interviewers 

had a significant negative effect on the odds of non-response occurring, as did interviewer experience with the SIPP 

instrument. Larger households had higher odds of observing non-response in the household, compared to smaller 

households. These results were expected, as senior field interviewers attain that status in part because they are better 

convincers than average interviewers. Meanwhile, larger households give more opportunities for refusal and for 

inability to locate respondents for interview, which can be independent of interview length. For similar interview 

lengths across respondents, the more respondents there are in a household, the longer the aggregate amount time 

spent, and we can expect this to increase the odds of some non-response. 

The results of the household analysis, Model 1, suggest that the effect of first person interview length on nonresponse is not linear, and that there may be a saturation point before which the odds of observing a non-response in 

the household is increasing slowly, and after which the odds of observation increase dramatically. In regards to 

future research, some next steps we plan to undertake involve doing a survival analysis to see if we can identify a 

saturation point, as well as a look into the effect of proxy interviews and their relationship to non-response. 

Although the person-level analyses are interesting, these results should be considered with caution. The dependent 

nature of the non-response behavior across household members suggests that the correct frame for analysis is a 

hierarchical model with persons nested in households, so that household- and person-level effects can be quantified. 

This will be the focus of our work in the spring of 2012. 

References 

Brennan, Mike and Hoek, Janet (1992). The Behavior of Respondents, Nonrespondents, and Refusers Across Mail 

Surveys. Public Opinion Quarterly, 56(4) pp. 530-535. Available at http://www.jstor.org/stable/2749208 . 

Campanelli, P. & O’Muircheartaigh, P. (2002). The Importance of Experimental Control in Testing the Impact of 

Interviewer Continuity on Panel Survey Nonresponse. Quality & Quantity, 33, 129-144.

Durrant, Gabriele B. and Steele, Fiona. (2009) Multilevel modeling of refusal and non-contact in household surveys: 

evidence from six UK Government surveys. Journal of the Royal Statistical Society. Series A (Statistics in Society), 

172(2) pp 361-381. Available at http://onlinelibrary.wiley.com/doi/10.1111/j.1467-985X.2008.00565.x/full

Groves,Robert M. and Cialdini, Robert B., and Couper, Mick P. (1992). Understanding The Decision to Participate 

in a Survey. The Public Opinion Quarterly, 56(4)pp. 475-495. Available at http://www.jstor.org/stable/2749203 . 

Groves, Robert M. and Steven G. Heeringa. (2006). Responsive Design for Household Surveys: Tools for Actively 

Controlling Survey Errors and Costs. Journal of the Royal Statistical Society. Series A (Statistics in Society), 

169(3) pp439-457. Available at http://www.jstor.org/stable/3877429 . 

Lipps, Oliver and Pollien, Alexandre. (2011). Effects of Interviewer Experience on Components of Nonresponse in 

the European Social Survey. Field Methods,23(2), 156-172. Available at http://fmx.sagepub.com/content/23/2/156. 

O'Muircheartaigh, Colm and Campanelli, Pamela (1999). A Multilevel Exploration of the Role of Interviewers in 

Survey Non-Response. Journal of the Royal Statistical Society, Vol. 162, No. 3, 437-446. Available at 

http://www.jstor.org/stable/pdfplus/2680489.pdf

Schatteman, Tom. (2004). Do Interviewer Characteristics Influcence Respondents’ Participation In Panel Surveys? 

An Analysis on a Belgian study of school-to-work transitions. Draft paper, available at 

http://www.ccsr.ac.uk/methods/events/attrition/documents/Schatterman.pdf
6 

Shtatland, Ernest S and Kleinman, Ken and Cain, Emily M. (2003). Stepwise Methods in Using SAS Proc Logistic 

and SAS Enterprise Miner for Prediction. SAS Users Group International Proceedings. Available at 

http://www2.sas.com/proceedings/sugi28/258-28.pdf. 

SIPP-EHC Data Evaluation Workgroup. (2011). An Initial Evaluation of the 2010 Field Test of Re-Engineered 

SIPP. SEHSD Working Paper Number 2011-9, U.S. Census Bureau, available at 

http://www.census.gov/sipp/DEWS/sippehc_eval.pdf

Snijders, Tom. Multilevel Analysis. 673-677 in M. Lewis-Beck, A.E. Bryman, and T.F. Liao (eds.), The SAGE 

Encyclopedia of Social Science Research Methods (Volume II). Sage, 2003. Available at 
7 
8 
9 
10
11
12
13
14
15