Planning an Adaptive Design Treatment in 2020 Census Tests 

Gina Walejko, Peter Miller, and Kevin Deardorff 

U.S. Census Bureau 

4600 Silver Hill Road 

Washington, DC 20233 

Proceedings of the 2013 Federal Committee on Statistical Methodology (FCSM) Research Conference 

Introduction 

Decennial Census field costs for the nonresponse followup operation (NRFU) in 2010 totaled nearly $1.6 billion. To 

control costs, the Census Bureau is testing strategies to decrease NRFU workload and conduct a more efficient data 

collection operation. 

The Census Bureau designed the 2013 Census Test to explore operational aspects of workload reduction and data 

collection. The primary goal of the test will be to assess whether the Census Bureau can implement a simulated 

NRFU data collection using different contact strategies and administrative records during production. Thus, the 

primary goal of the test will be to test operational feasibility. 

The test will look at: 

1. decreasing NRFU workload using administrative records to identify vacant housing units and enumerate 

occupied housing units, and 

2. achieving more efficient data collection by 

a. lowering the cap on computer-assisted personal interviewing (CAPI) contact attempts, 

b. using telephone - either a centralized computer-assisted telephone interviewing (CATI) operation or 

calls made by individual enumerators - to make initial contacts for cases whose addresses can be 

matched to telephone numbers, and 

c. using an adaptive approach to manage case assignments. 

The 2013 Census Test will consist of four treatments that examine these operations. (See Table 1.) Treatment 1will 

use administrative records to reduce workload and a fixed contact strategy of three CAPI contact attempts for each 

housing unit. Treatment 2 will not use administrative records to reduce workload but will have a fixed contact 

strategy of three personal visit contact attempts for each housing unit. In Treatments 1 and 2, individual enumerators 

will make initial contact attempts via telephone when possible. Treatment 3 will use administrative records to reduce 

workload, CATI to make initial contacts when possible, and an adaptive contact strategy to assign cases daily for 

face-to-face contacts. Treatment 4 will use administrative records to set the level of contact effort, CATI for initial 

contacts, and an adaptive contact strategy to assign cases daily for face-to-face contacts. All of these approaches are 

new for the Census Bureau. The 2013 Census Test is an exploration of how these novel approaches work in the 

field. 

Administrative Records to Reduce NRFU Workload 

The Census Bureau is committed to designing and conducting a 2020 Census that costs no more per housing unit 

than the 2010 Census, while striving to maintain high quality results. A major cost driver for the 2010 Census 

involved collecting information from housing units that did not respond to enumeration attempts. To reduce costs for 

the 2020 Census, the Census Bureau is investigating the strategic reuse of federal, state, and private data sources. 

The 2010 Census Match Study evaluated data from federal agencies and commercial vendors, finding that 

administrative records data were useful for determining housing unit and occupancy status, informing household 

population count, and assigning demographic characteristics. Using a composite of many commercial and federal 

files, the study found that administrative records were present for 89.2 percent of NRFU housing units in the 2010 

Census and for 79.5 percent of persons enumerated in NRFU units in the 2010 Census (Rastogi and O’Hara, 2012). 

1 
The Census Bureau is evaluating administrative records sources in various combinations to enhance data collection 

and processing methods for the NRFU operation. Different administrative records files and Census Bureau data will 

be used to explore agreement and disagreement observed in content across sources to develop approaches (such as 

modeling or business rules) to improve NRFU data quality and reduce costs from fewer in-person visits. Research 

outputs will include recommendations for which administrative records sources are fit for operational uses such as 

count imputation, characteristic imputation, and modeling for NRFU. Analyses will also help design and assign 

resources to carry out the 2020 Census; improve methods to unduplicate public, private, and census lists; and to 

determine which data are available to impute missing data. 

The Census Bureau will also use administrative records data to simulate NRFU operation results if fieldwork were 

curtailed or reduced and to evaluate the impact of reduced field operations on the vacant and delete check operations 

used in previous censuses. Administrative records will be evaluated to help determine the contact strategy for 

deciding if and when to interview a housing unit and to improve the contact strategies for non-responding housing 

units. These analyses will include estimated cost savings from the use of administrative records usage during the 

NRFU operations. 

In the 2013 Census Test, we will look at the use of record information in a small-scale field setting. Sample 

addresses will be matched to records from various sources. In Treatments 1 and 3, record information will be 

employed to reduce NRFU workload by removing vacants and “enumerating” some cases. The information will be 

employed in Treatment 4 to prioritize field effort. In this case, sample addresses will receive one personal visit if 

adequate record information is available to enumerate the households. Sample units without adequate record 

information will receive up to three in-person visits. 

Reducing Maximum Personal Visit Attempts to Increase NRFU Efficiency

In 2010, production salary cost and mileage contributed 63.9 percent and 22.7 percent to the total cost of the NRFU 

field operation, respectively. Field staff made 104,432,798 total contacts attempts during the operation. Enumerators 

were permitted up to six contact attempts for each case. About 41 percent of the 47,197,405 housing units included 

in the NRFU operation were enumerated on the first contact, and nearly 83 percent were enumerated in three or 

fewer contact attempts (Walker et al. 2012). 

In the 2013 Census Test there will be a maximum of three personal visit contact attempts. In Treatment 4, the 

presence or absence of administrative records suitable to enumerate the household will determine the level of 

contact effort. For cases with record information available, one contact attempt will be made to enumerate the 

household. For cases without record information, up to three contact attempts will be permitted. 

Implementing an Adaptive Contact Strategy to Increase NRFU Efficiency 

An adaptive survey design involves a tailored, dynamic approach to managing cases, enabled through a centralized 

data collection system. This approach utilizes paradata, process information such as contact history and case 

dispositions, response data, and auxiliary frame information to guide contact procedures and allocate resources 

(Miller, 2013). 

Adaptive survey designs are being tested to increase data collection efficiency while maintaining data quality. For 

example, the National Survey of College Graduates, conducted by the U.S. Census Bureau on behalf of the National 

Science Foundation, included a subsample of 4,000 cases in its 2013 adaptive design experiment. These cases were 

monitored throughout the course of data collection, and cases were switched to different modes or to active or 

inactive status to decrease nonresponse bias or to reduce cost (Coffey, 2013). The National Survey of Family 

Growth monitors subgroup representation in the sample during data collection in order to adjust field efforts to 

attack differential nonresponse (Groves and Heeringa, 2006).

The 2013 Census Test will use an adaptive approach to alert interviewers to the cases most likely to be completed on 

the next contact attempt. A model containing parameters from administrative record information, auxiliary frame 

data, and paradata will produce response propensity scores on a daily basis. Case management systems will

highlight each interviewer’s seven highest propensity-to-interview cases. The interviewers will be trained to attempt 

these “high priority” cases each day they work. 

2 
Site Selection and Sample 

For this study, 2,077 sample addresses were selected from a universe of all block groups in the Philadelphia 

Metropolitan Statistical Area. Philadelphia was chosen due to Census interviewer availability and operational and 

cost advantages provided by working with a Census regional office close to Census Bureau headquarters. In order to 

save time and money, the test will not contain a self-response component, and no questionnaire will be mailed. Only 

housing units from the six pairs of sampled block groups that did not mail back a self-response form during the 2010 

decennial census will be eligible for inclusion. Thus, the NRFU data collection environment will be simulated using 

information from the 2010 Census. 

Block groups with more than five percent of the population in group quarters were removed during sample selection 

because the research focused on individual housing units. Block groups with more than 10 percent of households 

having no one over age 14 who speak English ‘very well’ or better were also removed because of limited availability 

of interviewers to conduct interviews in non-English languages. 

From the remaining block groups, an iterative process was followed in which one block was randomly selected. A 

similarity score between this block group and all other eligible block groups was calculated to produce “pairs” of 

block groups, one to be assigned to the adaptive contact strategy and the other to the fixed contact strategy. The 

similarity score was based on a weighted combination of the absolute differences between block groups for the 

following ten block group variables found in the Census Bureau’s 2012 Planning Database (2012 Planning 

Database, 2013) that relate to likelihood of contacting the household (Groves & Couper, 1998): 

1. Percent vacant 

2. Average number of persons per household 

3. Percent owner occupied 

4. Percent single unit 

5. Percent multi-unit (ten or more) 

6. Percent NRFU 2010 cases 

7. Percent Black 

8. Percent Hispanic 

9. Percent under age 18 

10. Percent over age 64 

Each randomly sampled block group was paired with the block group with the highest similarity score. If the 

similarity score was above a threshold, this pair was included in the sample and removed from the remaining block 

group universe. The block group sampling strategy described above was repeated until there were over 2,000 

eligible housing units in the sample block groups. This approach resulted in eight pairs of blocks groups, two pairs 

of which were discarded due to logistical difficulties noted by the Philadelphia Regional Office. 

Treatments were then assigned to housing units using a two-step process. One block group in the pair was randomly 

assigned to the fixed contact strategy and the other to the adaptive contact strategy. Subsequent block groups were 

assigned to balance sample size between the two treatments. All sample units were then randomly assigned to two 

groups - those in which records, if available, were used to remove cases from the workload before going to the field 

and those in which administrative records, if available, were not used to remove cases. Nine housing units that had 

recently participated in other Census surveys were removed from the sample before this random assignment. This 

process resulted in a total of 2,077 eligible housing units across four treatments. 

Study Design 

Use of Administrative Records. Two treatments (Treatments 1 and 3 in Table 1) will employ administrative records 

to remove from field workload the occupied housing units that have records deemed suitable to enumerate them. 

Sample unit occupancy will be estimated by U.S. Postal Service information on whether advance letters, mailed two 

weeks prior to data collection, are deliverable or not and by examination of administrative records information. The 

suitability of records for enumerating these housing units will be determined through the Census Bureau’s research 

on matching administrative records information to 2010 Census NRFU housing units. In these treatments, the 

3 
Census Bureau also will remove housing units from data collection whose prenotice letters are returned with 

“undeliverable as addressed” United States Postal Service information and which have no other recorded evidence 

of occupancy. These housing units will be classified as “vacant” for purposes of the study. 

Table 1.Study Design for the 2013 Census Test 

Fixed Adaptive

Admin records 

identify vacants 

and enumerate 

occupied units 

before 

fieldwork 

Treatment 1

Administrative records 

• Remove cases from workload 

Telephone 

• If number, CAPI interviewers call 

• All numbers called twice 

Priority 

• None

Number of visits 

• Three personal visits before proxy 

Treatment 3

Administrative records 

• Remove cases from workload 

Telephone 

• If number, CATI before field 

• CATI call procedures 

Priority 

• Propensity models determine priority 

Number of visits 

• Three personal visits before proxy 

Admin records 

do not identify 

vacants and 

enumerate 

occupied units 

before 

fieldwork 

Treatment 2

Administrative records 

• Not used

Telephone 

• If number, CAPI interviewers call 

• All numbers called twice 

Priority 

• None

Number of visits 

• Three personal visits before proxy 

Treatment 4

Administrative records 

• Determine level of effort (number of 

contacts) 

Telephone 

• If number, CATI before field 

• CATI call procedures 

Priority 

• Propensity models determine priority 

Number of visits 

• If administrative record, one personal 

visit before proxy 

• Three personal visits before proxy 

The Census Bureau will not employ administrative records to reduce workload in Treatments 2 and 4. Instead, 

administrative records will be used to prioritize cases for contact in Treatment 4, and there will be no use of records 

in Treatment 2. 

Use of Telephone. The Census Bureau will attempt to match all sampled housing units to telephone numbers using 

purchased vendor files prior to data collection. These numbers will be flagged as landline or cell. The Census 

Bureau will check all landline telephone numbers to make sure they are working. In the fixed contact strategy 

treatments (Treatments 1 and 2), the Census Bureau will instruct CAPI interviewers to make two telephone calls to 

sample units before performing personal visits. Interviewers will attempt up to three in-person contact attempts for 

sampled housing units not reached by telephone and housing units without telephone numbers. If an interviewer 

cannot complete an interview after three in-person contact attempts, they will be instructed to obtain a proxy 

interview. 

In the adaptive contact strategy treatments (Treatments 3 and 4) the Census Bureau will send telephone numbers 

matched to sampled addresses to a CATI operation where interviewers will attempt interviews for up to two weeks. 

Nonresponding CATI cases will be transferred to CAPI interviewers who will attempt personal visits. Housing units 

without telephone numbers will be sent directly to CAPI interviewers at the beginning of the field period. 

High Priority Cases. CAPI interviewers in the adaptive contact strategy treatments (3 and 4) will be provided “high 

priority cases” on a daily basis, determined by response propensity models. Inputs to propensity models include 

auxiliary frame information and paradata. Paradata are obtained from the Contact History Instrument (CHI), which 

interviewers complete after each contact attempt. Auxiliary frame information comes from three sources: 

4 
1. government or other administrative records containing such information as household size and age 

composition, 

2. 2010 decennial contact history information used to determine a housing unit’s initial propensity to respond, 

and 

3. other frame information such as whether or not the housing unit is located in a multi-unit building. 

High priority is determined using a discrete-time hazard model (a type of logistical regression model) that 

determines each case’s propensity to respond at the next contact attempt given 12 parameters: 

initial propensity to respond (predicted by 2010 decennial contact history information); 

study treatment; 

whether or not the sample unit is in a multi-unit structure;

mode of each contact attempt (telephone or in-person); 

total number of contact attempts already made; 

whether or not prior contact with a person was made;

whether or not a potential respondent expressed reluctance during a contact attempt; 

if the contact was performed during “peak” hours, on the weekend or after 6:00 PM on a weekday; and 

four pieces of administrative record information predictive of contact and interview: 

o whether or not there is more than one person in the housing unit, 

o whether or not all sample unit members are less than 30 years-old, 

o whether or not all sample unit members are 70 years-old or older, and 

o whether or not there are children under 5 years-old in the house. 

The model runs each morning after new CHI data are transmitted from interviewer laptops. A SAS program orders 

all propensity scores within interviewer and selects the top seven cases in order to give interviewers their 

assignments for the next day’s contacts. 

Resources 

To conserve resources, the Census Bureau will use existing staff and office infrastructure for the 2013 Census Test. 

The Census Bureau has recruited interviewers that worked on the American Housing Survey. Two field supervisors 

from the Philadelphia Regional Office were chosen to manage these interviewers, one for the group working in the 

adaptive contact group and the other for the fixed contact group. Interviewers will be randomly assigned to these 

groups. 

The Census Bureau will use existing American Community Survey (ACS) systems for the 2013 Census Test, 

modifying where necessary. The test will use the ACS case management system for displaying and recording case 

information. The test will use a modified 2013 ACS production instrument that removes all person-level questions 

except relationship, gender, age, date of birth, Hispanic origin, and race. This instrument contains only two 

household-level questions, tenure and status of temporarily occupied units. The test also will use the Census 

Bureau’s contact history instrument to track contact attempts, strategies, and reluctance reasons. The case 

management system, production instrument, and contact history instrument will be made available to interviewers 

on laptops. 

The 2013 Census Test will rely on a modified ACS CAPI interviewer training because it will use the ACS case 

management system, modified production instrument, and contact history instrument. This interviewer training will 

recapitulate information provided to ACS interviewers but will emphasize the importance of interviewer behavior in 

following contact procedures prescribed for the test and in recording contact history information. 

In addition to using existing ACS infrastructure, the test will also utilize other Census Bureau resources. Paradata 

will be obtained on a daily basis using the Universal Tracking System (UTS). UTS is a centralized data repository 

for paradata and cost and progress data for several Census Bureau surveys. Its purpose is to track survey operations 

including case workloads and different modes. In addition to obtaining paradata, UTS will be used as the production 

environment in which to model propensity scores and execute business rules for the test. The 2013 Census Test will 

also utilize existing CAPI and CATI cost recording mechanisms. 

5 
The 2013 Census Test developed its own interviewer performance reports. These daily reports will track if 

interviewers followed prescribed contact procedures. Both adaptive and fixed groups will be monitored on how 

frequently they transmit their data, when cases are closed out after the maximum attempts have been made, and how 

faithfully interviewers record contact history information. Interviewers assigned to the adaptive treatment will also 

be monitored on contact attempts for high priority cases. Interviewers in the fixed contact group will be monitored 

for their use of telephone and in person contact attempts. 

Analysis Plan 

The primary goal of the test is to study the feasibility of conducting NRFU data collection using different contact 

strategies and administrative records during production. Thus, the primary goal of the test will be to examine 

operational design and execution. 

Operational feasibility will be measured throughout the entire test. The following components of operational 

feasibility will be examined: 

ability to remove cases that use administrative records to reduce the NRFU workload, 

ability to assign/reassign cases when workload has been reduced using administrative records, 

ability to staff effectively a field operation after cases are completed by CATI, 

ability of interviewers to follow instructions provided in case management, 

ability of interviewers to perform daily manual syncs between their laptops and a central server, 

capability of systems using a model and set of business rules to produce case priority information on a daily 

basis, and 

capability of a centralized paradata repository to receive paradata daily from several systems and transmit 

data daily to an operations control system, 

Secondary goals of the test are to measure, to the extent possible, the cost and data quality associated with the 

different treatments. One analysis will compare cost and data quality between groups that use and do not use 

administrative records to reduce the NRFU workload. Another analysis will compare cost and data quality between 

groups that use an adaptive contact strategy versus a fixed contact strategy. The aim of these analyses will be to 

develop comparable cost and quality measures for future Census tests. 

Cost will be operationalized in four ways: 

overall cost, which includes the costs of CAPI and CATI operations, 

average cost per case, 

average cost per contact attempt, and 

average case completion per contact attempt. 

Data quality will be operationalized in four ways: 

item nonresponse rates, 

percent proxy interviews, 

percent partial completes, and 

final response rates. 

Conclusion 

The research program for the 2020 Census is examining ways to reduce costs. The 2013 Census Test is an 

operational feasibility test of several new methods that hold promise of helping to achieve this goal. The research 

will examine different uses of administrative records, modes of data collection, and contact strategies for the 

nonresponse follow-up phase of Census data collection. The findings of the test will inform subsequent research into 

methods for the 2020 Census. 

References 

6 
(2013). 2012 Planning Database. Research @ Census. Retrieved from 

http://www.census.gov/research/2012_planning_database/ 

Groves, R. M. & Couper, M. P. (1998). Nonresponse in Household Interview Surveys. John Wiley & Sons. 

Groves, R. M. & Heeringa, S. G. (2006). “Responsive Design for Household Surveys: Tools for Actively 

Controlling Survey Errors and Costs.” Journal of the Royal Statistical Society. Series A. (Statistics in Society)

169(3), 439-57.

Rastogi, S. & O’Hara, A. (2012). 2010 Census Match Study. 2010 Census Planning Memoranda Series. No. 247.

Final Report. 

Walker, S., Winder, S., Jackson, G. & Heimel, S. (2012). 2010 Census Nonresponse Followup Operations 

Assessment. 2010 Census Planning Memoranda Series. No. 190. Final Report. 

7