2020 Census: Progress Report on Using Administrative Records to Control Enumeration Costs
Released: 2015-11-03T00:00:00

Letter

Background

We, the Bureau, and others have observed that some of the information the Bureau collects during the census has already been gathered by other government agencies in the course of administering their programs. Accessing that information could provide the Bureau with data to help conduct the census and, in some cases, complete census forms that have missing data. Such uses of administrative records have the potential to reduce the cost of the decennial census because, for example, the Bureau would need to hire fewer temporary workers and acquire less office space and equipment to support fieldwork. Moreover, some of the information collected through administrative records could be more accurate than information the Bureau collects through traditional door-to- door follow-up methods, such as when the Bureau’s enumerators need to interview neighbors or other “proxy” respondents because they cannot reach a household member to collect needed information.

While the Bureau has made some limited use of administrative records during past decennials, it plans to use them much more extensively in 2020. In its first operational plan for the 2020 Census (released October 6, 2015), the Bureau reported design decisions including the use of administrative records to identify vacant addresses in advance of follow- up field work and to enumerate nonresponding households when possible in order to reduce the need for repeated contact attempts during its nonresponse follow-up operation (NRFU). The Bureau also updated the life-cycle cost estimate for the 2020 Census to $12.5 billion (in constant 2020 dollars). This is slightly lower than the Bureau’s prior life-cycle cost estimate of $12.7 billion. We plan to assess the reliability of the new cost estimate and examine the practices the Bureau used to produce it after the Bureau makes the model and its supporting documentation available. As part of the operational plan, the Bureau also released an update of how much its fundamental redesign of the census would reduce costs in four major design areas compared to the cost of a 2020 Census conducted using the methods of the 2010 Census. Table 1 below compares the cost reduction in those four areas.

The operational plan describes over 100 other preliminary design decisions related to the Bureau’s efforts to build an address list and collect census responses. Its release marks a critical turning point in the decade-long countdown to the next decennial, as the Bureau completes its early research and testing plan, and pivots toward developing operations and systems and testing them to refine the census design.

Administrative Records Can Reduce 2020 Fieldwork, but the Bureau Will Need to Better Define Milestones and Deadlines to Help Manage Risks

Bureau officials have said they hope to use administrative records to reduce the field work involved in the most expensive census operation— NRFU, when Bureau staff traditionally knock on doors across the country at homes of people who did not respond to the census, or who were missed by census mailings. The Bureau has reported that the following three uses are key to potentially saving up to $1.4 billion compared to using traditional census methods.

Identify vacant housing units. The Bureau incurs a large part of its census cost while following up at residences that did not return a census questionnaire. However, during the 2010 Census, of the 48 million housing units enumerators visited for follow up, about 14 million were vacant. One of the largest efficiency gains to the census may come simply from using administrative records to remove these vacant units from the follow-up workload. In a test in Arizona earlier this year, this use of administrative records enabled the Bureau to reduce the NRFU workload by 11 percent. Since we completed our audit work, the Bureau announced that it would still send a reminder post card to units it identified as vacant, which will cost more, but will provide any missed household one more opportunity to respond to the census.

Identify and enumerate occupied nonresponding housing units. During the 2015 Census Test, the Bureau demonstrated it could use administrative records to accurately count some nonresponding occupied households if the household had administrative records meeting a certain quality threshold, without attempting any visits. In the test, the Bureau used this approach to reduce the NRFU workload by about 20 percent. Since we completed our audit work, the Bureau announced that before using administrative records to enumerate such households in 2020, it will still attempt one visit to the household. Attempting a visit will add costs, but will also provide the household one more opportunity to respond to the census.

Predict the best times to complete NRFU. One of the challenges the Bureau faces when knocking on doors is reaching a household when someone is home. In the 2015 Census Test, the Bureau used demographic information, such as age, from administrative records sources in addition to information about how households had responded to other Bureau surveys in order to help determine the contact strategy for deciding if and when to interview a household.

The Bureau tested each of these uses during its 2015 Census Test and plans further testing of them to refine the methods, but the Bureau has already decided to use them.

The Bureau Plans Additional Opportunities for Using Administrative Records in the 2020 Census

In addition to the three uses the Bureau has committed to, the Bureau has identified nine additional uses of administrative records that may help further reduce cost or improve the quality of the census (see figure 1). The Bureau has not separately estimated cost savings for these nine uses, but has begun researching the feasibility of most of them.

As shown in the figure, these uses would occur during various points relative to data collection.

Before data collection. The Bureau is already using administrative records to validate and update the address list. The Bureau is drawing on address lists and map information from state, local, and tribal governments, in addition to information obtained from commercial sources, to update its own address list continuously throughout the decade, reducing the need for a more costly door-to-door canvassing during the 2 years prior to the census, as was done for the 2010 Census. In addition, Bureau officials reported research is about to begin on how to better use records to identify group quarters, such as dormitories, prisons, nursing homes, and homeless shelters, and to target outreach, that is, encourage cooperation of staff at these locations with the census. The Bureau historically uses special procedures to enumerate at these places, and administrative records could potentially jump start the time and effort spent getting ready for them.

During data collection. The Bureau is considering using administrative records in lieu of some follow-up visits for the purpose of quality control of field work. In past decennials, the Bureau has called or sent enumerators to re-interview some respondents. Relying on administrative records could reduce fieldwork and respondent burden, or enable the Bureau to better target re-interviews of respondents. The Bureau is also researching how administrative records can be used to help process responses that do not have a census ID number on them (this activity is called non-ID processing). The Bureau may receive such responses from households that lost or never received mailings or other advance communication with an ID number from the Bureau. A test in 2015 in the Savannah, Georgia, media market area demonstrated that a large collation of administrative records from many sources was effective in helping the Bureau correct or fill in missing address information. This enabled the Bureau to better locate where those responses should be counted. The Bureau is also researching how other records may help it validate responses or the identities of those who submit responses as part of this processing.

After data collection. When the Bureau still does not have information on a housing unit after collecting data during field operations, it will attempt to impute the data—as it has done since 1970. According to Bureau officials, in 2020 the Bureau will use administrative records to help improve how it imputes three related types of data: (1) whether or not a unit is occupied, (2) how many people live in the unit, and (3) the residents’ demographic characteristics, such as sex, race, and ethnicity. Finally, the Bureau is considering how administrative records might help it evaluate census accuracy.

The Bureau Has Identified Potential Sources for Most of the Administrative Records It Hopes to Use for the 2020 Census

The Bureau has identified and obtained access to nearly all of the sources that it believes it needs in order to leverage all of the opportunities it has identified (see figure 2).

In addition to the records already obtained, the Bureau is working to gain access to the National Directory of New Hires (NDNH), a national database of wage and employment information used for child support enforcement, and KidLink, a database from the Social Security Administration that links parent and child Social Security numbers for children born after 1998 in U.S. hospitals. Both of these databases would help the Bureau improve its ability to find historically “hard-to-count” groups, such as certain minority groups or young children.

Bureau officials stated that they are examining ways to quantify the potential effect that their access to these additional sources could have on the 2020 Census. But they point out that there is value in accessing these records for the Bureau’s other statistical surveys as well, and that even if they are unable to obtain the additional records in time for the 2020 Census, they would continue pursuing them for these other purposes, as well as for use in future censuses.

Setting Deadlines Would Help the Bureau Ensure It Makes Timely Decisions on Administrative Records

While the Bureau is to be commended for its efforts to expand its use of administrative records to control costs and increase accuracy, we identified actions the Bureau could take to increase its chances of success. First, as of August 2015, the Bureau had not set deadlines to determine which of its identified uses of administrative records it will or will not implement for the 2020 Census, nor had it set deadlines for determining exactly which records from which sources it will tap in support of each use it implements. Moreover, the Bureau had no deadlines against which to measure progress for obtaining access to additional sources or scheduled milestones for when key steps may need to be taken in order to integrate them within 2020 preparations. For example, the Bureau will need time to review files in order to ensure their fitness for use before the Bureau can integrate them into the census design. Earlier in October the Bureau announced time frames for several decisions related to uses of administrative records and explained that the decisions are not yet included in its integrated schedule of activities. According to our scheduling guide, assurance of program success can be increased when management relies on credible schedules containing the complete scope of activities necessary to achieve established program objectives.

Bureau officials have stated that final decisions on the use of administrative records are needed by the end of fiscal year 2017 in order to be included in the Bureau’s 2018 full end-to-end test. However, these deadlines do not appear in schedule documents. We recommended that the Census Director ensure that resources focus on activities with promise to reduce cost by documenting milestones related to deciding which records to use, particularly for purposes not yet demonstrated as feasible or involving records it does not already have access to, such as NDNH and KidLink. The Department of Commerce—the Bureau’s parent agency—concurred with our recommendation. Deadlines for deciding on all potential uses—either committing to move forward with them or abandoning them as possibilities for 2020—and for deciding how all other records will be used would help to ensure the Bureau is using its resources cost-effectively.

The Bureau Is Taking Steps to Address Challenges to Using Administrative Records for the 2020 Census; Implementing Our Previous Recommendations Could Help

The Bureau is taking steps to address challenges it faces in using administrative records to control costs and improve the quality of the 2020 Census.

One challenge facing the Bureau is ensuring the quality of the records it receives from other agencies and levels of government. To meet this challenge, the Bureau has processes in place and is conducting research and testing to ensure quality of records. For example, to ensure accuracy, the Bureau routinely screens address and map files provided by state, local, and tribal governments to determine if they satisfy preset minimum quality standards for completeness of address information. This helps to improve the master list of addresses. The Bureau plans comprehensive testing of all records during an end-to-end test of its 2020 Census design (to be conducted in 2018). The Bureau plans additional testing of administrative records for the 2016 Census Test in the Los Angeles and Houston metro areas, a large test of address canvassing for 2016, and an additional site test in 2017 at an undetermined location.

The Bureau reported it will review imputation models it used during prior censuses to determine how it can integrate information from administrative records into them in fiscal year 2016. Tests will be included in the 2016 Census Test.

A second challenge involves protecting confidential data. We have previously reported that until the Bureau implements a complete and comprehensive security program, it will have limited assurance that its information and systems are being adequately protected against unauthorized access, use, disclosure, modification, disruption, or loss. In January 2013, we made over 100 recommendations aimed at addressing weaknesses in that program. The Bureau agreed and Bureau officials state that the Bureau has taken action on all 115 of our recommendations to improve its security program. In assessing the Bureau’s reported actions, we have reviewed documentation pertaining to 97 of the recommendations—66 of which we have confirmed have been addressed and 31 that require additional actions and/or documentation from the Bureau. We are currently analyzing the extent to which the remaining 18 recommendations have been addressed by the Bureau and expect to complete that review by the end of 2015. My colleague is further addressing the security challenge in her statement. Bureau officials pointed out that the Bureau is well positioned to prevent disclosure of administrative records, as it has long-standing experience in collecting data from other agencies and reporting on them. Furthermore, the Bureau and the agency providing the data agree to data safeguards during negotiations for access.

A third challenge concerns public acceptance and attitudes about sharing of personal data across government agencies for the purposes of the census. We have previously reported on the need within the federal statistical system for broader public discussion on balancing trade-offs among competing values, such as quality, cost, timeliness, privacy, and confidentiality. The public has related concerns involving trust in the government and perceptions about the burden on respondents as well the social benefits of agencies sharing data. We recommended in 2012 that the Bureau develop and implement an effective congressional outreach strategy, particularly on new design elements the Bureau is researching and considering as well the cost-quality trade-offs of potential design decisions. The Bureau agreed with the recommendation and, in November 2014, it provided us with a congressional engagement plan. The four-page plan brings together in one place a summary of the Bureau’s ongoing activity in this area, yet, by itself, lacks goals or strategies for attaining them, or accountability for who will work to implement them or when. We will continue monitoring the Bureau’s efforts to address this recommendation, particularly as these efforts may depend on scheduling of activities the Bureau may yet set related to making final decisions about administrative records.

Key Assumptions for Estimated Cost Savings from Administrative Records Are Logical; the Bureau Will Continue to Validate the Assumptions

As part of our recently released review, we determined that key assumptions in the Bureau’s administrative records cost area made sense. Table 2 shows the results of our analysis.

While we were reviewing these cost assumptions, the Bureau did not always have documentation readily available, and Bureau reporting on one of the assumptions needed to be corrected. We were able to identify the needed support, and Bureau staff said that they will change the methodology for future reporting on the cost estimate to involve more factors and variables, such as the ratio of field workers to supervisors they would need in 2020 in addition to the NRFU workload assumption. This change will help demonstrate the reliability of the estimates as well as ensure effective communication with others about them.

Since we released our report, the Bureau provided an updated estimate of the total 2020 Census life-cycle cost of $12.5 billion, as well as updated estimates of how much less in four major cost areas its 2020 plan would cost compared to a cost of the a 2020 Census conducted using the 2010 Census approaches and methods. We expect soon to begin reviewing the Bureau’s new cost model and its assumption. Bureau officials have told us that although the model has been updated, the key assumptions within the administrative records cost area are largely the same.

Bureau officials told us that the revised life-cycle cost estimate the Bureau released on October 6, 2015 was developed with leading practices from our cost estimating and assessment guide. After the Bureau releases the underlying model, methodology, and supporting documents for the estimate, we anticipate reviewing them to assess their reliability. Continued oversight efforts, such as this hearing, will be helpful to ensure that the Bureau’s efforts remain on track and focused on those most promising to result in a cost-effective 2020 Census.

GAO Contacts and Staff Acknowledgments

If you have any questions on matters discussed in this statement, please contact Robert Goldenkoff at (202) 512-2757 or by e-mail at goldenkoffr@gao.gov. Other key contributors to this testimony include Ty Mitchell, Assistant Director; Brett Caloia; Robert Gebhart; Richard Hung; Andrea Levine; Donna Miller; Tamara Stenzel; and Timothy Wexler.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.