2020 Census: Additional Actions Would Help the Bureau Realize Potential Administrative Records Cost Savings
Published: 2015-10-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The cost of the decennial census has steadily increased during the past 40 years, prompting the Bureau to reengineer key census-taking methods for the 2020 Census, including making greater use of information from administrative records. Given the potential cost savings associated with the use of administrative records, GAO reviewed (1) the Bureau's plans for using them and what opportunities and challenges the Bureau faces going forward; (2) the extent to which the Bureau's key 2015 test of them was implemented in accordance with objectives; and (3) the key assumptions supporting estimates of expected cost savings. To meet these objectives, GAO reviewed Bureau planning documents and test plans, interviewed Bureau officials, and observed implementation of the 2015 Census Test in Arizona. GAO also relied on its Schedule Assessment Guide.
section_title: What GAO Found
paragraphs: Increased reliance on administrative records—information already provided to the government as it administers other programs—has been discussed since the 1970s as a possible way to improve the quality or reduce the cost of the decennial census, and it may finally play a significant role in the decennial census in 2020. The U.S. Census Bureau (Bureau) estimates that it can save $1.4 billion using administrative records, compared to relying solely on traditional methods.
The Bureau recently completed its 2015 Census Test in Maricopa County, Arizona—a major test involving administrative records. The Bureau used this census test to demonstrate the feasibility of using administrative records to reduce the cost of its largest decennial field operation, following up door to door to enumerate households that do not respond to the census. Yet turning this estimated savings—and the potential savings from other uses of the records, such as using administrative records to help validate and update the address list rather than having to send temporary workers to every housing unit in the country—into a real cost reduction for the taxpayer will require detailed planning that includes milestones for ensuring outstanding challenges are addressed. This would include preventing disclosure of records and addressing concerns the public may have over their use, and obtaining access to remaining records. The Bureau has not set deadlines for deciding which records it will use and for which purposes, but doing so will help the Bureau complete needed activities on time and prioritize which activities—or records—to abandon if time and resources become a constraint.
Bureau officials said they consider the test a large success because it demonstrated a variety of new methods and advanced technologies that are under consideration for the 2020 Census. The test also demonstrated the feasibility of a prototype system for managing the field operation, yet implementation issues with some of the prototype technology were not systematically reported or tracked, and may have affected the usefulness of test data. Systematic problems arising during test interviews can affect key test measures, such as the number of hours spent going door to door. Knowing which cases experienced such problems can help link cost estimates to specific design features and prioritize future research, development, and acquisition efforts.
Key assumptions the Bureau used in estimating potential cost savings from administrative records are logical, and the Bureau plans to provide additional support for them. For example, the Bureau's assumption that it could reduce its follow-up workload follows clearly from the Bureau's use of administrative records to remove vacant units from among those housing units needing follow-up because people did not respond to the census, reducing that workload by 11.6 percent. This assumption was also validated by the Bureau's experience in its recent test, and the Bureau plans further testing of this assumption during future tests in 2016 and beyond. The Bureau released an updated life cycle cost estimate in October 2015, and GAO anticipates reviewing its reliability after the Bureau makes available support for the estimate.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Census Director ensure that resources focus on activities with promise to reduce cost by documenting milestones related to deciding which records to use for which purposes and by systematically recording better information about implementation issues affecting specific cases in future tests. The Department of Commerce concurred with GAO's findings and recommendations, and provided minor technical comments, which were included in the final report.

section_title: Letter
paragraphs: The cost of the decennial census has steadily increased during the past  40 years. For example, at about $13 billion, the 2010 Census was 56  percent more costly than the $8.1 billion 2000 Census (in constant 2010  dollars). Given budgetary realities, that cost growth is unsustainable.  Beginning in 1990, we reported that rising costs and difficulties in  securing public participation, among other challenges, required a new  approach to taking the census—a view that was shared by the U.S.  Census Bureau (Bureau) and other stakeholders.
The Bureau recently estimated that with a new approach it can conduct  the 2020 Census for $12.3 billion, more than $5 billion less than the  Bureau’s initial cost estimate of $17.5 billion to repeat the design and  methods of the 2010 Census. The Bureau plans to do this in part by  reengineering key census-taking methods, including making greater use  of information already provided to federal and state governments as they  administer programs—administrative records. The Bureau has  preliminarily estimated that expanded use of administrative records could  reduce the need for certain labor-intensive field work used in prior  decennials, and thus reduce the cost of the 2020 Census by as much as  $1.4 billion compared to the cost of relying on traditional census methods,  such as repeatedly visiting nonresponding households.
Earlier this year in the greater Phoenix, Arizona, area—specifically,  Maricopa County—the Bureau tested how well it can use administrative  records to reduce fieldwork and increase productivity for its largest field  operation—Non-Response Follow-Up (NRFU)—where census employees  attempt to follow up door-to-door with households that did not return a  census form.
You asked us to examine the Bureau’s plans for using administrative  records for the 2020 Census. We reviewed (1) the Bureau’s plans for  using administrative records for 2020 and what opportunities and  challenges the Bureau faces in using them; (2) the extent to which the  Bureau implemented its key 2015 test of administrative records in  accordance with its testing objectives, and what the Bureau’s experience  implementing selected aspects of the test was; and (3) key assumptions  supporting the cost savings estimates to be achieved from administrative  records.
For all three objectives, we reviewed documentary and testimonial  evidence from Bureau officials responsible for research and testing the  use of administrative records for 2020, such as testing plans. We  identified the administrative records the Bureau is considering and linked  them to their possible uses the Bureau is considering for 2020. We relied  on the GAO Schedule Assessment Guide as a source of criteria when  reviewing the activity the Bureau plans for administrative records, and  took steps to verify that the related schedule data we examined was  reliably representing the Bureau’s schedule. To review implementation of  the test, we used the Bureau’s test objectives as criteria, and conducted  direct observations and interviews in Maricopa County, Arizona,  identifying deviations as well problems appearing to arise with  implementation. To determine the logic of key assumptions for cost  savings, we isolated cost assumptions related to administrative records  and attempted to trace the assumptions through the Bureau’s relevant  calculations in the most recent cost model the Bureau made available to  us. To determine the support for the assumptions, we examined Bureau  documentation for its justifications, including results from recent Bureau  tests. When examining the extent to which implementation issues may  have affected measurement of key cost drivers in the 2015 Census Test,  we relied on our Cost Estimating and Assessment Guide for criteria.  More information on our scope and methodology can be found in  appendix I.
We conducted this performance audit from January 2015 to October 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: For decades, the Bureau has been considering how it could use  administrative records to help reduce the decennial’s costs. We, the  Bureau, and others have observed that some of the information needed  for the census has already been collected by other government agencies  in the course of administering their programs. Thus, accessing that  information and using it to administer and, in some cases, provide the  data for completing census forms, has the potential to reduce the cost of  the decennial census, such as its expensive acquisition of temporary  workspace and equipment to support fieldwork. Moreover, some of the  information collected through administrative records could be more  accurate than information collected through traditional methods, such as  when respondents provide the Bureau with incomplete information (or no  information at all), and when the Bureau’s enumerators need to interview  neighbors or other “proxy” respondents to collect needed information.  Depending on the source of the information, administrative records can  help the Bureau with such things as whether a housing unit is occupied or  vacant, improving the accuracy of the Bureau’s address list, and providing  demographic information on household members.
As far back as 1970, the Bureau has made limited use of administrative  records to help enumerate group quarters, such as college dormitories  and prisons. More recently, the Bureau conducted limited experiments  during the 2000 Census and found there was potential to use  administrative records to assist with follow-up and other operations, but  that further research would be needed. Additionally, since 2000, the  Bureau has used addresses provided by the U.S. Postal Service (USPS)  Delivery Sequence File (DSF) as a starting point to update its Master  Address File, a data file that contains a list of all known living quarters in  the United States and Puerto Rico. For the 2010 Census, the Bureau  used administrative records to help enumerate some group quarters and  select cases for an operation that followed up on potentially inaccurate  census responses.
In 2009, the Bureau’s earliest planning for the 2020 Census considered a  range of scenarios for using administrative records, from the most  expensive option—a traditional census with extensive field follow-up with  nonrespondents, and without increased use of administrative records—to  the least expensive option—a census conducted entirely by  administrative records. Considering concerns about cost and quality, the  Bureau ruled out the extreme scenarios and began exploring a “hybrid”  scenario that included a number of possible uses for administrative  records. In planning for the 2020 Census, Bureau research and testing  teams have been determining the possibilities, feasibilities, and cost and  quality implications of various uses of administrative records, also  including additional information, such as telephone numbers and  addresses, obtained from commercial vendors.
Earlier this year in Maricopa County, Arizona, the Bureau conducted its  2015 Census Test to see how well it can use administrative records to  reduce fieldwork and increase productivity for NRFU. The test also  included a new field management structure and an enhanced Operations  Control System supporting daily reassignments of cases. As part of this  effort, the Bureau  tested how well administrative records substituted for additional visits  to collect information from nonresponding households and from  proxies, such as neighbors;  compared the cost and productivity of traditional follow-up methods  to those relying on an enhanced operational control system,  demonstrating the potential benefits of automating the assignment of  work, scheduling the time of day for enumerators to conduct follow-up  to determine when residents were most likely to be home, as well as  efficient sequencing and routing of enumerator daily visits based on  administrative records and information from other surveys; and provided ground experience with prototypes of systems with leased  smartphone devices on which to collect data, which are not  necessarily reflective of the systems it may acquire or develop for  2020.
A key benefit of a test like the 2015 Census Test is being able to identify  potential problems with design alternatives. Untested systems and  unpracticed procedures will inherently experience implementation issues  that provide much of the basis on which lessons from such tests are  typically drawn. The test results are one source of input to the Bureau’s  preliminary design decisions for the 2020 Census. The Bureau included a  description of its preliminary decisions in the 2020 Census Operational  Plan it released on October 6, 2015. These decisions included using  administrative records to identify vacant addresses in advance of follow- up field work and to enumerate nonresponding households when possible  to reduce the need for repeated contact attempts during NRFU. The plan  also described an updated lifecycle cost estimate for the 2020 Census,  which we plan to review. The Bureau’s estimate of the total cost of the  2020 Census with the innovations it describes in its operational plan is  $12.3 billion. The Bureau has more tests planned including a 2016  Census Test in selected areas within Harris County, Texas and Los  Angeles County, California; a large test of address canvassing also in  2016; an additional site test in 2017 at an as yet undetermined location;  and a 2018 end-to-end test—the equivalent of prior decennial cycles’  “dress rehearsal.”
section_title: Administrative Records Can Reduce 2020 Fieldwork, but the Bureau Will Need to Better Define Milestones and Deadlines to Help Manage Risks
subsections: section_title: Decennial Uses of Administrative Records Offer the Bureau the Opportunity to Reduce the Fieldwork Needed for the 2020 Census
paragraphs: In key planning documents, the Bureau describes a goal of using  administrative records to reduce the field work involved in its NRFU  operation. To that end, the Bureau plans to use data from internal and  external sources, such as the 2010 Census, the United States Postal  Service (USPS), and the Internal Revenue Service (IRS), in a number of  ways, such as by identifying vacant housing units or enumerating  households in cases of nonresponse. The Bureau has reported that the  following three uses are key to the Bureau potentially saving up to $1.4  billion compared to traditional census methods. The Bureau tested each  of these uses during its 2015 Census Test and has decided to use them.
Identify vacant housing units. The Bureau incurs a large part of the  census’ cost while following up at residences that did not return a  census questionnaire. To ensure a complete count, Bureau guidance  in 2010 had enumerators visit some places up to six times to try to  obtain a response. During the 2010 Census, enumerators visited 48  million housing units for follow-up at least once. This number included  14 million vacant housing units. One of the largest potential efficiency  gains to the census may come from using administrative records to  remove these vacant units from the follow-up workload. Preliminary  findings from the Bureau’s 2015 site test found that administrative  records identified 11.6 percent of the NRFU workload as vacant.
Identify and enumerate nonresponding housing units that are  occupied. Another way the Bureau can reduce the NRFU workload is  to use administrative records to count households that did not return  census questionnaires. As part of its 2015 Census Test, the Bureau  successfully enumerated households using various administrative  records. The Bureau tested three approaches to counting  nonresponding households during the 2015 site test. In the approach  that most extensively used administrative records, the Bureau did not  attempt any NRFU visits and enumerated all occupied households  that had administrative records meeting a certain quality threshold.  The enumerators who used this approach had an initial workload of  approximately 29,000 households, and administrative records were  used to enumerate more than 5,800 of these households—which  reduced the workload for this approach by about 20 percent.
Predict best times to complete NRFU. One of the challenges the  Bureau faces in NRFU is reaching a household at a time when  someone is home. Catching respondents at home on an enumerator’s  first visit reduces the need for more follow-up fieldwork. In the 2015  Census Test, the Bureau used administrative records in addition to  information about how households had responded to other Bureau  surveys to help determine the contact strategy for deciding if and  when to interview a housing unit. For example, the Bureau used  demographic information, such as age, from administrative records  sources to determine the time of day to contact households.
section_title: The Bureau Has Identified Nine Additional Opportunities for Using Administrative Records in the 2020 Census
paragraphs: The Bureau has identified nine additional uses of administrative records  that may help control cost or improve the quality of decennial census data  or operations (see figure 1). The Bureau has not estimated cost savings  for these nine uses, but has begun researching the feasibility of most of  them.
As shown in the figure, these uses would occur during various points  relative to data collection.
Before data collection. The first use listed in the figure—validate and  update the address list—is one on which the Bureau is already working.  The Bureau is drawing on address lists and map information from state,  local, and tribal governments to update its own address list continuously  throughout the decade, reducing the need for a more costly door-to-door  canvassing during the 2 years prior to the census, as was done for the  2010 Census. According to Bureau officials, the Bureau is about to begin  research on how better to use records to identify group quarters, such as  dormitories, prisons, nursing homes, or homeless shelters; and to target  its outreach, that is, encourage cooperation with the census. The Bureau  uses special procedures to enumerate at these places, and administrative  records could potentially reduce the time and effort spent getting ready for  them.
During data collection. In addition to reducing the NRFU fieldwork, the  Bureau is considering using administrative records to help ensure quality  control of fieldwork such as by providing a near-real-time check on  interviews of households more at risk of being missed in the census. This  could reduce fieldwork and respondent burden or enable quality control  reinterviews of respondents to target other types of quality concerns. The  Bureau is also researching how administrative records can be used to  help process responses it receives either on paper or over the Internet  that do not have a census ID number on them (this activity is called non- ID processing). The Bureau may receive such responses from  households that may have lost or never received mailings or other  advance communication from the Bureau. The Bureau has done some  testing on this use—in 2015 in the Savannah, Georgia, media market  area, the Bureau invited test participants to respond over the Internet.  The Bureau demonstrated that a large collation of administrative records  from many sources was effective in helping the Bureau correct or fill in  missing address information, which enabled the Bureau to better locate  where those responses should be counted. The Bureau has other  ongoing research into how other records may help the Bureau validate  responses or the identities of those who submit responses as part of this  processing.
After data collection. If the Bureau still does not have information on a  housing unit after collecting data during field operations, it will attempt to  impute the data—it has done this since 1970. According to Bureau  officials, the Bureau plans to use administrative records to help improve  imputation of three related types of data the Bureau fills in for these  housing units. These data fields are (1) whether or not a unit is occupied,  (2) what the count of the unit might be, and (3) the demographic  characteristics of the residents. Finally, the Bureau is considering how  administrative records might help the Bureau evaluate the accuracy of the  census.
The Bureau’s research and testing on administrative records to help with  the address list, the two uses within non-ID processing, and the three  related imputation methods, are well underway. The Bureau reports  having achieved some early success already by demonstrating records’  use in updating the address list and in locating respondents’ addresses  as part of non-ID processing. Bureau officials said that they had not  started research and testing on the three remaining potential uses  because these uses are less likely to generate significant cost savings or  they fall much later in the decennial cycle, and the Bureau considered  them a lower priority for the limited funding available for research thus far  in the decade. According to Bureau officials, the Bureau will begin  research on the remaining uses during fiscal year 2016.
According to the Bureau, nearly every opportunity to use administrative  records for the 2020 Census would involve more than one source of  records and most opportunities would involve many sources. The Bureau  has identified and obtained access to nearly all of the sources it believes  it needs to leverage all of the opportunities it has identified, including the  three uses (identify vacant housing units; identify and enumerate  nonresponding housing units that are occupied; and predict best times to  complete NRFU) the Bureau believes will generate a large portion of its  estimated $1.4 billion savings from the cost of traditional methods. These  sources are summarized in figure 2. The Bureau reported it has tested all  of these sources already.
As of July 2015, the Bureau has memorandums of understanding in place  with seven federal agencies governing the use of data from 15 different  programs and activities.
The Bureau is leveraging data from state governments involved in three  federal grant programs for low-income individuals that tend to serve  sociodemographic groups (i.e., children and infants) that have historically  been undercounted in the census, the Temporary Assistance for Needy  Families (TANF), the Supplemental Nutrition Assistance Program (SNAP)  and the smaller Special Supplemental Nutrition Program for Women,  Infants, and Children (WIC). Bureau officials believe that data maintained  by states that administer the programs funded by these grants can  reliably identify beneficiaries and their addresses, and can potentially add  support to each of the potential uses the Bureau is considering. According  to the Bureau, while it has agreements in place with nine states thus far to  obtain their TANF, SNAP, or WIC program data, working individually with  states can be time consuming. Several states have declined the Bureau’s  requests to share data, citing various information technology limitations  and resource constraints. The Bureau has invited all states to share data  with it and officials said they are prepared to proceed with those that  choose to participate.
As part of its program to validate and update its address list throughout  the decade (rather than only during the 2 years prior to the census, as  was done for the 2010 Census), the Bureau is seeking participation from  state, local, and tribal governments. To participate, the governments must  reliably maintain address lists, such as for the purposes of emergency  response or property assessment, and be willing to share information with  the Bureau. Thus far, to validate and update its address list, the Bureau  has drawn on address lists and map information from more than 1,000  state, local, and tribal governments. Bureau officials have told us that they  expect to receive reliable data covering about two-thirds of the more than  3,200 counties in the country through this program.
The Bureau is working to gain access to additional sources of records to  better ensure the quality of the data it already has access to and to  improve its ability to find “hard to count” groups. The additional sources  include the following:
National Directory of New Hires (NDNH): NDNH is a national  database of wage and employment information used for child support  enforcement. Bureau officials believe that name and wage information  from NDNH could help corroborate the tax data from IRS that the  Bureau already has access to, improving the collective accuracy of  the records. The President’s 2016 budget submission included a  request for legislation that would authorize the Department of Health  and Human Services to share NDNH data with the Bureau for  statistical purposes such as the decennial census.
KidLink: KidLink is a database from the Social Security  Administration (SSA) that links parent and child Social Security  numbers for children born after 1998 in U.S. hospitals. It is valuable to  the Bureau because children, and babies less than 1 year old in  particular, have been historically undercounted. Bureau officials have  said that access to this database could help identify another 1 million  people. According to the Bureau, SSA raised issues about Bureau  access to these data. The Census Bureau Director says that he will  work with departmental staff and the Office of Management and  Budget to explore an administrative solution that may provide the  Bureau with access.
Obtaining access is challenging because federal, state, local, and tribal  agencies have different authorities and policies governing what, whether,  and how they share their administrative data. For example, the Bureau  has access authority to IRS tax data. Yet for other data, such as NDNH,  the Bureau is not authorized by statute to have access.
Bureau officials stated that they are examining ways to quantify the  potential effect that their access to these additional sources could have  on the 2020 Census. The Bureau estimated that the value of acquiring  the NDNH and using it to corroborate data from IRS, in conjunction with  other administrative and third-party data sources, would be approximately  $157.5 million (using 2010 figures and dollars). This assumes no  nonresponse follow-up visits for cases with administrative and/or third- party data, so that the actual savings would likely be less since the  Bureau recently decided to make at least one follow-up visit before  enumerating a household with administrative records. Bureau officials  state that there is value in accessing these records for the Bureau’s other  statistical surveys as well, and that even if they are unable to obtain the  additional records in time for the 2020 Census, they would continue  pursuing them for these other purposes, as well as for use in future  censuses.
section_title: The Bureau Lacks Deadlines for Ensuring Administrative Records Decisions Are Made in Time
paragraphs: As of August 2015, the Bureau had not set deadlines to determine when  to make final decisions on which of its 12 identified uses of administrative  records it will implement for the 2020 Census, nor had it set deadlines for  determining exactly which records from which sources it will tap in  support of each use it implements. Moreover, the Bureau has no  deadlines it can use against which to measure progress for obtaining  access to its additional sources or scheduled milestones for when key  steps may need to be taken to integrate them within 2020 preparations.  For example, time will be needed to review files to ensure their fitness for  use before the Bureau can integrate them into the census design.  According to our scheduling guide, assurance of program success can be  increased when management relies on credible schedules containing the  complete scope of activities necessary to achieve established program  objectives.
Bureau officials have stated that final decisions on the use of  administrative records are needed by the end of fiscal year 2017 to be  included in the Bureau’s 2018 end-to-end test. But, these deadlines do  not appear in schedule documents. Deadlines for deciding on the  remaining potential uses—either committing to move forward with them or  abandoning them as possibilities for 2020— and for deciding how all  other records will be used would help to ensure the Bureau is using its  resources cost-effectively.
section_title: The Bureau is Addressing Challenges to Using Administrative Records for the 2020 Census; Implementing Our Previous Recommendations Could Help
subsections: section_title: The Bureau Has Processes in Place and Is Conducting Research and Testing to Ensure Quality of Records
paragraphs: Although the Bureau has no control over the accuracy of data provided to  it by other agencies, it is responsible for ensuring that data it uses for  2020 Census are of sufficient quality for their planned uses. Data quality  can involve the accuracy, relevance, and timeliness of the data.
Steps taken: The Bureau has taken many steps to ensure the quality of  the records it is considering using for 2020.
The Bureau’s Center for Administrative Records Research and  Application screens all administrative records the Bureau has  obtained to ensure their fitness for use by assigning unique person  and address identifiers to facilitate record linkage, evaluating biases  associated with the linkages, and evaluating the quality and coverage  of data.
Similarly, the Bureau’s Geography Division routinely screens address  and map files provided by state, local, and tribal governments to  determine if they satisfy preset minimum quality standards for  completeness of address information. This helps to improve the  master list of addresses.
The Bureau’s Administrative Records Modeling project team has  researched several predictive models for identifying thresholds of  sufficient quality for administrative records used in identifying  occupied and vacant housing units during NRFU.
The Bureau has researched which combinations of records more fully  cover the population. Among other findings, this research has helped  the Bureau refine what combinations of administrative records work  better to determine whether a housing unit is occupied or unoccupied,  or for determining the number of people living in it.
Relatedly, in August 2015 the Bureau was measuring how well  administrative records cover hard-to-count groups such as children  and people who were born in other countries. The Bureau planned to  finish this work by October 2015.
The Bureau reported that 2014 Census Site Test results found that  administrative records matched some households better than others.  For example, 65 percent of households with one adult and zero  children matched to actual records, but only 33 percent of households  with three adults and one or more child matched to administrative  records.
Since records need to be timely to be most useful, according to the  Bureau, it negotiated to obtain monthly files beginning in February  2015 from IRS tax returns for use in the 2015 Census Test, which it  conducted around the Census Day of April 1, 2015. As a result, the  Bureau obtained the records several months earlier in the calendar  year than it had in the past.
Future plans: The Bureau plans comprehensive testing of all records  during an end-to-end test of its 2020 Census design (to be conducted in  2018). The Bureau plans additional testing of administrative records for  the 2016 Census Test in the Los Angeles and Houston metro areas, in a  large test of address canvassing also in 2016, and in an additional site  test in 2017 at an as yet undetermined location.
The Bureau reported it will review imputation models it used during prior  censuses to determine how it can integrate information from  administrative records into them in fiscal year 2016. Tests will be included  in the 2016 Census Site Test.
section_title: The Bureau Has Experience Preventing Disclosure, but Implementing Our Prior Recommendations Can Improve Protections
paragraphs: We have previously reported that until the Bureau implements a complete  and comprehensive security program, it will have limited assurance that  its information and systems are being adequately protected against  unauthorized access, use, disclosure, modification, disruption, or loss. In  January 2013, we made 115 recommendations aimed at addressing  weaknesses in that program. The Bureau expressed broad agreement  and said it would work to find the best ways to address our  recommendations. In July 2015, the Bureau reported that it experienced  an information technology attack to gain access to the Federal Audit  Clearinghouse that contains nonconfidential information on audit reporting  packages from state and local governments, nonprofit organizations, and  Indian tribes expending federal awards. Federal agencies use the single  audit reports to ensure program compliance. The Bureau is making  additional clearinghouse information available via the Internet next year.  According to Bureau officials, the breach was limited to this database on  a segmented portion of the Bureau’s network that does not touch  administrative records or sensitive respondent data protected under Title  XIII, and the hackers did not obtain the personally identifiable information  of census and survey respondents.
Steps taken: The Bureau cited examples of its past long-standing  experience in collecting data from other agencies and reporting on it as  evidence of the Bureau’s ability to prevent disclosure of information from  such sources:
Since 1972, the Bureau’s Survey of Business Owners has collected  data on businesses from administrative records, including data from  the Social Security Administration.
The Bureau’s Longitudinal Employer-Household Dynamics program  produces information combining federal, state, and Bureau data on  employers and employees. This program collects and secures  administrative records information from all 50 U.S. states, including  unemployment insurance earnings data.
During negotiations for access, the Bureau and the agency providing the  data agree to data safeguards. For example, the Bureau’s agreement  with IRS states that the Bureau will advise its employees of their  responsibility for handling federal tax information, and will annually certify  that all employees who access federal tax information have been advised  about their obligation to protect the information. Further, the Bureau is  required to provide annual reports to IRS that include the type of  computer system and type of medium on which the data are contained.  Once the Bureau obtains access to an administrative data source, it  transfers the information that it needs to Bureau servers and maintains  the information within the Bureau’s firewalls and information security  infrastructure.
Our open recommendations underscore the importance of the Bureau  safeguarding its systems. Bureau officials state that the Bureau has taken  action on all 115 of our recommendations to improve its security program.  In assessing the Bureau’s reported actions, we have reviewed  documentation pertaining to 75 of the recommendations—58 of which we  have confirmed have been addressed and 17 require additional actions  and/or documentation from the Bureau. We are currently analyzing the  extent to which the remaining 40 recommendations have been addressed  by the Bureau and expect to complete that review by the end of 2015.
section_title: The Bureau Is Taking Steps to Maintain Transparency, but Implementing Our Prior Recommendation Can Help Support Dialogue on Public Concerns
paragraphs: A third challenge is the extent to which the public will accept the sharing  of personal data across government agencies for the purposes of the  census. We have previously reported on the need within the federal  statistical system for broader public discussion on balancing trade-offs  among competing values, such as quality, cost, timeliness, privacy, and  confidentiality. Related concerns involve trust in the government and  perceptions about burden on respondents as well the social benefits of  agencies sharing data. We recommended in 2012 that the Bureau  develop and implement an effective congressional outreach strategy,  particularly on new design elements the Bureau is researching and  considering as well as on cost-quality trade-offs of potential design  decisions. The Bureau concurred with the recommendation and has  taken a number of steps since that are likely to help inform congressional  decision making, which we describe below.
Steps taken: In 2013, the Bureau contracted for regular polling of  nationally representative individuals on the extent to which they prefer  data to come from information already provided to federal and state  governments or from a survey they fill out. Findings included that  respondents were evenly divided when asked whether they prefer the  Bureau to obtain someone’s name and age directly from the Social  Security Administration rather than asking for this information on a  questionnaire. In 2013, the Bureau began hosting quarterly program  management reviews encouraging dialogue with oversight on selected  technical aspects of the Bureau’s ongoing research and testing. These  reviews are open to the public and viewable online over the Internet.  These supplement the Bureau’s monthly status reports on ongoing  research projects that the Bureau provides to Office of Management and  Budget and, later, Congress.
Future steps: The Bureau is developing a communications campaign for  2020, which it will formally launch in 2016. The campaign will include  information about how the Bureau intends to use administrative records in  the 2020 Census. Given the many potential uses of administrative  records the Bureau has identified, it will be important for the Bureau’s  messaging to consider the range of uses. For example, some people may  feel differently about the Bureau using administrative records for  enumerating as opposed to targeting the time of day they will be  contacted by the Bureau. Moving forward, to help support broader public  discussion on trade-offs that the Bureau may need to make on the role of  administrative records in the 2020 Census, the Bureau should address  our prior recommendation to develop and implement an effective  congressional outreach strategy, particularly on new design elements the  Bureau is researching and considering, as well as on cost-quality trade- offs of potential design decisions.
In response to our 2012 recommendation, in November 2014 the Bureau  provided us with a congressional engagement plan. The four-page plan  brings together in one place a summary of the Bureau’s ongoing activity  in this area, yet, by itself, lacks goals or strategies for attaining them, or  accountability for who will work to implement them or when. We will  continue monitoring the Bureau’s efforts to address this recommendation,  particularly as they may depend on deadlines the Bureau may yet set for  making final decisions about administrative records.
section_title: The Bureau Executed Its Recent Site Test According to Test Plan, but the Bureau Has Limited Information on How Some Implementation Issues Affected the Data Collected
subsections: section_title: The 2015 Site Test Was Carried Out on Schedule and Achieved Key Goals
paragraphs: The Bureau had several objectives for its 2015 Census Test. For  example, the Bureau wanted to begin the process of developing a field  operations control system that combined administrative records,  technology, and available real-time data to improve the efficiency of field  data collection. In addition, it planned to collect data on whether using  administrative records could reduce NRFU workload and increase NRFU  productivity. Senior Bureau officials told us that the test was also to  provide data to help inform future cost estimates and design decisions.
The Bureau designed the 2015 Census Test to test three approaches to  NRFU, and used one each within three different parts of the test sample:  one that followed procedures similar to the 2010 Census and two that  used experimental approaches. Each part of the sample had a workload  of around 23,000 housing units.
Control panel. This panel followed NRFU procedures similar to those  used in the 2010 Census. Like the 2010 Census, the operation was  managed from a field office, enumerators compiled their timesheets  by hand daily, and supervisory staff had regular face-to-face meetings  with their enumerators.
Hybrid Administrative Records Removal Panel. For this panel, the  Bureau used administrative records to identify vacant housing units  and remove them from the NRFU workload before enumerators  attempted contact. Then enumerators were to make one attempt to  contact the remaining housing units. If the attempt was unsuccessful,  the Bureau used administrative records to attempt to enumerate the  housing units.
Full Administrative Records Removal Panel. This panel followed  the same approach as the hybrid to remove vacant units from the  NRFU workload. But the Bureau also attempted to use administrative  records to count occupied households before enumerators attempted  any contacts.
Bureau officials said they consider the 2015 Census Test a large success  because it allowed them to employ a variety of new methods and  advanced technologies that are under consideration for the 2020 Census.  Our review of key tracking documents showed that the Bureau executed  key milestones for the test early or on schedule. Additionally, Bureau  officials stated that operational costs tracked closely to planned costs,  and that actual field workload was within a few percentage points of that  planned for each test panel.
Another success of the 2015 Census Test is that during our observations  the Bureau maintained control over test panels so that they did not  appear to influence each other. For example, field test managers with  whom we met appeared largely unaware that other test panels existed or  how their procedures may be different. According to the Bureau, the test  also demonstrated the usefulness of continuing work on the enhanced  operational control system.
Some of the tests biggest achievements were in demonstrating the  feasibility of using administrative records: a total of 8,370 vacant units were identified for removal from the  NRFU workload of 72,072 units (11.6 percent), a total of 14,312 cases (19.8 percent of workload) were identified as  occupied through the use of administrative records. Depending on the  test panel, this information was used instead of knocking on doors of  neighbors or others when respondents were not home and  enumeration attempts were exhausted, and  the race/ethnicity for persons who did not provide it in their responses  was identified.
These test results are linked to assumptions that need to be met to attain  the Bureau’s cost savings estimate and assisted the Bureau in making its  preliminary design decisions and updating its cost estimates. For  example, the Bureau’s October 2015 operational plan reports that the  Bureau will use an approach during NRFU like that for the hybrid  administrative records removal panel during the 2015 test, making only  one visit to nonresponding households where the Bureau has determined  administrative records are good enough to complete the roster and  enumerate the household.
section_title: Lack of Information on Site Test Implementation Issues May Affect Usefulness of Data for Future Decision Making
paragraphs: As part of its testing related to administrative records and proving  capabilities of prototype systems implementing selected design features  in the 2015 Census Test, the Bureau collected data on the extent to  which administrative records reduced field data collection and improved  productivity; two key cost-drivers in prior decennials. During the site test,  the Bureau experienced specific implementation issues, and these in turn  affected the measurement of the key cost drivers. We did not select a  generalizable sample of the over 400 enumerators hired during the test to  interview. Yet, our observations were consistent across the 30  enumerators we met and were echoed during the four debriefings the  Bureau held with field staff near the conclusion of the site test.
We observed enumerators spending extra time dealing with  enumeration devices that were not working properly, lacked  connectivity, or that they may not have been using properly.  Enumerators also told us about experiences similar to our  observations. These issues affected measures related to total hours  spent on the NRFU operation.
We observed several instances of enumerators being assigned to  enumerate repeatedly the same nonresponding or nonexistent  households, which the design alternative would not have permitted  had it been implemented correctly. This affected measures related to  reducing contact attempts.
We observed inefficiencies in automated route management, poor  cellular service leading to limited or no access to smart phone maps  while enumerators tried to locate addresses, and, to a lesser extent,  enumerators having difficulties closing out or completing their cases,  leading to additional visits. These issues are likely to have decreased  various productivity measures.
Additionally, we observed situations where multiple enumerators were  visiting multiunit structures and gated communities without  coordination and communication between each other. This affected  measures related to reducing contact attempts.
Based on our discussions with enumerators about these issues, it  appears that the incidence of the issues was underreported to the  Bureau, which had intended to track such issues. For example, all but 4  of the 30 temporary enumerators and field supervisors we spoke with said  they tried their own temporary solutions to problems they faced, such as  powering down and restarting their Bureau-issued phones when they  froze, rather than notifying supervisors or the Bureau’s call-in number  every time. We heard this repeated during the Bureau’s debriefings of  enumerators near the end of the test. Additionally, enumerators we spoke  with reported using the capability within their phones to record notes  about specific cases with which they experienced implementation  problems. According to senior managers responsible for managing the  test implementation, during the test there was no systematic review of  these notes planned other than when an enumerator had separately  flagged a case as involving a dangerous situation, or was being let go for  performance reasons and the enumerator’s work was being reviewed.  After we discussed the notes with senior Bureau officials, they told us that  they would explore the notes for information that would help the Bureau  better understand the implementation issues. They also told us that  during enumerators’ debriefings, they learned enumerators had not  always known to whom to report what types of issues. Bureau training  materials provided a toll-free number for enumerators to call regarding  any technical issue if first calling the enumerator supervisor could not  resolve it.
Bureau officials acknowledged that problems with the implementation of  the test likely affected productivity measures to be used in calculating  future cost estimates. According to leading practices reported in our cost  estimation guide, information about the extent to which implementation  issues affect test data should be collected, as they can be useful in future  cost estimation. However, the effect on the measures is difficult to  determine because the Bureau did not systematically keep data about the  extent of the problems. Bureau officials said they are analyzing the results  of the test to determine the extent to which identified implementation  issues affected the productivity measure, so that they could better control  for and understand the separate effects of implementation issues.
Moreover, in future tests, systematically collecting better information  about the effect of implementation issues may help inform how the  Bureau prioritizes specific design features underlying those issues. For  example, if in future tests the Bureau systematically tracks the cases  where, say, its automated routing may create problems for enumerators,  the Bureau could know the extent to which that implementation issue  affected its test measures, like average miles driven per case, and the  resulting effect on cost. With such information, the Bureau could make a  more informed decision about prioritizing its efforts and resources on  ensuring that routing works in systems it ultimately develops or acquires,  or whether to get rid of automated routing altogether. Such information  can also help inform cost and risk analyses the Bureau may undertake  based on the possibilities that such specific implementation issues may  also occur during the 2020 Census. Accordingly, steps the Bureau can  take to better capture information from enumerators about implementation  issues should they arise in future tests will also help future cost  estimation. Such steps might include additional or revised training  elements for enumerators on what details to report and the logistics of  reporting them: to whom—such as the help desk or their supervisors— where—such as within the notes capability on their phones or  elsewhere—and how.
section_title: Key Assumptions for Cost Saving Estimates from Administrative Records are Logical; the Bureau Will Continue to Validate the Assumptions
paragraphs: The Bureau had five key assumptions that support its estimate that using  administrative records for the 2020 enumeration will cost less than relying  on a traditional design and methods (see table 1). Bureau officials  responsible for cost estimation described each of the assumptions to us  and showed us where the assumptions were represented in key planning  documents.
The first three assumptions will only materialize if certain conditions are  met. For example, to reduce the number of field offices, the Bureau will  need to demonstrate that it can reduce the NRFU workload, as the  workload is a key driver of the number of field offices needed. The Bureau  articulated the last two assumptions as business decisions about its  operations. We reviewed whether the assumptions were logically  represented in the calculations used to produce the Bureau’s available  cost estimates, and how the Bureau had supported the assumptions  through prior Bureau experience, research, or testing. As shown in table  2, overall, the Bureau’s assumptions are logical and have support. The  Bureau plans to further test some of these assumptions to validate the  assumptions further or to identify needed revisions.
The Bureau’s planned tests in 2016 and beyond will inform several of  these assumptions:  Identify and remove vacant units from the NRFU workload. Now  that the Bureau has successfully demonstrated it can identify and  remove vacant units from the NRFU workload using administrative  records, it plans to continue testing this activity to gather additional  data on how much the NRFU workload can be reduced.
Reducing the number of field offices. Although the Bureau  announced in October that it would open no more than 250 offices in  2020, the Bureau plans to conduct future large tests at other sites and  to involve other operations in the tests to gather additional information  about the number of field offices needed for 2020.
Limiting the number of NRFU visits. Future tests will provide the  Bureau opportunities to try to control the maximum number of visits  enumerators make, and determine if it can reduce implementation  issues that caused repeat visits during the 2015 test.
Bureau officials told us that research is underway that will help validate  that the new uses of administrative records will provide the benefits that  the VDC and CFU operations formerly provided. They said that reliance  on administrative records can provide an alternative to field visits  intended simply to verify whether a housing unit exists or is vacant.  Further, they said that the use of administrative records before and during  the enumeration will remove the need for a CFU operation checking  against the records after the enumeration, as was done in the 2010  Census.
While we were reviewing these cost assumptions, documentation was not  always readily available, and Bureau reporting on one of the assumptions  needed to be corrected. We were able to identify the needed support.  Bureau staff said that, moving forward, they decided to change the  methodology for future reporting on the cost estimate to involve more  factors and variables, such as the ratio of field workers to supervisors  they would need in 2020 in addition to the NRFU workload assumption.  This change will help demonstrate the reliability of the estimates as well  as ensure effective communication with others about them.
Bureau officials told us that the revised total life-cycle cost estimate the  Bureau released on October 6, 2015, was developed with leading  practices from our cost estimating and assessment guide. After the  Bureau releases the underlying model, methodology, and supporting  documents for the estimate, we anticipate reviewing them to assess their  reliability.
section_title: Conclusions
paragraphs: Although administrative records have been discussed and used for the  decennial census since the 1970s, the Bureau plans a more significant  role for them to reduce the amount of data collection fieldwork to reduce  the cost of the decennial census in 2020. The Bureau appears to have  demonstrated the feasibility and potential effectiveness of administrative  records for several uses during NRFU, which the Bureau estimates could  save up to $1.4 billion compared to traditional census methods. The  Bureau has also identified several additional opportunities to leverage  administrative records to help improve the cost and quality of the 2020  Census. Yet, the Bureau will need to consider when to end research  pursuits that show less promise for substantially reducing the census’  cost or meeting other 2020 goals so that it can focus resources on  successfully refining and implementing activities that have greater  potential. Knowing deadlines for when final go/no go decisions need to be  made about which records the Bureau will use, how it will use them, and  for which purposes will help ensure necessary activities are completed on  time. Deadlines regarding still uncertain purposes or those involving  records the Bureau is still pursuing, such as NDNH and KidLink, as well  as those from some states, will also help the Bureau prioritize which  activities—or records—to continue pursuing or to abandon if time  becomes a constraint.
The Bureau’s 2015 Census Test was generally an operational success in  that it provided much useful information to inform cost estimates and  decisions about how to design future operations. Test results and the  information about implementation issues we and the Bureau documented  should prove useful as the Bureau moves forward to refine its business  requirements for further testing. As the Bureau plans for its 2016 tests,  getting better information from them about which fieldwork cases are not  being implemented as planned could help the Bureau link future  estimates of cost savings to, and prioritize, design features it wants or  needs to work well in final systems it develops, or to eliminate problematic  features it deems not worth the trouble. The Bureau could capture this  information more systematically through the help desk, within the notes  capability on the phones used for interviewing or elsewhere, and by  ensuring enumerators received training on where to record the issues,  who to contact, what details to include, and the importance of doing so.  The bottom line is that the amount of time the Bureau has to research and  test the range of options for 2020 and ensure their readiness is limited.
The Bureau has a lot of activity to follow through on in the time remaining,  so it will be important to reduce the number of open options to ones the  Bureau can manage well when turning the corner from research to a  focus on development and implementation.
The Bureau’s early cost savings assumptions related to its use of  administrative records are logical, and the Bureau is taking steps to  develop further support for them. We plan to review the Bureau’s October  2015 cost estimate and latest estimates of savings from using  administrative records after the Bureau makes supporting documentation  available.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Commerce direct the Under  Secretary of the Economics and Statistics Administration and the Director  of the U.S. Census Bureau to take the following two actions to help  ensure the Bureau focuses its resources on those activities that show  promise for substantially reducing enumeration cost.
Establish clearly documented deadlines for making final decisions  about which records to use for what purposes, particularly for  purposes not yet demonstrated as feasible or involving records it does  not already have access to, such as NDNH and KidLink.
In advance of the 2016 Census Test and later tests, ensure  systematic capture of information about fieldwork cases that  experience problems by including information in enumerator training  about where to record the issues, who to contact, what details to  include, and the importance of doing so.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Secretary of the Department of  Commerce for comment. In its written comments, reproduced in appendix  II, the Department of Commerce concurred with our recommendations.  The Department of Commerce also provided minor technical comments  that were incorporated, as appropriate.
We are sending copies of the report to the Secretary of Commerce, the  Under Secretary of Economic Affairs, the Director of the U.S. Census  Bureau, and interested congressional committees. The report also is  available at no charge on GAO’s website at http://www.gao.gov.
If you have any questions about this report please contact me at (202)  512-2757 or goldenkoffr@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. The GAO staff that made major contributions to this report  are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The purpose of our review was to examine the Census Bureau’s (Bureau)  plans for using administrative records for the 2020 Census. Specifically,  our objectives were to review (1) the Bureau’s plans for using  administrative records for 2020 and what opportunities and challenges  the Bureau faces in using them; (2) the extent to which the Bureau’s key  2015 test of administrative records were implemented in accordance with  its testing objectives and what the Bureau’s experience implementing  selected aspects of the test was; and (3) key assumptions supporting the  cost savings estimates to be achieved from administrative records.
For all objectives, we reviewed documentation from the Bureau on 2020  research and testing of administrative records, and reviewed  documentary and testimonial evidence from Bureau officials responsible  for research and testing the use of administrative records.
To address the first objective, we identified the administrative records the  Bureau is considering and linked them to their possible uses the Bureau  is considering for 2020. We identified which records the Bureau has  access to and examined the Bureau’s authority under Title XIII and other  statutes to use information from other agencies for the decennial census.  We identified what decisions remain for the Bureau regarding  administrative records as well as the Bureau’s timelines for making those  decisions. We relied on our Schedule Assessment Guide as a source of  criteria for assessing activity the Bureau plans for administrative records.  We took steps to verify that the related schedule data we examined was  reliably representing the Bureau’s schedule, such as by comparing the  respective inclusion of major projects as well checking that activities  occurred in both.
To address the second objective, we examined the Bureau’s 2015  Census Test, which took place in Maricopa County, Arizona. Using the  Bureau’s test objectives as criteria, we conducted direct observations and  interviews to assess the Bureau’s implementation of the test, and  collected performance metrics on the test from the Bureau. We also  documented where we observed implementation deviating from either  what Bureau temporary enumerators had been trained to expect, or what  we expected based on our prior experience with census field operations.  We also identified problems appearing to arise with implementation. We  conducted 30 in-field observations of Bureau enumerators conducting  Nonresponse Follow-up during the test and interviewed Bureau  employees managing the test. Our sample of field observations was  stratified for geographic balance across the test site area, but was not  designed necessarily to be generalizable. When examining the extent to  which implementation issues may have affected measurement of key cost  drivers in the 2015 Census Test, we relied on our Cost Estimating and  Assessment Guide for criteria. We communicated implementation issues  we observed in near-real time to Bureau officials for their consideration as  they conducted their own evaluation of the test and ongoing related  research.
To address the third objective, we inventoried the working cost  assumptions provided by the Bureau and isolated those related to  administrative records. To determine if assumptions were logical, we  traced their incorporation into respective calculations within the cost  model the Bureau used to produce the Bureau’s earlier cost estimates  available to us at the time of the audit. We examined the support and  justifications the Bureau had documented for each assumption, the  related results of the 2014 and 2015 Census Tests, and related materials  the Bureau made available to us during the audit. When examining the  extent to which implementation issues may have affected measurement  of key cost drivers in the 2015 Census Test, we relied on our Cost  Estimating and Assessment Guide for criteria. We did not review the  Bureau cost estimation methodology or the reliability of either its  preliminary cost savings estimates, or any cost estimation material  information that was part of the Bureau’s October 2015 release.
We conducted this performance audit from January 2015 through October  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Ty Mitchell, Assistant Director;  Jeffrey DeMarco; Robert Gebhart; Richard Hung; Andrea Levine; Donna  Miller; Shannin O’Neill and Timothy Wexler.