With a cost of about $12.3 billion, the 2010 Census was the most expensive population count in U.S. history, costing about 31 percent more than the $9.4 billion 2000 Census (in constant 2020 dollars). Some cost growth is to be expected because the population is growing and becoming more complex and difficult to count, which increases the Bureau’s workload. However, the cost of counting each housing unit has escalated from about $16 in 1970 to $92 in 2010 (in constant 2020 dollars), according to the Bureau. For the 2020 Census, the Bureau intends to limit its per-household cost to not more than that of the 2010 Census, adjusted for inflation. To achieve this goal, the Bureau is significantly changing how it conducts the census, in part by re-engineering key census-taking methods and infrastructure. The Bureau’s innovations include (1) using the Internet as a self-response option; (2) verifying most addresses using “in-office” procedures rather than costly field canvassing; (3) re-engineering data collection methods; and (4) in certain instances, replacing enumerator-collected data with administrative records (information already provided to federal and state governments as they administer other programs). The Bureau’s various initiatives have the potential to reduce costs. In October 2015, the Bureau estimated that with its new approach it can conduct the 2020 Census for a life-cycle cost of $12.5 billion, $5.2 billion less than if it were to repeat the design and methods of the 2010 Census (both in constant 2020 dollars). However, in June 2016, we reported that this $12.5 billion cost estimate was not reliable and did not adequately account for risk. Table 1 below shows the Bureau’s estimated cost savings it hopes to achieve in the following four innovation areas. The 2016 test was the latest major test of NRFU in the Bureau’s testing program. In 2014, the Bureau tested new methods for conducting NRFU in the Maryland and Washington, D.C., area. In 2015, the Bureau assessed NRFU operations, in Maricopa County, Arizona. In 2018, the Bureau plans to conduct a final “End-to-End” test which is essentially a dress rehearsal for the actual decennial. The Bureau needs to finalize the census design by the end of fiscal year 2017 so that key activities can be included in the End-to-End Test. The Bureau plans to conduct additional research through 2018 in order to further refine the design of the 2020 Census, but recently had to alter its approach. On October 18, 2016, the Bureau decided to stop two field test operations planned for fiscal year 2017 in order to mitigate risks from funding uncertainty. Specifically, the Bureau said it would stop all planned field activity, including local outreach and hiring, at its test sites in Puerto Rico, North and South Dakota, and Washington State. The Bureau will not carry out planned field tests of its mail-out strategy and follow up for non-response in Puerto Rico, or its door-to-door enumeration. The Bureau also cancelled plans to update its address list in the Indian lands and surrounding areas in the three states. However, the Bureau will continue with other planned testing in fiscal year 2017, such as those focusing on systems readiness and internet response. Further, the Bureau said it would consider incorporating the cancelled field activities elements within the 2018 End-to-End Test. The Bureau maintains that stopping the 2017 Field Test will help prioritize readiness for the 2018 End-to-End Test, and mitigate risk. Nevertheless it also represents a lost opportunity to test, refine, and integrate operations and systems, and puts more pressure on the 2018 Test to demonstrate that enumeration activities will function as needed for 2020. NRFU generally proceeded according to the Bureau’s operational plans. However, our observations and the Bureau’s preliminary data at both test sites found that (1) there were a large number of non-interviews, and (2) enumerators had difficulty implementing new census-taking procedures. The Bureau’s 2016 Census Test included a new field management structure that, among other things, included an enhanced operations control system supporting daily assignments of cases. A cornerstone of the Bureau’s efforts to reduce the cost of NRFU is the automation of decision-making on how to manage the follow-up caseload. Unlike previous censuses and one prior test, enumerators in the 2016 Census Test did not have an assigned set of cases that they alone would work until completion. Instead, the Bureau relied on an enhanced operational control system that was designed to provide daily assignments and street routing of non-response follow-up cases to enumerators in the most optimal and efficient way. The Bureau first tested this system in the 2015 Census Test. The test also included streamlined procedures for making contact at large apartment buildings. This was intended to reduce repeated attempts to contact property managers. A key objective of the 2016 Census Test was to refine procedures for collecting NRFU data from households using mobile devices leased from a contractor. In prior decennials, enumerators collected NRFU information using paper and pencil. The Bureau believes that replacing paper-based operations with automated case management and mobile devices for collecting interview data will provide a faster, more accurate, and more secure means of data collection in the 2020 Census (see figure 1). Some test activities that we observed at both test sites included streamlined multi-unit contact procedures and interviews with a proxy respondent. A proxy is someone who is a non-household member, at least 15 years old, and knowledgeable about the NRFU address. At multi- unit structures such as apartment buildings, the enumerator is trained to first interview the property manager to find out which units were occupied and which were vacant on Census Day. Such interviews help to streamline NRFU by removing vacant units from an enumerator’s workload. They also help build a rapport with property managers by ensuring they know when enumerators are working in their building and can also help them gain access to locked buildings. Preliminary data at both test sites indicate that the Bureau experienced a large number of non-interviews. According to the Bureau, non-interviews are cases where either no data or insufficient data were collected, in part because the cases reached the maximum number of six attempted visits without success or were not completed due to, for example, language barriers or dangerous situations. While not necessarily a precursor to the 2020 non-interview rate, because of its relationship to the cost and quality of the count, it will be important for the Bureau to better understand the factors contributing to it. According to preliminary 2016 Census Test data, there were 19,721 NRFU cases coded as non-interviews in Harris County, Texas and 14,026 in Los Angeles County, California, or about 30 and 20 percent of the test workload respectively. In such cases, the Bureau may have to impute attributes of the household based on the demographic characteristics of surrounding housing units as well as administrative records. Bureau officials expect higher numbers of non-interviews during tests in part because, compared to the actual enumeration, the Bureau conducts less outreach and promotion. Bureau officials hypothesized that another contributing factor could be related to NRFU methods used in the 2016 test compared to earlier decennials. For the 2010 and earlier decennials, enumerators collected information during NRFU using pencil and paper. Enumerators may have visited a housing unit more than the 6 maximum allowable visits to obtain an interview but did not record all of their attempts, thus enabling them to achieve a higher completion rate. For the 2020 Census, and as tested in 2016, the Bureau plans to collect data using mobile devices leased from a contractor, and an automated case management system to manage each household visit. The Bureau believes that this approach will provide a faster, more accurate, and more secure means of data collection. At the same time, the mobile device and automated case management system did not allow an enumerator to attempt to visit a housing unit more than once per day, reopen a closed case, or exceed the maximum allowable six attempts. One factor we observed that may have contributed to the non-interview rate was that enumerators did not seem to uniformly understand nor follow procedures for completing interviews with proxy respondents. According to the 2016 Census Test enumerator training manual, when an eligible respondent at the address cannot be located, the automated case management system on the mobile device will prompt the enumerator when to find a proxy to interview, such as when no one is home or the housing unit appears vacant. In such circumstances, enumerators are to find a neighbor or landlord to interview. However, in the course of our site visits, we observed that enumerators did not always follow these procedures. For example, one enumerator, when prompted to find a proxy, looked to the left and then right and, finding no one, closed the case. Similarly, another enumerator ignored the prompt to find a proxy and explained that neighbors are usually not responsive or willing to provide information about the neighbor, and did not seek to find a proxy. Enumerators we interviewed did not seem to understand the importance of obtaining a successful proxy interview, and many appeared to have received little encouragement during training to put in the effort to find a proxy. Proxy data for occupied households are important to the success of the census as the alternative is a non-interview. In 2010, about one-fourth of the NRFU interviews for occupied housing units were conducted using proxy data. We shared our observations with Bureau officials who told us that they are aware that enumerator training for proxies needs to be revised to convey the importance of collecting proxy data when necessary. Converting non-interviews by collecting respondent or proxy data can improve interview completion rates, and ultimately the quality of census data. The Bureau told us it will continue to refine procedures for 2020. According to the Bureau, its plans to automate the assignment of NRFU cases have the potential to deliver significant efficiency gains. At the same time, refinements to certain enumeration procedures and better communication could produce additional efficiencies by enabling the Bureau to be more responsive to situations enumerators encounter in the course of their follow-up work. Enumerators were unable to access recently closed incomplete cases. Under current procedures, if an enumerator is unable to make contact with a household member, the case management system closes that case and it is to be reattempted at a later date, perhaps by a different enumerator, assuming the enumerator has not exceeded six attempts. Decisions on when reattempts will be made—and by whom—are automated and not designed to be responsive to the immediate circumstances on the ground. This is in contrast to earlier decennials when enumerators, using paper-based data collection procedures, had discretion and control over when to re-attempt cases in the area where they were working. According to the Bureau, leaving cases open for re- attempts can undermine the efficiency gains of automation when enumerators depart significantly from their optimized route, circling back needlessly to previously attempted cases rather than progressing through their scheduled workload. During our test site observations, however, we preliminarily found how this approach could lead to inefficiencies in certain circumstances. For example, we observed enumerators start their NRFU visits in the early afternoon as scheduled, when many people are out working or are otherwise away. If no one answered the door, those cases were closed for the day and reassigned later. However, if a household member returned while the enumerator was still around, the enumerator could not reopen the case and attempt an interview. We saw this at both test site locations, typically in apartment buildings or at apartment-style gated communities, where enumerators had clear visibility to a large number of housing units and could easily see people arriving home. Bureau officials acknowledged that closing cases in this fashion represented a missed opportunity and plan to test greater flexibilities as part of the 2018 End-to-End Test. Programming some flexibility into the mobile device—if accompanied with adequate training on how and when to use it—should permit completion of some interviews without having to deploy staff to the same case on subsequent days. This in turn could reduce the cost of follow-up attempts and improve interview completion rates. Enumerators did not understand procedures for visits to property managers. Property managers are a key source of information on non- respondents when enumerators cannot find people at home. They can also facilitate access to locked buildings. Further, developing a rapport with property managers has helped the NRFU process, such as when repeated access to a secured building or residential complex is needed on subsequent days by different enumerators. In response to problems observed during the Bureau’s 2014 and 2015 Census tests and complaints from property managers about multiple uncoordinated visits by enumerators, the Bureau’s 2016 Census Test introduced specific procedures to conduct initial visits to property managers in large multi-unit apartment buildings. The procedures sought to identify up front which, if any, units needing follow-up at the location were vacant, eliminating the need for enumerators to collect this information from property managers with subsequent visits on a case-by- case basis. According to Bureau officials, the automated case management system was designed to allow for an enumerator to make up to three visits to property managers to remove vacant units. According to the Bureau, the 2016 Census Test demonstrated that vacant units could quickly be removed from the NRFU workload using these procedures in cases where a property manager was readily available; however, in other cases the procedures caused confusion. For example, whenever an initial visit was unsuccessful, all of the cases at that location—up until then collated into only one summary row of the enumerator’s on-screen case list—would suddenly expand and appear as individual cases to be worked, sometimes adding several screens and dozens of cases to the length of the list, which enumerators we spoke with found confusing. Furthermore, without the knowledge of which units were vacant, enumerators may have unnecessarily made visits to these units and increased the cost and the time required to complete NRFU . During debriefing sessions the Bureau held, Bureau enumerators and their supervisors identified training in these procedures as an area they felt needed greater attention in the future. Indeed, while training classes included a case study exercise on interviewing a property manager, this exercise in the enumerators training manual gives no warning to enumerators and does not refer to the procedures. Bureau officials said that they are pleased with the progress the test demonstrates they have made in automating case management at multi-unit locations a priority. They added that they recognize the need to better integrate procedures in the training moving forward. Timing of return visits did not leverage information on respondent availability. During our field visits, we encountered several instances where enumerators had been told by a respondent or otherwise learned that returning at a specific time on a later date would improve their chance of obtaining an interview from either a household respondent or a property manager. But the Bureau’s 2016 Census Test and automated case management did not have an efficient way to leverage that information. Attempting contact at non-responding households at times respondents are expected to be available can increase the completion rate and reduce the need to return at a later date or rely on proxy interviews as a source of information. The Bureau’s automated case management system assigned cases to 6- hour time windows after estimating hour-by-hour probabilities of when best to contact people. The estimation relied on various administrative records, information from other Bureau surveys that had successful contacts in the past, as well as area characteristics. The 2016 Census Test did not have a way to change or update these estimates when cases were subsequently reassigned. The goals of assigned time windows were intended to result in more productive visits and reduce costs. When enumerators identified potentially better times to attempt a contact, they were instructed to key in this information into their mobile devices. For example, one enumerator keyed in a mother’s request to come back on Thursday afternoon when her kids were in camp, while others keyed-in information like office hours and telephone contact numbers obtained from signs on the property they had seen for property managers. However, according to the Bureau, this updated information went unused, and we met enumerators who had been assigned to enumerate addresses at the same unproductive time after they had written notes documenting other better times to visit. Another enumerator reported visiting a property manager who complained that the enumerator was not honoring the manager’s earlier request made during a prior enumeration attempt that an enumerator return during a specified time window. Such repeat visits can waste enumerator time (and miles driven), and contribute to respondent burden or reduced data quality when respondents become annoyed and may become less cooperative. We discussed our preliminary observation with managers at the test sites, who expressed frustration that the automated case management system did not allow them to record the locally-obtained data on when to contact people whom they found in enumerator notes in a way to affect future case assignment. Headquarters staff told us that while they have not fully evaluated this yet, they are concerned that providing local managers with too much flexibility to override the results of optimized case and time assignments would undermine the efficiency gains achievable by the automation. They also explained that enumerators were to have been provided capability to record what day or what time of day for follow-up. This information could have been used by the automated case management to better target the timing of future assignments. However, they acknowledged that this procedure may not have been either fully implemented or explained during enumerator training. Bureau officials have said that this is another area they are planning to address. The Bureau has reengineered its approach to building its master address list for 2020. Specifically, by relying on multiple sources of imagery and administrative data, the Bureau anticipates constructing its address list with far less door-to-door field canvassing compared to previous censuses. One major change the Bureau is making consists of using in-office address canvassing–a two-phase process that systematically reviews small geographic areas nationwide, known as census blocks, to identify those that will not need to be canvassed in the field, as shown in figure 2. The Bureau estimates that the two phases of in-office canvassing will result in roughly 25 percent of housing units requiring in-field canvassing, instead of canvassing nearly all housing units in the field as done previously. With in-office address canvassing clerks compare current aerial imagery for a given block with imagery for that block dating to the time of the last decennial census in 2010. During this first phase, called Interactive Review, specially trained clerks identify whether a block appears to have experienced change in the number of housing units, flagging each block either as stable—free of population growth, decline, or uncertainty in what is happening in the imagery over time—or “active,” in which case it moves to the next phase. Addresses in stable blocks are not marked for in-field canvassing. For blocks where change is detected or suspected, the Bureau uses a second phase of in-office canvassing, known as Active Block Resolution, to attempt to resolve the status of each address and housing unit in question within that block. During this phase, clerks use aerial imagery, street imagery, and data from the U.S. Postal Service, as well as from state, local, and tribal partners when reviewing blocks. If a block can be fully resolved during this phase of in-office canvassing, the changes are recorded in the Bureau’s master address file. If a block cannot be fully resolved during the second phase of in-office canvassing, then the entire block, or some portion of the block, is flagged for inclusion in the in-field canvassing operation. In-office address canvassing began in September 2015 with plans for a first pass of the entire country to be completed by the end of fiscal year 2018. In-field canvassing for the 2020 Census is scheduled to begin in August 2019. Another major change the Bureau is making for its re-engineered address canvassing is significantly expanding the role that state, local, and tribal partners can play throughout the decade in contributing to an accurate, more up-to-date address list. Through the Geographic Support Systems Initiative, begun in fiscal year 2011, partner jurisdictions have been providing address and spatial data to the Bureau to help validate and supplement the Bureau’s address list. As of October 2016, the Bureau reported that it had received partner data covering 73 percent of all known housing units nationwide. It added that the vast majority of the addresses in the files that the Bureau had processed as of July 2015 have either been matched with existing addresses in its database, or added to the address list. As with previous decennial censuses, as directed by Congress, the Bureau will also engage with state, local, and tribal partners through its Local Update of Census Addresses program in fiscal years 2018 and 2019 in order to ensure that jurisdictions have the ability to comment on the address list prior to enumeration. The Bureau plans to rely on the in-office part of address canvassing to validate a large part of the addresses added to the list during that program where data are available to permit it. The Bureau is testing its re-engineered address canvassing operation in two sites through December 2016—in Buncombe County, North Carolina, and St. Louis, Missouri. In-office canvassing for the test sites began at the Bureau’s National Processing Center in Jeffersonville, Indiana, in August 2016. The exercise will test the Bureau’s assumptions about the cost and effectiveness of the re-engineered approach, as well as the quality of in- office canvassing, field staff training, and the use of new collection geography in the field. In addition to the 100 percent in-office canvassing the Bureau plans for 2020, the Bureau will also canvass 100 percent of the test areas in the field so that it can compare results it obtains for blocks where it would not otherwise have gone door to door. The Bureau hired 262 in-field listers across both sites to conduct the door-to-door canvassing, also beginning in October with a relisting operation commencing in November. Although the innovations the Bureau is planning with its reengineered address canvassing have the potential to reduce costs, they entail some risks that could affect the cost or quality of the address canvassing operation. According to the Bureau, these risks include: Locating Hidden Housing Units. The Bureau recognizes that certain kinds of dwellings are hard to identify and may not have been marked as housing units at the time of address list development and not included in any databases. This could lead to their being missed and occupants not being counted in the census. These units are referred to as hidden housing units and include such living arrangements as attics, basements, or garages converted into housing units. According to the Bureau, while in-field canvassing also has similar risks for missing these types of housing units, solely relying on the use of imagery to identify these units could lead to an incomplete address list. Monitoring Change in the Housing Stock. When the Bureau determines during the first phase of in-office canvassing that a block has not experienced population change, the Bureau plans to subject the block to later monitoring so that if later change is detected, the block can be reassigned for further review. The Bureau has developed the conditions or “triggers” for subjecting blocks to later monitoring, but has not yet determined how it will operationalize them. According to the Bureau, if the triggers that the Bureau is developing for this process do not adequately detect recent change, then housing unit growth may be missed, and the resulting address list may not be up-to-date. Obtaining Quality Data. For the Bureau to adequately review enough blocks in-office–and therefore reduce field costs of door-to-door canvassing–the Bureau needs to have data of sufficient quality to make reliable determinations about changes in housing units within those blocks. According to the Bureau, if it does not obtain sufficient satellite imagery (covering areas with both current and prior census imagery) or address and spatial data from state/local/tribal partners, then it may be forced to send more blocks than planned to in-field canvassing. We have ongoing audit work examining the Bureau’s re-engineered address canvassing approach. The justification of key cost and data quality assumptions, the approaches to mitigating key risks, and the Bureau’s adherence to timelines and canvassing schedules are all subjects of our ongoing work, which we plan to report on early next year. The Bureau goes to great lengths each decade to improve specific census-taking activities. But these incremental modifications have not kept pace with societal changes that make the population increasingly difficult to locate and cost-effectively count. This increasing difficulty and escalating costs led the Bureau to re-engineer its approach for the 2020 Census. While preparations for 2020 are still underway, and with testing still occurring, the Bureau’s experience in planning for 2010 can enhance its readiness for 2020. For example, as the Bureau continues its planning efforts for 2020, our prior work indicates that it will be essential for it to address the following three lessons learned: Ensure key census-taking activities are fully tested Develop and manage on the basis of reliable cost estimates Sustain workforce planning Ensure key census-taking activities are fully tested. The census is a large, complex operation comprised of thousands of moving parts, all of which must function in concert with one another to secure a cost-effective count. While the census is under way, the tolerance for any breakdowns is quite small. Given this difficult operating environment, rigorous testing is a critical risk mitigation strategy because it provides information on the feasibility and performance of individual census-taking activities, their potential for achieving desired results, and the extent to which they are able to function together under full operational conditions. Given the new four innovation areas for the 2020 Census, it will be imperative that the Bureau have systems and operations in place for the 2018 End-to-End Test that will take place in three locations, covering more than 700,000 housing units in total. The 2018 test locations are: Pierce County, Washington; Providence County, Rhode Island; and the Bluefield- Beckley-Oak Hill area of West Virginia. In our prior work on testing done for the 2010 Census, we noted that a sound study design should include such components as: clearly stated objectives with accompanying performance measures; research questions linked to test objectives and, as appropriate, a clear rationale for why sites were selected for field tests; a thoroughly documented data collection strategy; input from stakeholders and lessons learned considered in developing test objectives; and a data analysis plan including, as appropriate, methods for determining the extent to which specific activities contribute to controlling costs and enhancing quality. Develop and manage on the basis of reliable cost estimates. Reliable cost estimates that appropriately account for risks facing an agency can help an agency manage large complex activities like the 2020 Census, as well as help Congress make funding decisions and provide oversight. Cost estimates are also necessary to inform decisions to fund one program over another, to develop annual budget requests, to determine what resources are needed, and to develop baselines for measuring performance. The Bureau has a history of unreliable cost estimation and resultant overruns. For example, we placed the Decennial Census on our High Risk list in 2008 in part due to weaknesses in the Bureau's estimation of its 2010 Census life-cycle cost. Recently, we reported in our review of the Bureau’s October 2015 life- cycle cost estimate that in order for the Bureau to improve its ability to control the cost of the 2020 Census, it will be critical for it to have better control over its cost estimation process. While we found that the Bureau has taken significant steps toward improving its capacity to produce reliable cost estimates, those efforts had not yet resulted in a reliable decennial cost estimate. Among the four broad characteristics of a reliable cost estimate—none of which the Bureau fully met—the Bureau reported it was focusing its attention on improving the documentation of the cost estimate, in order to help improve other characteristics as well. While poor documentation affected our ability to assess the reliability of the Bureau’s cost estimate’s other characteristics, we believe the problems we observed related to an absence of internal control procedures over the cost estimation process, which resulted in poor documentation. Furthermore, we found the Bureau lacked guidance to control the cost estimation process. Investment in the planning documents to help control and support cost estimation early in the estimation cycle, such as with an operational plan, guidance on key steps and process flows, assignment of responsibilities, and job aids for staff can help institutionalize practices and ensure that otherwise disparate parties in the process operate consistently. As we reported, taking steps to ensure its cost estimate is reliable would help improve decision-making, budget formulation, progress measurement, course correction when warranted, and accountability for results. We made three recommendations including that the Bureau take specific steps to ensure its cost estimate meets the characteristics of a high- quality estimate and improve control over how risk and uncertainty are accounted for in cost estimation, with which the Department of Commerce agreed. Bureau officials have stated that they plan to address the recommendations with their update of the 2020 Census Lifecycle Cost Estimate in December 2016. We plan to assess this cost estimate as soon as it is available. Sustain attention to workforce planning. Strategic workforce planning encourages agency managers and stakeholders to systematically consider what is to be done, when and how it will be done, what skills will be needed, and how to gauge progress and results. Sustained workforce planning can help the Bureau stay on track for the 2020 Census and help avoid past staffing problems. For example, a Bureau assessment of its experience with the 2010 Census observed that areas such as the management of large programs and projects, cost estimation, and information technology (IT) lacked staff with core skills and experience. Moreover, the Bureau’s experience with the 2010 Census and prior enumerations has shown that not following leading practices in workforce planning can increase the risks of subsequent downstream operations, such as cost estimation. In 2012 we reported that while the Bureau’s workforce planning efforts were generally consistent with such key leading practices as identifying current and future critical occupations, the Bureau had not coordinated workforce planning efforts across its directorates for key occupations. Without a Bureau-wide competency assessment, for instance, the Bureau risked not having the necessary workforce in place to manage the multimillion dollar IT investments for its 2020 operations. We found the Bureau also needed to address having inadequately trained cost estimating staff so that it could produce credible, comprehensive, and accurate cost estimates. Moreover, the Bureau needed to devote greater attention to setting goals and monitoring progress for skills gaps—as well as engaging stakeholders in developing, communicating, and implementing its workforce plan—so that the Bureau could identify and avoid possible workforce plan implementation barriers. Since that time, the Bureau has taken actions in response to our recommendations to coordinate and set goals for its workforce planning. For example, in September 2014, the Bureau drafted action plans to address the skills gaps that had been identified as part of a Bureau-wide competency assessment. The Bureau has indicated that a 2020 directorate-wide workforce assessment report is in its final review stages and will include a comprehensive succession planning strategy. These actions taken by the Bureau to incorporate key leading workforce planning practices will help the Bureau meet its objective of having a workforce matched with the demands of the 2020 Census. Going forward, a sustained focus on workforce planning will be necessary to ensure the Bureau will be in a position to hire the optimal mix of managers and technical experts to carry out a cost-effective census. In summary, the key innovations the Bureau plans for 2020 show promise for controlling costs and maintaining accuracy, although there are significant risks involved. The Bureau is aware of these risks, and robust testing can help manage them by assessing the feasibility of key activities, their capacity to deliver desired outcomes, and their ability to work in concert with one another under operational conditions. While the Bureau decided to stop key field testing planned for fiscal year 2017 in order to mitigate a funding risk, this decision may have consequences for elements of field operations not getting tested as a result, and, ultimately, for the 2020 Census. Going forward, once the Bureau has the test results, past experience has also shown the importance of refining operations as needed based on the results of the tests, incorporating lessons learned from 2010 as appropriate, and making needed changes to its design in time to be included in the Bureau’s End-to-End Test scheduled for 2018. Chairman Meadows, Ranking Member Connolly, and Members of the Subcommittee, this completes my prepared statement. I would be pleased to respond to any questions that you may have. If you have any questions on matters discussed in this statement, please contact Robert Goldenkoff at (202) 512-2757 or by e-mail at goldenkoffr@gao.gov. Other key contributors to this testimony include Lisa Pearson, Assistant Director; Mark Abraham, Peter Beck; Devin Braun; Jeff DeMarco; Robert Gebhart; Emily Hutz; Richard Hung; Donna Miller; Ty Mitchell; Kayla Robinson; Kathleen Padulchick; Robert Robinson, and Timothy Wexler. This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.

Summary: With a life-cycle cost of about $12.3 billion, the 2010 Census was the most expensive U.S. census in history. To help control costs and maintain accuracy, the 2020 Census design includes new procedures and technology that have not been used extensively in earlier decennials, if at all. While these innovations show promise for a more cost-effective head count, they also introduce risks. As a result, it is important to thoroughly test the operations planned for 2020.This testimony focuses on (1) the preliminary results to date of the Bureau's 2016 Census Test in Los Angeles County, California, and Harris County, Texas; (2) the Bureau's plans for the upcoming test of address canvassing procedures in Buncombe County, North Carolina, and St. Louis, Missouri; and (3) the lessons learned from the 2010 Census that can be applied to preparations for 2020. This testimony is based on GAO's ongoing reviews of the 2016 Census Test and Address Canvassing Test. For these studies, GAO reviewed Bureau documents and preliminary data, interviewed Bureau officials, and made site visits to observe census operations. This testimony is also based on prior GAO work on lessons learned from the 2010 Census. GAO preliminarily found that during the 2016 Census Test, nonresponse follow-up (NRFU), where enumerators visit households that did not respond to the census, generally proceeded according to the Census Bureau's (Bureau) operational plans. However, data at both test sites indicate that the Bureau experienced a large number of non-interviews. The Bureau considers non-interviews to be cases where either no data or insufficient data are collected. Bureau officials are not certain why there were so many non-interviews for the test and are researching potential causes. In addition, the Bureau's plan to automate the assignment of NRFU cases to enumerators has the potential to deliver significant efficiency gains as compared to paper-based operations conducted in previous decennial censuses, according to the Bureau. GAO preliminarily found that improvements to certain enumeration procedures and better training could produce additional efficiencies by enabling the Bureau to be more responsive to situations enumerators encounter on the ground. These improvements include providing more flexible access to recently closed, incomplete cases; enumerator interview training with multi-unit property managers; and operational procedures to make use of local data on the best time to attempt interviews. The Bureau has reengineered its approach to building its master address list for 2020 in part by introducing a two-phase "in-office" process that systematically reviews small geographic areas nationwide. The goal is to limit the more expensive and traditional door-to-door canvassing to those areas most in need of updating, such as areas with recent housing growth. The in-office phases rely on aerial imagery, street imagery, geographic information systems and address file data from state, local, and tribal partners. The Bureau estimates that the new process will result in about 25 percent of housing units requiring field canvassing compared to the traditional process where all housing units were canvassed. The Bureau has identified a series of risks that could affect the cost or quality of the address canvassing operation, including locating hidden housing units such as converted garages, monitoring change in housing stock, and obtaining quality data. The Bureau is testing its reengineered address canvassing operation in two sites through December 2016--in Buncombe County, North Carolina, and St. Louis. The Bureau's experience in planning for the 2010 decennial can enhance its readiness for 2020. Going forward, GAO's prior work indicates it will be important for the Bureau to address several key lessons learned, including: (1) ensuring key census-taking activities are fully tested, (2) developing and managing on the basis of reliable cost estimates, and (3) sustaining workforce planning efforts to ensure it has the optimal mix of skills to cost-effectively conduct the enumeration. GAO has made several recommendations to the Census Bureau in prior reports on cost estimation and workforce planning. The Bureau has implemented the workforce planning recommendations, and agreed with and plans to implement the cost estimation recommendations.