Decennial Census: Progress Report on Preparations for 2020
Published: 2016-11-16T00:00:00

Fast Facts:
Is the Census Bureau ready for the 2020 Census?
If you don't send in your census form, you can expect a census-taker to interview you in person. However, in this year's tests, up to 25% of those types of cases were closed without a complete interview. This testimony discusses some of the possible reasons why this happened.
We also provide information about other preparations for the 2020 Census, including innovations in validating addresses, and applying lessons learned from the 2010 Census.

Highlights:
section_title: Why GAO Did This Study
paragraphs: With a life-cycle cost of about $12.3 billion, the 2010 Census was the most expensive U.S. census in history. To help control costs and maintain accuracy, the 2020 Census design includes new procedures and technology that have not been used extensively in earlier decennials, if at all. While these innovations show promise for a more cost-effective head count, they also introduce risks. As a result, it is important to thoroughly test the operations planned for 2020.This testimony focuses on (1) the preliminary results to date of the Bureau's 2016 Census Test in Los Angeles County, California, and Harris County, Texas; (2) the Bureau's plans for the upcoming test of address canvassing procedures in Buncombe County, North Carolina, and St. Louis, Missouri; and (3) the lessons learned from the 2010 Census that can be applied to preparations for 2020.
This testimony is based on GAO's ongoing reviews of the 2016 Census Test and Address Canvassing Test. For these studies, GAO reviewed Bureau documents and preliminary data, interviewed Bureau officials, and made site visits to observe census operations. This testimony is also based on prior GAO work on lessons learned from the 2010 Census.
section_title: What GAO Found
paragraphs: GAO preliminarily found that during the 2016 Census Test, nonresponse follow-up (NRFU), where enumerators visit households that did not respond to the census, generally proceeded according to the Census Bureau's (Bureau) operational plans. However, data at both test sites indicate that the Bureau experienced a large number of non-interviews. The Bureau considers non-interviews to be cases where either no data or insufficient data are collected. Bureau officials are not certain why there were so many non-interviews for the test and are researching potential causes. In addition, the Bureau's plan to automate the assignment of NRFU cases to enumerators has the potential to deliver significant efficiency gains as compared to paper-based operations conducted in previous decennial censuses, according to the Bureau. GAO preliminarily found that improvements to certain enumeration procedures and better training could produce additional efficiencies by enabling the Bureau to be more responsive to situations enumerators encounter on the ground. These improvements include providing more flexible access to recently closed, incomplete cases; enumerator interview training with multi-unit property managers; and operational procedures to make use of local data on the best time to attempt interviews.
The Bureau has reengineered its approach to building its master address list for 2020 in part by introducing a two-phase "in-office" process that systematically reviews small geographic areas nationwide. The goal is to limit the more expensive and traditional door-to-door canvassing to those areas most in need of updating, such as areas with recent housing growth. The in-office phases rely on aerial imagery, street imagery, geographic information systems and address file data from state, local, and tribal partners. The Bureau estimates that the new process will result in about 25 percent of housing units requiring field canvassing compared to the traditional process where all housing units were canvassed. The Bureau has identified a series of risks that could affect the cost or quality of the address canvassing operation, including locating hidden housing units such as converted garages, monitoring change in housing stock, and obtaining quality data. The Bureau is testing its reengineered address canvassing operation in two sites through December 2016--in Buncombe County, North Carolina, and St. Louis.
The Bureau's experience in planning for the 2010 decennial can enhance its readiness for 2020. Going forward, GAO's prior work indicates it will be important for the Bureau to address several key lessons learned, including: (1) ensuring key census-taking activities are fully tested, (2) developing and managing on the basis of reliable cost estimates, and (3) sustaining workforce planning efforts to ensure it has the optimal mix of skills to cost-effectively conduct the enumeration.
section_title: What GAO Recommends
paragraphs: GAO has made several recommendations to the Census Bureau in prior reports on cost estimation and workforce planning. The Bureau has implemented the workforce planning recommendations, and agreed with and plans to implement the cost estimation recommendations.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the progress of the U.S. Census  Bureau’s (Bureau) preparations for the 2020 Census. As requested, my  remarks will focus on (1) the preliminary results to date of the Bureau’s  2016 Census Test in Los Angeles (L.A.) County, California, and Harris  County, Texas; (2) the status of the Bureau’s test of address canvassing  procedures in Buncombe County, North Carolina, and St. Louis, Missouri;  and (3) the lessons learned from the 2010 Census that can be applied to  the Bureau’s preparations for 2020. In his statement today, my colleague  will discuss the Bureau’s approach to deliver an enterprise information  technology initiative, and to ensure the integrity and security of systems  and data in support of the 2020 Census. We anticipate issuing a report  on the Bureau’s 2016 test early in the new year.
Sufficient testing, while important to the success of any census, is even  more critical for the Bureau’s preparations for 2020. To help control costs  and maintain accuracy, the 2020 Census design includes new procedures  and technology that have not been used extensively in earlier decennials,  if at all. While these innovations show promise for a more cost-effective  head count, they also introduce new risks. As we have noted in our prior  work, it will be important to thoroughly test the operations planned for  2020 to ensure they will (1) produce needed cost savings, (2) function in  concert with other census operations, and (3) work at the scale needed  for the national head count. The Bureau’s failure to fully test some key  operations prior to the 2010 Census was a key factor that led us to  designate that decennial as one of our high-risk areas.
A key objective of the 2016 Census Test in Harris and L.A. Counties was  to refine the methodology for nonresponse follow-up (NRFU), where  enumerators personally visit households that do not self-respond to the  census. NRFU is the largest and costliest of all census-taking activities  because it is so labor intensive. The Bureau selected Harris and L.A.  counties as test sites for several reasons including language diversity,  demographic diversity, high vacancy rates, and varying levels of Internet  usage. There are around 225,000 housing units in each test area. The  Bureau estimates that re-engineering its field procedures could save as  much as $2.5 billion.
In conducting the 2016 Address Canvassing Test, which began in August  2016, the Bureau is to measure the effectiveness of new procedures for  building its address list. Buncombe County, North Carolina, is a mix of  urban, suburban, and rural territories while St. Louis is a principal city.  Accurate addresses and precise maps are critical for the census, in part  because census data are used for congressional apportionment,  redistricting, and allocations of federal aid to state and local governments.  In prior decades, the Bureau employed field staff to walk almost every  street in the nation as one of several operations to update the Bureau’s  inventory of addresses and geography.
For 2020, the Bureau plans to target its traditional or “in-field” canvassing  efforts to those areas most in need of updating such as those that have  experienced rapid recent housing development and for which the Bureau  has no data sources capturing those changes. The Bureau will rely on “in- office” procedures to update the majority of addresses in the country.  These procedures include validating addresses through aerial imagery  and by using data from the U.S. Postal Service as well as from state,  local, and tribal governments. The Bureau estimates it will save up to $1  billion with the successful implementation of this initiative.
My testimony is based on our ongoing reviews of the 2016 Census Test  and Address Canvassing Test. For these studies, we reviewed Bureau  documents and preliminary data, interviewed local and headquarters  Bureau officials, and for the 2016 Census Test, made several site visits to  Los Angeles and Harris Counties to observe NRFU procedures. For the  Address Canvassing Test, we made site visits to Buncombe County,  North Carolina and St. Louis to observe in-field address canvassing  procedures and to the Bureau’s National Processing Center in  Jeffersonville, Indiana to observe in-office canvassing. These  observations are not generalizable. On September 16, 2016, we shared  the information included in this statement with the Census Bureau for its  review. On September 21, 2016, we met with Bureau officials and they  provided technical comments which we included, as appropriate. My  testimony is also based on our prior work on the Bureau’s preparations  for 2020, as well as our work on lessons learned from the conduct of the  2010 Census. For those studies, we reviewed Bureau planning  documents and test plans, interviewed Bureau officials, and made site  visits to observe how census operations were being implemented in the  field.
The work on which this statement is based was conducted in accordance  with generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.
subsections: section_title: Background
paragraphs: With a cost of about $12.3 billion, the 2010 Census was the most  expensive population count in U.S. history, costing about 31 percent more  than the $9.4 billion 2000 Census (in constant 2020 dollars). Some cost  growth is to be expected because the population is growing and  becoming more complex and difficult to count, which increases the  Bureau’s workload. However, the cost of counting each housing unit has  escalated from about $16 in 1970 to $92 in 2010 (in constant 2020  dollars), according to the Bureau.
For the 2020 Census, the Bureau intends to limit its per-household cost to  not more than that of the 2010 Census, adjusted for inflation. To achieve  this goal, the Bureau is significantly changing how it conducts the census,  in part by re-engineering key census-taking methods and infrastructure.  The Bureau’s innovations include (1) using the Internet as a self-response  option; (2) verifying most addresses using “in-office” procedures rather  than costly field canvassing; (3) re-engineering data collection methods;  and (4) in certain instances, replacing enumerator-collected data with  administrative records (information already provided to federal and state  governments as they administer other programs).
The Bureau’s various initiatives have the potential to reduce costs. In  October 2015, the Bureau estimated that with its new approach it can  conduct the 2020 Census for a life-cycle cost of $12.5 billion, $5.2 billion  less than if it were to repeat the design and methods of the 2010 Census  (both in constant 2020 dollars). However, in June 2016, we reported that  this $12.5 billion cost estimate was not reliable and did not adequately  account for risk. Table 1 below shows the Bureau’s estimated cost  savings it hopes to achieve in the following four innovation areas.
The 2016 test was the latest major test of NRFU in the Bureau’s testing  program.  In 2014, the Bureau tested new methods for conducting NRFU  in the Maryland and Washington, D.C., area. In 2015, the Bureau  assessed NRFU operations, in Maricopa County, Arizona. In 2018, the  Bureau plans to conduct a final “End-to-End” test which is essentially a  dress rehearsal for the actual decennial. The Bureau needs to finalize the  census design by the end of fiscal year 2017 so that key activities can be  included in the End-to-End Test.
The Bureau plans to conduct additional research through 2018 in order to  further refine the design of the 2020 Census, but recently had to alter its  approach.  On October 18, 2016, the Bureau decided to stop two field test  operations planned for fiscal year 2017 in order to mitigate risks from  funding uncertainty. Specifically, the Bureau said it would stop all planned  field activity, including local outreach and hiring, at its test sites in Puerto  Rico, North and South Dakota, and Washington State. The Bureau will  not carry out planned field tests of its mail-out strategy and follow up for  non-response in Puerto Rico, or its door-to-door enumeration. The  Bureau also cancelled plans to update its address list in the Indian lands  and surrounding areas in the three states.
However, the Bureau will continue with other planned testing in fiscal year  2017, such as those focusing on systems readiness and internet  response. Further, the Bureau said it would consider incorporating the  cancelled field activities elements within the 2018 End-to-End Test. The  Bureau maintains that stopping the 2017 Field Test will help prioritize  readiness for the 2018 End-to-End Test, and mitigate risk. Nevertheless it  also represents a lost opportunity to test, refine, and integrate operations  and systems, and puts more pressure on the 2018 Test to demonstrate  that enumeration activities will function as needed for 2020.
section_title: 2016 Census Test Highlighted NRFU Challenges
paragraphs: NRFU generally proceeded according to the Bureau’s operational plans.  However, our observations and the Bureau’s preliminary data at both test  sites found that (1) there were a large number of non-interviews, and (2)  enumerators had difficulty implementing new census-taking procedures.
The Bureau’s 2016 Census Test included a new field management  structure that, among other things, included an enhanced operations  control system supporting daily assignments of cases. A cornerstone of  the Bureau’s efforts to reduce the cost of NRFU is the automation of  decision-making on how to manage the follow-up caseload. Unlike  previous censuses and one prior test, enumerators in the 2016 Census  Test did not have an assigned set of cases that they alone would work  until completion. Instead, the Bureau relied on an enhanced operational  control system that was designed to provide daily assignments and street  routing of non-response follow-up cases to enumerators in the most  optimal and efficient way. The Bureau first tested this system in the 2015  Census Test. The test also included streamlined procedures for making  contact at large apartment buildings. This was intended to reduce  repeated attempts to contact property managers.
A key objective of the 2016 Census Test was to refine procedures for  collecting NRFU data from households using mobile devices leased from  a contractor. In prior decennials, enumerators collected NRFU information  using paper and pencil. The Bureau believes that replacing paper-based  operations with automated case management and mobile devices for  collecting interview data will provide a faster, more accurate, and more  secure means of data collection in the 2020 Census (see figure 1).
Some test activities that we observed at both test sites included  streamlined multi-unit contact procedures and interviews with a proxy  respondent. A proxy is someone who is a non-household member, at  least 15 years old, and knowledgeable about the NRFU address. At multi- unit structures such as apartment buildings, the enumerator is trained to  first interview the property manager to find out which units were occupied  and which were vacant on Census Day. Such interviews help to  streamline NRFU by removing vacant units from an enumerator’s  workload. They also help build a rapport with property managers by  ensuring they know when enumerators are working in their building and  can also help them gain access to locked buildings.
subsections: section_title: The Bureau Needs to Better Understand Factors Contributing to NRFU Non-interviews
paragraphs: Preliminary data at both test sites indicate that the Bureau experienced a  large number of non-interviews. According to the Bureau, non-interviews  are cases where either no data or insufficient data were collected, in part  because the cases reached the maximum number of six attempted visits  without success or were not completed due to, for example, language  barriers or dangerous situations. While not necessarily a precursor to the  2020 non-interview rate, because of its relationship to the cost and quality  of the count, it will be important for the Bureau to better understand the  factors contributing to it.
According to preliminary 2016 Census Test data, there were 19,721  NRFU cases coded as non-interviews in Harris County, Texas and  14,026 in Los Angeles County, California, or about 30 and 20 percent of  the test workload respectively. In such cases, the Bureau may have to  impute attributes of the household based on the demographic  characteristics of surrounding housing units as well as administrative  records.
Bureau officials expect higher numbers of non-interviews during tests in  part because, compared to the actual enumeration, the Bureau conducts  less outreach and promotion. Bureau officials hypothesized that another  contributing factor could be related to NRFU methods used in the 2016  test compared to earlier decennials.  For the 2010 and earlier decennials,  enumerators collected information during NRFU using pencil and paper.  Enumerators may have visited a housing unit more than the 6 maximum  allowable visits to obtain an interview but did not record all of their  attempts, thus enabling them to achieve a higher completion rate. For the  2020 Census, and as tested in 2016, the Bureau plans to collect data  using mobile devices leased from a contractor, and an automated case  management system to manage each household visit. The Bureau  believes that this approach will provide a faster, more accurate, and more  secure means of data collection.  At the same time, the mobile device and  automated case management system did not allow an enumerator to  attempt to visit a housing unit more than once per day, reopen a closed  case, or exceed the maximum allowable six attempts.
One factor we observed that may have contributed to the non-interview  rate was that enumerators did not seem to uniformly understand nor  follow procedures for completing interviews with proxy respondents.  According to the 2016 Census Test enumerator training manual, when an  eligible respondent at the address cannot be located, the automated case  management system on the mobile device will prompt the enumerator  when to find a proxy to interview, such as when no one is home or the  housing unit appears vacant. In such circumstances, enumerators are to  find a neighbor or landlord to interview. However, in the course of our site  visits, we observed that enumerators did not always follow these  procedures. For example, one enumerator, when prompted to find a  proxy, looked to the left and then right and, finding no one, closed the  case. Similarly, another enumerator ignored the prompt to find a proxy  and explained that neighbors are usually not responsive or willing to  provide information about the neighbor, and did not seek to find a proxy.  Enumerators we interviewed did not seem to understand the importance  of obtaining a successful proxy interview, and many appeared to have  received little encouragement during training to put in the effort to find a  proxy.
Proxy data for occupied households are important to the success of the  census as the alternative is a non-interview. In 2010, about one-fourth of  the NRFU interviews for occupied housing units were conducted using  proxy data. We shared our observations with Bureau officials who told us  that they are aware that enumerator training for proxies needs to be  revised to convey the importance of collecting proxy data when  necessary. Converting non-interviews by collecting respondent or proxy  data can improve interview completion rates, and ultimately the quality of  census data. The Bureau told us it will continue to refine procedures for  2020.
section_title: Enumerators Faced Challenges Implementing New Procedures
paragraphs: According to the Bureau, its plans to automate the assignment of NRFU  cases have the potential to deliver significant efficiency gains. At the  same time, refinements to certain enumeration procedures and better  communication could produce additional efficiencies by enabling the  Bureau to be more responsive to situations enumerators encounter in the  course of their follow-up work.
Enumerators were unable to access recently closed incomplete  cases. Under current procedures, if an enumerator is unable to make  contact with a household member, the case management system closes  that case and it is to be reattempted at a later date, perhaps by a different  enumerator, assuming the enumerator has not exceeded six attempts.  Decisions on when reattempts will be made—and by whom—are  automated and not designed to be responsive to the immediate  circumstances on the ground. This is in contrast to earlier decennials  when enumerators, using paper-based data collection procedures, had  discretion and control over when to re-attempt cases in the area where  they were working. According to the Bureau, leaving cases open for re- attempts can undermine the efficiency gains of automation when  enumerators depart significantly from their optimized route, circling back  needlessly to previously attempted cases rather than progressing through  their scheduled workload.
During our test site observations, however, we preliminarily found how  this approach could lead to inefficiencies in certain circumstances. For  example, we observed enumerators start their NRFU visits in the early  afternoon as scheduled, when many people are out working or are  otherwise away. If no one answered the door, those cases were closed  for the day and reassigned later. However, if a household member  returned while the enumerator was still around, the enumerator could not  reopen the case and attempt an interview. We saw this at both test site  locations, typically in apartment buildings or at apartment-style gated  communities, where enumerators had clear visibility to a large number of  housing units and could easily see people arriving home.
Bureau officials acknowledged that closing cases in this fashion  represented a missed opportunity and plan to test greater flexibilities as  part of the 2018 End-to-End Test. Programming some flexibility into the  mobile device—if accompanied with adequate training on how and when  to use it—should permit completion of some interviews without having to  deploy staff to the same case on subsequent days. This in turn could  reduce the cost of follow-up attempts and improve interview completion  rates.
Enumerators did not understand procedures for visits to property  managers. Property managers are a key source of information on non- respondents when enumerators cannot find people at home. They can  also facilitate access to locked buildings. Further, developing a rapport  with property managers has helped the NRFU process, such as when  repeated access to a secured building or residential complex is needed  on subsequent days by different enumerators.
In response to problems observed during the Bureau’s 2014 and 2015  Census tests and complaints from property managers about multiple  uncoordinated visits by enumerators, the Bureau’s 2016 Census Test  introduced specific procedures to conduct initial visits to property  managers in large multi-unit apartment buildings. The procedures sought  to identify up front which, if any, units needing follow-up at the location  were vacant, eliminating the need for enumerators to collect this  information from property managers with subsequent visits on a case-by- case basis. According to Bureau officials, the automated case  management system was designed to allow for an enumerator to make  up to three visits to property managers to remove vacant units.
According to the Bureau, the 2016 Census Test demonstrated that vacant  units could quickly be removed from the NRFU workload using these  procedures in cases where a property manager was readily available;  however, in other cases the procedures caused confusion. For example,  whenever an initial visit was unsuccessful, all of the cases at that  location—up until then collated into only one summary row of the  enumerator’s on-screen case list—would suddenly expand and appear as  individual cases to be worked, sometimes adding several screens and  dozens of cases to the length of the list, which enumerators we spoke  with found confusing. Furthermore, without the knowledge of which units  were vacant, enumerators may have unnecessarily made visits to these  units and increased the cost and the time required to complete NRFU .
During debriefing sessions the Bureau held, Bureau enumerators and  their supervisors identified training in these procedures as an area they  felt needed greater attention in the future. Indeed, while training classes  included a case study exercise on interviewing a property manager, this  exercise in the enumerators training manual gives no warning to  enumerators and does not refer to the procedures. Bureau officials said  that they are pleased with the progress the test demonstrates they have  made in automating case management at multi-unit locations a priority.  They added that they recognize the need to better integrate procedures in  the training moving forward.
Timing of return visits did not leverage information on respondent  availability. During our field visits, we encountered several instances  where enumerators had been told by a respondent or otherwise learned  that returning at a specific time on a later date would improve their  chance of obtaining an interview from either a household respondent or a  property manager. But the Bureau’s 2016 Census Test and automated  case management did not have an efficient way to leverage that  information. Attempting contact at non-responding households at times  respondents are expected to be available can increase the completion  rate and reduce the need to return at a later date or rely on proxy  interviews as a source of information.
The Bureau’s automated case management system assigned cases to 6- hour time windows after estimating hour-by-hour probabilities of when  best to contact people. The estimation relied on various administrative  records, information from other Bureau surveys that had successful  contacts in the past, as well as area characteristics. The 2016 Census  Test did not have a way to change or update these estimates when cases  were subsequently reassigned. The goals of assigned time windows were  intended to result in more productive visits and reduce costs.
When enumerators identified potentially better times to attempt a contact,  they were instructed to key in this information into their mobile devices.  For example, one enumerator keyed in a mother’s request to come back  on Thursday afternoon when her kids were in camp, while others keyed-in  information like office hours and telephone contact numbers obtained  from signs on the property they had seen for property managers.  However, according to the Bureau, this updated information went unused,  and we met enumerators who had been assigned to enumerate  addresses at the same unproductive time after they had written notes  documenting other better times to visit. Another enumerator reported  visiting a property manager who complained that the enumerator was not  honoring the manager’s earlier request made during a prior enumeration  attempt that an enumerator return during a specified time window. Such  repeat visits can waste enumerator time (and miles driven), and  contribute to respondent burden or reduced data quality when  respondents become annoyed and may become less cooperative.
We discussed our preliminary observation with managers at the test sites,  who expressed frustration that the automated case management system  did not allow them to record the locally-obtained data on when to contact  people whom they found in enumerator notes in a way to affect future  case assignment. Headquarters staff told us that while they have not fully  evaluated this yet, they are concerned that providing local managers with  too much flexibility to override the results of optimized case and time  assignments would undermine the efficiency gains achievable by the  automation. They also explained that enumerators were to have been  provided capability to record what day or what time of day for follow-up.  This information could have been used by the automated case  management to better target the timing of future assignments. However,  they acknowledged that this procedure may not have been either fully  implemented or explained during enumerator training. Bureau officials  have said that this is another area they are planning to address.
section_title: The Bureau Has Fundamentally Re- engineered Address Canvassing for 2020
paragraphs: The Bureau has reengineered its approach to building its master address  list for 2020. Specifically, by relying on multiple sources of imagery and  administrative data, the Bureau anticipates constructing its address list  with far less door-to-door field canvassing compared to previous  censuses.
One major change the Bureau is making consists of using in-office  address canvassing–a two-phase process that systematically reviews  small geographic areas nationwide, known as census blocks, to identify  those that will not need to be canvassed in the field, as shown in figure 2.
The Bureau estimates that the two phases of in-office canvassing will  result in roughly 25 percent of housing units requiring in-field canvassing,  instead of canvassing nearly all housing units in the field as done  previously. With in-office address canvassing clerks compare current  aerial imagery for a given block with imagery for that block dating to the  time of the last decennial census in 2010. During this first phase, called  Interactive Review, specially trained clerks identify whether a block  appears to have experienced change in the number of housing units,  flagging each block either as stable—free of population growth, decline,  or uncertainty in what is happening in the imagery over time—or “active,”  in which case it moves to the next phase. Addresses in stable blocks are  not marked for in-field canvassing.
For blocks where change is detected or suspected, the Bureau uses a  second phase of in-office canvassing, known as Active Block Resolution,  to attempt to resolve the status of each address and housing unit in  question within that block. During this phase, clerks use aerial imagery,  street imagery, and data from the U.S. Postal Service, as well as from  state, local, and tribal partners when reviewing blocks. If a block can be  fully resolved during this phase of in-office canvassing, the changes are  recorded in the Bureau’s master address file. If a block cannot be fully  resolved during the second phase of in-office canvassing, then the entire  block, or some portion of the block, is flagged for inclusion in the in-field  canvassing operation. In-office address canvassing began in September  2015 with plans for a first pass of the entire country to be completed by  the end of fiscal year 2018. In-field canvassing for the 2020 Census is  scheduled to begin in August 2019.
Another major change the Bureau is making for its re-engineered address  canvassing is significantly expanding the role that state, local, and tribal  partners can play throughout the decade in contributing to an accurate,  more up-to-date address list. Through the Geographic Support Systems  Initiative, begun in fiscal year 2011, partner jurisdictions have been  providing address and spatial data to the Bureau to help validate and  supplement the Bureau’s address list. As of October 2016, the Bureau  reported that it had received partner data covering 73 percent of all  known housing units nationwide. It added that the vast majority of the  addresses in the files that the Bureau had processed as of July 2015  have either been matched with existing addresses in its database, or  added to the address list. As with previous decennial censuses, as  directed by Congress, the Bureau will also engage with state, local, and  tribal partners through its Local Update of Census Addresses program  in fiscal years 2018 and 2019 in order to ensure that jurisdictions have the  ability to comment on the address list prior to enumeration. The Bureau  plans to rely on the in-office part of address canvassing to validate a large  part of the addresses added to the list during that program where data are  available to permit it.
The Bureau is testing its re-engineered address canvassing operation in  two sites through December 2016—in Buncombe County, North Carolina,  and St. Louis, Missouri. In-office canvassing for the test sites began at the  Bureau’s National Processing Center in Jeffersonville, Indiana, in August  2016. The exercise will test the Bureau’s assumptions about the cost and  effectiveness of the re-engineered approach, as well as the quality of in- office canvassing, field staff training, and the use of new collection  geography in the field. In addition to the 100 percent in-office canvassing  the Bureau plans for 2020, the Bureau will also canvass 100 percent of  the test areas in the field so that it can compare results it obtains for  blocks where it would not otherwise have gone door to door. The Bureau  hired 262 in-field listers across both sites to conduct the door-to-door  canvassing, also beginning in October with a relisting operation  commencing in November.
Although the innovations the Bureau is planning with its reengineered  address canvassing have the potential to reduce costs, they entail some  risks that could affect the cost or quality of the address canvassing  operation. According to the Bureau, these risks include:
Locating Hidden Housing Units. The Bureau recognizes that certain  kinds of dwellings are hard to identify and may not have been marked  as housing units at the time of address list development and not  included in any databases. This could lead to their being missed and  occupants not being counted in the census. These units are referred  to as hidden housing units and include such living arrangements as  attics, basements, or garages converted into housing units. According  to the Bureau, while in-field canvassing also has similar risks for  missing these types of housing units, solely relying on the use of  imagery to identify these units could lead to an incomplete address  list.
Monitoring Change in the Housing Stock. When the Bureau  determines during the first phase of in-office canvassing that a block  has not experienced population change, the Bureau plans to subject  the block to later monitoring so that if later change is detected, the  block can be reassigned for further review. The Bureau has  developed the conditions or “triggers” for subjecting blocks to later  monitoring, but has not yet determined how it will operationalize them.  According to the Bureau, if the triggers that the Bureau is developing  for this process do not adequately detect recent change, then housing  unit growth may be missed, and the resulting address list may not be  up-to-date.
Obtaining Quality Data. For the Bureau to adequately review enough  blocks in-office–and therefore reduce field costs of door-to-door  canvassing–the Bureau needs to have data of sufficient quality to  make reliable determinations about changes in housing units within  those blocks. According to the Bureau, if it does not obtain sufficient  satellite imagery (covering areas with both current and prior census  imagery) or address and spatial data from state/local/tribal partners,  then it may be forced to send more blocks than planned to in-field  canvassing.
We have ongoing audit work examining the Bureau’s re-engineered  address canvassing approach. The justification of key cost and data  quality assumptions, the approaches to mitigating key risks, and the  Bureau’s adherence to timelines and canvassing schedules are all  subjects of our ongoing work, which we plan to report on early next year.
section_title: Lessons Learned from the 2010 Census Can Be Applied to Preparations for 2020
paragraphs: The Bureau goes to great lengths each decade to improve specific  census-taking activities. But these incremental modifications have not  kept pace with societal changes that make the population increasingly  difficult to locate and cost-effectively count. This increasing difficulty and  escalating costs led the Bureau to re-engineer its approach for the 2020  Census. While preparations for 2020 are still underway, and with testing  still occurring, the Bureau’s experience in planning for 2010 can enhance  its readiness for 2020. For example, as the Bureau continues its planning  efforts for 2020, our prior work indicates that it will be essential for it to  address the following three lessons learned:
Ensure key census-taking activities are fully tested
Develop and manage on the basis of reliable cost estimates
Sustain workforce planning  Ensure key census-taking activities are fully tested. The census is a  large, complex operation comprised of thousands of moving parts, all of  which must function in concert with one another to secure a cost-effective  count. While the census is under way, the tolerance for any breakdowns  is quite small. Given this difficult operating environment, rigorous testing  is a critical risk mitigation strategy because it provides information on the  feasibility and performance of individual census-taking activities, their  potential for achieving desired results, and the extent to which they are  able to function together under full operational conditions. Given the new  four innovation areas for the 2020 Census, it will be imperative that the  Bureau have systems and operations in place for the 2018 End-to-End  Test that will take place in three locations, covering more than 700,000  housing units in total. The 2018 test locations are: Pierce County,  Washington; Providence County, Rhode Island; and the Bluefield- Beckley-Oak Hill area of West Virginia.
In our prior work on testing done for the 2010 Census, we noted that a  sound study design should include such components as:  clearly stated objectives with accompanying performance measures;  research questions linked to test objectives and, as appropriate, a  clear rationale for why sites were selected for field tests; a thoroughly documented data collection strategy; input from stakeholders and lessons learned considered in developing  test objectives; and a data analysis plan including, as appropriate, methods for  determining the extent to which specific activities contribute to  controlling costs and enhancing quality.
Develop and manage on the basis of reliable cost estimates. Reliable  cost estimates that appropriately account for risks facing an agency can  help an agency manage large complex activities like the 2020 Census, as  well as help Congress make funding decisions and provide oversight.  Cost estimates are also necessary to inform decisions to fund one  program over another, to develop annual budget requests, to determine  what resources are needed, and to develop baselines for measuring  performance. The Bureau has a history of unreliable cost estimation and  resultant overruns. For example, we placed the Decennial Census on our  High Risk list in 2008 in part due to weaknesses in the Bureau's  estimation of its 2010 Census life-cycle cost.
Recently, we reported in our review of the Bureau’s October 2015 life- cycle cost estimate that in order for the Bureau to improve its ability to  control the cost of the 2020 Census, it will be critical for it to have better  control over its cost estimation process. While we found that the Bureau  has taken significant steps toward improving its capacity to produce  reliable cost estimates, those efforts had not yet resulted in a reliable  decennial cost estimate.
Among the four broad characteristics of a reliable cost estimate—none of  which the Bureau fully met—the Bureau reported it was focusing its  attention on improving the documentation of the cost estimate, in order to  help improve other characteristics as well. While poor documentation  affected our ability to assess the reliability of the Bureau’s cost estimate’s  other characteristics, we believe the problems we observed related to an  absence of internal control procedures over the cost estimation process,  which resulted in poor documentation.
Furthermore, we found the Bureau lacked guidance to control the cost  estimation process. Investment in the planning documents to help control  and support cost estimation early in the estimation cycle, such as with an  operational plan, guidance on key steps and process flows, assignment of  responsibilities, and job aids for staff can help institutionalize practices  and ensure that otherwise disparate parties in the process operate  consistently. As we reported, taking steps to ensure its cost estimate is  reliable would help improve decision-making, budget formulation,  progress measurement, course correction when warranted, and  accountability for results.
We made three recommendations including that the Bureau take specific  steps to ensure its cost estimate meets the characteristics of a high- quality estimate and improve control over how risk and uncertainty are  accounted for in cost estimation, with which the Department of Commerce  agreed. Bureau officials have stated that they plan to address the  recommendations with their update of the 2020 Census Lifecycle Cost  Estimate in December 2016. We plan to assess this cost estimate as  soon as it is available.
Sustain attention to workforce planning. Strategic workforce planning  encourages agency managers and stakeholders to systematically  consider what is to be done, when and how it will be done, what skills will  be needed, and how to gauge progress and results. Sustained workforce  planning can help the Bureau stay on track for the 2020 Census and help  avoid past staffing problems. For example, a Bureau assessment of its  experience with the 2010 Census observed that areas such as the  management of large programs and projects, cost estimation, and  information technology (IT) lacked staff with core skills and experience.  Moreover, the Bureau’s experience with the 2010 Census and prior  enumerations has shown that not following leading practices in workforce  planning can increase the risks of subsequent downstream operations,  such as cost estimation.
In 2012 we reported that while the Bureau’s workforce planning efforts  were generally consistent with such key leading practices as identifying  current and future critical occupations, the Bureau had not coordinated  workforce planning efforts across its directorates for key occupations.  Without a Bureau-wide competency assessment, for instance, the Bureau  risked not having the necessary workforce in place to manage the  multimillion dollar IT investments for its 2020 operations. We found the  Bureau also needed to address having inadequately trained cost  estimating staff so that it could produce credible, comprehensive, and  accurate cost estimates. Moreover, the Bureau needed to devote greater  attention to setting goals and monitoring progress for skills gaps—as well  as engaging stakeholders in developing, communicating, and  implementing its workforce plan—so that the Bureau could identify and  avoid possible workforce plan implementation barriers.
Since that time, the Bureau has taken actions in response to our  recommendations to coordinate and set goals for its workforce planning.  For example, in September 2014, the Bureau drafted action plans to  address the skills gaps that had been identified as part of a Bureau-wide  competency assessment. The Bureau has indicated that a 2020  directorate-wide workforce assessment report is in its final review stages  and will include a comprehensive succession planning strategy.
These actions taken by the Bureau to incorporate key leading workforce  planning practices will help the Bureau meet its objective of having a  workforce matched with the demands of the 2020 Census. Going forward,  a sustained focus on workforce planning will be necessary to ensure the  Bureau will be in a position to hire the optimal mix of managers and  technical experts to carry out a cost-effective census.
In summary, the key innovations the Bureau plans for 2020 show promise  for controlling costs and maintaining accuracy, although there are  significant risks involved. The Bureau is aware of these risks, and robust  testing can help manage them by assessing the feasibility of key  activities, their capacity to deliver desired outcomes, and their ability to  work in concert with one another under operational conditions. While the  Bureau decided to stop key field testing planned for fiscal year 2017 in  order to mitigate a funding risk, this decision may have consequences for  elements of field operations not getting tested as a result, and, ultimately,  for the 2020 Census.
Going forward, once the Bureau has the test results, past experience has  also shown the importance of refining operations as needed based on the  results of the tests, incorporating lessons learned from 2010 as  appropriate, and making needed changes to its design in time to be  included in the Bureau’s End-to-End Test scheduled for 2018.
Chairman Meadows, Ranking Member Connolly, and Members of the  Subcommittee, this completes my prepared statement. I would be  pleased to respond to any questions that you may have.
section_title: GAO Contacts and Staff Acknowledgments
paragraphs: If you have any questions on matters discussed in this statement, please  contact Robert Goldenkoff at (202) 512-2757 or by e-mail at  goldenkoffr@gao.gov. Other key contributors to this testimony include  Lisa Pearson, Assistant Director; Mark Abraham, Peter Beck; Devin  Braun; Jeff DeMarco; Robert Gebhart; Emily Hutz; Richard Hung; Donna  Miller; Ty Mitchell; Kayla Robinson; Kathleen Padulchick; Robert  Robinson, and Timothy Wexler.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.