2020 Census: Sustained Attention to Innovations, IT Systems, and Cost Estimation Is Needed
Released: 2017-05-03T00:00:00

Letter

Background

The cost of the census has been escalating over the last several decennials. The 2010 decennial was the costliest U.S. Census in history at about $12.3 billion, and was about 31 percent more costly than the $9.4 billion 2000 Census (in 2020 dollars). The average cost for counting a housing unit increased from about $16 in 1970 to around $92 in 2010 (in 2020 dollars). According to the Bureau, the total estimated cost of the 2020 Census is approximately $12.5 billion dollars (in 2020 dollars). Meanwhile, the return of census questionnaires by mail (the primary mode of data collection) declined over this period from 78 percent in 1970 to 63 percent in 2010 (see figure 1).

Declining mail response rates—a key indicator of a cost-effective census—are significant and lead to higher costs. This is because the Bureau sends temporary workers to each non-responding household to obtain census data. As a result, non-response follow-up is the Bureau’s largest and most costly field operation. In many ways, the Bureau has had to invest substantially more resources each decade to match the results of prior enumerations.

The 2020 Decennial Census is not the first decennial census to be on GAO’s High- Risk-List. Both the 2000 and the 2010 Decennial Censuses were also high-risk areas. All three of these censuses have had issues related to technology and design changes.

Achieving a complete and accurate census is becoming an increasingly daunting task, in part because the nation’s population is growing larger, more diverse, and more reluctant to participate. When the census misses a person who should have been included, it results in an undercount; an overcount occurs when an individual is counted more than once. Such errors are particularly problematic because of their differential impact on various subgroups. Minorities, renters, and children, for example, are more likely to be undercounted by the census.

The Bureau faces an additional challenge of locating unconventional and hidden housing units, such as converted basements and attics. For example, as shown in figure 2, what appears to be a small, single-family house could contain an apartment, as suggested by its two doorbells. If an address is not in the Bureau’s address file, its residents are less likely to be included in the census. The Bureau estimated a net undercount of 790,000 housing units in 2010.

The Bureau Has Redesigned the 2020 Census to Help Control Costs

The basic design of the enumeration—mail out and mail back of the census questionnaire with in-person follow-up for non-respondents—has been in use since 1970. A key lesson learned from 2010 and earlier enumerations is that this “traditional” design is no longer capable of cost- effectively counting the population. In response to its own assessments, our recommendations, and studies by other organizations, the Bureau has fundamentally reexamined its approach for conducting the 2020 Census. Its plan for 2020 includes four broad innovation areas (re- engineered field operations, administrative records, verifying addresses in-office, and the Internet self-response option). The Bureau believes these innovations will save it over $5 billion (2020 dollars) when compared to what it estimates conducting the census with traditional methods would cost. However, in June 2016, we reported that the Bureau’s life-cycle cost estimate of $12.5 billion cost was not reliable and did not adequately account for risk.

To help drive these innovations, the Bureau plans to rely on about 50 IT systems that are being developed or are already in production. Eleven of these systems are expected to be provided by an enterprise-wide initiative called Census Enterprise Data Collection and Processing (CEDCaP), which is managed within the Bureau’s IT Directorate. This initiative is a large and complex modernization program intended to deliver a system-of-systems to support all the Bureau’s survey data collection and processing functions, rather than continuing to rely on unique, survey-specific systems with redundant capabilities. According to Bureau officials, the 39 other systems (referred to as non-CEDCaP systems) are to be provided by the 2020 Census Directorate or other Bureau divisions.

The Bureau Needs to Manage Risks of Implementing Innovations

The Bureau Is Planning Many Previously Unused Innovations for the 2020 Census

The four innovation areas the Bureau plans for 2020 show promise for a more cost-effective head count (see table 1). However, they also introduce new risks, in part because they include new procedures and technology that have not been used extensively in earlier decennials, if at all. As a result, it will be imperative that the Bureau conduct a robust testing program culminating in the 2018 End-to-End Test that is to demonstrate the systems and operations that will be used for 2020 under operational conditions.

On October 18, 2016, the Bureau decided to stop two field test operations planned for fiscal year 2017 in order to mitigate risks from funding uncertainty. Specifically, the Bureau said it would stop all planned field activity, including local outreach and hiring, at its test sites in Puerto Rico, North and South Dakota, and Washington State. The Bureau is not carrying out the planned field tests of its mail-out strategy and follow up for non-response in Puerto Rico or its door-to-door enumeration.

However, the Bureau is continuing with other planned testing in fiscal year 2017, such as tests focusing on systems readiness and Internet response. One of these tests, which began in March 2017, included 28 of the roughly 50 systems to be used in 2020 operations. Further, the Bureau said it would consider incorporating the canceled field activities elements within the 2018 End-to-End Test, which is expected to begin in August 2017. The Bureau maintains that stopping the 2017 Field Test will help prioritize readiness for the 2018 End-to-End Test and mitigate risk. Nevertheless, as we reported in November 2016, it also represents a lost opportunity to test, refine, and integrate operations and systems, and it puts more pressure on the 2018 test to demonstrate that enumeration activities will function as needed for 2020.

New Uses of Administrative Records Are Promising but Introduce Challenges

Administrative records—information already provided to the government as it administers other programs, such as mail collection by the U.S. Postal Service—have been discussed and used for the decennial census since the 1970s, but the Bureau plans a more significant role for them for 2020 to reduce fieldwork and control costs. For example, the Bureau plans to use administrative records to help determine vacant and occupied housing units and thus improve the efficiency of following-up with non-respondents. The Bureau has estimated that using these records could save up to $1.4 billion compared to traditional census methods. In 2015, we found that the Bureau has already demonstrated the feasibility of using administrative records. However, it still faces challenges with using them for the 2020 Census. For example, although the Bureau has no control over the accuracy of data provided to it by other agencies, it is responsible for ensuring that data it uses for the 2020 Census are of sufficient quality for their planned uses.

Another challenge we identified is the extent to which the public will accept government agencies sharing personal data for the purposes of the census. Related concerns involve trust in the government and perceptions about burden on respondents as well the social benefits of agencies sharing data. Moreover, in addition to using administrative records to reduce fieldwork, the Bureau is considering several additional opportunities to leverage administrative records to help improve the cost and quality of the 2020 Census. It will be important for the Bureau to set deadlines for deciding which records it will use and for which purposes for those records. In this way, the Bureau can monitor its progress and prioritize which activities—or records—to continue pursuing, or to abandon, if time becomes a constraint.

The Bureau Needs to Identify and Analyze Root Causes of Non-interviews During Testing

When households do not respond to the census and when the Bureau does not obtain information about the household while knocking on doors during its non-response follow-up operation, the Bureau may have to impute attributes of the household based on the demographic characteristics of surrounding housing units as well as on administrative records. We reported in 2016 that during the Bureau’s 2016 Census Site Test, the Bureau experienced about 20 and 30 percent of its test workload as non-interviews at its two test sites in Harris County, Texas, and Los Angeles County, California, respectively. According to the Bureau, non-interviews are cases where no data or insufficient data are collected, either because enumerators make six attempted visits without success (the maximum number the Bureau allows), or interviews are not completed due to, for example, language barriers or dangerous situations. Identifying root causes of problems is something we look for when determining progress within a high-risk area. Accordingly, while the 2016 Census Test non-interview rate is not necessarily a precursor to the 2020 non-interview rate, because of its relationship to the cost and quality of the census, it will be important for the Bureau to better understand the factors contributing to it.

The Bureau Is Not Fully Prepared to Implement Key IT Capabilities

The Bureau Continues to Face Challenges in Managing and Overseeing IT Programs, Systems, and Contractors Supporting the Decennial

The Bureau is managing the acquisition and development of new and modified IT systems, which adds complexity to the design of the census. We have previously reported that the Bureau has faced challenges in managing internal coordination and governance, contracts, schedule, scope, and costs for its IT systems, and we have ongoing work reviewing each of these challenges: Internal coordination and governance. Different divisions are responsible for acquiring and developing different systems and technology supporting the 2020 Decennial Census. Eleven of the roughly 50 systems planned to be used in the 2018 End-to-End test are being delivered by the Chief Information Officer’s (CIO) IT Directorate through the CEDCaP program, with the other 39 being managed and developed within the 2020 Census Directorate or other Bureau divisions. We have reported the Bureau has had challenges in coordinating and governing between these divisions. In August 2016 we reported that the two programs lacked effective processes for managing their schedule, risk, and requirements interdependencies.We stated that until the two programs establish schedules that are completely aligned, develop an integrated list of all interdependent risks, and finalize processes for managing requirements, both programs are at risk of not delivering their programs as expected.

Effective IT governance can drive change, provide oversight, and ensure accountability for results. Further, effective IT governance was what Congress had in mind when it enacted the provisions known as the Federal Information Technology Acquisition Reform Act (FITARA), which reinforced the role of the agency CIO. To ensure executive-level oversight of the key systems and technology, the Bureau’s CIO (or a representative) is a member of governance boards that oversee all of the operations and technology for the 2020 Census. Moving forward, it will be important that the CIO and other agency executives continue to use a collaborative governance approach to effectively manage risks and ensure that the IT solutions meet the needs of the agency within cost and schedule.

Contract management. The Bureau is relying on contractor support in many key areas of the 2020 Census, including for integrating all of the key systems and infrastructure and developing many of the key data collection systems. In August 2016, the Bureau hired a contractor to integrate the 2020 Census systems and infrastructure. The contractor’s work was to include evaluating the systems and infrastructure and acquiring the infrastructure (e.g., cloud or data center) to meet the Bureau’s scalability and performance needs. It was also to include integrating all of the systems, supporting technical testing activities, and developing plans for ensuring the continuity of operations. Since the contract was awarded, the scope has grown to also include assisting with operational testing activities, conducting performance testing for two Internet self-response systems, and technical support for the implementation of the paper data capture system.

In addition, the Bureau is relying on contractors to develop a number of key systems and components of the IT infrastructure. These activities include (1) developing the IT platform that is to be used to collect data from a majority of respondents—those using the Internet, telephone, and non-response follow-up activities; (2) procuring the mobile devices and cellular service to be used for non-response follow-up; and (3) developing the IT infrastructure in the field offices. As we reported in November 2016, a greater reliance on contractors for these key components of the 2020 Census requires the Bureau to focus on sound management and oversight of the key contracts, projects, and systems.

Schedule management. In November 2016, we noted that a great deal of development work remained to be completed for the roughly 50 systems and that the Bureau was still developing the plans and schedules for the integration and testing of these systems leading up to the 2018 End-to-End test. This is still the case. In addition, the Bureau has delayed key milestone dates by about 2 months due to issues with developing the systems. This delay reduces the time available for the integration and security testing activities leading up to the 2018 End-to-End test. The Bureau is evaluating options to decrease the impact on those testing activities, and we have ongoing work reviewing the Bureau’s schedule changes and the impacts of those changes.

Scope and requirements management. The Bureau is still in the midst of determining the full complement of systems to be used for the 2018 End-to-End Test (beginning in August 2017) as well as the 2020 Census. For example, the Bureau is evaluating whether to remove several systems from the End-to-End Test. In addition, the agency is evaluating whether to change direction on certain systems and capabilities for the 2018 End-to-End Test. For example, it is evaluating whether to use the newly developed internet self-response system or a legacy prototype. Time is running out before the 2018 End-to-End Test begins, so it will be important for the Bureau to make these decisions with enough time to implement the chosen solutions. IT cost growth. The Bureau has identified at least $2 billion in IT costs for the 2020 program. These costs are associated with (among other things) contracts for the integration of 50 systems into a “system of systems,” the infrastructure on which these systems reside, the call centers for phone responses, the mobile devices and cellular service for field staff who conduct the door-to-door enumeration, and a portion of the costs for the CEDCaP program.

However, our ongoing work suggests that IT costs are likely to rise. For example, the scope of work for the integration contract has increased since the contract was awarded in August 2016, and the corresponding costs are likely to rise as well. Also, the Bureau recently reported that the overall costs for the CEDCaP program are expected to increase by at least 10 percent. In addition, the program is still trying to determine the full complement of systems for the 2018 test and 2020 operations, so these costs may also change. Bureau officials reported that the agency is developing a new cost estimate for the entire 2020 Census program, which they expect to publish later in the spring of 2017.

The Bureau’s Information Security Challenges Have Been Exacerbated by Short Time Frames

In November 2016, we described the significant challenges that the Bureau faced in securing systems and data, and we noted that tight time frames could exacerbate these challenges. One such challenge was making certain that security assessments were completed in a timely manner and that risks were at an acceptable level.

To determine the level of risk present in a system, guidance from the National Institute of Standards and Technology (NIST) states that, as part of their information security programs, agencies should develop security plans, assess security controls, and develop plans of action and milestones to address any deficiencies. The security plans, assessment results, and plans to address deficiencies are key artifacts that authorizing officials use to determine if the risks are acceptable. By issuing an authorization to operate, the authorizing official accepts both the plans to reduce identified deficiencies and any residual risk (e.g., unaddressed vulnerabilities) remaining from unaddressed deficiencies. Each of the systems that the 2020 Census IT architecture plans to rely on will need to develop security plans that identify controls to be implemented, undergo a full security assessment, remediate critical deficiencies, and obtain authorization to operate before being used for the 2020 Census.

However, development and acquisition delays compressed the time to conduct the security reviews and approvals for the 28 systems being used in the 2017 test. For 2 of the 28 systems in the test, the authorizing officials (the CIO and the Associate Director for Decennial Census Programs) accepted security risks and uncertainty due to these compressed time frames. For one of the systems, Bureau officials noted that they did not have time to thoroughly assess the low-impact components. The officials stated that, as a result, they had low confidence in the confidentiality, integrity, and availability of these components. To mitigate the risk, Bureau officials stated that they secured these components in a private network with limited user access. In another case, the Bureau did not have the time to complete penetration testing prior to the 2017 test. According to Bureau officials, the authorizing officials accepted these risks because they wanted to meet the deadline of March 20, 2017, to begin the test in order to meet the Census Day of April 1, 2017. Bureau officials noted that they were informed of the security situation and were comfortable with the approach and mitigations proposed to authorize these systems for the 2017 test.

Because many of the systems that will be a part of the 2018 End-to-End test are not yet fully developed, the Bureau has not finalized all of the controls to be implemented; assessed those controls; developed plans to remediate control weaknesses; and determined whether there is time to fully remediate any deficiencies before the system test. As we previously reported, while the large-scale technological changes (such as Internet self-response) introduce great potential for efficiency and effectiveness gains, they also introduce many information security challenges. Thus, it will be important that these security assessments are completed in a timely manner and that risks are at an acceptable level before the systems are deployed.

The Bureau Needs to Improve the Reliability of Its 2020 Cost Estimate

Estimation Does Not Conform to Best Practices

In June 2016, we reported that the Bureau’s October 2015 update of the life-cycle cost estimate for the 2020 Census did not conform to the four characteristics that constitute best practices, and, as a result, the estimate was unreliable. Cost estimates that appropriately account for risks facing an agency can help an agency manage large, complex activities like the 2020 Census, as well as help Congress make funding decisions and provide oversight. Cost estimates are also necessary to inform decisions to fund one program over another, to develop annual budget requests, to determine what resources are needed, and to develop baselines for measuring performance.

In 2016, we reported that although the Bureau had taken significant steps to improve its capacity to carry out an effective cost estimate, such as establishing an independent cost estimation office, its October 2015 version of the estimate for the 2020 Census only partially met the characteristics of two best practices (comprehensive and accurate) and minimally met the other two (well-documented and credible). All four characteristics need to be substantially met in order for an estimate to be deemed high-quality.

For example, to be comprehensive an estimate should have enough detail to ensure that cost elements are neither omitted nor double- counted, and all cost-influencing assumptions are detailed in the estimate’s documentation, among other things, according to best practices. In June 2016, we reported that while Bureau officials were able to provide us with several documents that included projections and assumptions that were used in the cost estimate, we found the estimate to be partially comprehensive because it is unclear if all life-cycle costs are included in the estimate or if the cost estimate completely defines the program.

Accurate estimates are unbiased and contain few mathematical mistakes. We reported that the estimate partially met best practices for this characteristic, in part because we could not independently verify the calculations the Bureau used within its cost model, which the Bureau did not have documented or explained outside its cost model. The Bureau’s cost-estimate was also not well-documented. Cost estimates are considered valid if they are well-documented to the point they can be easily repeated or updated and can be traced to original sources through auditing, according to best practices. In 2016, we reported that while the Bureau provided some documentation of supporting data, it did not describe how the source data were incorporated.

Finally, credible cost estimates must clearly identify limitations due to uncertainty or bias surrounding the data or assumptions, according to best practices. In June 2016 we reported that the estimate minimally met best practices for this characteristic in part because the Bureau carried out its risk and uncertainty analysis only for about $4.6 billion (37 percent) of the $12.5 billion total estimated life-cycle cost, excluding, for example, consideration of uncertainty over what the decennial census’s estimated part will be of the total cost of CEDCaP. In June 2016, we also reported that risks were not properly accounted for in the cost estimate and recommended that the Bureau properly account for risk.to ensure there are appropriate levels for budgeted contingencies and those recommendations have not yet been implemented. In April 2017, Bureau officials told us they were making progress toward implementing our recommendations and would provide us with that documentation when it is finalized.

Several Events Indicate That the Cost of Some Elements of the 2020 Census Design Are Increasing

The Bureau has not published an update to its October 2015 cost estimate, yet several events since then suggest that the cost of the current design could be higher. We believe that recent Bureau actions indicate that the current design will have more expensive cost elements that include the following:

According to Bureau official, in August an $886 million IT integration contract was awarded. According to Bureau officials, there was no reference to this contract in the documentation for the planned contract costs supporting the October 2015 life-cycle cost estimate.

In January 2017, the Bureau suspended part of how it is verifying addresses “in-office” procedures using on-screen imagery—one of its four key design innovations intended to control the cost of the 2020 Census. According to Bureau officials, the suspension of the one part of in-office canvassing will increase the workload of the more expensive in-field (door-to door address identification) by at least five percentage points over what had been assumed as part of the earlier cost estimate.

In April 2017, Bureau officials informed us that they are preparing additional reporting to the Department of Commerce and Office of Management and Budget because the CEDCaP program is now expected to exceed the original total contract by more than 10 percent.

According to Bureau officials these changes will be reflected in the next life-cycle cost estimate.

It is critical for the Bureau to have better oversight and control of its cost estimation process, particularly as the design of key cost elements continues to evolve, and we have recommended that the Bureau ensure its cost estimate is consistent with our best practices. The Bureau agreed with our recommendations and has told us it is making progress towards addressing them. Moreover, Bureau officials told us that they are finalizing an update to the life-cycle cost estimate that should be released this spring. However, until we are able to review the life-cycle cost estimate along with supporting documentation we are unable to determine whether the Bureau is fully addressing our recommendations, meeting best practices, and properly accounting for all costs associated with the 2020 Census.

In summary, while the Bureau has made substantial progress in revamping its approach to the census and testing the new design, considerable challenges and uncertainties remain in (1) implementing the cost-saving innovations; (2) managing key IT systems, including ensuring their security, to support the census; and (3) developing a quality cost estimate for the 2020 Census. For these reasons the 2020 Census is a GAO high-risk area. Continued management attention and congressional oversight will be important for ensuring risks are mitigated, the Bureau’s preparations stay on track, and Bureau officials are held accountable for implementing the enumeration as planned. We will continue to assess the Bureau’s efforts to conduct a cost-effective enumeration and look forward to keeping Congress informed of the Bureau’s progress.

Chairman Culberson, Ranking Member Serrano, and Members of the Subcommittee, this completes our prepared statement. We would be pleased to respond to any questions that you may have.

GAO Contacts and Staff Acknowledgments

If you have any questions on matters discussed in this statement, please contact Robert Goldenkoff at (202) 512-2757 or by e-mail at goldenkoffr@gao.gov or David A. Powner at (202) 512-9286 or by e-mail at pownerd@gao.gov. Other key contributors to this testimony include Ty Mitchell, Assistant Director; Colleen Phillips, Assistant Director; Justine Augeri; Brian Bothwell, Robert Gebhart; Jason Lee, Lisa Pearson; Kayla Robinson, Robert Robinson; Kathleen Sharkey; Andrea Starosciak; Umesh Thakkar; and Katherine Wulff.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.