2020 Census: Continued Management Attention Needed to Oversee Innovations, Develop and Secure IT Systems, and Improve Cost Estimation
Released: 2017-10-12T00:00:00

Letter

Background

The cost of the census has been escalating over the last several decennials. The 2010 decennial was the costliest U.S. Census in history at about $12.3 billion, and was about 31 percent more costly than the $9.4 billion 2000 Census (in 2020 dollars). The average cost for counting a housing unit increased from about $16 in 1970 to around $92 in 2010 (in 2020 dollars). According to the Bureau, the total cost of the 2020 Census is estimated to be approximately $12.5 billion dollars (in 2020 dollars). As discussed later in this statement, however, the cost of the 2020 Census will likely be higher than this current estimate.

Meanwhile, the return of census questionnaires by mail (the primary mode of data collection) declined over this period from 78 percent in 1970 to 63 percent in 2010 (see figure 1). Declining mail response rates—a key indicator in determining the cost-effectiveness of the census—are significant and lead to higher costs. This is because the Bureau sends temporary workers to each non-responding household to obtain census data. As a result, non-response follow-up is the Bureau’s largest and most costly field operation. In many ways, the Bureau has had to invest substantially more resources each decade to match the results of prior enumerations.

Further, achieving a complete and accurate census is becoming an increasingly daunting task, in part, because the nation’s population is growing larger, more diverse, and more reluctant to participate. When the census misses a person who should have been included, it results in an undercount; conversely, an overcount occurs when an individual is counted more than once. Such errors are particularly problematic because of their impact on various subgroups. Minorities, renters, and children, for example, are more likely to be undercounted by the census.

The Bureau faces an additional challenge of locating unconventional and hidden housing units, such as converted basements and attics. For example, as shown in figure 2, what appears to be a small, single-family house could contain an apartment, as suggested by its two doorbells. If an address is not in the Bureau’s address file, its residents are less likely to be included in the census.

The Bureau Has Redesigned the 2020 Census to Help Control Costs

The basic design of the enumeration—mail out and mail back of the census questionnaire with in-person follow-up for non-respondents—has been in use since 1970. However, a key lesson learned from the 2010 Census and earlier enumerations, is that this “traditional” design is no longer capable of cost-effectively counting the population.

In response to its own assessments, our recommendations, and studies by other organizations, the Bureau has fundamentally re-examined its approach for conducting the 2020 Census. Specifically, its plan for 2020 includes four broad innovation areas (re-engineering field operations, using administrative records, verifying addresses in-office, and developing an Internet self-response option).

The Bureau has estimated that these innovations could result in savings of over $5 billion (in 2020 dollars) when compared to its estimates of the cost for conducting the census with traditional methods. However, in June 2016, we reported that the Bureau’s life-cycle cost estimate of $12.5 billion, developed in October 2015, was not reliable and did not adequately account for risk, as discussed later in this statement.

Bureau Plans to Use IT to Drive Innovation To help drive these innovations, the Bureau plans to rely on both new and legacy IT systems and infrastructure. For example, the Bureau is developing or modifying 11 IT systems as part of an enterprise-wide initiative called Census Enterprise Data Collection and Processing (CEDCaP), which is managed within the Bureau’s IT Directorate. This initiative is a large and complex modernization program intended to deliver a system-of-systems to support all of the Bureau’s survey data collection and processing functions, rather than continuing to rely on unique, survey-specific systems with redundant capabilities. In addition, according to Bureau officials, the 2020 Census Directorate or other Bureau divisions are developing or modifying 32 other IT systems.

To help inform, validate, and refine the operational design of the 2020 Census, and to test several of the IT systems, the Bureau has held a series of operational tests since 2012. Among these, in March 2017, the Bureau conducted a nationwide test (referred to as the 2017 Census Test) of households responding to census questions using paper, the Internet, or the phone. This test evaluated key new IT components, such as the Internet self-response system and the use of a cloud-based infrastructure.

The Bureau is currently conducting the 2018 End-to-End Test, which began in August 2017 and runs through April 2019. It is the Bureau’s final opportunity to test all key systems and operations to ensure readiness for the 2020 Census. The Bureau’s plans for this test include, among other things, address canvassing, self-response (via paper, Internet, and phone), and nonresponse follow-up.

To support its 2018 End-to-End Test, the Bureau plans to deploy and use 43 systems incrementally to support nine operations from December 2016 through the end of the test in April 2019. These nine operations are: (1) in-office address canvassing, (2) recruiting staff for address canvassing, (3) training for address canvassing, (4) in-field address canvassing, (5) recruiting staff for field enumeration, (6) training for field enumeration, (7) self-response (i.e., Internet, phone, or paper), (8) field enumeration, and (9) tabulation and dissemination. Appendix I includes additional details about the 43 systems, the operation or operations they support, and key deployment dates.

The Bureau Needs to Manage Risks of Implementing Innovations

The Bureau Plans Four Innovation Areas for 2020, but Has Scaled Back Key Census Tests

The four innovation areas the Bureau plans for 2020 show promise for a more cost-effective head count (see table 1). However, the innovations also introduce new risks, in part, because they include new procedures and technology that have not been used extensively in earlier decennials, if at all. Our prior work has shown the importance of the Bureau conducting a robust testing program, including the 2018 End-to-End Test. However, because of funding uncertainty the Bureau canceled the field components of the 2017 Census Test including non-response follow- up, a key census operation.

In November 2016, we reported that the cancelation of the 2017 field tests was a lost opportunity to test, refine, and integrate operations and systems, and that it put more pressure on the 2018 End-to-End Test to demonstrate that enumeration activities will function under census-like conditions as needed for 2020. In May 2017, the Bureau scaled back the operational scope of the 2018 End-to-End and, of the three planned test sites, only the Rhode Island site would fully implement the 2018 End- to-End Test. The Washington and West Virginia state test sites would test address canvassing. In addition, due to budgetary concerns, the Bureau decided to remove three coverage measurement operations (and the technology that supports them) from the scope of the test. Without sufficient testing, operational problems can go undiscovered and the opportunity to improve operations will be lost, in part because the 2018 End-to-End Test is the last opportunity to demonstrate census technology and procedures across a range of geographic locations, housing types, and demographic groups.

New Uses of Administrative Records Are Promising, but Introduce Challenges

Administrative records—information already provided to the government as it administers other programs, such as mail collection by the U.S. Postal Service—have been discussed and used for the decennial census since the 1970s, and for 2020 the Bureau plans a more significant role for them. In July 2017, we reported that the Bureau had taken steps to ensure that its use of administrative records would lower the cost and improve the accuracy of the 2020 Census.

For example, the Bureau set a rule that it would only use administrative records to count a household when a minimum amount of information was present within data sources. According to the Bureau, this would help ensure that administrative records are used only in circumstances where research has shown them to be most accurate. Additionally, before using any administrative records to support census operations, the Bureau determined it will subject each source to a quality assurance process that includes, among other things, basic checks for data integrity as well as assessments by subject matter experts of the information’s fitness for various uses by the Bureau. (See figure 3.)

According to the Bureau, it links administrative records data sources to complement each other, improving their reliability and completeness. The Bureau also creates an anonymous personal identifier for each individual in the data to reduce the risk of disclosure once the data are linked across sources.

In July 2017, we reported that the Bureau had already tested the uses of administrative records that hold the most potential for reducing census costs, such as counting people who did not respond to census mailings. The Bureau planned to test additional applications of administrative records for the first time during the 2018 End-to-End Test. For example, the Bureau planned to use administrative records to support quality control during its non-response field enumeration. The Bureau planned to compare response data collected by enumerators to administrative records and flag significant differences based on predefined rules. The differences might be in the total count of persons in a household or in specific combinations of personal characteristics, such as age or race. According to the Bureau, flagging such differences could be used to help identify which enumeration cases to reinterview as part of the quality control operation.

However, we reported in October 2015 that the Bureau faced other challenges with using administrative records for the 2020 Census. For example, although the Bureau has no control over the accuracy of data provided to it by other agencies, it is responsible for ensuring that data it uses for the 2020 Census are of sufficient quality for their planned uses. Another challenge we identified in 2015 is the extent to which the public will accept government agencies sharing personal data for the purposes of the census. The Bureau has recognized these challenges within its risk registers.

The Bureau Has Fundamentally Re- Engineered Address Canvassing for 2020

In-Office Address Canvassing. The Bureau has re-engineered its approach to building its master address list for 2020. Specifically, by relying on multiple sources of imagery and administrative data, the Bureau anticipates constructing its address list with far less door-to-door field canvassing compared to previous censuses.

One major change the Bureau has made consists of using in-office address canvassing—a two-phase process that was to systematically review small geographic areas nationwide, known as census blocks, to identify those that will not need to be canvassed in the field, as shown in figure 4.

The Bureau estimated that the two phases of in-office canvassing would have resulted in roughly 25 percent of housing units requiring in-field canvassing, instead of canvassing nearly all housing units in the field as done in prior decennials. With in-office address canvassing census workers compare current aerial imagery for a given block with imagery for that block dating to the time of the last decennial census in 2010. During this first phase, called Interactive Review, specially trained census workers identify whether a block appears to have experienced change in the number of housing units, flagging each block either as stable—free of population growth, decline, or uncertainty in what is happening in the imagery over time—or “active,” in which case it moves to the next phase. Addresses in stable blocks are not marked for in-field canvassing.

For blocks where change is detected or suspected, the Bureau was to use a second phase of in-office canvassing, known as Active Block Resolution, to attempt to resolve the status of each address and housing unit in question within that block. During this phase, census workers use aerial imagery, street imagery, and data from the U.S. Postal Service, as well as from state, local, and tribal partners when reviewing blocks. If a block can be fully resolved during this phase of in-office canvassing, the changes are recorded in the Bureau’s master address file. If a block cannot be fully resolved during the second phase of in-office canvassing, then the entire block, or some portion of the block, is flagged for inclusion in the in-field canvassing operation. A first pass of the entire country for in-office address canvassing began in September 2015 and was completed in June 2017. In-field canvassing for the 2020 Census is scheduled to begin in August 2019.

However, in July 2017 we reported that the Bureau altered its design for re-engineered address canvassing because of budget uncertainty by suspending the second phase of in-office address canvassing. Without the second phase of in-office address canvassing, blocks that are not resolved by phase one will have a greater chance of requiring in-field canvassing. Bureau officials told us at that time that they anticipated that canceling the second phase of in-office address canvassing altogether would increase their estimated in-field canvassing workload by 5 percentage points, from 25 percent to 30 percent of housing units— increasing costs.

The Bureau did not develop cost and quality information on address canvassing projects, and detailed information on cost tradeoffs was not available when we requested it. The information the Bureau had did not break out the estimated cost of the different phases of in-office address canvassing through 2020. However, the total estimated cost for both phases one and two was approximately $22 million. Thus, this suspension might save a portion of the $22 million, but it will potentially increase the cost of the address canvassing operation downstream. Our July 2017 report recommended, and the Bureau agreed, that the Bureau should use its evaluations before 2020 to determine the implications of in- office address canvassing on the cost and quality of address canvassing, and use this information to justify decisions related to its re-engineered address canvassing approach.

In-Field Address Canvassing for the 2018 End-to-End Test. On August 28, 2017, temporary census employees known as address listers began implementing the in-field component of address canvassing for the 2018 End-to-End Test. Listers walked the streets of designated census blocks at all three test sites to verify addresses and geographic locations. The operation ended on September 27, 2017. As part of our ongoing work, we visited all three test sites and observed 18 listers conduct address canvassing. Generally, we found that listers were able to conduct address canvassing as planned. However, we also noted several challenges. We shared the following preliminary observations from our site visits with the Bureau: Internet connectivity was problematic at the West Virginia test site. We spoke to four census field supervisors that described certain areas as dead spots where Internet and cell phone service were not available. We also were told by those same supervisors that only certain cell service providers worked in certain areas. In order to access the Internet or cell service in those areas, census workers sometimes needed to drive several miles.

The allocation of lister assignments was not always optimal. Listers were supposed to be provided assignments close to where they live in order to optimize their local knowledge and to limit the numbers of miles being driven by listers to and from their assignment area. Bureau officials told us this was a challenge at all three test sites. Moreover, at one site the area census manager told us that some listers were being assigned work in another county even though blocks were still unassigned closer to where they resided. Relying on local knowledge and limiting the number of miles can increase both the efficiency and effectiveness of address canvassing.

The assignment of some of the large blocks early in the operations was not occurring as planned. At all three 2018 End-to-End Test sites Bureau managers had to manually assign some large blocks (some blocks had hundreds of housing units). It is important to assign large blocks early on because leaving the large blocks to be canvassed until the end of the operation could jeopardize the timely completion of address canvassing.

The global positioning system-derived location for the lister was not always corresponding to the location on the map. A Bureau official confirmed that at all three test sites, the location icon jumped around or was on the wrong street. According to a Bureau official, listers were told to override the global positioning system-derived location when confirming the geographic location of the residence.

We have discussed these challenges with Bureau officials who stated that overall they are satisfied with the implementation of address canvassing but also agreed that resolving challenges discovered during address canvassing, some of which can affect the operation’s efficiency and effectiveness, will be important before the 2020 Census. We will continue to monitor address canvassing operation and plan to issue a report in the winter of 2018.

The Bureau Continues to Face Challenges in Implementing and Securing Key IT Systems

The Bureau Continues to Face Challenges Implementing and Managing IT Systems

We have previously reported that the Bureau faced challenges in managing and overseeing IT programs, systems, and contractors supporting the 2020 Census. Specifically, it has been challenged in managing schedules, costs, contracts, and governance and internal coordination for its IT systems. As a result of these challenges, the Bureau is at risk of being unable to fully implement key IT systems necessary to support the 2020 Census. We have previously recommended that the Bureau take action to improve its implementation and management of IT in areas such as governance and internal coordination. We also have ongoing work reviewing each of these areas.

Our ongoing work has indicated that the Bureau faces significant challenges in managing the schedule for developing and testing systems for the 2018 End-to-End Test that began in August 2017. In this regard, the Bureau still has significant development and testing work that remains to be completed. As of August 2017, of the 43 systems in the test, the Bureau reported that 4 systems had completed development and integration testing, while the remaining 39 systems had not completed these activities.

Of these 39 systems, the Bureau reported that it had deployed a portion of the functionality for 21 systems to support address canvassing for the 2018 End-to-End Test; however, it had not yet deployed any functionality for the remaining 18 systems for the test. Figure 5 summarizes the development and testing status for the 43 systems planned for the 2018 End-to-End Test, and appendix I includes additional information on the status of development and testing for these systems.

Moreover, due to challenges experienced during systems development, the Bureau has delayed key IT milestone dates (e.g., dates to begin integration testing) by several months for the systems supporting six of the nine operations in the 2018 End-to-End Test. Figure 6 depicts the delays to the deployment dates for the operations in the 2018 End-to-End Test, as of August 2017.

However, our ongoing work also indicates that the Bureau is at risk of not meeting the updated milestone dates. For example, in June 2017 the Bureau reported that at least two of the systems expected to be used in the self-response operation (the Internet self-response system and the call center system) are at risk of not meeting the delayed milestone dates. In addition, in September 2017 the Bureau reported that at least two of the systems expected to be used in the field enumeration operation (the enumeration system and the operational control system) are at risk of not meeting their delayed dates.

Combined, these delays reduce the time available to conduct the security reviews and approvals for the systems being used in the 2018 End-to- End Test. We previously testified in May 2017 that the Bureau faced similar challenges leading up to the 2017 Census Test, including experiencing delays in system development that led to compressed time frames for security reviews and approvals. Specifically, we noted that the Bureau did not have time to thoroughly assess the low-impact components of one system and complete penetration testing for another system prior to the test, but accepted the security risks and uncertainty due to compressed time frames. We concluded that, for the 2018 End-to- End Test, it will be important that these security assessments are completed in a timely manner and that risks are at an acceptable level before the systems are deployed.

The Bureau noted that, if it continues to be behind schedule, field operations for the 2018 End-to-End Test will not be performed as planned. Bureau officials are evaluating options to decrease the impact of these delays on integration testing and security review activities by, for example, utilizing additional staff. We have ongoing work reviewing the Bureau’s development and testing delays and the impacts of these delays on systems readiness for the 2018 End-to-End Test.

The Bureau faces challenges in reporting and controlling IT cost growth. In April 2017, the Bureau briefed us on its efforts to estimate the costs for the 2020 Census, during which it presented IT costs of about $2.4 billion from fiscal years 2018 through 2021. Based on this information and other corroborating IT contract information provided by the Bureau, we testified in May 2017 that the Bureau had identified at least $2 billion in IT costs.

However, in June 2017, Bureau officials in the 2020 Census Directorate told us that the data they provided in April 2017 did not reflect all IT costs for the 2020 program. The officials provided us with an analysis of the Bureau’s October 2015 cost estimate that identified $3.4 billion in total IT costs from fiscal years 2012 through 2023. These costs included, among other things, those associated with system engineering, test and evaluation, and infrastructure, as well as a portion of the costs for the CEDCaP program.

Yet, our ongoing work determined that the Bureau’s $3.4 billion cost estimate does not reflect its current plans for acquiring IT to be used during the 2020 Census and that the related costs are likely to increase: In August 2016, the Bureau awarded a technical integration contract for about $886 million, a cost that was not reflected in the $3.4 billion expected IT costs. More recently, in May 2017, we testified that the scope of work for this contract had increased since the contract was awarded; thus, the corresponding contract costs were likely to rise above $886 million, as well.

In March 2017, the Bureau reported that the contract associated with the call center and IT system to support the collection of census data over the phone was projected to overrun its initial estimated cost by at least $40 million.

In May 2017, the Bureau reported that the CEDCaP program’s cost estimate was increasing by about $400 million—from its original estimate of $548 million in 2013 to a revised estimate of $965 million in May 2017.

In June 2017, the Bureau awarded a contract for mobile devices and associated services for about $283 million, an amount that is about $137 million higher than the cost for these devices and services identified in its October 2015 estimate.

As a result of these factors, the Bureau’s $3.4 billion estimate of IT costs is likely to be at least $1.4 billion higher, thus increasing the total costs to at least $4.8 billion. Figure 7 identifies the Bureau estimate of total IT costs associated with the 2020 program as of October 2015, as well as anticipated cost increases as of August 2017.

IT cost information that is accurately reported and clearly communicated is necessary so that Congress and the public have confidence that taxpayer funds are being spent in an appropriate manner. However, changes in the Bureau’s reporting of these total costs, combined with cost growth since the October 2015 estimate, raise questions as to whether the Bureau has a complete understanding of the IT costs associated with the 2020 program. In this regard, we have previously reported on issues with the Bureau’s cost estimating practices (which are discussed in more detail later in this statement). To address these issues, in October 2017, officials stated that the Bureau is developing a new cost estimate for the entire 2020 Census program, which they expect to release by the end of this fall.

Our ongoing work also determined that the Bureau faces challenges in managing its significant contractor support. The Bureau is relying on contractor support in many key areas of the 2020 Census. For example, it is relying on contractors to develop a number of key systems and components of the IT infrastructure. These activities include (1) developing the IT platform that is to be used to collect data from a majority of respondents—those using the Internet, telephone, and non- response follow-up activities; (2) procuring the mobile devices and cellular service to be used for non-response follow-up; and (3) developing the infrastructure in the field offices. According to Bureau officials, contractors are also providing support in areas such as fraud detection, cloud computing services, and disaster recovery.

In addition to the development of key technology, the Bureau is relying on contractor support for integrating all of the key systems and infrastructure. The Bureau awarded a contract to integrate the 2020 Census systems and infrastructure in August 2016. The contractor’s work was to include evaluating the systems and infrastructure and acquiring the infrastructure (e.g., cloud or data center) to meet the Bureau’s scalability and performance needs. It was also to include integrating all of the systems, supporting technical testing activities, and developing plans for ensuring the continuity of operations. Since the contract was awarded, the Bureau has modified the scope to also include assisting with operational testing activities, conducting performance testing for two Internet self-response systems, and technical support for the implementation of the paper data capture system.

However, our ongoing work has indicated that the Bureau is facing staffing challenges that could impact its ability to manage and oversee the technical integration contractor. Specifically, the Bureau is managing the integration contractor through a government program management office, but this office is still filling vacancies. As of October 2017, the Bureau reported that 35 of the office’s 58 federal employee positions were vacant. As a result, this program management office may not be able to provide adequate oversight of contractor cost, schedule, and performance.

The delays during the 2017 Test and preparations for the 2018 End-to- End Test raises concerns regarding the Bureau’s ability to effectively perform contractor management. As we reported in November 2016, a greater reliance on contractors for these key components of the 2020 Census requires the Bureau to focus on sound management and oversight of the key contracts, projects, and systems. As part of our ongoing work, we plan to monitor the Bureau’s progress in managing its contractor support.

Effective IT governance can drive change, provide oversight, and ensure accountability for results. Further, effective IT governance was envisioned in the provisions referred to as the Federal Information Technology Acquisition Reform Act (FITARA), which strengthened and reinforced the role of the departmental CIO.

To ensure executive-level oversight of the key systems and technology, the Bureau’s CIO (or a representative) is a member of the governance boards that oversee all of the operations and technology for the 2020 Census. However, in August 2016 we reported on challenges the Bureau has had with IT governance and internal coordination, including weaknesses in its ability to monitor and control IT project costs, schedules, and performance. We made eight recommendations to the Department of Commerce to direct the Bureau to, among other things, better ensure that risks are adequately identified and schedules are aligned. The department agreed with our recommendations. However, as of October 2017, the Bureau had only fully implemented one recommendation and had taken initial steps toward implementing others.

Further, given the schedule delays and cost increases previously mentioned, and the vast amount of development, testing, and security assessments left to be completed, we remain concerned about executive- level oversight of systems and security. Moving forward, it will be important that the CIO and other agency executives continue to use a collaborative governance approach to effectively manage risks and ensure that the IT solutions meet the needs of the agency within cost and schedule. As part of our ongoing work, we plan to monitor the steps the Bureau is taking to effectively oversee and manage the development and acquisition of its IT systems.

The Bureau Has Significant Information Security Steps to Complete for the 2018 End-to-End Test

In November 2016, we described the significant challenges that the Bureau faced in securing systems and data for the 2020 Census, and we noted that tight time frames could exacerbate these challenges. Two such challenges were (1) ensuring that individuals gain only limited and appropriate access to the 2020 Census data, including personally identifiable information (PII) (e.g., name, address, and date of birth), and (2) making certain that security assessments were completed in a timely manner and that risks were at an acceptable level. Protecting PII, for example, is especially important because a majority of the 43 systems to be used in the 2018 End-to-End Test contain PII, as reflected in figure 8.

To address these and other challenges, federal law and guidance specify requirements for protecting federal information and information systems, such as those to be used in the 2020 Census. Specifically, the Federal Information Security Management Act of 2002 and the Federal Information Security Modernization Act of 2014 (FISMA) require executive branch agencies to develop, document, and implement an agency-wide program to provide security for the information and information systems that support operations and assets of the agency.

Accordingly, the National Institute of Standards and Technology (NIST) developed risk management framework guidance for agencies to follow in developing information security programs. Additionally, the Office of Management and Budget’s (OMB) revised Circular A-130 on managing federal information resources required agencies to implement the NIST risk management framework to integrate information security and risk management activities into the system development life cycle.

In accordance with FISMA, NIST guidance, and OMB guidance, the Office of the CIO established a risk management framework. This framework requires that system developers ensure that each of the systems undergoes a full security assessment, and that system developers remediate critical deficiencies. In addition, according to the Bureau’s framework, system developers must ensure that each component of a system has its own system security plan, which documents how the Bureau plans to implement security controls. As a result, system developers for a single system might develop multiple system security plans (in some cases as many as 34 plans), which all have to be approved as part of the system’s complete security documentation. We have ongoing work that is reviewing the extent to which the Bureau’s framework meets the specific requirements of the NIST guidance.

According to the Bureau’s framework, each of the 43 systems in the 2018 End-to-End Test will need to have complete security documentation (such as system security plans) and an approved authorization to operate prior to their use in the 2018 End-to-End Test. However, our ongoing work indicates that, while the Bureau is completing these steps for the 43 systems to be used in the 2018 End-to-End Test, significant work remains. Specifically:

None of the 43 systems are fully authorized to operate through the completion of the 2018 End-to-End Test. Bureau officials from the CIO’s Office of Information Security stated that these systems will need to be reauthorized because, among other things, they have additional development work planned that may require the systems to be reauthorized; are being moved to a different infrastructure environment (e.g., from a data center to a cloud-based environment); or have a current authorization that expires before the completion of the 2018 End-to-End Test. The amount of work remaining is concerning because the test has already begun and the delays experienced in system development and testing mentioned earlier reduce the time available for performing the security assessments needed to fully authorize these systems before the completion of the 2018 End-to-End test.

Thirty-seven systems have a current authorization to operate, but the Bureau will need to reauthorize these systems before the completion of the 2018 End-to-End Test. This is due to the reasons mentioned previously, such as additional development work planned and changes to the infrastructure environments.

Two systems have not yet obtained an authorization to operate.

For the remaining four systems, the Bureau has not yet provided us with documentation about the current authorization status.

Figure 9 depicts the authorization to operate status for the systems being used in the 2018 End-to-End Test, as reported by the Bureau.

Because many of the systems that will be a part of the 2018 End-to-End Test are not yet fully developed, the Bureau has not finalized all of the security controls to be implemented; assessed those controls; developed plans to remediate control weaknesses; and determined whether there is time to fully remediate any deficiencies before the systems are needed for the test. In addition, as discussed earlier, the Bureau is facing system development challenges that are delaying the completion of milestones and compressing the time available for security testing activities.

As we previously reported, while the large-scale technological changes (such as Internet self-response) increase the likelihood of efficiency and effectiveness gains, they also introduce many information security challenges. The 2018 End-to-End Test also involves collecting PII on hundreds of thousands of households across the country, which further increases the need to properly secure these systems. Thus, it will be important that the Bureau provides adequate time to perform these security assessments, completes them in a timely manner, and ensures that risks are at an acceptable level before the systems are deployed. We plan to continue monitoring the Bureau’s progress in securing its IT systems and data as part of our ongoing work.

The Bureau Needs to Improve the Reliability of Its 2020 Cost Estimate

2020 Census Cost Estimate Does Not Reflect Best Practices

In June 2016, we reported that the Bureau’s October 2015 update of its life-cycle cost estimate for the 2020 Census did not conform to the four characteristics that constitute best practices, and, as a result, the estimate was unreliable. Cost estimates that appropriately account for risks facing an agency can help an agency manage large, complex activities like the 2020 Census, as well as help Congress make funding decisions and provide oversight. Cost estimates are also necessary to inform decisions to fund one program over another, to develop annual budget requests, to determine what resources are needed, and to develop baselines for measuring performance.

In June 2016, we reported that, although the Bureau had taken steps to improve its capacity to carry out an effective cost estimate, such as establishing an independent cost estimation office, its October 2015 version of the estimate for the 2020 Census only partially met the characteristics of two best practices (comprehensive and accurate) and minimally met the other two (well-documented and credible). All four characteristics need to be substantially met in order for an estimate to be deemed high-quality:

Comprehensive. To be comprehensive an estimate should have enough detail to ensure that cost elements are neither omitted nor double-counted, and all cost-influencing assumptions are detailed in the estimate’s documentation, among other things, according to best practices. In June 2016, we reported that, while Bureau officials were able to provide us with several documents that included projections and assumptions that were used in the cost estimate, we found the estimate to be partially comprehensive because it was unclear if all life-cycle costs were included in the estimate or if the cost estimate completely defined the program.

Accurate. Accurate estimates are unbiased and contain few mathematical mistakes. We reported in June 2016 that the estimate partially met best practices for this characteristic, in part because we could not independently verify the calculations the Bureau used within its cost model, which the Bureau did not have documented or explained outside its cost model.

Well-documented. Cost estimates are considered valid if they are well-documented to the point they can be easily repeated or updated and can be traced to original sources through auditing, according to best practices. In June 2016, we reported that, while the Bureau provided some documentation of supporting data, it did not describe how the source data were incorporated.

Credible. Credible cost estimates must clearly identify limitations due to uncertainty or bias surrounding the data or assumptions, according to best practices. In June 2016, we reported that the estimate minimally met best practices for this characteristic in part because the Bureau carried out its risk and uncertainty analysis only for about $4.6 billion (37 percent) of the $12.5 billion total estimated life-cycle cost, excluding, for example, consideration of uncertainty over what the decennial census’s estimated part will be of the total cost of CEDCaP.

In June 2016, we recommended that the Bureau take action to ensure its 2020 Census cost estimate meets all four characteristic of a reliable cost estimate. The Bureau agreed with our recommendation. We also reported in June 2016 that risks were not properly accounted for in the cost estimate and recommended that the Bureau properly account for risk to ensure there are appropriate levels for budgeted contingencies, and those recommendations have not yet been implemented.

In October 2017, Bureau officials told us they were making progress towards implementing our recommendations and would provide us with that documentation when the cost estimate and supporting documentation are finalized. Moreover, Bureau officials also told us that an updated cost estimate would be available by the end of this fall. However, until the Bureau updates its estimate and we have the opportunity to review its reliability, questions will surround the quality of the 2020 Census cost estimate and the basis for any 2020 Census annual budgetary figures.

The Cost of the 2020 Census Will Likely Be Higher Than Originally Planned

While the Bureau has not updated its October 2015 cost estimate, several events since then indicate that the cost of the current design will be higher. For example:

As previously mentioned, in August 2016 an $886 million IT integration contract was awarded. According to Bureau officials, there was no reference to this contract in the documentation for the planned contract costs supporting the October 2015 life-cycle cost estimate.

In March 2017, the Bureau suspended part of how it is verifying address in-office procedures using on-screen imagery—one of its four key design innovations intended to control the cost of the 2020 Census. According to Bureau officials, the suspension of the one part of in-office canvassing will increase the workload of the more expensive in-field (door-to-door address identification) by at least five percentage points, from 25 percent to 30 percent of housing units— increasing the cost over what had been assumed as part of the earlier cost estimate. Based on cost assumptions underlying its October 2015 life-cycle cost estimate, we found, as part of our prior work, that the potential addition of five percentage points to the field workload alone could reduce the Bureau’s cost savings by $26.6 million.

As earlier discussed, in May 2017, Bureau officials reported that the cost of the CEDCaP program has now increased by over $400 million, from about $548 million to $965 million.

2020 Census Cost Estimate May Not Fully Inform Annual Budget Requests

Cost estimates are also used by the Bureau as a tool to inform the annual budget process. However, since the Bureau did not fully follow best practices for developing and maintaining the life-cycle cost estimate, as previously described, annual budget requests based on that cost estimate may not be fully informed.

A high-quality cost estimate is the foundation of a good budget. A major purpose of a cost estimate is to support the budget process by providing an estimate of the funding required to efficiently execute a program. Because most programs do not remain static but evolve over time, developing a cost estimate should not be a onetime event but rather a recurrent process. Effective program and cost control requires ongoing revisions to the cost estimate and budget.

Using a reliable life-cycle cost estimate to formulate the budget could help the Bureau ensure that all costs are fully accounted for so that resources are adequate to support the program. Credible cost estimates could also help the Bureau effectively defend budgets to the Department of Commerce, OMB, and Congress. Concerns about the soundness of the life cycle cost estimate and the quality of annual budgets related to the 2020 Census are particularly important because the bulk of funds will be obligated in fiscal years 2019 through 2020. In our June 2016 report on the Bureau’s life-cycle cost estimate we made several recommendations with which the Bureau agreed. We will continue to monitor the Bureau’s efforts to address these recommendations.

In conclusion, the Bureau has made progress in revamping its approach to the census and testing the new design. However, it faces considerable challenges and uncertainties in (1) implementing the cost-saving innovations; (2) managing the development and security of key IT systems; and (3) developing a quality cost estimate for the 2020 Census. For these reasons, the 2020 Census is a GAO high risk area.

Continued management attention is vital for ensuring risks are managed, the Bureau’s preparations stay on-track, and the Bureau is held accountable for implementing the enumeration as planned. We will continue to assess the Bureau’s efforts to conduct a cost-effective enumeration and look forward to keeping Congress informed of the Bureau’s progress.

Chairman Gowdy, Ranking Member Cummings, and Members of the Committee, this completes our prepared statement. We would be pleased to respond to any questions that you may have.

GAO Contacts and Staff Acknowledgments

If you have any questions about this statement, please contact David A. Powner at (202) 512-9286 or by e-mail at pownerd@gao.gov or Robert Goldenkoff at (202) 512-2757 or by e-mail at goldenkoffr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Other key contributors to this testimony include Lisa Pearson (Assistant Director); Jon Ticehurst (Assistant Director); Kate Sharkey (Analyst in Charge); Mark Abraham, Dewi Djunaidy; Hoyt Lacy; Andrea Starosciak; Umesh Thakkar; Timothy Wexler; and Katherine Wulff. Staff who made key contributions to the reports cited in this statement are identified in the source products.

Appendix I: Status as of August 2017 of Development and Integration Testing for Systems in the 2018 End-to-End Test

As part of its 2018 End-to-End Test, the Census Bureau (Bureau) plans to deploy 43 systems incrementally to support nine operations from December 2016 through the end of the test in April 2019. The nine operations are: (1) in-office address canvassing, (2) recruiting for address canvassing, (3) training for address canvassing, (4) in-field address canvassing operation, (5) recruiting for field enumeration, (6) training for field enumeration, (7) self-response (i.e., Internet, phone, or paper) operation, (8) field enumeration operation, and (9) tabulation and dissemination. According to the Bureau, a single system may be deployed multiple times throughout the test (with additional or new functionality) if that system is needed for more than one of these operations.

Table 1 describes the status as of August 2017 of development and integration testing for each system in the 2018 End-to-End Test. Specifically, as of August 2017, the Bureau had completed both development work and integration testing for 4 systems, and was in the process of completing development and testing for 39 systems.