2020 Census: Bureau Has Made Progress with Its Scheduling, but Further Improvement Will Help Inform Management Decisions
Released: 2018-07-26T00:00:00

Letter

Background

A reliable schedule is critically important for a successful 2020 Census. In February 2017, we added the 2020 Census to our High-Risk List because operational and other issues including scheduling are threatening the Bureau’s ability to deliver a cost-effective enumeration. We reported on concerns about the quality of the Bureau’s schedule and cost assessment, the Bureau’s capacity to implement innovative census-taking methods, and uncertainties surrounding critical information technology systems. Underlying these issues are challenges in such essential management functions as the Bureau’s ability to collect and use real-time indicators of schedule, cost, and performance; follow leading practices for scheduling, cost estimation, risk management, and IT acquisition, development, testing, and security; and cost effectively deal with contingencies including, for example, fiscal constraints, potential changes in design, and natural disasters that could affect the enumeration.

Reliable scheduling practices are essential for managing tradeoffs between cost, schedule, and scope. Among other things, scheduling allows program managers to decide between possible sequences of activities, determine the flexibility of the schedule according to available resources, predict the consequences of managerial action or inaction in events, and allocate contingency plans to mitigate risk. Following changes in a program, the schedule is used to forecast the effects of delayed, deleted, and added effort, as well as possible avenues for time and cost recovery.

Scheduling is important because the cost of counting the nation’s population has been escalating with each decade. The 2010 Census was the most expensive in U.S. history at about $12.3 billion, and was about 31 percent more costly than the $9.4 billion 2000 Census (in 2020 constant dollars). According to the Bureau, the total cost of the 2020 Census is now estimated to be approximately $15.6 billion dollars, more than $3 billion higher than previously estimated by the Bureau.

Moreover, as shown in figure 1, the average cost for counting a housing unit increased from about $16 in 1970 to around $92 in 2010 (in 2020 constant dollars). At the same time, the return of census questionnaires by mail (the primary mode of data collection) declined over this period from 78 percent in 1970 to 63 percent in 2010. Declining mail response rates have led to higher costs because the Bureau needs to send temporary workers to each nonresponding household to obtain census data.

The schedules we reviewed for this report—2020 Census Geographic Programs, 2018 End-to-End Test Address Canvassing, and 2018 End-to- End Census Test Nonresponse Follow-up—relate to the key activities of developing an accurate address list and following up with households that did not mail back their census forms. The Bureau relies on a complete and accurate address list to maximize the more cost-efficient self- response rate.

The three projects we selected contribute to two of the Census Bureau’s largest field operations. The Bureau’s Geographic Programs Operation maintains the Bureau’s master address file and mapping data used to conduct the 2020 Census. The Bureau’s Geographic Programs Operation provides the most current address list to the Bureau’s Address Canvassing Operation, where Bureau staff make updates to the address list via in-office and in-field procedures. These updates are processed on an ongoing basis throughout the decade. The Bureau conducts its Nonresponse Follow-up Operation after Census Day by having enumerators go door-to-door to determine the housing unit status for addresses that do not self-respond to the 2020 Census, and enumerate households that are determined to be occupied.

Selected Census Schedules Better Reflect Characteristics of a Reliable Schedule Compared to Prior Assessment, though Weaknesses Remain

We have previously reported in our Schedule Assessment Guide that a reliable schedule can provide a road map for systematic execution of a program, and the means by which to gauge progress, identify and address potential problems, and promote accountability. The guide identifies four characteristics of a reliable schedule:

Comprehensive: The schedule should identify all activities and resources necessary to accomplish the project. The schedule should cover the scope of work to be performed so that the full picture is available to managers.

Well-constructed: Activities should be logically sequenced and critical activities that would affect the timelines of the schedule should be identified.

Credible: All schedules should be linked to a complete master schedule for managers to reference and analyzed for how risk impacts the outcome of the schedule.

Controlled: There should be a documented process for changes to the schedule so that the integrity of the schedule is assured.

For a schedule to be reliable, it must substantially or fully meet all criteria for these four characteristics. These characteristics, their related leading practices, and their criteria are described in more detail in appendix II.

In 2013, we assessed the Bureau’s 2020 Research and Testing and Geographic Support System Initiative schedules using these criteria. While the results exhibited some of the characteristics of a reliable schedule, important weaknesses remained. Both schedules substantially met one of the four characteristics (controlled) and minimally or partially met the other three characteristics (comprehensive, well-constructed, and credible).

For this review, we assessed the 2018 End-to-End Census Test Address Canvassing, 2018 End-to-End Census Test Nonresponse Follow-up, and 2020 Census Geographic Programs projects’ schedules. We found that overall the selected schedules better reflected two of the four characteristics of a reliable schedule compared to our 2013 assessment (see figure 2).

Examples of the extent to which these characteristics were met are provided below. For a more detailed explanation of our assessment results, see appendix III.

Comprehensive–Selected Schedules Partially Meet Characteristic but Do Not Identify Needed Resources

As with our 2013 schedule assessment, our 2018 analysis found that the Bureau is partially meeting the characteristics of a comprehensive schedule. For example, the projects we assessed reflect the work to be accomplished for the project schedules, and each project schedule includes estimates of the duration of each activity. Additionally, the 2018 End-to-End Census Test Address Canvassing and the 2018 End-to-End Census Test Nonresponse Follow-up project schedules contain clear start and finish milestones, and map to the census program work breakdown structure—a detailed definition of the work necessary to accomplish a program’s objectives.

This leading practice of capturing all activities was substantially met, not fully met (see appendix III for a more detailed explanation), because while for each project all activities and milestones are mapped to their work breakdown structures by codes, there are no corresponding dictionaries to define the work. The absence of such a dictionary could potentially lead to confusion among staff in different census offices about the scope of the work they are responsible for performing.

Our schedule guide states that a work breakdown structure dictionary is a valuable communication tool between systems engineers, program management, and other stakeholders because it provides a clear picture of what efforts have to be accomplished. Bureau officials stated that although their 2020 Schedule Management Plan requires each project to have a schedule work breakdown structure dictionary, as project schedules are updated, they have not created these required dictionaries. As an alternative, they noted that the 2020 Census Operational Plan includes details and definitions of the projects.

Additionally, none of the three schedules we assessed include information about what levels of resources, such as labor and equipment, are required to complete the planned work—including this information is called resource loading. The Bureau’s 2020 Schedule Management Plan states that it is the responsibility of a representative from a project team and the schedule staff to assign resources to an individual project schedule, and that defining and assigning resources should be done following the testing phase of the 2020 Census Lifecycle.

The Bureau is now in its implementation phase (see figure 3 below), so according to its management plan, resource loading should have begun. But it has not. For example, the 2018 End-to-End Census Test Address Canvassing project schedule did not include any resource information on the recruiting and hiring goals for the address canvassing field work.

Instead, Bureau officials stated that they are estimating the cost of activities using a software tool separate from the current schedule management tool. They further stated that this Bureau-wide solution includes all 2020 Decennial Census staff as Decennial funded resources. However, the information in this separate tool has no effect on the durations or forecasted start and finish dates of detailed activities within individual projects. Furthermore, the separate tool does not always track all activities at the lowest level in the schedule, so that Bureau managers do not have reliable visibility with it on the efforts of the lowest level of detailed activities.

Resource loading is important for any agency, but is particularly important for the Census Bureau, given its statutorily mandated deadlines. Missed deadlines or schedule slippage can easily jeopardize the quality of the 2020 Census, and there is little room for error given that census data are used to apportion the seats of the House of Representatives, redraw congressional districts, and allocate billions of dollars each year in federal financial assistance. In our schedule guide, we reported that including resources such as labor, materials, and overhead costs can make a schedule a more useful management tool.

A resource-loaded schedule can help management with things such as computing labor and equipment hours, calculating total project and per- period cost, resolving resource allocation conflicts, determining whether all required resources will be available when they are needed, and establishing the reasonableness of the plan. For example, information on the resource needs of field operations in the 2018 End-to-End Census Test would assist management in determining if the appropriate resource allocations have been made for any given test activity. It would also aide in forecasting the likelihood that those resources will be available to complete the 2018 End-to-End Census Test Address Canvassing and Nonresponse Follow-up activities as scheduled. If the schedule does not allow insight into the current or projected allocation of resources for these test activities, the Bureau’s risk of key end-to-end test milestones slipping increases significantly.

In 2009, we reviewed the Bureau’s schedule and recommended that the Bureau include estimates of the resources in the 2020 integrated schedule for each activity as the schedule was built. The Department of Commerce did not respond to the recommendation at that time. In our 2013 assessment of the Bureau’s schedule, Bureau officials stated that they hoped to begin identifying the resources needed for each activity in their schedules by early 2014. However, as of May 2018, the Bureau has not yet implemented this recommendation. Senior Bureau officials have now stated that the Bureau would require additional staffing in the Schedule Management Branch in the Decennial Census Management Division in order to plan for and implement resource loading. When the Bureau has resource loaded its schedule, it will be able to use the schedule more effectively as a management tool.

Well-Constructed–The Bureau Demonstrated Improvement in Selected Schedules Compared to Prior Assessment

Our 2013 assessment of the Bureau’s schedule reported that the Bureau only minimally met the characteristics of a well-constructed schedule. Our 2018 assessment found that two of the selected project schedules now substantially met this characteristic and one partially met it. In this assessment, Bureau officials linked many of the activities clearly and in a straightforward sequence in the schedule, which was not always the case in prior assessments.

This improvement is important because it helps staff identify next steps as they progress through such things as acquiring and mobilizing the staff needed to conduct the address canvassing and nonresponse follow-up test field work, and helps managers identify the impact of changes in one activity on subsequent activities. For example, the schedule lays out the sequence of activities needed, such as developing training materials, recruiting field staff, training staff and equipping them with the tools needed to complete the test. Our assessment also concluded that two of the three project schedules we assessed have valid critical paths, which is the sequence of activities in the schedule that, according to their current status, lead to the program’s earliest completion date. A valid critical path allows management to focus on activities that will lead to the project’s success.

The 2020 Census Geographic Programs project schedule partially met the well-constructed characteristic due to problems existing within the schedule’s sequencing logic. In particular, we found a large number of unjustified date constraints and lags. In part because of these sequencing issues, total float calculations—that is, the amount of time a predecessor activity can slip before its delay affects the program’s estimated finish date—appear unreasonably high. Additionally, this project schedule has activities on the critical path with long durations. For example, the project schedule for Geographic Programs included several long-duration activities on its critical path that relate to the Bureau’s collection of community boundary data—information essential to delineating geographic boundaries used in the tabulation of census data. These critical long-duration activities make it difficult to measure time- critical progress on such activities in the near term.

These issues with how the schedule is constructed can also cause schedule users to lack confidence in the forecasted dates. Bureau officials acknowledged that the 2020 Census Geographic Programs project schedule had logic issues at the time because it was in the middle of a revision. Bureau officials stated that the standard process is to update the project schedule in an offline version and then assess the quality and impacts of changes before acceptance. According to Bureau officials, the Geographic Programs project schedule did not follow this process and was instead updated in the live version of the schedule because of time constraints.

Credible–Selected Schedules Partially Meet Characteristic, but the Bureau Has Not Carried Out a Schedule Risk Analysis

Our 2013 assessment of the Bureau’s schedule found that the Bureau minimally met the characteristics of a credible schedule. Our 2018 assessment of the Bureau’s schedule found that the Bureau’s scheduling practices for a credible schedule have improved. We found that there is now a clear relationship between lower-level activities and higher-level activities and milestones, and there is generally better consistency of dates between the project schedule and higher-level management documents.

However, the Bureau has not carried out a systematic quantitative risk analysis on its schedule. A schedule risk analysis is a statistical simulation of the possible effects of threats, opportunities, and general uncertainty to a program’s schedule that results in a quantifiable level of confidence in meeting the program’s key milestone dates. While the Bureau has identified and continues to track risks to its 2018 End-to-End Test address canvasing and nonresponse follow-up efforts in risk registers, a quantitative risk analysis would illustrate the impact of risks on the project schedule, and how those risks would affect the Bureau’s ability to meet milestones on time. Such an analysis would also provide a measure of how much time contingency should be built in the schedule to help manage prioritized risks, and, implicitly, provide indications of where additional resources might be needed to stay on schedule.

In response to our 2013 schedule assessment, Bureau officials said they were waiting for decisions about scheduling software before making decisions about a schedule risk analysis. As of May 2018, the Bureau has conducted three risk analyses to prove the software’s capability. However, the Bureau still had not conducted a schedule risk assessment on the current integrated master schedule used to manage the 2020 Census program.

Without a schedule risk analysis, the Bureau cannot determine the likelihood of each project’s completion date; how much schedule risk contingency is needed to provide an acceptable level of certainty for completion by a specific date; which risks are most likely to affect the schedule; how much contingency time each risk requires; and the sequence of activities that are most likely to delay the project. Senior Bureau officials stated that a schedule risk assessment plan and process was approved by Bureau management in late May 2018 and that they hope to implement this plan in summer 2018. They intend to conduct an internal review over the next couple months to determine how to best use the information this risk assessment would yield. Follow through on their plans is critical to ensuring our recommendation is implemented.

Controlled–Selected Schedules Substantially Meet the Characteristic

As with our 2013 schedule assessment, our 2018 analysis reported that the Bureau’s scheduling practices are substantially meeting the characteristics of a controlled schedule. Our analysis determined that there are no date anomalies in the project schedules, such as planned dates in the past or actual dates in the future. We found the schedule was current as of the date delivered to us, and according to Bureau documents, the schedule is updated weekly following an established schedule process.

Additionally, the Bureau reported that it has a schedule management process in place and a method for logging changes to the schedule in accordance with leading practices. Bureau officials reported that they monitor schedule trends, including bi-weekly schedule reliability checks using the Defense Contract Management Agency 14-Point assessment, a commonly used set of schedule integrity and reliability measures. Bureau officials also provided the May 2014 Program Change Management Process Strategy which defines the process for initiating changes to the integrated performance measurement baseline configuration; analyzing the impact of changes to project cost, schedule and scope; approving or disapproving changes; and updating project or product specifications and baselines.

However, our assessment found that the Bureau did not fully meet this characteristic for a controlled schedule. The Bureau lacked sound documentation of the schedule in the form of a schedule basis document, and changes to the current schedules in the form of a schedule narrative. The current schedule should be documented in a schedule narrative with each update, including changes made to the schedule during status updates and changes that are justified along with their likely effect on future activities. The Bureau had not prepared such narratives. Additionally, none of the three schedules were supported by a schedule baseline document—a single document that defines the organization of a schedule, describes the logic of the network, describes the basic approach to managing resources, and provides a basis for all parameters used to calculate dates. Sound documentation helps with analyzing changes in the program schedule and identifying the reasons why actual schedule results vary from their estimates, thereby contributing to the collection of data that can be useful to evaluations of schedule efforts, and that can be used to support future estimates.

While the Bureau has made improvements to implement the recommendations regarding the comprehensiveness and construction characteristics of the Bureau’s scheduling practices, the Bureau’s lack of resource loading and a risk assessment of the schedule continue to affect the reliability of the Bureau’s schedule. The schedule would be a more useful management tool if the Bureau increased the schedule’s reliability by addressing these weaknesses. To address these remaining weaknesses, we continue to believe that these recommendations are valid in order to ensure the 2020 schedule can support key management decisions.

Agency Comments and Our Evaluation

We provided a draft of this report to the Department of Commerce. In its written comments, reproduced in appendix IV the Department of Commerce agreed with our findings.

We are sending copies of this report to the Secretary of Commerce, the Under Secretary of Economic Affairs, the Acting Director of the U.S. Census Bureau, and interested congressional committees. The report also is available at no charge on GAO’s website at http://www.gao.gov.

If you have any questions about this report, please contact me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff that made major contributions to this report are listed in appendix V.

Appendix I: Objective, Scope, and Methodology

This report assesses the extent to which the Bureau is using leading practices for scheduling key projects. We did this by focusing on three 2020 projects. We selected the three projects from the December 17, 2017, version of the 2020 Census integrated master schedule. That schedule consists of 255 total projects, of which 134 were remaining to be completed. We made our selections based on the cost of the projects, their significance to the 2020 Census, and the fact that they were in progress. The 3 projects selected for review are 2018 End-to-End Census Test Address Canvassing, 2018 End-to-End Census Test Nonresponse Follow-up, and 2020 Census Geographic Programs.

We reviewed the project schedules and underlying sub-schedules to assess them against the 10 scheduling leading practices by:

Checking for specific problems that could hinder the schedule’s ability to respond to changes. For example, we:

Examined if there are any open-ended activities (i.e., activities with no predecessor and/or successors),

Searched for activities with poor logic:

For example, Start to Start successor only or Finish to Finish predecessor only which represent dangling logic, or

Logic on summary tasks rather than attached to detailed tasks (summary tasks are for organizing the schedule and should not drive the logic).

Looked for activities with constraints which keep the schedule rigid (e.g., start no earlier than, finish no later than, etc.), Identified any lags or leads which should only be used to show how two tasks interact and not to represent work,

Determined if activities were resource loaded—which helps to cost out the schedule—and examine whether resources are over allocated or not available when needed,

Examined the length of activity durations and compared them to the program management review cycle,

Checked for horizontal and vertical integration within the schedule,

Examined the schedule critical path to determine whether or not it was reliable and logical,

Examined schedule float and determine if it was reasonable, and

Examined whether the schedule was baselined, its status cycle, and what deviations there were from the original plan. We also determined if there were any actual start or finish dates recorded in the future and whether there was any broken logic between planned tasks.

We also interviewed Bureau officials responsible for the 2020 schedule. We scored each scheduling leading practice on a five-point scale ranging from “not met” to “met.” We determined the characteristic assessment rating by assigning each best practice rating a number and taking the average. The numerical ratings and ranges of the resulting averages are as follows.

We then compared these results with our prior assessments of the Bureau’s schedule, particularly those where recommendations were made, and we updated the status of those recommendations.

Assessing only three key projects limits possible statements about the Bureau’s entire schedule. For example, if the Bureau is not following best practices in creating and maintaining the three project schedules, we can conclude that the larger integrated schedule is unreliable. This is because an integrated master schedule consolidates lower-level project schedules; errors and reliability issues in lower levels will be transferred to higher- level schedules. However, if the selected lower-level projects are deemed reliable, we cannot definitively determine the reliability of the integrated master schedule because the other projects that were not assessed may be unreliable.

Appendix II: Description of Scheduling Leading Practices

Characteristic Comprehensive Capturing all activities

Description A schedule should reflect all activities defined in the project’s work breakdown structure and include all activities to be performed by the government and contractor.

The schedule should realistically reflect the resources (i.e., labor, material, and overhead) needed to do the work, whether all required resources will be available when needed, and whether any funding or time constraints exist.

The schedule should reflect how long each activity will take to execute.

The schedule should be planned so that all activities are logically sequenced in the order they are to be carried out.

The schedule should identify the critical path, or those activities that, if delayed, will negatively impact the overall project completion date. The critical path enables analysis of the effect delays may have on the overall schedule.

The schedule should identify float—the amount of time an activity can slip in the schedule before it affects other activities—so that flexibility in the schedule can be determined. As a general rule, activities along the critical path have the least amount of float.

Credible

The detailed schedule should be horizontally traceable, meaning that it should link products and outcomes associated with other sequenced activities. The integrated master schedule should also be vertically traceable—that is, varying levels of activities and supporting subactivities can be traced. Such mapping or alignment of levels enables different groups to work to the same master schedule.

The schedule should include a schedule risk analysis that uses statistical techniques to predict the probability of meeting a completion date. A schedule risk analysis can help management identify high priority risks and opportunities.

Controlled

Progress updates and logic provide a realistic forecast of start and completion dates for program activities. Maintaining the integrity of the schedule logic at regular intervals is necessary to reflect the true status of the program. To ensure that the schedule is properly updated, people responsible for updating should be trained in critical path method scheduling.

A baseline schedule represents the original configuration of the program plan and is the basis for managing the project scope, the time period for accomplishing it, and the required resources. Comparing the current status of the schedule to the baseline can help managers target areas for mitigation.

Appendix III: Assessment of the Extent to Which the Bureau Followed Scheduling Leading Practices

Appendix IV: Comments from the Department of Commerce

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

Other key contributors to this report include Ty Mitchell, Assistant Director; Juaná Collymore; Rob Gebhart; Yvette Gutierrez; Jason Lee; Kayla Robinson; Cynthia Saunders; and Timothy Wexler.