GAO 

United States Government Accountability Office

Report to Congressional Requesters

2010 CENSUS 

Census Bureau Has 

Made Progress on 

Schedule and 

Operational Control 

Tools, but Needs to 

Prioritize Remaining 

System Requirements 

November 2009 

GAO-10-59 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

November 2009

2010 CENSUS

Census Bureau Has Made Progress on Schedule and 

Operational Control Tools, but Needs to Prioritize 

Highlights of GAO-10-59, a report to Remaining System Requirements

congressional requesters 

To carry out the decennial census, 

the U.S. Census Bureau (Bureau) 

conducts a sequence of thousands 

of activities and numerous 

operations. As requested, GAO 

examined (1) the Bureau’s use of 

scheduling tools to maintain and 

monitor progress and (2) the status 

of two systems key to field data 

collection: the control system the 

Bureau will use to manage the 

work flow for paper-based 

operations, including nonresponse 

follow-up, and the system used to 

manage quality control of two 

major field operations. 

GAO applied schedule analysis 

tools; reviewed Bureau evaluations, 

planning documents, and other 

documents on work flow 

management; and interviewed 

Bureau officials. 

What GAO Recommends

GAO recommends, among other 

things, that the Bureau include 

resource needs in its schedule for 

2020 and implement reliable 

progress measures of the 

development of a key system. 

In commenting on a draft of this 

report, the Department of 

Commerce provided additional 

details on steps the Bureau is 

already taking and plans to take to 

address the intent of GAO’s 

recommendations. GAO agrees that 

the monitoring efforts the 

department describes can help 

assess progress on key system 

development. GAO maintains that 

without prioritized requirements 

and reliable progress measures, the 

Bureau is unable to fully gauge its 

progress on system development. 

The Bureau’s master schedule provides a useful tool to gauge progress, 

identify and address potential problems, and promote accountability as the 

Bureau carries out the census. GAO found that the Bureau’s use of its master 

schedule generally follows leading scheduling practices that enable such highlevel oversight. However, errors GAO found in the Bureau’s schedule hinder 

the Bureau’s ability to identify the effects of activity delays and to plan for the 

unexpected. The Bureau has recently begun taking systematic steps to 

identify and correct remaining errors. However, within its schedule, the 

Bureau does not identify the resources needed to complete activities, making 

it difficult for the Bureau to evaluate the costs of schedule changes or the 

resource constraints that may occur at peak levels of activity. Leveraging the 

2010 scheduling experience and including resource needs in the 2020 schedule 

should facilitate planning for the 2020 Census, already underway. 

The automated control system that the Bureau plans to use to help manage 

major field data collection operations has significant development and testing 

milestones remaining, with some scheduled to finish shortly before the system 

needs to be deployed. This aggressive schedule leaves little time for resolving 

problems that may arise, and without prioritized and final software 

specifications and reliable progress measures, the Bureau may not get what it 

needs from the system to conduct the operations. Additionally, development 

of quality control software for two major field operations faces delays, 

although detailed specifications and test plans are final. 

Development and Testing Operations Control System Proceeding on a Tight Schedule 

2009 2010

Selected Census Operations

Source: GAO summary of 2010 U.S. Census Master Activity Schedule (9/25/2009).

Remote Alaska Enumeration

Remote Update/Enumerate

Update/Enumerate

Nonresponse Follow-up

Group Quarters Enumeration

Field Verification

Enumuration of Transitory 

Locations

Group Quarters Advance Visit

Development dates Testing dates Operation dates

1

Release

number

2

3

Apr. 1, 2010: 

Census Day

Dec. 31, 

2010: 

Delivery of 

apportionment counts

to the 

President

View GAO-10-59 or key components.

For more information, contact Robert 

Goldenkoff at (202) 512-2757 or 

goldenkoffr@gao.gov. 
Page i GAO-10-59 

Contents 

Letter 1

Background 3 

The Bureau Is Successfully Using Schedule Tools to Track the 

Implementation of the Census, but Opportunities Exist for 

Improvement in 2010 and Beyond 5 

Bureau Plans to Deliver Two Key Systems on a Tight Schedule 10 

Conclusions 16 

Recommendations for Executive Action 17 

Agency Comments and Our Evaluation 18 

Appendix I Scope and Methodology for Reviewing the Bureau’s 

Schedule 21 

Appendix II Comments from the Department of Commerce 23 

Appendix III GAO Contact and Staff Acknowledgments 27 

Related GAO Products 28 

Figure 

Figure 1: Development and Testing Schedule for Paper-Based 

Operations Control System Is Proceeding on a Tight 

Schedule 12 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately. 

 2010 Census 
United States Government Accountability Office

Washington, DC 20548 

 

November 13, 2009 

The Honorable Thomas R. Carper 

Chairman 

The Honorable John McCain 

Ranking Member 

Subcommittee on Federal Financial Management, Government 

 Information, Federal Services, and International Security 

Committee on Homeland Security and Governmental Affairs 

United States Senate 

The Honorable William Lacy Clay 

Chairman 

The Honorable Patrick T. McHenry 

Ranking Member 

Subcommittee on Information Policy, Census, and National Archives 

Committee on Oversight and Government Reform 

House of Representatives 

The Honorable Tom Coburn 

United States Senate 

The Constitution mandates that the federal government count the U.S. 

population every 10 years—a large and complex undertaking known as the 

decennial census. To carry out the census, the U.S. Census Bureau 

(Bureau) conducts numerous operations and coordinates a sequence of 

thousands of interdependent activities. Given mandated deadlines that the 

Bureau faces for delivering census tabulations, the Bureau uses a master 

activity schedule to help managers stay on track and alert them to 

potential delays. With less than 5 months until Census Day, April 1, 2010, 

there is little time remaining for the Bureau to deal with any unexpected 

problems that may arise, so early recognition of potential delays is 

essential. 

The largest census field operation is Nonresponse Follow-up (NRFU), 

where field staff visit households that have not mailed back their census 

forms. The Bureau will rely on a Paper-Based Operations Control System 

(PBOCS) to manage this follow-up operation and other activities 

scheduled to be completed in the coming months. Field office managers 

will use PBOCS to assign work to roughly 1.2 million temporary 

employees in about 500 offices nationwide. Among other activities, field 

office managers will also track response data collected by field staff from 

Page 1 GAO-10-59 2010 Census
 

an estimated 47 million census forms from households that did not return 

their forms by mail. 

Another key part of managing upcoming census field operations is 

ensuring the quality of the data collected. The quality control operation 

helps the Bureau determine whether the first interviews held to collect 

census data were done correctly, look for workers who may need 

additional training, and identify workers who are falsifying census data. 

For some operations, the quality control workload will be determined by 

matching and coding software that will identify cases that need follow-up 

and provide that case selection information to PBOCS. While the Bureau 

developed a similar operations control system for the 2000 Census and has 

used the core functionality of the matching software in census-like tests 

before, neither PBOCS nor the matching software was used in a “dress 

rehearsal” for the decennial held in 2008. PBOCS and its integration with 

the matching software has not been fully tested in a census-like 

environment. 

Because of the importance of staying on schedule to meet mandated 

deadlines, effectively managing field operations, and ensuring the quality 

of the data collected, as you requested, we examined (1) the Bureau’s use 

of scheduling tools to maintain and monitor progress and (2) the status of 

two systems critical to conducting field data collection: the control system 

the Bureau will use to manage the workflow for paper-based operations 

including NRFU, and the system used to manage quality control of two 

major field operations. 

To meet these objectives, we reviewed Bureau planning documents, 

evaluations, schedules, scheduling best practices, and operational and 

software specifications. We also interviewed Bureau and contractor staff 

regarding the planning, scheduling, and status of selected areas related to 

field office workflow management. To address the first objective, we 

examined the schedule, comparing its estimates to relevant best practices 

and testing its data for reliability using methods described in detail in 

appendix I. To address the second objective, we compared the schedule of 

Bureau operations to the testing schedule of systems needed to conduct 

those operations, examined previous evaluations of those systems, and 

evaluated the status of operational and software specifications. We also 

reviewed past GAO reports as appropriate to meet our objectives. 

We conducted this performance audit from October 2008 to November 

2009 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

Page 2 GAO-10-59 2010 Census
 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives. 

The Secretary of Commerce is legally required to (1) conduct the census 

on April 1 of the decennial year, (2) report the state population counts to 

the President for purposes of congressional apportionment by December 

31 of the decennial year, and (3) send population tabulations to the states 

for purposes of redistricting no later than April 1 of the year following 

Census Day. The Bureau has defined over 40 different operations in its 

high-level requirements document, describing all of the planned operations 

and systems needed to meet these mandates.1

Background 

For the 2010 Census, the Bureau is using a comprehensive master 

schedule to integrate the work to be carried out in the dozens of 

operations. The schedule provides a high-level roadmap for Bureau 

executives and is used to alert executives to activities that are behind 

schedule or experiencing issues, allowing problems to be addressed so the 

census can continue to proceed on track. Staying on schedule is crucial to 

accomplishing all of the tasks involved in conducting the census. In fact, 

scheduling and planning are so important that the Bureau has already 

established a high-level schedule for planning the 2020 Census. 

While the schedule can be used to manage census operations at a high 

level and dictates major time allocations and deadlines, local census 

offices across the nation require more detailed plans to conduct 

enumeration that exceed the detail included in the master schedule. A 

successful census depends, in large part, on the field work carried out in 

these local census offices where employees on the ground in local 

communities build a list of where to count people and count people who 

do not return their census forms. As we have previously reported2

, the 

Bureau had initially planned to carry out major field data collection 

activities using hand held computing devices. Development and 

performance problems with the hand held device led the Secretary of 

 

1

U.S. Census Bureau, 2010 Census Operational Plan, 2010 Census Information 

Memoranda Series No. 2 (July 7, 2009). 

2

GAO, 2010 Census: Fundamental Building Blocks of a Successful Enumeration Face 

Challenges, GAO-09-430T (Washington, D.C.: Mar. 5, 2009). 

Page 3 GAO-10-59 2010 Census
 

Commerce in April 2008 to abandon using the device for most of its 

intended operations and resulted in the Bureau removing the NRFU 

operation from the 2008 Dress Rehearsal. As a result, the Bureau was not 

able to use the dress rehearsal as a comprehensive end-to-end test of the 

interoperability of all of its planned systems, and the Bureau has had to 

develop plans to support and conduct the affected operations on paper as 

it did for the 2000 Census. 

For the 2010 Census, the Bureau will manage remaining fieldwork 

activities with PBOCS. This system is intended to provide managers with 

essential real-time information, such as worker productivity and 

completion rates for field operations. It also allows managers in the field 

to assign or reassign cases among workers. If the system does not work as 

intended, it could hinder or delay field operations and introduce errors 

into files containing collected data. 

Another responsibility of field offices is implementing the quality control 

process established by the Bureau to ensure that correct information is 

collected by field staff and data are not falsified. To ensure data quality 

and consistency of quality control procedures, the Bureau will manage, 

track, match, and review answers provided during re-interview operations 

using its Census Matching Review and Coding System (Census MaRCS). 

This system is also to designate quality control assignments where 

selected households will be re-interviewed in order to determine that the 

original enumerator correctly conducted the interview. Census MaRCS is 

also to assist in identifying interviews where the data from the re-interview 

do not match the data from the original interview, indicating that a 

mistake has been made. 

Both PBOCS and Census MaRCS are key systems that have not been fully 

tested. Since 2005, we have reported concerns with the Bureau’s 

management and testing of key information technology systems.3

 In March 

2009, we reviewed the status of and plans for the testing of key 2010 

Census systems.4

 We reported that while the Bureau has made progress in 

conducting systems, integration, and end-to-end testing, critical testing 

against baseline requirements still remained to be performed before 

 

3

GAO, 2010 Census: Basic Design Has Potential, but Remaining Challenges Need Prompt 

Resolution, GAO-05-9 (Washington, D.C.: Jan. 12, 2005). 

4

GAO, Information Technology: Census Bureau Testing of 2010 Decennial Systems Can 

Be Strengthened, GAO-09-262 (Washington, D.C.: Mar. 5, 2009). 

Page 4 GAO-10-59 2010 Census
 

systems would be ready to support the 2010 Census and the planning for 

the testing needed much improvement. While the Bureau has made 

noteworthy progress in gearing up for the enumeration, with less than a 

year remaining until Census Day, uncertainties surround the Bureau’s 

overall readiness for 2010.5

The Bureau has implemented processes around its master schedule that 

comply with a number of scheduling process criteria that are important to 

maintaining a schedule that is a useful management tool. Such a schedule 

can provide a road map for systematic execution of a program and the 

means by which to gauge progress, identify and address potential 

problems, and promote accountability. We have documented the 

importance of adhering to these criteria and of implementing associated 

best practices in our GAO Cost Estimating and Assessment Guide.

6

According to these criteria, a schedule should be 

• comprehensive, with logically sequenced activities spanning the scope 

of work to be performed so that the full picture is available to 

managers; 

• current, with the progress on ongoing activities updated regularly so 

that managers can readily know the status of the project; and 

The Bureau Is 

Successfully Using 

Schedule Tools to 

Track the 

Implementation of the 

Census, but 

Opportunities Exist 

for Improvement in 

2010 and Beyond 

• controlled, with a documented process for changes to the schedule so 

that the integrity of the schedule is assured. 

The Bureau’s master schedule represents all 44 of the operations 

described by the broad requirements for the census in its 2010 Census 

Operational Plan.

7

 While the Bureau continues to add activities to its 

central schedule, by including at least all the activities described in these 

broad requirements, the Bureau is ensuring that it has a comprehensive 

schedule that will be less likely to miss critical interactions between 

operations. The Bureau ensured a complete scope of the schedule with 

input from stakeholders throughout the agency, with reviews of previous 

schedules, and building on a number of census tests during the decade. As 

a result, the schedule is the primary source for senior managers, on a 

weekly basis, to determine what census activity is ahead of or behind 

 

5

GAO-09-430T

6

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: Mar. 2009). 

7

U.S. Census Bureau, 2010 Census Operational Plan. 

Page 5 GAO-10-59 2010 Census
 

schedule and provides a resource for determining any impact to the 

overall project of delays in major activities. 

The Bureau has documented and implemented a formal process for 

keeping the data in the schedule current. Staff within each Bureau division 

are responsible for ensuring that schedule activities within their division 

have their status updated on a weekly basis. Staff update the actual start 

and finish dates, the percentage of an activity completed so far, and 

estimates of the time remaining to complete each activity in progress. The 

Bureau is recording status information on an average of more than 1,300 

activities in the schedule ongoing during any given week, generating 

historical data that could provide valuable input to future schedule 

estimates. 

Finally, the Bureau has implemented a formal change control process that 

preserves a baseline of the schedule so that progress can be meaningfully 

measured. The Bureau’s criteria for justifying changes are clearly 

documented and require approval by a team of senior managers and 

acknowledgment of the impact by each affected team within the Bureau. 

Since the master schedule was baselined in May 2008, about 300 changes 

have been approved. Even corrections to the schedule for known errors, 

such as incorrect links between activities, must be approved through the 

change control process, helping to ensure the integrity of the schedule. 

In addition to these practices, the Bureau has positioned itself to monitor 

the schedule regularly to help ensure that the census is progressing and 

that work is being completed as planned. A central team of staff working 

with the schedule implements a process that begins with the weekly 

updates of the schedule status and involves subject matter experts from 

multiple divisions, and monitors and resolves schedule-related issues, 

resulting in a weekly briefing to the Deputy Director and the Director of 

the Census, which includes documented explanations for critical activities 

scheduled to start late. For example, the central team began reporting 

from the master schedule in March 2009 that the printing of questionnaires 

for a field operation to validate locations of group quarters8

 in September 

and October 2009 might be running late. The schedule showed that late 

printing of the questionnaires would trigger their late delivery and the late 

 

8

The Bureau conducts an operation where field staff visit potential group quarters—

locations of group housing, such as prisons and nursing facilities—to confirm each location 

and establish a contact person for later enumeration. 

Page 6 GAO-10-59 2010 Census
 

assembly of job assistance kits needed to support the operation, and thus 

put the timeliness of the operation in danger. According to a Bureau 

official, the Bureau then addressed the issue by deciding to unlink the kit 

assembly from the questionnaire printing, allowing kits to begin assembly 

on time and having questionnaires delivered directly to field offices when 

they were ready, letting the operation begin on time. 

Bureau Is Removing Logic 

Errors from Its Master 

Schedule to Improve Its 

Reliability 

When we began analyzing the Bureau’s master schedule, we discovered a 

significant number of activities in the schedule that had either missing or 

inaccurate information describing their relationships with other activities 

in the schedule. We brought these to the Bureau’s attention, and the 

Bureau has begun systematically identifying such activities and correcting 

their information in the schedule. 

In accordance with scheduling best practices, activities in the schedule 

should be linked logically with relationships to other activities that 

precede or follow them, and they should be linked in the correct order. 

Since reports that the Bureau uses to manage the census depend on the 

schedule having been built properly, inconsistent adherence to these 

scheduling practices has occasionally created false alarms about the 

schedule and created unnecessary work for those who have had to resolve 

them. 

In our analysis of the Bureau’s schedule, we found that nearly all 

relationships between activities are generally in place in the schedule and 

some activities in the schedule do not need relationships. However, many 

activities appeared in the schedule missing one of their logical 

relationships. From January 2009 through August, an average of more than 

1,200 of the more than 11,000 activities in the entire schedule were missing 

relationships to other activities from either their start or end dates. Each 

month, on average, over 1,100 of the over 6,100 as yet not completed 

activities were missing relationships. For example, within the Bureau’s 

master schedule, an activity listed for receiving finished materials from the 

NRFU re-interview operation appeared in the schedule with no 

relationship to subsequent activities, making it appear that any delays in 

its completion would have no impact on subsequent census activities. 

While the absence of such a relationship in the schedule does not imply 

that the Bureau would miss the potential impact of any delays in the 

completion of this activity, the incidence of a large number of such 

missing relationships can confound attempts to trace the chain of impacts 

that any delays may have throughout the schedule. 

Page 7 GAO-10-59 2010 Census
 

Similarly, we found a small number of activities in the schedule that had 

been linked together in the wrong order, so that one activity might appear 

to finish before a necessary prior activity had been completed. Such an 

incorrect relationship can unnecessarily complicate the use of the 

schedule to guide work or measure progress. The number of such 

apparent out-of-sequence activities in the entire schedule has decreased 

from on average more than 100 each month in January through March to 

60 in August. 

Since June 2009, Bureau staff have been running structured queries on the 

data supporting the master schedule to identify activities with missing or 

incorrect data; researching each activity to determine what, if any, 

corrections to the data are needed; forwarding proposed changes to 

affected activities, operation by operation, to program officials for review; 

and submitting changes to the Bureau’s formal change control process. 

The Bureau reports that since this concerted effort began to correct such 

errors that affect activities in 37 different census operations, the Bureau 

had completed research for activities in 15 of the operations and approved 

changes in 12 of them by early October 2009. The Bureau also informed us 

that this review process involving the program officials responsible for the 

logic errors has provided an educational opportunity for the officials to 

see how their programs can directly affect others, and as a result has 

heightened awareness about the importance of getting schedule 

information keyed in correctly. 

Improvements to the 

Schedule Could Help the 

Bureau Manage the 2020 

Census 

A schedule provides an estimate of how long a given work plan will take to 

complete. Since the duration of the work described by the activities listed 

in a schedule is generally uncertain, a schedule can be analyzed for the 

amount of risk that its underlying work plan is exposed to. Schedule risk 

analysis—the systematic analysis of the impact of a variety of “what if” 

scenarios—is an established best practice to help identify areas of a 

schedule that need additional management attention. Conducting a 

schedule risk analysis helps establish the level of confidence in meeting 

scheduled completion dates and the amount of contingency time needed 

for given levels of confidence, and helps identify high-priority risks to a 

schedule. The Bureau is tracking risks to the census and managing those 

risks on a regular basis, as documented in the 2010 Census Risk 

Page 8 GAO-10-59 2010 Census
 

Management Plan,

9

 but these data are not being mapped into the schedule 

at a level that can be used for a systematic schedule risk analysis. 

A well-defined schedule should help identify the amount of human capital 

and financial resources that are needed to execute the programs within 

the scope of the schedule, providing a real-time link between time and cost 

and helping to reduce uncertainty in cost estimates and the risk of cost 

overruns. However, the Bureau does not link within its schedule estimates 

of resource requirements—such as labor hours and materials—to 

respective activities. Having this information linked in a schedule 

enhances an organization’s capability to monitor, manage, and understand 

resource productivity; plan for the availability of required resources; and 

understand and report cost and staffing requirements. For example, if the 

Bureau were to find itself behind schedule with major operations to be 

completed, and resource requirements were linked in the schedule, the 

Bureau could then better assess the trade-offs between either adding more 

resources or reducing the scope of the operations. In addition, when 

resources are linked to activities in the schedule, scheduling tools can 

identify periods of their peak usage and assist managers with reordering 

activities to level out demands on potentially scarce or costly resources. 

When we met with Bureau officials and discussed this, they pointed out 

that incorporating this schedule best practice would be difficult to do late 

in the preparations for 2010, but they expressed interest in incorporating 

this schedule best practice as a step forward in the Bureau’s use of the 

schedule to manage decennial censuses. 

Finally, the Bureau’s use of a master schedule in 2010 that is, according to 

the Bureau, more highly integrated into the management of the decennial 

census provides an opportunity to draw many potential lessons for 2020. 

The Bureau learned lessons from its use of the 2000 master schedule as 

documented in a 2003 Bureau management evaluation,10 in particular with 

its adoption of the formal change control process implemented for 2010. 

Yet as noted in the evaluation, there were questions about the quality of 

the data maintained in the schedule. Without a reliable change control 

process, the schedule did not provide a reliable baseline, making 

evaluation of schedule and activity duration estimates difficult, if not 

 

9

U.S. Census Bureau, 2010 Census Risk Management Plan, 2010 Census Information 

Memoranda Series No. 5 (June 6, 2008). 

10U.S. Census Bureau, Management Evaluation of Census 2000, Census 2000 Evaluation 

Q.1 (Oct. 8, 2003). 

Page 9 GAO-10-59 2010 Census
 

impossible. The Bureau is generating a large amount of data—and 

experience—with its efforts in developing, maintaining, and using the 2010 

master schedule. Unless the Bureau prioritizes the need for documenting 

lessons learned from the current experience—as it did for the 2000 

Census—and formally puts in place an effort to capture and analyze 

schedule data, changes to baselines, and variances between estimated and 

actual durations, it runs the risk of missing out on another opportunity for 

using additional lessons some of its staff may already be learning. 

The automated control system that the Bureau plans to use to help manage 

the data collection operations of the decennial census still faces significant 

development and testing milestones, some of which are scheduled to be 

completed just before the system needs to be deployed before respective 

field operations begin. As a result, should the Bureau encounter any 

significant problems during final testing, there will be little time to make 

changes before systems are needed to support field operations. PBOCS 

will help manage both paper, and people, and it needs to exchange data 

successfully with several other Bureau systems, such as one used for 

processing payroll. The Bureau plans to complete development and testing 

of PBOCS in three major releases, grouping the releases of parts of PBOCS 

together loosely by the timing of the field operations those parts are 

needed to support. The Bureau already completed a preliminary release of 

PBOCS with limited functionality in June 2009 to support some initial 

testing. 

Bureau Plans to 

Deliver Two Key 

Systems on a Tight 

Schedule 

Figure 1 shows the development, testing, and operation periods for the 

three remaining releases and the operations that PBOCS supports. 

According to the baseline of the Bureau’s master schedule, PBOCS should 

be deployed and operational anywhere from 1 to 6 weeks before each 

operation begins for operations leading up to and including NRFU. 

According to the Bureau, the system should ideally be ready for use during 

training periods so that managers can familiarize themselves with the 

system they will have to use and can begin using the system to assign work 

to new staff. This also requires that PBOCS be ready in time so that 

production data can be loaded into the system, as well as information 

about the employees to whom work will be assigned. For example, PBOCS 

for NRFU is to finish its final testing in March 2010, about 9 weeks before 

NRFU is scheduled to begin on May 1, 2010, and deployment is scheduled 

to take place about 6 weeks before NRFU starts, leaving 3 weeks of 

contingency time in the event that unexpected problems arise during 

PBOCS development. This means that if any significant problems are 

identified during the testing phases of PBOCS, there is generally little time 

Page 10 GAO-10-59 2010 Census
 

to resolve the problems before the system needs to be deployed. In 

addition, it will be more difficult for the Bureau to integrate into PBOCS 

training for users on any late changes in the PBOCS software. While the 

Bureau relies on last-minute additions to training and procedures 

documents to communicate late changes to workers, Bureau officials 

agreed that it can be difficult to incorporate such last-minute additions 

into training sessions and for users to learn them, and doing so should be 

avoided if possible. 

Page 11 GAO-10-59 2010 Census
 

Figure 1: Development and Testing Schedule for Paper-Based Operations Control System Is Proceeding on a Tight Schedule 

Operation name and description

Remote Alaska Enumeration: Field staff visit 

limited-access households in Alaska’s isolated 

areas before the spring thaw.

Update/Leave: Field staff update addresses and 

leave census forms at housing units in areas that 

primarily do not have house numbers and/or 

street names.

Remote Update/Enumerate: Field staff update 

addresses and complete census forms in sparsely 

inhabited areas of Maine and Alaska.

Update/Enumerate: Field staff update addresses

and complete census forms in areas with special 

enumeration needs, such as American Indian 

reservations and seasonal resort areas.

Nonresponse Follow-up: Field staff follow up in 

person with nonresponding households.

Group Quarters Enumeration: Field staff visit 

group housing such as prisons and nursing 

facilities.

Vacant Delete Check: Field staff confirm earlier 

assessments that a housing unit is vacant or that 

it is no longer a housing unit.

Field Verification: Field staff verify the existence 

of housing units whose addresses do not match 

an address in the Bureau’s master address file.

Enumeration of Transitory Locations: Field 

staff conduct personal interviews with individuals

who do not have a usual home elsewhere.

Group Quarters Advance Visit: Field staff visit 

probable group quarters to confirm location and 

establish a contact person.

2009 2010

Source: GAO summary of 2010 U.S. Census Master Activity Schedule (9/25/2009).

Apr. 1, 2010: 

Census Day

Dec. 31, 2010: 

Delivery of 

apportionment 

counts to the 

President

Development dates

Operation dates

Testing dates

Release 1

Release

number

Release 2

Release 3

The Bureau also faces the significant challenge of developing the detailed 

specifications for the software to be developed. As of early-September 

2009, the Bureau had established high-level requirements for PBOCS and 

has reported completing development of release 1 of PBOCS. The Bureau 

reports that as of late-October its requirements development, system 

development, and system testing for phase 1 is largely completed. 

Page 12 GAO-10-59 2010 Census
 

However, the Bureau has not yet finalized the detailed requirements for 

this release or for later releases. High-level requirements describe in 

general terms what functions the system will accomplish, such as 

producing specific management reports on the progress of specific paperbased operations. Detailed requirements describe more specifically what 

needs to be done in order to accomplish such functions. For example, a 

detailed requirement would specify the specific data that should be pulled 

from the specific data set to produce specific columns of a specific report. 

While high-level requirements provide software programmers with general 

guidelines on, for example, what types of reports should be produced, 

without a clear understanding of the detailed requirements, the 

programmers cannot be sure that they are identifying the correct source of 

information for producing such reports, and reports can thus be 

inaccurate. According to Bureau officials, previous contract programmers 

with little decennial census experience and no involvement with current 

development efforts made erroneous assumptions about which data to use 

when preparing some quality control reports that became problematic in 

the dress rehearsal. 

Without detailed requirements, the Bureau also cannot be sure how 

frequently such reports should be updated or which staff should have 

access to which reports. Further, software developers may not have the 

required information to meet the Bureau’s needs. Also, as we have 

previously reported, detailed operational requirements determine system 

development, and without well-defined requirements, systems are at risk 

of cost increases, schedule delays, or performance shortfalls. As we have 

reported and testified numerous times, the Bureau experienced this with 

an earlier contract to automate the support of its field data collection 

activity, which included the failed handheld computing device.11

The Bureau’s PBOCS development managers have told us that they are 

working closely with stakeholders in an iterative process of short 

development cycles to help mitigate PBOCS development risks caused by 

not having detailed requirements written in advance. Embedding subject 

matter experts within the software development process can help mitigate 

risk inherent in the short time frame the Bureau has remaining to develop 

and test PBOCS. Yet the absence of well-documented and prioritized 

 

11For example, see GAO, Information Technology: Census Bureau Needs to Improve Its 

Risk Management of Decennial Systems, GAO-08-79 (Washington, D.C.: Oct. 5, 2007); 

Census 2010: Census at Critical Juncture for Implementing Risk Reduction Strategies, 

GAO-08-659T (Washington, D.C.: Apr. 9, 2008); and GAO-09-262. 

Page 13 GAO-10-59 2010 Census
 

detailed requirements for PBOCS, which still need to be developed and 

tested, remains among the most significant risks to getting PBOCS ready 

on time. Furthermore, the Bureau lacks reliable development progress 

measures that permit estimating which requirements may not get 

addressed and that are important to ensuring the visibility of the 

development program to Bureau leadership. Aggressive monitoring of 

system development and testing progress and of the effort remaining will 

help ensure that program officials who will rely on these systems can 

anticipate what risks they face and what mitigation activities they may 

need for shortfalls in the final systems. 

In recognition of the serious implications that a failed PBOCS would have 

for the conduct of the 2010 Census, and to see whether there were 

additional steps that could be taken to mitigate the outstanding risks to 

successful PBOCS development and testing, in June 2009 the Bureau 

chartered an assessment of PBOCS. The assessment team, chaired by the 

Bureau’s Chief Information Officer (CIO), reported initially in late July 

2009 and provided an update report in late August 2009. According to the 

August update and our discussion of it with the CIO, the team increased its 

risk rating in two areas of PBOCS that it is monitoring in part because of 

the absence of fully documented requirements, testing plans, progress 

measures, and deployment plans. 

In its comments on the draft of this report, the Department of Commerce 

provided information describing several steps the Bureau was taking to 

monitor the progress of PBOCS development. According to Commerce, 

the Bureau already does the following: 

• Daily Project Management Standup meetings, which cover action item 

management, calendar review, activity sequencing, and any threats or 

action-blocking issues. 

• Daily Architecture Review Board and team leads meetings. 

• Weekly Program Management Review Board meetings, and thriceweekly Product Architecture Review Board meetings. 

• At least weekly review of progress by the PBOCS Internal Assessment 

Team—chaired by the CIO. The team briefs the Bureau Director at 

least monthly. 

• Monthly Quality Assurance Board meetings and twice-monthly Risk 

Review Board meetings. 

At the end of our review, the PBOCS development team demonstrated two 

software tools it said it was using to help manage its iterative process of 

short development cycles. We did not fully assess their use of the tools. 

Page 14 GAO-10-59 2010 Census
 

The progress measures they demonstrated with one of the tools predicted 

that the completion dates would be missed. It also showed that the 

development team was underestimating the development effort required to 

achieve its iterative development goals. When we noted this, the 

presenters told us that the information in their management system was 

not current. Until the Bureau completes the detailed requirements for 

PBOCS and prioritizes them and its PBOCS development monitoring is 

relying on current and reliable progress measures, such as those the 

development team attempted to demonstrate to us for estimating the effort 

needed to complete remaining development, it will not be able to fully 

gauge the PBOCS development progress and have reasonable assurance 

that PBOCS will meet the program’s needs. The Bureau is continuing to 

examine how improvements will be made. 

Quality Control Matching 

and Coding Software for 

Nonreponse Follow-up and 

Update/Enumerate Faces 

Testing Delays and 

Revisions to Requirements 

The Bureau has experienced delays in the development and testing of 

software that will play a key role along with PBOCS in controlling and 

managing field data collection activity for the quality assurance programs 

of NRFU and Update/Enumerate. Census MaRCS will help manage the 

process of identifying systematic or regular violations in the door-to-door 

data collection procedures. In particular, Census MaRCS will be a tool to 

help target additional households needing reinterview as part of the 

quality assurance program for these two major census data collection 

operations in the field. Therefore, fully developing and testing Census 

MaRCS will be important to the successful conduct of the census. 

Like other systems at the Bureau, Census MaRCS had to undergo design 

changes when the Bureau made the April 2008 decision to switch to paperbased operations. Detailed performance requirements—such as the 

information to be included on reports and its sources, including 

performance metrics, such as the number of users the system is designed 

to handle—are documented and were baselined in May 2009. However, 

specifications have been added or clarified as software development has 

progressed and improvements have been suggested. 

Software development has at times been slower than expected, leading to 

delays in some testing. Test plans for Census MaRCS software and 

interfaces are in place, having been documented in May 2009. According to 

those plans, the Bureau is in the second of three stages of testing Census 

MaRCS and is scheduled to complete its final stage in December 2009, 

almost 2 months before its first deployment for the Update/Enumerate 

operation. Slower-than-expected development has delayed some parts of 

the second phase of testing, which will thus finish late according to the 

Page 15 GAO-10-59 2010 Census
 

Bureau, but Bureau officials have indicated that they believe that delay 

can be absorbed into the schedule, and that the system will be delivered as 

scheduled in February 2010. 

The compressed testing schedule leaves little time for additional delays in 

writing software or conducting tests. The Bureau is working on additional 

plans to test the interfaces between systems like PBOCS and Census 

MaRCS to ensure that they work together, but those test plans have not yet 

been finalized. Since Census MaRCS was not used in the dress rehearsal, 

and a full end-to-end system test—that is, a test of whether all interrelated 

systems collectively work together as intended in an operational 

environment—is not planned for in the time remaining before the system 

is required to be deployed, successful testing of the interfaces with other 

systems is critical to the system’s readiness. If development or testing 

delays persist, it will be more important than ever that system 

requirements be prioritized so that effort is spent on the “must haves” 

necessary for system operation. 

Our review of the Bureau’s master schedule for conducting the 2010 

Census and the processes the Bureau uses to manage it suggests that it is 

doing a commendable job conducting such a large and complex 

undertaking consistent with leading scheduling practices. Furthermore, 

the Bureau’s systematic effort to correct errors that we have identified in 

the schedule will further improve the ability of the master schedule to 

support senior management oversight and decision making as 2010 

approaches. Other improvements, such as embedding estimates of 

resource needs into the schedule, may take more time to implement. Yet, 

the Bureau’s generally well-defined and integrated schedule provides an 

essential road map for the systematic execution of the census and the 

means by which to gauge progress, identify and address potential 

problems, and promote accountability. Leveraging the Bureau’s 

experience with scheduling for 2010 by documenting it should provide 

lessons learned for similar efforts in 2020 as well. 

Conclusions 

Moreover, since we testified on the status of the 2010 Census in March 

2009,12 the Bureau has made progress on a number of key elements needed 

to manage the work flow in field operations. In particular, the Bureau has 

made progress in developing and testing systems to support paper-based 

 

12GAO-09-430T

Page 16 GAO-10-59 2010 Census
 

operations in the wake of the Secretary of Commerce’s April 2008 decision 

to switch to paper-based operations for most field data collection activity. 

That said, some delays are also occurring, and since so much still remains 

to be done in the months leading up to Census Day, the Bureau has limited 

time to fix any potential problems that arise in systems that are not 

thoroughly tested. 

While the Bureau has made significant progress in developing test 

schedules for key systems, careful monitoring of the progress in 

addressing, and setting priorities among, the remaining detailed 

requirements for the control system supporting paper-based operations is 

critical for the Bureau to anticipate what risks it faces and mitigations it 

may need for shortfalls in the final system. Based on the challenges faced 

in the earlier program implementing handheld computing devices, the 

Bureau has already experienced the ill effect of having to change its plans 

when a system does not fully meet planned program needs. With limited 

time before implementation, it is uncertain whether the Bureau may be 

able to complete development and fully test all key aspects of its systems, 

like PBOCS and Census MaRCS, which are still under development. 

Continued aggressive monitoring by the Bureau and improvements in the 

progress measures on system development and testing, of effort 

remaining, and of the risks relating to these efforts is needed. Such effort 

will help ensure that Bureau leadership, as well as Bureau program 

officials who will rely on these systems, have early warning on what, if 

any, desired system features will be unavailable in the final systems, 

maximizing time available to implement mitigation strategies as needed. 

We recommend that the Secretary of Commerce require the Director of 

the U.S. Census Bureau to take the following three actions: 

To improve the Bureau’s use of its master schedule to manage the 2020 

decennial census: 

Recommendations for 

Executive Action 

• Include estimates of the resources, such as labor, materials, and 

overhead costs, in the 2020 integrated schedule for each activity as the 

schedule is built, and prepare to carry out other steps as necessary to 

conduct systematic schedule risk analyses on the 2020 schedule. 

• Take steps necessary to evaluate the accuracy of the Bureau’s 

baselined schedule and determine what improvements to the Bureau’s 

schedule development and management processes can be made for 

2020. 

Page 17 GAO-10-59 2010 Census
 

To improve the Bureau’s ability to manage paper-based field operations in 

the 2010 Decennial Census, 

• finalize and prioritize detailed requirements and implement reliable 

progress reporting on the development of the paper-based operations 

control system, including estimates of effort needed to complete 

remaining development. 

The Secretary of Commerce provided written comments on a draft of this 

report on November 3, 2009. The comments are reprinted in appendix II. 

Commerce did not comment on the first two recommendations, but 

provided additional information on steps it had already been taking to 

monitor progress of PBOCS development related to the third 

recommendation. Commerce commented on how we characterized the 

status of system development and testing, and provided additional 

statements about the status. Commerce also made some suggestions 

where additional context or clarification was needed, and where 

appropriate we made those changes. 

Agency Comments 

and Our Evaluation 

With respect to our third recommendation to finalize and prioritize 

detailed requirements and implement reliable progress reporting on 

PBOCS, including estimates of effort needed to complete remaining 

development, Commerce described numerous regular meetings that the 

development team holds, as well as efforts by others in the Bureau to 

monitor and report on PBOCS development progress. We added 

references to these monitoring efforts within the report. We agree that the 

monitoring efforts the department describes can help assess the progress 

being made; however, the monitoring efforts can only assess the progress 

being reported to them. First, a complete set of requirements is needed to 

understand the work that has been accomplished and the work remaining. 

As we have noted, the Bureau has not yet developed a full set of 

requirements. Second, the management tool’s measurement of the 

development activity successfully addressing the requirements is directly 

related to the effectiveness of test cases developed for those requirements. 

However, as we have previously noted, if a requirement has not been 

adequately defined, it is unlikely that a test will discover a defect. 

Accordingly, until the Bureau completes the detailed requirements for 

PBOCS and prioritizes them it is unable to use these tools to fully gauge its 

progress toward meeting the overall project’s goals and objectives of 

system development. We clarified our discussion of this in the report and 

reworded the recommendation to better focus on the need for reliable 

information. 

Page 18 GAO-10-59 2010 Census
 

Commerce maintained that our draft report implied that no PBOCS 

development had begun and that no testing would be completed until 

March 2010. The draft report described development and testing dates that 

clearly illustrate that both development and testing have been occurring 

over several months leading up to their conclusion. We have included 

additional language in the text to clarify that development and testing take 

place over a period of time. We have also included a statement from 

Commerce that much of this activity has largely been completed for the 

first of three phases. 

Commerce commented on the accuracy of the dates used in the figures in 

the draft report. We verified that the dates we used were the correct dates 

that the Bureau had provided to us earlier. We made minor adjustments to 

some of the gridlines in the graphic for presentation purposes. 

The Bureau also provided additional information on the prior testing of 

matching software, the context for NRFU being dropped from the dress 

rehearsal, how PBOCS errors could potentially introduce errors into 

census files, and contract programmers we reported being involved in 

dress rehearsal PBOCS not being the same ones helping the Bureau with 

PBOCS development now. We revised the report as appropriate in 

response. 

Commerce commented on our discussion of the unreliability of 

information that the Bureau PBOCS development team provided us during 

a demonstration of two tools that it uses to help manage its iterative 

process of short development cycles. Commerce described the use of one 

of the two tools but not the one whose progress tracking measures were 

demonstrated to us and for which the team told us that data were not 

current. We have revised the text to more clearly state that more than one 

tool was used to demonstrate how development and testing was being 

managed by the PBOCS development team, and we have added additional 

language to describe the progress information that should have been 

current but that was not. 

Finally, as we had noted in the draft report, the Bureau has already begun 

taking action to address errors we had identified in its master schedule. 

Since we sent a draft of this report to Commerce, the Bureau provided 

additional information on the status of that effort, and we have updated 

this report accordingly. 

Page 19 GAO-10-59 2010 Census
 

We are sending copies of this report to the Secretary of Commerce, the 

Director of the U.S. Census Bureau, and interested congressional 

committees. The report also is available at no charge on GAO’s Web site at 

http://www.gao.gov. 

If you have any questions about this report please contact me at (202) 512-

2757 or goldenkoffr@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this report. GAO staff who made major contributions to this report are 

Robert Goldenkoff 

listed in appendix III. 

Director 

Strategic Issues 

Page 20 GAO-10-59 2010 Census
Appendix I: Scope and Methodology for Appendix I: Scope and Methodology for Reviewing the Bureau’s Schedule 

Reviewing the Bureau’s Schedule 

• We reviewed the U.S. Census Bureau (Bureau) program’s schedule 

estimates and compared them with relevant best practices in GAO 

Cost Estimating and Assessment Guide: Best Practices for 

Developing and Managing Capital Program Cost1

 to determine the 

extent to which they reflect key practices that are fundamental to 

having a reliable schedule. These practices address whether the 

schedule is 

• comprehensive, with logically sequenced activities spanning the scope 

of work to be performed so that the full picture is available to 

managers; 

• current, with progress on ongoing activities updated regularly so that 

managers can readily know the status of the project; and 

• controlled, with a documented process for changes to the schedule so 

that the integrity of the schedule is ensured. 

In doing so, we independently assessed a copy of the program’s integrated 

master schedule and its underlying schedules against our best practices. 

We also interviewed knowledgeable program officials to discuss their use 

of best practices in creating the program’s current schedule and we 

attended a schedule walk-through to better understand how the schedule 

was constructed and maintained. 

We tested the Bureau’s schedule data for reliability by 

• running a schedule check report in Pertmaster which is a scheduling 

analysis software tool that identifies missing logic, constraints, and so 

forth; 

• using the schedule information from Pertmaster, copying the schedule 

data into Excel, and checking for specific problems that could hinder 

the schedule’s ability to dynamically respond to changes; 

• examining whether there were any open-ended activities (i.e., activities 

with no predecessors, successors, or both); 

• searching for activities with poor logic; 

• identifying whether there were any lags or leads that should only be 

used to show how two tasks interact and not to represent work; 

• determining if activities were resource loaded, which helps to cost out 

the schedule and examine whether resources are overstretched or not 

available when needed; 

• examining whether the schedule was baselined, when it had its status 

updated, and what deviations there were from the plan; and 

 

1

GAO-09-3SP

Page 21 GAO-10-59 2010 Census 
Appendix I: Scope and Methodology for 

Reviewing the Bureau’s Schedule 

• examining if there were any actual start or finish dates recorded in the 

future and whether there was any broken logic. 

Page 22 GAO-10-59 2010 Census
Appendix II: Comments from the Department Appendix II: Comments from the Department of Commerce 

of Commerce 

Page 23 GAO-10-59 2010 Census 
Appendix II: Comments from the Department 

of Commerce 

Page 24 GAO-10-59 2010 Census
Appendix II: Comments from the Department 

of Commerce 

Page 25 GAO-10-59 2010 Census
Appendix II: Comments from the Department 

of Commerce 

Page 26 GAO-10-59 2010 Census
Appendix III: GAO

A

 Contact and Staff 

cknowledgments 

Page 27 GAO-10-59 

Appendix III: GAO Contact and Staff 

Acknowledgments 

Robert Goldenkoff, (202) 512-2757 or goldenkoffr@gao.gov GAO Contact 

In addition to the contact named above, Ty Mitchell, Assistant Director; 

Virginia Chanley; Vijay D’Souza; Jason Lee; Andrea Levine; Donna Miller; 

Crystal Robinson; Jessica Thomsen; Jonathon Ticehurst; and Katherine 

Wulff made key contributions to this report. 

Acknowledgments 

 2010 Census 
Related GAO Products Related GAO Products 

2010 Census: Census Bureau Continues to Make Progress in Mitigating 

Risks to a Successful Enumeration, but Still Faces Various Challenges. 

GAO-10-132T. Washington, D.C.: October 7, 2009. 

2010 Census: Fundamental Building Blocks of a Successful 

Enumeration Face Challenges. GAO-09-430T. Washington, D.C.: March 5, 

2009. 

Information Technology: Census Bureau Testing of 2010 Decennial 

Systems Can Be Strengthened. GAO-09-262. Washington, D.C.: March 5, 

2009. 

Census 2010: Census Bureau’s Decision to Continue with Handheld 

Computers for Address Canvassing Makes Planning and Testing 

Critical. GAO-08-936. Washington, D.C.: July 31, 2008. 

Census 2010: Census at Critical Juncture for Implementing Risk 

Reduction Strategies. GAO-08-659T. Washington, D.C.: April 9, 2008. 

Information Technology: Census Bureau Needs to Improve Its Risk 

Management of Decennial Systems. GAO-08-79. Washington, D.C.: 

October 5, 2007. 

2010 Census: Basic Design Has Potential, but Remaining Challenges 

Need Prompt Resolution. GAO-05-9. Washington, D.C.: January 12, 2005. 

(450717) Page 28 GAO-10-59 20 10 Census
GAO’s Mission The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday afternoon, GAO 

posts on its Web site newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.” 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s Web site, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information. 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, DC 20548 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548 

Public Affairs 

Please Print on Recycled Paper