U.S. CENSUS BUREAU 

2020 Census Planning: 

Delays with 2010 Census 

Research Studies May 

Adversely Impact the 2020 

Decennial Census 

FINAL REPORT NO. OIG-12-023-1 

APRIL 5, 2012 

U.S. Department of Commerce 

Office of Inspector General 

Office of Audit and Evaluation 

FOR PUBLIC RELEASE 
UNITED STATES DEPARTMENT OF COMMERCE 

Office of Inspector General 

Washington, D.C. 20230 

April 5, 20 12 

MEMORANDUM FOR: 	 Robert M. Groves 


Director, U.S. Census Bureau 


FROM: 	 Ron Prevos~~~~j/~~'"'-- --- Assistant Inspector eneral for Economic 

and Statistical Program Assessment 

SUBJECT: 2020 Census Planning: Delays with 20 I 0 Census Research Studies 

May Adversely Impact the 2020 Decennial Census 

Final Report No. 0/G-/2-023-1 

We are providing our final report on the Census Bureau's ("the Bureau's") early 2020 Census 

planning efforts. Our review (a) evaluated the Bureau's efforts to improve the 2020 Census 

design, focusing on how the results of the Bureau's evaluation plans will inform the new 

decennial design, and (b) evaluated and monitored the implementation of the Bureau's 

management initiatives designed to enhance organizational change. 

We found that the assessments and other studies from the previous census cycle are behind 

schedule, raising concerns that complete results will not emerge in time to inform Census 2020 

research-or help the Bureau contain life cycle costs within its $12.8-18 billion (nominal 

dollars) estimate. We also highlighted the importance of developing a comprehensive risk 

management plan, which recognizes that the roles and responsibilities of the Department's and 

the Bureau's new risk management offices are evolving. Finally, we summarized the various 

organizational changes that will impact both the planning and the implementation of the next 

decennial census. 

We have received your response to our draft report. Where appropriate, we have modified 

this final report based on this response. The formal Bureau response is included as appendix D. 

(We summarized your response, and OIG comments, starting on page I 0.) The final report will 

be posted on the OIG's website pursuant to section 8l of the Inspector General Act of 1978, 

as amended. 

In accordance with the Department Administrative Order 213-5, please provide us with an 

action plan responding to all report recommendations within 60 days of the date of this 

memorandum. 

We thank you and your staff for the courtesies extended to us during this review. Please direct 

any inquiries regarding the report to me at (202) 482-3052 or Carol Rice at (202) 482-6020. 

Attachment 
cc: 	 Mark Doms, Chief Economist, Economics and Statistics Administration 

Frank Vitrano, Associate Director for the 2020 Census, U.S. Census Bureau 

Pam Moulder, Program Analyst, U.S. Census Bureau 

Adam Miller, Audit Liaison, U.S. Census Bureau 
CENSUS BUREAU 

2020 Census Planning: Delays with 2010 Census Research 

Studies May Adversely Impact the 2020 Decennial Census 

OIG-12-023-I 

WHAT WE FOUND 

The Census Bureau recognizes that it must fundamentally change its decennial design to 

improve the enumeration quality of the 2020 Census and contain life cycle costs within 

its preliminary estimates of $12.8–18 billion (nominal dollars). More specifically, the 

Bureau should reassess how it updates its address list, enumerates households, and 

utilizes field and IT infrastructure in this critical effort. The Census Program Evaluation 

and Experiments (CPEX) program provides lessons learned from the previous decennial 

that inform the next design. With the Bureau’s timely completion of 2010 CPEX studies 

in question, we found that: 

Delays In Completing 2010 Census Studies Could Adversely Impact the Bureau’s Efforts to 

Improve the Decennial Census Design. Our review of select early 2020 Census planning 

efforts found that the assessments and other studies from the previous census cycle 

lagged behind schedule. Bureau officials stated that delays occurred because of new 

processes, untimely funding, and project team challenges. We found that attrition, staff 

reassignments, and competition for resources may have also impacted the project 

schedules. By February 2012, the Bureau had released 14 reports—however, 

according to its revised schedule, 38 reports should have been released by December 

31, 2011. Delays in completing the 109 studies from the 2010 CPEX create a risk that 

results may not be available as inputs for the thirty-five 2020 Census research projects 

slated to begin during FY 2012. 

The 2020 Census Requires a Comprehensive Risk Management Plan. The 2020 Census 

Directorate is finalizing a risk management plan to ensure project delivery on time, 

within budget, and with the promised functionality. The Bureau is working to integrate 

this plan with its enterprise risk management program and the Department of 

Commerce’s risk management efforts, including reviews of IT and non-IT major 

programs. This should help the Bureau monitor 2020 Census IT development—and thus 

help the Bureau avoid issues, such as the cost overruns and problems developing 

requirements, encountered with the 2010 Census field data collection automation contract. 

Coordination with the Department on major IT and non-IT systems will be critical in helping 

to control costs and ensure that systems support mission requirements. 

WHAT WE RECOMMEND 

We have recommended to the director of the Census Bureau the following actions 

(which, since December 2011, the Bureau has already begun implementing): 

 Prioritize further the 2010 CPEX studies, and focus program resources, to ensure 

that the most critical studies affecting the cost and quality of the 2020 Census are 

completed; and 

 Improve the transparency of the 2010 CPEX program by posting study plans, 

expected publication dates for the 109 studies, and final reports online as soon as 

practicable so that stakeholders can review and monitor the Bureau’s progress in 

redesigning the 2020 Census. 

Report In Brief

 APRIL 5, 2012

Background 

The Census Bureau is directed by law 

to carry out the decennial census, 

which is mandated by the Constitution. Decennial census data are used 

to apportion seats in Congress; redraw congressional districts; and 

allocate hundreds of billions of dollars in federal funds for state and 

local governments, as well as other 

public and private sector purposes. 

For the 2020 Census, the Bureau has 

reorganized to improve its research 

and testing, as well as develop more 

cost-effective program processes and 

methods. Among the changes are a 

reorganization of its regional office 

structure, the creation of new directorates, and the launch of a new Office of Risk Management and Program 

Evaluation. Also, the Bureau is working to improve its cost estimation 

process for the next decennial census; it will update estimates as research and testing progress, so that 

budget requests align with decennial 

census requirements. 

Why We Did This Review 

As planning for the 2020 Census 

proceeds, these changes must contribute to the design research and 

testing needed to develop a costeffective and high quality decennial 

census. This decade, the OIG plans 

to monitor the 2020 Census 

throughout its life cycle to help the 

Bureau achieve its goal of containing 

the costs of the next decennial by 

recommending changes to promote a 

more cost-effective design. 

For this review, we sought to 

(1) evaluate the Bureau’s efforts to 

improve the 2020 Census design, 

focusing on how the results of the 

Bureau’s evaluation plans will inform 

the new decennial design, and (2) 

evaluate and monitor the implementation of the Bureau’s management 

initiatives designed to enhance organizational change. 

 


 


 


 


 


 


 


 


 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Contents 

Introduction ....................................................................................................................................................... 1
 

Findings and Recommendations .................................................................................................................... 4
 

I. Delays In Completing 2010 Census Studies Could Adversely Impact the Bureau’s 

Efforts to Improve the Decennial Census Design.............................................................................4
 

II. The 2020 Census Requires a Comprehensive Risk Management Plan ........................................7
 

Summary of Agency and OIG Comments................................................................................................ 10
 

Appendix A: Objectives, Scope, and Methodology ................................................................................ 12
 

Appendix B: Census Bureau Actions Taken in Preparation for the 2020 Census .......................... 13
 

Appendix C: Status of 2010 Census Studies ........................................................................................... 17
 

Appendix D: Agency Response .................................................................................................................. 23
 

FINAL REPORT NO. OIG-12-023-1 
   

   

   

—

   

—

   

     

   

   

   

   

—

   

   

—

   

   

—

   

   

   

   

—

   

—

—

   

—

   

   

   

     

   

—

   

   

—

   

   

—

   

   

           

   

   

   

   

   

   

  

   

   

      

   

Supplemental

Research and

Testing

Develop and

baseline

supplemental

research and testing

plan

Conduct

supplemental

research and testing

Begin critical

planning for the

2020 Census

Operational

Development and

Systems Testing

Develop and

baseline

operational

requirements

Conduct systems

development and

testing

Conduct

operational and

system tests

Establish field

infrastructure

Readiness Testing,

Execution, and

Closeout

Test systems

readiness

Update address

frame

Conduct

enumeration

activities

Execute Census Day

Deliver

apportionment

counts and

redistricting data

Research and

Testing

Conduct research

and testing and

field tests

Develop and

baseline systems

engineering

processes

Determine strategy

for major

acquisitions

Determine and

refine initial

operational designs

FYs 2012–2014 FYs 2015–2018 FYs 2019–2023

Options Analysis

Perform

operational design

and program

management

options analysis

—

Develop and

baseline IT

integration plan

—

Develop 2020

Census business

plan

—

Create a new

2020 Census

Directorate

FY 2012

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Introduction 

The Census Bureau (“the Bureau”) is directed by law to carry out the decennial census, which 

is mandated by the Constitution. Decennial census data are used to apportion seats in 

Congress; redraw congressional districts; and allocate hundreds of billions of dollars in federal 

funds for state and local governments, as well as other public and private sector purposes. Since 

1970, the design of the decennial census has involved mailing out and delivering questionnaires 

to households nationwide and asking households to return completed forms by mail. For those 

households that do not receive or return questionnaires, the Bureau sends out temporary 

workers to collect the information during field operations. 

The 2010 Census encompassed a fiscal year (FY) 2001–2013 life cycle that included planning, 

testing, and reporting of results. This effort—managed by the Decennial Management Division 

under the Decennial Census Directorate—involved 11 major field operations employing more 

than 800,000 temporary employees assigned to nearly 500 field offices. For the 2020 Census, 

the Bureau has planned the following life cycle, which follows a similar model divided into 

several phases with important deliverables. Figure 1 shows the different phases of the 2020 

Census life cycle and some major deliverables. 

Figure 1. 2020 Census Life Cycle and Corresponding Major Deliverables 

Source: U.S. Census Bureau information 

For the 2020 Census, the Bureau has undertaken a reorganization to improve its research and 

testing, foster innovation, and develop more cost-effective program processes and methods. 

FINAL REPORT NO. OIG-12-023-1 1 
                                                           

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Among the changes are a reorganization of its regional office structure and the creation of the 

2020 Census Directorate, the Research and Methodology Directorate, and the Office of Risk 

Management and Program Evaluation. Also, the Bureau is working to improve its cost 

estimation process for the next decennial census, generating rough order of magnitude (ROM) 

estimates for several potential designs. It will update estimates as research and testing progress, 

so that budget requests align with decennial census requirements (see section II for the role of 

the Bureau’s risk management office and appendix B for a summary of the remaining changes). 

As planning for the 2020 Census proceeds, these organizational changes must contribute to the 

design research and testing needed to develop a cost-effective and high quality decennial census. 

The final cost of the 2010 Census was more than $12 billion—nearly twice the cost of the 2000 

Census (nominal dollars)—due in part 

to a late-stage design change and Figure 2. Five Major Research Tracks 

higher-than-expected contractor costs. for the 2020 Census Design 

If the Bureau implements the changes 

currently under consideration, 

preliminary 2020 Census cost 

estimates rise to $12.8–18 billion 

(nominal dollars). However, if the 

design remains unchanged, the 2020 

decennial census could reach $30 

billion—based on the decennial’s 

history of doubling in cost on average 

every cycle, according to a recent 

GAO estimate.1 The Bureau has 

announced its goal of a 2020 Census 

costing less per housing unit (adjusted 

for inflation) than the 2010 Census, 

while maintaining quality. 

To produce a more efficient and 

effective decennial census—that 

contains costs, maintains or improves 

information quality,2

 and reduces 

program risks—the Bureau must 

make fundamental changes to its 

design, which would impact planning, development, and implementation. Planning for the 2020 

Census has begun and, over the next several years, staff will examine five different research 

tracks to formulate the 2020 Census design (see figure 2). Whatever design the Bureau 

chooses, it must allow sufficient time for testing, improvement, and implementation. 

1

 Government Accountability Office, December 2010. 2010 Census: Data Collection Operations Were Generally 

Completed as Planned, but Long-standing Challenges Suggest Need for Fundamental Reforms, GAO-11-193, Washington, 

DC: General Accountability Office, 27. 

2

 Final results from a post-enumeration survey that provides estimates of persons missed or erroneously counted 

are scheduled for release in 2012. These results will gauge the accuracy of the 2010 Census count. 

Source: U.S. Census Bureau 

FINAL REPORT NO. OIG-12-023-1 2 
U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

The period between FYs 2011 and 2014 will prove crucial for 2020 Census success. In previous 

decennial census cycles, Congress—and consequently the public—paid minimal attention to the 

early stages of planning and design formulation, holding one hearing during the first half of the 

decade prior to the 2010 Census. This decade, the OIG plans to monitor the 2020 Census 

throughout its life cycle to help the Bureau achieve its goal of containing the costs of the next 

decennial by recommending changes to promote a more cost-effective design. 

For this review, we sought to (1) evaluate the Bureau’s efforts to improve the 2020 Census 

design, focusing on how the results of the Bureau’s evaluation plans will inform the new 

decennial design, and (2) evaluate and monitor the implementation of the Bureau’s management 

initiatives designed to enhance organizational change. See appendix A for the complete 

objectives, scope, and methodology of this evaluation. 

FINAL REPORT NO. OIG-12-023-1 3 
     

           

           

         

     

         

     

       

           

           

         

       

         

         

     

 

 

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Findings and Recommendations 

The Census Bureau recognizes that it must fundamentally change the decennial census design to 

contain costs and improve enumeration quality. Rather than make marginal changes to a design 

used for the past 5 decennial census cycles, the Bureau needs to reassess how it updates its 

address list, enumerates households, and utilizes field and IT infrastructure to carry out this 

critical effort. In FY 2011, the Bureau began planning for the next decennial census. During our 

review of these efforts, we recommended management ensure that 2010 Census studies are 

completed on schedule and posted online quickly upon completion for stakeholders to review 

and monitor. These studies, carried out by the Census Program Evaluation and Experiments 

(CPEX) program, provide lessons learned from the previous decennial that inform the next 

design. After the 2000 Census, the Bureau, due in part to resource constraints, cancelled more 

than a quarter of the program evaluations—and, along with them, potential lessons learned 

needed to make changes to the 2010 Census. If the Bureau cannot complete the 2010 Census 

studies on time, their results may not provide input for 2020 Census research projects— 

jeopardizing the Bureau’s goal of developing a design that contains costs while maintaining data 

quality. 

I. Delays In Completing 2010 Census Studies Could Adversely Impact the 

Bureau’s Efforts to Improve the Decennial Census Design 

Our review of select early 2020 Census planning efforts 

found that the assessments and other studies from the 

previous census cycle lagged behind schedule. These 

provide data necessary for 2020 Census research 

projects that will help formulate the new design. 

Although major decennial census field operations ended 

in 2010, the Bureau continues work to complete the 

comprehensive 2010 CPEX—the assessments, quality 

profiles, evaluations, and experiments (see box at right) 

scheduled for release into FY 2013. However, tracking 

information provided by CPEX staff shows some 

studies have missed their original report release dates. 

This situation materialized after the 2000 Census— 

when the Bureau’s former Testing, Experimentation, 

and Evaluation Program cancelled 29 percent of its 

evaluations due, in part, to resource constraints. For 

example, the Bureau cancelled a 2000 Census 

evaluation on the use of U.S. Postal Service data to 

update its address database, as well as 4 of 7 

evaluations measuring how accurately the Bureau enumerated hard-to-count populations living 

in group housing such as dormitories and shelters. The inability to complete its 2010 reviews 

on time, or at all, could adversely impact the Bureau’s efforts to formulate a new 2020 

decennial census design and realize cost savings while maintaining or improving the accuracy of 

the census. 

Assessments document final volumes,

rates, and costs for individual operations

using data from production and quality

assurance files—as well as information

collected from debriefings.

Quality Profiles document the results of

quality control operations.

Evaluations analyze the effectiveness of

census components and their impact on

data quality and coverage, using data

collected from census operations and

systems.

Experiments are quantitative studies

(involving national surveys) during a

decennial census that informs future

decennial census planning.

FINAL REPORT NO. OIG-12-023-1 4 
                                                           

     

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Since 1950, the Bureau has incorporated a testing, evaluation, and experimental program to 

evaluate the current decennial census and facilitate planning for the next one—two activities 

that support efforts to improve the decennial census design. Teams of employees across the 

Bureau are working to complete 109 studies evaluating decennial census data for insight into 

improving its processes. The Bureau plans to issue reports on the 2010 Census studies on a 

rolling basis through April 2013.3 Figure 3 shows the number of reports completed or 

scheduled for completion (represented by columns, as measured by the left-most y-axis) and 

the cumulative totals (represented by lines, as measured by the right-most y-axis) by fiscal year. 

However, over the past few months, the release of these studies has been delayed, in some 

cases by several months. Of the 30 reports (13 of which the Bureau has designated as critical 

for 2020 Census research and testing) planned for release by the end of FY 2011, Census issued 

only 2. A third report—the Address Canvassing Operations Assessment, the one critical report 

of the three—was in final review at the end of FY 2011, 9 months after the scheduled 

December 2010 release date that was published in the official study plan. 

Figure 3. CPEX Study Release Schedule by Quarter, as of September 30, 2011 

0

15

30

45

60

0

5

10

15

20

FY11‐Q1FY11‐Q2FY11‐Q3FY11‐Q4FY12‐Q1FY12‐Q2FY12‐Q3FY12‐Q4FY13‐Q1FY13‐Q2FY13‐Q3FY13‐Q4

Number of CPEX Studies (Cumulative)

Number of CPEX Studies

Fiscal Year and Quarter

Non‐Critical Critical Cumulative Non‐Critical Cumulative Critical

Source: U.S. Census Bureau 

Bureau officials stated that delays occurred because the study development process was new, 

funding to conduct fieldwork was delayed, and project teams were unable to obtain the 

required information according to the project schedule. By reviewing study plans and 

interviewing staff, we found that attrition, staff reassignments, and competition for resources 

may have also impacted the project schedules. Since Census released the plans, 13 studies have 

3

 During our review, we were provided revised report release dates. Of the 10 reports scheduled to be released 

by the end of FY 2011 per the new schedule, only 3 were completed. We present the entire list of studies as 

appendix C. 

FINAL REPORT NO. OIG-12-023-1 5 
                                                           

         

             

     

               

          

               

         

           

           

          

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

lost at least one author and 11 others have had authors reassigned to new divisions. For 

example, the Address Canvassing Targeting and Cost Reduction Study, whose results are 

essential to a 2020 Census design that uses an efficient and effective address canvassing 

operation, lost an author to attrition. This left one full-time person to work on this critical 

evaluation. The 2010 Census Match Study, which analyzes the use of administrative records to 

help enumerate the population, has recently lost key staff. (See box, below, for brief 

explanations of these two studies.) Twenty study plans cited risk of delay because staff assigned 

to research, develop, and author the reports are also assigned to other critical Census 

operations that may take priority over study activities. 

The Address Canvassing Targeting Study will

assess the feasibility of a targeted address

canvassing operation—which could save

millions of dollars in nationwide field costs that,

in 2010, totaled $444 million.

The Census Match Study will analyze the use of

administrative records to help enumerate

households nationwide. This could reduce the

need for large nonresponse followup operations

requiring thousands of temporary workers

To help the studies catch up to their schedules, 

Bureau officials have devised a new briefing 

strategy to streamline the process. The lower-level 

Census Integration Group (CIG), which consists of 

the various managers responsible for 2010 

operations, is no longer briefed on study results. 

Instead, study briefings now occur during the 

report drafting stage with the high-level Executive 

Steering Committee for the 2010 Census 

Assessments and Research.4 This committee, which 

was chartered in April 2011, is tasked with (1) 

providing guidance on the results from research on 

and assessment of 2010 Census operations and (2) helping direct early 2020 Census life cycle 

testing. At the CIG meeting held on February 8, 2012, the Bureau reported 51 studies briefed 

and 14 reports released. Since, according to the revised schedule, 38 reports should have been 

released by December 31, 2011, stakeholders cannot yet determine whether this new process 

will allow the Bureau to meet the release dates for the remaining CPEX studies, particularly if 

projects continue to lack the necessary resources. Nonetheless, we recommend posting final 

reports online as soon as practicable for stakeholder review. 

Delays in completing 2010 Census studies create a risk that the results and information may 

not be available as inputs for the thirty-five 2020 Census research projects slated to begin 

during FY 2012. These research projects seek to improve major aspects—such as the address 

list, counting the population, response options and processing, data products, and 

infrastructure-related issues—as well as inform the redesign of the 2020 Census. Of particular 

importance are specific projects aimed at improving map and address accuracy with continual 

updating (to allow for a less-costly targeted address canvassing) and using administrative 

records (to reduce the size and scope of expensive field operations). With the Bureau 

expecting to have major design components in place starting in FY 2015, the timely completion 

of research projects becomes critical. 

The CPEX program—which cost $68 million for FYs 2009–2011 and, according to the Bureau, 

is budgeted for $11 million in FY 2012—faces significant challenges from the reporting delays 

4

 Members of this Executive Steering Committee include associate and division directors involved in decennial 

census planning. 

FINAL REPORT NO. OIG-12-023-1 6 
                                                           


 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

encountered to date.5 With 2020 research projects scheduled to have already started and with 

the prospect of constrained future budgets, the Bureau should further prioritize which 2010 

CPEX studies to complete. Among the 109 studies, Census management had identified 52 (7 of 

which had been released as of February 8) that would provide information critical to planning 

and implementing the thirty-five 2020 Census research projects. 

Following the FY 2012 appropriations, the Bureau revised its plans for the 2010 CPEX studies.6 

The Bureau eliminated 20 CPEX studies, 2 of which it deemed critical to informing the 2020 

Census research projects7

 and 10 of which were quality profiles designed to assess quality 

control processes for certain decennial operations. This reduced the number of critical studies 

to 50. Given the federal government’s current constrained fiscal environment, the Bureau could 

face funding challenges beyond FY 2012. To prepare for such a possibility, the Bureau should 

further prioritize which 2010 CPEX studies are most critical to 2020 Census planning efforts— 

and focus on providing program resources (including full-time subject matter experts) to them 

to ensure their completion. 

II. The 2020 Census Requires a Comprehensive Risk Management Plan 

During FY 2011, the Bureau was still developing risk management activities while it prepared to 

enter its research and testing phase. During our 2010 Census oversight, we noted that risk 

management activities began late in the life cycle. In our final 2010 Census report to Congress, 

we recommended that these activities commence from the outset of the life cycle. The 2020 

Census Directorate is finalizing a risk management plan to ensure project delivery on time, 

within budget, and with the promised functionality. The plan aims to avoid potential problems 

during the life cycle by helping staff identify project risks and develop strategies to mitigate 

them. Effective risk management will require the involvement of relevant stakeholders and 

oversight organizations, including the Bureau’s Office of Risk Management and Program 

Evaluation and the Department’s Office of Program Evaluation and Risk Management and 

Investment Review Board (IRB). 

OIG quarterly reports on the 2010 Census8

 identified significant problems in project planning 

(e.g., not employing critical path management or thoroughly reviewing project start and end 

dates up front) and risk management (e.g., starting such activities late in the decennial census life 

cycle and not completing contingency plans on time) that the Bureau needed to resolve to 

contain costs and maintain information quality for the 2020 Census. To accomplish this, we 

recommended that risk management activities begin from the outset of the current decennial 

census life cycle, rather than just before field operations (as happened for the 2010 effort). 

5

 For FY 2013, the Bureau has stated it requested slightly more than $7 million to fund the 2010 CPEX program. 

6

 The Consolidated and Further Continuing Appropriations Act of 2012 (Public Law 112-55) was signed into law 

on November 18, 2011. 

7

 The two critical cancelled studies are the (1) Sample Design Assessment and the (2) Respondent Debriefings of 

Person Interview and Person Followup Evaluation. These pertain to the Census Coverage Measurement program, 

which examines coverage errors from the 2010 Census to improve future censuses. 

8

OIG-19791-1, August 2009; OIG-19791-2, December 2009; OIG-19791-3, February 2010; OIG-19791-4, May 2010; 

OIG-19791-5, January 2011; and OIG-11-030-I, June 2011.
 

FINAL REPORT NO. OIG-12-023-1 7 
                                                           

           

           

             

               

           

               

             

     

     

         

           

  

         

       

   

     

           

 


 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Similarly, a recently issued National Research Council report9 on redesigning the 2020 Census 

encouraged planning for the next decennial to start early in the life cycle to ensure a more 

cost-effective design. To accomplish these goals, the Bureau needs to effectively monitor risk 

management activities. 

The 2020 Census Directorate recently 

developed a risk management plan modeled 

after the 2010 effort. The Bureau’s 2020 

Census risk management plan establishes 

processes to identify and analyze risks, 

populate a risk register,10 develop mitigation 

and contingency plans, assess mitigation 

activities, and periodically reassess the risks as 

needed. The Bureau’s draft business plan 

identifies several risks for the research and 

testing phase of the decennial life cycle (see 

box, opposite). Census management is 

currently developing an enterprise risk 

management program to complement and 

oversee individual plans developed by Bureau 

units. 

In addition to planning documents, Census staff 

is exploring best practices for research and 

testing. For example, it cohosted a June 2011 

seminar to learn about successful government 

system engineering projects from three major 

consulting firms. Through these concepts and 

case studies, the Bureau aimed to learn how to 

use systems engineering11 for planning and 

developing the 2020 Census. 

Select 2020 Census Program Risks Identified

During the Research and Testing Phase

 Alignment of acquisition policy with 2020 Census

needs

 Gaps and weakness in the research and testing

plan

 Insufficient funding for research and testing

 Lack of Census manager support of the planning

approach

 Lack of external stakeholder support for 2020

Census design alternatives

 Mandatory cost constraints

 Premature life cycle cost estimate

 Resistance to improved program management and

accountability

 Shortage of necessary skills—operational and

technical subject matter, systems engineering,

program/project management, acquisition

 Unanticipated mandatory change

Source: OIG, based on U.S. Census Bureau data

The Bureau is working to integrate its 2020 Census risk management plan with its enterprise 

risk management program and the Department of Commerce’s risk management efforts, 

including reviews of IT and non-IT major programs. Effective monitoring of decennial risk 

management activities will require coordination with several Bureau and Departmental 

oversight bodies. During the previous decennial census, the Bureau’s Risk Review Board, 

composed of senior Census managers involved with the decennial census, met weekly to 

oversee risk management activities during field operations. Census managers informed us they 

9

 National Research Council. 2011. Change and the 2020 Census: Not Whether But How. Washington, DC: The 

National Academies Press.
 

10 A risk register contains the results of a project’s risk analysis and includes risk descriptions, probability of 

occurrence, impact on objectives, proposed responses, owners, and current status. 

11 Systems engineering is an interdisciplinary approach that focuses on defining customer needs and required 

functionality early in the development cycle, documenting requirements, and then proceeding with design synthesis 

and system validation—with the goal of providing a quality product that meets the user needs. 

FINAL REPORT NO. OIG-12-023-1 8 
                                                           

	 

	 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

will again use the Board for the next decennial. Additionally, the Bureau’s newly-created Office 

of Risk Management and Program Evaluation (which reports to the deputy director) will 

oversee Bureau program risks. Their standardized risk management provides a consistent 

methodology to apply to all programs and operating units. One task will be to monitor 2020 

Census IT development—which may help the Bureau avoid issues, such as the cost overruns 

and problems developing requirements, encountered with the 2010 Census field data collection 

automation contract. Independent of the Bureau directorates, the risk management office can 

help identify common risks across directorates and facilitate communication among them to 

resolve problems quickly. 

Finally, Census will need collaboration between the Department’s Office of Program Evaluation 

and Risk Management and the IRB. The Department’s risk management office will serve an 

important role as the Department’s authority on enterprise and programmatic risk matters— 

including implementing enterprise-wide policies and procedures, identifying and applying best 

practices, and advising Department and Bureau management on programmatic risks. Based on 

our interview with staff, we concluded that the Department’s risk management office had yet to 

finalize its oversight role with the 2020 Decennial Census program. The IRB’s role in advising 

the Secretary and Deputy Secretary on major IT and non-IT systems will be critical in helping to 

control costs and ensure that systems support mission requirements. Any significant delay in 

the development of these relationships presents a risk to the Bureau in its efforts to plan for 

the 2020 Census. 

Recommendations12 

We recommend the director of the Census Bureau take the following actions: 

	 Prioritize further the 2010 CPEX studies, and focus program resources, to ensure that 

the most critical studies affecting the cost and quality of the 2020 Census are 

completed; and 

	 Improve the transparency of the 2010 CPEX program by posting study plans, expected 

publication dates for the 109 studies, and final reports online as soon as practicable so 

that stakeholders can review and monitor the Bureau’s progress in redesigning the 2020 

Census. 

12 We do not offer any new recommendations to address finding II, which concerns a comprehensive risk 

management plan. Currently, the Bureau is addressing an OIG recommendation—from our final quarterly report 

to Congress—to “[i]mplement improved project planning and management techniques early in the decade.” 

Among management priorities are plans to address risk management activities for the 2020 Census (see table B-1 

in appendix B of this report). 

FINAL REPORT NO. OIG-12-023-1 9 
                                                           

	 

	 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Summary of Agency and OIG Comments 

In its March 21, 2012, response to our draft report, the Bureau reported it had already taken 

steps to address our recommendations. The Bureau has made progress implementing our 

recommendations following our December 20, 2011, exit conference. Regarding the first 

recommendation, the Bureau has completed a series of internal reviews to ensure the most 

critical studies are completed. In addition, the Bureau stated that it monitors the progress of 

reports through exchanges with authors, managers, and stakeholders—as well as through the 

use of tracking tools to ensure allocating appropriate resources for their completion. In 

response to our second recommendation, the Bureau has created a website to disseminate the 

CPEX reports to the public. The Bureau’s response further stated it had posted online 21 

reports. 

In addition to responding to our recommendations, the Bureau provided technical comments 

to our draft report. After consideration of the comments, we modified our final report to 

reflect appropriate changes. The Bureau also suggested modifying our report to reflect the 

updated number of studies in its current schedule. However, we retained the numbers in our 

draft report that were listed in the original CPEX studies schedule—which the Bureau provided 

to us in October 2011—while later changes provided by the Bureau reflect their subsequent 

prioritization. (For further details about the Bureau’s changes, find the Bureau’s response in 

appendix D.) We outline below specific changes involving the number of studies and those 

designated critical: 

	 We reported 109 studies in our draft report based on a list of studies the Bureau 

provided to us in October 2011 and a list of 20 canceled studies contained in an internal 

memorandum dated February 2012.13 After examining its response (and through a 

follow-up interview with staff), we determined that—since October 2011—the Bureau 

split 1study into 12 individual studies, effectively adding 11 new studies. Adding those 11 

studies to the remaining 89 studies results in 100 current studies, as reported by the 

Bureau. 

	 On page 7, we reported that 52 CPEX studies designated as critical were reduced to 50 

when 2 studies were canceled. However, the Bureau’s response to our draft report 

states that 53 critical studies now exist. In addition to the 2 canceled studies, the 

Bureau redesignated 2 critical studies as noncritical14 and named 5 others critical. 

13 See 2010 Decennial Census Program Decision Memorandum Series No. 38—Decision to Reduce Scope of the 2010 

Census Program for Evaluations and Experiments (February 14, 2012). 

14 The studies no longer considered critical are (1) Missing Housing Unit Assessment (renamed “Evaluation to 

Assess Effect of Census Coverage Measurement [CCM] Search Area and Census Address List Formation Rules on 

CCM Estimates”) and (2) Behavior Coding of the 2010 Nonresponse Followup Interviews Evaluation. 

FINAL REPORT NO. OIG-12-023-1 10 
	 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

	 On page 7, we stated that the Bureau released 7 critical reports as of February 8, 2012. 

In the Bureau’s response they report the release of 9. This discrepancy results from the 

definition of the release date. We define “release date” as the date of the signed 

memorandum that formally releases the report for distribution. As such, the Bureau had 

released only 7 critical reports by the aforementioned date. 

FINAL REPORT NO. OIG-12-023-1 11 
	 

	 

	 

	 

	 

	 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Appendix A: Objectives, Scope, and 

Methodology 

To conduct this review, we sought to (1) evaluate the Census Bureau’s efforts to improve the 

2020 Census design, focusing on how the results of the Bureau’s evaluation plans will inform 

the new decennial design; and, (2) evaluate and monitor the implementation of the Bureau’s 

management initiatives designed to enhance organizational change. 

Our methodology included interviewing senior Bureau managers and staff and reviewing 

planning documentation for the 2020 Census. Specifically, we: 

	 Interviewed Bureau staff to (1) understand their roles in support of the 2020 Census 

and (2) monitor development of the decennial census design; 

	 Interviewed Bureau regional directors to obtain (1) their feedback on the 2010 Census 

and (2) their suggestions for improving the 2020 Census; 

	 Interviewed outside stakeholders to understand their concerns and receive suggestions 

for improving the 2020 Census design; 

	 Reviewed the Bureau’s draft 2020 Census business plan and rough order of magnitude 

for long-term cost estimates; 

	 Assessed the study plans for 2010 Census assessments, evaluations, experiments, and 

quality profiles and research plans for the 2020 Census; and, 

	 Evaluated Bureau organizational changes and new management initiatives put in place in 

preparation for the 2020 Census. 

We conducted this review from February through October 2011, under the authorities of the 

Inspector General Act of 1978, as amended, and Departmental Organization Order 10-13, 

dated August 31, 2006, as amended. The review was conducted in accordance with the Quality 

Standards for Inspection and Evaluation (January 2011) issued by the Council of the Inspectors 

General on Integrity and Efficiency. 

FINAL REPORT NO. OIG-12-023-1 12 
                                                           

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Appendix B: Census Bureau Actions Taken 

in Preparation for the 2020 Census 

To effect changes in support of a more cost-effective decennial census in 2020 that improves 

information quality, the Census Bureau has begun implementing several important 

organizational and process changes. In addition, it has taken initial steps to address major 

challenges to the 2020 Census that we presented in our culminating sixth and final quarterly 

report to Congress.15 Below is a summary of the organizational and process changes and the 

Bureau’s actions to address the 2020 Census challenges from that report. 

Reorganization of the Regional Office Structure. In June 2011, the Bureau announced a 

significant change to its regional office structure, the first such change since 1961. Expected to 

be completed by January 1, 2013, the number of offices that manage nationwide surveys using 

thousands of permanent workers (field representatives) will decrease from 12 to 6 (see figure 

B-1). 

Figure B-1. Census Regions, Effective January 1, 2013 

Source: U.S. Census Bureau 

This reorganization aims to lower costs, improve efficiency, and increase responsiveness. The 

Bureau utilized objective criteria to select which offices to close and expects to save $15–18 

15 U.S. Department of Commerce Office of Inspector General, June 2011. Census 2010: Final Report to Congress, 

OIG-11-030-I. Washington, DC: Department of Commerce OIG. 

FINAL REPORT NO. OIG-12-023-1 13 
U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

million annually, starting in FY 2014, with few staff reductions. The Bureau will continue the 

current practice of providing administrative functions (e.g., payroll, human resources) at each 

office. The Bureau intends to increase regional office participation and leverage field office 

infrastructure to support planning efforts for the 2020 Census. With the new 2020 Census 

design still under development, it is unknown how these changes in the field will impact the 

Bureau’s goal of containing the cost of the decennial census while preserving data quality. 

2020 Census Directorate. In late 2010, Bureau management proposed a plan to separate the 

tasks of handling remaining 2010 Census operations and planning for the 2020 Census. The 

2020 Census Directorate fulfills the Bureau’s goal of redesigning the decennial census by 

allowing it to focus on planning for the 2020 Census without responsibility for finalizing the 

2010 Census. This directorate is composed of the associate director, the Office of 2020 Census 

Planning, and the American Community Survey (ACS) office. The current associate director is 

the former head of the decennial management division who oversaw the 2010 Census as field 

operations were carried out. The 2020 Census planning office is tasked with planning and 

implementing program management processes for research and testing and with evaluating 

proposed designs. The ACS office will continue to carry out its mandatory, continuous survey 

(which, after 2000, replaced the decennial census long form containing additional population 

and housing questions) but also be available to support research and planning for the next 

decennial census. Specifically, the Bureau plans to develop IT, field, and operational 

infrastructure that can support the ACS and adapt to the much larger 2020 Census. The ACS 

will serve as a test bed for carrying out tests, such as new survey methods, for the proposed 

decennial census design using existing permanent staff located throughout the country. ACS 

field representatives will test out new hardware platforms and software functionalities as they 

conduct their regular monthly surveys. 

Research and Methodology Directorate. The new Research and Methodology Directorate 

is expected to support mission-critical research for the 2020 Census and collaborate with 

program divisions throughout the life cycle. The new directorate strengthens the former 

Methodology and Standards Directorate by tasking it with functions critical to redesigning the 

2020 Census. This directorate will support teams working on 2020 research projects as 

needed. Headed by the associate director, this unit is divided into 5 centers: 

 Statistical Research and Methodology 

 Economic Studies 

 Administrative Records Research and Applications 

 Disclosure Avoidance and Research 

 Survey Measurement 

The Research and Methodology Directorate is expected to work closely with the 2020 Census 

Directorate in areas such as administrative records and survey measurement. 

New Effort at 2020 Census Cost Estimation. The Bureau is working to improve its cost 

estimate for the proposed decennial census design. Since 1970, the cost of the decennial census 

has increased significantly with each successive cycle, despite the Bureau utilizing virtually the 

FINAL REPORT NO. OIG-12-023-1 14 
 

                                                           

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

same design to collect information. For 2010, the cost was more than $12 billion,16 which was 

more than the original life cycle estimate of $11.7 billion made in 2001 but less than the $14.7 

billion estimate made in 2008 following the announcement of a return to a paper-based 

Nonresponse Followup operation. With the final 2020 Census design unknown, the Bureau has 

generated rough order of magnitude (ROM) cost estimates for four preliminary design options 

spanning the entire life cycle (FYs 2012–2023) based on inputs from the 2010 Census (such as 

enumerator training and salaries, operation workloads, and office space costs). The most 

expensive option, which makes few changes to the 2010 Census design, could cost up to $18 

billion—while the least expensive option, which utilizes targeted address canvassing, greater use 

of administrative records to help enumerate the population, and fewer field offices than in 2010, 

carries a $12.8 billion estimate. The costs of the remaining two designs fall in between these 

figures.17 The preliminary costs of these designs are in contrast to GAO’s independent estimate 

of up to $30 billion based on doubling the cost on average of the previous decennial census 

design. 

Bureau staff generated many thousands of simulations for each design option to increase the 

accuracy of its cost estimates. The Bureau plans on updating these estimates on an annual basis 

to take into account changes made during the research and testing phase and avoid having one 

static estimate, as developed for the 2010 Census, that certainly will change over time. Per its 

decennial business plan, the Bureau will have an initial operational design by FY 2015 based on 

initial research and testing of the five research tracks through FY 2014. From then on, the 

budget for 2020 Census should provide more accurate numbers for the finalized design and 

associated costs, particularly for administrative records and targeted address canvassing. 

Status of Challenges to the 2020 Census. For our June 2011 final quarterly report to 

Congress on the status of 2010 Census operations, we enumerated 7 major challenges to the 

2020 Census that the Bureau must address to ensure a cost-effective and quality decennial 

census. During this review, we found that the Bureau has taken initial steps to address some of 

these challenges with specific actions as it continues planning for the next decennial census 

(summarized in table B-1, next page): 

16 According to the Bureau, this represented a savings of $1.9 billion due to efficiencies gained during field 

operations and not spending approximately $800 million in contingency funds for emergencies that did not 

materialize. 

17 There were two other potential designs for which they did not generate ROM estimates, because the designs 

rely heavily or nearly exclusively on the use of administrative records and include no address canvassing. Testing 

involving these two areas remains ongoing and the extent of their potential use is unknown. 

FINAL REPORT NO. OIG-12-023-1 15 
 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Table B-1. Summary of Bureau Actions to Address OIG’s 2020 Census Challenges 

Challenges to the 2020 Censusa Actions Taken by the Census Bureau in FY 2011 

Revamp cost estimation and budget processes to 

increase accuracy, flexibility, and transparency 

The Bureau has generated ROM estimates for budget years 

FY 2012–2014 that encompass the research and testing 

phase. Census Bureau staff stated that they will update 

budget estimates annually to inform the annual budget 

process. 

Use the Internet and administrative records to 

contain costs and improve accuracy 

Bureau staff plan to incorporate these two activities as part 

of its research and testing projects to redesign the 2020 

Census. The Bureau expects to implement its final 

decennial census design in FY 2015 that may utilize the 

Internet and administrative records. 

Implement a more effective decennial test program 

using the ACS as a test environment 

Bureau managers plan on utilizing resources from the ACS 

to test new enumeration methods, hardware and software 

products, and conduct smaller field tests in support of the 

2020 Census redesign effort. Specific plans have not yet 

been finalized. 

Effectively automate field data collection 

As part of its 2020 research and testing program, the 

Bureau plans on researching how to increase the efficiency 

of its field infrastructure, exploring ways to automate the 

collection and processing of information during field 

operations. 

Avoid a massive end-of-decade field operation 

through continuous updating of address lists and 

maps 

As part of its planning process, the Bureau has proposed 

several decennial census designs, some of which include a 

targeted address canvassing operation to supplement 

continuous map and address updating. The decision on the 

extent to which targeted address canvassing will be used as 

part of the final decennial census design will be made after 

initial research and testing concludes in FY 2014. 

Implement improved project planning and 

management techniques early in the decade 

The Bureau formulated a draft business plan to support its 

FY 2012 budget initiative that provides a road map on FYs 

2012–2014 planning activities. Management has begun to 

formulate plans to address project and risk management 

activities for the 2020 Census. 

Source: OIG analysis of U.S. Census Bureau data a The challenge to create a director position that spans Presidential administrations requires Congressional action 

and cannot be addressed solely by the Census Bureau. 

FINAL REPORT NO. OIG-12-023-1 16 
  

       

  

     

   

   

   

   

   

   

                     

                     

           

   

   

             

   

   

               

   

   

             

   

   

   

               

   

   

   

   

   

   

   

             

   

   

   

             

     

   

   

   

                   

   

   

             

   

   

                

   

   

               

       

   

   

   

   

               

     

   

   

   

                 

   

   

             

           

     

   

   

   

                 

   

   

   

 

 

 

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Appendix C: Status of 2010 Census Studies 

STUDY TITLE

(critical studies highlighted in yellow)

Original

Release

Date

Status,

End of

FY 2011 Based

on Original

Release Date

Revised

Release

Date

Current

Status, Based

on Revised

Release Date

Census Coverage Measurement

CCM Independent Listing Quality Profile FY 11

(4Q)

FY 11

(3Q) FY 11 (3Q)

Demographic Analysis Operation Assessment FY 11

(4Q) OVERDUE FY 11(4Q) FY 12 (1Q)

CCM Sample Design Assessment FY 12

(1Q)

FY 12

(2Q) CANCELLED

CCM Person Interview Quality Profile FY 12

(3Q)

FY 12

(3Q) CANCELLED

CCM Initial Housing Unit Operations Assessment FY 12

(4Q)

FY 11

(4Q) OVERDUE

CCM Person Interview Operations Assessment FY 12

(4Q)

FY 12

(2Q)

CCM Final Housing Unit Operations Assessment FY 12

(4Q)

FY 12

(4Q)

CCM Person Matching and Followup Operations

Assessment

FY 12

(4Q)

FY 12

(4Q)

Recall Bias Panel Plan Evaluation FY 12

(4Q)

FY 12

(4Q)

CCM Respondent Debriefings of Person Interview and

Person Followup Evaluation

FY 13

(1Q)

FY 12

(1Q) CANCELLED

CCM Initial Housing Unit Clerical Matching Quality Profile FY 13

(2Q)

FY 12

(3Q) CANCELLED

CCM Person Followup Quality Profile FY 13

(1Q)

FY 12

(3Q) CANCELLED

CCM Person Clerical Matching Quality Profile FY 13

(1Q)

FY 12

(4Q) CANCELLED

Investigation of Methods to Evaluate Coverage of the

Group Quarters Population Evaluation

FY 13

(1Q)

FY 13

(1Q)

Developing an Error Structure in Components of Census

Coverage Error Evaluation

FY 13

(1Q)

FY 13

(1Q) CANCELLED

CCM Initial Housing Unit Followup Quality Profile FY 13

(2Q)

FY 11

(4Q) OVERDUE

Explaining How Census Errors Occur through Comparing

Census Operations History with Census Coverage

Measurement Results Evaluation

FY 13

(2Q)

FY 12

(2Q) CANCELLED

CCM Final Housing Unit Followup Quality Profile FY 13

(3Q)

FY 12

(4Q) CANCELLED

FINAL REPORT NO. OIG-12-023-1 17 
  

       

  

     

   

   

   

   

   

             

   

             

   

           

   

   

   

               

   

   

   

           

   

   

   

               

   

   

   

                 

   

   

   

           

   

   

   

         

   

   

   

               

   

   

   

               

   

   

   

           

   

   

   

   

   

     

       

   

   

   

   

           

   

   

   

           

   

           

   

   

  

               

   

               

   

                 

               

               

   

 

 

 

 

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

STUDY TITLE

(critical studies highlighted in yellow)

Original

Release

Date

Status,

End of

FY 2011 Based

on Original

Release Date

Revised

Release

Date

Current

Status, Based

on Revised

Release Date

Coverage Improvement

Non‐ID Processing Assessment FY 11

(3Q) OVERDUE FY 13(1Q)

Count Review Assessment FY 11

(4Q) OVERDUE FY 12(2Q)

Small Multiunit Structures Evaluation FY 12

(1Q)

FY 12

(2Q)

Alternative Coverage Followup Questions and Design

Evaluation

FY 12

(1Q)

FY 12

(3Q)

LUCA Participant Survey Evaluation FY 12

(1Q)

FY 12

(2Q)

Address Canvassing Targeting and Cost Reduction

Evaluation

FY 12

(2Q)

FY 12

(2Q)

Data‐based Extraction Processes for the Address Frame

Evaluation

FY 12

(2Q)

FY 12

(2Q)

Effectiveness of Unduplication Evaluation FY 12

(4Q)

FY 12

(4Q)

Avoid Followup Evaluation FY 12

(4Q)

FY 13

(1Q)

Administrative Records Use for Coverage Problems

Evaluation

FY 12

(4Q)

FY 13

(1Q)

Address Frame Accuracy and Quality Evaluation FY 13

(1Q)

FY 13

(1Q)

Address List Maintenance Using Supplemental Data

Sources Evaluation

FY 13

(1Q)

FY 12

(3Q)

Comparative Ethnographic Studies of Enumeration

Methods and Coverage Evaluation

FY 13

(1Q)

FY 13

(1Q)

Census Match Study Evaluation FY 13

(1Q)

FY 12

(2Q)

Matching Error Study Evaluation Unavail‐

able

FY 13

(1Q) CANCELLED

Missing Housing Unit Assessment Unavail‐

able

FY 13

(1Q)

Field Operations

Address Canvassing Operations Assessment FY 11

(1Q) OVERDUE FY 12(1Q)

Global Positioning System Evaluation FY 11

(2Q) OVERDUE FY 12(4Q)

Group Quarters Validation Quality Profile FY 11

(3Q) OVERDUE FY 11(1Q)

CANCELLED

OVERDUE

Address Canvassing Quality Profile FY 11

(3Q) OVERDUE FY 11(4Q) OVERDUE

Group Quarters Validation Assessment FY 11

(3Q) OVERDUE FY 12(1Q)

FINAL REPORT NO. OIG-12-023-1 18 
  

       

  

     

   

   

   

   

   

                 

   

               

   

             

   

               

   

             

   

             

   

             

   

       

   

   

   

         

   

   

   

           

   

   

   

               

   

   

   

       

   

   

   

   

   

             

   

   

   

             

   

   

           

   

   

   

           

   

   

   

               

   

   

   

           

   

   

   

         

   

   

   

           

   

   

                   

   

   

   

           

   

   

       

   

   

 

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

STUDY TITLE

(critical studies highlighted in yellow)

Original

Release

Date

Status,

End of

FY 2011 Based

on Original

Release Date

Revised

Release

Date

Current

Status, Based

on Revised

Release Date

Enumeration at Transitory Locations Assessment FY 11

(3Q) OVERDUE FY 12(3Q)

Nonresponse Followup Operations Assessment FY 11

(3Q) OVERDUE FY 12(1Q)

Field Verification Assessment FY 11

(4Q) OVERDUE FY 12(1Q)

Group Quarters Enumeration Assessment FY 11

(4Q) OVERDUE FY 12(3Q)

Shipboard Enumeration Operation Assessment FY 11

(4Q) OVERDUE FY 12(3Q)

Island Areas Assessment FY 11

(4Q) OVERDUE FY 12(1Q)

Telephone Questionnaire Assistance Assessment FY 11

(4Q) OVERDUE FY 12(1Q)

Update/Leave Assessment FY 12

(1Q)

FY 12

(3Q)

Coverage Followup Assessment FY 12

(1Q)

FY 12

(2Q)

Be Counted/Questionnaire Assistance Center Assessment FY 12

(1Q)

FY 12

(1Q)

Federally Affiliated Overseas Count Operation Assessment FY 12

(1Q)

FY 12

(2Q)

Observing Census Enumeration of Non‐English‐Speaking

Households Evaluation

FY 12

(1Q)

FY 12

(3Q)

Nonresponse Followup Contact Strategy Experiment FY 12

(2Q)

FY 12

(1Q)

Group Quarters Enumeration Quality Profile FY 12

(2Q)

FY 12

(2Q) CANCELLED

Update Enumerate Operations Assessment FY 12

(2Q)

FY 12

(3Q)

Nonresponse Followup Quality Profile FY 12

(3Q)

FY 12

(1Q)

Enumeration of Transitory Locations Quality Profile FY 12

(3Q)

FY 12

(2Q)

Update Enumerate Quality Profile FY 12

(3Q)

FY 12

(2Q)

Update/Leave Quality Profile FY 12

(3Q)

FY 12

(2Q)

Field Verification Quality Profile FY 12

(3Q)

FY 12

(3Q) CANCELLED

Automation in Field Data Collection for Address Canvassing

Evaluation

FY 12

(3Q)

FY 12

(2Q)

Island Areas Quality Profile FY 12

(4Q)

FY 12

(3Q) CANCELLED

Military Enumeration Assessment FY 13

(1Q)

FY 13

(1Q) CANCELLED

FINAL REPORT NO. OIG-12-023-1 19 
  

       

  

     

   

   

   

   

   

       

   

   

   

         

   

   

   

   

                         

               

   

           

                 

   

                 

   

             

   

                   

   

               

   

             

   

           

      

   

   

   

   

           

   

   

   

           

   

   

           

   

   

           

   

   

   

               

   

   

   

               

   

   

   

             

   

   

   

           

   

   

           

   

   

   

            

   

   

   

 

 

 

 

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

STUDY TITLE

(critical studies highlighted in yellow)

Original

Release

Date

Status,

End of

FY 2011 Based

on Original

Release Date

Revised

Release

Date

Current

Status, Based

on Revised

Release Date

Service‐Based Enumeration Assessment FY 13

(1Q)

FY 13

(1Q)

Puerto Rico Assessment FY 12

(4Q)

FY 13

(1Q)

Field and Headquarters Infrastructure

Recruiting and Hiring Field Staff Assessment FY 11

(2Q) OVERDUE FY 12(1Q) FY 12 (1Q)

Cost and Progress Assessment FY 11

(2Q) OVERDUE FY 12(1Q)

Logistics Assessment FY 11

(3Q) OVERDUE FY 11(4Q) OVERDUE

Forms Printing and Distribution Assessment FY 11

(3Q) OVERDUE FY 12(1Q)

ELCO/LCO IT Equipment Deployment and De‐installation

Assessment

FY 11

(4Q) OVERDUE FY 12(1Q)

Mail Response/Return Rates Assessment FY 11

(4Q) OVERDUE FY 12(1Q)

DRIS Paper Questionnaire Data Capture Assessment FY 11

(4Q) OVERDUE FY 12(1Q)

RCC/LCO Space Acquisition Assessment FY 11

(4Q) OVERDUE FY 12(1Q)

Web Discovery Evaluation FY 11

(4Q) OVERDUE FY 12(3Q)

Universe Control and Management and Response

Processing System Assessment

FY 12

(1Q)

FY 13

(1Q)

Program Management Process: Risk Assessment FY 12

(1Q)

FY 12

(4Q)

Program Management Process: Schedule Assessment FY 12

(1Q)

FY 12

(4Q) CANCELLED

Program Management Process: Change Control

Assessment

FY 12

(1Q)

FY 12

(1Q) CANCELLED

Program Management Process: Issues Assessment FY 12

(1Q)

FY 12

(2Q)

Field Office Administration and Payroll Assessment FY 12

(2Q)

FY 12

(1Q)

Decennial Applicant, Personnel, and Payroll System

Assessment

FY 12

(2Q)

FY 12

(2Q)

2010 Census Postal Tracking Assessment FY 12

(2Q)

FY 12

(1Q)

Census Unedited File Assessment FY 12

(3Q)

FY 12

(3Q) CANCELLED

Late Adds Mailout Assessment FY 13

(1Q)

FY 12

(2Q)

Personally Identifiable Information Assessment FY 12

(1Q)

FY 12

(2Q)

FINAL REPORT NO. OIG-12-023-1 20 
  

       

  

     

   

   

   

   

   

   

             

   

             

               

   

                 

         

   

   

               

   

   

   

         

   

   

   

           

   

   

   

             

   

   

   

         

   

   

   

                   

   

   

   

             

   

   

   

           

   

   

   

   

             

   

   

             

       

   

   

   

   

           

   

                 

               

   

   

   

           

   

   

   

         

   

   

   

                 

   

   

   

         

   

   

   

 

 

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

STUDY TITLE

(critical studies highlighted in yellow)

Original

Release

Date

Status,

End of

FY 2011 Based

on Original

Release Date

Revised

Release

Date

Current

Status, Based

on Revised

Release Date

Questionnaire Content

Behavior Coding of the 2010 Nonresponse Followup

Interviews Evaluation

FY 11

(3Q) OVERDUE FY 11(4Q) FY 11 (4Q)

Item Nonresponse and Imputation Assessment FY 11

(4Q) OVERDUE FY 12(1Q)

Bilingual Questionnaire Assessment FY 11

(4Q) OVERDUE FY 12(1Q) FY 11 (4Q)

Confidentiality Notification Experiment FY 12

(1Q)

FY 11

(3Q) OVERDUE

Content and Forms Design Program Assessment FY 12

(1Q)

FY 12

(1Q)

Language Program Assessment FY 12

(1Q)

FY 12

(1Q)

Census Quality Survey Evaluation FY 12

(1Q)

FY 12

(2Q)

AQE—Census 2000 Form Replication Experiment FY 12

(2Q)

FY 12

(2Q)

Content Reinterview Evaluation FY 12

(2Q)

FY 12

(1Q)

Behavior Coding of the 2010 Coverage Followup Interviews

Evaluation

FY 12

(2Q)

FY 12

(2Q)

AQE—Race and Hispanic Origin Alternative Questionnaire

Experiment

FY 12

(3Q)

FY 12

(4Q)

2010 Census and American Community Survey

Comparability Evaluation

FY 12

(3Q)

FY 13

(1Q) CANCELLED

DRIS Data Capture QA and Paper Data Quality ‐ Quality

Profile

FY 13

(1Q)

FY 12

(4Q) CANCELLED

Behavior Coding of the 2010 Census Coverage

Measurement Person Interview Evaluation

FY 13

(1Q)

FY 13

(1Q)

Communications

National Partnerships Evaluation FY 11

(4Q) OVERDUE FY 13(1Q)

Census In Schools Research Evaluation FY 11

(4Q) OVERDUE FY 11(3Q) OVERDUE

Deadline Messaging and Compressed Mailing Schedule

Experiment

FY 12

(1Q)

FY 12

(1Q)

Paid Advertising Heavy‐Up Experiment FY 12

(1Q)

FY 12

(4Q)

Integrated Communications Program Assessment FY 12

(1Q)

FY 13

(1Q)

Census Barriers, Attitudes, and Motivators Survey II

Evaluation

FY 12

(1Q)

FY 12

(2Q)

Integrated Communications Program Evaluation Unavail‐

able

FY 12

(1Q)

Geographic Programs

FINAL REPORT NO. OIG-12-023-1 21 
  

       

  

     

   

   

   

   

   

                   

   

         

         

   

   

   

   

               

   

   

   

       

   

   

   

  

       

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

STUDY TITLE

(critical studies highlighted in yellow)

Original

Release

Date

Status,

End of

FY 2011 Based

on Original

Release Date

Revised

Release

Date

Current

Status, Based

on Revised

Release Date

Type of Enumeration Area Delineation Assessment FY 11

(1Q) OVERDUE FY 12(1Q)

Service‐Based Enumeration, Group Homes, and Carnival

Locations Address List Update Assessment

FY 12

(2Q)

FY 12

(3Q)

Local Update of Censuses Addresses Assessment FY 12

(3Q)

FY 12

(3Q)

New Construction Assessment FY 12

(3Q)

FY 12

(4Q)

Reports Overdue Based on: Original

Date 30 RevisedDate 7

Source: OIG analysis of U.S. Census Bureau information 

FINAL REPORT NO. OIG-12-023-1 22 

 

MEMORANDUM FOR /\on Eilers 

Principal Assistant lmpector General for Audil and ~valuation 

~ Through: Rebecca 

Under Sec

M. 

retary 

Blankfor ~ 

Econo1nic AlTa irs 

From: Robert M. Groves f) I rl- W1 /.. 

Director /~ fl { ~ 

Subject: Draft Report: 2020 Census Plnuning: Delays wirh 2010 Census 

Research Studias May Adversely lmpact the 2(120 Decennial 

Census 

The attached comments are in response to your draft audit report, 2020 Census !'/arming; 

Delays with 2010 Census Research Studies May Adversely Impact the 2020 Decennial Census. 

The U.S. Censu.~ Bureau appreciates the comments and recommendations developed by the 

Office of the Principal Assistant inspector General in producing this audit report. 

Attachment 

cc: US/EA 

UNITED STATES DEPARTMENT OF COMMERCE 

Economics and Statistic• Admlnl•tra1lon 

U.S. Census Bureau 

Wasllll'~lon. OC 20233·0001 

OFFICE or THE DIRECTOR 

~:M' Z 1 ?017 

USCENSUSBUREAU 

www.census.gQv 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Appendix D: Agency Response 

FINAL REPORT NO. OIG-12-023-1 23 

 

OIG Draft Report 

2020 Census Planning: Delays with 2010 Census Resea rch Studies 

May Adversely lmpacl the 2020 Decennial Census 

U.S. Census Bureau Response 

The Census Bureau has reviewed the Department of Commerce, Office of Inspector General, 

(OIG) Draft Report dated March 5, 2012, and has the following comments: 

Snccific Comments 

Page 4, first paragnph: The sentence reads ... "After the 2000 Census. the Bureau caned led 

more than a quarter of the program eva/uations ... to tn(tke changes to the 2010 Census. " 

Census Bureau response: We agree with this statement. We, in fact, canceled some 

evaluations after the 2000 Census due in part to resource constraints. To make the connection 

with this paragraph to the subsequent paragraph on this page, we suggest you modily this 

statement to read ... "After the 2000 Census, the Bureau, due in part to resource constraints, 

cancelled more than a quarter of the program evaluations .... " 

l>age 6, parngrapb 3: "Delays in completing 2010 Census studies ... the timely completion of 

research proj ects becomes critical. " 

Census Bureau response: As we move forward with 20 l 0 CPEX, we are scheduled to brief the 

findings or all I 00 studies to the Executive Steering Commillee (ESC) by the end of fiscal year 

2012. The ESC, which is comprised mostly of Associate Directors and Division Chiefs, was 

established to "provide guidance on me results from research and the assessments of the 20 l 0 

Census operations." We are working under the assumption that review and approval of tina I 

reports (a tier the presentation or findings to the ESC) would on average require approximately 

four months after the presentation. We estimate that we will still be able to release all final 

reports by i\pril201 3. ln the meantime, we have made all preliminary findings available 

internally to staff worki''l! on early 2020 Census efforts. A s well, the staff wo1·king on early 

2020 Census research and testing have, in many cases, worked also on the 20 I 0 Census 

operations, and !here fore already have knowledge of operational outcomes that they rely on 

when assisting in deve loping research plans. We wiU also continue to make preliminary findings 

available - as we did with tl1e assessment of the Address Canvassing Operations - to the OIG 

and GAO as requested to support specific audits. 

1'11ges 6-7, last parag•·aph: "The CPEX program- which costs $1)8 million for FYs 2009-

201 I ... and implementil1g the thirty:five 2020 Cei1Sus research projects. " 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL
 

FINAL REPORT NO. OIG-12-023-1 24 

 

Census Bureau response: The actual expenditures for the 20 l 0 CJ>EX in fiSCal years 2009 

through 2011 were $68.1 million combined, while the budget for tiscal year 2012 is currentl y 

$11.2 million and the funding request for fiscal year 20 13 is $7.3 million. 

Page 9: Recommendation 1 - Prioritize further the 2010 CPEX studies, and .focus program 

resources, to ensure thCII the most critical swdies affecting the cost and quality of the 

2020 Census are completed 

Census Bureau response: Since the time the 0 10 conducted this revic\v, !he Census Bureau 

underwent a series of activities to ensure that the most critical studies affecting the cost and 

quality of the 2020 Census arc completed. First, the 20 10 CPEX staff worked directly with 

management who oversee curly 2020 Census research and testing efforts to identify the 

53 2010 CPEX studies from which resultS were deemed critical to early decennial census 

planning. When we received significantly lower than requested appropriations from Congress 

for fiscal year 2012, the Census Bureau bad to identify 20 studies for cancelation leaving us with 

the current I 00 studies. We removed from consideration any studies where findings had already 

been presented to the ESC tor Census Assessments and Research, as well as any studies that 

were designated as critical to early 2020 Census efforts (see 2010 Decennial Census Program 

Decision Memorandum Series, Memorandum 38 for more information on this process). 

Therefore, it is important to note that none of the studies designated as critical to early 2020 

Census resea.rch and testing were eliminated from the 

20 l 0 CI'EX program. 

We conduct regular briefings on the progress of the prognun to various stakeholders including: 

• weekly check-ins \\~th the Associate Directors for 20 10 and 2020 Censuses, and Division 

Chief for 2020 Research and Planning Office where we present the 20 I 0 CPEX 

dashboard showing the number of reports briefed to the ESC, reports submitted for final 

review, and reports that have been finalized and released for the overall program, 

• monthly progress reports to the Deputy Director on the 20 I 0 CPEX dashboards and the 

ESC briefing dates for each 2010 CPEX study, and 

• monthly swtus reports to the Office of Management and Budget and DOC on the overall 

progress of the 20 I 0 CI'EX program. 

We also monitor the progress of individual report.~ through the use of l.'J)Cciali~..ed tracking tools 

and through continuous exchanges with individual authors, managers, and stakeholders. We 

utilized this approach to ensure appropriate resource loading and timely availability of accurate 

findings from all 2010 CPEX studies. 

In the case of d1e concerns raised about the delayed public release of the Address Canvassing 

Assessment, we identilied the need for more critical review of the preliminary findings that were 

delivered on March 24,201 1. Between March 2011 and January 2012, thc Bureau worked to 

discuss. debate. and fine-tune our interpretations of the findings so that we could present the 

most accurate and well-rounded portrayal of the Address Canvassing Operation. As part of this 

2 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL
 

FINAL REPORT NO. OIG-12-023-1 25 
3 

process, the Bureau held discussions on fi ndings from the Address Canvassing Operation with 

oversight groups including the National Academy of Sciences and the Office of lnspector 

General in August 2011. As a result of these efforts, preliminary findings were available to 

relevant stakeholders to assist in the development of research plans related to future address 

canvassing operations. While not all extemal stakeholders had the benefit of this briefing before 

the final reports were posted on the web, we believe we an: balancing the maximum gains of 

sharing information from the CPEX program while addressing fiscal constraints. Also, we 

consistently monitor the resources available to individual studies to ensure the timely production 

of results. To date) authors or managers have not contacted us with a request for additional 

human resources due to loss of key staff during critical periods. 

Page 9: Recommcndntion 2 - Improve the transparency of the 20 I 0 CP EX program by po.sling 

study plans, expected publication dutes for the 109 studies, und final reports online as soon as 

practicable so that stakeholders can review and moniwr the Bureau 's progress in redesigning 

the 2020 Census. 

Census Bureau response: Currently, there are I 00 sLudics in scope for the 2010 CPEX, not 

109 as the report indicates. Of these I 00 studies, we have already received the first drafts for 

two-thirds of them that detail preliminary findings. The findings for 56 of these studies have 

been briefed to the ESC. We have finalized 21 of the reports and published Lhem on our Web 

site at <http://2010.census.gov/2010census/about/cpex.php>. This Web site provides some 

backgroLmd information on the 2010 CPEX process as well as provides various methods to 

access the final reports from 20 I 0 CPEX studies- including which 2020 Census investment 

area they pertain to. 

General Comments 

Page 5: 

• Correct the report throughout to show total l 00 CPEX studies, not 109. 

• The Address Canvas:;ing Operations assessment was released January 2012, not 

September 20 II as the chart caption indicates. 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL
 

FINAL REPORT NO. OIG-12-023-1 26 

 

4 

Page 6: 

• The eSC that is hearing briefi ngs of CPEX results is not the 2020 ESC as st:ucd in the 

rcpon, but the ESC for Census Assessments and Research. l 'he briefings actually suu1cd 

in J\pril 2011 , not July20 l l. 

Page 7: 

• Correct the first sentence in paragraph I to show that there were SJ. not 52, studies 

among the I 00 thnt hnd been identified as critical, and nine, ro.ther than seven, were 

released by February 8. 

• Correct paragraph 2 to show there were 53 critical Studies. not 50. Although the report· 

claims 2 critical studies were eliminated, none of the 53 studies originall y described as 

critical have been eliminated. The total remains at 53. 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL
 

12CENSUS2120 

FINAL REPORT NO. OIG-12-023-1 27