Information Technology: Census Bureau Needs to Implement Key Management Practices
Published: 2012-09-18T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The 2010 Decennial Census, at a cost of approximately $13 billion, was the most expensive headcount in our nations history. Prior to the 2010 Decennial Census, the Census Bureau experienced significant challenges in managing its information systems leading to cost overruns and performance shortfalls which increased the cost of the 2010 census by almost $3 billion. Given the bureaus extensive use of IT in collecting, analyzing, and distributing information, GAO was asked to determine to what extent the bureau has developed (1) effective policies, procedures, and processes for managing IT investments and system development; and (2) effective practices for acquiring and maintaining IT human capital skills. To address these objectives, GAO identified leading practices in these areas, reviewed bureau policies and procedures to determine whether they followed these practices, and interviewed bureau officials.
section_title: What GAO Found
paragraphs: The U.S. Census Bureau (Census Bureau) has drafted a new investment management plan, system development methodology, and requirements development and management processes to improve its ability to manage information technology (IT) investments and system development, but additional work is needed to ensure these processes are effective and successfully implemented across the bureau. GAO and others have identified the importance of implementing critical processes within an agency to allow it to select, control, and evaluate its IT investments and effectively manage system development. The bureau has developed a new draft investment management plan which contains policies and guidance for managing IT projects; however, the plan does not explain when investments with cost or schedule variances should be escalated to higher-level boards for review, or when managers should provide updated investment information to a planned bureau-wide tracking tool. The bureau has also developed a new system development methodology guide, but the guide has critical gaps. For example, although there are five development process models allowed, including the traditional sequential approach and newer more iterative approaches, the guide does not explain how to adapt processes and related work products for newer iterative approaches. Furthermore, while the bureau has developed new draft requirements development and management processes for system development within individual bureau directorates, it has not established a consistent process bureau-wide as GAO recommended in 2005. Lack of a consistent bureau-wide process contributed to significant cost and performance issues in the 2010 Decennial Census. Although the bureau plans to begin operational development for the 2020 Decennial Census in fiscal year 2015, it has not finalized plans for implementing its new investment management and system development processes across the bureau. Until the bureau takes additional action to finalize and implement consistent, bureau-wide processes, it faces the risk that IT governance issues that adversely affected the 2010 Decennial Census will also impact the 2020 Decennial Census.
The bureau has begun to take steps to improve its IT workforce planning; however, many key practices consistent with principles for effective workforce planning remain to be put in place. In particular, there is no bureau-wide coordination of these workforce planning efforts. Each directorate is responsible for its own IT workforce planning and the bureau has not established any efforts to coordinate activities among directorates. While the bureau identified mission critical IT occupations and began an assessment of select mission critical competencies in June 2011, it does not plan to perform a bureau-wide IT competency assessment until the fall of 2012. Until bureau-wide IT workforce planning processes are established and the bureau develops specific plans to conduct an IT skills inventory and gap analysis, the bureau faces the risk that the appropriate IT workforce will not be in place to effectively develop and manage multimillion dollar investments in information systems and technology that will be needed for the 2020 Decennial Census.
section_title: What GAO Recommends
paragraphs: To strengthen and improve the Census Bureaus management of IT, GAO recommends that the Acting Secretary of Commerce take eight actions, including improvements to guidance for its planned IT investment process, a consistent requirements development and management process, an implementation plan and time frames for its investment management process and system development methodology, and coordination of IT workforce planning efforts. In written comments, the Acting Secretary concurred with our recommendations and described steps the bureau was taking to implement them.

section_title: Letter
paragraphs: Providing current and relevant data about the economy and people of the  United States is the mission of the U.S. Census Bureau (Census Bureau)  of the Department of Commerce. The data collected are vital for  determining reapportionment and redistricting of the congressional  districts for the U.S. House of Representatives; realigning the boundaries  of the legislative districts of each state; allocating money for federal  financial assistance; and providing a social, demographic, and economic  profile of the nation’s people to guide policy decisions at each level of  government. To improve the coverage, accuracy, and efficiency of  gathering data from the public, the Census Bureau relies on automation  and information technology (IT).
Given the Census Bureau’s extensive use of IT in collecting, analyzing,  and distributing information, you asked us to determine to what extent the  Census Bureau has developed (1) effective policies, procedures, and  processes for managing IT investments and system development; and (2)  effective practices for acquiring and maintaining IT human capital skills.
To meet these objectives, we reviewed the bureau’s policies and  procedures related to IT investment management, system development,  and human capital management. We also interviewed bureau officials  responsible for providing oversight in these areas to learn how the bureau  is implementing changes to those management processes. For more  information on our scope and methodology, see appendix I.
We conducted this performance audit from February 2012 through  September 2012 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The Census Bureau’s mission is to collect and provide comprehensive  data about the nation’s people and economy. Core activities include  conducting decennial, economic, and government censuses; conducting  demographic and economic surveys; managing international  demographic and socioeconomic databases; providing technical advisory  services to foreign governments; and performing other activities such as  producing official population estimates and projections.
The Census Bureau is part of the Department of Commerce and is in the  department’s Economics and Statistics Administration, led by the Under  Secretary for Economic Affairs. The Census Bureau is headed by a  Director and is organized into directorates corresponding to key  programmatic and administrative functions as depicted in figure 1.
According to the bureau, while planning, taking, processing, and  publishing the results of censuses and surveys still requires the work of  thousands of people, advances over the years have been made in the  speed of collection, analysis, and publication of data through the  development of mechanical and electronic tools. For nearly 100 years,  census data were tabulated by clerks who made tally marks or added  columns of figures with a pen or a pencil. As the nation grew and there  were more people, items, and characteristics to count, speedier tabulation  methods had to be invented or the results of one census would not be  processed before it was time for the next one. In 1880, the bureau used a  “tabulating machine”—a wooden box in which a roll of paper was  threaded past an opening where a clerk marked the tallies in various  columns and then added up the marks when the roll was full—that made  tabulating at least twice as fast as the previous manual process.
By 1950, mechanical tabulating improved; its speed had increased to  2,000 items per minute. In 1951, the first large-scale electronic computer,  UNIVAC I, was designed and built specifically for the Census Bureau.  This machine was able to tabulate 4,000 items per minute. From 1970 on,  the bureau took advantage of new high-speed composers that converted  the data on computer tape directly to words and numbers on off-set  negative film used in publishing. Beginning in the mid-1980s, some  statistics were made available on diskettes for use in microcomputers and  users began to obtain statistics online. In the later 1980s, the bureau  began testing CD-ROM (compact disk/read-only memory) laser disks as a  medium for releasing data.
The 2000 Census demonstrated probably the biggest leap forward in the  use of technology for collecting and disseminating data. According to the  bureau, its previous response scanning system (which dated to the  1950s) was replaced with optical character recognition technology,  allowing the bureau to design a respondent-friendly (instead of machine- friendly) questionnaire in which write-in responses could also be captured  electronically. In addition, the bureau’s previous online data system from  the 1990s evolved into online data available through the Census Bureau’s  website. Further technological advances were made in the 2010 census  through the use of handheld computers for certain parts of Census  operations and integration of Global Positioning System information into  Census Bureau maps. Although specific technical decisions for the 2020  Census remain to be made, both ongoing Census operations and the  next decennial census will be highly reliant on the effective use of  information technology.
The 2010 Decennial Census cost $13 billion and was the costliest U.S.  census in history. One reason for the high cost was the increased use of  paper-based processing over what was originally intended due to  performance issues with key IT systems, which increased the cost of the  census by up to $3 billion. The total cost of the census was 56 percent  more than the $8.1 billion 2000 Decennial Census (in constant 2010  dollars). Based on past trends, if the growth rate continues unchecked,   For 2020, the  the census could cost approximately $25 billion in 2020.bureau intends to focus on several measures to reduce costs, including  better use of IT, but still is planning to spend roughly $12 to $18 billion to  conduct the census.
To support its IT operations for all of its activities, including those related  to decennial censuses, the Census Bureau reported that it plans to spend  $384 million on major IT investments in fiscal year 2012. Of this, $130  million is to be spent on systems managed by the IT Directorate and $254  million is to be spent on systems managed in other directorates. To  support these efforts, a bureau official from the Human Resource Division  reported that as of July 2012, the bureau employed 1,148 IT staff among  its approximately 14,000 employees. IT staff are spread throughout the  bureau: the IT Directorate has 256 staff, Economic Programs has 262  staff, the Decennial Census has 156 staff, Field Operations has 185 staff,  and Demographic Programs has 123 staff.
subsections: section_title: Overview of Investment Management and GAO’s IT Investment Management Maturity Framework
paragraphs: GAO’s Information Technology Investment Management (ITIM)  framework can be used by agencies to improve their organizational   A central tenet of  processes and measure progress in attaining them.this framework is the select/control/evaluate model. Figure 2 illustrates  the central components of this model.
During the select phase the organization (1) identifies and analyzes each  project’s risks and returns before committing significant funds to any  project and (2) selects those IT projects that will best support its mission  needs. This process should be repeated each time funds are allocated to  projects, reselecting even ongoing investments as described below.
During the control phase the organization ensures that, as projects  develop and investment expenditures continue, the project continues to  meet mission needs at the expected levels of cost and risk. If the project  is not meeting expectations or if problems have arisen, steps are quickly  taken to address the deficiencies. If mission needs have changed, the  organization is able to adjust its objectives for the project and  appropriately modify expected project outcomes.
During the evaluate phase, actual versus expected results are compared  after a project has been fully implemented. This is done to (1) assess the  project’s impact on mission performance, (2) identify any changes or  modifications to the project that may be needed, and (3) revise the  investment management process based on lessons learned.
The ITIM framework consists of five progressive stages of maturity that  an agency can achieve in its investment management capabilities. The  maturity stages are cumulative; that is, in order to attain a higher stage,  an agency must institutionalize all of the critical processes at the lower  stages, in addition to the higher stage critical processes.
The framework’s five maturity stages (see fig. 3) represent steps toward  achieving stable and mature processes for managing IT investments. The  successful attainment of each stage leads to improvement in the  organization’s ability to manage its investments. With the exception of the  first stage, each maturity stage is composed of critical processes that  must be implemented and institutionalized. These critical processes are  further broken down into key practices that describe the types of activities  that an organization should be performing to successfully implement each  critical process. It is not unusual for an organization to be performing key  practices from more than one maturity stage at the same time. However,  our research shows that agency efforts to improve investment  management capabilities should focus on implementing all the lower- stage practices before addressing the higher-stage practices.
Stage 2 critical processes lay the foundation by establishing successful,  predictable, and repeatable investment control processes at the project  level. Stage 3 is where the agency moves from project-centric processes  to portfolio-based processes and evaluates potential investments  according to how well they support the agency’s missions, strategies, and  goals. Organizations implementing these Stage 2 and 3 practices have in  place selection, control, and evaluation processes that are consistent with  the Clinger-Cohen Act of 1996.evaluation techniques to continuously improve both investment processes  and portfolios in order to achieve strategic outcomes.
Stages 4 and 5 require the use of  The ITIM framework can be used to assess the maturity of an agency’s  investment management processes and as a tool for organizational  improvement. The overriding purpose of the framework is to encourage  investment processes that promote business value and mission  performance, reduce risk, and increase accountability and transparency  in the decision-making process. We have used the framework in several  of our evaluations and a number of agencies have adopted it.agencies have used ITIM for purposes ranging from self-assessment to  redesign of their IT investment management processes.
Effective management of federal IT investments remains an ongoing  challenge. In December 2010, the White House released a plan to reform  federal IT management that includes greater attention to several of the  management processes described in ITIM. The plan includes efforts to  increase accountability for IT investments, strengthen IT program  management, increase the authority of agency chief information officers,  and strengthen the ability of agency investment review boards to oversee  agency IT investments.
section_title: System Development Methodologies
paragraphs: The decision to invest in IT often leads to the acquisition or development  of IT systems. To manage that development, organizations often employ  a system development methodology. There are several different  methodologies that can be used to develop IT systems, which range from  the traditional waterfall model to the spiral model and to iterative models  such as the Agile model.
The waterfall model begins with requirements development and  continues sequentially through other phases—design, build, and  test—using the output of one phase as the input to the next to develop  a finished product at the end. This model allows the status of a  development project to be easily identified and tracked based on the  current phase of the project.
The spiral model uses a risk-based approach to incrementally build a  system by cycling through the four development phases. Using this  model, each spiral, or incremental cycle, typically starts by  determining the development objectives and scope for the increment.  Next, alternative solutions are evaluated and risk management  techniques are employed to identify and reduce risks. Then, a product  for the increment (such as a prototype) is developed. Finally, the  product is evaluated to determine whether the increment’s initial  objectives have been met.
The Agile model focuses on short-duration, small-scope  development phases that produce segments of a functional product.  This model operates with similar phases to the traditional waterfall  model—requirements, design, build, and test—but uses a shorter  development cycle to achieve multiple iterations in similar time  frames. Recently, several agencies have tried Agile, as it calls for  producing software in small, short increments. Shorter, more  incremental approaches to IT development have been identified as  having the potential to improve the way in which the federal  government develops and implements IT. In a recent report, we  identified 32 practices and approaches as effective for applying Agile  software development methods to IT projects. Officials who have  used Agile methods on federal projects generally agreed that these  practices are effective. In addition, the Office of Management and  Budget recently issued guidance that advocates the use of shorter  delivery time frames, an approach consistent with Agile. See figure 4  for a comparison of the Agile and waterfall development methods.
According to the Software Engineering Institute (SEI) Capability Maturity  Model Integration (CMMI) for Development, a noted reference for best  practices in system life-cycle development processes, when establishing  an organizational process, the organization should establish and maintain  criteria and guidelines that can be tailored for a particular project based  on the development model chosen, and other issues, such as customer  needs, cost, schedule, and technical difficulty.
Also according to SEI, within a given system development model, a  number of specific development activities should be addressed. These  include requirements development and requirements management.  Requirements development includes activities such as identifying  desirable functionality and quality attributes through an analysis of  scenarios with relevant stakeholders; analyzing and qualifying  functionality required by end users; and partitioning requirements into  groups based on established criteria such as similar functionality to  facilitate and focus the requirements analysis. Requirements  management provides management of the business and system  requirements, and identification of inconsistencies among requirements  and the project’s plans and work products.
section_title: GAO and Office of Personnel Management Guidance Help Federal Agencies Strategically Manage Human Capital
paragraphs: A strategic approach to human capital management includes viewing  personnel as assets whose value can be enhanced by investing in them.  Such an approach enables an organization to use their people effectively  and to determine how well they integrate human capital considerations  into daily decision making and planning for mission results. It also helps  organizations to remain aware of and be prepared for current and future  needs as an organization, and ensure that personnel have the  knowledge, skills, and abilities needed to pursue the mission of the  organization.
In 2003, we identified a set of key practices for effective strategic human  capital management, including workforce planning.based on our reports and testimonies, reviews of studies by leading  workforce planning organizations, and interviews with officials from the  Office of Personnel Management and other federal agencies. Strategic  workforce planning addresses two critical needs: (1) aligning an  organization’s human capital program with its current and emerging  mission and programmatic goals and (2) developing long-term strategies   These practices are  for acquiring, developing, and retaining staff to achieve programmatic  goals. While agency approaches to workforce planning will vary, we and  the Office of Personnel Management have identified key practices in  effective strategic workforce planning, six of which are:
Align workforce planning with strategic planning and budget  formulation.
Involve top management, employees, and other stakeholders in  developing, communicating, and implementing the strategic workforce  plan.
Identify the critical skills and competencies that will be needed to  achieve current and future programmatic results.
Develop strategies that are tailored to address gaps in number,  deployment, and alignment of human capital approaches for enabling  and sustaining the contributions of all critical skills and competencies.
Build the capability needed to address administrative, educational,  and other requirements important to support workforce planning  strategies.
Monitor and evaluate the agency’s progress toward its human  capital goals and the contribution that human capital results have  made toward achieving programmatic results.
section_title: Activities for Implementing New Organizational Governance Processes
paragraphs: We have previously reported on the challenges associated with  implementing a new IT governance framework, such as the Census  Bureau is trying to do. Implementing a new governance framework and  system development methodology are challenging tasks that can be  aided by having robust implementation plans. Such a plan is instrumental  in helping agencies coordinate and guide improvement efforts. As we  have previously reported, several steps are important for successfully  implementing new organizational governance processes related to   For example,  investment management and system development.organizations should:
Have a commitment from agency leadership to putting the process in  place. Buy-in of key stakeholders should be obtained to ensure that  their perspectives are considered and to facilitate adoption. This  includes obtaining top management support and creating forums for  involving business representatives.
Select an implementation team and develop a detailed  implementation plan that lays out a roadmap for implementing the  new process. An effective implementation team should include key  stakeholders from both business and IT components. An  implementation plan should build on existing strengths and  weaknesses; specify measurable goals, objectives, and milestones;  specify needed resources; assign responsibility and accountability for  accomplishing tasks; and be approved by senior-level management.  On the other end, measures to assess progress in meeting the  objectives of the implementation efforts should be developed and  should include lessons learned.
Perform pilot testing of the new process to evaluate the process and  identify potential problems. Pilot testing is an effective—and usually  necessary—tool for moving the agency successfully to full  implementation. Pilot testing allows the agency to (1) evaluate the  soundness of the proposed process in actual practice, (2) identify and  correct problems with the new design, and (3) refine performance  measures. Also, successful pilot testing will help strengthen support  for full-scale implementation from employees, outside stakeholders,  Congress, and the public, and help secure the funding needed for a  smooth rollout.
Develop a formal evaluation process to determine the effectiveness of  the new process in meeting the agency’s goals. The team should  develop a formal evaluation process to determine the efficiency and  effectiveness of the new process, both during pilot tests and full  implementation, in meeting the agency’s performance goals. The  process should also allow the agency to pinpoint trouble spots, so that  corrective actions can be developed quickly.
section_title: Prior GAO and Inspector General Reports Identified Management Challenges
paragraphs: Our prior work has identified the importance of having sound  management processes in place to help the bureau as it manages  multimillion dollar investments needed for its decennial census. For the  last decennial, we issued multiple reports and testimonies from 2005  through 2010 on weaknesses in the Census Bureau’s management and  testing of key 2010 Decennial Census IT systems. For example, in June  2005, we found that while the Census Bureau had initiated key practices  in areas such as providing investment oversight, project planning,  requirements management, and risk management, they were not fully and  consistently performed across the bureau. Accordingly, we made  recommendations to the Census Bureau to develop procedures to ensure  consistent investment management and decision-making practices and to  institutionalize a process improvement initiative, such as the CMMI  framework, to strengthen bureau-wide system development and  management processes. We noted that unless these recommendations  were implemented, the bureau would face increased risk that cost  overruns, schedule slippages, and performance shortfalls would occur  and it would not be able to effectively manage its multimillion dollar  investments in IT.
As development of the IT systems progressed, these problems were  realized. In 2007, we reviewed the status of four key IT acquisitions  needed for the 2010 Decennial Census. The bureau was still in the  process of addressing our 2005 recommendations and our review found  that there were increases in cost estimates and projected cost overruns of  at least $51 million for the Field Data Collection Automation program  due to changes in requirements. There were also schedule slippages with  two other projects. Furthermore, these four projects were not consistently  implementing key risk management practices. We concluded that unless  the bureau addressed our recommendations to strengthen system testing  and risk management activities, there would be an increased probability  that decennial systems would not be delivered on schedule and within  budget or perform as expected.
Subsequently, in March 2008, we added the 2010 Decennial Census to  our list of high-risk programs in part because of long-standing  weaknesses in the Census Bureau’s IT acquisition and contract  management function, difficulties in developing reliable life-cycle cost  estimates, and key operations that were not tested under operational  conditions. We also testified on significant risks facing the 2010 census.  In particular, we testified in March 2008 that the Field Data Collection  Automation program was experiencing significant problems, including  schedule delays and cost increases from changes in requirements, which  required additional work and staffing. In April 2008, the Census Bureau  dropped the use of handheld devices developed as part of this program  for nonresponse follow-up and reverted to a paper-based operation,  requiring the development of a Paper-Based Operations Control System  to manage the operation. Dropping the use of handhelds for nonresponse  follow-up and replacing it with the paper-based system increased the cost  of the 2010 Decennial Census by up to $3 billion.
In March 2009, we reported that the bureau continued to face a number  of problems related to testing of key IT systems, such as the Paper- Based Operations Control Systems, that included weaknesses in test  plans and schedules, and a lack of executive-level oversight and  guidance. We recommended that the bureau complete key system- testing activities and improve testing oversight and guidance or the  Census Bureau would face the risk that systems were not thoroughly  tested and or would perform as planned. Later that year, we reported in  November 2009 that the bureau had not finalized detailed requirements  for releases of the Paper-Based Operations Control System, which put  the system at risk for cost increases, schedule delays, or performance  shortfalls. Although the bureau worked aggressively to improve the  Paper-Based Operations Control System, we reported in December 2010  that the system had experienced significant issues when it was put in  operation. The bureau attributed these issues, in part, due to a  compressed development and testing schedule, as well as inadequate  performance and interface testing.
At a cost of about $13 billion, 2010 was the costliest decennial census in  history. While the 2010 census was removed from GAO’s high-risk list in  February 2011, we reported in April 2011 that the bureau needed to  continue to improve key practices for managing IT and strengthen its  ability to develop reliable life-cycle cost estimates.
GAO, 2020 Census: Additional Steps Are Needed to Build on Early Planning,  GAO-12-626 (Washington, D.C.: May 17, 2012). organizational transformation, long-term project planning, and strategic workforce planning, but we did identify opportunities for improvement.
In addition, the Department of Commerce Office of the Inspector General  recently identified several management challenges the Census Bureau  faces as it prepares for the 2020 Decennial Census. Office of the Inspector General noted that the bureau needed to  implement improved project planning and management techniques early  in the decade to address the weaknesses in project management, cost  estimation, and risk management. Officials from the Census Bureau  stated that those recommendations were consistent with their current  plans.
section_title: Census Bureau Has Begun to Improve Its Investment Management and System Development Processes, but Has Not Finalized Plans for Implementation
paragraphs: U.S. Department of Commerce Office of Inspector General, Top Management  Challenges Face the Department of Commerce, OIG-11-015 (Washington, D.C.: Dec. 20,  2010). developing systems necessary for the 2020 Decennial Census to help  avoid a repeat of the cost and performance issues that occurred during  the 2010 Decennial Census.
subsections: section_title: Census Bureau Has Not Finalized Its Investment Management Plan
paragraphs: The bureau has developed a new investment management plan, called  the Enterprise Investment Management Plan, which is to apply to all  investments, including IT investments. The draft plan outlines a portfolio  management process that is to operate in two interdependent cycles, one  to align current and planned investments to ensure the right investments  are selected to support the bureau’s mission, and one to monitor the  development, deployment, and operation of approved investments in  order to ensure new projects are developed as planned and ongoing  systems are regularly evaluated for their impact on and relevancy to the  bureau’s mission.
The plan outlines key investment management roles and responsibilities  for various groups within the bureau. The groups include the
Operating Committee, which is comprised of the bureau’s senior  executive team, including the Deputy Director, who chairs the  committee, and associate directors for all nine of the bureau’s  directorates, one of whom is also the Chief Information Officer of the  bureau. The Operating Committee has ultimate responsibility for  directing the bureau’s resource allocations and overseeing program  performance. It also has overall responsibility for all IT investments  costing more than $10 million that are high priority and medium or  high priority investments that cost more than $50 million.
Office of Risk Management and Program Evaluation, which is to  manage the bureau’s enterprise investment portfolio, review business  cases for major investments and all other projects and portfolio  investments within the bureau, regardless of cost or priority, and track  project, program, and portfolio performance information. The office  began operating in January 2011.
Directorate, division, and program-level investment review boards,  which are to assess, review, and prioritize all existing and proposed  investments at the appropriate directorate, division, or program level,  and escalate investment issues to a higher-level board when required.  Directorate-level review boards have overall responsibility for  investments that are high priority and cost less than $10 million,  medium-priority investments between $10 million and $50 million, and  low-priority investments costing more than $50 million. Division and  program-level boards have responsibility for medium-priority  investments of less than $10 million and low-priority investments that  cost less than $50 million.
As shown in the following table, the bureau’s new investment  management plan is consistent with key practices outlined in the ITIM  framework for Stage 2, including having documented policies and  procedures in place for identifying IT projects that support business needs  and selecting investments for funding. However, other policies within the  plan only partially address the ITIM framework. In particular, the plan  does not include guidelines for the membership of investment review  boards, the frequency of board meetings, and the thresholds for  escalating issues to higher-level boards, as these decisions are left up to  individual directorates to determine. Table 1 summarizes our assessment  of the policies contained in the bureau’s draft plan against relevant  practices in Stage 2 of the ITIM framework.
In addition to lacking key guidelines for investment review board  operations and the enterprise portfolio management tool, the plan is still a  draft. The Chief of the Office of Risk Management and Program  Evaluation stated that the plan would be finalized in late September 2012.  According to the official, it has taken the bureau time to finalize the plan  due to its review process with stakeholders, which included holding  desktop exercises with key staff in various directorates to walk through  the new governance processes and obtain feedback, and the naming of a  new Deputy Director for the bureau.
While development of a draft investment management plan is a useful  first step toward more rigorous investment management, until the Office  of Risk Management and Program Evaluation establishes guidelines for  the frequency and membership of the investment review boards,  thresholds for escalating cost or schedule variance issues, and time  frames for project managers to make periodic updates of investment  information in its enterprise portfolio management tool, the bureau is likely  to face inconsistent application of its investment management plan.
section_title: Census Bureau Has Created a New System Development Methodology, but It Does Not Address Modifications Needed for Newer Models of Software Development
paragraphs: In February 2012, the bureau’s IT and 2020 Census Directorates, with the  assistance of a contractor, developed a new system development life- cycle methodology to improve the bureau’s ability to develop IT systems  and to address our prior 2005 recommendation to strengthen bureau- wide system development and management processes. The main  elements of the bureau’s new methodology include:
Ten defined life-cycle phases, including initiation, concept  development, planning, requirements analysis, design, development,  integration and test, deployment, operations and maintenance, and  disposition, along with corresponding activities and work products that  must be completed.
Five development process models, including waterfall, prototyping,  incremental, iterative,when developing new systems or modifying or adding functionality to  existing systems.  and spiral, that project managers can use
A tool that helps project managers choose the appropriate  development process model.
An appendix that helps project managers identify the work products to  complete for each phase of the life-cycle based on the project’s life- cycle cost and priority.
Although the methodology lays out a foundation for system development  activities at the bureau, it has critical gaps. In particular, SEI’s CMMI for  Development, a noted reference for best practices in system life-cycle  development processes, recommends that an organization establish and  maintain criteria and guidelines that can be tailored to suit changing  situations. However, the bureau’s methodology is based on a waterfall  development model and work products identified in the guide are only tied  to categories of projects assigned according to each project’s life-cycle  cost and priority. The guide does not have guidance on how to adapt the  process and related work products to alternate development models.  Specifically, since other life-cycle models do not follow the waterfall model  in terms of phases, types of activities, and work products for each phase,  the methodology should explain how it can be adapted to alternate  software development models, including identifying mandatory work  products for other non-waterfall development models and phases. For  example, one project manager for a pilot project of the new guide that is  using an iterative process development model confirmed that he had  difficulty in using the guide and had to deviate from it as he was  developing work products.
The current Chief Information Officer stated that the bureau’s process  does not currently include alternate development models because the  methodology is in its first iteration and he wanted to evaluate the new  process with pilots before making changes or additions. Census Bureau  officials also acknowledged that the system development life-cycle  methodology will not be as useful with other life-cycle models, such as an  iterative model, and that they intend to develop additional guidance for  non-waterfall models. However, they could not provide dates for when  development of this guidance would be started or finalized.
Both private and public-sector organizations are making increasing use of  non-waterfall development models as a means of reducing development   Until the bureau specifies  times and reducing risk for software projects.how to adapt the new system development life-cycle development  methodology to non-waterfall models, the methodology will be of limited  use for managing system development and acquisition efforts.
section_title: Both IT and 2020 Census Directorates Have Independently Drafted New Requirements Development and Management Processes, but Bureau-wide Approach Is Needed
paragraphs: SEI states that a disciplined process for developing and managing  requirements can help reduce the risks of developing or acquiring a  system. The practices underlying requirements development and  management include eliciting, documenting, verifying and validating, and  managing requirements through a system’s life cycle. This set of activities  translates customer needs from statements of high-level business  requirements into validated, testable system requirements. A well-defined  and managed requirements baseline can, in addition, improve  understanding among stakeholders and increase stakeholder buy-in and  acceptance of the resulting system.
Both the IT Directorate and the 2020 Census Directorate have drafted  new requirements development and management processes, though only  the IT Directorate’s guidance has been finalized. In particular, the IT  Directorate has developed a new process, the Application Services  Division’s Requirements Elicitation, Analysis, and Documentation  Process, for working with customers to develop requirements for projects  during the requirements phase. This includes activities for creating a  requirements work plan, identifying high-level and detailed project  requirements, as well as assessing the feasibility and managing the risks  associated with these requirements. The 2020 Census Directorate has  developed a draft Requirements Engineering Management Plan that  establishes processes for developing both enterprise-level mission  requirements for the 2020 Decennial Census and specific project-level  business, capability, and solution requirements. The process includes  four stages for developing and managing these sets of requirements  including discovery, analysis, agreement, and solution acceptance, and  outlines roles and responsibilities for stakeholders.
While both the IT and 2020 Census Directorates have established new  requirements development and management processes, which are to be  used in the requirements analysis phase of the bureau’s new system  development methodology, the bureau has not established a consistent  process bureau-wide as we have previously recommended. In 2005 we  found that individual project teams within the bureau had not consistently  implemented key practices for requirements management and we  recommended that a consistent approach be established bureau-wide.  These weaknesses in the bureau’s processes for requirements  management were not sufficiently addressed and we reported in 2007,  2008, and 2009 that these issues contributed to increases in life-cycle  cost estimates and cost overruns of hundreds of millions of dollars for key  investments necessary for the 2010 Decennial Census. Furthermore,  the bureau had developed a handheld device for the 2010 census that did  not operate as intended because of the lack of a robust requirements  process. Instead, the bureau had to rely on a paper-based system to  replace the handheld devices for key aspects of the census, which  increased the cost of the 2010 census by up to $3 billion.
Nevertheless, both the Chief Information Officer, who heads the IT  Directorate and the Associate Director of the 2020 Census Directorate, as  well as the Chief of the Office of Risk Management and Program  Evaluation, stated that there are no plans to standardize a requirements  development and management process across the bureau. Currently, for  future investments, bureau officials will decide which directorate  requirements process to use, depending on which directorate has  responsibility for developing the system. For example, the 2020 Census  and IT Directorates are currently working together to develop a new  electronic document management system Decennial Census. For this project, only the IT Directorate’s requirements  process is being used because the IT Directorate has overall  responsibility for developing the system.
The Electronic Document Management System will be used to streamline the American  Community Service Office and 2020 Census program processes for developing,  reviewing, and approving documents at the program and project levels. future in preparation for the 2020 Decennial Census, but did not specify  when this would occur.
By utilizing overlapping requirements development and management  processes without a specified time frame for when these processes will  be integrated, the bureau is increasing the risk that IT investments,  particularly those intended for the 2020 Decennial Census, will face cost  overruns, schedule slippages, and performance shortfalls. Until the  bureau establishes and implements a consistent requirements  development and management process across the bureau that has clear  guidance for developing requirements at the strategic mission, business,  and project levels and is integrated with its new system development  methodology, it will not have assurance that requirements for IT systems  intended for the 2020 Decennial Census will be effectively developed or  managed.
section_title: Census Bureau Has Not Finalized Plans for Implementing Investment Management and System Development Processes across the Bureau
paragraphs: While the bureau has drafted a new investment management plan, and  system development methodology, including requirements development  and management processes, key activities for effectively implementing  these processes across the bureau remain to be undertaken. In  particular, while the bureau’s leadership has made a commitment to  putting the new investment plan in place across the bureau, no specific  plans have been made to implement the system methodology bureau- wide. In addition, detailed implementation plans for putting these  processes in place, including having sufficient pilot testing and formal  evaluations to determine the effectiveness of the new processes remain  to be developed. Table 2 shows our assessment of the bureau’s  implementation efforts based on our prior work on implementing new  processes within an organization.
With respect to the investment management plan, the Chief of the Office  of Risk Management and Program Evaluation said that the bureau had  not finalized a plan for implementing the new investment management  structure because the office was still in the process of incorporating  feedback on the plan. For the system development methodology, the  Chief Information Officer stated that an implementation plan had not been  finalized because more work was needed to refine the methodology. The  methodology would be implemented across the bureau once it had been  refined and all issues were resolved.
As we noted in 2005, the bureau’s lack of a consistent bureau-wide  approach for IT investment management contributed to the bureau not  effectively and efficiently managing multimillion dollar investments,  including taking consistent and appropriate action when cost, schedule, or  performance expectations were not being met. In addition, a lack of a  consistent approach for system development and management, including  requirements development and management, led to project teams  managing systems in an ad hoc manner and increased the risk that cost  overruns, schedule slippages, and performance shortfalls would, and did,  occur, as we reported in 2007 and 2009.
According to the bureau’s timeline for 2020 Decennial Census planning, it  will begin operational development and system testing starting in fiscal  year 2015. While the exact design for 2020 information systems is not yet  defined, it is likely to be complex, involve multiple directorates, and use  both contractors and bureau staff based on the prior decennial census  and the bureau’s initial plans for 2020.
Unless the investment management plan and system development life- cycle methodology, including a requirements development and  management process, are fully implemented by this time, the Census  Bureau will face increased risk that similar challenges that occurred in the  2010 Decennial Census will occur for the 2020 Decennial Census’s  multimillion dollar investments. In addition, although only full  implementation can identify all the potential problems with the new  investment management plan and system methodology, until the bureau  conducts additional pilot testing, including various software development  models and projects of the scope and complexity of those needed for the  2020 Decennial Census, there is increased risk of not identifying potential  problems with the investment management plan and system development  life-cycle methodology.
Moreover, while feedback sessions on issues with implementing  processes can be useful, until the Office of Risk Management and  Program Evaluation and the Application Services Division establish a  documented evaluation process to assess the effectiveness of the new  processes, the bureau will lack assurance as to whether these processes  are effective in meeting the bureau’s goals. Lastly, while any significant  change cannot be accomplished overnight, clear leadership and  deadlines are essential to implement changes. Failure to address these  issues in a timely manner puts the bureau at risk of the same cost  overrun, schedule slippage, and performance shortfall issues that  affected the previous census.
section_title: Census Bureau Does Not Have Bureau- wide Workforce Planning Practices for IT Staff
paragraphs: Over the past year, the Census Bureau has taken limited steps to develop  IT human capital practices, including identifying critical IT occupations  and select competencies and conducting an inventory of these  competencies among its IT staff in June 2011. However, many key  practices remain to be implemented. In particular, the Census Bureau has  not developed a bureau-wide IT workforce plan, identified gaps in  mission-critical IT occupations, skills, and competencies, or developed  strategies to address gaps. Table 3 summarizes our assessment of the  bureau’s efforts against key principles for effective workforce planning.
According to officials in the Human Resources Division, the bureau has  not yet begun key workforce planning activities because it first needs to  complete an initial bureau-wide competency assessment. In planning for  this assessment, the bureau conducted a pilot assessment of IT  competencies in June 2011. Officials stated that the pilot assessment  provided several lessons that they intend to use for the bureau-wide  competency assessment. For instance, while managers in the  directorates with IT staff found that the information from the pilot  assessment was useful, those managers were more interested in  identifying the current skills of their IT staff than the competency  information that was gathered. The bureau’s Human Resources Division  therefore collected additional information for managers in May 2012. The  bureau is to conduct the bureau-wide competency assessment, including  a reassessment of IT competencies, in late 2012.
Once this assessment is completed, the bureau’s directorates are to  conduct additional workforce planning activities. For example, bureau  officials stated that individual directorates plan to conduct gap analyses  for mission-critical skills and competencies. According to the bureau’s  Human Resources Division, once the gap analysis is completed, the  bureau is to begin refining existing strategies and its capacity to address  workforce gaps. The bureau provided a document with high-level goals to  incorporate results of its competency assessment into individual  directorate workforce plans by July 2013. However, it did not provide time  frames for when specific activities would be completed for its IT  workforce, such as a gap analysis for occupations, skills, or competencies  of IT staff, nor did it provide plans to integrate these activities bureau- wide.
The Chief of the Human Resources Division stated that the Census  Bureau has traditionally been decentralized in its IT workforce planning  efforts because its IT staff is located in several directorates. The bureau  has also faced budget constraints in conducting workforce planning  specifically for its IT staff but had recently begun to undertake efforts in  this area within the IT Directorate after a new division chief was hired.
Effective IT workforce planning efforts are critical to ensuring the bureau  has the appropriate workforce in place to achieve its mission and  strategic goals, particularly in regards to the 2020 Decennial Census.  While the bureau’s Human Resources Division plans to undertake a  thorough assessment of the competencies of its workforce this year,  unless the division establishes a repeatable process for performing skills  assessments and gap analysis that can be implemented in a timely  manner, managers may not be able to make decisions to address any  skills gaps in preparation for the 2020 Decennial Census. In addition, until  the Human Resources Division establishes a process for directorates to  coordinate on IT workforce planning in line with key principles for effective  workforce planning, the bureau may not have sufficient assurance that it  has the appropriate IT workforce needed for 2020 Decennial Census  activities.
section_title: Conclusions
paragraphs: While the Census Bureau has begun to make improvements to  investment management and system development processes, including  requirements development and management, more work remains to be  done to refine these processes and implement them across the bureau.  As we have previously reported, a lack of robust processes in these areas  contributed to the cost overruns, schedule slippages, and performance  shortfalls in key IT investments that were needed for the 2010 Decennial  Census, which increased its cost by up to $3 billion. However, the bureau  has not established key guidelines and thresholds within its investment  management plan, nor has it developed guidance to tailor the bureau’s  new system development methodology to alternate development models,  or established plans to implement these new processes across the  bureau. Until the bureau takes action in these key areas, there is the risk  that similar issues will arise for the 2020 Decennial Census.
Furthermore, having an IT workforce that has the appropriate mission-  critical skills and competencies will be necessary to help the bureau  effectively develop and manage its multimillion dollar investments in  information systems and technology. While the bureau has begun to  improve its IT human capital practices including conducting an inventory  of select IT competencies, many key workforce planning practices remain  to be put in place, including conducting gap analyses and integrating IT  workforce planning bureau-wide. Having effective IT workforce planning  practices will help ensure the bureau can achieve its mission and  strategic goals for the 2020 Decennial Census.
section_title: Recommendations for Executive Action
paragraphs: To strengthen and improve the bureau’s new investment management,  system development, and IT workforce management processes, we  recommend that the Acting Secretary of Commerce direct the Under  Secretary for Economic Affairs who oversees the Economics and  Statistics Administration, as well as the Acting Director of the U.S.  Census Bureau, take eight actions to address weaknesses in the  following IT management areas:
Establish guidelines for the frequency and membership of bureau  investment review boards and thresholds for these boards to escalate  cost or schedule variance issues to higher-level boards.
Establish time frames for project managers to provide periodic  updates of investment information in the enterprise investment  management tool.
Adapt the bureau’s new system development life-cycle methodology,  including the mandatory work products, activities, and phases of the  project, to the additional software development models beyond the  waterfall model that are specified in the methodology.
Establish and implement a consistent requirements development and  management process across the bureau that is integrated with its new  system development life-cycle methodology and includes guidance for  developing requirements at the strategic mission, business, and  project levels.
Finalize a plan for implementing the Enterprise Investment  Management Plan, including time frames for implementation by fiscal  year 2015, pilot testing of the new process, and a documented  evaluation process.
Establish a plan for implementing the new system development life- cycle methodology, including requirements development and  management processes, across the bureau, to include time frames for  implementation by fiscal year 2015, additional pilots of the  methodology prior to full implementation, and a documented  evaluation process.
Establish a repeatable process for performing IT skills assessments  and gap analysis that can be implemented in a timely manner.
Establish a process for directorates to coordinate on IT workforce  planning, including: (1) aligning IT workforce planning with strategic  planning and budget formulation; (2) involving appropriate  stakeholders and staff from each directorate; (3) identifying critical  occupations, skills, and competencies, and analyzing workforce gaps;  (4) developing strategies to address IT workforce gaps; (5) building  capacity to address workforce gaps; and (6) monitoring and  evaluating IT workforce planning efforts across the bureau, and  ensure this process is implemented across the bureau.
section_title: Agency Comments
paragraphs: We received comments from the Acting Secretary of Commerce on a  draft of this report. The comments are included in appendix II. In its  comments, the department stated that the Census Bureau concurred with  our eight recommendations and outlined steps it was taking to implement  the recommendations. The bureau acknowledged that the Enterprise  Investment Management Plan was in draft, but indicated it had deployed  components of the plan, such as initiating governing boards in three  directorates. The bureau also noted that its enterprise portfolio  management tool had been placed into production in July and that the  bureau was working to migrate to the new tool.
Regarding the system development life-cycle methodology, the bureau  stated that the methodology was one of several initiatives to improve the  bureau’s delivery of IT services. The bureau said that later iterations of  the methodology would include more iterative models such as Agile that  were applicable to its business needs. The bureau also indicated it was  planning to include feedback from its pilot projects in the revised  methodology and to provide additional guidance on required  documentation.
For its requirements management processes, the bureau stated it was  planning to integrate the bureau’s different requirements management  processes and emphasized that joint commitment of the 2020 Decennial  Census and IT Directorates was critical to the success of a requirements  management process for the 2020 Decennial Census.
The bureau also indicated that it has provided extensive training and  education for its IT workforce even though it lacked an integrated plan  based on best practices. The bureau plans to develop a workforce plan to  incorporate strategies for hiring, developing, and contracting to meet its  identified requirements. It noted, however, that this could be affected by  potential budget cuts. The bureau stated its bureau-wide assessment and  competency gap analysis would be completed by fall 2012.
We are sending copies of this report to the Acting Secretary of  Commerce, the Senior Advisor to the Acting Director and the Deputy  Director of the U.S. Census Bureau, and interested congressional  committees. The report also is available at no charge on GAO’s website  at http://www.gao.gov.
If you or your staffs have any questions on the matters discussed in this  report, please contact me at (202) 512-9286 or pownerd@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to evaluate the (1) effectiveness of the Census  Bureau’s policies, procedures, and processes for managing information  technology (IT) investments and system development and (2) the Census  Bureau’s development of effective practices for acquiring and maintaining  IT human capital skills.
To address our first objective, we reviewed the effectiveness of policies  and procedures in two areas—investment management and system  development and management. We compared the bureau’s new IT  investment management policies and procedures in its draft Enterprise  Investment Management Plan to the criteria for policies and procedures  associated with maturity stage 2 of the Information Technology  Investment Management (ITIM) framework. The ITIM framework consists  of five progressive stages of maturity that an agency can achieve in its  investment management capabilities. The maturity stages are cumulative;  that is, in order to attain a higher stage, an agency must first  institutionalize all of the critical processes at the lower stages. To  determine whether the bureau satisfied the criteria for maturity stage 2 we  compared the bureau’s policies and procedures to the critical processes  outlined in stage 2. We did not evaluate the bureau at maturity stage 1  because our prior review in 2005 determined that it had passed that  stage. We did not evaluate the Census Bureau at maturity stages 3, 4, or  5 because our prior work has shown that an agency should focus on  implementing all practices associated with a lower phase before  addressing the higher-stage practices. We also interviewed officials from  the Office of Risk Management and Program Evaluation regarding the  development and implementation of the plan across the bureau.
To assess the effectiveness of the bureau’s processes for managing  system development, including requirements development and  management, we reviewed the bureau’s System Development Life Cycle  Users’ Guide and project templates, Requirements Engineering  Management Plan, and the Application Services Division’s Requirements  Elicitation, Analysis, and Documentation Process, and compared these  documents with Software Engineering Institute’s (SEI) Capability Maturity  Model Integration for Development criteria in two areas: establishing  organizational processes, and requirements management and  development. Although SEI specifies criteria in numerous areas, we  focused on establishing organizational processes because the bureau’s  system development methodology is newly developed. We focused on  requirements development and management because of the bureau’s  challenges in managing requirements for the 2010 Decennial Census. For  each of the two areas, we analyzed bureau plans and procedures to  determine if the practices described were consistent with those in the SEI  criteria. In addition, we interviewed officials from the IT Directorate and  2020 Census Directorate regarding development and implementation of  the guide and the requirements management and development  processes.
To assess the Census Bureau’s efforts to effectively implement these  new investment management and system development processes across  the bureau, we interviewed officials from the Office of Risk Management  and Program Evaluation, IT Directorate, and 2020 Census Directorate  regarding implementation efforts and reviewed related documentation,  which we compared to our reported best practices for implementing new  organizational governance processes. We evaluated whether these  efforts satisfied each of the components of each key activity from the best  practices and assigned ratings of “implemented”, “partially implemented”,  or “not implemented” based on that assessment. A rating of “partially  implemented” was given if the bureau’s activities satisfied at least one  component of the key activity.
To address our second objective, we reviewed bureau workforce planning  documents, including the Human Capital Management Plan (FY 2011- 2016 ), IT Directorate’s 2011-2016 Strategic Information Technology  Plan, and the IT Directorate’s workforce plan, and other bureau  documentation related to the bureau’s pilot workforce competency  assessment and compared these to the six leading principles for  workforce planning that we and the Office of Personnel Management  have identified to determine whether the bureau’s practices were  consistent with these principles. We evaluated whether the bureau’s  activities satisfied each of the detailed practices in a principle and  assigned ratings of “implemented”, “partially implemented”, or “not  implemented” based on that assessment. A rating of “partially  implemented” was assessed if the bureau’s activities satisfied at least one  of the detailed practices in the principle. We also interviewed officials from  the Human Resources Division and IT Directorate to obtain information  about the pilot assessment and the bureau’s workforce planning efforts.
We performed our work from March 2012 through September 2012 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact listed above, the following staff made key  contributions to this report: Vijay D’Souza (Assistant Director); Justin  Booth; Nancy Glover; Valerie Hopkins; Paul Middleton; Tarunkant  Mithani; and Karl Seifert.