Information Technology: Census Bureau Needs to Implement Key Management Practices
Released: 2012-09-18T00:00:00

Letter

Background

The Census Bureau’s mission is to collect and provide comprehensive data about the nation’s people and economy. Core activities include conducting decennial, economic, and government censuses; conducting demographic and economic surveys; managing international demographic and socioeconomic databases; providing technical advisory services to foreign governments; and performing other activities such as producing official population estimates and projections.

The Census Bureau is part of the Department of Commerce and is in the department’s Economics and Statistics Administration, led by the Under Secretary for Economic Affairs. The Census Bureau is headed by a Director and is organized into directorates corresponding to key programmatic and administrative functions as depicted in figure 1.

According to the bureau, while planning, taking, processing, and publishing the results of censuses and surveys still requires the work of thousands of people, advances over the years have been made in the speed of collection, analysis, and publication of data through the development of mechanical and electronic tools. For nearly 100 years, census data were tabulated by clerks who made tally marks or added columns of figures with a pen or a pencil. As the nation grew and there were more people, items, and characteristics to count, speedier tabulation methods had to be invented or the results of one census would not be processed before it was time for the next one. In 1880, the bureau used a “tabulating machine”—a wooden box in which a roll of paper was threaded past an opening where a clerk marked the tallies in various columns and then added up the marks when the roll was full—that made tabulating at least twice as fast as the previous manual process.

By 1950, mechanical tabulating improved; its speed had increased to 2,000 items per minute. In 1951, the first large-scale electronic computer, UNIVAC I, was designed and built specifically for the Census Bureau. This machine was able to tabulate 4,000 items per minute. From 1970 on, the bureau took advantage of new high-speed composers that converted the data on computer tape directly to words and numbers on off-set negative film used in publishing. Beginning in the mid-1980s, some statistics were made available on diskettes for use in microcomputers and users began to obtain statistics online. In the later 1980s, the bureau began testing CD-ROM (compact disk/read-only memory) laser disks as a medium for releasing data.

The 2000 Census demonstrated probably the biggest leap forward in the use of technology for collecting and disseminating data. According to the bureau, its previous response scanning system (which dated to the 1950s) was replaced with optical character recognition technology, allowing the bureau to design a respondent-friendly (instead of machine- friendly) questionnaire in which write-in responses could also be captured electronically. In addition, the bureau’s previous online data system from the 1990s evolved into online data available through the Census Bureau’s website. Further technological advances were made in the 2010 census through the use of handheld computers for certain parts of Census operations and integration of Global Positioning System information into Census Bureau maps. Although specific technical decisions for the 2020 Census remain to be made, both ongoing Census operations and the next decennial census will be highly reliant on the effective use of information technology.

The 2010 Decennial Census cost $13 billion and was the costliest U.S. census in history. One reason for the high cost was the increased use of paper-based processing over what was originally intended due to performance issues with key IT systems, which increased the cost of the census by up to $3 billion. The total cost of the census was 56 percent more than the $8.1 billion 2000 Decennial Census (in constant 2010 dollars). Based on past trends, if the growth rate continues unchecked, For 2020, the the census could cost approximately $25 billion in 2020.bureau intends to focus on several measures to reduce costs, including better use of IT, but still is planning to spend roughly $12 to $18 billion to conduct the census.

To support its IT operations for all of its activities, including those related to decennial censuses, the Census Bureau reported that it plans to spend $384 million on major IT investments in fiscal year 2012. Of this, $130 million is to be spent on systems managed by the IT Directorate and $254 million is to be spent on systems managed in other directorates. To support these efforts, a bureau official from the Human Resource Division reported that as of July 2012, the bureau employed 1,148 IT staff among its approximately 14,000 employees. IT staff are spread throughout the bureau: the IT Directorate has 256 staff, Economic Programs has 262 staff, the Decennial Census has 156 staff, Field Operations has 185 staff, and Demographic Programs has 123 staff.

Overview of Investment Management and GAO’s IT Investment Management Maturity Framework

GAO’s Information Technology Investment Management (ITIM) framework can be used by agencies to improve their organizational A central tenet of processes and measure progress in attaining them.this framework is the select/control/evaluate model. Figure 2 illustrates the central components of this model.

During the select phase the organization (1) identifies and analyzes each project’s risks and returns before committing significant funds to any project and (2) selects those IT projects that will best support its mission needs. This process should be repeated each time funds are allocated to projects, reselecting even ongoing investments as described below.

During the control phase the organization ensures that, as projects develop and investment expenditures continue, the project continues to meet mission needs at the expected levels of cost and risk. If the project is not meeting expectations or if problems have arisen, steps are quickly taken to address the deficiencies. If mission needs have changed, the organization is able to adjust its objectives for the project and appropriately modify expected project outcomes.

During the evaluate phase, actual versus expected results are compared after a project has been fully implemented. This is done to (1) assess the project’s impact on mission performance, (2) identify any changes or modifications to the project that may be needed, and (3) revise the investment management process based on lessons learned.

The ITIM framework consists of five progressive stages of maturity that an agency can achieve in its investment management capabilities. The maturity stages are cumulative; that is, in order to attain a higher stage, an agency must institutionalize all of the critical processes at the lower stages, in addition to the higher stage critical processes.

The framework’s five maturity stages (see fig. 3) represent steps toward achieving stable and mature processes for managing IT investments. The successful attainment of each stage leads to improvement in the organization’s ability to manage its investments. With the exception of the first stage, each maturity stage is composed of critical processes that must be implemented and institutionalized. These critical processes are further broken down into key practices that describe the types of activities that an organization should be performing to successfully implement each critical process. It is not unusual for an organization to be performing key practices from more than one maturity stage at the same time. However, our research shows that agency efforts to improve investment management capabilities should focus on implementing all the lower- stage practices before addressing the higher-stage practices.

Stage 2 critical processes lay the foundation by establishing successful, predictable, and repeatable investment control processes at the project level. Stage 3 is where the agency moves from project-centric processes to portfolio-based processes and evaluates potential investments according to how well they support the agency’s missions, strategies, and goals. Organizations implementing these Stage 2 and 3 practices have in place selection, control, and evaluation processes that are consistent with the Clinger-Cohen Act of 1996.evaluation techniques to continuously improve both investment processes and portfolios in order to achieve strategic outcomes.

Stages 4 and 5 require the use of The ITIM framework can be used to assess the maturity of an agency’s investment management processes and as a tool for organizational improvement. The overriding purpose of the framework is to encourage investment processes that promote business value and mission performance, reduce risk, and increase accountability and transparency in the decision-making process. We have used the framework in several of our evaluations and a number of agencies have adopted it.agencies have used ITIM for purposes ranging from self-assessment to redesign of their IT investment management processes.

Effective management of federal IT investments remains an ongoing challenge. In December 2010, the White House released a plan to reform federal IT management that includes greater attention to several of the management processes described in ITIM. The plan includes efforts to increase accountability for IT investments, strengthen IT program management, increase the authority of agency chief information officers, and strengthen the ability of agency investment review boards to oversee agency IT investments.

System Development Methodologies

The decision to invest in IT often leads to the acquisition or development of IT systems. To manage that development, organizations often employ a system development methodology. There are several different methodologies that can be used to develop IT systems, which range from the traditional waterfall model to the spiral model and to iterative models such as the Agile model.

The waterfall model begins with requirements development and continues sequentially through other phases—design, build, and test—using the output of one phase as the input to the next to develop a finished product at the end. This model allows the status of a development project to be easily identified and tracked based on the current phase of the project.

The spiral model uses a risk-based approach to incrementally build a system by cycling through the four development phases. Using this model, each spiral, or incremental cycle, typically starts by determining the development objectives and scope for the increment. Next, alternative solutions are evaluated and risk management techniques are employed to identify and reduce risks. Then, a product for the increment (such as a prototype) is developed. Finally, the product is evaluated to determine whether the increment’s initial objectives have been met.

The Agile model focuses on short-duration, small-scope development phases that produce segments of a functional product. This model operates with similar phases to the traditional waterfall model—requirements, design, build, and test—but uses a shorter development cycle to achieve multiple iterations in similar time frames. Recently, several agencies have tried Agile, as it calls for producing software in small, short increments. Shorter, more incremental approaches to IT development have been identified as having the potential to improve the way in which the federal government develops and implements IT. In a recent report, we identified 32 practices and approaches as effective for applying Agile software development methods to IT projects. Officials who have used Agile methods on federal projects generally agreed that these practices are effective. In addition, the Office of Management and Budget recently issued guidance that advocates the use of shorter delivery time frames, an approach consistent with Agile. See figure 4 for a comparison of the Agile and waterfall development methods.

According to the Software Engineering Institute (SEI) Capability Maturity Model Integration (CMMI) for Development, a noted reference for best practices in system life-cycle development processes, when establishing an organizational process, the organization should establish and maintain criteria and guidelines that can be tailored for a particular project based on the development model chosen, and other issues, such as customer needs, cost, schedule, and technical difficulty.

Also according to SEI, within a given system development model, a number of specific development activities should be addressed. These include requirements development and requirements management. Requirements development includes activities such as identifying desirable functionality and quality attributes through an analysis of scenarios with relevant stakeholders; analyzing and qualifying functionality required by end users; and partitioning requirements into groups based on established criteria such as similar functionality to facilitate and focus the requirements analysis. Requirements management provides management of the business and system requirements, and identification of inconsistencies among requirements and the project’s plans and work products.

GAO and Office of Personnel Management Guidance Help Federal Agencies Strategically Manage Human Capital

A strategic approach to human capital management includes viewing personnel as assets whose value can be enhanced by investing in them. Such an approach enables an organization to use their people effectively and to determine how well they integrate human capital considerations into daily decision making and planning for mission results. It also helps organizations to remain aware of and be prepared for current and future needs as an organization, and ensure that personnel have the knowledge, skills, and abilities needed to pursue the mission of the organization.

In 2003, we identified a set of key practices for effective strategic human capital management, including workforce planning.based on our reports and testimonies, reviews of studies by leading workforce planning organizations, and interviews with officials from the Office of Personnel Management and other federal agencies. Strategic workforce planning addresses two critical needs: (1) aligning an organization’s human capital program with its current and emerging mission and programmatic goals and (2) developing long-term strategies These practices are for acquiring, developing, and retaining staff to achieve programmatic goals. While agency approaches to workforce planning will vary, we and the Office of Personnel Management have identified key practices in effective strategic workforce planning, six of which are:

Align workforce planning with strategic planning and budget formulation.

Involve top management, employees, and other stakeholders in developing, communicating, and implementing the strategic workforce plan.

Identify the critical skills and competencies that will be needed to achieve current and future programmatic results.

Develop strategies that are tailored to address gaps in number, deployment, and alignment of human capital approaches for enabling and sustaining the contributions of all critical skills and competencies.

Build the capability needed to address administrative, educational, and other requirements important to support workforce planning strategies.

Monitor and evaluate the agency’s progress toward its human capital goals and the contribution that human capital results have made toward achieving programmatic results.

Activities for Implementing New Organizational Governance Processes

We have previously reported on the challenges associated with implementing a new IT governance framework, such as the Census Bureau is trying to do. Implementing a new governance framework and system development methodology are challenging tasks that can be aided by having robust implementation plans. Such a plan is instrumental in helping agencies coordinate and guide improvement efforts. As we have previously reported, several steps are important for successfully implementing new organizational governance processes related to For example, investment management and system development.organizations should:

Have a commitment from agency leadership to putting the process in place. Buy-in of key stakeholders should be obtained to ensure that their perspectives are considered and to facilitate adoption. This includes obtaining top management support and creating forums for involving business representatives.

Select an implementation team and develop a detailed implementation plan that lays out a roadmap for implementing the new process. An effective implementation team should include key stakeholders from both business and IT components. An implementation plan should build on existing strengths and weaknesses; specify measurable goals, objectives, and milestones; specify needed resources; assign responsibility and accountability for accomplishing tasks; and be approved by senior-level management. On the other end, measures to assess progress in meeting the objectives of the implementation efforts should be developed and should include lessons learned.

Perform pilot testing of the new process to evaluate the process and identify potential problems. Pilot testing is an effective—and usually necessary—tool for moving the agency successfully to full implementation. Pilot testing allows the agency to (1) evaluate the soundness of the proposed process in actual practice, (2) identify and correct problems with the new design, and (3) refine performance measures. Also, successful pilot testing will help strengthen support for full-scale implementation from employees, outside stakeholders, Congress, and the public, and help secure the funding needed for a smooth rollout.

Develop a formal evaluation process to determine the effectiveness of the new process in meeting the agency’s goals. The team should develop a formal evaluation process to determine the efficiency and effectiveness of the new process, both during pilot tests and full implementation, in meeting the agency’s performance goals. The process should also allow the agency to pinpoint trouble spots, so that corrective actions can be developed quickly.

Prior GAO and Inspector General Reports Identified Management Challenges

Our prior work has identified the importance of having sound management processes in place to help the bureau as it manages multimillion dollar investments needed for its decennial census. For the last decennial, we issued multiple reports and testimonies from 2005 through 2010 on weaknesses in the Census Bureau’s management and testing of key 2010 Decennial Census IT systems. For example, in June 2005, we found that while the Census Bureau had initiated key practices in areas such as providing investment oversight, project planning, requirements management, and risk management, they were not fully and consistently performed across the bureau. Accordingly, we made recommendations to the Census Bureau to develop procedures to ensure consistent investment management and decision-making practices and to institutionalize a process improvement initiative, such as the CMMI framework, to strengthen bureau-wide system development and management processes. We noted that unless these recommendations were implemented, the bureau would face increased risk that cost overruns, schedule slippages, and performance shortfalls would occur and it would not be able to effectively manage its multimillion dollar investments in IT.

As development of the IT systems progressed, these problems were realized. In 2007, we reviewed the status of four key IT acquisitions needed for the 2010 Decennial Census. The bureau was still in the process of addressing our 2005 recommendations and our review found that there were increases in cost estimates and projected cost overruns of at least $51 million for the Field Data Collection Automation program due to changes in requirements. There were also schedule slippages with two other projects. Furthermore, these four projects were not consistently implementing key risk management practices. We concluded that unless the bureau addressed our recommendations to strengthen system testing and risk management activities, there would be an increased probability that decennial systems would not be delivered on schedule and within budget or perform as expected.

Subsequently, in March 2008, we added the 2010 Decennial Census to our list of high-risk programs in part because of long-standing weaknesses in the Census Bureau’s IT acquisition and contract management function, difficulties in developing reliable life-cycle cost estimates, and key operations that were not tested under operational conditions. We also testified on significant risks facing the 2010 census. In particular, we testified in March 2008 that the Field Data Collection Automation program was experiencing significant problems, including schedule delays and cost increases from changes in requirements, which required additional work and staffing. In April 2008, the Census Bureau dropped the use of handheld devices developed as part of this program for nonresponse follow-up and reverted to a paper-based operation, requiring the development of a Paper-Based Operations Control System to manage the operation. Dropping the use of handhelds for nonresponse follow-up and replacing it with the paper-based system increased the cost of the 2010 Decennial Census by up to $3 billion.

In March 2009, we reported that the bureau continued to face a number of problems related to testing of key IT systems, such as the Paper- Based Operations Control Systems, that included weaknesses in test plans and schedules, and a lack of executive-level oversight and guidance. We recommended that the bureau complete key system- testing activities and improve testing oversight and guidance or the Census Bureau would face the risk that systems were not thoroughly tested and or would perform as planned. Later that year, we reported in November 2009 that the bureau had not finalized detailed requirements for releases of the Paper-Based Operations Control System, which put the system at risk for cost increases, schedule delays, or performance shortfalls. Although the bureau worked aggressively to improve the Paper-Based Operations Control System, we reported in December 2010 that the system had experienced significant issues when it was put in operation. The bureau attributed these issues, in part, due to a compressed development and testing schedule, as well as inadequate performance and interface testing.

At a cost of about $13 billion, 2010 was the costliest decennial census in history. While the 2010 census was removed from GAO’s high-risk list in February 2011, we reported in April 2011 that the bureau needed to continue to improve key practices for managing IT and strengthen its ability to develop reliable life-cycle cost estimates.

GAO, 2020 Census: Additional Steps Are Needed to Build on Early Planning, GAO-12-626 (Washington, D.C.: May 17, 2012). organizational transformation, long-term project planning, and strategic workforce planning, but we did identify opportunities for improvement.

In addition, the Department of Commerce Office of the Inspector General recently identified several management challenges the Census Bureau faces as it prepares for the 2020 Decennial Census. Office of the Inspector General noted that the bureau needed to implement improved project planning and management techniques early in the decade to address the weaknesses in project management, cost estimation, and risk management. Officials from the Census Bureau stated that those recommendations were consistent with their current plans.

Census Bureau Has Begun to Improve Its Investment Management and System Development Processes, but Has Not Finalized Plans for Implementation

U.S. Department of Commerce Office of Inspector General, Top Management Challenges Face the Department of Commerce, OIG-11-015 (Washington, D.C.: Dec. 20, 2010). developing systems necessary for the 2020 Decennial Census to help avoid a repeat of the cost and performance issues that occurred during the 2010 Decennial Census.

Census Bureau Has Not Finalized Its Investment Management Plan

The bureau has developed a new investment management plan, called the Enterprise Investment Management Plan, which is to apply to all investments, including IT investments. The draft plan outlines a portfolio management process that is to operate in two interdependent cycles, one to align current and planned investments to ensure the right investments are selected to support the bureau’s mission, and one to monitor the development, deployment, and operation of approved investments in order to ensure new projects are developed as planned and ongoing systems are regularly evaluated for their impact on and relevancy to the bureau’s mission.

The plan outlines key investment management roles and responsibilities for various groups within the bureau. The groups include the

Operating Committee, which is comprised of the bureau’s senior executive team, including the Deputy Director, who chairs the committee, and associate directors for all nine of the bureau’s directorates, one of whom is also the Chief Information Officer of the bureau. The Operating Committee has ultimate responsibility for directing the bureau’s resource allocations and overseeing program performance. It also has overall responsibility for all IT investments costing more than $10 million that are high priority and medium or high priority investments that cost more than $50 million.

Office of Risk Management and Program Evaluation, which is to manage the bureau’s enterprise investment portfolio, review business cases for major investments and all other projects and portfolio investments within the bureau, regardless of cost or priority, and track project, program, and portfolio performance information. The office began operating in January 2011.

Directorate, division, and program-level investment review boards, which are to assess, review, and prioritize all existing and proposed investments at the appropriate directorate, division, or program level, and escalate investment issues to a higher-level board when required. Directorate-level review boards have overall responsibility for investments that are high priority and cost less than $10 million, medium-priority investments between $10 million and $50 million, and low-priority investments costing more than $50 million. Division and program-level boards have responsibility for medium-priority investments of less than $10 million and low-priority investments that cost less than $50 million.

As shown in the following table, the bureau’s new investment management plan is consistent with key practices outlined in the ITIM framework for Stage 2, including having documented policies and procedures in place for identifying IT projects that support business needs and selecting investments for funding. However, other policies within the plan only partially address the ITIM framework. In particular, the plan does not include guidelines for the membership of investment review boards, the frequency of board meetings, and the thresholds for escalating issues to higher-level boards, as these decisions are left up to individual directorates to determine. Table 1 summarizes our assessment of the policies contained in the bureau’s draft plan against relevant practices in Stage 2 of the ITIM framework.

In addition to lacking key guidelines for investment review board operations and the enterprise portfolio management tool, the plan is still a draft. The Chief of the Office of Risk Management and Program Evaluation stated that the plan would be finalized in late September 2012. According to the official, it has taken the bureau time to finalize the plan due to its review process with stakeholders, which included holding desktop exercises with key staff in various directorates to walk through the new governance processes and obtain feedback, and the naming of a new Deputy Director for the bureau.

While development of a draft investment management plan is a useful first step toward more rigorous investment management, until the Office of Risk Management and Program Evaluation establishes guidelines for the frequency and membership of the investment review boards, thresholds for escalating cost or schedule variance issues, and time frames for project managers to make periodic updates of investment information in its enterprise portfolio management tool, the bureau is likely to face inconsistent application of its investment management plan.

Census Bureau Has Created a New System Development Methodology, but It Does Not Address Modifications Needed for Newer Models of Software Development

In February 2012, the bureau’s IT and 2020 Census Directorates, with the assistance of a contractor, developed a new system development life- cycle methodology to improve the bureau’s ability to develop IT systems and to address our prior 2005 recommendation to strengthen bureau- wide system development and management processes. The main elements of the bureau’s new methodology include:

Ten defined life-cycle phases, including initiation, concept development, planning, requirements analysis, design, development, integration and test, deployment, operations and maintenance, and disposition, along with corresponding activities and work products that must be completed.

Five development process models, including waterfall, prototyping, incremental, iterative,when developing new systems or modifying or adding functionality to existing systems. and spiral, that project managers can use

A tool that helps project managers choose the appropriate development process model.

An appendix that helps project managers identify the work products to complete for each phase of the life-cycle based on the project’s life- cycle cost and priority.

Although the methodology lays out a foundation for system development activities at the bureau, it has critical gaps. In particular, SEI’s CMMI for Development, a noted reference for best practices in system life-cycle development processes, recommends that an organization establish and maintain criteria and guidelines that can be tailored to suit changing situations. However, the bureau’s methodology is based on a waterfall development model and work products identified in the guide are only tied to categories of projects assigned according to each project’s life-cycle cost and priority. The guide does not have guidance on how to adapt the process and related work products to alternate development models. Specifically, since other life-cycle models do not follow the waterfall model in terms of phases, types of activities, and work products for each phase, the methodology should explain how it can be adapted to alternate software development models, including identifying mandatory work products for other non-waterfall development models and phases. For example, one project manager for a pilot project of the new guide that is using an iterative process development model confirmed that he had difficulty in using the guide and had to deviate from it as he was developing work products.

The current Chief Information Officer stated that the bureau’s process does not currently include alternate development models because the methodology is in its first iteration and he wanted to evaluate the new process with pilots before making changes or additions. Census Bureau officials also acknowledged that the system development life-cycle methodology will not be as useful with other life-cycle models, such as an iterative model, and that they intend to develop additional guidance for non-waterfall models. However, they could not provide dates for when development of this guidance would be started or finalized.

Both private and public-sector organizations are making increasing use of non-waterfall development models as a means of reducing development Until the bureau specifies times and reducing risk for software projects.how to adapt the new system development life-cycle development methodology to non-waterfall models, the methodology will be of limited use for managing system development and acquisition efforts.

Both IT and 2020 Census Directorates Have Independently Drafted New Requirements Development and Management Processes, but Bureau-wide Approach Is Needed

SEI states that a disciplined process for developing and managing requirements can help reduce the risks of developing or acquiring a system. The practices underlying requirements development and management include eliciting, documenting, verifying and validating, and managing requirements through a system’s life cycle. This set of activities translates customer needs from statements of high-level business requirements into validated, testable system requirements. A well-defined and managed requirements baseline can, in addition, improve understanding among stakeholders and increase stakeholder buy-in and acceptance of the resulting system.

Both the IT Directorate and the 2020 Census Directorate have drafted new requirements development and management processes, though only the IT Directorate’s guidance has been finalized. In particular, the IT Directorate has developed a new process, the Application Services Division’s Requirements Elicitation, Analysis, and Documentation Process, for working with customers to develop requirements for projects during the requirements phase. This includes activities for creating a requirements work plan, identifying high-level and detailed project requirements, as well as assessing the feasibility and managing the risks associated with these requirements. The 2020 Census Directorate has developed a draft Requirements Engineering Management Plan that establishes processes for developing both enterprise-level mission requirements for the 2020 Decennial Census and specific project-level business, capability, and solution requirements. The process includes four stages for developing and managing these sets of requirements including discovery, analysis, agreement, and solution acceptance, and outlines roles and responsibilities for stakeholders.

While both the IT and 2020 Census Directorates have established new requirements development and management processes, which are to be used in the requirements analysis phase of the bureau’s new system development methodology, the bureau has not established a consistent process bureau-wide as we have previously recommended. In 2005 we found that individual project teams within the bureau had not consistently implemented key practices for requirements management and we recommended that a consistent approach be established bureau-wide. These weaknesses in the bureau’s processes for requirements management were not sufficiently addressed and we reported in 2007, 2008, and 2009 that these issues contributed to increases in life-cycle cost estimates and cost overruns of hundreds of millions of dollars for key investments necessary for the 2010 Decennial Census. Furthermore, the bureau had developed a handheld device for the 2010 census that did not operate as intended because of the lack of a robust requirements process. Instead, the bureau had to rely on a paper-based system to replace the handheld devices for key aspects of the census, which increased the cost of the 2010 census by up to $3 billion.

Nevertheless, both the Chief Information Officer, who heads the IT Directorate and the Associate Director of the 2020 Census Directorate, as well as the Chief of the Office of Risk Management and Program Evaluation, stated that there are no plans to standardize a requirements development and management process across the bureau. Currently, for future investments, bureau officials will decide which directorate requirements process to use, depending on which directorate has responsibility for developing the system. For example, the 2020 Census and IT Directorates are currently working together to develop a new electronic document management system Decennial Census. For this project, only the IT Directorate’s requirements process is being used because the IT Directorate has overall responsibility for developing the system.

The Electronic Document Management System will be used to streamline the American Community Service Office and 2020 Census program processes for developing, reviewing, and approving documents at the program and project levels. future in preparation for the 2020 Decennial Census, but did not specify when this would occur.

By utilizing overlapping requirements development and management processes without a specified time frame for when these processes will be integrated, the bureau is increasing the risk that IT investments, particularly those intended for the 2020 Decennial Census, will face cost overruns, schedule slippages, and performance shortfalls. Until the bureau establishes and implements a consistent requirements development and management process across the bureau that has clear guidance for developing requirements at the strategic mission, business, and project levels and is integrated with its new system development methodology, it will not have assurance that requirements for IT systems intended for the 2020 Decennial Census will be effectively developed or managed.

Census Bureau Has Not Finalized Plans for Implementing Investment Management and System Development Processes across the Bureau

While the bureau has drafted a new investment management plan, and system development methodology, including requirements development and management processes, key activities for effectively implementing these processes across the bureau remain to be undertaken. In particular, while the bureau’s leadership has made a commitment to putting the new investment plan in place across the bureau, no specific plans have been made to implement the system methodology bureau- wide. In addition, detailed implementation plans for putting these processes in place, including having sufficient pilot testing and formal evaluations to determine the effectiveness of the new processes remain to be developed. Table 2 shows our assessment of the bureau’s implementation efforts based on our prior work on implementing new processes within an organization.

With respect to the investment management plan, the Chief of the Office of Risk Management and Program Evaluation said that the bureau had not finalized a plan for implementing the new investment management structure because the office was still in the process of incorporating feedback on the plan. For the system development methodology, the Chief Information Officer stated that an implementation plan had not been finalized because more work was needed to refine the methodology. The methodology would be implemented across the bureau once it had been refined and all issues were resolved.

As we noted in 2005, the bureau’s lack of a consistent bureau-wide approach for IT investment management contributed to the bureau not effectively and efficiently managing multimillion dollar investments, including taking consistent and appropriate action when cost, schedule, or performance expectations were not being met. In addition, a lack of a consistent approach for system development and management, including requirements development and management, led to project teams managing systems in an ad hoc manner and increased the risk that cost overruns, schedule slippages, and performance shortfalls would, and did, occur, as we reported in 2007 and 2009.

According to the bureau’s timeline for 2020 Decennial Census planning, it will begin operational development and system testing starting in fiscal year 2015. While the exact design for 2020 information systems is not yet defined, it is likely to be complex, involve multiple directorates, and use both contractors and bureau staff based on the prior decennial census and the bureau’s initial plans for 2020.

Unless the investment management plan and system development life- cycle methodology, including a requirements development and management process, are fully implemented by this time, the Census Bureau will face increased risk that similar challenges that occurred in the 2010 Decennial Census will occur for the 2020 Decennial Census’s multimillion dollar investments. In addition, although only full implementation can identify all the potential problems with the new investment management plan and system methodology, until the bureau conducts additional pilot testing, including various software development models and projects of the scope and complexity of those needed for the 2020 Decennial Census, there is increased risk of not identifying potential problems with the investment management plan and system development life-cycle methodology.

Moreover, while feedback sessions on issues with implementing processes can be useful, until the Office of Risk Management and Program Evaluation and the Application Services Division establish a documented evaluation process to assess the effectiveness of the new processes, the bureau will lack assurance as to whether these processes are effective in meeting the bureau’s goals. Lastly, while any significant change cannot be accomplished overnight, clear leadership and deadlines are essential to implement changes. Failure to address these issues in a timely manner puts the bureau at risk of the same cost overrun, schedule slippage, and performance shortfall issues that affected the previous census.

Census Bureau Does Not Have Bureau- wide Workforce Planning Practices for IT Staff

Over the past year, the Census Bureau has taken limited steps to develop IT human capital practices, including identifying critical IT occupations and select competencies and conducting an inventory of these competencies among its IT staff in June 2011. However, many key practices remain to be implemented. In particular, the Census Bureau has not developed a bureau-wide IT workforce plan, identified gaps in mission-critical IT occupations, skills, and competencies, or developed strategies to address gaps. Table 3 summarizes our assessment of the bureau’s efforts against key principles for effective workforce planning.

According to officials in the Human Resources Division, the bureau has not yet begun key workforce planning activities because it first needs to complete an initial bureau-wide competency assessment. In planning for this assessment, the bureau conducted a pilot assessment of IT competencies in June 2011. Officials stated that the pilot assessment provided several lessons that they intend to use for the bureau-wide competency assessment. For instance, while managers in the directorates with IT staff found that the information from the pilot assessment was useful, those managers were more interested in identifying the current skills of their IT staff than the competency information that was gathered. The bureau’s Human Resources Division therefore collected additional information for managers in May 2012. The bureau is to conduct the bureau-wide competency assessment, including a reassessment of IT competencies, in late 2012.

Once this assessment is completed, the bureau’s directorates are to conduct additional workforce planning activities. For example, bureau officials stated that individual directorates plan to conduct gap analyses for mission-critical skills and competencies. According to the bureau’s Human Resources Division, once the gap analysis is completed, the bureau is to begin refining existing strategies and its capacity to address workforce gaps. The bureau provided a document with high-level goals to incorporate results of its competency assessment into individual directorate workforce plans by July 2013. However, it did not provide time frames for when specific activities would be completed for its IT workforce, such as a gap analysis for occupations, skills, or competencies of IT staff, nor did it provide plans to integrate these activities bureau- wide.

The Chief of the Human Resources Division stated that the Census Bureau has traditionally been decentralized in its IT workforce planning efforts because its IT staff is located in several directorates. The bureau has also faced budget constraints in conducting workforce planning specifically for its IT staff but had recently begun to undertake efforts in this area within the IT Directorate after a new division chief was hired.

Effective IT workforce planning efforts are critical to ensuring the bureau has the appropriate workforce in place to achieve its mission and strategic goals, particularly in regards to the 2020 Decennial Census. While the bureau’s Human Resources Division plans to undertake a thorough assessment of the competencies of its workforce this year, unless the division establishes a repeatable process for performing skills assessments and gap analysis that can be implemented in a timely manner, managers may not be able to make decisions to address any skills gaps in preparation for the 2020 Decennial Census. In addition, until the Human Resources Division establishes a process for directorates to coordinate on IT workforce planning in line with key principles for effective workforce planning, the bureau may not have sufficient assurance that it has the appropriate IT workforce needed for 2020 Decennial Census activities.

Conclusions

While the Census Bureau has begun to make improvements to investment management and system development processes, including requirements development and management, more work remains to be done to refine these processes and implement them across the bureau. As we have previously reported, a lack of robust processes in these areas contributed to the cost overruns, schedule slippages, and performance shortfalls in key IT investments that were needed for the 2010 Decennial Census, which increased its cost by up to $3 billion. However, the bureau has not established key guidelines and thresholds within its investment management plan, nor has it developed guidance to tailor the bureau’s new system development methodology to alternate development models, or established plans to implement these new processes across the bureau. Until the bureau takes action in these key areas, there is the risk that similar issues will arise for the 2020 Decennial Census.

Furthermore, having an IT workforce that has the appropriate mission- critical skills and competencies will be necessary to help the bureau effectively develop and manage its multimillion dollar investments in information systems and technology. While the bureau has begun to improve its IT human capital practices including conducting an inventory of select IT competencies, many key workforce planning practices remain to be put in place, including conducting gap analyses and integrating IT workforce planning bureau-wide. Having effective IT workforce planning practices will help ensure the bureau can achieve its mission and strategic goals for the 2020 Decennial Census.

Recommendations for Executive Action

To strengthen and improve the bureau’s new investment management, system development, and IT workforce management processes, we recommend that the Acting Secretary of Commerce direct the Under Secretary for Economic Affairs who oversees the Economics and Statistics Administration, as well as the Acting Director of the U.S. Census Bureau, take eight actions to address weaknesses in the following IT management areas:

Establish guidelines for the frequency and membership of bureau investment review boards and thresholds for these boards to escalate cost or schedule variance issues to higher-level boards.

Establish time frames for project managers to provide periodic updates of investment information in the enterprise investment management tool.

Adapt the bureau’s new system development life-cycle methodology, including the mandatory work products, activities, and phases of the project, to the additional software development models beyond the waterfall model that are specified in the methodology.

Establish and implement a consistent requirements development and management process across the bureau that is integrated with its new system development life-cycle methodology and includes guidance for developing requirements at the strategic mission, business, and project levels.

Finalize a plan for implementing the Enterprise Investment Management Plan, including time frames for implementation by fiscal year 2015, pilot testing of the new process, and a documented evaluation process.

Establish a plan for implementing the new system development life- cycle methodology, including requirements development and management processes, across the bureau, to include time frames for implementation by fiscal year 2015, additional pilots of the methodology prior to full implementation, and a documented evaluation process.

Establish a repeatable process for performing IT skills assessments and gap analysis that can be implemented in a timely manner.

Establish a process for directorates to coordinate on IT workforce planning, including: (1) aligning IT workforce planning with strategic planning and budget formulation; (2) involving appropriate stakeholders and staff from each directorate; (3) identifying critical occupations, skills, and competencies, and analyzing workforce gaps; (4) developing strategies to address IT workforce gaps; (5) building capacity to address workforce gaps; and (6) monitoring and evaluating IT workforce planning efforts across the bureau, and ensure this process is implemented across the bureau.

Agency Comments

We received comments from the Acting Secretary of Commerce on a draft of this report. The comments are included in appendix II. In its comments, the department stated that the Census Bureau concurred with our eight recommendations and outlined steps it was taking to implement the recommendations. The bureau acknowledged that the Enterprise Investment Management Plan was in draft, but indicated it had deployed components of the plan, such as initiating governing boards in three directorates. The bureau also noted that its enterprise portfolio management tool had been placed into production in July and that the bureau was working to migrate to the new tool.

Regarding the system development life-cycle methodology, the bureau stated that the methodology was one of several initiatives to improve the bureau’s delivery of IT services. The bureau said that later iterations of the methodology would include more iterative models such as Agile that were applicable to its business needs. The bureau also indicated it was planning to include feedback from its pilot projects in the revised methodology and to provide additional guidance on required documentation.

For its requirements management processes, the bureau stated it was planning to integrate the bureau’s different requirements management processes and emphasized that joint commitment of the 2020 Decennial Census and IT Directorates was critical to the success of a requirements management process for the 2020 Decennial Census.

The bureau also indicated that it has provided extensive training and education for its IT workforce even though it lacked an integrated plan based on best practices. The bureau plans to develop a workforce plan to incorporate strategies for hiring, developing, and contracting to meet its identified requirements. It noted, however, that this could be affected by potential budget cuts. The bureau stated its bureau-wide assessment and competency gap analysis would be completed by fall 2012.

We are sending copies of this report to the Acting Secretary of Commerce, the Senior Advisor to the Acting Director and the Deputy Director of the U.S. Census Bureau, and interested congressional committees. The report also is available at no charge on GAO’s website at http://www.gao.gov.

If you or your staffs have any questions on the matters discussed in this report, please contact me at (202) 512-9286 or pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to evaluate the (1) effectiveness of the Census Bureau’s policies, procedures, and processes for managing information technology (IT) investments and system development and (2) the Census Bureau’s development of effective practices for acquiring and maintaining IT human capital skills.

To address our first objective, we reviewed the effectiveness of policies and procedures in two areas—investment management and system development and management. We compared the bureau’s new IT investment management policies and procedures in its draft Enterprise Investment Management Plan to the criteria for policies and procedures associated with maturity stage 2 of the Information Technology Investment Management (ITIM) framework. The ITIM framework consists of five progressive stages of maturity that an agency can achieve in its investment management capabilities. The maturity stages are cumulative; that is, in order to attain a higher stage, an agency must first institutionalize all of the critical processes at the lower stages. To determine whether the bureau satisfied the criteria for maturity stage 2 we compared the bureau’s policies and procedures to the critical processes outlined in stage 2. We did not evaluate the bureau at maturity stage 1 because our prior review in 2005 determined that it had passed that stage. We did not evaluate the Census Bureau at maturity stages 3, 4, or 5 because our prior work has shown that an agency should focus on implementing all practices associated with a lower phase before addressing the higher-stage practices. We also interviewed officials from the Office of Risk Management and Program Evaluation regarding the development and implementation of the plan across the bureau.

To assess the effectiveness of the bureau’s processes for managing system development, including requirements development and management, we reviewed the bureau’s System Development Life Cycle Users’ Guide and project templates, Requirements Engineering Management Plan, and the Application Services Division’s Requirements Elicitation, Analysis, and Documentation Process, and compared these documents with Software Engineering Institute’s (SEI) Capability Maturity Model Integration for Development criteria in two areas: establishing organizational processes, and requirements management and development. Although SEI specifies criteria in numerous areas, we focused on establishing organizational processes because the bureau’s system development methodology is newly developed. We focused on requirements development and management because of the bureau’s challenges in managing requirements for the 2010 Decennial Census. For each of the two areas, we analyzed bureau plans and procedures to determine if the practices described were consistent with those in the SEI criteria. In addition, we interviewed officials from the IT Directorate and 2020 Census Directorate regarding development and implementation of the guide and the requirements management and development processes.

To assess the Census Bureau’s efforts to effectively implement these new investment management and system development processes across the bureau, we interviewed officials from the Office of Risk Management and Program Evaluation, IT Directorate, and 2020 Census Directorate regarding implementation efforts and reviewed related documentation, which we compared to our reported best practices for implementing new organizational governance processes. We evaluated whether these efforts satisfied each of the components of each key activity from the best practices and assigned ratings of “implemented”, “partially implemented”, or “not implemented” based on that assessment. A rating of “partially implemented” was given if the bureau’s activities satisfied at least one component of the key activity.

To address our second objective, we reviewed bureau workforce planning documents, including the Human Capital Management Plan (FY 2011- 2016 ), IT Directorate’s 2011-2016 Strategic Information Technology Plan, and the IT Directorate’s workforce plan, and other bureau documentation related to the bureau’s pilot workforce competency assessment and compared these to the six leading principles for workforce planning that we and the Office of Personnel Management have identified to determine whether the bureau’s practices were consistent with these principles. We evaluated whether the bureau’s activities satisfied each of the detailed practices in a principle and assigned ratings of “implemented”, “partially implemented”, or “not implemented” based on that assessment. A rating of “partially implemented” was assessed if the bureau’s activities satisfied at least one of the detailed practices in the principle. We also interviewed officials from the Human Resources Division and IT Directorate to obtain information about the pilot assessment and the bureau’s workforce planning efforts.

We performed our work from March 2012 through September 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Commerce

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact listed above, the following staff made key contributions to this report: Vijay D’Souza (Assistant Director); Justin Booth; Nancy Glover; Valerie Hopkins; Paul Middleton; Tarunkant Mithani; and Karl Seifert.