Information Technology: HHS Has Several Investment Management Capabilities in Place, but Needs to Address Key Weaknesses
Released: 2005-11-28T00:00:00

Letter

Background

HHS’s Mission, Organizational Structure, and Use of IT

HHS is the primary organization within the federal government that is devoted to protecting the health of Americans. It provides essential human services, such as ensuring food and drug safety and assisting needy families. HHS administers more grant dollars than all other federal agencies combined, providing over $200 billion of the more than $350 billion in federal funds that were awarded to states and other entities in fiscal year 2002, the most recent year for which these data are available. For fiscal year 2005, HHS had a budget of $581 billion and a workforce of over 67,000 employees.

To accomplish its mission, HHS is comprised of 12 component agencies and several staff offices that cover a wide range of activities—including conducting and sponsoring medical and social science research, guarding against the outbreak of infectious diseases, assuring the safety of food and drugs, and providing health care services and insurance. The Office of the Secretary consists of several staff divisions and offices, including the Office of the Assistant Secretary for Budget, Technology, and Finance. The HHS Office of the Chief Information Officer (CIO) is located within this staff office (see fig. 1).

Information technology investments play a critical role in helping HHS carry out its diverse mission. According to the President’s most recent budget, HHS expects to spend about $5 billion in IT in fiscal year 2006, making the department’s IT investment budget the third largest in the federal government. As figure 2 illustrates, approximately $3 billion is designated as grants to states for investments for Medicaid programs and other purposes, such as child support enforcement systems. Approximately $2 billion is for discretionary investment spending, of which 89 percent is used to fund IT investments for component agencies; 7 percent is invested in HHS enterprisewide initiatives; and 4 percent is used to fund other initiatives, including Office of the Inspector General IT investments.

Table 1 provides additional information about the component agencies and their estimated IT budget for fiscal year 2006.

HHS’ investments reflect the diversity of the department’s missions and operating environments. For example, HHS currently has several enterprisewide IT initiatives that enable stakeholders to advance the causes of better health, safety, and well-being for American people. These initiatives include: Unified Financial Management System, a new core financial system, to help management monitor budgets, conduct operations, evaluate program performance, and make financial and programmatic decisions. As a core financial system, it will interface with an estimated 110 other HHS information systems.

The Office of the Assistant Secretary for Public Health Emergency Preparedness maintains a command center where it can coordinate the response to public health emergencies from one centralized location. This center is equipped with satellite teleconferencing capability, broadband Internet hookups, and analysis and tracking software.

In addition, HHS’s component agencies have several projects and systems that are critical to the effective implementation of HHS’s mission, including the following: The Food and Drug Administration’s Automated Drug Information Management System is to be developed as a fully electronic information management system that will receive, evaluate, and disseminate information about investigational and marketing submissions for human drugs and therapeutic biologics.

The National Institutes of Health’s major IT initiative, the Clinical Research Information System, is a comprehensive effort to modernize the systems that support clinical care and the agency’s collection of research data for the intramural clinical research programs.

The Centers for Disease Control and Prevention’s major IT initiative, Public Health Information Network, is a national initiative to implement a multiorganizational business and technical architecture for public health information systems.

Prior Reviews Identified Weaknesses in HHS’s IT Investment Management Process

In January 2004, we reported on a broad view of the government’s implementation of investment management practices at 26 major departments and agencies, including HHS. We also reported—and HHS acknowledged—that there were serious weaknesses in investment management. Notably, the department had not yet established selection criteria for project investments or a requirement that investments support work processes that have been simplified or redesigned. In addition, the department did not have decision-making rules to guide oversight of IT investments, review projects at major milestones, or systematically track corrective actions. Accordingly, we made several recommendations, including that HHS revise its investment management policy and require PIRs to address validating benefits and costs. In response to our recommendations, the department has been modifying several of its investment management policies, including its capital planning and investment control guidance and its governance policies.

More recently, in June 2005, we reported that the HHS IT Investment Review Board had conducted only budgetary reviews of the Centers for Disease Control and Prevention’s Public Health Information Network and some of its initiatives, until this past February, when HHS initiated steps for better monitoring of system development projects. We concluded that until management implements a systematic method for IT investment reviews, it will have difficulty minimizing risks while maximizing returns on these critical public health investments.

HHS’s Approach to Investment Management

HHS has several groups and individuals involved in managing both the enterprisewide and component agency IT investments. They are involved from reviewing and approving a proposed IT project, through the process of budgeting for it, monitoring it through implementation, and evaluating it at its conclusion. The composition, roles, and responsibilities of these individuals and groups are described below: Information Technology Investment Review Board (ITIRB)—Chaired by HHS’s CIO, this board is responsible for selecting, controlling, and evaluating all departmental IT investments. Members include the Deputy Assistant Secretary for Budget, Finance, Performance and Planning; the Directors for Acquisition Management Policy and Human Resources; and the component agency CIOs. The board is supported by an executive secretary who is responsible for, among other things, managing the flow of IT investment documentation, scheduling meetings, and assisting the members in preparing for their meetings. Currently, this board reviews all enterprisewide investments and delegates responsibilities for component agency investments to each individual component agencies investment review boards in accordance with departmental policies and procedures.

CIO Council—Also chaired by the HHS CIO and comprised of component agency CIOs, this board advises the HHS ITIRB on the technical soundness of all IT investments that require departmental review and provides recommendations regarding, among other things, technical aspects of affordability, soundness of design, risk, and compliance with architectural and security standards.

Critical Partners—Comprised of departmental officials from various functional areas, including enterprise architecture, security and privacy, acquisition management, finance, budget, human resources, and e-government; this group is responsible for ensuring that most investments comply with the HHS policy in each of the functional areas and for advising the HHS ITIRB and individual IT investment managers on issues in their areas of expertise. Each review results in a determination whether the investment is approved, conditionally approved, or not approved. A not approved result is flagged for executive review.

Business Case Quality Review Team—Comprised of component agency officials, this group evaluates the justifications for IT investments—both formal business cases and information documented in the department’s portfolio management tool’s Select forms—against the criteria used by the Office of Management and Budget’s to evaluate business cases agencies submit to the office as part of the formulation of the federal budget and provides recommendations for improving these justifications.

Capital Planning and Investment Control (CPIC) Reengineering/Portfolio Management Tool (PMT) Implementation Team—Chaired by the Office of the CIO officials with representatives from the Critical Partners and the Business Case Quality Review Team, this group advises the board on issues regarding investment management policies and procedures and the implementation of the department’s portfolio management tool.

Investment Managers—Responsible for managing investments in accordance with approved cost, schedule, and performance baselines, and for maintaining information on project status, control, performance, risk, and corrective actions.

Process for Managing Investments

The department has defined a three-phase process for managing investments that involves selecting proposed projects and reselecting ongoing projects (select phase), controlling ongoing projects through development (control phase), and evaluating projects that have been deployed (evaluate phase). The department retains direct management of HHS enterprisewide IT investments and delegates considerable authority for other investments to component agencies. Specifically, the department selects ongoing and new component agency investments through the process for selecting enterprisewide IT investments described below. Controlling and evaluating component agency IT investments are delegated to the component agencies, which are required by the department to follow a process similar to the one described below.

Each phase of the process for enterprisewide investments is comprised of multiple steps that set out requirements needed for the HHS ITIRB to make the decision to move forward with the project.

The purpose of the select phase is to ensure that HHS chooses the projects that best support its mission and applies resources to the most important and valuable investments. The select phase is also intended to help the department justify budget requests by demonstrating sound business cases and project plans. To select investments, HHS has established two separate components—investment screening for new investment proposals and investment scoring and screening for ongoing investments.

During the new investment screening, the investment manager is expected to develop a project prospectus, which identifies a specific business need and preliminary, high-level system requirements. A high-level determination of resource and schedule requirements is also to be conducted as part of the business need identification activities. Approval of the project prospectus by the HHS ITIRB signifies that the agency agrees that the need is critical enough to proceed to the next step in which the business case is developed. During business case development, the investment manager is required to develop the business case, which establishes the lifecycle cost, schedule, benefits, and performance baselines and includes an analysis for each investment to identify alternatives that may satisfy the needs of the department. In addition, the investment managers sign a document called the accountability agreement form to accept responsibility for reporting on the project status in achieving performance baselines throughout the remaining phases of the investment management process.

After the project is initially approved by the HHS ITIRB, the business cases and Select forms for most IT investments are updated annually as part of the budget formulation process. (The Select forms are a collection of forms with HHS’s portfolio management tool that capture investment data to justify funding and ensure adequate project planning during the select phase.) The first step within the annual budget formulation process requires that all component agencies use the Select forms to report the project cost estimates that best represent the level of funding required to meet program or business needs. At this point, the Critical Partners and the Business Case Quality Review Team score and rank the Select forms using the department’s portfolio management tool to create a single HHS portfolio as well as component agency portfolios to provide recommendations to the component agencies for making final adjustments to their portfolio ranking.

Once the component agencies have made the appropriate changes, the Office of the CIO develops prioritized IT portfolios for HHS as a whole as well as each component agency to present to the HHS ITIRB. The departmental board and CIO Council review and comment on the prioritized portfolio and submit it to the Secretary’s Budget Council for input into their budget deliberations. The Secretary’s Budget Council then makes recommendations to the Secretary regarding HHS and component agencies’ budgets. Finally, the department submits its approved Secretary’s IT budget to the Office of Management and Budget for inclusion in the President’s Budget.

Once selected for inclusion in the department’s IT portfolio, each project is to be managed by an investment manager and reviewed by the ITIRB on a quarterly basis throughout the end of development. The board performs reviews of projects that deviate from predetermined budget, schedule, or performance milestones established in the business case and works with the investment managers to develop a correction action plan. The ITIRB must also decide whether to continue to fund the project; rebaseline the scope, schedule, or budget; or to terminate the project.

Once a project has been fully implemented, the HHS ITIRB is to conduct annual reviews of all HHS enterprisewide steady state investments—that is, investments in operations and maintenance—to determine whether they continue to meet the business needs. In addition, investments that have recently completed implementation or a significant phase are to undergo PIRs to evaluate actual development events against project management plans and to identify lessons learned that can be applied to current and future investments.

Figure 3 illustrates HHS’s investment management process phases and steps. The highlighted steps represent the activities that the department conducts for both enterprisewide and component agency investments.

ITIM Maturity Framework

The ITIM framework is a maturity model composed of five progressive stages of maturity that an agency can achieve in its investment management capabilities. It was developed on the basis of our research into the IT investment management practices of leading private- and public- sector organizations. In each of the five stages, the framework identifies critical processes for making successful IT investments. The maturity stages are cumulative; that is, in order to attain a higher stage the agency must have institutionalized all of the critical processes at the lower stages, in addition to the higher stage critical processes.

The framework can be used to assess the maturity of an agency’s investment management processes and as a tool for organizational improvement. The overriding purpose of the framework is to encourage investment processes that increase business value and mission performance, reduce risk, and increase accountability and transparency in the decision process. We have used the framework in several of our evaluations, and a number of agencies have adopted it. These agencies have used ITIM for purposes ranging from self-assessment to redesign of their IT investment management processes.

ITIM’s five maturity stages represent steps toward achieving stable and mature processes for managing IT investments. Each stage builds on the lower stages; the successful attainment of each stage leads to improvement in the organization’s ability to manage its investments. With the exception of the first stage, each maturity stage is composed of “critical processes” that must be implemented and institutionalized in order for the organization to achieve that stage. These critical processes are further broken down into key practices that describe the types of activities that an organization should be performing to successfully implement each critical process. It is not unusual for an organization to be performing key practices from more than one maturity stage at the same time, but efforts to improve investment management capabilities should focus on implementing all lower stage practices before addressing higher stage practices.

In the ITIM framework, Stage 2 critical processes lay the foundation for sound IT investment processes by helping the agency to attain successful, predictable, and repeatable investment control processes at the project level. Specifically, Stage 2 encompasses building a sound investment management foundation by establishing basic capabilities for selecting new IT projects. It also involves developing the capability to control projects so that they finish predictably within established cost and schedule expectations and the capability to identify potential exposures to risk and put in place strategies to mitigate that risk. The basic selection processes established in Stage 2 lays the foundation for more mature selection capabilities in Stage 3, which represents a major step forward in maturity, in which the agency moves from project-centric processes to a portfolio approach, evaluating potential investments by how well they support the agency’s missions, strategies, and goals.

Stage 3 requires that an organization continually assess both proposed and ongoing projects as parts of a complete investment portfolio—an integrated and competing set of investment options. It focuses on establishing a consistent, well-defined perspective on the IT investment portfolio and maintaining mature, integrated selection (and reselection), control, and evaluation processes, which are to be evaluated during PIRs. This portfolio perspective allows decision makers to consider the interaction among investments and the contributions to organizational mission goals and strategies that could be made by alternative portfolio selections, rather than to focus exclusively on the balance between the costs and benefits of individual investments.

Stages 4 and 5 require the use of evaluation techniques to continuously improve both the investment portfolio and the investment processes in order to better achieve strategic outcomes. At Stage 4 maturity, an organization has the capacity to conduct IT succession activities and, therefore, can plan and implement the deselection of obsolete, high-risk, or low-value IT investments. An organization with Stage 5 maturity conducts proactive monitoring for breakthrough information technologies that will enable it to change and improve its business performance. Organizations implementing Stages 2 and 3 have in place the selection, control, and evaluation processes that are required by the Clinger-Cohen Act of 1996. Stages 4 and 5 define key attributes that are associated with the most capable organizations.

Figure 4 shows the five ITIM stages of maturity and the critical processes associated with each stage.

As defined by the model, each critical process consists of “key practices” that must be executed to implement the critical process.

HHS Has Established Many Key Practices for Managing Its Investments, but Has Provided Limited Guidance and Oversight to Component Agencies Processes

In order to have the capabilities to effectively manage IT investments, an agency, at a minimum, should, (1) build an investment foundation by putting basic, project-level control and selection practices in place (Stage 2 capabilities) and (2) manage its projects as a portfolio of investments, treating them as an integrated package of competing investment options and pursuing those that best meet the strategic goals, objectives, and mission of the agency (Stage 3 capabilities). These practices may be executed at various organizational levels of the agency, including at the component level. However, overall responsibility for their success remains at the department level. Therefore, at a minimum, the department should effectively oversee component agencies’ IT investment management processes.

HHS has executed 24 of the 38 key practices that the ITIM framework requires to build a foundation for IT investment management (Stage 2) and 8 of the 27 key practices required to manage investments as a portfolio (Stage 3). However, the department has only provided limited oversight of component agencies’ ITIM processes. Until HHS implements and oversees a stable investment management process throughout the department, it will lack essential management controls over all of its IT investments, and it will be unable to ensure that it is appropriately selecting, managing, and evaluating the mix of investments that will maximize returns to the organization, taking into account the appropriate level of risk.

HHS Has Established Over Half of the Foundational Practices Needed to Manage Its Investments

At the ITIM Stage 2 level of maturity, an organization has attained repeatable, successful IT project-level investment control processes and basic selection processes. Through these processes, the organization can identify expectation gaps early and take the appropriate steps to address them. According to the ITIM, critical processes at Stage 2 include (1) defining IT investment board operations, (2) identifying the business needs for each IT investment, (3) developing a basic process for selecting new IT proposals and reselecting ongoing investments, (4) developing project-level investment control processes, and (5) collecting information about existing investments to inform investment management decisions. Table 2 describes the purpose of each of these Stage 2 critical processes.

In the federal government, the agency head and the CIO are responsible for effectively managing information technology. The agency head, through the department-level CIO, is responsible for providing leadership and oversight for foundational critical processes by ensuring that written policies and procedures are established, repositories of information are created that support investment decision making, resources are allocated, responsibilities are assigned, and all the activities are properly carried out where they may be most effectively executed. In a large and diverse organization such as HHS, it is especially critical that the CIO create this structure and framework to ensure that the organization is effectively managing its investments at every level. This means that the CIO must ensure that component agencies have investment management processes in place that adequately support the department’s investment management process to make certain that funds are being expended on component agency investments that will fulfill mission needs.

Because of the management attention that has been given to IT investment management, the department has put in place over half of the key practices needed to establish the investment foundation. The department has satisfied all of the key practices associated with ensuring that projects and systems support organizational needs and meet users’ needs. It has satisfied most of the key practices associated with identifying and collecting investment information, selecting new proposals and reselecting ongoing investments, and instituting the department’s investment review board.

However, because of its limited involvement in overseeing component agency investments, the department has not executed any of the key practices related to providing investment oversight.

Table 3 summarizes the status of HHS’s critical processes for Stage 2 and shows how many key practices HHS has executed in managing its IT investments.

HHS Has Established an Investment Review Board, but It Is Operating without a Comprehensive Process Guide

The establishment of decision-making bodies or boards is a key component of the IT investment management process. At the Stage 2 level of maturity, organizations define one or more boards, provide resources to support the boards’ operations, and appoint members who have expertise in both operational and technical aspects of proposed investments. The boards should operate according to a written IT investment process guide that is tailored to the organization’s unique characteristics, thus ensuring that consistent and effective management practices are implemented across the organization. The organization selects board members to ensure that they are knowledgeable about policies and procedures for managing investments. Organizations at the Stage 2 level of maturity also take steps to ensure that executives and line managers support and carry out the decisions of the investment board. According to the ITIM, organizations should (1) use an investment management guide as an authoritative document to initiate and manage investment processes and (2) provide a comprehensive foundation for the policies and procedures that are developed for all of the other related processes. (The complete list of key practices is provided in table 4.)

The department has executed 5 of the 8 key practices for this critical process. The department established an IT investment review board as its corporate-level investment board that consists of senior officials, including the CIO and the Deputy Assistant Secretaries for Budget, Finance, and Performance & Planning. The board is adequately resourced, with most support being provided by the Office of the CIO, whose responsibilities include developing and modifying the department’s criteria for selecting, controlling, and evaluating potential and existing IT investments. In addition, the CIO Council reviews the enterprisewide investments for technical soundness and provides its recommendations to the board. The Critical Partners and Business Case Quality Review Team provide additional support to the board by reviewing and scoring most of their IT investments.

To ensure that the board’s decisions are carried out for enterprisewide investments, the ITIRB approves an accountability agreement document and business case that identify the benefits, costs, and schedule for the approved investments. The board then monitors the investments through the end of development. HHS requires the component agencies to follow a similar process in accordance with departmental policies and procedures. We verified that an accountability agreement document was signed and the business case identified performance expectations for the two enterprisewide IT investments we reviewed—Public Key Infrastructure and Enterprise Architecture initiatives. Additionally, the board has oversight of the development and maintenance of the documented IT investment process through the CPIC Reengineering/PMT Implementation Team, who provides investment management policy change recommendations to the board for approval.

Although HHS has implemented these key practices, it does not have a comprehensive organization-specific process guide to direct the operations of the investment board. While the Information Resources Management policy, guidelines, and standard operating procedures provide general guidance on the organization’s investment management process, they do not reflect the current investment management process. Moreover, they do not constitute an IT investment process guide because they do not sufficiently define the investment process. Specifically, the policies and procedures do not include information on the roles of the key players such as the CIO Council, Critical Partners, Business Case Quality Review Team, or the component agency investment review boards. In addition, they do not identify the manner in which investment board’s processes are to be coordinated with other key organizational plans and processes (such as the budget formulation process). HHS has recently drafted a revised investment management policy addressing many of these weaknesses; however, it has not been finalized, and HHS officials could not provide a final issuance date. Without a comprehensive investment management process guide, the department lacks the assurance that IT investment activities will be coordinated and performed in a consistent and cost- effective manner.

Moreover, while HHS has established an IT investment board, the board does not have business representation (that is, mission representation) from component agencies. Instead, Chief Information Officers represent the component agencies. According to HHS’s CIO, the membership of the board is adequate for carrying out the investment activities it currently performs—primarily focusing on enterprisewide IT investments. However, because allocating resources among major IT investments may require fundamental trade-offs among a multitude of business objectives, portfolio management decisions are essentially business decisions, and therefore require sufficient business representation on the board. Until the department adjusts its board membership to include business representation from component agencies, it will not have assurance that it includes those executives who are in the best position to make the full range of decisions needed to enable the agency to meet its mission most effectively, particularly as it begins to execute its full range of responsibility.

Finally, the HHS ITIRB is not operating according to its assigned authority and responsibility. The department’s investment management policy and the HHS ITIRB’s charter state that the board has oversight responsibility for both enterprisewide and a defined set of component agency IT investments, including projects that are high risk, crosscutting, and require review by the Office of Management and Budget. However, the board currently oversees only enterprisewide IT investments. According to HHS officials, the department has delegated authority to the component agencies to conduct investment reviews; however, the board does not have a mechanism in place for ensuring that component agencies are conducting such reviews in accordance with department policies and procedures. Until the board operates according to its assigned authority, it cannot ensure that component agency investments are properly aligned with the organization’s objectives or reviewed by the appropriate board.

Table 4 shows the rating for each key practice required to institute the investment board. Each of the “executed” ratings shown below represents instances where, on the basis of the evidence provided by HHS officials, we concluded that the specific key practices were executed by the organization.

Defining business needs for each IT project helps to ensure that projects and systems support an organization’s business needs and meet users’ needs. This critical process ensures that an organization’s business objectives and its IT management strategy are linked. According to the ITIM, effectively meeting business needs requires, among other things, (1) documenting business needs with stated goals and objectives; (2) identifying specific users and other beneficiaries of IT projects and systems; (3) providing adequate resources to ensure that projects and systems support the organization’s business needs and meet users’ needs; and (4) periodically evaluating the alignment of IT projects and systems with the organization’s strategic goals and objectives. (The complete list of key practices is provided in table 5.)

The department has in place all of the key practices for meeting business needs. Specifically, HHS has policy and procedures that call for business needs to be identified in the business case or the portfolio management tool’s Select forms for both proposed and ongoing enterprisewide and component agency IT projects. Resources devoted to ensuring that IT projects and systems support the organization’s business needs and meet users’ needs include the Business Case Quality Review Team, the Critical Partners, the portfolio management tool, and detailed procedures and associated templates for developing business cases. HHS’s specific business mission, with stated goals and objectives, is defined in the HHS Strategic Plan for fiscal years 2004 through 2009.

Further, HHS defines and documents business needs for both proposed and ongoing enterprisewide and component agency IT projects, and identifies users and other beneficiaries during its selection activities. In addition, according to HHS IT officials, end users participate in project management throughout the IT project’s life cycle. For the four projects we reviewed, we verified that business needs and specific users and other beneficiaries were identified and documented in the business case or in the Select forms within HHS’s portfolio management tool. In addition, end users are involved in project management throughout the life cycle of the enterprisewide investments. For example, users of HHS’s Public Key Infrastructure and Enterprise Architecture initiatives participate in project management through integrated project teams, which meet approximately once a month and are comprised of representatives from the component agencies. Because the department has executed all of the key practices associated with identifying business needs, it has increased confidence that its IT projects will meet both business needs and users’ needs.

Table 5 shows the rating for each key practice required to meet business needs and summarizes the evidence that supports these ratings.

Selecting new IT proposals and reselecting ongoing investments require a well-defined and disciplined process to provide the agency’s investment boards, business units, and developers with a common understanding of the process and the cost, benefit, schedule, and risk criteria that will be used both to select new projects and to reselect ongoing projects for continued funding. According to the ITIM, this critical process requires, among other things, (1) making funding decisions for new proposals according to an established process; (2) providing adequate resources for investment selection activities; (3) using a defined selection process to select new investments and reselect ongoing investments; (4) establishing criteria for analyzing, prioritizing, and selecting new IT investments and for reselecting ongoing investments; and (5) creating a process for ensuring that the criteria change as organizational objectives change. (The complete list of key practices is provided in table 6.)

HHS has executed 7 of the 10 key practices associated with selecting an investment. For example, resources devoted to selection activities include the Critical Partners, Business Case Quality Review Team, and portfolio management tool, which contains several forms for selecting IT projects and systems. HHS also has detailed procedures for using its portfolio management tool and developing business cases. The criteria for analyzing, prioritizing, selecting and reselecting new and ongoing investments address the President’s Management Agenda, HHS strategic goals, and IT strategic goals, value, and risk. They are incorporated into the department’s portfolio management tool and are reviewed by the investment review board and adjusted within the tool annually at the beginning of each budget cycle to reflect organizational objectives. This year, HHS added additional criteria—a quality score.

HHS uses its annual budget formulation process to select both enterprisewide and component agency proposed and ongoing IT investments. We verified that the four projects we reviewed were reselected by the department using the annual budget formulation process.

Although HHS has the above strengths, the department has not executed any of the practices associated with documenting policies and procedures. Specifically, HHS has not fully documented its process for selecting new IT proposals and reselecting ongoing IT investments. Although a number of documents address investment selection, they are not linked to provide decision makers with a clear understanding of the selection and reselection processes. In addition, they do not define the roles and responsibilities for all key players involved in these processes. Moreover, although the HHS Office of the CIO works directly with the department’s Office of the Budget, HHS does not have policies and procedures documenting the integration of funding with the process of selecting and reselecting investments. Until the department fully documents policies and procedures for selecting new IT proposals and reselecting ongoing IT investments, the department will not be adequately certain that it is consistently and objectively selecting and reselecting investments that best meet the needs and priorities of the department.

Table 6 shows the rating for each key practice required to select an investment and summarizes the evidence that supports these ratings.

An organization should effectively oversee its IT projects throughout all phases of their life cycles. Its investment board should observe each project’s performance and progress toward predefined cost and schedule expectations as well as each project’s anticipated benefits and risk exposure. This does not mean that a departmental board, such as the ITIRB, should micromanage each project to provide effective oversight; rather it means that the departmental board should be actively involved in all IT investments and proposals that are high cost or high risk or have significant scope and duration and at a minimum, should, have a mechanism for maintaining visibility of other investments. The board should also employ early warning systems that enable it to take corrective actions at the first sign of cost, schedule, and performance slippages. According to the ITIM, effective project oversight requires, among other things, (1) having written policies and procedures for management oversight; (2) developing and maintaining an approved management plan for each IT project; (3) making up-to-date cost and schedule data for each project available to the oversight boards; (4) having regular reviews by each investment board of each project’s performance against stated expectations; and (5) ensuring that corrective actions for each underperforming project are documented, agreed to, implemented, and tracked until the desired outcome is achieved. (The complete list of key practices is provided in table 7.)

The department has not executed any of the seven key practices associated with effective project oversight, primarily because of its limited role in overseeing component agency IT investments. Specifically, while the department has documented standard operating procedures and instructional memorandums for oversight of enterprisewide IT investments, they are not comprehensive in that they do not specify the board’s responsibilities for investment oversight; procedural rules for the ITIRB operations and decision making during project oversight; or policies and procedures for overseeing component agency IT investments.

The HHS ITIRB is currently performing regular reviews of enterprisewide IT projects and systems against stated expectations through reports that are available to decision makers on the HHS Intranet. However, the department is not regularly reviewing component agency investments that are high risk, crosscutting, and require review by the Office of Management and Budget, although their policy calls for it. The board also does not have a mechanism for maintaining visibility of other component agency investments.

The department delegates oversight of these investments to the component agencies but believes it is nonetheless effectively overseeing component agency investments through (1) reviews of these investments as part of the annual Critical Partner and Business Case Quality reviews performed during the annual selection process and the use of (2) earned value management data. Although the annual reviews may provide insight into the status of investments, they are not frequent enough to allow for timely identification of problems. Moreover, while HHS officials told us that staff responsible for collecting earned value management data on component agency investments share significant concerns about the data with the ITIRB, they did not have formal documentation clearly supporting this issue. In addition, formal procedures for elevating issues to the board have not been developed. In the absence of effective board oversight, HHS executives will not have the information they need to determine whether component agency projects are being developed on schedule and within budget. In addition, the department will run the risk that underperforming component agency projects will not be identified in time for corrective actions to be taken.

We verified that HHS provided oversight for the two enterprisewide investments, but had delegated oversight activities for the two component agency investments we reviewed.

Table 7 shows the rating for each key practice required to provide investment oversight and summarizes the evidence that supports these ratings.

To make good IT investment decisions, an organization must be able to acquire pertinent information about each investment and store that information in a retrievable format. During this critical process, an organization identifies its IT assets and creates a comprehensive repository of investment information. This repository provides information to investment decision makers to help them evaluate the potential impacts and opportunities created by proposed or continuing investments. It can provide insights into major IT cost and management drivers and trends. The repository can take many forms and need not be centrally located, but the collection method should, at a minimum, identify each IT investment and its associated components. This critical process may be satisfied by the information contained in the organization’s current enterprise architecture, augmented by additional information—such as financial information and information on risk and benefits—that the investment board may require to ensure that informed decisions are being made. According to the ITIM, effectively managing this repository requires, among other things, (1) developing written policies and procedures for identifying and collecting the information; (2) assigning responsibilities for ensuring that the information being collected meets the needs of the investment management process; (3) identifying IT projects and systems and collecting relevant information to support decisions about them; and (4) making the information easily accessible to decision makers and others. (The complete list of key practices is provided in table 8.)

HHS has executed 5 of the 6 key practices for capturing investment information. For example, the department has several documents that define the policies and procedures for identifying and collecting investment information in its repositories and also assign responsibility to the HHS CIO for ensuring that the information collected during project and systems identification meets the needs of the investment management process. HHS maintains a portfolio management tool, which serves as the primary repository for identifying and collecting information about both department and component agency IT projects and systems. The department’s portfolio management tool is easily accessible to decision makers at both the department and component level and the Office of the CIO has provided decision makers with various training manuals and guidance memorandums. In addition, the department also identifies and collects information about enterprisewide IT investments using its Intranet. Further, the department recently began collecting earned value information through spreadsheets on major HHS IT investments that compares planned and actual cost and schedule information. These repositories are easily accessible to the board members.

The key practice HHS has not executed has to do with the captured investment information not yet being used by the HHS ITIRB to fully support decisions about component agency investments. For example, the earned value investment data received from each component agency has not been used by the HHS ITIRB for control and evaluation decisions. According to agency officials, the department has recently begun monitoring the earned value data to identify investments that report cost and schedule variances and these officials acknowledge a need to formalize the process for doing so. Until HHS’s decision makers use the information in the repository to fully support the investment management process, it will be unable to effectively evaluate the impacts and opportunities created by proposed or continuing investments.

Table 8 shows the rating for each key practice required to capture investment information and summarizes the evidence that supports these ratings.

Once an agency has attained Stage 2 maturity, it needs to implement critical processes for managing its investments as a portfolio (Stage 3). An IT investment portfolio is an integrated, agencywide collection of investments that are assessed and managed collectively based on common criteria. Managing investments as a portfolio is a conscious, continuous, and proactive approach to allocating limited resources among an organization’s competing initiatives in light of the relative benefits expected from these investments. Taking an agencywide perspective enables an organization to consider its investments comprehensively, so that collectively the investments optimally address the organization’s missions, strategic goals, and objectives. Managing IT investments as a portfolio also allows an organization to determine its priorities and make decisions about which projects to fund and continue to fund based on analyses of the relative organizational value and risks of all projects, including projects that are proposed, under development, and in operation. Although investments may initially be organized into subordinate portfolios—based on, for example, business lines or life cycle stages—and managed by subordinate investment boards; they should ultimately be aggregated into this enterprise-level portfolio.

According to the ITIM framework, Stage 3 maturity includes (1) defining the portfolio criteria, (2) creating the portfolio, (3) evaluating the portfolio, and (4) conducting postimplementation reviews. Table 9 summarizes the purpose of each critical process in Stage 3.

HHS has executed 8 of the 27 key practices required by Stage 3. For example, the department’s core IT portfolio selection criteria, including cost, benefit, schedule, and risk are approved by the HHS ITIRB. In addition, the investment board examines the mix of new and ongoing investments and their respective data and analyses to select investments to fund. However, many key practices still need to be executed before HHS can effectively manage its IT investments from a portfolio perspective. For example, HHS has not addressed any of the key practices related to evaluating the portfolio or conducting PIRs. Until HHS fully implements the critical processes associated with managing its investments as a complete portfolio, it will not have the data it needs to make informed decisions about competing investments.

Table 10 summarizes the status of HHS’s critical processes for Stage 3, showing how many associated key practices it has executed.

To manage IT investments effectively, an organization needs to establish rules or “portfolio selection criteria” for determining how to allocate scarce funding to existing and proposed investments. Thus, developing an IT investment portfolio requires defining appropriate cost, benefit, schedule, and risk criteria with which to evaluate individual investments in the context of all other investments. To ensure that the organization’s strategic goals, objectives, and mission will be satisfied by its investments, the criteria should have an enterprisewide perspective. Further, if an organization’s mission or business needs and strategies change, criteria for selecting investments should be reexamined and modified as appropriate. Portfolio selection criteria should be disseminated throughout the organization to ensure that decisions concerning investments are made in a consistent manner and that this critical process is institutionalized. To achieve this result, project management personnel and others should be aware of the criteria and address the criteria in funding submissions for projects. Resources required for this critical process typically include the time and attention of executives involved in the process, adequate funding, and supporting tools. (The complete list of key practices is provided in table 11.)

The department has executed 5 of the 7 key practices for this critical process. For example, responsibility has been assigned to the HHS Lead Capital Planner for managing the development and modification of the IT portfolio selection criteria, and adequate resources have been committed for portfolio selection activities, including the Critical Partners, portfolio management tool project manager, and the Office of the CIO staff. Moreover, the project management personal and other stakeholders are aware of the portfolio selection criteria that are embedded into the department’s portfolio management tool and also contained within policies and procedures.

Finally, the HHS ITIRB approves the core IT selection criteria, including cost, benefit, schedule, and risk criteria, based on the organization’s mission, goals, strategies, and priorities. Beginning in fiscal year 2004, HHS began scoring and ranking approximately 80 percent of its IT investments against alignment, value, and risk criteria in order to determine a priority score, which is the sum of alignment, value, and risk criteria scores, weighted for relative importance. Similarly, for the fiscal year 2007 budget formulation process, HHS began collecting investment information on the business case quality, Critical Partner reviews, and cost and schedule variance to determine a quality score, which is the sum of the business case quality, Critical Partner reviews, and cost and schedule variance scores, weighted for relative importance. The HHS ITIRB evaluates and annually adjusts its portfolio selection criteria within the portfolio management tool.

Despite these important steps in defining portfolio selection criteria, weaknesses remain. The department has not developed policies or procedures for modifying the portfolio selection criteria to reflect changes to HHS mission, goals, strategies, and priorities. In addition, the HHS ITIRB began reviewing the IT portfolio selection criteria this year. However, the process for modifying portfolio selection criteria is not institutionalized because the process to do so was only used once and there are no documented policies and procedures to ensure that it will be used again. Until HHS defines and implements the practices required for defining the portfolio criteria definition, it will not have the tool it needs to select investments that support its mission, organizational strategies, and business priorities.

Table 11 shows the rating for each key practice required to define portfolio selection criteria and summarizes the evidence that supports these ratings.

At Stage 3, organizations create a portfolio of IT investments to ensure that IT investments are analyzed according to the organization’s portfolio selection criteria and to ensure that an optimal IT investment portfolio with manageable risks and returns is selected and funded. According to ITIM, creating the portfolio requires organizations to, among other things, document policies and procedures for analyzing, selecting, and maintaining the portfolio; provide adequate resources, including people, funding, and tools for creating the portfolio; and capture the information used to select, control, and evaluate the portfolio and maintain it for future reference. In creating the portfolio, the investment board must also (1) examine the mix of new and ongoing investments, and their respective data and analyses and select investments for funding and (2) approve or modify the performance expectations for the IT investments they have selected. (The complete list of key practices is provided in table 12.)

HHS has executed 3 of the 7 key practices associated with creating the portfolio. Beginning in fiscal year 2004, the department began to create a portfolio by using its portfolio management tool to collect cost, benefit, schedule, risk, strategic alignment, and enterprise architecture information on investments accounting for 80 percent of the dollar value of the HHS IT investment portfolio. Each component agency’s IT portfolio is displayed in priority order along with where each investment falls within the overall IT portfolio. Further, according to HHS IT officials, the agency has adequate resources for portfolio selection activities, including the Critical Partners, the portfolio management tool project manager, and the Office of the CIO staff. These officials also stated that HHS ITIRB members are also knowledgeable about the process of creating a portfolio.

Nevertheless, HHS has a number of significant weaknesses in the way it creates a portfolio. First, it does not have policies and procedures that sufficiently address this critical process. Although the department has policies and procedures for creating IT portfolio selection criteria, they lack policies and procedures for using these criteria to analyze, select, and maintain the investment portfolio. Second, even though the HHS ITIRB has quarterly reviews to compare project and system performance with expectations for enterprisewide IT investments, the board is not provided with information comparing the performance of component agency investments against expectations. In addition, the board approves or modifies the performance expectations for the enterprisewide IT investments it has selected, but does not regularly approve or modify the performance expectations for component agency IT investments or ensure that this is done. Moreover, as previously mentioned, investment information has not been used to fully support control and evaluate decisions for component agency investments. Unless HHS defines and implements the practices for creating a comprehensive portfolio of IT investments, it will not be able to determine whether it has selected the mix of investments that best meets its needs considering resource and funding constraints.

Table 12 shows the rating for each key practice required to create a portfolio and summarizes the evidence that supports these ratings.

This critical process builds upon the Stage 2 critical process, Providing Investment Oversight, by adding the elements of portfolio performance to an organization’s investment control capacity. Compared with less mature organizations, Stage 3 organizations will have the foundation they need to control the risks faced by each investment and to deliver benefits that are linked to mission performance. In addition, a Stage 3 organization will have the benefit of performance data generated by Stage 2 processes. Executive- level oversight of risk management outcomes and incremental benefit accumulation provides the organization with increased assurance that each IT investment will achieve the desired results. (The complete list of key practices is provided in table 13.)

HHS has not executed any of the seven key practices for evaluating a portfolio. It has yet to develop policies and procedures that address performance oversight from a portfolio perspective. Moreover, while the department annually reviews its portfolio as part of its selection process, it does not evaluate the investment portfolio on a continuing basis to assess its performance. Finally, the results of Providing Investment Oversight reviews from Stage 2 are important to this critical process. However, as previously mentioned, while the HHS ITIRB has oversight of enterprisewide investments, it does not regularly review a defined set of component agencies’ investments and maintain visibility of other investments. Although the department’s portfolio management tool has the ability to summarize performance metrics for each investment and quickly understand the status of each investment and any potential emerging problem area, the tool is currently only being used on an ad hoc basis to make portfolio oversight decisions. Defining and implementing processes to evaluate the performance of its entire portfolio would provide HHS with greater assurance that it is controlling the risks and achieving the benefits associated with the mix of investments it has selected.

Table 13 shows the rating for each key practice required to evaluate the portfolio and summarizes the evidence that supports these ratings.

The purpose of a PIR is to evaluate an investment after it has completed development (that is, after its transition from the implementation phase to the operations and maintenance phase) in order to validate actual investment results. This review is conducted to (1) examine differences between estimated and actual investment costs and benefits and possible ramifications for unplanned funding needs in the future and (2) extract “lessons learned” about the investment selection and control processes that can be used as the basis for management improvements. Similarly, PIRs should be conducted for investment projects that were terminated before completion, to readily identify potential management and process improvements. (The complete list of key practices is provided in table 14.)

HHS has not executed the six key practices for conducting PIRs. Although its policy calls for postimplementation reviews of IT investments that have recently completed implementation of the entire investment or a significant phase of the investment, the department does not have specific procedures for conducting such reviews, including specifying who conducts and participates in the PIR, what information is presented in a PIR, or how results are to be disseminated to decision makers. To date, HHS has conducted closeout reviews of two enterprisewide investments following their implementation; however, while these reports do cover investment cost expectations, they cannot be considered PIRs because the reports do not address general conclusions, lessons learned, or schedule deviations. Unless PIRs are conducted on a regular basis, HHS will not be able to effectively evaluate the results of its IT investments to determine whether continuation, modification, or termination of an IT investment would be necessary in order to meet stated HHS mission objectives.

Table 14 shows the rating for each key practice required to conduct PIRs and summarizes the evidence that supports these ratings.

The ability of a department-level CIO to effectively oversee IT investment management processes throughout the agency depends on the existence of appropriate management structures with adequate authorities and sufficient guidance. Under the Clinger-Cohen Act of 1996, the CIO of each agency is responsible for effectively managing all of the agency’s IT resources. To comply with the act, HHS designates its CIO to be responsible for ensuring that the component agencies are defining and implementing effective investment management processes that are appropriately aligned with the department’s processes.

Although each component agency has staff responsible for gathering, maintaining, and analyzing IT investment information, the HHS Office of the CIO has the responsibility to define and implement overall HHS IT investment management practices, and monitor component agency investment management practices to ensure a cohesive departmental process and the capability exists to carry out the process. In accordance with this, the department’s investment management policies and guidelines state that the component agencies are to establish and manage investment management processes and governance structures that are aligned with the department’s policies and procedures. However, as mentioned in previous sections, the department’s investment management policies and procedures have several weaknesses. For example, HHS does not have a set of documented procedures that provide decision makers with a clear understanding of the selection and reselection process.

Moreover, HHS currently has no structured mechanism in place to ensure that the component agencies are adhering to the department’s policies and procedures. According to HHS officials, the CIO has the authority to audit a component agencies IT investment management process. However, they were unable to provide us evidence of having performed any such audits. These officials also stated that the department’s portfolio management tool is another method that will enable HHS to oversee component-level investment management processes. However, since not all component agencies are using the portfolio management tool to individually make select, control, and evaluate decisions, its usefulness in this regard is limited. Until the department develops a mechanism for ensuring that component agencies define and implement investment management processes that align with those of the department, it is running the risk that effective processes are being institutionalized at both the department and the component agency level. In addition, the department will be unable to ensure that it is optimizing its investments in IT and effectively assessing and managing the risks of these investments.

HHS Does Not Have a Plan to Coordinate and Guide Improvement Efforts

HHS has initiated several efforts to improve its investment management process. Specifically, it has drafted a revised investment management guide that addresses the weaknesses with current guidance that we identify in this report. In addition, in February 2005, HHS incorporated capabilities into its portfolio management tool to enhance performance of control and evaluate functions. Specifically, the tool now has the capabilities to produce (1) scorecards to provide data for each investment in a portfolio, allowing cross investment comparisons on data elements collected; (2) investor maps to provide a graphical depiction of a portfolio in terms of up to six data categories, with the ability to show target and actual values; and (3) a workbook module to track the identification and resolution of issues that may arise regarding the management of an investment or set of investments.

Although HHS has initiated these efforts, they only fully address 2 of the 14 Stage 2 key practices the department did not execute.

The draft investment management guidance, when finalized, will address weaknesses associated with one of the key practices for instituting the investment board by reflecting the current management process, including information on the roles of key working groups involved in the organization’s IT investment processes, and identifying the manner in which investments board’s processes are to be coordinated with other key organizational plans and processes. The guidance will also address the integration of the funding and selection processes, a key practice the department has not executed that is associated with selecting an investment.

The enhanced portfolio management tool capabilities will enhance the department’s ability to oversee investments’ performance and position the board to perform portfolio evaluation activities, but they will not fully address any of the weaknesses we identify.

HHS has not coordinated these and additional efforts that would address the weaknesses we identify in this report in a comprehensive plan that (1) specifies measurable goals, objectives, and milestones; (2) specifies needed resources; (3) assigns clear responsibility and accountability for accomplishing tasks; and (4) is approved by senior management. We have previously reported that such a plan is instrumental in helping agencies coordinate and guide improvement efforts. Until HHS develops a plan that would allow for the systematic prioritization, sequencing, and evaluation of improvement efforts, the agency risks not being able to effectively establish the mature investment management processes that result in greater certainty about the outcomes of future IT investments.

Conclusions

Because of the attention that has been given to investment management, HHS has established several of the practices needed to effectively manage its investments. These practices have strengthened the department’s basic capabilities for selecting and controlling projects and begun to equip the department with the capabilities it needs to make informed decisions about competing investments. However, several significant weaknesses remain in the foundational practices needed to manage individual investments, the portfolio-level investments needed to manage investments as a collection, and in the level of guidance and oversight provided to component agency investment management processes. These weaknesses hamper the department’s ability to ensure that it is managing the mix of investments that will maximize returns to the organization, taking into account the appropriate level of risk.

Critical to HHS’s success, going forward will be the development of an implementation plan that (1) is based on an assessment of strengths and weaknesses; (2) specifies measurable goals, objectives, and milestones; (3) specifies needed resources; (4) assigns clear responsibility and accountability for accomplishing tasks; and (5) is approved by senior management. Although the department has initiated improvement efforts, it has not developed a comprehensive plan to guide these and other efforts needed to improve its investment management process. Without such a plan and procedures for implementing it, it is unlikely that the department will effectively establish mature investment management capability. As a result, HHS will continue to be challenged in its ability to make informed and prudent investment decisions in managing its annual multibillion-dollar IT budget.

Recommendations for Executive Action

To strengthen HHS’s investment management capability and address the weaknesses discussed in this report, we recommend that the Secretary of the Department of Health and Human Services direct the Chief Information Officer to develop and implement a plan for improving the department’s IT investment management processes. The plan should address the weaknesses described in this report, beginning with those we identified in our Stage 2 analysis and continuing with those we identified in our Stage 3 analysis. The plan should, at a minimum, provide for accomplishing the following: Develop comprehensive guidance and additional supporting guidance that defines and describes the complete investment management process, unifies existing processes enterprisewide, reflects changes in processes as they occur; define the operations and decision-making processes of the HHS investment review board and other management entities, such as the component agencies, involved in managing IT investments.

Ensure that HHS’s investment review board’s membership includes business representation of its component agencies as it begins to execute its full range of responsibilities.

Develop well-defined and disciplined written procedures that outline the process for selecting new IT proposals, reselecting ongoing IT investments, and integrating funding with the process of selecting an investment.

Establish a process for the investment board to regularly review and track the performance of a defined set of component agency IT systems against expectations, and take corrective actions when these expectations are not being met; and establish a mechanism for maintaining visibility into other investments.

Develop and implement policies and procedures for modifying IT portfolio selection criteria.

Develop policies and procedures for using the portfolio selection criteria to create its portfolio.

Develop, review, and modify criteria for assessing portfolio performance at regular intervals to reflect current performance expectations.

Define and implement processes for carrying out PIRs for all IT investments.

We also recommend that the HHS Secretary direct the CIO to ensure that the plan draws together ongoing efforts and additional efforts that are needed to address the weaknesses identified in this report. The plan should also (1) specify measurable goals, objectives, and milestones; (2) specify needed resources; (3) assign clear responsibility and accountability for accomplishing tasks; and (4) be approved by senior management.

Finally, to improve the department oversight of its component agency investment management process, we are recommending that the HHS Secretary direct the HHS CIO to establish a mechanism for ensuring component agencies define and implement investment management processes that are aligned with those of the department.

Agency Comments

The Department of Health and Human Services’s Inspector General provided written comments on a draft of this report (reprinted in app. II). In these comments, HHS generally agreed with our findings and recommendations and stated that the report represented a fair assessment of the department’s progress in IT investment management. The department added that it will leverage the report in its efforts to improve its investment management processes.

HHS expressed differing perspectives on the inclusion of component agency business representation on the investment review board and the performance of postimplementation reviews. Specifically, regarding business representation on the board, the department commented that it used a hierarchy of investment reviews (with the first review occurring at the component agency) combined with ITIRB members representing mission support areas, such as Finance, Acquisition, and Human Resources, to provide a structure for making the business decisions regarding the department’s investments. We disagree with the department that this arrangement provides an adequate structure for managing the department’s investments. Because allocating resources among major IT investments may require fundamental trade-offs among a multitude of business objectives, portfolio management decisions are essentially business decisions, and therefore require sufficient business representation on the board. CIOs and executives responsible for mission- support functions do not constitute sufficient business representation because, by virtue of their responsibilities, they are not in the best position to make business decisions. Portfolio management decisions are better made by executives with business line decision-making authority.

Regarding PIRs, HHS commented that it was currently informally performing them by conducting closeout reviews of recently implemented investments and annual reviews of systems in operations and maintenance. PIRs are conducted to determine whether cost, benefit, schedule, and risk expectations that were set for investments were achieved and develop lessons learned about the investment selection and control processes that can be used as the basis for management improvements. However, neither the closeout reviews, nor the reviews of systems in operations and maintenance, are addressing all these elements. Specifically, as we stated in our report, the closeout reviews do not address schedule deviations, determine whether the benefits were achieved, or identify lessons learned. In addition, the reviews of projects in operations and maintenance do not capture the benefits realized or identify lessons learned.

Commenting on departmental-level oversight of component agency investments, HHS stated that it agrees with our recommendation to improve its oversight of component agency investments. It stated that it would use a number of mechanisms to do this, including performing audits to ensure alignment of component agency’s processes with those of the department, using earned value management data to identify potential performance problems with most investments, and directly reviewing investments determined to be of high priority. We agree with HHS that these steps would help address some of the weaknesses in project oversight that we identify in this report.

As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from the date of this report. At that time, we will send copies to other interested congressional committees, the Secretary of Health and Human Services, and other interested parties. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

Should you or your offices have questions on matters discussed in this report, please contact me at (202) 512-9286 or pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Objectives, Scope, and Methodology

The objectives of our review were to (1) assess the Department of Health and Human Services’s capabilities for managing its IT investments and (2) determine any plans HHS might have for improving those capabilities.

To address our first objective, we reviewed the results of the department’s self-assessment of Stages 2 and 3 practices using our ITIM framework and validated and updated the results of the self-assessment through document reviews and interviews with officials. We reviewed written policies, procedures, and guidance and other documentation providing evidence of executed practices, including HHS’s Capital Planning and Investment Control Policy and Guidelines, standard operating procedures, portfolio management tool training manuals, and various instructional memorandums. We also reviewed the HHS ITIRB meeting materials, including quarterly status reports, meeting minutes, and records of decisions. We did not assess progress in establishing the capabilities found in Stages 4 and 5 because the department acknowledged that it had not executed any of the key practices in higher maturity stages. In addition, we conducted interviews with officials from the Office of the CIO, whose main responsibility is to oversee and ensure that HHS’s IT investment management process is implemented and followed to determine the level of oversight and guidance the department is providing to its component agencies. We also interviewed the Centers for Medicare & Medicaid’s Director for Investment Tracking and Assessment to determine the level of investment management guidance and oversight that is provided by the department.

As part of our analysis, we selected two HHS enterprisewide and two component agency IT projects as case studies to verify that the critical processes and key practices were being applied. The projects selected (1) are recognized as major systems, (2) were in different life cycle phases, (3) represent a mix of headquarters and component agency investments, (4) support different functional areas, and (5) required different levels of funding. The four projects are described below: HHS Public Key Infrastructure—This project supports digital signatures and other public key-enabled security services; it is intended to be the underlying architecture to support secure transmissions of electronic communication, such as encrypted email, by linking a digital key to a specific person, and issues and manages digital certificates. The intent of the project is to provide an identity proofing process that is both fast and certificate authority neutral. It is an agencywide strategic initiative that provides security services. The project is a major enterprisewide investment and is in the operations and maintenance phase. The project has a planned completion date of July 2011 and is estimated to spend $7.7 million for fiscal year 2006.

HHS Enterprise Architecture Initiative—This initiative is to provide the overall framework for planning and managing the technology-supported information assets of HHS and give the department the ability to identify data and process redundancies and inefficiencies in its information systems. The program’s objectives focus on development of operational policies and support that enable identification, analysis and ongoing management of the business, and information and related technology architectures. It is to provide leadership, direction, and support to HHS’s component agencies in planning and implementing information systems to support required business processes. As of fiscal year 2005, the initiative is a major enterprisewide program investment and is estimated to spend $15.0 million for fiscal year 2006.

National Institutes of Health’s Electronic Research Administration— This initiative is the National Institutes of Health’s infrastructure for conducting interactive electronic transactions for the receipt, review, monitoring, and administration of grant awards to biomedical investigators worldwide. It is to provide the technology capabilities for the agency to efficiently and effectively perform grants administration functions. The system is to provide end-to-end support of the grants administration process, including receipt of applications, review and selection of grantees, financial and progress reporting, issuance of final reports and grant dole-out, invention reporting, and interface with accounting systems. It is a major component agency investment and is expected to have a useful life of 13 years. The project is estimated to spend $42.1 million for fiscal year 2006.

Food and Drug Administration’s Mission Accomplishment and Regulatory Compliance Services—This program is a comprehensive redesign and reengineering of core mission-critical systems at the agency, including the Field Accomplishments and Compliance Tracking System and the Operation and Administration Support System. The first of these systems is to support the investigation, tracking of compliance, and laboratory operations related to domestic operations under the agency’s purview; the second is to primarily support the review and decision-making process of products imported into the United States. Both are legacy systems that execute on client-server platforms; while currently viable, the current systems cannot address many of the business needs due to the exponential growth in functionality on a rigid platform that was not designed to support the extent of change that has been required. The Mission Accomplishment and Regulatory Compliance Services is a major component agency investment and is expected to move to production in September 2007 and have a useful life of 10 years. The project is estimated to spend $10.2 million for fiscal year 2006.

For these projects, we reviewed project management documentation, such as business cases, status reports, and meeting minutes. We also interviewed officials from the Office of the CIO for the two component agency investments and the project managers for the two HHS enterprisewide projects.

We compared the evidence collected from our document reviews and interviews to the key practices in ITIM. We rated the key practices as “executed” on the basis of whether the agency demonstrated (by providing evidence of performance) that it had met the criteria of the key practice. A key practice was rated as “not executed” when we found insufficient evidence of a practice during the review or when we determined that there were significant weaknesses in HHS’s execution of the key practice. In addition, HHS was provided the opportunity to produce evidence for key practices rated as “not executed.”

To address our second objective, we obtained and evaluated documents showing what management actions had been taken and what initiatives had been planned by the agency. This documentation included the Policy Advisory Board charter, draft investment management policies and procedures, as well as procedures and guidance for control and evaluate functionalities within HHS’s portfolio management tool. We also interviewed officials from the Office of the CIO to determine efforts undertaken to improve IT investment management processes.

We conducted our work at HHS headquarters in Washington, D.C., from January through September 2005, in accordance with generally accepted government auditing standards.

Comments from the Department of Health and Human Services

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the person named above, Neil Doherty, Joanne Fiorino, Sabine Paul, Nik Rapelje, Niti Tandon, and Amos Tevelow made key contributions to this report.