United States General Accounting Office

GAO Report to Congressional Requesters

July 1998 VA INFORMATION

TECHNOLOGY

Improvements Needed

to Implement

Legislative Reforms

GAO/AIMD-98-154

GAO United States

General Accounting Office

Washington, D.C. 20548

Accounting and Information

Management Division

B-278709 

July 7, 1998

The Honorable Arlen Specter

Chairman, Committee on Veterans’ Affairs

United States Senate

The Honorable John D. Rockefeller IV

Ranking Minority Member

Committee on Veterans’ Affairs

United States Senate

The Honorable Bob Stump

Chairman, Committee on Veterans’ Affairs

House of Representatives

The Honorable Lane Evans

Ranking Minority Member

Committee on Veterans’ Affairs

House of Representatives

In your September 1997 letter, you requested that we review and evaluate

how well the Department of Veterans Affairs (VA) is selecting and

managing its information technology investments and identify specific

areas where improvements could be made. Information technology

represents about $1 billion of VA’s fiscal year 1999 budget request of

$43 billion. This report provides the results of our review.

The Clinger-Cohen Act and other related legislative reforms provide

guidance on how VA and other agencies should plan, manage, and acquire

information technology as part of their overall information resources

management responsibilities. They also require federal agencies to appoint

Chief Information Officers (CIOs) responsible for providing leadership in

the acquisition and management of information resources. Accordingly, as

agreed with your offices, our specific objectives were to examine how VA

has implemented the following specific provisions of the Clinger-Cohen

Act and other legislative reforms: reengineering business processes before

acquiring information technology; completing an integrated information

Page 1 GAO/AIMD-98-154 VA Information Technology
B-278709 

technology architecture;1 institutionalizing a disciplined information

technology investment decision-making process; and appointing an agency

CIO.

Results in Brief VA has not fully implemented critical provisions of the Clinger-Cohen Act

and other legislative reforms. Although VA has taken some initial steps, it

has not adequately implemented these legislative reforms. Specifically, the

Clinger-Cohen Act requires agencies to analyze their mission-related and

administrative processes, and on the basis of this analysis, revise and

improve these processes before making significant investments in

supporting information technology. Although GAO’s business process

reengineering guide2

 states that agencies should have an overall business

process improvement strategy to accomplish reengineering, VA has not

developed such a strategy. VA also has not yet defined the departmentwide

integrated information technology architecture needed to efficiently utilize

information systems across the department.

In addition, VA has not institutionalized a disciplined process for selecting,

controlling, and evaluating information technology as investments as

required by the Clinger-Cohen Act. Specifically, VA decisionmakers did not

have current and complete information such as cost, benefit, schedule,

risk, and performance data at the project level, which is essential to

making sound investment decisions. In addition, VA’s process for

controlling and evaluating its investment portfolio is incomplete and, as a

result, decisionmakers do not have the information needed to (1) detect or

avoid problems early and (2) improve VA’s investment process. As a

consequence, the department does not know whether it is making the right

investments, how to control these investments effectively, or whether

these investments have provided mission-related benefits in excess of

their costs.

Finally, although the Clinger-Cohen Act requires agencies’ CIOs to have

information management as their primary duty, the responsibilities of VA’s

CIO are not limited primarily to information management. Instead, the CIO

1

An integrated information technology architecture is a blueprint, consisting of logical and technical

components, to guide and constrain the development and evolution of a collection of related systems.

At the logical level, the architecture provides a high-level description of an organization’s mission, the

business functions being performed and the relationships among the functions, the information needed

to perform the functions, and the flow of information among functions. At the technical level, the

architecture provides the rules and standards needed to ensure that the interrelated systems are built

to be interoperable and maintainable. These include specifications of critical aspects of component

systems’ hardware, software, communication, data, security, and performance characteristics.

2

Business Process Reengineering Assessment Guide (GAO/AIMD-10.1.15, April 1997, Version 3).

Page 2 GAO/AIMD-98-154 VA Information Technology
B-278709 

also functions as the department’s Assistant Secretary for Management

and Chief Financial Officer (CFO). As a result, information technology

issues are not addressed promptly.

In commenting on a draft of this report, VA concurred with all six

recommendations. VA stated, among other things, that it will establish the

position of Assistant Secretary to serve as the department’s CIO, reporting

directly to the Secretary on all information resources issues.

Background VA comprises three major components: the Veterans Benefits

Administration (VBA), the Veterans Health Administration (VHA), and the

National Cemetery System (NCS).3 VA’s mission is “to administer the laws

providing benefits and other services to veterans and dependents. . . .” The

department’s vision is to be a more customer-focused organization,

functioning as “One VA.” This vision stemmed from the recognition that

veterans think of VA as a single entity, but often encounter a confusing,

bureaucratic maze of uncoordinated programs that put them through

repetitive and frustrating administrative procedures and delays. The “One

VA” vision is to create versatile new ways for veterans to obtain services

and information by streamlining interactions with its customers and

integrating information technology resources to enable VA employees to

help customers more quickly and effectively. This will require modifying

or replacing separate information systems with integrated systems using

common standards to share information across VA programs and with

external partner organizations, such as the Department of Defense.

Information technology accounted for approximately $1 billion of VA’s

fiscal year 1999 budget request of $43 billion. Of the $1 billion, about

$847 million, $146 million, and $5 million were for VHA, VBA, and NCS,

respectively.

Over the past several years, we have identified weaknesses in VA’s efforts

to modernize its operations and manage its information technology

resources. As we reported in 1992, VBA’s procurement of hardware was not

supported by a defined information architecture, thereby increasing the

risk of developing systems that would not work as intended.4

 In June 1996,

we testified that VBA needed to develop a much improved investment

3

VBA provides nonmedical benefits to veterans and their dependents; VHA provides services through

the nation’s largest health-care system; and NCS provides burial services in 115 national cemeteries.

4

Veterans Benefits: Acquisition of Information Resources for Modernization Is Premature

(GAO/IMTEC-93-6, November 4, 1992).

Page 3 GAO/AIMD-98-154 VA Information Technology
B-278709 

strategy for selecting and managing information technology projects in a

more disciplined, businesslike manner.5

 In January 1998, we reported6 that

while VA made significant progress in preparing a strategic plan, dated

September 30, 1997, the plan needed improvement in four major areas:

(1) development of results-oriented goals, (2) descriptions of how the

goals are to be achieved, (3) discussion of external factors, and

(4) discussion of coordination efforts with other agencies. Finally, in

October 1997, we testified on the importance of having strong CIOs at

major federal agencies, such as VA, to bring about much-needed reforms in

the government’s management of information technology.7

 In addition, a

panel of the National Academy of Public Administration reported in

August 1997, that VBA lacks strategic planning and management

capabilities that are necessary for leadership to define where the

organization wants to be, enable development of specific operational plans

for getting there, and provide a set of coordinating and integrating

capacities for implementing planned initiatives.8

Recognizing the need to better manage information technology, recent

legislative reforms—the Clinger-Cohen Act of 1996,9

 the Paperwork

Reduction Act of 1995, and the Federal Acquisition Streamlining Act of

1994—provide guidance to federal agencies on how to plan, manage, and

acquire information technology as part of their overall information

resources management responsibilities. These legislative reforms highlight

the need for business process reengineering, integrated architectures,

investment processes, and CIOs to help with major information resource

management responsibilities.

5

Veterans Benefits Modernization: Management and Technical Weaknesses Must Be Overcome If

Modernization Is To Succeed (GAO/T-AIMD-96-103, June 19, 1996).

6

Managing for Results: Agencies’ Annual Performance Plans Can Help Address Strategic Planning

Challenges (GAO/GGD-98-44, January 30, 1998), app. XV, “Observations on the Department of Veterans

Affairs’ Strategic Plan.”

7

Chief Information Officers: Ensuring Strong Leadership and an Effective Council (GAO/T-AIMD-98-22,

October 27, 1997).

8

Management of Compensation and Pension Benefits Claim Processes for Veterans—A Report by a

Panel of the National Academy of Public Administration for Congress and the Department of Veterans

Affairs, August 1997.

9

The Omnibus Consolidated Appropriations Act, 1997, renamed both the Federal Acquisition Reform

Act of 1996 and the Information Technology Management Reform Act of 1996 as the Clinger-Cohen Act

of 1996.

Page 4 GAO/AIMD-98-154 VA Information Technology
B-278709 

Scope and

Methodology

In assessing VA’s implementation of the Clinger-Cohen Act and other

legislative reforms, we reviewed and analyzed numerous documents

pertaining to VA’s business process reengineering, integrated information

technology architecture, information technology investment

decision-making process, and appointment of an agency CIO. These

documents include VA’s draft entitled One VA: Vision of Information

Technology Enhanced Customer Service, dated January 22, 1998; OMB’s

Memorandum on Information Technology Architectures, dated June 18,

1997; VA’s draft FY 1999 Department Capital Plan, dated October 1997; and

VA’s April 1997 Progress Report on the Department of Veterans Affairs CIO

Program.

We discussed VA’s implementation of the Clinger-Cohen Act and other

legislative reforms with Office of Management and Budget (OMB) officials

and with various VA headquarters and component officials, including the

Offices of the CIO, Information Resources Management, and Policy and

Planning. We also interviewed a VA representative of the contractor

responsible for developing VA’s information technology vision document.

We used our investment guide10 to evaluate and assess VA’s information

technology investment process.

We performed our work from October 1997 through April 1998, in

accordance with generally accepted government auditing standards. We

requested comments on a draft of this report from the Secretary of

Veterans Affairs and they are reprinted in appendix I. More details of our

objectives, scope, and methodology are included as appendix II.

VA Lacks Overall

Business Process

Improvement Strategy

The Clinger-Cohen Act requires agency heads to analyze the missions of

the agency and, on the basis of this analysis, revise and improve the

agency’s mission-related and administrative processes before making

significant investments in supporting information technology. Specifically,

agencies should maximize the potential of technology to improve

performance, rather than simply automating inefficient processes.

According to our business process reengineering guide,11 an agency should

have an overall business process improvement strategy that provides a

means to coordinate and integrate the various reengineering and

improvement projects, set priorities, and make appropriate budget

decisions.

10Assessing Risks and Returns: A Guide for Evaluating Federal Agencies’ IT Investment

Decision-making (GAO/AIMD-10.1.13, February 1997).

11GAO/AIMD-10.1.15, April 1997, Version 3.

Page 5 GAO/AIMD-98-154 VA Information Technology
B-278709 

VA has not analyzed its business processes in terms of implementing its

“One VA” vision. In addition, it does not have a departmentwide business

process improvement strategy specifying what reengineering and

improvement projects are needed, how they are related, and how they are

prioritized. VA’s Directive 6000 instructs administration heads, assistant

secretaries, and other key officials to apply sound business process

improvement or business reengineering methods to enhance the benefits

of information technology, but the directive does not provide the guidance

needed to accomplish this departmentwide effort. Specifically, VA’s

strategy does not identify needed reengineering and improvement

projects, describe how they are interrelated, determine the order in which

they will be pursued, and define specific goals, time frames, resource

requirements, and key participants for each.

In the absence of a departmentwide strategy, VA components are

proceeding with separate, uncoordinated efforts, which undermines the

department’s “One VA” vision. For example, both VBA and VHA are building

information centers which enable callers, through the use of an 800

number, to obtain information on general benefits and basic services.

However, these efforts are not currently coordinated. As a result, both

projects are unnecessarily providing the same functionality. A senior VA

official acknowledged that VA should not be building two separate

information center systems, and that doing so is not consistent with the

“One VA” vision.

Similarly, VA and its components have not adequately coordinated and

integrated their business process improvement efforts for the development

of a master veteran record (MVR). This departmentwide project is intended

to electronically link VHA, VBA, NCS, and Board of Veterans’ Appeals

information systems and databases to share vital information, such as

death notification, change of address, representation and family status

about veterans. However, according to the project manager, this project is

experiencing difficulties because VBA will not fund a segment of the project

necessary to establish a link between VBA’s compensation and pension

program—VBA’s largest program—and other VA components. As a result,

VBA is not in a position to obtain timely information, such as death

notifications, which can result in overpayments to veterans.

VA has acknowledged the need to develop a strategy to achieve the “One

VA” vision and has hired a contractor to analyze the department’s business

plans and information technology projects to determine how well they fit

within the vision. However, to date, VA has not committed to when it will

Page 6 GAO/AIMD-98-154 VA Information Technology
B-278709 

have an overall business process improvement strategy to accomplish

reengineering.

VA Lacks an

Integrated

Information

Technology

Architecture

The Clinger-Cohen Act and recent OMB guidelines require agency CIOs to

implement an architecture to provide a framework for evolving or

maintaining existing information technology, and for acquiring new

information technology to achieve the agency’s strategic and information

technology goals. Leading organizations both in the private sector and in

government use systems architectures to guide mission-critical systems

development and to ensure the appropriate integration of information

systems through common standards.12

Despite the importance of doing so, VA and its components have yet to

define a departmentwide integrated architecture. For example, as we

reported in May 1997, VBA did not have a complete, integrated systems

architecture to help guide its new systems development activities.13 We

therefore recommended that VBA develop such an architecture, including a

security architecture and performance characteristics and standards. VA

concurred with our recommendation.

To formulate an approach for developing an integrated architecture, VA in

March 1997 established an architecture team consisting of representatives

from VA’s Office of Information Resources Management, VBA and VHA. This

team issued a report to the VA CIO Council14 in May 1997 adopting the

National Institute of Standards and Technology (NIST)15 five-layer model

for its departmentwide information technology architecture. The five

layers—business processes, information flows and relationships,

applications processing, data descriptions, and technology—provide a

framework for defining an information technology architecture. VA can use

this model to help it document the baseline architecture, identify a target

architecture, and develop a migration plan showing how the department

will make the necessary transition from its existing architecture to the

target architecture.

12Executive Guide: Improving Mission Performance Through Strategic Information Management and

Technology—Learning From Leading Organizations (GAO/AIMD-94-115, May 1994).

13Veterans Benefits Computer Systems: Risks of VBA’s Year-2000 Efforts (GAO/AIMD-97-79, May 30,

1997).

14VA’s CIO Council is comprised of VA’s Assistant Secretary for Management and his Deputy CIO, VA’s

Assistant Secretary for Policy and Planning, VBA’s CIO, VHA’s CIO, NCS’ Director of Operations

Support, and Board of Veterans’ Appeals’ Director of Management and Administration.

15NIST Special Publication 500-167, “Information Management Directions: The Information Challenge.”

Page 7 GAO/AIMD-98-154 VA Information Technology
B-278709 

Despite the VA architecture team’s efforts, VA does not yet have a

departmentwide target architecture and migration plan. While a baseline

architecture has been established, VA has not addressed key aspects of the

target architecture, such as information flows, data descriptions, and

common technical standards that would apply to VBA, VHA, and NCS.

According to VA’s CIO, the department has not addressed these aspects

because it is waiting for the Strategic Management Steering Committee16

to take a position on the proposal to develop a departmentwide target

architecture and establish a program office to implement the architecture

and related efforts, including business process reengineering and

customer service improvements.

VA Has Not

Institutionalized a

Disciplined

Information

Technology

Investment Process

While information technology represents $625 million or 80 percent of VA’s

proposed $786 million capital investment budget for fiscal year 1999, the

department lacks an effective process for selecting, controlling, and

evaluating its information technology projects as investments. VA’s newly

developed selection process, used for the first time in the fiscal year 1999

budget cycle, is incomplete, undisciplined, and does not satisfy the

selection process requirements specified in the Clinger-Cohen Act. For

example, decisionmakers did not have adequate information pertaining to

project cost, benefits, risk, and performance measures to make

well-informed decisions. Also, VA’s process for monitoring and controlling

its investment portfolio was incomplete and provided little information to

VA decisionmakers reviewing ongoing projects. Finally, VA’s process for

evaluating completed projects did not include reviews to (1) determine the

causes of major differences between actual and expected results in terms

of cost, schedule, and performance, and (2) revise investment processes

on the basis of lessons learned. As a result of these weaknesses, the

department does not know whether it is making the right investments,

how to control these investments effectively, or whether these

investments have provided mission-related benefits in excess of their

costs.

The Clinger-Cohen Act

Prescribes an Investment

Management Approach

The Clinger-Cohen Act requires agency heads to implement an approach

for maximizing the value and assessing and managing the risks of

information technology investments. It stipulates that this approach

should be integrated with the agency’s budget, financial, and program

management processes.

16The members of this committee include the Chief of Staff, the Deputy Under Secretaries for Health

and Benefits, the CIOs for VHA and VBA, the CFOs for VHA and VBA, the Assistant Secretary for

Policy and Planning, and the NCS Directors for Field Operations and Operations Support.

Page 8 GAO/AIMD-98-154 VA Information Technology
B-278709 

According to our investment guide,17 an information technology

investment process is an integrated approach that provides for disciplined,

data-driven identification, selection, control, life-cycle management, and

evaluation of information technology investments. Information from one

phase is used to support activities in the other phases. When identifying

information technology investments to be managed at the department

level, leading organizations use criteria that include (1) high-dollar,

high-risk projects, (2) cross-functional projects (two or more

organizational units benefitting from the projects), and (3) common

infrastructure support, such as hardware and telecommunications.

Once selected, information technology projects in the investment portfolio

are consistently controlled and managed through progress reviews at key

milestones in a project’s life cycle. Progress reviews should include

assessing deliverables, technical issues, schedule, costs, and risks. Finally,

once a project has been fully implemented, a post-implementation review

or evaluation should be conducted, comparing actuals against estimates in

order to assess performance and identify areas where future

decision-making can be improved.

VA defined a new decision-making process for information technology

investments and conducted a “dry run” in formulating the fiscal year 1999

budget. As depicted in figure 1, this process began with VA’s staff offices

and components submitting information packages about their capital

investment projects to an Information Technology Strategic Planning

Working Group. This group—composed of project decisionmakers

representing the VA Central Office’s Office of Information Resources

Management, Office of Financial Management, VBA, VHA, Board of Veterans’

Appeals, Office of Planning and Policy, and NCS—was created to assist VA’s

CIO Council in its review of prospective projects for funding. The working

group used risk and return criteria, which included factors such as

investment size, project longevity, technical risk, business impact on

mission, and customer needs.

17GAO/AIMD-10.1.13, February 1997.

Page 9 GAO/AIMD-98-154 VA Information Technology
B-278709 

Figure 1: VA’s New Investment

Decision-Making Process

VA's Staff Offices and Components

Information Technology

Strategic Planning

Working Group

VA CIO Council

VA Capital Investment Panel

VA Capital Investment Board

The working group then forwarded the scored and ranked projects to VA’s

CIO Council, which is responsible for ensuring that the information

technology projects are well planned, completely documented, support the

strategic plan and corporate goals, and are mission critical. After its

review, the council forwarded the scored and ranked projects to VA’s

Capital Investment Panel. This panel, which is composed of project

decisionmakers representing the VA CFO, VA CIO, VHA, VBA, VA Office of

Planning and Policy, and NCS, was created to assist VA’s Capital Investment

Page 10 GAO/AIMD-98-154 VA Information Technology
B-278709 

Board. The panel reviewed the projects from the CIO Council and

recommended to the VA Capital Investment Board that these projects be

included in the department’s capital investment plan for the upcoming

year.18 The board is composed of the Deputy Secretary, Assistant

Secretary for Management, Assistant Secretary for Policy and Planning,

Under Secretaries for VHA and VBA, and the Director of NCS and is

responsible for making decisions on capital investment projects and

ensuring that the projects conform with VA mission, goals, priorities, and

strategies.

VA Has Not Followed a

Disciplined Process for

Selecting Projects

Under the Clinger-Cohen Act, agencies need to compare and prioritize

projects using explicit quantitative and qualitative decision criteria, such

as data on hardware and software life-cycle costs, technical risks, and

mission-related benefits. In conducting their selection processes, leading

organizations assess and manage all information technology projects,

including mission-critical or infrastructure projects, at all phases of their

life cycles, in order to create a complete strategic investment portfolio and

help ensure that the benefits of their investments will be realized.19 By

continually scrutinizing and analyzing their entire information technology

investment portfolio, managers can examine the costs of maintaining

existing systems versus investing in new ones and, on the basis of mission

priorities, reach decisions on systems’ overall contributions to

organizational goals.

As stated in our investment guide,20 good decisions require good data. To

help make decisions on information technology investments, leading

organizations require all projects to have complete and up-to-date project

information. This information includes cost and benefit data, risk

assessments, implementation plans, and initial performance measures.

Further, this information allows senior executives to rigorously evaluate

each project, make project comparisons across the organization, and

establish project review schedules for projects selected for funding in

order to monitor and track project cost, benefits, and risks.

VA’s selection criteria requires that relevant reports (e.g., congressionally

requested audits and studies, VA in-process and post-implementation

review findings), cost-benefit analyses, risk analyses, and risk

18VA’s Fiscal Year 1999 Department Capital Plan supports VA’s annual budget request and summarizes

how each proposal addresses the Capital Investment Board’s criteria.

19GAO/AIMD-94-115, May 1994.

20GAO/AIMD-10.1.13, February 1997.

Page 11 GAO/AIMD-98-154 VA Information Technology
B-278709 

management plans, be provided to decisionmakers reviewing projects for

funding.

VA did not follow a disciplined process for selecting its information

technology projects. Specifically, the VA Capital Investment Board was not

provided sufficient data with which to make good funding decisions. In

our analysis of VA’s selection process, we examined 7 of 16 projects

approved by the board. These seven projects represent about $223 million

or 36 percent of VA’s fiscal year 1999 information technology capital

investment budget of $625 million. As shown in table 1, none of the seven

projects we examined contained all the required information. Further,

despite the importance of VA’s Veteran-Focused Information Technology

Architecture (ITA) program to defining and achieving the “One VA” vision,

none of the required information for this project was provided to the

board. Nonetheless, the board decided to fund all seven projects. Further,

the board did not establish a schedule for conducting project reviews, at

key milestones, for each approved project.

Page 12 GAO/AIMD-98-154 VA Information Technology
B-278709 

Table 1: Summary of Supporting

Analyses and Documentation

Presented to the VA Information

Technology Strategic Planning

Working Group

ESa ICb MVRc HRLink$d C&Pe IDCUf ITAg

Relevant reportsh X

Market research results X Z

Benefit-cost analysis X Z X X X

Formal risk analysis X Z X

Risk management plan X Z X

Acquisition

strategy/plan X X

Legend: X represents documents presented to VA’s Information Technology

Strategic Planning Working Group, which was created to assist VA’s

CIO Council in its review of the projects.

Z represents a document provided to decisionmakers that contained 

elements of these types of analyses and documents.

aVHA’s VA Medical Care Enrollment System.

bVBA’s Information Centers.

cVA’s Master Veteran Record.

dVA’s replacement human resources and payroll system (formerly known as PAY-VA).

eVBA’s Replacement of the Compensation and Pension Payment System.

f

VA’s Integrated Data Communications Utility Follow-On.

gVA’s Veteran-Focused Information Technology Architecture Program.

hRelevant reports include congressionally requested audits and studies as well as VA in-process

and post-implementation review findings.

Source: VA. We did not independently assess the quality and validity of the analyses and

documents provided.

Recognizing the weaknesses in the investment selection process, the

Deputy CIO, in an August 4, 1997, memorandum to VA’s CIO, recommended

that this process be improved in the next budget cycle. For example, she

recommended that (1) adequate documentation be provided for all

information technology projects and (2) adequate time be provided for

thorough reviews of the documentation prior to scoring and ranking.

Seven months later, the CIO in a memorandum to VA’s administration heads,

assistant secretaries, and other key officials, specified that changes would

be made to the department’s capital investment process for the fiscal year

2000 budget cycle to ensure the provision of adequate documentation and

adequate documentary review. In addition, the memorandum stated that

projects with incomplete documentation will be returned to the originating

Page 13 GAO/AIMD-98-154 VA Information Technology
B-278709 

office for the missing information. The memorandum did not address what

action will be taken if the missing documentation is not provided.

Control Process Does Not

Provide for Adequate

Monitoring and

Management of

Investments

Leading organizations continue to manage their investments once

selection has occurred, maintaining a cycle of continual control and

monitoring.21 Senior managers review the project at specific milestones as

the project moves through its life cycle and as the dollar amounts spent on

the project increase. At these milestones, the executives compare the

expected costs, risks, and benefits of earlier phases with the actual costs

incurred, risks encountered, and performance benefits realized to date.

This enables senior executives to (1) identify and focus on managing

high-potential or high-risk projects, (2) reevaluate investment decisions

early in a project’s life cycle if problems arise, (3) respond to changing

external and internal conditions in mission priorities and budgets, and

(4) learn from past successes and mistakes in order to make better

decisions in the future. During the control phase, senior executives

determine if projects should be functionally modified, continued,

accelerated, delayed, or terminated. As executives responsible for

implementing legislative reform, it is critical that senior managers stay

actively involved in the process for controlling information technology

projects and receive complete and up-to-date information related to the

projects under review.

To control and monitor its information technology projects, VA relies on

periodic project status reviews and formal in-process reviews. Periodic

project status reviews are conducted at the VA component level. Formal

in-process reviews are conducted at the department level. According to

VA’s policy, formal in-process reviews are only conducted ad hoc, such as

when it becomes apparent that a project is behind schedule, over-budget,

not performing as planned, or when oversight agencies raise issues.

VA’s process for monitoring and managing its investment portfolio is not

timely and provides little information to VA decisionmakers. First, VA does

not conduct formal in-process reviews before significant dollars are

expended or substantial risks are encountered. For example, VA had

initially scheduled an in-process review of a VBA project to replatform and

redesign a system that provides educational benefits to reservists.

However, the in-process review was canceled when the project ran into

problems and the project is now being reassessed. The problems

21GAO/AIMD-94-115, May 1997.

Page 14 GAO/AIMD-98-154 VA Information Technology
B-278709 

associated with this project might have been avoided had VA conducted

proactive, risk-based in-process reviews at all critical project milestones.

Second, to the extent that periodic project status reviews and formal

in-process reviews are conducted, the results of the reviews were not

provided to decisionmakers reviewing projects for funding. For example,

of the 15 major ongoing or maintenance projects22 that the VA investment

board approved for funding, only one, VBA’s Replacement of the

Compensation and Pension Payment System project, received a formal

in-process review during fiscal year 1997. However, as shown in table 1,

decisionmakers were not provided with the results of this review.

Therefore, they were not in a position to effectively monitor and manage

this project.

Evaluation Process Does

Not Identify How to

Improve the Investment

Process

Once projects have been implemented and become operational, leading

organizations conduct post-implementation reviews (PIRs) to determine

whether they have achieved expected benefits, such as lowered cost,

reduced cycle time, increased quality, or increased speed of service

delivery.23 Our information technology investment guide24 points out that

each PIR should have a dual focus. First, it should provide an assessment of

the implemented project, including an evaluation of customer/user

satisfaction and mission/program impact in terms of achieving the

estimated cost, schedule, and mission-related benefits. Second, it should

provide lessons learned so that the investment decision-making processes

can be improved.

VA has developed a standard methodology for conducting PIRs. This

methodology focuses on elements, such as: (1) customer/user satisfaction,

(2) strategic impact and effectiveness, and (3) impact on organization’s

internal operations including security, internal controls, standards and

compliance, and maintenance.

Our review identified deficiencies with VA’s process for evaluating

completed projects. First, while the three PIRs VA performed during fiscal

years 1996 and 1997 gathered information on customer/user satisfaction

and discussed development and implementation challenges, none of them

22An ongoing project is a project that VA’s Capital Investment Board initially approved as a new project

and which is now seeking out-year funding; a maintenance project is a project that is fully

operational/implemented and in a maintenance mode.

23GAO/AIMD-94-115, May 1994.

24GAO/AIMD-10.1.13, February 1997.

Page 15 GAO/AIMD-98-154 VA Information Technology
B-278709 

compared actuals to estimates in terms of cost, schedule, and

mission-related benefits.25 For example, while the PIRs discuss cost

savings, they do not provide information on whether the projects met,

exceeded, or fell short of expectations.

Second, VA did not identify lessons learned that can be used to improve

VA’s investment process for selecting, controlling, and evaluating

information technology initiatives. Our review of the three PIRs VA

performed disclosed that the PIRs did identify some project specific

improvements. For example, based on a PIR of VHA’s Integrated Funds

Distribution, Control Point Activity, Accounting and Procurement system,

VHA subsequently modified this system to ensure appropriate security

access. However, none of the PIRs assessed the completed projects to

identify improvements that could be made to VA’s information technology

investment process.

VA’s CIO

Responsibilities Not

Limited Primarily to

Information

Technology

Management

The Paperwork Reduction Act and the Clinger-Cohen Act direct federal

agency heads to appoint CIOs to (1) promote improvements to the work

processes used by the agency to carry out its programs, (2) implement an

integrated agencywide systems or technology architecture, and (3) help to

establish a sound investment review process to select, control, and

evaluate spending for information technology. To help ensure that these

responsibilities are effectively executed, the Clinger-Cohen Act also

requires that the CIO’s primary responsibility be related to information

management.

VA’s CIO responsibilities are not limited primarily to information

management. The CIO also serves the department in a variety of top

management positions, including Assistant Secretary for Management, CFO,

and Deputy Assistant Secretary for Budget.

In an agency as decentralized as VA, its CIO is faced with many significant

information management responsibilities, such as ensuring (1) that the

department’s operations will not be disrupted by the Year 2000 problem,

(2) that its systems developments are not handicapped by incomplete

architectures, and (3) that a sound information management investment

review process that provides a systematic, data-driven means of selecting,

controlling, and evaluating information technology projects will be

25The three reviews were of VHA’s Automated Medical Information Exchange; VHA’s Integrated Funds

Distribution, Control Point Activity, Accounting and Procurement; and NCS’ Burial Operations Support

System.

Page 16 GAO/AIMD-98-154 VA Information Technology
B-278709 

institutionalized. As we testified in October 1997,26 each of these

responsibilities is formidable. Taken together, they certainly constitute a

full-time job for any CIO.

We have raised concerns in the past about agencies that have vested CIO

and CFO responsibilities in one person.27 Agencies face challenges in

improving both financial and information management. In our opinion,

each management area requires full-time leadership by separate

individuals with appropriate talent, skills, and experience in these two

areas. The Clinger-Cohen Act calls for CIOs to have information resources

management as their primary duty. We have stressed the importance of

this principle in testimony and in our February 1997 high-risk report, in

which we emphasized that the CIO’s duties should focus sharply on

strategic information management issues and not include other major

responsibilities.28

In a May 1997 report29 to OMB, VA’s Assistant Secretary for Management

acknowledged that he was the department’s CIO as well as its CFO. He

indicated that the VA Secretary felt that assigning multiple responsibilities

to the department’s CIO would establish clear accountability for

information resources management activities at VA, where financial

systems represent a substantial part of the agency’s information systems

portfolio. However, officials familiar with the current information

management environment at VA and its components told us that VA’s CIO is

unable to get involved in the normal, day-to-day business of a CIO unless a

problem arises that absolutely demands his attention.

Moreover, VA’s CIO told us that because he does not have a technical

background in information resources management, he relies on his deputy.

VA’s Deputy CIO, however, told us that since she has not been officially

delegated the decision-making authority that the CIO has, she can not make

important information technology decisions promptly. For example, the

Deputy CIO recognized problems VBA was having with the Veterans Service

Network (VETSNET). Consequently, she wrote a plan to correct the

26GAO/T-AIMD-98-22, October 27, 1997.

27GAO/T-AIMD-98-22, October 27, 1997.

28Government Reform: Legislation Would Strengthen Federal Management of Information and

Technology (GAO/T-AIMD-95-205, July 25, 1995); Managing Technology: Best Practices Can Improve

Performance and Produce Results (GAO/T-AIMD-97-38, January 31, 1997); High-Risk Series:

Information Management and Technology (GAO/HR-97-9, February 1997); and GAO/T-AIMD-98-22,

October 27, 1997.

29Progress Report on the Department of Veterans Affairs Chief Information Officer Program,

April 1997.

Page 17 GAO/AIMD-98-154 VA Information Technology
B-278709 

problems and briefed the CIO. Despite the problems that VETSNET has

encountered and the significance of this project to VA, the CIO has not acted

yet on this plan beyond presenting the plan to the VBA CIO. The Deputy CIO

stated that she does not have the authority to ensure that this corrective

action plan is enacted. As a result, she added, such issues, when left

unaddressed, tend to evolve into different issues or problems later.

The CIO recently told us that he would soon step down from his Assistant

Secretary for Management/CFO/CIO positions and assume the position of

VA’s Deputy Assistant Secretary for Budget. It is not known at this time

whether the new Assistant Secretary will also hold the CIO and CFO

positions.

According to VA’s Director of Information Resource Management Policy

and Standards Service, VA recently formed a working group30 to determine

whether to separate the department’s CIO and CFO positions. The working

group has submitted several options on this matter to VA’s Secretary for

consideration.

Conclusions VA has not fully implemented critical provisions of the Clinger-Cohen Act

and other information technology legislative reforms to achieve its “One

VA” vision of becoming more customer-focused and delivering seamless

service to veterans. It lacks a departmentwide strategy for reengineering

and improving business processes. As a result, business process

reengineering efforts at the component levels are uncoordinated,

duplicative, and do not provide VA with opportunities to share information.

Further, while VA recognizes the importance of defining a departmentwide

integrated information technology architecture, it has not yet done so.

Without an integrated architecture, VA will continue to develop duplicative

and redundant information systems and will not accomplish its vision of

“One VA.”

In addition, VA has not institutionalized a disciplined investment

management process. Decisionmakers continue to make investment

decisions involving millions of dollars without reliable data on expected

and actual costs, benefits, and risks. Moreover, VA’s process for controlling

information technology projects through periodic status and in-process

30The working group consists of VA’s Chief of Staff, who is also the chairperson of the working group;

VA’s Deputy to the Assistant Secretary for Management; VA’s Deputy Assistant Secretary for

Information Resources Management; VA’s Deputy General Counsel; and VA’s Deputy Assistant

Secretary for Human Resources Management.

Page 18 GAO/AIMD-98-154 VA Information Technology
B-278709 

reviews does not adequately monitor and manage its investments so as to

detect or avoid problems early. Further, VA’s process for evaluating

completed projects does not modify and improve the investment process

based on lessons learned.

Finally, given the size of VA’s information technology budget and the many

serious information management issues its CIO must face, such as ensuring

that the department’s operations will not be disrupted by the Year 2000

problem, it is important that information resources management be the

CIO’s primary duty. A full-time CIO would help ensure adequate coverage of

information management issues. Information resources management is not

the primary duty of VA’s CIO. He also serves as Assistant Secretary for

Management, CFO, and Deputy Assistant Secretary for Budget.

Recommendations We recommend that the Secretary of Veterans Affairs direct the Assistant

Secretary for Policy and Planning to develop a departmentwide strategy

that details how VA will reengineer its business processes, including

identifying and prioritizing process improvement projects, and delineating

their interrelationships.

To fulfill the requirements of the Clinger-Cohen Act and other information

technology legislative reforms, we also recommend that the Secretary

direct VA’s CIO to

• develop a detailed implementation plan with milestones for completing an

integrated, departmentwide information technology architecture;

• fully implement a disciplined process for selecting information technology

investments in which all decisions are based upon complete and current

project information including estimated project costs, expected

mission-related benefits, projected schedule, and risks;

• conduct formal in-process reviews at key milestones in a project’s life

cycle, including comparing actual and estimated project costs, benefits,

schedule, and risks, and provide these results, as well as the results of

periodic project status reviews performed by VA components, to

decisionmakers who will determine whether to continue, accelerate, or

terminate information technology projects; and

• initiate post-implementation reviews for information technology projects

within 12 months of implementation, to compare completed project cost,

schedule, performance, and mission improvement outcomes with original

estimates, and provide the results of these reviews to decisionmakers so

Page 19 GAO/AIMD-98-154 VA Information Technology
B-278709 

that improvements can be made to VA’s information technology investment

process.

In addition, we recommend that the Secretary appoint a CIO with full-time

responsibilities for information resources management.

Agency Comments

and Our Evaluation

In commenting on a draft of this report, the Department of Veterans

Affairs concurred with all six of our recommendations. The department

also stated that it recognizes that its information resources management

challenges are broad and critical to the success of the department’s

mission, and, therefore, established the position of Assistant Secretary to

serve as CIO reporting directly to the Secretary on all information

resources issues. This new Assistant Secretary will be responsible for

ensuring that all of the department’s information technology initiatives

support the overall “One VA” vision.

Finally, in concurring with our recommendation to complete an integrated,

departmentwide information technology architecture, the department did

not specify how and when it plans to do so. Until it completes and

implements an integrated architecture, VA will continue to develop

duplicative and redundant information systems and will not accomplish its

vision of “One VA.”

As agreed with your offices, we will not distribute this report until 5 days

after its date. At that time, we will send copies to the Chairman and

Ranking Minority Member of the Subcommittee on Oversight and

Investigations, House Committee on Veterans’ Affairs; and the Chairman

and Ranking Minority Member of the Subcommittee on Benefits, House

Committee on Veterans’ Affairs. We will also provide copies to the

Chairmen and Ranking Minority Members of the Senate and House

Committees on Appropriations; the Secretary of Veterans Affairs; and the

Director of the Office of Management and Budget. Copies will also be

made available to others upon request.

Page 20 GAO/AIMD-98-154 VA Information Technology
B-278709 

Please contact me at (202) 512-6253 or by e-mail at

willemssenj.aimd@gao.gov if you have any questions concerning this

report. Major contributors to this report are listed in appendix III.

Joel C. Willemssen

Director, Civil Agencies Information Systems

Page 21 GAO/AIMD-98-154 VA Information Technology
Contents

Letter 1

Appendix I 

Comments From the

Department of

Veterans Affairs

24

Appendix II 

Objectives, Scope,

and Methodology

28

Appendix III 

Major Contributors to

This Report

30

Table Table 1: Summary of Supporting Analyses and Documentation

Presented to the VA Information Technology Strategic Planning

Working Group

13

Figure Figure 1: VA’s New Investment Decision-Making Process 10

Abbreviations

CIO Chief Information Officer

CFO Chief Financial Officer

ITA Veteran-Focused Information Technology Architecture

Program

MVR Master Veteran Record

NCS National Cemetery System

NIST National Institute of Standards and Technology

OMB Office of Management and Budget

PIR Post-implementation review

VA Department of Veterans Affairs

VBA Veterans Benefits Administration

VETSNET Veterans Service Network

VHA Veterans Health Administration

Page 22 GAO/AIMD-98-154 VA Information Technology
Page 23 GAO/AIMD-98-154 VA Information Technology
Appendix I 

Comments From the Department of

Veterans Affairs

Page 24 GAO/AIMD-98-154 VA Information Technology
Appendix I 

Comments From the Department of

Veterans Affairs

Page 25 GAO/AIMD-98-154 VA Information Technology
Appendix I 

Comments From the Department of

Veterans Affairs

Page 26 GAO/AIMD-98-154 VA Information Technology
Appendix I 

Comments From the Department of

Veterans Affairs

Page 27 GAO/AIMD-98-154 VA Information Technology
Appendix II 

Objectives, Scope, and Methodology

Our objectives were to examine how VA has implemented the following

specific provisions of the Clinger-Cohen Act and other legislative reforms:

reengineering business processes before acquiring information

technology, completing an integrated information technology architecture,

institutionalizing a disciplined information technology investment

decision-making process; and appointing an agency CIO.

In examining VA’s reengineering of its business processes, we applied GAO’s

guide for business process reengineering.1

 We also reviewed VA’s draft One

VA: Vision of Information Technology Enhanced Customer Service, dated

January 22, 1998, and its Strategic Plan—Fiscal Years 1998-2003, dated

September 30, 1997. In addition, we discussed VA business process revision

activities with VA, VBA, VHA, NCS, and OMB officials.

Regarding VA’s information technology architecture, we applied OMB’s

Memorandum on Information Technology Architecture, dated June 18,

1997, and the National Institute of Standards and Technology’s Special

Publication 500-167, “Information Management Directions: The

Information Challenge.” We also reviewed agency documents and

interviewed VA officials on the department’s efforts to develop an

integrated information technology architecture.

To assess VA’s information technology investment process, we applied

applicable requirements from the Clinger-Cohen Act of 1996, the

Paperwork Reduction Act of 1995, the Government Performance and

Results Act of 1993, the Federal Acquisition Streamlining Act of 1994, the

Chief Financial Officers Act of 1990, OMB Circular A-130, GAO’s best

practices report on strategic information management,2

 and OMB’s guide

Evaluating Information Technology Investments: A Practical Guide. We

reviewed and analyzed numerous documents provided by VA, including its

(1) Strategic Plan—Fiscal Years 1998-2003, dated September 30, 1997,

(2) Information Technology Strategic Plan—FY 1999-FY 2003, dated July

1997, (3) Office of Information Resources Management—IRM Policy and

Standards Service—Information Technology Evaluation Process, dated

November 4, 1997, (4) Directive 6000—VA Information Resources

Management (IRM) Framework, dated September 17, 1997, (5) draft

Department of Veterans Affairs FY 1999 Department Capital Plan, dated

October 1997, and (6) VA’s Information Technology Strategic Planning,

dated March 1997.

1

GAO/AIMD-10.1.15, April 1997, Version 3.

2

Executive Guide: Improving Mission Performance Through Strategic Information Management and

Technology—Learning From Leading Organizations (GAO/AIMD-94-115, May 1994).

Page 28 GAO/AIMD-98-154 VA Information Technology
Appendix II 

Objectives, Scope, and Methodology

In addition, we compared VA’s information technology investment plans

and process documents with selected criteria following GAO’s guide for

evaluating and assessing federal agencies’ selection and management of

information technology resources3

 as well as OMB’s Capital Programming

Guide (Version 1.0) Supplement to OMB Circular A-11, Part 3: Planning,

Budgeting, and Acquisition of Capital Assets, dated July 1997. We also

applied criteria from our investment guide to our review of seven VA

information technology projects approved for funding in VA’s fiscal year

1999 budget cycle. These seven projects were selected based on a variety

of factors, including some of interest to congressional oversight

committees, some that exhibited potential duplication of project

functionality, and the single highest cost departmentwide project. We

interviewed key VA, VBA, VHA, NCS, and OMB officials regarding the

department’s information technology investment process.

Finally, to assess VA’s implementation of the CIO provision of the

Clinger-Cohen Act, we analyzed (1) VA’s April 1997 Progress Report on the

Department of Veterans Affairs CIO Program, (2) VA’s strategic, IRM, and

information technology plans mentioned above, and (3) OMB

documentation regarding CIOs. We also interviewed key VA, VBA, VHA, NCS,

and OMB officials regarding the duties and responsibilities of CIOs.

3

GAO/AIMD-10.1.13, February 1997.

Page 29 GAO/AIMD-98-154 VA Information Technology
Appendix III 

Major Contributors to This Report

Accounting and

Information

Management Division,

Washington, D.C.

Helen Lew, Assistant Director

Nabajyoti Barkakati, Technical Assistant Director

John T. Christian, Senior Business Process Analyst

Mary J. Dorsey, Information Systems Analyst-in-Charge

Michael P. Fruitman, Communications Analyst

Thomas M. McDonald, Senior Business Process Analyst

John P. Rehberger, Senior Information Systems Analyst

John A. Riley, Senior Business Process Analyst

(511233) Page 30 GAO/AIMD-98-154 VA Information Technology
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100