a

GAO

United States General Accounting Office

Report to the Subcommittee on Coast 

Guard and Maritime Transportation, 

Committee on Transportation and 

Infrastructure, House of Representatives

October 2001 COAST GUARD

Update on Marine 

Information for Safety 

and Law Enforcement 

System

GAO-02-11

Why GAO Did This Study

The Coast Guard is developing a

Web-based information system to

replace an aging computer

system that it uses to track safety

and law-enforcement actions—

such as inspections, drug

interdiction, and oil spill

assistance—involving

commercial and recreational

vessels. In 1995, the Coast Guard

awarded a contract to develop

the new system, called MISLE.

After the project encountered

cost and schedule problems,

development responsibility was

transferred to the Coast Guard’s

systems development center in

1999, and the contract terminated

after about $26 million had been

spent.

The Coast Guard’s history of

systems development problems

and information technology

weaknesses prompted the

Subcommittee to ask GAO to

review MISLE’s current status

and risks.

October 2001

COAST GUARD

Update on Marine Information for Safety

and Law Enforcement System

This is a test for developing Highlights for a GAO report. For additional information about the report, call Linda Koontz at (202) 512-

6240. To provide comments on this test Highlights page, call Keith Fultz at (202) 512-3200 or send an email to HighlightsTest@gao.gov.

Highlights of GAO-02-11, a report to the Subcommittee on Coast Guard and Maritime Transportation,

House Committee on Transportation and Infrastructure

What GAO Recommends

GAO is making several

recommendations in the four key

areas of system requirements,

software testing, transition

planning, and risk mitigation.

In comments on a draft of our

report, Coast Guard officials

generally agreed with GAO’s

recommendations, but noted that

some of the risks were “tradeoff

decisions,” necessary to deliver an

initial version of MISLE.

United States General Accounting Office

What GAO Found

The Coast Guard has made progress since taking over its Marine

Information for Safety and Law Enforcement (MISLE) system

development; it is now poised to deploy the system with limited

capabilities. But much remains to be accomplished in order to deliver the

complete system, with only about $11 million of its $61 million estimate

remaining.

MISLE is facing risks in four key areas:

• Changing system requirements. MISLE requirements have

repeatedly changed; several have been dropped, and others

postponed to later development phases due to problems found during

testing and the Coast Guard’s emphasis on replacing its current

system as soon as possible. Other requirements are being added or

accelerated. Such a continually changing scenario increases the risk

that MISLE will fall short of user needs.

• Software testing. The Coast Guard undertook risky software

testing practices in that it deferred testing some functions and did not

resolve all critical problems uncovered before moving on to the next

testing stage. This approach increases the likelihood that the system

will not perform as expected and/or may take longer to develop than

anticipated.

• Transition planning. Deployment of MISLE involves planning for

accurately moving data from the older system and training system

users. However, critical transition plans are not yet complete.

Beyond adding to possible delays, the absence of transition plans and

insufficient training increase the chance of user discomfort with the

new system.

• Risk management. Finally, the Coast Guard’s risk management

approach has been ineffective: risks were not assigned severity

ratings, and not all have been prioritized. Further, the Coast Guard

has not developed detailed mitigation plans for all significant risks.

Unless these challenges are successfully addressed, performance

shortcomings, cost escalation, and schedule delay are likely.

G A O

Accountability Integrity Reliability

Highlights
Page 1 GAO-02-11 Coast Guard

United States General Accounting Office

Washington, D.C. 20548

A

October 17, 2001

The Honorable Frank LoBiondo

Chairman

The Honorable Corrine Brown

Ranking Minority Member

Subcommittee on Coast Guard and Maritime Transportation

Committee on Transportation and Infrastructure

House of Representatives

Over the last decade, the United States Coast Guard (USCG) has 

experienced difficulties in acquiring the Marine Information for Safety and 

Law Enforcement (MISLE) system—an information system to track marine 

safety and law-enforcement activities involving commercial and 

recreational vessels. In 1999, the Coast Guard terminated a contract to 

acquire MISLE, after spending about 4 years and $26 million, and is instead 

developing the system at its Operations Systems Center. As you requested, 

our objective was to provide an update on MISLE’s status, plans, and 

technical and programmatic risks. 

To fulfill this objective, we evaluated MISLE project plans, costs, and 

schedules by comparing original and current job management documents. 

We also assessed technical and programmatic risks facing the MISLE 

acquisition and USCG’s plans for addressing those risks. We performed our 

work from May through September 2001, in accordance with generally 

accepted government auditing standards. USCG officials provided us with 

comments on a draft of this report; they are discussed in the “Agency 

Comments” section.

On September 18, 2001, we provided a detailed briefing to your office on 

the results of this work. The briefing slides are included in appendix I. The 

purpose of this letter is to provide the published briefing slides to you and 

to officially transmit our recommendations to the Secretary of 

Transportation.

In brief, we reported that USCG has made progress in developing MISLE 

and was poised to deploy a minimum level of functionality in November 

2001. However, the system was several months behind schedule, and 

USCG had already spent most of the $61 million acquisition cost estimate—

most of it on the original contract. With only about $11 million remaining, 

much remains to be done to deliver the complete system. In its efforts to 

develop and deploy a complete MISLE system, USCG faces significant 
Page 2 GAO-02-11 Coast Guard

challenges and risks in several areas, including managing system 

requirements and user expectations, testing the system, transitioning to an 

operational system, and managing program risks. We made specific 

recommendations to address these risks. 

Recommendations To mitigate USCG’s MISLE risks, we recommend that the Secretary of 

Transportation direct the USCG Commandant to ensure that the 

appropriate officials complete the following actions.

In the system requirements area, 

• define and prioritize, in conjunction with system users, all needed 

system functions, corrections, and enhancements that must occur to 

meet valid user needs; and

• develop cost and schedule estimates for providing these functions, 

corrections, and enhancements.

In the software testing area,

• close all critical problems before initiating the next state of testing.

In the area of transition planning,

• finalize and implement Vessel Documentation System (VDS) transition 

plans, and

• develop and implement VDS training materials. 

In the risk mitigation area, 

• develop a single list of system risks,

• evaluate system risks to determine their severity and prioritize these 

risks,

• develop and implement comprehensive mitigation strategies for each of 

the risks, and 

• regularly oversee the status of risks and risk mitigation efforts to 

determine whether additional mitigation activities are warranted. 
Page 3 GAO-02-11 Coast Guard

Agency Comments We provided drafts of our briefing and this report to Department of 

Transportation and USCG officials, including representatives of the Office 

of the Secretary of Transportation and the USCG Acquisition Management 

Office. We met with USCG officials to obtain their comments on our drafts. 

These officials generally agreed with our recommendations, but 

characterized the changing requirements and testing issues we raised as 

“tradeoff decisions,” necessary to deliver an initial version of MISLE. 

USCG officials also noted that, given the recent terrorist attacks, MISLE 

deployment will likely be delayed until at least December 2001.

We are sending copies of this report to the Secretary of Transportation, the 

USCG Commandant, the Director of the Office of Management and Budget, 

and other interested parties. Copies will also be made available to others 

upon request.

Should you or your staffs have any questions concerning this report, please 

contact me at (202) 512-6240 or by e-mail at koontzl@gao.gov. Nabajyoti 

Barkakati, Barbara Collier, Michael Fruitman, Colleen Phillips, Margaret 

Sullivan, and Glenda Wright were major contributors to this report.

Linda D. Koontz

Director, Information Management Issues 
Page 4 GAO-02-11 Coast Guard

Appendix I

GAO’s September 18, 2001, Briefing ApendxiI

1

United States Coast Guard’s (USCG)

Marine Information for Safety and Law Enforcement (MISLE) System

An Update

Briefing for Staff

Subcommittee on Coast Guard and Maritime Transportation

Committee on Transportation and Infrastructure

House of Representatives

September 18, 2001
Appendix I

GAO’s September 18, 2001, Briefing

Page 5 GAO-02-11 Coast Guard

2

Briefing Outline

• Objective, Scope, and Methodology

• MISLE Background

• MISLE Status and Plans

• Key Risk Areas

• System requirements

• Software testing

• Transition planning

• Risk management

• Conclusions

• Recommendations

• Agency Comments and Our Evaluation 

 
Appendix I

GAO’s September 18, 2001, Briefing

Page 6 GAO-02-11 Coast Guard

3

Objective

To provide an update on MISLE’s status, plans, and technical and

programmatic risks

Scope and Methodology

We evaluated MISLE project plans, costs, and schedules by comparing

original and current job management documents. We also assessed technical

and programmatic risks facing the MISLE acquisition, as well as USCG’s

plans for addressing those risks, by evaluating USCG program documents,

comparing them to accepted system engineering principles, and interviewing

project officials, developers, and system users. Further, we reviewed plans

for future MISLE enhancements and discussed these enhancements with

system users and developers.

We conducted our review at USCG headquarters in Washington, D.C., and at

USCG’s Operations Systems Center and National Vessel Documentation

Center in the Martinsburg, WV, area. We conducted our work from May

through September 2001, in accordance with generally accepted government

auditing standards.

Objective, Scope, and Methodology
Appendix I

GAO’s September 18, 2001, Briefing

Page 7 GAO-02-11 Coast Guard

4

MISLE is intended to be a Web-based information system for entering and

obtaining data on Coast Guard activities concerning commercial and

federally documented recreational vessels in support of USCG’s marine

safety and law enforcement missions.

A software-intensive system, the MISLE project also includes the purchase

of hardware, including network servers and data storage devices.

MISLE is to support over 5000 system users at about 550 sites using the

existing Coast Guard intranet.

MISLE Background

Overview
Appendix I

GAO’s September 18, 2001, Briefing

Page 8 GAO-02-11 Coast Guard

5

In the mid-1990s, MISLE was envisioned to develop and integrate two major

systems—the Marine Safety Network and the Vessel Identification and

Documentation System—and to integrate them with a third, already developed

system, the Law Enforcement Information System II.

Since then, however, this concept has evolved. A description of the

subsystems and recent changes to these subsystems are discussed below:

• Marine Safety Network (MSN)

a system to allow Coast Guard personnel to input and obtain information

on USCG marine safety activities, such as vessel inspections and

boardings; it relies on vessel data from the Vessel Documentation System

MISLE Background

Overview (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 9 GAO-02-11 Coast Guard

6

MISLE components (cont’d)

• Vessel Identification and Documentation System

a system planned to integrate two subsystems in response to Public Law

100-710 (commonly called the Ship Mortgage Act of 1988):

• Vessel Identification System—planned to automate and integrate

vessel registration information from participating states; due to

complications in implementing this system, USCG no longer plans to

provide VIS functionality as part of MISLE (see Key Risk Area—System

Requirements)

• Vessel Documentation System (VDS)—a system to support the

National Vessel Documentation Center’s processes for documenting

vessels

• Law Enforcement Information System II (LEIS-II)

a system for tracking USCG law enforcement activities, such as drug

interdiction activities; due to the age of the LEIS-II platform, USCG now

plans to replace LEIS-II rather than integrate it with MISLE

MISLE Background

Overview (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 10 GAO-02-11 Coast Guard

7

As of early September 2001, USCG’s estimates for MISLE costs

and deployment schedules included the following:

• Acquisition cost: $61 million through 2003

• Life cycle cost: $94 million through 2008

• Initial capability: November 2001

• Final capability: September 2003

MISLE Background

Overview (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 11 GAO-02-11 Coast Guard

8

In 1986, USCG began developing requirements for replacing its

legacy Marine Safety Information System (MSIS) by 1995.

In the early 1990s, USCG delayed plans to replace MSIS in order to

integrate requirements for multiple systems into one system

development effort—called MISLE.

In 1995, USCG awarded a contract to Computer Sciences

Corporation (CSC) to develop and deliver a complete MISLE system

by 2002. At that time, USCG officials estimated that this contract

could cost up to $35 million.

• A March 1999 change required the developer to replace MSIS

functions by June 2001.

MISLE Background

History
Appendix I

GAO’s September 18, 2001, Briefing

Page 12 GAO-02-11 Coast Guard

9

The MISLE development contract encountered escalating costs and

schedule delays; it was “partially terminated for government convenience”

in October 1999.

• CSC delivered two components under this contract:

• Vessel Identification System—a system to automate and

integrate vessel registration information from participating states.

This system was never implemented due to problems in

integrating state data. USCG no longer plans to provide VIS

functionality as part of MISLE.

• Mission Analysis and Planning—a decision support tool for

obtaining statistical information from the legacy MSIS system

• Total spent on the CSC contract: $26 million

In October 1999, USCG transferred responsibility for MISLE development

from CSC to its Operations Systems Center (OSC)—a government-owned,

contractor-operated facility. USCG’s acquisition office manages the

project.

MISLE Background

History (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 13 GAO-02-11 Coast Guard

10

OSC’s strategy is to develop and deploy a minimal level of MISLE

functionality as soon as possible to replace MSIS.

• MSIS—once scheduled for termination in 1995—is still serving as the

primary information system for marine safety programs.

• MSIS has failed repeatedly over the past several years, causing days

and weeks of data processing backlogs.

• USCG officials stated that MSIS hardware is no longer supportable;

the original vendor is no longer in business, and the agency is

currently reusing parts to keep the system running.

MISLE Background

Acquisition Strategy
Appendix I

GAO’s September 18, 2001, Briefing

Page 14 GAO-02-11 Coast Guard

11

OSC’s system development approach:

• Rapid Application Development/Joint Application Design

a system development methodology that focuses on involving users

early and often in the design and development of a system in order to

develop it quickly.

• Timebox Methodology

a system development methodology, often used in conjunction with

the Rapid Application Development methodology, in which a time

limit is established for developing a system (this forces users to focus

on a minimal set of requirements and allows them to drop some

functions if necessary to meet the time limit).

MISLE Background

Acquisition Strategy (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 15 GAO-02-11 Coast Guard

12

MISLE Phase 3

Feb. 2002 to Sept. 2003

Deliverables:

• completion of remaining MSN and VDS capabilities

• expanded data warehouse

MISLE Phase 1

Oct. 1999 to Nov. 2001

Deliverables:

• initial MSN capability needed to replace MSIS

• initial VDS capability

MISLE Phase 2

Feb. 2001 to March 2002

Deliverables:

• replacement of LEIS-II

• extended MSN and VDS capabilities

• initial data warehouse

MISLE Background

Planned Delivery Schedule

OSC’s three-phased delivery approach:
Appendix I

GAO’s September 18, 2001, Briefing

Page 16 GAO-02-11 Coast Guard

13

MISLE Phase 1

40 high level requirements have been translated to over 13,000 detailed

software requirements supporting two key components: the Marine Safety

Network (MSN) and the Vessel Documentation System (VDS)

MSN Status:

• Completed system testing1

• User input obtained throughout

VDS Status:

• Major components postponed to later releases

• Completed system testing

MISLE Phase 2

Development began February 2001

MISLE Status and Plans

Progress to Date

1USCG tests increasingly larger portions of MISLE in stages: (1) unit testing--informal tests of software modules; (2) functional testing--formal tests of

software modules; (3) integration testing--testing the integration of all modules in a core area; and (4) system testing--testing all of the core areas in a

deliverable. MISLE then undergoes operational test and evaluation (OT&E)--testing of operational effectiveness and suitability by system users.
Appendix I

GAO’s September 18, 2001, Briefing

Page 17 GAO-02-11 Coast Guard

14

USCG’s $61 million total estimated MISLE acquisition cost includes:

• about $7 million spent on requirements analysis, design, and

validation in the early 1990s

• $26 million spent on the 1995 MISLE contract that was terminated

• about $28 million for current MISLE development efforts, including

• almost $17 million spent through fiscal year 2001 to develop

• most of MISLE Phase 1

• part of MISLE Phase 2

• about $11 million in fiscal year 2002 and beyond

• to complete residual tasks supporting MISLE Phases 1 and 2

• to develop and implement MISLE Phase 3

.

MISLE Status and Plans

Costs
Appendix I

GAO’s September 18, 2001, Briefing

Page 18 GAO-02-11 Coast Guard

15

 MISLE Phase 1 has fallen behind schedule.

MISLE Status and Plans

Schedule

MISLE Phase 1

milestones

Dec. 1999

baseline

changea

June and

Dec. 2000

estimatesb

May 2001

estimate

Aug. 1, 2001

estimate

Sept. 5, 2001

estimate

System testing complete N/IcN/I 7/20/01 MSN 8/15/01

VDS 9/15/01

MSN 8/27/01

VDS 8/27/01

Operational Test and

Evaluation complete

6/01 2/01 8/15/01 MSN 8/30/01

VDS 9/30/01

MSN 9/21/01

VDS 9/21/01

MISLE month complete N/I N/I 9/15/01 9/30/01 10/19/01

Data migration complete N/I N/I 9/30/01 10/10/01 11/2/01

MISLE Phase 1 deployed N/I 4/01 10/01 10/15/01 11/5/01

a

In commenting on a draft of this briefing, the MISLE project manager noted this baseline document also includes

an OT&E completion range from January to September 2001, and that OT&E would be completed within this

range. However, USCG internal management reports reflect the June 2001 completion date. In fact, these reports

show MISLE in breach of its schedule.

b

These estimates are identified in the June 2000 MISLE Implementation Plan, as well as the December 2000

MISLE Acquisition and Project Management Plans. In commenting on a draft of this briefing, USCG officials

characterized these documents as internal, optimistic project plans--not firm schedule estimates.

c

Not identified.
Appendix I

GAO’s September 18, 2001, Briefing

Page 19 GAO-02-11 Coast Guard

16

USCG officials identified the following reasons for schedule delays:

• Problems in integrating VDS and MSN components caused a

major VDS redesign (October 2000)

• USCG needed to retrofit MSN to work with the current version of

Internet Explorer (January 2001)

• VDS functional testing was delayed because of system instability

(January 2001)

• MSN integration testing identified a missing function (June 2001):

• ability to communicate with USCG’s financial center

• VDS integration testing encountered problems (July 2001):

• test scripts would not run at users’ facility

• shortcoming in ability to edit MSIS data migrated to MISLE

(estimated 4 week delay)

MISLE Status and Plans

Schedule (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 20 GAO-02-11 Coast Guard

17

The MISLE acquisition is facing considerable challenges in the following

key risk areas:

• System Requirements

• Software Testing

• Transition Planning

• Risk Management

Key Risk Areas
Appendix I

GAO’s September 18, 2001, Briefing

Page 21 GAO-02-11 Coast Guard

18

System requirements define the minimum functions and performance

levels needed to satisfy user needs. USCG identified system

requirements for MISLE in 1994. Also, when the agency restructured the

MISLE program in 1999, it identified specific functions to be delivered in

each of the three MISLE phases.

However, MISLE requirements and the functions to be provided in each

of the three deliverables are now changing. USCG plans to drop several

of MISLE’s original system requirements and has recently postponed key

functions due to problems encountered during testing and the agency’s

need to field a system quickly. Also, in response to system users’ needs,

USCG plans to add and accelerate key functions. Further, while testing

and training on the system, users identified new and missing functions

which USCG plans to add to MISLE later.

USCG’s plans for dropping, delaying, accelerating, and adding functions

are discussed below.

Key Risk Areas

System Requirements
Appendix I

GAO’s September 18, 2001, Briefing

Page 22 GAO-02-11 Coast Guard

19

USCG is planning to drop significant MISLE requirements:

• Vessel Identification System—This system, planned to automate

states’ vessel registration data and integrate it with data on federally

documented vessels in response to the Ship Mortgage Act of 1988,

was developed as part of MISLE but never successfully

implemented. USCG is considering developing a new system outside

MISLE.

• Portable/Remote Access—This function was planned to provide the

ability to access MISLE information from remote locations, such as

Coast Guard cutters. USCG officials stated that MISLE will allow for

remote access via laptop and dial-up modem, but that providing

cutter connectivity will require decisions on the USCG infrastructure

that are beyond MISLE’s scope. USCG is considering leasing

satellite time to provide cutter connectivity.

Key Risk Areas

System Requirements (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 23 GAO-02-11 Coast Guard

20

Dropped requirements (cont’d)

• External Access—This function was planned to provide non-USCG

personnel with access to MISLE data. USCG officials stated that MISLE

has this capability but will not provide external access because of

security concerns. USCG plans to reassess the need for external

access on a case-by-case basis.

• Merchant Mariner Licensing and Documentation—This function was

to provide licensing and documentation capabilities within MISLE.

However, instead of waiting for MISLE, USCG decided to develop a

stand-alone system to provide this capability. MISLE Phase 1 is

planned to link to the Merchant Mariner Licensing and Documentation

stand-alone system.

Key Risk Areas

System Requirements (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 24 GAO-02-11 Coast Guard

21

Dropped requirements (cont’d)

• Pollution Funds Management—This function was to provide the

ability to manage pollution funds within MISLE. Instead, USCG

developed an alternative system outside MISLE to provide this

functionality. MISLE will provide a link to that system.

• Personnel Resource Management—This function was to provide the

ability to manage personnel within MISLE. Instead, USCG developed

an alternative system outside MISLE to provide this functionality. USCG

officials expect that MISLE will be able to access data in that system.

Key Risk Areas

System Requirements (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 25 GAO-02-11 Coast Guard

22

USCG has delayed some MISLE functions:

Significant Phase 1 deliverables are now planned for later releases.

Examples include VDS’ ability to

• provide Abstracts of Titles

• provide Certificates of Ownership

• automate NVDC’s paper-intensive work processes

Key Phase 2 deliverables are now planned to be Phase 3 deliverables.

Two high priority examples include

• VDS’ ability to handle requests for nonvessel information (this

involves non-U.S. corporations and oil spill response

organizations)

• MSN’s ability to capture information on the approval and oversight

of the construction of portable quarters on vessels

Key Risk Areas

System Requirements (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 26 GAO-02-11 Coast Guard

23

 Delayed functions (cont’d)

Other Phase 2 deliverables are being reconsidered—that is, USCG

reported that they may not be “doable” in Phase 3. Examples include

MSN’s ability to

• submit Web forms, such as vessel arrival, inspection request, and

marine event application forms

• capture and track information on reviews of manuals (such as

Operations Manuals, Emergency Evacuation Plans, and

Passenger Terminal Security Plans)

 Other examples include VDS’ ability to

• provide electronic access to data in response to FOIA requests

• allow e-commerce, such as credit card transactions and

submission of Web forms

Key Risk Areas

System Requirements (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 27 GAO-02-11 Coast Guard

24

USCG has added and accelerated MISLE functions:

In response to user needs, USCG has decided to

• replace LEIS-II in Phase 2, rather than integrate it with MISLE

• accelerate a critical function, incident response planning, from

Phase 3 to Phase 2

In commenting on a draft of this briefing, USCG officials noted that the

agency has also moved multiple functions from Phase 2 to later releases

of Phase 1 in response to user requests to get these functions sooner.

However, in prior discussions, USCG officials acknowledged that they

have not yet scheduled these later releases and, in fact, some of the later

Phase 1 releases may not be delivered until after Phase 2 is delivered.

Key Risk Areas

System Requirements (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 28 GAO-02-11 Coast Guard

25

 USCG plans to add functions to MISLE

While testing and training on MISLE, system testers and users are

identifying both needed system corrections and new user needs that

MISLE does not currently provide. USCG plans to add these functions to

MISLE as “future enhancements.”

• As of August, USCG identified 72 future enhancements. Examples

include

• making vessel details on maps readable

• incorporating data from additional information systems outside

MISLE

• making a print icon work

In commenting on a draft of this briefing, MISLE project officials stated

that these future enhancements are low-level requirements and user

preferences, and that they do not jeopardize the project’s ability to deliver

on its agreed-upon scope.

Key Risk Areas

System Requirements (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 29 GAO-02-11 Coast Guard

26

USCG has developed a schedule for the functions that it deems are the

highest priority—MSN release 1.1 and 1.2, and MISLE Phase 2.

However, USCG has not yet prioritized all of the additional and delayed

MISLE functions, nor has it developed schedules for providing them. For

example, USCG has not yet established a schedule for delivering the

significant VDS functions that were deferred from Phase 1 to the VDS

release 1.1, or for the many other MSN and VDS functions anticipated in

later releases of Phase 1.

By dropping and/or delaying key MISLE functions, USCG runs the risk

that the system will not function as intended or expected, and that the

deployed system will fall short of user needs. Further, without a schedule

or cost estimate for the delayed functions and enhancements, it is not

clear when these functions will be implemented nor how much it will cost

to implement them.

Key Risk Areas

System Requirements (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 30 GAO-02-11 Coast Guard

27

According to leading information technology organizations, to be effective,

software testing practices should be planned and conducted in a structured

and disciplined fashion. Typically, this involves testing increasingly larger

increments of a system until the complete system is tested and accepted,

and resolving critical problems before moving to the next phase of testing.

USCG policies call for

• testing MISLE software in successively larger increments2

• unit testing

• functional testing

• integration testing

• system testing

• operational test and evaluation (OT&E)

• managing problems reported during testing:

• problem reports are to be prioritized in levels 1 to 5 (levels 1 and 2

are most critical), fixed, tested, and closed

• all critical problem reports (levels 1 and 2) are to be closed before the

next stage of testing begins

Key Risk Areas

Software Testing

2We define similar concepts in our guide for managing testing activities,

Year 2000 Computing Crisis: A Testing Guide (GAO/AIMD-10.1.21, November 1998).
Appendix I

GAO’s September 18, 2001, Briefing

Page 31 GAO-02-11 Coast Guard

28

MSN Integration

Testing

MSN System

Testing

MSN OT&E

VDS Integration

Testing

VDS System

Testing

April May June July Aug Sept

(4/1/01) (6/26/01)

(6/30/01) (8/27/01)

 (9/10/01) (9/21/01)

(5/21/01) (8/10/01)

(8/16/01) (8/27/01)

2001

VDS OT&E

Key Risk Areas

Software Testing (cont’d)

USCG’s current testing schedule for MISLE components:

 (9/10/01) (9/21/01)
Appendix I

GAO’s September 18, 2001, Briefing

Page 32 GAO-02-11 Coast Guard

29

USCG has undertaken risky testing practices:

• Key VDS functions were not tested during functional testing because

they were under development at the time and were deferred to

integration testing. For example:

• system security

• processing and printing Abstracts of Titles

• printing Certificates of Documentation

Key Risk Areas

Software Testing (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 33 GAO-02-11 Coast Guard

30

Risky testing practices (cont’d):

• Significant problems were not closed before the next stage of testing

began. For example:

• 9 critical problems identified during MSN integration testing were

not closed before system testing began

USCG’s testing practices increase the risk that MISLE will not perform as

expected, or may take longer to develop than expected.

Key Risk Areas

Software Testing (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 34 GAO-02-11 Coast Guard

31

According to the CIO Council’s guidance on enterprise architectures,

agencies should carefully plan the transition from legacy systems to new

systems. To ease the transition from MSIS to MISLE, USCG developed a

transition plan that calls for developing data migration plans and training

system users.

USCG has acknowledged that transitioning from MSIS to MISLE will be

difficult. It will involve moving data from one system to the next, ensuring

the accuracy of the transported data, and training users on new business

processes. USCG has undertaken several transition efforts:

• MSN data migration plan has been developed

• Hundreds of MSN users have undergone training

• A 4-week period, known as MISLE month, is planned in which MSIS

and MISLE will be run concurrently to allow MSN users to become

acquainted with the new system and new work processes

Key Risk Areas

Transition Planning
Appendix I

GAO’s September 18, 2001, Briefing

Page 35 GAO-02-11 Coast Guard

32

Despite its transition guidance, USCG has not yet completed critical VDS

transition planning because the initial system deliverables changed

dramatically in July 2001. Specifically,

• VDS transition plan has not been finalized

• VDS training plan has not been developed

• VDS users have not been trained

VDS users stated that training and transition planning are not critical for

the first release of VDS, because the functions being delivered will not

dramatically affect their operations. However, they acknowledge that

transition and training are critical to the next software release,3 which is

planned to reengineer and automate their work processes.

Incomplete plans for transitioning to a new system increase the likelihood

that the system will not be deployed on schedule. Insufficient training

plans also increase the likelihood that users will not readily adapt to new

work processes and systems.

Key Risk Areas

Transition Planning (cont’d)

3USCG has not yet scheduled the next software release for VDS.
Appendix I

GAO’s September 18, 2001, Briefing

Page 36 GAO-02-11 Coast Guard

33

Risk management is a key component of a sound systems development

approach. An effective risk management approach typically includes

identifying, prioritizing, resolving, and monitoring project risks. In support of

this approach, USCG’s risk management plan calls for

• assigning a severity rating (high, medium, or low) to risks that bear

particular attention and placing these risks on a “risk watch list,”

• prioritizing these risks,

• planning a response or strategy for each risk on the risk watch list, and

drafting a detailed response plan, and

• reviewing and evaluating all risks on the the risk watch list during

monthly management meetings.

Key Risk Areas

Risk Management
Appendix I

GAO’s September 18, 2001, Briefing

Page 37 GAO-02-11 Coast Guard

34

 USCG has not effectively implemented its risk management plan.

• USCG developed three different risk lists, and none of the risks were

assigned a severity rating.

• Risks on two of the risk lists have not been prioritized.

• USCG developed detailed mitigation strategies for some, but not all risks.

For example,

• detailed plans exist for data migration and transition risks

• such plans do not exist for managing user expectations and VDS

instability risks

• USCG officials addressed some, but not all, active risks at monthly status

briefings. Additionally, USCG officials stated that they discuss all active

risks during monthly management meetings; however, we found no

evidence that all risks were addressed at these informal meetings.

Inadequate risk management practices increase the likelihood that MISLE will

not be delivered on schedule, within budget, and able to perform as expected.

USCG officials acknowledged areas where they could improve their risk

management and have undertaken improvement efforts.

Key Risk Areas

Risk Management (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 38 GAO-02-11 Coast Guard

35

After a long history of problems in developing MISLE, USCG ended the

original MISLE contract in October 1999. Since that time, USCG has made

progress in developing MISLE and is now poised to deploy a minimum level

of functionality in November 2001. However, MISLE is several months

behind schedule and USCG has already spent most of its $61 million

acquisition cost estimate--most of it on the original contract. With only about

$11 million remaining, much remains to be done.

In its efforts to develop and deploy a complete MISLE system, USCG faces

significant challenges in several areas:

• managing system requirements and user expectations

• testing the system

• transitioning to an operational system, and

• managing program risks

Unless USCG can effectively address these challenges, MISLE deployment

schedules are likely to slip further and costs are likely to increase. Further,

MISLE may fall short of user expectations and its promised functionality.

Conclusions
Appendix I

GAO’s September 18, 2001, Briefing

Page 39 GAO-02-11 Coast Guard

36

To mitigate MISLE risks, we are making several categories of

recommendations to USCG’s MISLE program managers:

System Requirements

• Define and prioritize, in conjunction with system users, all needed

system functions, corrections, and enhancements that must occur to

meet valid user needs.

• Develop cost and schedule estimates for providing these functions,

corrections, and enhancements.

Software Testing

• Close all critical problems before initiating the next stage of testing.

Recommendations
Appendix I

GAO’s September 18, 2001, Briefing

Page 40 GAO-02-11 Coast Guard

37

Transition Planning

• Finalize and implement VDS transition plans.

• Develop and implement VDS training materials.

Risk Mitigation

• Develop a single list of system risks.

• Evaluate system risks to determine their severity and prioritize these

risks.

• Develop and implement comprehensive mitigation strategies for each

of the risks.

• Regularly oversee the status of risks and risk mitigation efforts to

determine whether additional mitigation activities are warranted.

Recommendations (cont’d)
Appendix I

GAO’s September 18, 2001, Briefing

Page 41 GAO-02-11 Coast Guard

38

In providing oral comments on a draft of this briefing, USCG officials

generally agreed with our recommendations, but disagreed with our

conclusion that unless the agency can address key risk areas, MISLE

deployment schedules are likely to slip further, costs are likely to increase,

and MISLE may fall short of user expectations and its promised

functionality.

 USCG officials stated that many of the changing requirements and testing

issues we raised were “tradeoff” decisions made to meet the primary goal

of delivering an initial version of MISLE to replace MSIS. Additionally,

USCG officials stated that some MISLE functions were deferred to later

releases in order to accelerate other user-requested functions. USCG

officials expressed confidence that they will deliver MISLE within budget

and on schedule.

Agency Comments and our Evaluation
Appendix I

GAO’s September 18, 2001, Briefing

Page 42 GAO-02-11 Coast Guard

39

Our evaluation: We acknowledge that USCG expects to meet its primary

goal of replacing MSIS in the next few months. However, MISLE will now

provide fewer capabilities than originally planned, and significant functions

have been delayed because of problems in testing.

 By dropping and delaying key MISLE functions, USCG runs the risk that the

system will fall short of user needs. Further, because there are no schedule

or cost estimates for most of the delayed functions, it is not clear when they

will be delivered or what they will cost.

Agency Comments and our Evaluation

(310331)

(310331) Letr
GAO’s Mission The General Accounting Office, the investigative arm of Congress, exists to 

support Congress in meeting its constitutional responsibilities and to help improve 

the performance and accountability of the federal government for the American 

people. GAO examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of accountability, 

integrity, and reliability.

Obtaining Copies of 

GAO Reports and 

Testimony

The fastest and easiest way to obtain copies of GAO documents is through the 

Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-text files of 

current reports and testimony and an expanding archive of older products. The 

Web site features a search engine to help you locate documents using key words 

and phrases. You can print these documents in their entirety, including charts and 

other graphics.

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO E-mail 

this list to you every afternoon, go to our home page and complete the easy-to-use 

electronic order form found under “To Order GAO Products.”

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. A check 

or money order should be made out to the Superintendent of Documents. GAO 

also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single 

address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

P.O. Box 37050

Washington, D.C. 20013

To order by Phone: Voice: (202) 512-6000 

TDD: (301) 413-0006 

Fax: (202) 258-4066

Visit GAO’s Document 

Distribution Center

GAO Building 

Room 1100, 700 4th Street, NW (corner of 4th and G Streets, NW)

Washington, D.C. 20013

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm,

E-mail: fraudnet@gao.gov, or

1-800-424-5454 (automated answering system).

Public Affairs Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G. Street NW, Room 7149, 

Washington, D.C. 20548

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Presorted Standard

Postage & Fees Paid

GAO

Permit No. GI00