Coast Guard: Change in Course Improves Deepwater Management and Oversight, but Outcome Still Uncertain
Released: 2008-06-24T00:00:00

Letter

Background

The Coast Guard is a multimission, maritime military service within DHS. The Coast Guard’s responsibilities fall into two general categories—those related to homeland security missions, such as port security, vessel escorts, security inspections, and defense readiness; and those related to non-homeland security missions, such as search and rescue, environmental protection, marine safety, and polar ice operations. To carry out these responsibilities, the Coast Guard operates a number of vessels and aircraft and, through its Deepwater Program, is currently modernizing or replacing those assets. At the start of the Deepwater Program, the Coast Guard chose to use a system-of-systems acquisition strategy that would replace its assets with a single, integrated package of aircraft, vessels, and communications systems through ICGS, a lead system integrator that was responsible for designing, constructing, deploying, supporting and integrating the assets to meet Coast Guard requirements. Under this approach, the Coast Guard provided the contractor with broad, overall performance specifications—such as the ability to interdict illegal immigrants—and ICGS determined the specifications for the Deepwater assets. The decision to use a lead system integrator was driven in part because of the Coast Guard’s lack of expertise in managing and executing an acquisition of this magnitude.

In past reports on Deepwater, as well as the Army’s Future Combat Systems that is pursuing a similar acquisition approach for similar reasons, we have raised a number of concerns about this approach to acquiring complex systems. The role of a system integrator differs from that of a traditional prime contractor in that it includes increased responsibilities for ensuring that the design, development, and implementation of the system-of-systems it is under contract to produce meet established budget and schedule. The close working relationship with the government that this arrangement engenders has advantages and disadvantages. An advantage is that such a relationship allows flexibility in responding to shifting priorities. Disadvantages are the government’s weakened ability to provide oversight over the long term and the potential for increased costs.

In a series of reports since 2001, we have noted the risks inherent in the lead system integrator approach to the Deepwater Program and have made a number of recommendations intended to improve the Coast Guard’s management and oversight. In particular, we raised concerns about the agency’s ability to keep costs under control in future program years by ensuring adequate competition for Deepwater assets and pointed to the need for better oversight and management of the system integrator. We, as well as the DHS Inspector General and others, have also noted problems in specific acquisition efforts, notably the NSC and the 110-Foot Patrol Boat Modernization, which the Commandant of the Coast Guard permanently halted in November 2006 because of operational and safety concerns.

Coast Guard Has Established a More Accountable Acquisition Organization but Faces Challenges in Building Acquisition Workforce

Acknowledging that the initial approach to Deepwater gave too much control to the contractor, the Coast Guard has reoriented its acquisition organization to position itself to execute systems integration and program management responsibilities formerly carried out by industry. Project managers, whose role in the past was largely one of monitoring ICGS without the authority to make decisions, have now been vested with accountability for program outcomes. In addition, integrated product teams (IPT)—a key program management tool—are now led by Coast Guard officials, not contractor representatives. The Coast Guard has also increased its leverage of its own technical authorities and third party expertise. In the midst of these positive changes, the Coast Guard, like other federal agencies, faces challenges in building a capable government workforce to manage this large acquisition. While it attempts to reduce vacancy rates, it is relying on support contractors in key positions.

Reorganization of the Acquisition Directorate Has Potential Benefits for Deepwater

Since July 2007, the Coast Guard has consolidated acquisition responsibilities into a single acquisition directorate, known as CG-9, and is making efforts to standardize operations within this directorate. Previously, Deepwater assets were managed independently of other Coast Guard acquisitions by the Deepwater Program Executive Office in an insulated structure. The Coast Guard’s goal for the reorganization is to provide greater consistency in its oversight and acquisition approach by concentrating acquisition activities under a single official and allowing greater leveraging of knowledge and resources across programs. The Coast Guard’s consolidation of the acquisition function into a single directorate is consistent with best practices as it allows the agency to operate strategically to meet its overall missions and needs. Figure 1 depicts the changes to the Coast Guard’s acquisition structure.

Deepwater Management Improved with Better Use of Project Managers and Government Control over Integrated Product Teams

In conjunction with the restructuring of its acquisition directorate, Coast Guard officials have begun to increase the responsibilities and accountability of the project managers who oversee the acquisition of Deepwater assets. Previously, ICGS was charged with a number of key program management responsibilities—ranging from designing and constructing assets to developing concepts for deployment and operations—while Coast Guard program and project managers tracked and monitored the contractor’s activities. The Coast Guard’s new approach increases government control over these key elements of program management while vesting project managers with authority and accountability they lacked in the past. For example, a previous Deepwater management plan emphasized “partnership” between the Coast Guard and ICGS in managing Deepwater and “joint and ICGS responsibility for overall management and execution of the program, including authorization of necessary resources and resolving performance, cost, schedule, and risk tradeoff issues.” Under this scenario, according to Coast Guard officials, project managers could not provide as much direction as they wanted because of the terms of the contract, where ICGS bore ultimate responsibility for outcomes. In contrast, Coast Guard project managers are now responsible for defining, planning, and executing the acquisition projects within established cost, schedule, and performance constraints.

Another significant shift has been to assert government control over Deepwater integrated product teams. These teams, a key program management tool, consist of groups of project officials and technical experts responsible for discussing options for problem solving relating to cost, schedule, and performance objectives. In the past, the teams were led and managed by the contractor, while government team members acted as “customer” representatives. Now, the teams are led by Coast Guard personnel.

Figure 2 shows examples of how responsibility for program outcomes has shifted from ICGS to the Coast Guard.

Shift Away from System Integrator Involves Increased Use of Coast Guard Technical Experts and Third Parties

The Coast Guard is also establishing technical authorities within the agency who review, approve, and monitor technical standards and ensure that assets meet those standards. The Coast Guard has established a technical authority for engineering to oversee issues related to Deepwater, and officials state that a similar authority for C4ISR is pending. Previously, the Coast Guard held only an advisory role in making technical decisions, and in some cases this arrangement led to poor outcomes. For example, Coast Guard officials told us their engineering experts had raised concerns during the NSC’s design phase about its ability to meet service life requirements and recommended design changes, but were ignored. If the recommendations had been heeded, changes to the ship’s design could have been made earlier and some additional costs may have been avoided.

To supplement and enhance the use of its internal expertise, the Coast Guard has increased its use of third-party, independent sources of technical expertise and advice. For example, the Coast Guard is increasing its use of the American Bureau of Shipping (ABS), an independent organization that establishes and applies standards for the design and construction of ship and other marine equipment, to assist the Coast Guard in certifying that Deepwater vessels meet certain safety and performance standards. As a case in point there are 987 standards pertaining to hull, mechanical, and electrical systems on the first NSC which must be certified. Currently, ICGS is responsible for submitting documentation to the Coast Guard for 892 of the standards, while ABS and other third parties have a minimal role. In contrast, the Coast Guard plans for ABS to be responsible for reviewing approximately 200 certifications starting with the third NSC and to have an even broader role in certifying the design and production of future assets such as the Offshore Patrol Cutter (OPC) and Fast Response Cutter (FRC). In addition, the Coast Guard is using the U.S. Navy’s Space and Naval Warfare Systems Command to verify the security of certain communications systems and has established partnerships with Naval Sea Systems Command, the Navy Board of Inspection and Survey (INSURV), Naval Air Systems Command, and Naval Surface Warfare Center to leverage their expertise. INSURV, for example, conducted acceptance trials of the NSC in April 2008.

Coast Guard Faces Challenges in Building Acquisition Workforce

Effective management of acquisition programs depends on appropriately trained individuals properly placed within the acquisition workforce. In the initial development of the Deepwater contract, the Coast Guard sought a system integrator because it recognized that it lacked the experience and depth in workforce to manage the acquisition internally. The Coast Guard’s 2008 acquisition human capital strategic plan sets forth a number of acquisition workforce challenges that pose the greatest threats to acquisition success. Key challenges and Coast Guard actions to address them are cited below.

Shortage of Civilian Acquisition Staff

Like many federal agencies that acquire major systems, the Coast Guard faces challenges in recruiting and retaining a sufficient number of government employees in acquisition positions such as contract specialists, cost estimators, system engineers, and program management support. The Coast Guard has taken a number of steps to hire acquisition professionals, including the increased use of recruitment incentives and relocation bonuses, utilizing direct hire authority, and rehiring government annuitants. While some vacancies are to be expected in any organization and especially in an acquisition organization given current trends across the government, the Coast Guard is experiencing vacancy rates of almost 20 percent.

Lack of an Acquisition Career Path for Coast Guard Military Personnel

The Coast Guard also recognizes the impact of military personnel rotation on its ability to maintain people in key positions. The Coast Guard’s policy of regular three-year rotations of military personnel among units, including to and from the acquisition directorate, limits continuity in key project roles filled by military officers and can have a serious impact on the acquisition expertise gained and maintained by those officers. The presence of Coast Guard officers in the acquisition directorate is important, as they provide specialized expertise in Coast Guard operations and fill key positions as program and project managers and technology leads. While the Coast Guard concedes that it does not have the personnel required to form a dedicated acquisition career field for military personnel, such as that found in the Navy, it is seeking to improve the base of acquisition knowledge throughout the Coast Guard by exposing more officers to acquisitions as they follow their regular rotations. To build this base, the Coast Guard is creating acquisition policy courses at the Coast Guard Academy and other institutions and is working with the academy to create an internship program where interested officer candidates can work within the acquisition directorate.

Reliance on Contractors for Technical and Programmatic Expertise

Some of the positions that rely on technical and other expertise, such as project technology leads and contracting officials, remain vacant. In the absence of new personnel to fill these positions, the Coast Guard is forced to turn elsewhere. Officials stated that for some specialties, such as cost estimation, the Coast Guard can leverage existing relationships, such as with the Navy. However, because of a shortage of acquisition personnel across government, support contractors are often used to supplement government staff. For example, all the cost and earned value analysts currently employed by the aviation program are support contractors. Program managers stated that they would prefer these positions be filled by government employees. The head of contracting activity for the Coast Guard cited similar concerns, specifically for using contractors as contract specialists. The issue of support contractors in acquisition is not unique to the Coast Guard. In our recent report on the acquisition of major weapons systems in the Department of Defense (DOD), we found that it too relies heavily on contractors to perform roles in program management, cost estimation, and engineering and technical functions. For example, of the 52 programs we reviewed, support contractors represented 34 percent of program office staff for engineering and technical positions and 22 percent for program management functions.

While support contractors can provide a variety of essential services, their use must be carefully overseen to ensure that they do not perform inherently governmental roles. As we recently reported in our work on Army contracting practices, for example, using contractors as contract specialists can create the risk of decreased government control over and accountability for policy and program decisions when contractors provide services that closely support inherently governmental functions. Conflicts of interest, improper use of personal services contracts, and increased costs are also potential risks of reliance on contractors. According to officials, the Coast Guard is currently analyzing its workforce to better determine which roles are appropriate for contractors and to what extent support contractors can be used. In addition, it is investigating practices and policies to improve oversight of contractors and ensure their work remains in a supporting role.

Identifying Long-Term Needs Is Important for Sustained Progress

In order to provide a clearer picture of its future needs for acquisition personnel, the Coast Guard evaluated two potential workforce forecasting tools: one developed internal to the Coast Guard and another developed by the Air Force and tested as part of a broader effort by DHS. The Coast Guard tool is intended to forecast the potential workload of a project office and its acquisition staff requirements by determining the number of hours spent on specific acquisition-oriented work functions, such as contract management, business management, and systems engineering. Coast Guard officials stated that this tool has the potential, if managed correctly, to forecast workforce needs beyond the current fiscal year to enable long-term planning and workforce development. A potential weakness of the tool, according to the Coast Guard, is the significant time investment required of project managers to establish and maintain it. The other forecasting tool relies on historical DOD and Air Force data on program management, supplemented with annual interviews with appropriate project managers, to create estimates of workforce and workload needs. According to the Coast Guard, testing of both tools has been completed and a decision has been made to implement the Air Force staffing model.

Transition to Asset- by-Asset Approach Results in Greater Visibility and Control, but the Coast Guard Has Not Determined How to Manage System-Level Aspects

The Coast Guard’s move away from the ICGS contract and the system-of- systems model to a more traditional, asset-level acquisition strategy has resulted in greater government visibility and control. For example, cost and schedule information are now captured at the individual asset level rather than at the overall, system-of-systems program level, which was difficult to manage. At the same time, however, key aspects of Deepwater still require a system-of-systems approach. These aspects include the C4ISR system and the numbers of each Deepwater asset the Coast Guard requires to achieve its missions. The Coast Guard has not yet determined how to manage these aspects under its new paradigm, yet it is proceeding with Deepwater acquisitions.

Asset-Based Acquisition Approach Facilitates Improved Management

The Coast Guard’s transition away from the ICGS system-of-systems contract to an asset-by-asset acquisition strategy is enabling increased government visibility and control over its acquisitions. Cost and schedule information are now captured at the individual asset level rather than at the system-of-systems program level, which did not yield useful information for decision making. For example, while cost and schedule breaches in the past were to be reported to DHS at the Deepwater system- of-systems level only—an unlikely occurrence as only a catastrophic event would ever trigger a threshold breach under that approach—the Coast Guard is now reporting breaches by asset. In 2007, for example, the Coast Guard reported breaches for the NSC and for the C-130J. Because of a number of factors including changes to the ship’s design and requirements, the total acquisition cost of the NSC class increased by $520 million, or 15 percent, and the schedule for lead ship delivery was delayed by approximately 2 years. The cost increase for the C-130J is projected to be between 10 and 20 percent of the original contract price and stems from issues such as changes in requirements and concurrent design and installation activities.

The Coast Guard recently demonstrated this new approach of increased control over acquiring Deepwater assets by holding its own competition for the Fast Response Cutter-B (FRC-B), in lieu of obtaining the asset through the ICGS contract after determining that it could better control costs by doing so. According to the Coast Guard’s head of contracting activity, the contract award is expected in July 2008. The Coast Guard plans to hold other competitions outside of the ICGS contract for additional assets in the future. However, Coast Guard officials told us that, in the near term, they may continue to issue task orders under the ICGS contract for specific efforts, such as logistics, or for assets that are already well under way.

Coast Guard Is Not Fully Positioned to Manage Crucial System-Level Aspects of Deepwater

Although the shift to individual acquisitions is intended to provide the Coast Guard with more visibility and control, key aspects still require a system-level approach. These aspects include an integrated C4ISR system, which is needed to provide critical information to field commanders and facilitate interoperability with DHS and DOD, and the numbers of each Deepwater asset the Coast Guard requires to achieve its missions. The Coast Guard is not fully positioned to manage these aspects under its new paradigm. It has not approved an acquisition strategy for C4ISR and lacks at present the ability to model the capabilities of existing and planned assets in a way that could inform the numbers of Deepwater assets it requires. The Coast Guard maintains, however, that it must proceed with its acquisitions in the absence of this information.

C4ISR Acquisition Strategy Not Finalized

C4ISR is a key aspect of the Coast Guard’s ability to meet its homeland security, as well as its traditional, missions. How the Coast Guard structures C4ISR—referred to as the “architecture”—is fundamental to the success of the Deepwater Program. C4ISR encompasses the connections between surface, aircraft, and shore-based assets, the means by which information is communicated through them and the way information is displayed across that architecture—referred to as a common operating picture. C4ISR is intended to provide operationally relevant information to Coast Guard field commanders to allow for the efficient and effective execution of their missions across the full range of Coast Guard operations. The Coast Guard plans to integrate the Deepwater C4ISR architecture with legacy cutters and shore facilities as well in order to establish common components across all the assets and further enhance this effort. The Coast Guard recently had an unscheduled demonstration of new capabilities made possible through C4ISR improvements. In February 2008, a Maritime Patrol Aircraft (MPA) diverted from a training flight to participate in the rescue of two downed fighter pilots. With the C4ISR capabilities on board, the aircraft coordinated search and rescue efforts with a number of civilian and military assets it identified in the area.

According to Coast Guard officials, a C4ISR acquisition strategy is still in development. The Coast Guard recognizes the need to develop an architecture with common components for use on all assets. However, no agreement has been reached on whether to acquire C4ISR on an asset basis or at a system level. An asset-based approach for C4ISR would entail some risk, as interoperability among all Coast Guard units and DHS components, as well the Navy and others, must be assured.

Officials stated that the Coast Guard is revisiting the C4ISR incremental acquisition approach proposed by ICGS and analyzing that approach’s requirements and architecture. In the meantime, the Coast Guard is continuing to contract with ICGS for C4ISR. The first increment, now drawing to a close, is providing core capabilities for Deepwater assets, including common software. Program officials state that the second increment is planned to reduce the reliance on proprietary software and begin the migration toward government owned software where it is practical to do so. The third increment is anticipated to be a new C4ISR solution for the Coast Guard. As the Coast Guard continues to analyze its strategy for procurement of these and other C4ISR increments, a key concern will be to negotiate the data rights it needs to maintain and upgrade the necessary software.

Coast Guard Developing Modeling Capability to Assist in Determining Number of Each Asset to Procure

An additional risk in transitioning from a system-of-systems based acquisition strategy to an asset-based strategy is that the Coast Guard may lose the strategic vision needed to know how many of each Deepwater asset to procure to meet Coast Guard needs. When deciding how many of a specific vessel or aircraft to procure, it is important to consider not only the capabilities of that asset, but how it can complement or duplicate the capabilities of the other assets with which it operates. The Coast Guard has stated that it will continue to use a systems approach in determining the overall capabilities it needs but has not yet developed the tools necessary to make this assessment. For example, the Coast Guard recently contracted for a Deepwater alternatives analysis that revisited the acquisition approach for many of the individual assets and made a number of recommendations on options for future procurements. The analysis, in general, did not make recommendations about the number of each asset to be procured. It did, however, suggest revisiting the number of NSCs if the capabilities of the OPC allowed it to fill the same missions and eliminating the vertical unmanned aerial vehicle for technical and manufacturing reasons. Coast Guard officials stated that the study was abbreviated in scope because of the limited time available.

Senior Coast Guard officials, while stating that the mix of Deepwater assets identified in the alternatives analysis—such as small, medium, and large cutters—is generally reasonable, acknowledge the need to revisit the numbers of each of these assets to be procured in light of Deepwater capabilities as a whole and the move away from the ICGS solution. Officials state, however, that increased capabilities in modeling and simulations are necessary to fully inform this effort. According to officials, the Coast Guard is working to upgrade a model that plots the planned capabilities of Deepwater assets, as well as the capabilities and operations of existing assets, against the requirements for Coast Guard missions. The Coast Guard intends to use this model as a means of testing each planned asset to ensure its capabilities fill stated deficiencies in the Coast Guard’s force structure and to inform how many of a particular asset are needed given the capabilities of the rest of the force. Officials stated that they intend to use this analysis to inform their development of the Deepwater acquisition strategy. In the meantime, the Coast Guard continues to plan for asset acquisitions in numbers very similar to those determined by ICGS, such as procurement of 8 NSCs and 25 OPCs.

Disciplined Project Management Approach Is Beneficial Going Forward, but Key Decision Point Is Missing and Consequences of Prior Decisions May Be Costly

As the Coast Guard moves the Deepwater Program from a system-of- systems acquisition to a more traditional asset-based approach, it is introducing the use of a more disciplined and formalized process under its Major Systems Acquisition Manual (MSAM). While the introduction of this process is a significant improvement over the prior acquisition process, the absence of a key milestone decision point before low-rate initial production begins and the lack of formal approvals of acquisition decisions by DHS could be problematic. The consequences of not following a more disciplined acquisition approach, especially for the establishment and demonstration of mission requirements, are now apparent for assets already in production and are likely to pose continued problems—such as increased costs—for the Coast Guard.

Use of Major Systems Acquisition Manual Process Improves Oversight

The Coast Guard is now following the process set forth in its MSAM for all Deepwater assets. This process requires documentation and approval of program activities at key points in a program’s life-cycle. The MSAM represents a disciplined management approach that begins with an identification of deficiencies in overall Coast Guard capabilities and then proceeds through a series of structured phases and decision points to identify requirements for performance, develop and select candidate systems that match these requirements, demonstrate the feasibility of selected systems, and produce a functional capability. At each decision point, referred to as a “milestone,” entities across the Coast Guard, such as those responsible for oversight of the budget process or command and control, are to be consulted. Designated officials at high levels—including the Vice Commandant of the Coast Guard—then formally approve the program to proceed to the next phase. Each milestone requires documentation that captures key information needed for decision making. For example, when the Coast Guard makes its milestone decision, under the MSAM process, to proceed with the OPC from the initiation phase into development, the project office presented documentation that described the capabilities the ship is expected to provide, a draft concept for operations, and an initial assessment of cost and schedule. Figure 3 presents the key phases and milestones of the MSAM process and the current status of Deepwater assets within the process.

The MSAM process provides a number of benefits that have the potential to improve acquisition outcomes. Primarily, it requires event-driven decision-making by high ranking acquisition executives at a number of key points in an asset’s lifecycle. The process also requires documentation to provide the information and criteria necessary for these decisions. In addition, as the assets proceed through each phase of the process and the requirements and capabilities of the assets become more defined, these assets’ ability to fill deficiencies identified by the Coast Guard must be established.

Previously, the Coast Guard authorized the Deepwater Program to deviate from its major systems acquisition process, stating that the process was focused on acquiring discrete assets and contains requirements and documentation that may be inappropriate for the Deepwater system-of- systems approach. Instead, Deepwater Program reviews were required on a schedule-driven basis—planned quarterly or annually—to report the status and performance of the contractor’s efforts. Key decision points were focused primarily at the Deepwater Program as a whole and held only occasionally. Coast Guard officials told us that little, if any, formal documentation of key decisions was maintained.

Lack of Key Milestone in MSAM Process Poses Risks

GAO’s work on best practices for major acquisitions has demonstrated that a knowledge-based approach to decision making, where specific knowledge is gathered and measured against standards at key points in the acquisition process to inform decisions about the path forward, can significantly improve program outcomes. While the MSAM process contains many characteristics of a knowledge-based approach, there are key differences that could affect acquisition outcomes. For example, the Milestone 2 decision to approve low-rate initial production precedes the majority of the design activities in the capability development and demonstration phase. By following such a process, the Coast Guard may decide to enter production before a design is proven, a decision that could result in increased costs as design and production activities are conducted concurrently. In a previous report, we reviewed DHS’ acquisition process, with which the Coast Guard’s MSAM process is aligned and intended to complement, and found a similar weakness. Recognition and correction of this weakness in the MSAM approach is particularly important as key assets within Deepwater, most noticeably the FRC, approach a low-rate production decision.

Improved Oversight by DHS Necessary

The MSAM requires the Coast Guard to obtain approval from DHS on all major program decisions beginning with the start of an acquisition program. This requirement would apply to Deepwater, as it has been designated a DHS major investment program. However, DHS approval of Deepwater acquisition decisions as part of its investment review process is not technically necessary because the department deferred decisions on specific assets to the Coast Guard in 2003. The department did require notification of changes to the Deepwater Program that could result in significant changes to cost, schedule, and performance, but this requirement was at the overall systems level. In practice, the Coast Guard has increased communication and coordination with DHS through good will and informal procedures such as personal working relationships. While increased communication between the Coast Guard and DHS is to be applauded, without a formal process in place, DHS could lose the ability to make strategic decisions—such as how and whether to fund certain projects—across its components if informal procedures and relationships should change. Coast Guard and DHS officials told us that the processes and procedures for coordinating acquisitions with DHS’ Investment Review Board, which is tasked with reviewing major acquisition programs, are currently undergoing revision, and changes to the process are expected near the end of fiscal 2008.

Consequences of Not Following MSAM Process Are Apparent for Assets in Production

The Coast Guard is facing the consequences of its decision not to follow the MSAM process as it attempts to better define requirements for individual assets already being procured, such as the NSC, Long-Range Interceptor (LRI), and the MPA, and to ensure that desired capabilities are met within cost and schedule constraints. Under the MSAM, the requirements generation process takes broad mission needs and translates them to operational capability requirements and then to asset performance specifications. Figure 4 depicts this traceability from mission needs to performance specifications.

For example, under the MSAM process, before the design of an asset is selected, representatives of the operational forces within the Coast Guard are required to generate the Operational Requirements Document that determines the capabilities or characteristics considered essential to achieve their mission. Operational requirements described in this document—such as operating environment, functions to be performed, and the need for interoperability with other assets—ultimately drive the performance and capability of an asset and should be traceable throughout development, design, and testing. They should also include basic asset requirements such as speed, maneuvering, and range to serve as threshold and objective values for future trade-off analyses.

Under the ICGS-led system-of-systems acquisition approach, the Coast Guard developed high-level system requirements for capabilities, such as the ability to interdict illegal migrants. ICGS then developed an integrated force mix of specific aircraft, vessels, and communications systems to meet those needs. But because the disciplined MSAM approach was not followed, the Coast Guard could not trace the ICGS-proposed asset performance to actual mission needs. Program and project managers are “backfilling” the necessary requirements documentation in programs that are already well underway, with the intent of providing the traceability that was previously lacking. For example, in 2006, the Coast Guard acknowledged that the lack of a traditional requirements document for the NSC, which was then under construction, would inhibit the Coast Guard’s ability to evaluate the vessel’s suitability and effectiveness for Coast Guard missions. To resolve this problem, the Coast Guard developed a document that lists all the operational requirements for the NSC, as derived from identified mission needs, to guide operational testing. According to Coast Guard officials, operational testing based on these requirements will commence when the third NSC is complete. Under the MSAM, operational requirements would have been established prior to design and production to serve as the basis to link the asset’s performance to its ability to fill a mission need.

Failure to follow a disciplined approach in requirements generation is also apparent with problems related to the LRI, a small boat intended to be launched from larger cutters such as the NSC. The Coast Guard accepted the ICGS-proposed performance specifications for the LRI as part of the overall Deepwater specification, but the specifications were not tied to Coast Guard mission requirements. Thus, the Coast Guard had no assurance that the boat it was buying was what it needed to accomplish its missions. As a result of Coast Guard-identified technical deficiencies in the performance specifications, design changes were required after the LRI task order was issued. For example, a number of C4ISR specifications had to be added; the initial specification for the fuel tank size was deleted, as its capacity would not enable the boat’s 400 nautical mile range to be met; and a more powerful electrical system was needed. These and other changes, which were required for the boat to accomplish what ICGS had proposed, drove the price for design and production from $744,621 to almost $3 million. The Coast Guard is beginning to define needed capabilities for the LRI under the MSAM process, with an eye towards developing the service’s own requirements for the asset. For example, Coast Guard officials told us that ICGS’ proposed top speed of 45 knots is unrealistic and would under no circumstances be needed to accomplish Coast Guard missions. The LRI has been equipped with a C4ISR suite that officials believe to be much more extensive than they need. They are also concerned that the boat is too heavy, at 22,000 pounds.

The ramifications of accepting asset performance specifications not tied to Coast Guard mission requirements also became apparent during recent testing of the system that launches and recovers small boats, such as the LRI, from the NSC’s stern. Design changes to the launch system had to be made because it was found to be inadequate to handle the heavy weight of the LRI. The Coast Guard will pay for this change because the NSC is a cost-plus incentive fee contract. In addition, Coast Guard officials told us that the LRI’s inboard spray rail, which had initially been installed to enable the boat to reach 45 knots, had to be removed to allow the boat to effectively launch from the NSC, a cost ICGS will bear under that fixed- price contract. Coast Guard officials stated that the current LRI acquisition will be terminated with delivery of the first boat (now being considered a prototype with the potential to be used to test launch and recovery mechanisms on future NSCs).

The Coast Guard’s procurement of MPAs is another example of the consequences of not following a disciplined acquisition approach, as key program documents that establish the Coast Guard’s requirements for this asset and a plan for operational testing to those requirements have not been finalized. The testing is expected to occur between June 2008 and December 2008. The Coast Guard has contracted with ICGS for eight MPAs and accepted delivery of three aircraft between December 2006 and June 2007. In March 2008, it also accepted delivery of three mission system pallets, which provide the aircraft with C4ISR capabilities. The Coast Guard anticipates putting another 4 MPAs on contract with ICGS in fiscal year 2008 and has requested funding for the 13th and 14th aircraft.

Coast Guard Is Working to Improve the Use and Quality of Program Information, but Key Information Is Unreliable

The proper functioning of an acquisition organization and the viability of the decisions made through its acquisition process are only as good as the information it receives. The Coast Guard is developing two new means of communicating information related to the Deepwater Program. Quarterly project reports will consolidate and standardize how it communicates information to decision makers, and the probability of project success tool is intended to help officials discern and correct problems before they have cost and schedule impacts. However, Coast Guard officials have concerns about the reliability of the data they receive from the contractor as they lack the visibility required to determine the causes of cost and schedule variances. In addition, Coast Guard officials have stated that Northrop Grumman’s earned value system, which provides the necessary cost and schedule information, may need to be re-certified for compliance with government standards. While the Coast Guard is taking steps to improve its visibility into and confidence in data received from the contractor, it plans to proceed with issuance of a task order for long lead materials for the fourth NSC.

New Project Tools Designed to Better Convey Key Information to Decision Makers

The Coast Guard recently developed quarterly project reports, a compilation of cost and schedule information created by the project managers that summarizes the status of each acquisition for reporting through the Coast Guard as well as to DHS and the Congress. The Coast Guard developed these reports to standardize and consolidate asset reporting across all acquisitions, including those outside of Deepwater. Currently, the quarterly performance reports are being developed for 14 separate assets. The reports present general information about the project such as contract value and type, as well as more specific, timely information such as project accomplishments and risks. Project risks are rank-ordered by probability of occurrence and severity of impact, and include such things as technical challenges and production issues.

The Coast Guard has also begun to analyze program information using the “probability of project success” tool. This tool was developed by the Army and the Air Force to evaluate projects on factors other than basic cost, schedule, and performance data and is being considered by DHS for application across its acquisitions. Currently, the tool is being applied to the same 14 projects covered under the quarterly performance reports. Coast Guard acquisition officials told us they will use this tool to grade each asset on 19 different elements in 5 categories, including project resources and project execution, to assess the risk of assets failing to meet their goals. Figure 5 lists these categories and elements.

The probability of project success tool is developed by acquisition support staff separate from the program and project offices. Of the 19 different elements, only one, health of the contractor, is graded by the project manager. The results of this tool are not seen as an assessment of the project manager, but of the support that the acquisition directorate has given them. Officials stated that the tool allows acquisition executives to identify projects that require assistance before they experience cost breaches or other problems and also allows for a comparison of risks and challenges across all Coast Guard acquisition projects to identify trends.

Usefulness of Earned Value Management Information Is Questionable

The production and analysis of earned value management data—the cost and schedule data reported by the contractor and used to evaluate progress toward program goals—are critical to informing both the quarterly performance reports and the probability of project success tool. However, Coast Guard officials are concerned about the utility of the earned value data they receive because, under the terms of the ICGS contract, they lack visibility at the levels required to inform decision- makers and manage projects. In addition, officials believe that Northrop Grumman’s earned value system may require re-certification to meet government standards to ensure the reliability of the data. Receiving useful and reliable earned value data could be particularly important for the Deepwater Program, as these data are also used to inform decisions on future projects, such as the pending orders to Northrop Grumman for the materials and production of the fourth NSC.

Coast Guard officials expressed concerns about the level of detail of the earned value data provided by ICGS. A Coast Guard official responsible for analyzing the contractor’s reported earned value data for the NSC stated that the data do not provide sufficient visibility for decision making at the asset level. The concerns stem in part from the system-of-systems contract structure with ICGS and how the terms for reporting earned value data to the government were negotiated. Earned value data are reported at different levels of activity, descending in order from the general to the specific, as determined in advance by the government. The levels of activity required for earned value reporting are very important and can determine the usefulness of the data received. Under the ICGS contract, the earned value data are reported at seven levels, beginning with the Deepwater system-of-systems level—”ICGS”—and stopping at the major component level—such as propulsion and armaments. Coast Guard officials stated that previously data on the NSC was reported to the fifth level, which only presents data on the progress of production of the cutter as a whole. A Coast Guard official stated that in order to gain adequate visibility into reported cost variances, a deeper level of reporting is necessary. While the Coast Guard has negotiated a more detailed level of earned value reporting on the first three NSCs to receive data at the major component level, according to an official, the Coast Guard may seek even more detailed levels of cost data in upcoming negotiations for the fourth NSC.

In addition to concerns about visibility into contractor earned value data, Coast Guard officials have concerns about the reliability of the underlying systems the contractors use to collect this data. An important consideration in relying on contractor-provided earned value management data is ensuring that the contractor’s process for generating the data is compliant with government standards. Contractors are expected to have earned value management plans that document the methodology, products, and tools they have in place to track earned value. Independent third parties, such as the Defense Contract Management Agency (DCMA) or the Defense Contract Audit Agency, ensure the contractor’s initial compliance with government standards and perform surveillance reviews to ensure that the contractor remains compliant. While Lockheed Martin’s earned value management system has been certified as compliant by DCMA, Coast Guard officials have stated that Northrop Grumman—the first tier subcontractor responsible for work on the NSC—may require re- certification. Previously, Northrop Grumman’s earned value management system had been certified by the Navy, but this certification is no longer considered acceptable by the Coast Guard. According to officials, the Coast Guard is working with DCMA and the Navy to review and, if necessary, re-certify Northrop Grumman’s earned value system. In the meantime, the Coast Guard intends to improve its insight into how the contractor produces and reports earned value data by executing a memorandum of agreement with the DCMA for on-site surveillance at the shipyard. Such on-site presence is critical to increase the likelihood that the Coast Guard receives accurate earned value data.

These concerns about visibility into, and reliability of, earned value data affect not only the information the Coast Guard needs for decision making on current projects, but also the information necessary for decisions on future projects, such as the production of the fourth NSC. As the Coast Guard compiles earned value information on the ships being constructed by Northrop Grumman, it can use this information in the estimates of future costs used to establish target prices for additional work to be performed. Because the Coast Guard lacks confidence in how Northrop Grumman is representing its cost and schedule performance on current projects, it may be in the position of paying the contractor for future projects, such as the long lead material and production of the fourth NSC, without the understanding necessary to evaluate proposed prices.

Conclusions

In response to significant problems in achieving its intended outcomes under Deepwater, Coast Guard leadership has made a major change in course in its management and oversight of this program. Even with this change, the Coast Guard continues to face numerous risks of varying magnitude in moving forward with an acquisition program of this size. While the initiatives the Coast Guard has underway have already begun to have a positive impact on reducing these risks, the extent and durability of their impact depends on positive decisions that continue to increase and improve government management and oversight.

The current reliance on informal procedures to keep DHS informed of Deepwater developments is not appropriate for an acquisition of this magnitude. The Deepwater Program will continue for some time to come, and the full burden of transcending the inevitable challenges should not rest solely with the initiatives of the current Coast Guard leadership. The Coast Guard’s major systems acquisition process requires DHS approval of milestone decisions; however, the 2003 DHS delegation to the Coast Guard of such approval means that DHS does not have formal approval authority, and it could lack the information needed to strategically allocate funding by balancing requirements and needed capabilities across departmental components. In addition, the Coast Guard’s acquisition process calls for a decision to authorize initial production before knowledge is gathered about the stability of an asset’s design and production processes, which is contrary to best practices and could result in cost increases and schedule delays because of redesign. And because the Coast Guard’s knowledge of the reasonableness of contractors’ proposed cost and schedule targets for Deepwater assets relies in part on visibility into and confidence in the contractors’ earned value management data, the Coast Guard may lack a solid basis to evaluate future proposals by Northrop Grumman until known problems with its data are resolved.

Recommendations for Executive Action

To help ensure that the initiatives to improve Deepwater management and oversight continue as intended and to facilitate decision-making across the department, we recommend that the Secretary of Homeland Security direct the Under Secretary for Management to rescind the delegation of Deepwater acquisition decision authority.

We also recommend that the Commandant of the Coast Guard take the following two actions: To improve knowledge-based decision-making for its acquisitions, revise the procedures in the Major Systems Acquisition Manual related to the authorization of low-rate initial production by requiring a formal design review to ensure that the design is stable as well as a review before authorizing initial production.

To improve program management of surface assets contracted to Northrop Grumman Ship Systems, develop an approach to increase visibility into that contractor’s earned value management data reporting before entering into any further contractual relationships, such as for long lead material for and production of the fourth NSC.

Agency Comments and Our Evaluation

In written comments on a draft of this report, the Department of Homeland Security concurred with our findings. The department stated that it would take our recommendation on rescinding the delegation of Deepwater acquisition decision authority under advisement, but neither concurred nor disagreed with the recommendation. The Coast Guard concurred with our recommendation on requiring a formal design review before low-rate initial production, and plans to incorporate such a review in its next revision of the MSAM process. In addition, it partially concurred with our recommendation to improve program management of surface assets by developing an approach to increase visibility into Northrop Grumman’s earned value management data. The Coast Guard stated that it agrees with the recommendation and is in the process of funding DCMA for surveillance of Northrop’s earned value system and increasing the level of visibility into Northrop’s data starting with the fourth NSC production contract. However, the Coast Guard stated that earned value data would provide limited utility for the fixed-price long lead materials contract for this acquisition and that obtaining the data would pose a significant cost and schedule impact. To determine a fair and reasonable price for the long lead and production contracts, the Coast Guard plans to obtain and review Northrop’s certified cost and pricing data. It appears to us that the Coast Guard has developed an approach for increasing visibility into the earned value management data for future contracts with Northrop Grumman. We believe this approach, if implemented as planned, will address our recommendation.

The comments from the Department of Homeland Security are included in their entirety in appendix III. Technical comments were also provided and incorporated into the report as appropriate.

We are sending copies if this report to interested congressional committees, the Secretary of Homeland Security, and the Commandant of the Coast Guard. We will provide copies to others on request. This report will also be available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have any questions about this report or need additional information, please contact me at (202) 512-4841 or huttonj@gao.gov.

Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Staff acknowledgements are provided in appendix IV.

Appendix I: Scope and Methodology

Overall, in conducting this review, we relied in part on the information and analysis in our March 2008 report, Status of Selected Aspects of the Coast Guard’s Deepwater Program and testimony, Coast Guard: Deepwater Program Management Initiatives and Key Homeland Security Missions. We also reviewed the Coast Guard’s 2007 Deepwater expenditure plan and fiscal year 2009 budget request. Additional scope and methodology information on each objective of this report follows.

To assess the Coast Guard’s efforts to increase accountability and program management through its reorganized acquisition function, we reviewed the Coast Guard’s July 2007 Blueprint for Acquisition Reform, organizational structures before and after the July 2007 reorganization, 2004 and 2008 Deepwater Program Management Plans, and project manager and integrated product team charters. We also interviewed senior acquisition directorate officials, program and project managers, and Integrated Coast Guard Systems (ICGS) representatives to discuss the Coast Guard’s increased role in program management and oversight and changes in roles and responsibilities of key positions. We held discussions with officials from the Coast Guard’s engineering and C4ISR technical authorities and the American Bureau of Shipping, and reviewed lists of certifications for the National Security Cutter (NSC). To assess Coast Guard actions to improve the acquisition workforce, we reviewed additional documentation such as the acquisition human capital strategic plan, documentation of workforce initiatives, and organization charts for aviation, surface, and C4ISR components showing government, contractor, and vacant positions. We supplemented the documentation review with interviews of acquisition directorate officials, including contracting and Office of Acquisition Workforce Management officials and program and project managers. We discussed workforce initiatives, challenges and obstacles to building an acquisition workforce, recruiting, difficulty in filling key positions, use of support contractors, inherently governmental positions, and tools for projecting acquisition workforce needs. We spoke with representatives of a support contractor developing one of the workforce planning tools. We also relied on our past work identifying management and workforce problems within the Deepwater Program and the Department of Defense (DOD).

To evaluate the Coast Guard’s transition to an asset-based paradigm for Deepwater, including how system-level aspects such as C4ISR are being managed, we analyzed a 2007 alternatives analysis prepared for the Coast Guard. We also discussed the planned C4ISR procurement strategy with the acquisition directorate C4ISR program manager and the Coast Guard Chief, Office of Cyber Security and Telecommunications. We reviewed the overall Deepwater and the NSC acquisition program baselines. Other acquisition program baselines were in draft form and not made available to us. We reviewed reports on NSC and C-130J missionization cost breaches to understand the change in how cost breaches are reported to DHS under the new approach. We analyzed the Long-Range Interceptor (LRI) task order and associated modifications and interviewed senior acquisition directorate officials, the surface asset program manager, and the LRI project manager about problems with the LRI’s design and its ability to interface with the NSC’s launch and recovery system during testing. We reviewed documentation of the Coast Guard’s acceptance of the first three Maritime Patrol Aircraft and associated mission system pallets and interviewed the aviation program manager.

To assess the Coast Guard’s implementation of a disciplined, project management process for Deepwater acquisitions, we reviewed the Major Systems Acquisition Manual and compared its processes with the knowledge-based, best practices processes we have identified through our prior work on large acquisitions at DOD. We reviewed the Coast Guard’s April 2000 memorandum waiving the acquisition manual requirements for the Deepwater Program to understand the rationale for the waiver, as well as the 2003 DHS memorandum giving the Coast Guard acquisition decision authority for Deepwater assets. We interviewed acquisition directorate officials and program and project managers to discuss efforts to transition the acquisition of Deepwater assets to the MSAM process, particularly for assets already under way. We also spoke with DHS officials about the DHS major acquisition review process and reporting requirements.

We assessed Coast Guard initiatives to improve the quality of program management information by analyzing Deepwater asset quarterly project reports for the fourth quarter, fiscal year 2007, and probability of project success information. We also analyzed selected earned value management cost performance reports for the NSC and reviewed earned value management system compliance letters for Northrop Grumman and Lockheed Martin, the Coast Guard’s standard operating procedure for earned value management systems, the Deepwater work breakdown structure dictionaries for Northrop Grumman and Lockheed Martin, and ICGS’ earned value management plan. We discussed the information contained within this documentation with acquisition directorate officials, the NSC business finance manager, Coast Guard support contractors responsible for analyzing the earned value management data, and ICGS and Northrop Grumman representatives.

We conducted this performance audit from October 2007 to June 2008 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Information on Selected Deepwater Surface and Air Assets

Appendix III: Comments from the Department of Homeland Security

Appendix IV: GAO Contact and Staff Acknowledgments

Acknowledgments

In addition to the individual named above, Michele Mackin, Assistant Director; J. Kristopher Keener; Martin G. Campbell; Maura Hardy; Angie Nichols-Friedman; Scott Purdy; Kelly Richburg; Raffaele Roffo; Sylvia Schatz; and Tatiana Winger made key contributions to this report.

GAO Products Related to the Deepwater Program

Status of Selected Aspects of the Coast Guard’s Deepwater Program. GAO-08-270R (Washington, D.C.: Mar. 11, 2008).

Coast Guard: Deepwater Program Management Initiatives and Key Homeland Security Missions. GAO-08-531T (Washington, D.C.: Mar. 5, 2008).

Coast Guard: Challenges Affecting Deepwater Asset Deployment and Management and Efforts to Address Them. GAO-07-874 (Washington, D.C.: June 18, 2007).

Coast Guard: Status of Efforts to Improve Deepwater Program Management and Address Operational Challenges. GAO-07-575T (Washington, D.C.: Mar. 8, 2007).

Coast Guard: Status of Deepwater Fast Response Cutter Design Efforts. GAO-06-764 (Washington, D.C.: June 23, 2006).

Coast Guard: Changes to Deepwater Plan Appear Sound, and Program Management Has Improved, but Continued Monitoring is Warranted. GAO-06-546 (Washington, D.C.: Apr. 28, 2006).

Coast Guard: Progress Being Made on Addressing Deepwater Legacy Asset Condition Issues and Program Management, but Acquisition Challenges Remain. GAO-05-757 (Washington, D.C.: July 22, 2005).

Coast Guard: Preliminary Observations on the Condition of Deepwater Legacy Assets and Acquisition Management Challenges. GAO-05-651T (Washington, D.C.: June 21, 2005).

Coast Guard: Deepwater Program Acquisition Schedule Update Needed. GAO-04-695 (Washington, D.C.: June 14, 2004).

Contract Management: Coast Guard’s Deepwater Program Needs Increased Attention to Management and Contractor Oversight. GAO-04-380 (Washington, D.C.: Mar. 9, 2004).

Coast Guard: Actions Needed to Mitigate Deepwater Project Risks. GAO-01-659T (Washington, D.C.: May 3, 2001).