Next Generation Enterprise Network: Navy Implementing Revised Approach, but Improvement Needed in Mitigating Risks
Published: 2012-09-19T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DON, a component of the Department of Defense (Defense), is replacing its existing network system with NGEN. Capabilities for the new system include secure transport of voice and data, data storage, and e-mail, at a cost of about $38 billion through fiscal year 2024. In March 2011, GAO reported that the approach for acquiring NGEN was not grounded in a reliable analysis of alternatives, the execution of NGEN was not based on a reliable schedule, and acquisition decisions were not always performance- and risk-based. GAO recommended that Defense, among other things, reconsider its approach. The department has not yet fully implemented GAOs recommendations but revised its approach to include acquiring certain NGEN services simultaneously instead of staggering their implementation.
GAO was asked to review the revised approach to determine (1) the extent to which DONs selected approach to acquire NGEN is the most cost effective and (2) the current status of and plans for acquiring NGEN. To do this, GAO reviewed analyses supporting the cost effectiveness of the acquisition approach, the programs revised acquisition strategy, integrated master schedule, key milestone decisions, and other relevant documents.
section_title: What GAO Found
paragraphs: While the Department of the Navy (DON) has revised its acquisition approach for its new network system, the Next Generation Enterprise Network (NGEN), it still has not shown that it has selected the most cost-effective approach for acquiring NGEN capabilities. Cost effectiveness is shown by comparing life-cycle costs and quantifiable and nonquantifiable benefits among alternatives, which can be accomplished by conducting a thorough analysis of alternatives. GAO previously identified weaknesses with the NGEN analysis of alternatives related to cost estimates and analysis of operational effectiveness and made associated recommendations. However, DON did not revisit the analysis of alternatives to address the weaknesses previously identified, nor did it conduct any other analysis that would show whether its revised approach is the most cost effective. For example, while DON developed a draft economic analysis in February 2012, the analysis assessed only the status quo and revised approach, and not other alternatives. As a result, GAO remains concerned with the analysis measuring NGEN cost effectiveness and DON does not know whether its revised approach for acquiring NGEN is the most cost effective.
Even though DON lacks assurance that it is pursuing the most cost-effective approach to acquiring NGEN capabilities, it has moved forward with implementing its revised approach. For example, the agency has completed activities to support the acquisition and transition to NGEN, prepared plans and analyses required for program initiation, and conducted oversight reviews to support the release of the request for proposals for transport and enterprise services (secure data and e-mail services, among other things). However, the programs schedule for acquiring NGEN capabilities has been delayed, thus making it more likely that DON will not be able to fully transition by the end of the continuity of services contract in April 2014. For example, the release of the request for proposals was delayed, and upcoming milestones, such as contract award and program initiation, have slipped (see table for major delays). Program officials attributed the delays to the need for additional planning and to revisions to the request for proposals. Compounding this situation is that identified risks are not being adequately mitigated. For example, not all mitigation plans are comprehensive because they do not always include all the elements of an effective plan (e.g., identification of resources needed) nor do they always contain the current status of the mitigation actions. According to program officials, weaknesses in these mitigation plans were due, in part, to the lack of a priority in establishing and maintaining comprehensive and current mitigation plans. As a result, the program faces an increased probability that transition from its existing system to NGEN will face further delays and cost overruns.
section_title: What GAO Recommends
paragraphs: GAO is recommending that Defense develop comprehensive risk mitigation plans for program-wide risks. In its comments, Defense concurred with GAOs recommendation and noted that it will continue to build on efforts to improve NGEN risk mitigation.

section_title: Letter
paragraphs: The Department of the Navy’s (DON) Next Generation Enterprise  Network (NGEN) program is expected to replace and improve information  technology (IT) services provided by the Navy Marine Corps Intranet  (NMCI), which currently provides about 382,000 computer workstations to  approximately 700,000 users across 2,500 United States Navy (USN) and  United States Marine Corps (Marine Corps) locations around the world.  As envisioned, NGEN is to provide secure data and IT services, such as  data storage, e-mail, and video teleconferencing, and its capabilities are  to be incrementally acquired through multiple providers. The first  increment is planned to provide comparable NMCI capabilities, as well as  enhanced information assurance and increased government control over  network operations. This increment is expected to be fully operational by  March 2014 and is estimated to cost approximately $38 billion through  fiscal year 2024.
In March 2011, we reported weaknesses in DON’s NGEN acquisition,  noting, for example, that its approach for acquiring the network  capabilities was not grounded in a reliable analysis of alternatives (AOA).As a result, we recommended, among other things, that the department  conduct an interim review to reconsider the acquisition approach. DON  subsequently reported that it had made changes to the NGEN acquisition  approach.
At your request, we conducted a study of DON’s revised approach for the  NGEN acquisition. Our specific objectives were to determine (1) the  extent to which DON’s selected approach to acquire NGEN is the most  cost effective and (2) the current status of and plans for this acquisition.
To determine the extent to which the selected acquisition approach is the  most cost effective, we reviewed DON documentation describing the cost  effectiveness of the acquisition approach, including the draft economic  analysis and other analyses supporting specific acquisition approach  changes. In addition, we interviewed cognizant DON and Office of the  Secretary of Defense officials about the use of these analyses in  acquisition decision making to ensure that NGEN capabilities are  acquired in the most cost-effective manner.
To determine the current status of and plans for acquiring NGEN, we  analyzed the program’s revised acquisition approach, integrated master  schedule, performance assessments, risk reports, and executive  acquisition decision briefings and meeting minutes, among other things.  We also discussed progress made on NGEN acquisition efforts with DON  program officials. Additional details on our objectives, scope, and  methodology are discussed in appendix I.
We conducted this performance audit from November 2011 to September  2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: DON is a major component of Department of Defense (Defense), and  consists of USN and Marine Corps service components. It is a large and  complex organization, whose primary mission is to organize, train,  maintain, and equip combat-ready naval forces capable of winning wars,  deterring aggression from foes, preserving freedom of the seas, and  promoting peace and security for the nation. To support this mission,  DON performs a variety of interrelated and interdependent IT-dependent  functions. In fiscal year 2012, DON’s IT budget was approximately $7.8  billion for 841 investments. NGEN, with a budget of $1.7 billion in fiscal  year 2012, is one such system investment.
subsections: section_title: Overview of NGEN
paragraphs: NGEN is to replace and improve the enterprise network and services  provided by NMCI, which were delivered through a DON-wide network  services contract with a single service provider (Hewlett Packard  Enterprise Services) that ended in September 2010. To bridge the time  between the end of the NMCI contract and the full transition to the first  increment of NGEN, DON awarded a $3.4 billion continuity of services  contract to Hewlett Packard Enterprise Services, which is scheduled to  run from October 2010 through April 2014. In addition to providing  continuity of network services, the contract includes transition services  and the transfer of NMCI infrastructure and intellectual property to DON.
When implemented, NGEN is to provide secure data and IT services,  such as data storage, e-mail, and video teleconferencing to USN and the  Marine Corps. It is also intended to provide the foundation for DON’s  future Naval Networking Environment. The network is to be developed incrementally, with the first increment expected to inherit the same  architecture and design, and provide the same capabilities and services  as does NMCI. In addition, NGEN is to provide increased DON control  over network operations and additional mandatory information assurance  capabilities to meet new Defense security requirements and the  implementation of an independent security validation function. Future  increments of the network have yet to be defined.
While NGEN’s first increment capabilities are not expected to differ from  those of NMCI, the operational environment for the network is expected to  change—from the contractor-owned and contractor-operated model  previously used by both services to a government-owned and contractor- operated model for USN and to a government-owned and government- operated model for the Marine Corps. In particular, USN plans to have  ownership and oversight of network operations while it relies on  contractors to execute and provide NGEN services. The Marine Corps  also plans to have ownership and oversight of network operations, but will  serve as its own service provider and obtain supplemental contractor  support as needed. The different operational models are intended to allow  USN and Marine Corps to operate their respective domains in the manner  best suited to support their different mission needs.
section_title: Oversight and Acquisition Processes for NGEN
paragraphs: To manage the acquisition and deployment of NGEN, DON established a  program management office within the Program Executive Office for  Enterprise Information Systems. In February 2011, DON merged the  NGEN program management office with the NMCI program management  office to form the Naval Enterprise Networks program management office.  This office manages the program’s cost, schedule, and performance and  is responsible for ensuring that the program meets its objectives. In  addition, various Defense and DON organizations share program  oversight and review responsibilities. These key entities include the
Under Secretary of Defense for Acquisition, Technology, and  Logistics. Serves as the Milestone Decision Authority, which is the  individual designated with overall responsibility for the program, to  include approving the program to proceed through its acquisition cycle  on the basis of, for example, the acquisition strategy, an  independently evaluated economic analysis, and the acquisition  program baseline. The Milestone Decision Authority is accountable for  cost, schedule, and performance reporting, including reporting to  Congress.
Assistant Secretary of the Navy, Research, Development, and  Acquisition. Serves as DON’s acquisition oversight organization for  the program, to include implementation of Under Secretary of Defense  for Acquisition, Technology, and Logistics policies and procedures.  Determines when all key milestones are ready to be submitted to the  Milestone Decision Authority.
Department of the Navy, Program Executive Office for Enterprise  Information Systems. Oversees a portfolio of large-scale projects  and programs designed to enable common business processes and  provide standard capabilities. Reviews the acquisition strategy,  economic analysis, and the acquisition program baseline prior to  approval by the Milestone Decision Authority.
Department of the Navy Chief Information Officer. Supports DON’s  planning, programming, budgeting, and execution processes by  ensuring that the program has achievable and executable goals and  conforms to financial management regulations and to DON, Defense,  and federal IT policies in several areas (e.g., security, architecture,  and investment management). Works closely with the program office  during milestone review assessments.
NGEN is designated as a Major Automated Information System (MAIS)  and is subject to both the Office of the Secretary of Defense’s and DON’s  MAIS acquisition policy and guidance, which require the program to  comply with defense acquisition system requirements. The defense  acquisition system consists of five key program life-cycle phases and  three related milestone decision points: (1) materiel solution analysis, (2)  technology development (milestone A held prior to entering this phase),  (3) engineering and manufacturing development (milestone B held prior to  entering this phase), (4) production and deployment (milestone C held  prior to entering this phase), and (5) operations and support. The  Milestone Decision Authority is to review the initial capabilities document,  which defines operational goals and needed capabilities, and authorize  the phase in which a MAIS program is to enter the defense acquisition  system.
In May 2010, the Under Secretary of Defense for Acquisition, Technology,  and Logistics authorized the NGEN program to enter the defense  acquisition system at production and deployment. NGEN was approved to  enter at this later phase because the technology was considered mature  and already operational under NMCI. Prior to entering the production and  deployment phase, a milestone C review must be held to review the  capability production document and the test and evaluation master plan,  among other things, and to authorize limited deployment to support  operational testing. The purpose of the phase is to achieve an operational  capability that satisfies the mission needs and is verified through  independent operational test and evaluation, and to implement the  system at all applicable locations. At milestone C, the NGEN program is  planned to be initiated and the acquisition program baseline to be  approved, establishing the cost, schedule, and performance thresholds  and objectives for the program. On approval of milestone C, DON will  proceed with award of the NGEN contracts for the transport and  enterprise services segments.
In addition to the defense acquisition system requirements, DON  guidance and policy require all MAIS programs to go through a “two- pass/six-gate” acquisition review process. The first pass, which consists  of gates 1 through 3, is focused on requirements development and  validation and is led by the Chief of Naval Operations or the Commandant  of the Marine Corps. The second pass, which consists of gates 4 through  6, is focused on developing and delivering a solution via systems  engineering and acquisition and is led by the Assistant Secretary of the  Navy (Research, Development and Acquisition). In addition to meeting  specific criteria for passing a given gate and proceeding to the next gate,  all gate reviews are to consider program health (i.e., satisfactory cost and  schedule performance, known risks, and budget adequacy) in deciding  whether to proceed. Table 1 lists the key purpose of each gate review.
In March 2011, we evaluated DON’s AOA for NGEN, which had  examined four acquisition alternatives. All of the alternatives were  assumed to deliver the same NMCI capabilities and the technology  considered for each alternative was assumed to be substantially the  same. As a result, DON officials stated that the AOA was not intended to  be a traditional analysis to determine a system solution, but rather was an  analysis of alternative acquisition approaches. The primary differences  among the alternatives related to how NGEN was to be acquired,  managed, and operated. Specifically, the alternatives varied in terms of  the number of contracts to be awarded and in the scope of government  versus contractor responsibilities. Table 2 provides a description and  comparison of each alternative that was examined in the AOA.
However, we reported that the approach pursued by DON did not match  any of the alternatives assessed in the AOA, and it was riskier and  potentially costlier than the alternatives assessed because it included a  higher number of contractual relationships. In particular, the chosen  approach was one that included more contracts, a different segmentation  scheme, and a different transition timeline than any of the alternatives  that had been assessed. We also reported that DON’s November 2009  risk-adjusted preliminary program life-cycle cost estimate for the  approach for fiscal years 2011 through 2015 showed that this approach  would cost at least an estimated $4.7 billion more than the alternatives  assessed in the AOA. Moreover, DON had not analyzed the impact of  these differences in terms of how they compared to the original  alternatives. Further, we identified key weaknesses in the cost estimates  and operational effectiveness analysis included in the NGEN AOA.  Specifically, we reported that, while the AOA cost estimates were  substantially well documented, they were not substantially accurate, and  they were neither comprehensive nor credible. Additionally, we reported  that, while the AOA identified program capabilities and goals, it did not  sufficiently assess the alternatives’ ability to satisfy the capabilities and  goals.
Given our findings, we recommended that Defense reconsider the  acquisition approach based on a meaningful analysis of all viable  alternative acquisition approaches. The department did not fully concur  with our recommendation and stated that it had concluded that DON’s  AOA was sufficient and that the analysis had been approved by the Office  of the Secretary of Defense, Cost Assessment and Program Evaluation.  The department added that it would complete an economic analysis for  milestone C, which would include a follow-on independent cost estimate  and an updated determination of the most cost-effective solution.  However, in response, we pointed out that DON planned to assess only  the status quo and the current approach in the economic analysis, not  other alternatives such as those that had been included in the AOA, and  we maintained that without a meaningful analysis of alternatives, the  department would be unable to determine the most cost-effective  solution.
We also reported that DON’s schedule for NGEN did not adequately  satisfy key schedule estimating best practices by, for example,  establishing the critical path (the sequence of activities that, if delayed,  impacts the planned completion date of the project) and assigning  resources to all work activities. Because it did not satisfy these practices,  the schedule did not provide a reliable basis for program execution.  According to program officials, schedule estimating had been constrained  by staffing limitations. However, these weaknesses contributed to delays  in the completion of NGEN events and milestones, including multiple  major acquisition reviews and program plans. Accordingly, we  recommended that Defense ensure that the NGEN schedule substantially  reflect key schedule estimating practices. The department partially agreed  with our recommendation.
Additionally, we reported that NGEN acquisition decisions were not  always performance- and risk-based. In particular, senior executives had  approved the program’s continuing progress in the face of known  performance shortfalls and risks. For example, in November 2009, the  program was approved at a key acquisition review despite the lack of  defined requirements, which officials recognized as a risk that would  impact the completion of other key documents, such as the test plan.  According to DON officials, the decision to proceed was based on their  view that they had sufficiently mitigated known risks and issues. We  recommended that the department ensure future NGEN acquisition  reviews and decisions fully reflect the state of the program’s performance  and its exposure to risks. The department agreed with our  recommendation.
section_title: DON Revised the NGEN Acquisition Approach
paragraphs: Subsequent to the issuance of our March 2011 report,reconsidered and made certain changes to the NGEN acquisition  approach. Specifically, in April 2012, the Office of the Secretary of  Defense approved NGEN acquisition approach changes that were  intended to support program executability and reduce program risk for  USN. Like the original approach, the revised approach emphasized  segmentation of the network, with the same five segments that had been  defined in the previous acquisition approach: the two primary segments  are enterprise services and transport services and the remaining three  segments are end user hardware; enterprise software licenses; and  verification, validation, and reporting (see table 3 for details on these  segments). Further, each segment is expected to be delivered by either a  contractor or government provider, with multiple competitive awards.  However, DON made changes to how certain NGEN segments are to be  acquired and transitioned. For example, it plans to solicit transport and  enterprise services using a single request for proposals and has said it  may award a combined contract for both segments; in addition, it plans to  transition both segments to the new provider(s) simultaneously instead of  staggering their implementation. According to DON officials, these  changes were made primarily because the transport and enterprise  services segments were integrally related under NMCI, so acquiring them  simultaneously would potentially reduce labor costs and administrative  burden, and reduce risk. Another change is that USN is expected to  acquire end user hardware as a service from the enterprise services  contractor rather than purchase the equipment and provide it as  government-furnished property to the contractor. According to program  officials, this change was made to mitigate a critical NGEN risk that the  program may not be fully funded if end user hardware must be purchased  in fiscal year 2014 and, in the long term, procuring the end user hardware  as a service is not more expensive than government-purchased  equipment. As an additional change to the acquisition approach, USN is  no longer expected to award a contract for the verification, validation, and  reporting segment because it now has an internal entity—the Tenth Fleet  Cyber Command—that is to perform this function. Table 3 summarizes  the previous and current plans for acquiring NGEN.
section_title: DON Has Not Reevaluated Alternatives to Ensure It Is Pursuing the Most Cost-Effective NGEN Acquisition Approach
paragraphs: According to cost estimating and acquisition guidance, cost  effectiveness is shown by a comparative analysis of all life-cycle costs  and quantifiable and nonquantifiable benefits among the competing  alternatives. Such an analysis should be used to examine viable  alternatives to inform acquisition decision making on the most promising  solution, without assuming a specific means of achieving the desired  result. For example, an AOA is initiated to examine potential solutions  with the goal of identifying the most promising option and can  subsequently be updated, as needed, to refine the proposed solution and  reaffirm the rationale in terms of cost effectiveness.economic analysis assesses net costs and benefits of the proposed  solution relative to the status quo and can identify and examine additional  alternatives that are considered feasible methods of satisfying the  objective.
Even after having revised its acquisition approach, DON has not yet  shown that it is pursuing the most cost-effective approach for acquiring  NGEN capabilities because it did not revisit the AOA to address the   nor did it conduct any other  weaknesses we previously identified, analysis that would show that the current approach is the most cost  effective. Officials told us they believe the approach they are now  pursuing remains consistent with the AOA we previously assessed.  However, the revised approach DON is now pursuing was not one of the  alternatives assessed because it differs from the AOA alternatives in  terms of transition timeline, segmentation scheme, and potentially the  number of contracts and the AOA still contains the issues we identified in  our previous report.
GAO-11-150. approach is the most cost-effective solution. Further, according to  program officials, the draft economic analysis is to be refined and updated  based on a revised service cost position, and is not expected to be final  until the acquisition program baseline is to be approved and about 3  months before the planned time frame for awarding the primary NGEN  contracts for transport and enterprise services, and thus, would be limited  in its ability to inform decision makers on the best NGEN approach to  pursue. Program officials agreed that the final economic analysis would  not be able to show the most cost-effective solution; they stated that the  economic analysis is being prepared because it is a required document  for program initiation (milestone C review).
DON also developed analyses to support changes to its acquisition  approach by examining whether a specific change to a particular segment  would be more cost effective. For example, DON examined whether it  should release one request for proposals instead of two for the transport  and enterprise services segments. Additionally, USN examined whether it  should acquire existing end user hardware owned by the incumbent and  provide it to the enterprise services contractor as government-furnished  property or acquire the end user hardware from the enterprise services  contractor as a service. However, because these analyses focus on  specific changes, they do not provide an understanding of whether DON’s  overall acquisition approach is the most cost effective. Without a  meaningful analysis of acquisition alternatives, DON does not know  whether its approach for acquiring NGEN capabilities and meeting NGEN  goals is the most cost effective among other viable alternatives.
section_title: DON Is Proceeding with the NGEN Acquisition, but Is Experiencing Schedule Delays and Not Adequately Mitigating Risks
paragraphs: Notwithstanding the lack of assurance that it is pursuing the most cost- effective acquisition, DON nonetheless has moved forward with its  revised approach for acquiring NGEN. In this regard, the department has  undertaken activities to support its acquisition and transition to NGEN,  prepared plans and analyses required for program initiation at milestone  C, and conducted oversight reviews to support the release of the request  for proposals for transport and enterprise services. However, the  program’s schedule for acquiring NGEN capabilities has been delayed,  resulting in a compressed timeline for transitioning to the new network  and increased risks associated with transitioning to the new network  before the end of the continuity of services contract. Compounding this  situation is the fact that identified risks that can further impact schedule  delays are not being adequately mitigated.
subsections: section_title: Execution of the NGEN Program Is Proceeding, but Major Milestones Have Slipped
paragraphs: DON has undertaken activities to support its acquisition and transition to  NGEN, prepared plans and analyses required for program initiation at  milestone C, and conducted oversight reviews to support the release of  the request for proposals for transport and enterprise services.  Specifically,
As of December 2011, DON had completed early transition activities,  such as developing IT service management strategies, processes,  procedures, and tools to serve as the overarching governance  framework for delivering NGEN capabilities; analyzing and validating  the current NMCI infrastructure inventory; and conducting job task  analyses and assessing learning tools for contractor technical  representatives. Additionally, the Marine Corps assumed control of  the NMCI infrastructure currently supporting its operations and  awarded the Marine Corps Common Hardware Suite contract to  procure NGEN end user hardware in May 2012. Also in May 2012,  DON reached agreement on the first of 12 planned enterprise  software license agreements. Further, the department released NGEN  documents and technical data to industry to ensure all competitors  have full access to NMCI technical data and to reduce the potential for  a protest. DON also released multiple requests for information and  solicited input from industry on a draft request for proposals in order to  better understand the capabilities of the current IT marketplace with  respect to NGEN requirements. Finally, DON released the request for  proposals for transport and enterprise services in May 2012.
DON has also prepared several plans and analyses required for  program initiation at milestone C, when the acquisition program  baseline is to be approved. In particular, in October 2011, DON  approved the cost analysis requirements description, which defines  the programmatic and technical features of NGEN increment 1 and  serves as the basis for estimating program costs. Additionally, the  Naval Center for Cost Analysis developed the service cost position,  which was based on the reconciliation of a completed program life- cycle cost estimate and an independent cost estimate. DON also  approved the systems engineering plan and the test and evaluation  master plan, which describes the overall test and evaluation strategy  for how the network’s capabilities will be assessed. Subsequently, in  November 2011, the program office developed the capability  production document, which clarified and solidified the capabilities for  NGEN increment 1 and became the primary source requirements  document for the program. Finally, the revised acquisition strategy,  which was required prior to release of the transport and enterprise  services request for proposals, was approved in April 2012.
According to Defense and DON policy, acquisition programs must  proceed through a series of gate and milestone reviews (as described  earlier in this report in table 1). Since our prior report, DON has  conducted two gate reviews and an Office of the Secretary of  Defense-level decision review to support the release of the request for  proposals for transport and enterprise services. In particular, in  October 2011, it completed an acquisition gate review to endorse the  NGEN increment 1 capability production document. Subsequently,  DON conducted a second NGEN acquisition gate review in January  2012 to approve the transport and enterprise services request for  proposals, during which it reviewed the current status and health of  the program including the key activities remaining to release the  request for proposals. Subsequently, in April 2012, the Office of the  Secretary of Defense, Milestone Decision Authority, reviewed the  NGEN program to approve the updated acquisition strategy and  authorize the release of the transport and enterprise services request  for proposals.
While DON has made progress on these efforts to acquire and transition  to NGEN, key program activities remain to be completed. For example,  DON will need to demonstrate that it is prepared to execute control and  governance of the network through four government readiness reviews;  baseline the program by establishing cost, schedule, and performance  thresholds and objectives; award the primary NGEN contract(s); and  transition to the new NGEN provider(s). Table 4 lists planned completion  dates for these remaining key activities.
However, a number of acquisition activities are facing schedule delays,  even though the incumbent is scheduled to end service delivery in April  2014. Specifically, while the request for proposals for transport and  enterprise services was issued, as we previously stated, it was delayed  by 17 months and 9 months, respectively, resulting in current delays in  NGEN program milestones, including the dates for conducting the  milestone C review and awarding the contract(s) for transport and  enterprise services. Additionally, the schedule for assessing USN’s  readiness to transition (i.e., government readiness reviews) is tied to  milestone C review and contract award, meaning that they are expected  to occur a certain number of days before or after their associated event;  thus, the government readiness reviews would also be impacted by  delays in milestone C and contract award. Moreover, these delays have  compressed the timeline for and increased the risks associated with  transitioning to the new network before the end of the continuity of  services contract. For example, the date for USN’s initial transition from  the current service provider to the new service provider(s) has slipped by  5 months and final transition is scheduled for March 2014, thus  compressing the period for shutting down network services with the  incumbent and transitioning them to the new NGEN contractor(s) by  about 5 months. Further, USN has identified a number of factors that  could impact transition and increase the risk that NGEN may not be  completed on time and may experience cost overruns, such as proposals  not meeting NGEN requirements and lack of coordination among  contractors and the government in operating the network.
Program officials attributed these schedule delays to the department’s  need to conduct more detailed planning before issuing the transport and  enterprise services request for proposals and for addressing industry  comments on the draft request for proposals to reduce the potential of a  bid protest. Figure 1 illustrates the delays in major NGEN milestones.
Our prior report highlighted the significance of DON not having a reliable  schedule for executing NGEN and its contribution to delays in key  program milestones. The lack of a reliable schedule, as we previously  noted, and the continuing delays in DON’s efforts to complete the network  transition as planned, raise concerns that it will be unable to complete the  transition within the time frames of the current continuity of services  contract. As a result, ensuring that the NGEN schedule substantially  reflects the key estimating practices, as we discussed and recommended  in our previous report, continues to be a vital step for DON to take.
section_title: Program Risk Mitigation Plans Have Not Been Fully Defined
paragraphs: According to industry best practices,process identifies potential problems before they occur, so that risk- handling activities may be planned and invoked, as needed, across the  life of the product and project in order to mitigate adverse impacts on  achieving objectives. Key activities of a comprehensive risk management  process include (1) identifying and analyzing risks, (2) escalating key  risks to the attention of senior management, and (3) developing risk  mitigation plans and milestones for key mitigation deliverables. In  particular, effective plans for risk mitigation should be developed for the  most important risks to the project, which includes a period of  performance, identification of resources needed, and responsible parties.  In addition, the status of each risk should be monitored periodically to  determine whether established thresholds have been exceeded and risk  mitigation plans should be implemented as appropriate to ensure that  systems will operate as intended.
NGEN program-wide and project-specific risks are managed by different  offices, with the program office identifying and tracking program-wide  risks—those that affect the overall NGEN program. In accordance with  best practices, the NGEN program identifies and analyzes program-wide  risks, by assigning a severity rating to risks, tracking these risks in a  database, and planning response strategies for each risk in the database.  In addition, NGEN program officials escalate these risks by reviewing and  evaluating these risks during monthly program risk management board  meetings. As of July 2012, the program office had identified eight  program risks that it considered critical (moderate- or high-level risks) and  that could result in schedule delays and cost increases. These risks  included potential delays in transition from the incumbent to the new  service provider(s) and in contract award for the transport and enterprise  services, as well as the potential lack of coordination among contractors  and the government in operating the network. Table 5 describes the  program-identified critical risks for NGEN as of July 2012.
While DON is working to mitigate seven of the eight program risks, its  mitigation plans did not always include all the elements of an effective  plan (e.g., identification of resources needed, responsible parties, and  period of performance). Specifically, the reported mitigation strategies did  not fully identify the resources needed, such as the staff and funds, nor  fully identify organizations that are responsible and accountable for  accomplishing risk mitigation activities. Additionally, while five of the  seven mitigation plans had activities with planned completion dates, most  did not include an estimated start date; thus, the plans did not fully define  the period of performance to ensure that the mitigation activities are being  implemented appropriately. Moreover, three of the seven plans did not  identify the status of activities for which completion dates had already  occurred. In particular, to mitigate the risk of potential lack of coordination  among contractors and the government in operating the network, DON  was to develop, implement, and automate key processes by February 15,  2012. However, the plan does not reflect whether this activity has been  completed or, otherwise discuss its status.
Additionally, two of the seven plans did not fully reflect the current status  of the program. For example, to mitigate transition risks, DON officials  identified that the enterprise and transport services contract(s) must be  awarded no later than December 2012 in order to ensure continuous  network availability during the transition from the continuity of services  contract to the NGEN contract(s). However, the current mitigation plan  does not document this milestone or reflect the current status of the  program, which now plans to award the contract(s) in February 2013.
Further, according to program documentation, a mitigation plan was  required and was being updated for the service offering descriptions risk;  however, according to other program documentation, the plan to mitigate  service offering descriptions is still under development, even though it has  been identified as a program risk since August 2011. According to  program officials, weaknesses in these mitigation plans were due, in part,  to the lack of a priority in establishing and maintaining comprehensive  and current plans.
Several of the risks identified are significant to ensure that NGEN  transition occurs as planned and within the estimated costs. Therefore it  is essential to ensure that for a given risk, techniques and methods will be  invoked to avoid, reduce, and control the probability of occurrence.
section_title: Conclusions
paragraphs: Even though DON does not know whether it is pursuing the most cost- effective approach to acquiring NGEN capabilities, it has proceeded to  implement its revised acquisition approach, completed various plans and  analyses including the first official program life-cycle cost estimate, and  held oversight reviews to support the issuance of the request for  proposals for the two primary NGEN segments. While these steps have  been taken, DON faces delays in upcoming milestones, which have  resulted in a compressed transition timeline and increased risks  associated with transitioning to the new network before the end of the  continuity of services contract. Compounding this are weaknesses in  DON’s risk mitigation efforts that could further impact schedule delays  and result in cost increases. Without a well-defined schedule, as we  previously reported, and adequate risk mitigation, DON cannot ensure  that needed NGEN capabilities will be in place in time to ensure that  services will continue to operate when the incumbent is scheduled to shut  down its services.
section_title: Recommendation for Executive Action
paragraphs: To strengthen risk mitigation activities for the NGEN program, we  recommend that the Secretary of Defense direct the Secretary of the  Navy to develop comprehensive mitigation plans and strategies for  program-wide critical risks that identify the mitigation period of  performance, resources needed, and responsible parties, and that fully  reflect the current status of the program.
section_title: Agency Comments
paragraphs: The Department of Defense provided written comments on a draft of this  report, signed by the Deputy Assistant Secretary of Defense (C3 and  Cyber), and reprinted in appendix II. In its comments, the department  agreed with our recommendation and noted that the program office will  continue to build on efforts to improve NGEN’s risk management and  mitigation process. For example, the department stated that it plans to  increase the speed at which NGEN risk management board action items  are closed.
We are sending copies of this report to the appropriate congressional  committees; the Director, Office of Management and Budget; the  Congressional Budget Office; the Secretary of Defense; and the  Secretary of the Navy. In addition, the report is available at no charge on  the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last   page of this report. GAO staff who made major contributions to this report  are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine (1) the extent to which the Department  of the Navy’s (DON) selected approach to acquiring the Next Generation  Enterprise Network (NGEN) is the most cost effective and (2) the current  status of and plans for acquiring NGEN.
To determine the extent to which DON’s approach to acquiring NGEN is  the most cost effective, we reviewed our prior work evaluating the NGEN  analysis of alternatives and analyzed current documentation that DON  had completed to describe and justify the cost effectiveness of its  acquisition approach. These included the draft economic analysis and  analyses to support specific acquisition approach changes. We assessed  DON’s supporting analyses against relevant Department of Defense  guidance and our Cost Estimating and Assessment Guide. In this regard,  we evaluated the purpose and use of these analyses in examining viable  alternatives to inform acquisition decision making on the most cost- effective solution. We also interviewed cognizant DON program officials  and Office of the Secretary of Defense Cost Assessment and Program  Evaluation officials about the use of these analyses in acquisition decision  making.
To determine the current status of and plans for acquiring NGEN, we  analyzed the revised NGEN acquisition strategy, integrated master  schedule, program performance assessments, risk reports, transport and  enterprise services request for proposals, planned system requirements,  cost estimates, draft operational readiness plan, and executive acquisition  decision briefings and meeting minutes, among other things. We also  reviewed these documents to determine how the program had changed  since our prior review of NGEN. To assess the status of risk management for the NGEN program initiative, we compared the risk  management plans and supporting documentation against leading  practices, such as Carnegie Mellon Software Engineering Institute’s  Capability Maturity Model® Integration for Services,whether such practices had been specified in the plans. Further, we  interviewed relevant DON program officials and Office of the Secretary of  Defense Cost Assessment and Program Evaluation officials to clarify  information in documents we reviewed and to more fully understand the  program’s progress to acquire NGEN. To assess the reliability of the data  that we used to support the findings in this report, we reviewed relevant  program documentation to substantiate evidence obtained through  interviews with agency officials. We determined that the data used in this  report are sufficiently reliable. We have also made appropriate attribution  indicating the sources of the data used.
We conducted this performance audit from November 2011 to September  2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, key contributors to this report  were Eric Winter (assistant director), Nabajyoti Barkakati, Harold Brumm,  Neil Doherty, Nancy Glover, Madhav Panwar, Jeanne Sung, and Niti  Tandon.