DOD Major Automated Information Systems: Improvements Can Be Made in Applying Leading Practices for Managing Risk and Testing
Released: 2017-03-30T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: DOD's MAIS programs include systems that are intended to help the department sustain its key operations. The National Defense Authorization Act for Fiscal Year 2012 includes a provision for GAO to select, assess, and report on the department's MAIS programs annually through March 2018. This is GAO's fifth report and (1) describes the extent to which selected MAIS programs have changed their planned cost and schedule estimates and met technical performance targets and (2) assesses the extent to which selected MAIS programs have used leading IT acquisition practices, including risk management.

GAO selected and reviewed cost, schedule, and performance data for 18 of DOD's MAIS programs that were non-classified and had an acquisition performance baseline. In addition, GAO performed an in-depth review of 5 of the programs, comparing selected IT management practices used by them to leading practices for requirements and risk management and systems testing and integration. The five selected programs were from at least two military services and had not been assessed by GAO in the past year. GAO also interviewed relevant program officials.

section_title: What GAO Found
paragraphs: Most of the 18 selected Department of Defense (DOD) major automated information system (MAIS) programs that GAO reviewed had experienced changes in their planned cost and schedule estimates and half of the programs had met their technical performance targets. Specifically, 16 programs experienced changes in their cost estimates ranging from a 39 percent decrease ($1.47 billion) to a 469 percent increase ($1.63 billion). The average cost increase was $457.2 million among the 11 programs reporting an increase. Fourteen programs experienced schedule delays, which ranged from 2 months to over 13 years. Finally, half of the MAIS programs fully met all of their technical performance targets. Of the remaining nine programs, 4 four had partially met their target because each was still conducting tests. The other five programs were in the early stages of system development and had not begun testing.

In addition, for the five MAIS programs GAO selected for in-depth review, all had either fully or partially applied leading practices for managing requirements, risks, and for conducting systems testing and integration.

Managing requirements. Three of the five programs had fully implemented the practices for managing requirements, while the other two had partially implemented some practices. Leading practices in this area include establishing requirements and ensuring traceability between requirements and work products.

Managing risks. Three of the five programs had fully implemented the risk management practices, while two had partially implemented some practices. An effective risk management process identifies potential problems before they occur. For example, one Army program did not have standard operating procedures for defining thresholds or bounds to manage risk. Unless such procedures are defined, the program will not have the tools needed to define risk management activities, including whether and how certain risks are prioritized. Further, programs should include practices to identify potential problems so that risk-handling activities may be planned and invoked across the project to mitigate the potential for adverse impacts. However, one Air Force program did not develop an overall risk mitigation plan to guide the implementation of individual risk mitigation activities. Without an overall risk plan to guide individual development efforts, those efforts cannot be managed cohesively.

Testing and integration. Four of the five programs had fully implemented practices for systems testing and integration. Programs should, among other activities, establish roles and responsibilities to manage testing and integration activities, including a chief developmental tester to oversee testing activities. However, one Air Force program reported difficulty in hiring a qualified individual to perform these duties. Until this position is filled, the program may not effectively manage risks and verify compliance with system acquisition and operational requirements.

section_title: What GAO Recommends
paragraphs: GAO recommends that DOD improve the management of specific MAIS programs, including establishing procedures for defining risk thresholds, developing an overall risk mitigation plan, and filling a key test management position. DOD concurred with all of the recommendations.

section_title: Letter
paragraphs: The Department of Defense (DOD) is one of the largest and most  complex organizations in the world. To meet its mission to protect the  security of our nation and to deter war, it relies heavily on the use of  information technology (IT) to support our warfighters. In this regard,  according to DOD’s IT investment portfolio for fiscal year 2016, the  department spent approximately $31 billion for such investments. Of this  amount, approximately $2.5 billion was spent on major automated  information system (MAIS) programs, which include systems that help the  department sustain its key operations in communications, business, and  command and control, and provide the department with access to  information to organize, plan, direct, and monitor mission operations.

A DOD IT acquisition investment that exceeds one of the following cost  thresholds is designated as a MAIS program: (1) program costs in any  single year exceed $40 million, (2) total program acquisition costs exceed  $165 million, or (3) total life-cycle costs exceed $520 million in fiscal year  2014 constant dollars. The Secretary of Defense can also use discretion  to designate a program as a MAIS if it does not meet these cost  thresholds.

The National Defense Authorization Act for Fiscal Year 2012 includes a  provision that we select, assess, and report on DOD MAIS programs  annually through March 2018. This report is the fifth in a series of annual  assessments. Our objectives for this review were to (1) describe the  extent to which selected MAIS programs had changed their planned cost  and schedule estimates, and met technical performance targets and (2)  assess the extent to which selected MAIS programs have used leading IT  acquisition practices, including requirements and risk management, and  systems testing and integration.

To address the first objective, we used DOD’s official list of 35 MAIS  programs, as of May 1, 2016, to establish a basis for selecting programs.  We selected 18 of 35 MAIS programs based on our criteria that programs  must be unclassified and have a first acquisition program baseline that  could be used as a reference point for evaluating cost, schedule, and  technical performance characteristics.

We then compared each program’s cost (in then-year dollars) and  schedule estimates established in the first acquisition baseline to the  latest total life-cycle cost and schedule estimates. In addition, to  determine whether technical performance targets were met, we reviewed  each program’s initial and the most recent baseline targets. We  summarized the results of our analyses for each individual MAIS  program.

To address the second objective, we selected 5 programs for a more in- depth review from among those used by at least two military services and  that had not been included in a GAO assessment in the past year to focus  on programs that have not been recently evaluated. Using these criteria,  we selected the 5 programs listed in table 1.

To evaluate DOD’s approach in managing the acquisition of MAIS  systems, we identified risk management and requirements management  practices from the Software Engineering Institute’s Capability Maturity  Model® Integration for Acquisition (CMMI-ACQ), and assessed each of  the programs against these leading practices. With regard to risk  management, we analyzed each program’s key documents, such as risk  management plans, risk charts, and other artifacts, and compared them  to the leading practices. For requirements management, we compared  requirements documents, such as the requirements management plan,  traceability matrix, and procedural tools to the leading practices.

We also identified key systems testing and integration practices from the  Institute of Electrical and Electronics Engineers, Incorporated (IEEE),  and DOD’s policy and guidance (DODI 5000.02, Defense Acquisition  Guidebook, and Test & Evaluation Management Guide) and assessed  each of the five programs against these practices. For example, we  evaluated the test manager roles and responsibilities, test plans, and test  reports to determine the extent to which these practices were applied. We  interviewed program officials to obtain additional information on each  program’s risk and requirements management processes, and systems  testing and integration practices.

We conducted this performance audit from April 2016 to March 2017 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. See appendix I for a more  detailed discussion of our objectives, scope, and methodology.
subsections: section_title: Background
paragraphs: DOD’s organizational structure includes the Office of the Secretary of  Defense, the Joint Chiefs of Staff, the military services, numerous  defense agencies and field activities, and various unified combatant  commands that contribute to the oversight of DOD’s MAIS programs.  Figure 1 provides a simplified depiction of DOD’s organizational structure.

The Under Secretary of Defense for Acquisition, Technology, and  Logistics (USD (AT&L)) serves as the Defense Acquisition Executive and  is the official responsible for supervising the acquisition and oversight of  MAIS programs. This official is the principal acquisition official of the  department and is the acquisition advisor to the Secretary of Defense.  The USD (AT&L) has policy and procedural authority for the defense  acquisition system, which establishes the steps that DOD programs  generally take as the department plans, designs, acquires, deploys,  operates, and maintains its IT systems (discussed in more detail following  this section). The USD (AT&L)’s authority also includes directing the  military services and defense agencies on acquisition matters and— unless the authority is delegated to the service or agency—making  milestone decisions for MAIS programs.
subsections: section_title: DOD’s Acquisition Guidance and Framework for Managing MAIS Acquisitions
paragraphs: The Department of Defense Instruction 5000.02 is the policy that  establishes guidelines for the management of all acquisition programs.  The department’s policy guidance includes six defense acquisition  models based on the type of product being acquired (e.g., software- intensive programs and hardware-intensive programs). A generic  acquisition model that shows all of the program life-cycle phases and key  decision points is shown in figure 2 and described below.

Materiel solution analysis: Refine the initial system solution (concept)  and create a strategy for acquiring the solution. A decision—referred to as  Milestone A, the risk reduction decision point—is made at the end of this  phase to authorize entry into the technology maturation and risk reduction  phase.

Technology maturation and risk reduction: Determine the preferred  technology solution and validate that it is affordable, satisfies program  requirements, and has acceptable technical risk. A decision—referred to  as Milestone B, the program initiation decision point—is made at the end  of this phase to authorize entry of the program into the engineering and  manufacturing development phase and award development contracts. An  acquisition program baseline is first established at the Milestone B  decision point. A program’s first acquisition program baseline contains the  original life-cycle cost estimate (acquisition and operations and  maintenance costs), the schedule estimate (major milestones and  decision points), and performance parameters that were approved for that  program by the Milestone Decision Authority. The first acquisition  program baseline is established after the program has refined user  requirements and identified the most appropriate technology solution that  demonstrates that it can meet users’ needs.

Engineering and manufacturing development: Develop a system and  demonstrate through testing that the system meets all program  requirements. A decision—referred to as Milestone C, the deployment  decision point—is made during this phase to authorize entry of the  system into the production and deployment phase or into limited  deployment in support of operational testing.

Production and deployment: Achieve an operational capability that  meets program requirements, as verified through independent operational  tests and evaluation, and implement the system at all applicable  locations.

Operations and support: Operationally sustain the system in the most  cost-effective manner over its life cycle.

section_title: Leading Practices for Managing an IT Acquisition Program
paragraphs: Entities such as the Project Management Institute, the Software  Engineering Institute at Carnegie Mellon University, IEEE, and GAO have  developed and identified leading practices to help guide organizations to  effectively plan and manage the acquisition of major IT systems, such as  the MAIS programs. Our work has shown that proper implementation of  such practices can significantly increase the likelihood of delivering  promised system capabilities on time and within budget. These practices  include, but are not limited to,

Risk management: A process for anticipating problems and taking  appropriate steps to mitigate risks and minimize their impact on  program commitments. It involves identifying and documenting risks,  categorizing them based on their estimated impact, prioritizing them,  developing risk mitigation strategies, and tracking progress in  executing the strategies.

Requirements management: Requirements establish what the  system is to do, how well it is to do it, and how it is to interact with  other systems. Appropriate requirements development involves  eliciting and developing customer and stakeholder requirements, and  analyzing them to ensure that they will meet users’ needs and  expectations. It also consists of validating requirements as the system  is being developed to ensure that the final systems to be deployed will  perform as intended in an operational environment.

Systems testing and integration management: A rigorous and  efficient test and evaluation program provides early knowledge of  developmental and operational issues. Correcting these issues early  can mitigate risks of cost overruns and schedule slippages, and can  ultimately contribute to delivery of effective and suitable systems in a  timely manner. System integration ensures that all the “pieces” of the  system can work together to realize the program’s goals. It involves  the collaborative planning and execution of test phases and events to  provide shared data in support of independent analysis, evaluation,  and reporting by all stakeholders.

section_title: Most Selected MAIS Programs Had Changes in Their Planned Cost or Schedule Estimates, and Half Met Their Technical Performance Targets
paragraphs: The majority of the 18 selected MAIS programs that we reviewed had  experienced changes in their planned cost estimates and in their planned  schedule estimates when comparing the first acquisition program  baseline to the most recent acquisition program baseline estimate.  These cost estimate changes ranged from a decrease of $1,466.2 million  (-39 percent) to an increase of $1,625.7 million (469 percent), and  schedule estimate changes ranged from a slippage of 13 years and 9  months to no change. Further, nine of the programs had met all of their  technical performance targets, while four of the programs had partially  met the performance targets. The remaining five programs had not yet  conducted testing activities. As a result, it is too early to report on the  status of technical performance targets. Table 2 shows the extent of  changes in planned cost and schedule estimates for each of the selected  MAIS programs we reviewed since the first program baseline estimate  and the number of technical performance targets met. See appendix II for  the detailed profiles of each program.
subsections: section_title: Sixteen of Eighteen Selected MAIS Programs Had Changes in Their Planned Cost Estimates
paragraphs: Of the 18 MAIS programs we reviewed, 16 had experienced changes in  their planned cost estimates. Specifically, five of the 16 programs had  cost decreases that ranged from $58.5 million (-1.5 percent) for the  Army’s Global Combat Support System, Increment 1 (GCSS-A Inc 1)  program to $1,466.2 million (-39 percent) for the Air Force’s Base  Information Transport Infrastructure Wired (BITI Wired) program.

Officials reported various reasons for the decreases in planned cost  estimates. For example, the Defense Health Agency’s integrated  Electronic Health Record Increment 1 (iEHR Inc 1) program reported that  the 70 percent cost decrease was due to a reduction in the program’s  scope. Officials for the Air Force’s BITI Wired program reported that the  39 percent cost decrease was due to early assumptions about risks not  materializing. For example, when the program had solicited bids for the  contract, they included risk factors that never materialized.

In addition, 11 of the 16 programs had experienced cost increases. These  increases ranged from $2.6 million (1 percent) for the Defense Logistics  Agency’s Defense Agencies Initiative Increment 2 (DAI Inc 2) program to  $1,625.7 million (469 percent) for the Navy’s Common Aviation Command  and Control System Increment 1 (CAC2S Inc 1) program. Among the 11  programs that reported an increase, the average increase was $457.2  million.

Program officials reported a variety of reasons for the increases in  planned cost estimates. For example, officials for the Army’s Tactical  Mission Command (TMC) program attributed its 159 percent increase to,  in part, higher than expected costs for research and development testing.  In addition, officials for the Air Force’s Defense Enterprise Accounting and  Management System Increment 1 (DEAMS Inc 1) program attributed a 9  percent cost increase to the program’s growth in scope and the addition  of software upgrade enhancements. Further, officials for the Navy’s  CAC2S system attributed cost increases of 469 percent to the program’s  early developmental challenges, which included program scope growth  and restructuring.

Two programs had not experienced any changes in their cost estimates:  These include the Defense Health Agency’s Department of Defense  Healthcare Management System (DHMSM) program and the Air Force’s  Integrated Strategic Planning and Analysis Network Increment 4 (ISPAN  Inc 4) program.

section_title: Fourteen of Eighteen Selected MAIS Programs Had Changes in Their Planned Schedule Estimate
paragraphs: Of the 18 selected MAIS programs we reviewed, 14 programs had  experienced changes in their planned schedule estimates. These  changes consisted of schedule slippages that ranged from 2 months for  the Air Force’s BITI Wired program to 13 years and 9 months for the  Navy’s CAC2S program.

Program officials reported various reasons for the changes in their  planned schedule estimates. For example, officials for the Navy’s CAC2S  program attributed the schedule slippages to the addition of new  requirements and program restructuring. Officials for the Air Force’s AOC- WS Inc 10.2 program reported that the 3 years of schedule slippages  were due to a combination of test problems and system maturity issues.  In addition, officials for the Air Force’s JMS Inc 2 system cited a larger  than expected number of discrepancies in testing that had caused  schedule delays totaling 2 years and 10 months. In turn, the program  delayed completion of its Milestone C and Full Deployment Decision by  more than 1 year beyond the original estimate.

Four MAIS programs experienced no change to their planned schedule  estimates—the DAI Inc 2, DHMSM, IPPS-A Inc 2, and the ISPAN Inc 4  programs.

section_title: Nine Selected MAIS Programs Had Met Technical Performance Targets, while Others Were at Various Stages of Progress
paragraphs: Among other information, DOD uses key performance parameters as a  metric to report on each program’s progress toward meeting the system  technical performance targets. This information includes a description of  the performance characteristics, the objective and threshold value for  each target and, importantly, whether the target has been met in  demonstrating performance. For example, one program identified  “network readiness and exchange of information” as a technical  performance target. The program also established an objective value for  this target as successfully processing 98 percent of the information into  the system, and established an threshold value for this target as  successfully processing 95 percent of the information into the system.

Among the 18 selected MAIS programs we reviewed that had established  technical performance targets, nine had met all of their defined targets.  As an example of technical performance targets that had been met, the  Army’s GCSS Inc 1 program reported in the 2016 Major Automated  Information System Annual Report that both of its technical performance  targets had been met. These targets related to the readiness of the  system to fully support operational activities and information exchanges in  the architecture (referred to as Net Ready) and compliance with the  operational activities that represent the system’s mission critical functions  (referred to as Mission Critical Functions).

As another example, the Defense Health Agency’s iEHR Inc 1 program  reported that all 3 performance targets had been met. These targets  related to ensuring that end user devices display the same information,  context, and display of the patient, expediting user access to the  applications, and ability of users to move from one device to another.

Further, of the 18 MAIS programs, four programs—the Air Force’s BITI- Wired and DEAMS Inc 1 programs, Navy’s CANES and Defense  Information Systems Agency’s Teleport Gen 3 programs—had partially  met their technical performance targets. For example, BITI-Wired had  partially implemented all 4 of its performance targets—related to  readiness of the network, availability of redundant features such as dual  power supplies and spares, reliability of systems, and the installation of  support equipment at all 178 sites. According to program officials, support  equipment had been installed at 167 of the 178 total sites and the  program expects all 4 performance targets to be met in September 2017,  the planned full deployment date. As another example, the Air Force’s  DEAMS Inc 1 program officials reported that 6 of 9 targets had been met.  For the 3 performance targets that had not been met, officials reported  that processing issues had prevented them from meeting these targets,  which included establishing accurate fund balances, reporting in a timely  manner, and ensuring readiness of the network. As of June 2016,  program officials reported they were working to address these issues.

Finally, five programs—IPPS-A Inc 2, AOC-WS Inc 10.2, ISPAN Inc 4,  JMS Inc 2, and DHMSM—were in the early stages of the system  development and had not begun testing. As a result, it is too early to  determine the status of performance targets because information on the  status of technical performance is not yet available.

section_title: All Five Selected MAIS Programs Had Fully or Partially Implemented Leading Practices for Managing Risk, Requirements, and Systems Testing and Integration
paragraphs: According to the Software Engineering Institute’s Capability Maturity  Model® Integration for Acquisition (CMMI- ACQ), leading practices for  managing an IT program include risk management, and requirements  management. Effective requirements management includes establishing  an agreed-upon set of requirements and ensuring traceability between  requirements and work products. Further, changes to requirements  should be managed in collaboration with stakeholders for the acquisition  of systems within their established cost, schedule, and delivery of the  expected product. Likewise, an effective risk management process  includes identifying problems before they occur, so that risk-handling  activities may be planned and invoked, as needed, across the life of the  project in order to mitigate the potential for adverse impacts. Further,  regarding systems testing and integration, other leading practices from  the Institute of Electrical and Electronics Engineers, Incorporated (IEEE)  and DOD policy guidance (DOD Instruction 5000.02, Defense  Acquisition Guidebook, and Test & Evaluation Management Guide) state  that roles and responsibilities should be established and test-related  plans, schedules, and reports should be developed to better manage test  and integration activities.

All five MAIS programs that we selected for an in-depth review had fully  or partially implemented leading practices for managing risk,  requirements, and systems testing and integration. Table 3 shows the  extent to which these programs had implemented the practices for  requirements management, risk management, and systems testing and  integration management. We assessed one program as “not applicable”  because it was initiated after a limited production decision point (post- Milestone C) and, therefore, had not implemented certain practices.
subsections: section_title: Two of the Five MAIS Programs Had Not Fully Implemented All Practices for Managing Requirements
paragraphs: Leading requirements management practices include establishing an  agreed-upon set of requirements, ensuring traceability between  requirements and work products, and managing any changes to the  requirements in collaboration with stakeholders. Such leading practices  help organizations to better manage the design, development, and  delivery of systems within established cost and schedule time frames.  These practices include developing an understanding with the requirements providers on the  meaning of the requirements, obtaining commitment to requirements from project participants, managing changes to requirements as they evolve during the project, maintaining bidirectional traceability among requirements and work, ensuring that project plans and work products remain aligned with  requirements.

Three of the five programs had fully implemented the requirements  management practices, while the other two had partially implemented  some practices.

Air Force— Air and Space Operations Center-Weapon System  Increment 10.2 (AOC-WS Inc 10.2)

The Air Force had fully implemented four, and partially implemented one,  of the five requirements management practices for the AOC WS Inc 10.2  program. For example, the program developed an understanding with  requirements providers on the meaning of the requirements and  established a requirements working group that oversees the development  and management of program requirements. In addition, the AOC-WS Inc  10.2 program obtained commitment to requirements from project  participants through the DOD configuration control board process, in  which requirements were reviewed and, upon acceptance, approved,  documented, and tracked. The program also managed changes to  requirements as they evolved during the project; however, although  changes to requirements were documented and maintained, the program  did not provide evidence of its rationale for the changes. The AOC-WS  Inc 10.2 program has since completed its acquisition phase and,  therefore, providing a rationale for changes made during the requirements  development phase would not impact the program moving forward.

Air Force— Base Information Transport Infrastructure Wired (BITI  Wired)

The Air Force had fully implemented the two applicable requirements  management practices for the BITI Wired program. Specifically, the  program had developed an understanding with requirements providers on  the meaning of the requirements and had managed changes to  requirements as they evolved during the project. The BITI Wired  requirements management plan established the criteria for the evaluation  and acceptance of the requirements.

On the other hand, three of the five requirements—obtain commitment to  requirements from project participants; maintain bidirectional traceability  among requirements and work products; and ensure that project plans  and work products remain aligned with requirements—were not  applicable for the BITI Wired Program. This was due to the Air Force  beginning the program after Milestone C.

Air Force— Joint Space Operations Center, Mission System  Increment 2 (JMS Inc 2)

The Air Force had fully implemented the five requirements management  practices for the JMS Inc 2 program. Specifically, the program had  developed an understanding with requirements providers on the meaning  of the requirements. For example, the program’s Requirements  Engineering and Management Plan established criteria for distinguishing  appropriate requirements and for evaluating and accepting requirements.  In addition, changes to requirements were managed as they evolved  during the project. For example, requirements change requests had been  sent through the Lead Command, which validated them and sent them to  the Program Management Office for approval. Lastly, the program  maintained requirements traceability from source requirements to lower- level requirements to ensure that all source requirements had been  completely addressed. Specifically, the program had established  processes for managing the program’s Requirements Traceability Matrix  database, which included activities to enter approved requirements,  sequence traceability changes, update development, test status, and  report data.

Army— Global Combat Support System-Army Increment 1 (GCSS-A  Inc 1)

The Army had partially implemented all five requirements management  practices for the GCSS-A Inc 1 program. Officials reported that, although  the program had progressed to the sustainment phase, and its  requirements activities had been completed since 2014, the Configuration  Steering Board review process had been used to manage, oversee, and  approve requirements as they evolved during the project. The program  had developed a Software Development Overview Schedule that  provided a general overview of their software development life cycle  process.

Nevertheless, an approved and signed Requirements Management Plan  at program initiation had not been implemented. Such a plan would have  been important to ensuring an understanding with the requirements  providers of the meaning of the requirements. Further, the program’s  Release Management Plan included a section on the requirements  process and references to a central information repository in the  Requirements Traceability database. However, the requirements  traceability matrix provided did not contain all relevant information such  as “priority.” These important management tools could have provided the  program with the discipline, structure, and oversight typically associated  with successful organizations that apply this practice. However, since the  GCSS-A Inc 1 program has completed its acquisition phase, fully  implementing these practices now would not impact the program moving  forward.

Army—Integrated Personnel and Pay System—Army Increment 2  (IPPS-A Inc 2)

The Army had fully implemented the five requirements management  practices for the IPPS-A Inc 2 program. Specifically, the program had  developed an understanding with requirements providers on the meaning  of the requirements and had managed changes to requirements as they  evolved during the project. For example, the program’s Requirements  Management Plan described the criteria and process through which  requirements are evaluated and accepted. In addition, the IPPS-A  Requirements/Configuration Control Board Charter requires that all  changes be vetted through senior governance boards, such as the  Council of Colonels and General Officer’s Steering Committee. Further,  the program maintained bidirectional traceability of requirements.  Specifically, a history for all requirements changes was maintained within  a repository for requirements that were managed by the Application  Lifecycle Management Tool.

section_title: Two of the Five MAIS Programs Had Not Fully Implemented All Practices for Managing Risk
paragraphs: According to leading practices, an effective risk management process  identifies potential problems before they occur, so that risk-handling  activities may be planned and invoked, as needed, across the life of the  project in order to mitigate the potential for adverse impacts. These  practices include determining risk sources and categories; defining parameters used to analyze and categorize risks and to  control the risk management effort; establishing and maintaining the strategy to be used for risk  identifying and documenting risks; evaluating and categorizing each identified risk using defined risk  categories and parameters, and determining its relative priority; developing a risk mitigation plan in accordance with the risk monitoring the status of each risk periodically and implementing the  risk mitigation plan as appropriate.

Three of the five programs had fully implemented the risk management  practices, while two had partially implemented some practices.

The Air Force had fully implemented four, and partially implemented  three, of the seven risk management practices for the AOC WS Inc 10.2  program. For example, the program had determined risk sources and  categories used to examine and oversee changes that impact the project,  as described in the program’s Risk Management Plan. The program also  had defined parameters used to analyze and categorize risks and to  control the risk management effort. Specifically, the Risk Management  Plan evaluated and assigned values to each risk by calculating the  probability and consequence of a risk, as well as its likelihood.

The AOC-WS Inc 10.2 program had partially implemented the key  management practice to identify risks that could negatively affect work  efforts. For example, although the program identified risk in specific risk  categories, it did not identify risks that were outside the normal scope of  the project.

In addition, while the program had developed criteria for evaluating and  qualifying risk likelihood and severity levels, it did not develop an overall  risk mitigation plan in accordance with the risk management strategy.  This occurred because the program underwent a critical change and,  according to program officials, this critical change altered the  management of the program due to its complexity. Until an overall risk  mitigation plan is developed, the program will not be able to effectively  manage risk mitigation and contingency planning activities.

The Air Force had fully implemented all seven of the risk management  practices for the BITI Wired program. Specifically, the program had  defined parameters used to analyze and categorize risks and to control  the risk management effort. For example, the program evaluated and  assigned values to each risk by calculating the probability and  consequence of a risk as well as its likelihood and consequence. The  program had also identified and documented risks that could negatively  affect work efforts. For example, the Risk Management Plan established  risk conditions, thresholds, and probabilities of occurrence. In addition,  the BITI Wired program monitored the status of each risk periodically and  implemented risk mitigation plans as appropriate through the use of a risk  management tool and risk board.

The Air Force had fully implemented all seven of the risk management  practices for the JMS Inc 2 program. Specifically, the program had  defined parameters used to analyze and categorize risks and to control  the risk management effort. For example, as detailed in the program’s  Risk and Opportunity Management Plan, risk ratings, including likelihood  and consequence of occurrence, were established through the risk  matrix; this provided a consistent, standardized method for risk analysis.  In addition, defined thresholds were used to prioritize risks and trigger  management action. Further, the program had developed risk  management plans establishing a strategy used for risk management.  Specifically, risks were monitored and risk status was reviewed in the  issue database as well as through meetings used to report on  programmatic metrics. Standard operating procedures had also been  developed to manage risk, including one to manage risk identification,  analysis, mitigation, and tracking.

The Army had fully implemented five, and partially implemented two, of  the risk management practices for the GCSS-A Inc 1 program.  Specifically, the program had a Risk and Opportunity Management Plan  that identified risk sources and categories. Further, in order to control the  risk management effort, risks were categorized and grouped according to  defined risk categories in the program’s Risk Management Plan. Further,  identified risks were evaluated using defined risk parameters. For  example, the Risk Register identified risk consequence and probability.

On the other hand, the program did not define thresholds or bounds for  mitigation or management action. According to leading practices, the risk  management effort can be prioritized, controlled, and managed better by  defining thresholds and categorizing risks for mitigation/management  action. Although program officials stated that they were in the process of  continuing to optimize their risk management efforts, and were updating  their standard operating procedures, this key leading practice has not yet  been implemented. Until the program has, and is guided by, updated  procedures, it will not be positioned to ensure that risks can be effectively  prioritized, controlled, and managed.

The Army had fully implemented all seven of the risk management  practices for the IPPS-A Inc 2 program. Specifically, the program’s Risk  Management Plan provided guidelines that determined risk sources and  categories. These risks were categorized in the Risk Management Tool  based on the “Risk Taxonomy” found in the Risk Management Plan.  Specifically, the risks were assigned to three main categories—technical,  programmatic, and external risks. The program also ensured potential  risks were identified and documented. For example, the Defense  Information Systems Agency, which hosts the two IPPS-A Defense  Enterprise Computing Centers (in St. Louis, MO, and Ogden, UT),  conducted assessments to ensure the sites were in compliance with  environmental controls. Further, the program developed risk mitigation  plans that identified the person or group responsible for addressing each  risk through the Risk Register.

section_title: Four of the Five MAIS Programs Had Fully Implemented All Practices for Managing Systems Testing and Integration
paragraphs: According to leading practices for managing systems testing and  integration activities, roles and responsibilities should be established and  test-related plans, schedules, and reports should be developed. Doing so  provides a baseline by which to measure progress so that appropriate  corrective actions can be taken when the program’s performance  deviates from the plans. These practices include establishing roles and responsibilities to manage testing and developing test-related plans, schedules, and reporting.

Four of the five programs had fully implemented practices for systems  testing and integration.

The Air Force had fully implemented both systems testing and integration  practices for the AOC-WS Inc 10.2 program. The program established  roles and responsibilities to manage testing/integration activities and  formulated test related plans, schedules, and reports. Specifically, the  program’s Integrated Test Team Charter and the Test and Evaluation  Master Plan (TEMP) established organizational and individual roles and  responsibilities for testing and integration. Further, the program  developed a test plan that included a test and evaluation strategy and  schedule.

The Air Force had fully implemented both systems testing and integration  practices for the BITI Wired program. In this regard, the program had  established testing roles and responsibilities. The program also  formulated all applicable test related plans, schedules, and reports. For  example, the BITI Wired Program developed testing plans and schedules  for each base installation site.

The Air Force had fully implemented one, and partially implemented one,  of the systems testing and integration practices for the JMS Inc 2  program. For example, the program had developed a Test and Evaluation  Master Plan that provided the framework and strategy for planning and  executing its testing and evaluation activities. Although the program had  established roles and responsibilities to manage testing and integration  activities, a chief developmental tester to oversee testing activities had  not been identified. Program officials stated that it had been difficult to fill  this position due to the lack of qualified personnel. Until this position is  filled, it may be difficult to manage and mitigate risks during development,  verify that products are compliant with operational requirements, and  inform decision makers throughout the program’s life cycle.

The Army had fully implemented both systems testing and integration  practices for the GCSS-A Inc 1 program. In this regard, the program had  established and documented roles and responsibilities for all testing and  evaluation activities, including identifying a chief developmental tester. In  addition, a GCSS-A Inc 1 Test and Evaluation Master Plan had been  developed, which described the total testing and evaluation planning from  component development through operational testing and evaluation. The  program also had developed test plans, such as the Capstone Test Plan  and Systems Engineering Plan, which further detailed the program’s  testing and evaluation strategy.

The Army had fully implemented both systems testing and integration  practices for the IPPS-A Inc 2 program. In this regard, the program had  established testing roles and responsibilities and formulated test plans,  schedules, and reports, where applicable. Specifically, the Increment II  Test and Evaluation Working Integrated Product Team Charter was  established to provide guidance in overseeing testing activities. In  addition, a chief developmental tester had been appointed to serve as the  focal point for all testing and evaluation efforts. Further, the IPPS-A Inc 2  program developed a Test and Evaluation Master Plan, which described  the overall testing and evaluation plans from system development through  operational testing and evaluation for all releases.

section_title: Conclusions
paragraphs: MAIS programs are important for sustaining DOD’s key operations in  communications, business, and command and control, and provide the  department with access to information to organize, plan, direct, and  monitor mission operations. Therefore, it is vital that these programs  follow leading practices to ensure success. While a number of leading  practices for risk, requirements, and systems testing and integration had  been fully or partially implemented by five programs that we reviewed in- depth—the Air Force’s AOC WS Inc 10.2, BITI Wired and JMS Inc 2  programs; and the Army’s GCSS-A Inc 1 and IPPS-A Inc 2 programs— three programs lacked practices to better manage the development of  systems. The program office for GCSS-A Inc 1 did not have standard  operating procedures for defining thresholds or bounds to manage risks,  thus putting at risk the program’s ability to effectively prioritize, control,  and manage for mitigation and management actions. In addition, the  program office for AOC-WS Inc 10.2 did not have an overall Risk  Mitigation Plan to guide the implementation of individual risk mitigation  and contingency planning activities. An overall risk plan is necessary to  help ensure that individual efforts are managed in a cohesive and  repeatable way. Further, the program office for JMS Inc 2 had not  assigned a chief developmental tester, an important role to ensure that  risks are managed and mitigated effectively throughout testing and  integration. Unless this position is filled, it will be difficult for the program  to effectively manage risks and verify compliance with certain operational  requirements throughout the acquisition.

section_title: Recommendations for Executive Action
paragraphs: To help improve the management of DOD’s MAIS programs, we are  recommending that the Secretary of Defense take the following three  actions: direct the Secretary of the Army to direct the program manager for  GCSS-A Inc 1 to establish standard operating procedures for  managing risks that include guidance for establishing thresholds and  bounds for key risk areas. direct the Secretary of the Air Force to direct the program manager for  AOC-WS Inc 10.2 to develop an overall risk mitigation plan to guide  the implementation of individual risk mitigation and contingency plan  activities. direct the Secretary of the Air Force to direct the program manager for  JMS to appoint a chief developmental tester to oversee systems  testing and integration activities.

section_title: Comments from the Department of Defense
paragraphs: DOD provided written comments on a draft of this report, which are  reproduced in appendix III. In its comments, the department agreed with  all three of our recommendations. Further, the department provided  evidence that it had fully addressed one of the recommendations and it  described plans and actions for addressing the remaining two  recommendations.

Regarding our recommendation that the GCSS-A Inc 1 program establish  standard operating procedures for managing risks, DOD provided  evidence that the Army had updated its risk management plan.  Specifically, the Army’s GCSS-A Inc 1 program manager amended the  Risk Management Plan for Project Manager Army Enterprise Systems  Integration Program on March 1, 2017. The plan now describes how risk  prioritization occurs; it requires that risk owners work with the risk  management officer to define appropriate trigger conditions in order to  ensure a mitigation plan can be initiated at the appropriate point. By  taking these actions to implement this recommendation, the program  should be in a better position to more effectively manage risks.

DOD also concurred with our recommendation to develop an overall risk  mitigation plan to guide the implementation of individual risk mitigation  and contingency planning activities on the AOC WS Inc 10.2 program.

Toward this end, the Air Force reported that it plans to restore the  program’s risk management practices to include weekly meetings to  review program risks and publish a risk management plan in April 2017.  Once implemented, these actions should help the Air Force to better  ensure that individual risk mitigation and contingency plan activities are  conducted in a consistent and repeatable way.

Further, DOD concurred with our recommendation that JMS appoint a  chief developmental tester to oversee systems testing and integration  activities. In this regard, the Air Force reported that it had identified a  qualified candidate to fill the Chief Developmental Tester position.  According to the Air Force, the program expects the new test chief to be  in place in July 2017. Once the position is filled, the program should be  better able to oversee important systems testing and integration activities.

We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; the Secretary of the Army; the  Secretary of the Air Force; and other interested parties. This report also is  available at no charge on the GAO website at http://www.gao.gov.

Should you or your staffs have any questions on information discussed in  this report, please contact me at (202) 512-4456 or harrisc@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix IV.

section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The National Defense Authorization Act for Fiscal Year 2012 mandated  that we select, assess, and report on Department of Defense’s (DOD)  major automated information system (MAIS) programs annually through  March 2018. This report is the fifth in our series of annual assessments.  Our objectives were to: (1) describe the extent to which selected MAIS  programs have changed their planned cost and schedule estimates and  met technical performance targets and (2) assess the extent to which  selected MAIS programs have used leading IT acquisition practices,  including requirements and risk management, and systems testing and  integration.

To address the first objective, we used DOD’s official list of 35 MAIS  programs as of May 1, 2016, to establish a basis for selecting programs.  We selected 18 of 35 MAIS programs based on our criteria that programs  must be unclassified and have a first acquisition program baseline that  could be used as a reference point for evaluating cost, schedule, and  technical performance characteristics.

For each selected program, we asked relevant DOD officials to complete  a data collection instrument with questions on estimated cost, schedule,  and technical performance goals, including the latest program status in  meeting those estimated goals. Using the responses from the data  collection instrument, we then compared each program’s first acquisition  program baseline cost estimate to the latest estimate to determine the  extent to which planned program costs had changed. Similarly, to  determine the extent to which these programs had changed their planned  schedule estimates, we compared each program’s first acquisition  program baseline schedule estimate to the latest schedule.

To determine whether the selected programs met their performance  targets, we reviewed the most recent program management reports and  examined each program’s self-identified system performance targets and  the status of their progress. We also reviewed additional information on  each program’s cost, schedule, and performance, including program  documentation, such as DOD’s MAIS annual and quarterly reports,  acquisition program baselines, system test reports, and our prior reports.  We then aggregated and summarized the results of these analyses  across the programs, and developed individual profiles for each program.

To address the second objective, we started with the list of 18 MAIS  programs that were selected specifically for the first objective. From this  list, we identified programs for a more in-depth review based on two  criteria: the programs selected must represent at least two military  services, and the programs must have not been evaluated by us within  the last year. Using this criteria, we identified 5 programs from two  military services: the Army’s Global Combat Supply System Increment 1  (GCSS-A Inc 1) and Integrated Personnel and Pay System Increment 2  (IPPS-A Inc 2); and the Air Force’s Air and Space Operations Center– Weapon System Increment 10.2 (AOC-WS Inc 10.2), Base Information  Transport Infrastructure Wired (BITI Wired), and Joint Space Operations  Center, Mission System Increment 2 (JMS Inc 2).

Based on prior work evaluating MAIS programs, we identified three  management practice areas—requirements management, risk  management, and systems testing and integration that are applicable to  these programs. For evaluating how the selected five programs applied  practices in the risk and requirements management areas, we identified  key practices related to these areas from the Software Engineering  Institute’s Capability Maturity Model® Integration for Acquisition (CMMI- ACQ).

Specifically, for requirements management, we evaluated each of the five  programs against the following 5 practices: develop an understanding with the requirements providers of the  meaning of the requirements, obtain commitment to requirements from project participants, manage changes to requirements as they evolve during the project, maintain bidirectional traceability among requirements and work, and ensure project plans and work products remain aligned with  requirements.

In doing so, we analyzed program requirements documentation against  the practices. This documentation included requirements management  plans, traceability matrices, requirements change forms, technical  performance assessments, and requirements board meeting minutes. We  also interviewed program officials to obtain additional information about  their requirements management practices.

For the risk management area, we evaluated each of the five programs  against the following 7 practices: determine risk sources and categories; define parameters used to analyze and categorize risks and to control  the risk management effort; establish and maintain the strategy to be used for risk management;  identify and document risks; evaluate and categorize each identified risk using defined risk  categories and parameters, and determine its relative priority; develop a risk mitigation plan in accordance with the risk management monitor the status of each risk periodically and implement the risk  mitigation plan as appropriate.

In doing so, we analyzed each program’s key documents, such as risk  management plans, risk charts, and other artifacts, to determine the  extent to which these practices had been applied. We also interviewed  program officials to obtain additional information about their risk  management practices.

For the systems testing and integration area, we evaluated each of the  five programs against the following 2 practices: establish roles and responsibilities to manage testing and integration develop test-related plans, schedules, and reporting.

In this regard, we evaluated the test manager’s roles and responsibilities,  as well as test plans, schedules, and test reports to determine the extent  that these practices had been applied. We also conducted interviews with  program officials to obtain additional information about their systems test  and integration practices.

Further, in determining the extent to which each of the five MAIS  programs had implemented the three practice areas—requirements  management, risk management, and systems testing and integration—we  rated a practice as “fully implemented” when all of the practices for the  area had been implemented. Similarly, when a program had implemented  at least one of the practices within the area, we rated the practice as  “partially implemented” to indicate the practice had been partially  implemented. For a MAIS program that did not conduct activities for any  of the practices because the program had been initiated after a limited  production decision point (post-Milestone C), we assessed the practice as  “not applicable” because such programs do not apply practices to  develop, manage requirements, or test the systems.

To assess the reliability of the data that we used to support the findings in  this report, we corroborated program office responses with relevant  program documentation and interviews with agency officials. We found no  data reliability issues and determined that the data used in this report  were sufficiently reliable for our reporting purposes. We have also made  appropriate attribution indicating the sources of the data.

We conducted this performance audit from April 2016 to March 2017 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

section_title: Appendix II: Profiles of Selected MAIS Programs
paragraphs: This section contains profiles of the 18 selected MAIS programs that we  analyzed to determine whether they had changed their planned cost and  schedule estimates and met performance measures. Each profile  presents data on the program’s purpose and status, its latest cost and  schedule estimates compared to the first acquisition program baseline  and system performance data. The first page of each two-page profile  provides (1) a description of the program’s purpose, (2) essential details,  such as the responsible DOD component, program owner, prime  contractor, the total number of active contractors supporting the program;  and (3) program costs (in then-year dollars), comparing the program’s  latest life-cycle cost estimate (separated into acquisition and operations  and maintenance costs) to its first acquisition program baseline  (subsequent acquisition program baselines that may have been  established are not identified).

The profile also provides information on the program’s status, costs,  schedule, and performance.

In the program status section, we discuss recent milestones achieved  and acquisition events.

In the cost section, we identify the extent to which the program’s life- cycle cost estimate has changed from its first acquisition program  baseline, as well as any notable causes for any changes.

In the schedule section, we discuss the extent to which the program’s  schedule has changed from its first acquisition program baseline and  causes for significant changes.

In the performance section, we identify the extent to which each program  has met its established measures. These performance ratings represent  point-in-time assessments as reported by the program. System  performance targets are rated as “met” when the program fully met all of  its key performance parameters. System performance was rated as “did  not fully meet” when a program did not pass system testing. System  performance was rated as “not yet applicable” when the system had not  yet been tested because the program was not yet at the appropriate  acquisition phase.
subsections: section_title: Air Force—Air and Space Operations Center- Weapon System Increment 10.2 (AOC-WS Inc 10.2)
paragraphs: The AOC-WS Inc 10.2 program is expected to provide system hardware,  software, technical documents, and technology refresh to ensure it  remains a viable weapons system. The program will enable personnel at  selected air and space operations centers to better plan, execute, and  assess theater-wide air and space operations. It will replace the currently  fielded AOC 10.1 system and provide additional capabilities, such as  dynamic planning and execution; data management; information  assurance; predictive battlespace awareness; and airspace management.

Program essentials (as of October 2016)

DOD component: Air Force  Program owner: Air Combat Command   Prime contractor: Northrop Grumman, Mission Systems  Total number of contractors (as of May 2016): 10  As of June 2016, the AOC-WS Inc 10.2 program was in the engineering  and manufacturing development phase of acquisition in the Defense  Acquisition Management System. In March 2016, the program declared a  critical change because of significant cost and schedule growth due to  schedule and technical difficulties in reaching Milestone C, which is the  key juncture for approval to move into limited production. According to the  program office, major factors contributing to the cost and schedule delays  were the prime contractor (1) underestimating the software development  effort; (2) insufficiently allocating manpower; and (3) underestimating the  end-to-end testing capability.

As of June 2016, the AOC-WS Inc 10.2 program’s total life-cycle cost was  $5,642.0, an increase of $56.8 million (1 percent) from the original  estimate of $5,585.2. The estimated total acquisition cost is $706.0  million, an increase of $243.3 million (53 percent) from the original  estimate of $462.7 million.

The AOC-WS Inc 10.2 program experienced significant schedule delays  since establishing the original schedule in 2007. Specifically, Milestone C  was estimated to be completed on July 2015, but, as of September 2016,  the program had not reached the milestone. According to program  officials, the program was undergoing a critical change that will result in a  3-year schedule delay.

The AOC-WS Inc 10.2 program was still early in development and system  performance data was not available at the time of our review.

section_title: Air Force—Base Information Transport Infrastructure Wired (BITI Wired)
paragraphs: The BITI Wired program is to upgrade the enterprise network and provide  the core cyber infrastructure. The upgrade includes wired and wireless  systems, and network cables and servers for more than 150 active duty,  Air Force Reserve, and Air National Guard bases. The program is  expected to update the fixed local area network and all necessary  information transport infrastructure to support current and future  communications needs of the Air Force and Joint Command warfighter.  This includes enabling base level data, voice, video, imagery, sensor, and  telemetry requirements.

Program essentials (as of October 2016)

DOD component: Air Force  Program owner: Air Force Space Command (AFSPC)  Prime contractors: General Dynamics, Harris Corp, Lockheed Martin,  Telos Corp, Three Wire Systems, and Red River for the procurement  contracts and Affigent and ThunderCat for the operations and support  contracts  Total number of contractors (as of May 2016): 8  The BITI-Wired program achieved a full deployment decision on February  18, 2014, and expects to reach full deployment in September 2017. The  program completed 167 out of 178 turnkey network upgrades and  expects to remain within the cost, schedule, and performance thresholds.

As of July 2016, the BITI Wired program’s total life-cycle cost estimate  was $2,323.7 million, a decrease of $1,466.2 million (-39 percent) from  the original estimate of $3,789.9 million. The decrease was due to more  accurate cost estimates being provided by vendors. Specifically, initial  estimates included large risk factors because, at the time, the government  could not provide vendors a definitive list of items. As a result, vendors’  earlier estimates reflected the potential for risks in their estimates. The  new list of refined equipment resulted in reduced costs. In addition, while  officials cited insufficient human resources as a potential risk to cost as  the program moves forward, they fully expect to stay within the cost  estimates.

The BITI Wired program’s full deployment decision was 2 months’ behind  schedule. However, program officials report they are on track to meet full  deployment in September 2017.

Did not fully meet all of the system performance targets  The BITI-Wired program had partially implemented all 4 of performance  targets—net ready, availability, support, and reliability. According to  program officials, installation of equipment had been installed at 167 of  the 178 sites. The program expects to report all 4 performance targets as  fully met in September 2017, the full deployment date, when the  equipment has been installed at all sites.

section_title: Navy—Consolidated Afloat Networks and Enterprise Services (CANES)
paragraphs: The CANES program is expected to consolidate and standardize the  Navy’s existing network infrastructures and services by reducing and  eliminating existing standalone afloat (i.e., surface ships and submarines)  networks, providing a technology platform that can rapidly adjust to  changing warfighting requirements, and reducing the hardware footprint  on 259 afloat and maritime operations center platforms.

Program essentials (as of October 2016)

DOD component: Navy  Program owner: C4I  Prime contractors: Northrop Grumman, Serco, BAE Systems, DRS Laurel  Technologies, General Dynamics, CGI Federal, Global Technical  Systems  Total number of contractors (as of May 2016): 82  The CANES program had moved past the full deployment milestone. The  program is expected to have all systems fully fielded by September 2024.  According to program officials, risks remain that could result in delaying  the fielding of CANES, thereby increasing risk to the program’s ability to  achieve full deployment on schedule.

As of July 2016, the CANES program’s total life-cycle cost estimate was  $11,910.6 million, a decrease of $830.3 million (-7 percent) from the  original estimate of $12,740.9 million. In February 2016, in response to a  full deployment decision, the life-cycle cost estimate was updated to  reflect reductions in procurement costs, the program’s technology refresh  schedule, and manpower costs.

The CANES program experienced a significant delay in reaching a full  deployment decision. In June 2014, the program slipped the objective  date for full deployment decision 2 years, from December 2013 to  December 2015. A full deployment decision was achieved in October  2015, and full deployment is estimated for September 2024. According to  program officials, an unstable funding environment is the largest risk to  achieving full deployment.

Did not fully meet all of the system performance targets  The CANES program had not met all 9 of the performance targets.  Specifically, of the 9 performance targets, 8 have been met, while 1  performance target was partially met—the net ready performance target.  According to the program office, this performance target was only partially  met because testing for afloat core services has been deferred.

section_title: Navy—Common Aviation Command and Control System Increment 1 (CAC2S Inc 1)
paragraphs: The CAC2S Inc 1 program is expected to provide an integrated and  coordinated modernization effort for the equipment of the Marine Air  Command and Control System. The system is expected to provide  enhanced capability for three air defense centers to support aviation  employment in joint, combined, and coalition operations. It will provide  tactical situational display, information management, sensor and data link  interface, and operational facilities for planning and execution of Marine  Aviation missions.

Program essentials (as of October 2016)

DOD component: Navy  Program owner: USMC, Program Executive Officer, Land Systems  Prime contractors: General Dynamics, C4 Systems, Inc.  Total number of contractors (as of May 2016): 5  As of March 2016, Phase 1 of the CAC2S Inc 1 program was in the  operations and sustainment acquisition phase and Phase 2 was in the  production and deployment phase. The program has been rebaselined  once following a critical change that occurred in 2009 due to the  program’s inability to meet the initial operational capability date.

As of March 2016, the CAC2S Inc 1 program’s total estimated life-cycle  estimate was $1,972.7 million, an increase of $1,625.7 million (469  percent) from the original life-cycle estimate of $347 million. Similarly, the  program’s current total acquisition estimate is $882 million, an increase of  $708.7 million (409 percent) from the original total acquisition estimate of  $173.4 million. As we reported in March 2016, factors that contributed to  the cost increase were early challenges in estimating costs due to  program scope growth and need for restructuring. Program officials noted  funding instability as a potential area of risk in meeting cost estimates.

As of October 2015, the CAC2S Inc 1 program had a 13-year, 9-month  slippage in their full deployment date due to a major change in project  scope and restructuring of the project.

Met all of the system performance targets  CAC2S Inc. 1 had met both of the technical performance targets which  were (1) net ready, and (2) data fusion.

section_title: Defense Logistics Agency—Defense Agencies Initiative Increment 2 (DAI Inc 2)
paragraphs: The DAI Inc 2 program’s mission is expected to deliver auditable,  accurate, timely, authoritative financial data to support DOD’s goal of  standardizing financial management practices, improving financial  decision support, and supporting audit readiness. Defense agencies use  more than ten different non-compliant financial management systems  supporting diverse operational functions and the warfighter in decision  making and financial reporting.

Program essentials (as of October 2016)

DOD component: Defense Logistics Agency  Program owner: OSD (C)   Prime contractor: Credence Management Solutions, LLC; TeraThink  Corp; IBM; CACI, Inc; Northrop Grumman Systems Corp; CACI ISS, Inc;  Amyx, Inc  Total number of contractors (as of May 2016): 25  As of March 2016, the DAI Inc 2 program was in the limited deployment  phase of the business capability life cycle. According to a 2016 MAIS  annual report, the program was on track to remain within the schedule,  cost, and performance thresholds identified in the original estimate.  According to program officials, the program was rebaselined to adjust  release 2 objective and threshold dates for limited fielding to  accommodate the time frame for resolving a bid protest. Notwithstanding  this additional time for the bid protest, the program achieved the original  acquisition program baseline objective date for this event.

As of March 2016, the DAI Inc 2 program’s total life-cycle estimate was  $868.8 million, an increase of $2.6 million (<1 percent) from the original  life-cycle cost estimate of $866.2 million. Similarly, the program’s current  total acquisition estimate was $184.9 million, an increase of $2.6 million  (1 percent) from the original acquisition estimate of $182.3 million.  However, these cost increases were within the planned cost objective  estimates. Officials cited funding instability as a potential risk to cost as  the program moves forward.

The DAI Inc 2 program had met the first acquisition program baseline  objective schedule estimate for limited fielding in September 2015.  Officials noted they were on schedule to stay within the threshold date of  April 2017 for a full deployment decision.

Did not fully meet all of the system performance targets  According to the 2016 MAIS annual report, the DAI Inc 2 program had not  met any of the five of the technical performance targets which are (1)  financial system performance, (2) business enterprise architecture  compliance, (3) support net centric DOD military operations, (4) managed  in network, and (5) effectively exchange information.

section_title: Department of Defense— Healthcare Management System Modernization (DHMSM)
paragraphs: The DHMSM Program is expected to provide a configurable and scalable  modernized electronic Health Record System. It will replace DOD’s  legacy healthcare systems including the Armed Force’s Health  Longitudinal Technology Application, Composite Health Care System  (inpatient), and most components of the Theater Medical Information  Program-Joint program, with a commercial Off-The-Shelf electronic health  record system. It is expected to address the current state of the Military  Health System, where multiple legacy systems and data stores,  developed over decades, are in need of modernization.

Program essentials (as of October 2016)

DOD component: Defense Health Agency  Program owner: PEO Defense Healthcare Management Systems  Prime contractor: Leidos  Total number of contractors (as of May 2016): 35  Cost, schedule, and performance summary  Not applicable: The program obtained its first baseline on May 3, 2016.  Due to the recent establishment of the program’s first baseline, a history  of cost, schedule, and technical performance has not been established.  Therefore, the cost, schedule, and technical performance is not  discussed.

section_title: Air Force—Defense Enterprise Accounting and Management System-Increment 1 (DEAMS Inc 1)
paragraphs: The DEAMS Inc 1 program is expected to provide the Air Force with an  entire spectrum of financial management capabilities, including  collections; commitments and obligations; cost accounting; general  ledger; funds control; receipts and acceptance; accounts payable and  disbursement; billing; and financial reporting. The DEAMS program is  also intended to be a key component of DOD’s solution for achieving  fully-auditable financial statements by September 30, 2017, as required  by the National Defense Authorization Act for Fiscal Year 2010.

Program essentials (as of October 2016)

DOD component: Air Force  Program owner: Air Force Financial Management  Prime contractors: Accenture and Kearney  Total number of contractors (as of May 2016): 29  As of June 2016, the DEAMS Inc 1 program was in the limited  deployment phase. In March 2016, the program was approved for limited  deployments at smaller major commands. Since then, the program has  undergone a Critical Change due to schedule slippage of the threshold  full deployment date of more than a year. However, the Critical Change  report was not completed at the time of our review. Specifically, the  DEAMS Inc 1 program was not demonstrating enough progress towards  operational effectiveness, suitability, and survivability to reach the full  deployment date within one year of the milestone objective. In addition,  the program office identified requirements management and technical  risks as areas that pose risks to successfully meeting schedule  milestones.

As of July 2016, the DEAMS Inc 1 program’s total life-cycle estimate was  $1,563.7, an increase of $124.9 million (9 percent) from the original life- cycle estimate of $1,434.3 million. In addition, the program’s cost  estimate decreased by $49.0 million (-8 percent) when compared to the  original total acquisition estimate of $641.1.

The DEAMS Inc 1 program was undergoing a critical change due to  schedule slippages. The full deployment date has been exceeded by  more than a year. However, a Critical Change report had not yet been  issued at the time of our review.

Did not fully meet all of the system performance targets  The DEAMS Inc 1 program had partially met the system performance  targets. Specifically, 6 of the 9 performance targets have been met, while  3 have not.

section_title: Global Combat Support System—Army Increment 1 (GCSS-A Inc 1)
paragraphs: The GCSS-A Inc 1 program is a commercial off-the-shelf enterprise  resources planning system that is expected to provide users with supply,  maintenance, property, integrated materiel management, and  management functionality and support. It uses non-developmental items  made up of software and server hardware and has two components:  GCSS-Army, which provides logistics and financial operations, and the  Army Enterprise System Integration Program, which is expected to  provide enterprise hub services, master data management, and business  intelligence.

Program essentials (as of October 2016)

DOD component: Army  Program owner: Assistant Secretary of the Army, Acquisition, Logistics  and Technology  Prime contractor: Northrop Grumman  Total number of contractors (as of May 2016): Northrop Grumman  As of March 2016, the GCSS-A Inc 1 program was in the production and  deployment phase of the Defense Acquisition Management System.  According to the 2016 Annual Report, the program was on track to remain  within schedule, cost, and performance thresholds. However, the GCSS- A Inc 1 program realized a decrease of $16 million in funding that could  potentially cause, in part, a six-month slip in fielding with a completion  date of March 2018.

As of May 2016, the GCSS-A Inc 1 program’s total life-cycle estimate was  $3,910.4 million, a decrease of $58.5 million (-1.5 percent) from the  original life-cycle estimate of $3,968.9 million. The current total acquisition  estimate was $1,932.7 million, an increase of $100.8 million (5.5 percent)  from the original total acquisition estimate of $1,831.9 million.

The GCSS-A Inc 1 program reached the full deployment decision in  December 2012, 10 months after the objective date of February 2012,  and 4 months after the threshold date of August 2012. This represents a  significant change from the original estimate of February 2012. The  program office cited the potential for a 6-month slip in schedule to meet  the full deployment date of September 2017. According to a MAIS  summary report, the program conducted a trade-off analysis to minimize  a 6-month slip in fielding, as a result of a $16 million decrement in 2016  funding. Other challenges in meeting schedule include requirements  management, insufficient human resources, and the possibility for Army  schedule changes. However, the program reports they are on schedule  for full deployment prior to the threshold schedule date of March 2018.

Met all of the system performance targets  The GCSS-A Inc 1 program had met both of the systems performance  targets which were (1) net ready, and (2) mission critical functions.

section_title: Marine Corps—Global Combat Support System- Marine Corps Logistics Chain Management Increment 1 (GCSS-MC LCM Inc 1)
paragraphs: The GCSS-MC LCM Inc 1 program is a portfolio of systems that is  expected to support logistics elements of command and control, joint  logistics interoperability, and secure access to and visibility of logistics  data. As a program within GCSS-MC, Increment 1 is based on the  implementation of Oracle e-Business Suite 11i as the core software  package. It is expected to provide the foundation for all future Marine  Corps logistics systems modernization.

Program essentials (as of October 2016)

DOD components: Department of the Navy, United States Marine Corps  Program owner: Deputy Commandant Installations and Logistics  Headquarters Marine Corp  Prime contractor: N/A, managed by the federal government  Total number of contractors (as of May 2016): Not applicable  As of March 2016, the GCSS-MC LCM Inc 1 program had been fully  implemented and reported as a closed acquisition program.

As of March 2016, the GCSS-MC LCM Inc 1 program’s total life cycle  cost estimate was $1,821.70 million, an increase of $1,360.3 million (295  percent) from the original total life cycle cost estimate of $461.4 million.  The GCSS-MC LCM Inc 1 program’s current total acquisition cost  estimate was $556.7 million, an increase of $362.3 million (186 percent)  from the original estimate of $194.4 million.

The GCSS-MC LCM Inc 1 program had a significant change in reaching  full deployment. The program reached the full deployment date in  December 2015. When compared to the original date of May 2010 for full  operational capability (equivalent is full deployment), this resulted in a 5  year and 7 month delay. The program is fully implemented.

section_title: Defense Health Agency— Integrated Electronic Health Record Increment 1 (iEHR Inc 1)
paragraphs: The iEHR Inc 1 program is expected to deliver two user capabilities—a  medical single sign on that streamlines the login process to allow users to  sign in once and leverage securely stored credentials to automatically  access the other available applications, and a context management  capability that will automatically present a patient’s data within all  applications. Other deliverables include enhanced infrastructure services,  a development test center/environment configuration, and clinical data  repository upgrades.

Program essentials (as of October 2016)

DOD component: Defense Health Agency  Program owner: PEO DHMS  Prime contractor: General Dynamics Information Technology   Total number of contractors (as of May 2016): Not applicable  As of March 2016, the iEHR Inc 1 program was in the sustainment phase  and, according to a 2016 MAIS Annual Report, the program is closed out  as an acquisition program.

The iEHR Inc 1 program’s March 2015 estimate for total life-cycle cost  was $307.6 million, a decrease of $718.3 million (-70 percent) from the  original total life-cycle estimate of $1,025.9 million. Likewise, the  program’s total estimated acquisition cost was $249.3 million, a decrease  of $117 million (-32 percent) from the original total acquisition cost  estimate of $366.3 million. The reduction in estimated cost was primarily  because of reductions in the program’s scope.

The iEHR Inc 1 program’s original objective schedule estimate for full  deployment decision was December 2013. When compared to the actual  full deployment decision date of November 2014 with the original date,  the program’s schedule slipped 11 months. The program achieved full  deployment in April 2015, 1 month ahead of schedule.

section_title: Army—Integrated Personnel and Pay System—Army Increment 1 (IPPS-A Inc 1)
paragraphs: The IPPS-A Inc 1 program is expected to provide a 24-hour, web-based,  integrated human resources system to soldiers, human resource  professionals, combatant commanders, personnel and pay managers,  and other authorized users. Specifically, the system is expected to  provide a single, multi-component, trusted database with a single record  for all Army soldiers, and serve as a trusted data source for personnel  and human resources data for the entire Army. Soldiers will have web- based access to their personnel data, enabling them to better manage  their careers and ensure accuracy of information. The system will allow  for interface communications and new multi-component reports that  include a Soldier Record Brief for all Army components (Active, National  Guard and Reserve).

Program essentials (as of October 2016)

DOD component: Department of the Army  Program owner: Department of the Army, Deputy Chief of Staff, G-1  Prime contractors: EDC Consulting, LLC - System Integration, Booz Allen  Hamilton-Support Services  Total number of contractors (as of May 2016): N/A: Program is in  sustainment  As of March 2016, the IPPS-A Inc 1 program had achieved full  deployment and was in the sustainment phase. The program was re- baselined after implementing strategic recommendations by an  independent review team established in June 2013 that determined the  system design was sufficient to meet Increment I requirements. The re- baselined acquisition program baseline was signed by the Milestone  Decision Authority in February 2015.

As of January 2015, the IPPS-A Inc 1 program’s total estimated life-cycle  cost was $368.9 million, an increase of $10.5 million (3 percent) from the  original total estimated life-cycle cost of $358.4 million. The total  estimated acquisition cost was $191.4 million (as of February 2015), an  increase of $34.4 million (22 percent) from the original total acquisition  cost of $157 million. According to program officials, the increase in cost  was a direct result of deferring continuity of operations requirements and  capturing the testing environment requirement under Increment 2.

The IPPS-A Inc 1 program achieved Milestone C and Full Deployment  Decision about 6 months after the original threshold dates in August 2013  and October 2013, respectively. During the acquisition, the delay  associated with the slip for a Milestone C decision triggered a significant  change to the program’s schedule. The program has achieved full  deployment and is now in the sustainment phase.

section_title: Army—Integrated Personnel and Pay System—Army Increment 2 (IPPS-A Inc 2)
paragraphs: The IPPS-A Inc 2 program is expected to provide a 24-hour, web-based,  integrated Human Resources system for Army personnel. It is to deliver  an integrated personnel and pay services for all Army components  (Active, National Guard, and Reserve), building on a trusted database  delivered by the IPPS-A Inc 1 program. The IPPS-A Inc 2 program is  planned to improve the Army’s financial management processes in  support of military personnel and payroll by linking the personnel and  payroll functions for all Army personnel, eliminating duplicate data entry,  reducing complex system maintenance, and minimizing pay  discrepancies. The IPPS-A Inc 2 program is expected to account for  status changes between Active, Reserve, and National Guard  components to ensure accurate service time, minimize impact on  individual pay, credit for service, and other benefits.

Program essentials (as of October 2016)  DOD component: Department of the Army  Program owner: Department of the Army, Deputy Chief of Staff, G-1  Prime contractor: CACI-System Integration, Booz Allen Hamilton-Support  Services  Total number of contractors (as of May 2016): 38  As of March 2016, the IPPS-A Inc 2 program is in the development and  deployment phase of the defense acquisition life cycle. A Milestone B  decision was achieved in December 2014. In fiscal year 2016, the  program completed several reviews—system requirements, system  functional, and integrated baseline. The program indicated requirements  growth as an area of potential risk.

As of March 2016, the IPPS-A Inc 2 program’s total life-cycle cost  estimate was $2,081.9 million, an increase of $330.9 million (19 percent)  from the original estimated life-cycle cost of $1,751.0 million. The current  total estimated acquisition cost estimate was $820.2 million, an increase  of $94.3 million (13 percent) from the original estimated acquisition cost of  $725.8 million.

The IPPS-A Inc 2 program achieved Milestone B on 19 December 2014,  within the planned threshold schedule.

The IPPS-A Inc 2 program was still early in development and system  performance data was not available.

section_title: Air Force—Integrated Strategic Planning and Analysis Network Increment 4 (ISPAN Inc 4)
paragraphs: The ISPAN Inc 4 program is expected to be a system-of-systems that  spans multiple security enclaves for strategic and operational level  planning and leadership decision making. The system is composed of two  primary components: (1) a global adaptive planning collaborative  information environment that manages strategy-to-execution planning  across all United States Strategic Command (USSTRATCOM) mission  areas and (2) a mission planning and analysis system that supports  developing nuclear and conventional force application plans. Both  components establish a framework to support the USSTRATCOM’s  effects-based planning and analysis activities.

Program essentials (as of October 2016)

DOD component: Air Force  Program owner: USSTRATCOM  Prime contractors: BAE, Leidos, Lockheed Martin, and Northrop  Grumman  Total number of contractors (as of May 2016): 19  As of June 2016, the ISPAN Inc 4 program was in the engineering and  manufacturing development phase of the Defense Acquisition  Management System. According to officials, the program was on track to  remain within the schedule, cost, and performance thresholds identified in  the original estimate, with no major risks. The program has not been  rebaselined.

The ISPAN Inc 4 program’s current cost estimates have not changed  since their first estimate.

The ISPAN Inc 4 program’s current schedule estimates have not changed  since their first estimate  The ISPAN Inc 4 program was still early in development and system  performance data was not yet available.

section_title: Air Force—Joint Space Operations Center, Mission System Increment 2 (JMS Inc 2)
paragraphs: The JMS Inc 2 Program is expected to replace the legacy Space Defense  Operations Center hardware and services with extensible and sustainable  infrastructure. The effort is planned to integrate components of the Space  Situational Awareness mission applications and Command and Control  capabilities into the Joint Space Operations Center to create timely,  actionable knowledge necessary for maintaining space superiority and  exercising command and control capabilities of space forces.

Program essentials (as of October 2016)

DOD component: Air Force  Program owner: Air Force Space Command  Prime contractor: N/A, managed by federal government   Total number of contractors (as of May 2016): 7  As of March 2016, the JMS Inc 2 program was in the engineering and  manufacturing development acquisition phase. Due to issues associated  with schedule delays of more than 12 months and acquisition  development costs increasing more than 25 percent, the program  underwent a critical change. A critical change report was delivered to  Congress in September 2016 due to the delayed completion of the  program’s Milestone C and full deployment decision by more than 12  months. According to the report, three factors that caused a critical  change were (1) the program’s original schedule was aggressive and  contained a significant level of concurrency, (2) funding and manpower  issues, and (3) contracting issues.

As of March 2016, the JMS Inc 2 program’s total life-cycle cost estimate  was $965.9 million, a decrease of $134.6 million (-12 percent) from the  original total estimate of $1,100.5 million. The latest estimate for total  estimated acquisition cost was $462.8 million, an increase of $150.1  million (48 percent) from the original acquisition cost estimate of $312.7  million. The decrease for the total life-cycle cost estimate was because  the original estimate for operations and sustainment costs was 20 years  after fielding while the new estimate is for 10 years after fielding.

Due to significant delays in the JMS Inc 2 program’s aggressive schedule,  the program underwent a rebaseline effort and issued a critical change  report in September 2016. The new estimate for a full deployment  decision is May 2019. With an original schedule date of July 2016, this  represents a 2-year, 10-month slippage.

The JMS Inc 2 program was still early in development and system  performance data was not yet available.

section_title: Army—Logistics Modernization Program Increment 2 (LMP Inc 2)
paragraphs: The LMP Inc 2 program is expected to deliver an enterprise solution that  builds, sustains, and generates warfighting capabilities using a fully- integrated supply chain and maintenance, repair, and overhaul system.  The LMP Inc 2 program’s support is critical to the Army’s ability to  achieve an integrated system that enables materiel readiness and asset  management and accountability, architecture, and acquisition compliancy.  When deployed, it is expected to deliver capabilities that address critical  Army Materiel Command requirements for automation of the industrial  base shop floor operations, standardization of ammunition automatic  identification technology, non-Army managed items management, Army  prepositioned stock planning, national maintenance program, and the  expansion and refinement of the integration between Army enterprise  resource planning systems.

Program essentials (as of October 2016)

DOD component: Army  Program owner: PEO EIS  Prime contractor: CSRA, LLC  Total number of contractors (as of May 2016): 188  As of June 2016, the LMP Inc 2 program was in the operation and  support phase of the acquisition life cycle. The program went to full  deployment in September 2016. The program will continue to provide  national level logistics and financial operations to Army arsenals and  depots.

As of July 2016, the LMP Inc 2 program’s total estimated life-cycle cost  estimate was $736 million, an increase of $6.9 million (1 percent) from the  original total life-cycle cost estimate of $729.9 million. Likewise, the total  current estimated acquisition cost estimate was $419.2 million, an  increase of $14.1 million (3 percent) from the original estimate of $405.1  million. However, these increases were within the threshold estimate  established for the program.

The LMP Inc 2 program had a schedule slip of 5 months when comparing  the original objective full deployment decision date of October 2015 to the  current estimate of July 2016. However, the schedule slip was within the  established threshold date of April 2016.

Met all of the system performance targets  The LMP Inc 2 program had met all seven of the system performance  targets which were (1) support net-centric military operations, (2) enter  and be managed in the network, (3) net readiness exchange information,  (4) survivability system information assurance, (5) survivability information  assurance system disaster recovery, (6) sustainment operational  availability, and (7) training proficiency level.

section_title: Defense Information Systems Agency— Teleport Generation 3 (Teleport Gen 3)
paragraphs: The Teleport Gen 3 program is expected to be a satellite communications  gateway that will enable the transportation of voice, video, and data  information to deployed forces, and links the deployed warfighter to the  sustaining base. The system is expected to provide expanded system  capacity, throughput, and functional capabilities to greatly enhance  support to tactical and deployed warfighters worldwide.

Program essentials (as of October 2016)

DOD component: Defense Information Systems Agency  Program owner: DISA Component Acquisition Executive  Prime contractors: Booz Allen Hamilton, Femme Comp Inc., and Itility  Total number of contractors (as of May 2016): 5  As of July 2016, the Teleport Gen 3 program was in the production and  deployment acquisition phase.

As of July 2016, the Teleport Gen 3 program’s current total life-cycle cost  estimate was $575.3 million, an increase of $11.6 million (2 percent) from  the original estimate of $563.7 million.

The Teleport Gen 3 program had a schedule slip of 3 years and 2  months, from January 2013 to March 2016 when comparing the original  baseline Milestone C (Gen 3, Phase 3) objective schedule estimate.

Did not fully meet all of the system performance targets  The Teleport Gen 3 program’s Phase 1 performance target had been  met. However, performance targets for Phases 2 and 3 had not been met.  According to the program office, all developmental and operational testing  had been completed for Teleport Gen 3 Phases 1 and 2. Additional  testing is planned for Phase 3.

section_title: Army—Tactical Mission Command (TMC)
paragraphs: The TMC program is a suite of systems that is expected to provide  mission command computing capabilities to Army commanders and their  staffs, consisting of a user-customizable common operating picture  enabled with real-time collaboration. These capabilities, together with  voice-over-internet-protocol, will provide real-time situational awareness  and decision support across the formation (battalion thru Army Service  Component Command). A component of the mission command capability  is the ability to layer information from multiple functions on the same map  display and share among command posts, allowing all parties to have a  common view.

Program essentials (as of October 2016)

DOD component: Army  Program owner: Mission Command  Prime contractor: General Dynamics (previous); currently a government  led effort by Army Research and Development Engineering Center   Total number of contractors (as of May 2016): N/A. Army is the lead  As of March 2016, the TMC program is currently post Milestone C. The  program has been rebaselined two times. A critical change report was  issued in response to a cost breach caused by a growth in the program’s  scope growth that was considered a “fact of life” change. As of March  2016, program officials reported they are on track to remain within the  schedule, cost, and performance thresholds identified in the original  estimate.

As of March 2016, the TMC program’s total life cycle cost estimate was  $2,315.50 million, an increase of $1,421.7 million (159 percent) from the  original total life cycle cost estimate of $893.80 million. The TMC  program’s latest total acquisition cost estimate was $2,192.1 million, an  increase of $1,371.7 million (167 percent) from the original acquisition  estimate of $820.40 million in March 2016. We reported in March 2016  that the Army’s TMC program estimated program development costs had  increased by 45 percent over the original estimate due to program scope  changes derived from the realignment of Command Post of the Future as  a foundation for Mission Command Collapse, the integration of  Personalized Assistant that Learns, and the incorporation of future force  requirements. According to a 2016 MAIS Annual Report, the program was  on track to remain within the schedule, cost, and performance thresholds  identified in the original estimate.

As we reported in our March 2016 report, the TMC program experienced  a slight slippage in schedule (3 months), however, it was within the  established threshold date. Program officials report they are still on track  to reach full deployment by December 2018.

Met all of the system performance targets  As of March 2016, the TMC program had met all three of the performance  targets which were (1) net ready, (2) display the common operational  picture, and (3) disseminate (create and exchange) orders.

section_title: DOD—Theater Medical Information Program— Joint Increment 2 (TMIP-J Inc 2)
paragraphs: The TMIP-J Inc 2 program is expected to integrate components of the  Military Health System sustaining base systems and the Services´  medical information systems to ensure timely interoperable medical  support for mobilization, deployment, and sustainment of all theater and  deployed forces. The system is expected to enhance clinical care and  information capture at all levels of care in theater, transmits critical  information to the theater commander; the evacuation chain for combat  and non-combat casualties; and forges the theater links of the health  record to the sustaining base and the Department of Veterans Affairs.  The TMIP-J program is the medical component of the Global Combat  Support System.

Program essentials (as of October 2016)

DOD components: DOD, Joint Participants: U.S. Army; U.S. Navy and  Marine Corps; U.S. Air Force   Program owner: PEO Defense Healthcare Management Systems   Prime contractor: KRATOS Inc.   Total number of contractors (as of April 2016): 6  Program status  As of July 2016, the TMIP-J Inc 2 program was reported as fully deployed  and was in the sustainment phase. The system consists of two  increments—Increment 1 was fielded in 2003, and development of  Increment 2 was being done in three releases. Release 1 was fielded in  2009 and Release 2 was fielded in 2013. Release 3, is the last major  release. In February 2016, the Under Secretary of Defense for  Acquisition, Technology, and Logistics declared that the TMIP-J Inc 2  program had met its full deployment decision action items. He delegated  Milestone Decision Authority to the Program Executive Officer, Defense  Healthcare Management Systems, and authorized the Services to declare  full deployment in accordance with their Service acquisition strategies.

Exceeded planned cost estimate  As of July 2016, the TMIP-J Inc 2 program’s total life cycle cost estimate  was $1,056.4 million, an increase of $380.3 million (56 percent) from the  original estimate of $676.1 million. The total acquisition cost estimate was  $608.2 million, an increase of $374.9 million (161 percent) from the  original estimate of $233.3 million.

Exceeded planned schedule estimate  The TMIP-J Inc 2 program had a 5-year, 6-month change to the full  deployment decision date when a breach occurred. Program officials  stated the schedule change was due to a mandate to increase the scope  of the program. In turn, a revised acquisition program baseline was  approved in January 2012. The objective date for full deployment  changed from June 2008 to December 2013, and a threshold date  changed from December 2008 to June 2014. The TMIP-J Inc 2 program  reached full deployment in May 2016 and has transitioned into the  sustainment phase.

Met all of the system performance targets  The TMIP-J Inc 2 program had met all ten of the performance targets  which were (1) net ready, (2) data transfer data availability, (3) data  availability, currency, and responsiveness—medical logistics visibility, (4)  data availability, currency and responsiveness—patient visibility, (5) data  availability, currency and responsiveness—medical infrastructure  readiness, (6) data availability, currency and responsiveness—individual  medical readiness, (7) data availability, currency and responsiveness— illness and injury rates, (8) data availability, currency and  responsiveness—reportable medical events, (9) standards compliance,  and (10) system operational availability and responsiveness.

section_title: Appendix III: Comments from the Department of Defense

section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact name above, the following staff also made key  contributions to this report: Eric Winter (Assistant Director), John Ortiz  (Analyst in Charge), Neha Bhatt, Nancy Glover, Kate Nielsen, and Teresa  Smith.


Question-summary annotations: questions: question: What did GAO notice about the MAIS programs it reviewed?
answer: Most of the 18 selected Department of Defense (DOD) major automated information system (MAIS) programs that GAO reviewed had experienced changes in their planned cost and schedule estimates and half of the programs had met their technical performance targets.
child_questions: question: What cost changes did the MAIS programs experience?
answer: Specifically, 16 programs experienced changes in their cost estimates ranging from a 39 percent decrease ($1.47 billion) to a 469 percent increase ($1.63 billion). The average cost increase was $457.2 million among the 11 programs reporting an increase.

question: What schedule and target changes did the MAIS programs experience?
answer: Fourteen programs experienced schedule delays, which ranged from 2 months to over 13 years. Finally, half of the MAIS programs fully met all of their technical performance targets.
child_questions: question: Why did 4 programs partially meet their targets?
answer: Of the remaining nine programs, 4 four had partially met their target because each was still conducting tests. The other five programs were in the early stages of system development and had not begun testing.

questions: question: How can a risk management process be effective?
answer: An effective risk management process identifies potential problems before they occur.
child_questions: question: How did one program not fully implement risk management processes?
answer: For example, one Army program did not have standard operating procedures for defining thresholds or bounds to manage risk. Unless such procedures are defined, the program will not have the tools needed to define risk management activities, including whether and how certain risks are prioritized.

question: What practices should programs have?
answer: Further, programs should include practices to identify potential problems so that risk-handling activities may be planned and invoked across the project to mitigate the potential for adverse impacts.
child_questions: question: What is the result of a program not having these practices?
answer: However, one Air Force program did not develop an overall risk mitigation plan to guide the implementation of individual risk mitigation activities. Without an overall risk plan to guide individual development efforts, those efforts cannot be managed cohesively.

questions: question: What should the programs do?
answer: Programs should, among other activities, establish roles and responsibilities to manage testing and integration activities, including a chief developmental tester to oversee testing activities.
child_questions: question: What issue was there with this requirement?
answer: However, one Air Force program reported difficulty in hiring a qualified individual to perform these duties.
child_questions: question: How might the program with this issue not be as effective?
answer: Until this position is filled, the program may not effectively manage risks and verify compliance with system acquisition and operational requirements.

questions: question: What purpose do MAIS programs serve?
answer: DOD's MAIS programs include systems that are intended to help the department sustain its key operations.

question: Why did GAO report on MAIS programs?
answer: The National Defense Authorization Act for Fiscal Year 2012 includes a provision for GAO to select, assess, and report on the department's MAIS programs annually through March 2018.
child_questions: question: What does GAO's report describe about MAIS?
answer: This is GAO's fifth report and (1) describes the extent to which selected MAIS programs have changed their planned cost and schedule estimates and met technical performance targets and (2) assesses the extent to which selected MAIS programs have used leading IT acquisition practices, including risk management.

questions: question: What did GAO review?
answer: GAO selected and reviewed cost, schedule, and performance data for 18 of DOD's MAIS programs that were non-classified and had an acquisition performance baseline.
child_questions: question: What in-depth reviews did GAO perform?
answer: In addition, GAO performed an in-depth review of 5 of the programs, comparing selected IT management practices used by them to leading practices for requirements and risk management and systems testing and integration.
child_questions: question: Why were in-depth reviews performed on the selected programs?
answer: The five selected programs were from at least two military services and had not been assessed by GAO in the past year.

question: How did GAO interact with the programs for the reviews?
answer: GAO also interviewed relevant program officials.