Patriot Modernization: Oversight Mechanism Needed to Track Progress and Provide Accountability
Published: 2016-08-25T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Patriot is a mobile Army surface-to-air missile system deployed worldwide to defend critical assets and forces. To respond to emerging threats and address a diverse set of capability needs, the Army has spent nearly $1.1 billion and requested $1.8 billion over the next 5 years to upgrade Patriot, begin developing a long-term radar solution, and integrate Patriot components into a central network and command and control system—the Integrated Air and Missile Defense.
A House report included a provision for GAO to assess, among other things, the status of the Patriot system and the Army's strategy for completing the upgrades. Among other things, this report examines (1) the extent to which the latest upgrades will address Patriot capability needs and (2) the level of oversight and accountability provided for the upgrade efforts. To conduct this review, GAO examined Army and program documents including test plans and schedules. GAO also interviewed Department of Defense (DOD) and other relevant officials.
section_title: What GAO Found
paragraphs: While the currently fielded version of the Army's Patriot surface-to-air missile system is an improvement over prior versions, the Army currently plans to spend about $2.9 billion between fiscal years 2013 and 2021 on an upgrade strategy to address a variety of capability needs. These efforts are intended to improve the system's performance, reliability, and communications as well as address obsolescence and sustainment issues. The figure below shows planned costs for ongoing efforts, near-term upgrades which begin fielding prior to fiscal year 2017, mid-term upgrades which begin fielding between fiscal years 2017 and 2021, and long-term upgrades—including a long-term radar solution. Key among the mid-term efforts are major software upgrades called Post Deployment Build-8 (PDB-8) and PDB-8.1, which are intended to improve communications and system capabilities against threats. The Army plans to begin operational testing for PDB-8 and PDB-8.1 in fiscal years 2016 and 2019, respectively. These testing results will reveal the extent to which the near and mid-term upgrades work as intended.
Although the Army estimated in 2013 that costs for Patriot upgrades would meet the threshold to be considered a major defense acquisition program (MDAP), the Army chose to incorporate the Patriot upgrade efforts into the existing Patriot program which made certain oversight mechanisms inapplicable. Further, it decided not to put a mechanism in place to track or report the upgrades' progress against initial cost, schedule, or performance estimates, similar to those generally required of MDAPs, which GAO considers essential for program oversight. Operational testing for PDB-8 and PDB-8.1 provides the Army with an opportunity to increase oversight. If performance shortfalls indicate a need for further development, the Army will have an opportunity to track progress on these upgrades to provide the oversight tools decisionmakers need to make important investment decisions.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Secretary of Defense direct the Army to establish oversight mechanisms, similar to those for major defense acquisition programs, if additional development is required for upgrades operationally tested with PDB-8 and PDB-8.1. DOD partially concurred, focusing its response on plans to track other MDAPs, but did not clarify how or if it would track current PDB-8 and PDB-8.1 progress. GAO maintains DOD should provide oversight for any additional PDB-8 and PDB-8.1 development.

section_title: Letter
paragraphs: The Patriot system is a cornerstone of the Army’s air and missile defense  architecture deployed worldwide in defense of the United States and its  allies for the protection of critical assets and forces. The Army has spent  approximately $1.1 billion since 2013 and requested another $1.8 billion  over the next 5 years for its latest iteration of Patriot system upgrades as  well as a long-term radar solution. Prompted by an evolving threat, these  efforts are designed to improve system performance and reliability,  upgrade the system’s communications, and address obsolescence and  sustainment issues. In addition, some of the upgrades will also enable the  Patriot radars and launchers to become a part of the Army’s future  Integrated Air and Missile Defense (IAMD) system–of-systems. The Army  IAMD system-of-systems integrates Patriot and other air and missile  defense systems’ weapons and sensors to a fire control quality network  and a central command and control system to enable any sensor to be  paired with the correct launcher.
Since 2012, members of the Senate Committee on Appropriations have  noted concerns with Patriot upgrade plans, citing its large budget  requests and issues with understanding requirements, specific  technologies required, development and fielding schedules, and costs of  the overall effort. A report accompanying a bill for the National Defense  Authorization Act for Fiscal Year 2016 included a provision that GAO  review the Patriot System. As part of our review, we provided an oral  briefing to the congressional defense committees in February and March  2016.This report assesses: (1) the current status of the Patriot system’s  performance and the extent to which it addresses warfighter needs; (2)  the cost, schedule, and testing plans to upgrade the Patriot system and  the extent to which planned upgrades will address Patriot capability  needs; (3) the level of oversight and accountability provided for the  upgrades; and (4) the extent to which the Army is planning to synchronize  Patriot modernization fielding and training schedules under high  operational demands. In addition, we assessed the extent to which the  Department of Defense’s (DOD) guidance for conducting its analysis of  alternatives (AOA) to evaluate materiel modernization solutions for the  current Patriot radar and launchers for use with IAMD meets GAO best  practices. This AOA is called the Lower Tier Air and Missile Defense  (LTAMD) Capability AOA. We discuss the AOA in appendix II.
To assess the Patriot system’s current performance status, we reviewed  operational test reports and discussed the results with officials in the  Office of the Director of Operational Test and Evaluation and Army Test  and Evaluation Command. We also reviewed current Combatant  Command warfighter operational needs statements for Patriot system  upgrades and discussed their status and mitigation plans with Army  officials. To determine Patriot’s upgrade plans, we analyzed detailed cost  data derived from program budgets, program schedules for testing and  fielding, and test and evaluation master plans, and discussed these plans  with Army and Patriot program officials. To determine the level of  oversight and accountability provided for the upgrades, we received  information from Army officials regarding how and why the upgrades were  executed under the existing Patriot program. We also reviewed prior  legislation and related reports since 2012 to understand Congress’s  concerns on oversight and accountability. To assess the Army’s fielding  and training schedules, we analyzed the Army’s fielding plan as well as  operational and training schedules. We also interviewed knowledgeable  officials on the Army’s process for choosing the fielding plan as well as its  benefits and challenges. Lastly, to assess DOD’s LTAMD AOA process,  we obtained DOD’s LTAMD AOA guidance documents and compared the  processes outlined in them to GAO best practices. We met with officials in  the Office of the Secretary of Defense for Cost Assessment and Program  Evaluation to discuss our findings and obtain additional information. For  more information on our scope and methodology, see appendix I.
We conducted this performance audit from June 2015 to August 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Patriot is a mobile Army surface-to-air missile system designed to counter  tactical ballistic missiles; cruise missiles; and other threats such as  airplanes, helicopters, and unmanned aerial vehicles. Patriot was first  deployed in the early 1980s and since that time has received a number of  substantial updates to keep pace with the growing threat. Patriot is  deployed worldwide in defense of the United States and its allies’ key  national interests, ground forces, and critical assets.
A Patriot fire unit is made up of four basic components: (1) a ground- based radar to detect and track targets; (2) launchers; (3) interceptor  missiles; and (4) a command, control, and communication station. Patriot  fire units are organized to fight in groups known as battalions. Each  battalion is controlled by its own command and control station and can  manage up to six fire units, although a battalion is typically deployed with  four. For a notional configuration of a Patriot battalion, see figure 1.  Several battalions can be commanded by an Army brigade. Brigades are  also responsible for certifying that the equipment can be employed as  required and for training the battalions. The brigade manages battalion  personnel under its command, with the ability to transfer personnel  among battalions to fill personnel gaps as needed.
The air and missile defense architecture consists of several systems  deployed together to provide a layered defense against various threats in  a range of battlespaces. Other air and missile defense systems can  contain, like Patriot, a sensor, a launcher, and a system-centric command  and control station. These systems’ command and control stations can  share information with other air and missile defense systems or with other  joint systems through external communication links, as seen in figure 1.  The air and missile defense architecture includes systems designed to  counter threats at a low altitude—such as rockets, artillery, and mortar— as well as systems designed to defeat high-altitude threats intercepted  above the earth’s atmosphere. Patriot serves as the Army’s primary  element deployed to intercept targets in this middle range of  battlespace—above the range of rockets, artillery, and mortar, but within  the earth’s atmosphere.
subsections: section_title: To Address Evolving Threats and Related Capability Gaps, the Army Adopted a New Integrated Air and Missile Defense Architecture
paragraphs: The Army has identified a number of air and missile defense  communication and performance capability gaps in its ability to address  evolving global threats. Over the last decade, adversaries have acquired  more robust, diverse, and complex threats. According to a 2010 Ballistic  Missile Defense Review Report, ballistic missiles are more technically  sophisticated, more proliferated, include more advanced  countermeasures, and continue to challenge U.S. ballistic missile defense  system capabilities. Cruise missiles have also become relatively simple  to develop, are cheaper than ballistic missiles or aircraft, and are easy to  export. Additionally, advanced electronic attacks, such as jamming or  spoofing, have become more widespread and easier to effectively  produce. Sophisticated enemies also have the ability to use a  combination of integrated attacks including electronic and cyber warfare,  a variety of inbound ballistic and cruise missiles, special operation forces,  and other methods to complicate the battlespace. The Army has identified  some high-priority air and missile defense gaps in its ability to respond to  the growing threats, as seen in table 1.
The Army announced an Air and Missile Defense Strategy in 2012 to  address communication and performance capability gaps by integrating  its current air and missile defense system components (e.g. sensors and  launchers), including Patriot, under a central network and command and  control system and linking them with joint and potential coalition allies.  The Integrated Air and Missile Defense (IAMD) program is currently  developing the IAMD Battle Command System (IBCS) that plans to  connect Patriot radars and launchers into IBCS’s central network and  command and control stations. By connecting these components directly  with IBCS, the Army intends to divest air and missile defense systems of  their system-specific command and control stations and allow them to  become network enabled sensors and launchers. See figure 2 below for  a notional representation of the future Integrated Air and Missile Defense  architecture.
The Army intends for the integrated air and missile defense architecture  to address communication and performance capability gaps by allowing  IBCS to collect information from a variety of sensors, fuse that data into a  single battlespace picture, and use that information to engage targets.  Receiving sensor data from a range of sensors could enable longer- distance engagements and provide commanders with more decision time  to select the appropriate response, prevent fratricide, and allow any joint  sensor to pair with the best available launcher. In addition, by integrating  several individual sensors’ data, IBCS could compare and resolve  conflicts within the individual systems’ abilities to accurately classify,  identify, and discriminate potential threat objects to provide more accurate  data back to the systems. IBCS could also help mitigate the risk of  electronic attack since additional sensor data could help confirm where  targets are when individual radars are being jammed or spoofed. In  addition, because launchers would have access to additional sensor data,  they could see more of the battlespace and use that information to more  effectively engage threats. IBCS is intended to multiply the performance  capabilities of the individual sensors and launchers connected to its  network. Therefore, the capability of the networked architecture relies  upon the ability of Patriot, as well as other air and missile defense  systems, to connect with IBCS and provide the needed quality data for  enhanced performance capabilities. Similar endeavors to create a  system-of-systems architecture with an extensive communication and  information network have proved challenging for DOD in the past. For  example, prior work on the Army’s Future Combat Systems, a multibillion  dollar development program originally consisting of 18 manned and  unmanned systems tied together by an extensive communications and  information network, faced rising costs and technical challenges that  eventually led to its cancellation.
section_title: The Army Is Conducting an Analysis of Alternatives for a Patriot Radar and Launcher
paragraphs: In 2014, DOD provided guidance to the Army for conducting its LTAMD  analysis of alternatives (AOA) to explore options for an efficient and cost- effective long-term radar and launcher solution—with considered  alternatives ranging from the current Patriot assets with modifications up  to total replacements—that will be able to connect with IBCS and address  capability needs related to radar reliability, range, and 360-degree  surveillance. The AOA results will support a decision for a new radar  acquisition program, known as the LTAMD sensor, that will require a  significant long-term financial investment. Issues with the Patriot radar  have been raised in the past. For example, the Director of Operational  Test and Evaluation identified performance and reliability issues with the  current Patriot radar in its annual report since 2013. In addition, the  Army conducted a business case analysis in 2013 and found that  upgrades to the Patriot radar could result in operations and support  savings, performance improvements, and reliability enhancements. An  AOA is a key first step in the acquisition process, intended to assess  alternative solutions for addressing a validated need. AOAs are generally  performed or updated to support key acquisition decision points. During  the course of our audit, an official in the Office of the Secretary of  Defense for Cost Assessment and Program Evaluation (CAPE) stated  that he expected the final LTAMD AOA report to receive approval in the  third quarter of fiscal year 2016. As of August 2016, the report was still  under independent review with the CAPE.
section_title: The Patriot Program Has Identified a Need for Training and Obsolescence and Sustainment Upgrades
paragraphs: To prepare the warfighter for the transition from the current, or legacy,  Patriot system to IBCS-integrated Patriot radars and launchers, the  Patriot program identified a need for training upgrades. Upgraded training  aids and devices are necessary because transitioning to IBCS changes  the way the warfighter employs the Patriot equipment.
The Patriot program has also identified a need to continue substantial  investments to address obsolescence and sustainment issues. For  example, the process of upgrading all of the legacy Patriot battalions to  IBCS-integrated radars and launchers is an 8-year process that officials  expect to begin in fiscal year 2017 and complete in fiscal year 2025. The  legacy Patriot system components need ongoing obsolescence and  sustainment improvements to improve reliability and availability, remain  affordable, and be compatible with the different versions of operational  Patriot battalions during that time. In addition, the program intends to  continue obsolescence and sustainment investments to maintain  readiness, improve reliability, and lower sustainment costs to support  deployed forces with legacy radars until the legacy radar is fully replaced.  Officials estimate that a new radar development could begin fielding in the  fiscal year 2028 time frame with tactical fielding completing within 7 years.  However, these plans are still preliminary and the milestone approval  process is still underway. Lastly, obsolescence and sustainment  improvements support legacy versions of Patriot systems, which foreign  military partners continue to buy and operate. Patriots have been sold  world-wide to 12 foreign military partners who share costs for sustainment  and capability improvements in addition to investing in development to  mitigate system obsolescence.
section_title: Current Version of Patriot Has Capability Improvements, Performance Shortfalls, and Does Not Yet Meet All Warfighter Needs
paragraphs: The currently fielded version of Patriot represents an improvement over  prior versions through upgraded software, a more capable missile, and  increased processor capabilities. However, the current version  demonstrated a number of performance shortfalls against its documented  requirements. In addition, warfighters from various combatant commands  have expressed critical needs for additional performance capabilities and  training equipment for the Patriot system that are currently unmet.
subsections: section_title: Current Patriot System Includes Capability Improvements over Prior Version but Operational Testing Revealed Some Performance Shortfalls
paragraphs: The current version of the Patriot system added performance capabilities  through a software and processor upgrade in 2013 and an upgraded  missile and launcher that began fielding in fiscal year 2016. In 2013, the  Patriot program released its current system software upgrade known as  Post Deployment Build-7 (PDB-7) that provided improvements in threat  tracking, debris mitigation, and user interface. The software is supported  by a new modern processor in the command and control station. This  new processor provides Patriot with the ability to process more complex  algorithms that improve the system’s capabilities against advanced  threats. It also provides a platform for future capability improvements.  Lastly, a launcher upgrade allows the system to launch and support use  of the new Patriot Advanced Capability-3 (PAC-3) Missile Segment  Enhancement (MSE) missile. The PAC-3 MSE, budgeted for and  managed under a separate acquisition program, was fielded in the first  quarter of fiscal year 2016 and is an upgrade to the predecessor PAC-3  missile by providing better lethality and a longer range—flying  approximately 50 percent higher in altitude and 100 percent farther  downrange.
While the system has made improvements, operational testing revealed  that the system requires significant upgrades to the radar and software to  bring the system up to the level of capabilities required. Operational  testing is a field test of a system or item under realistic operational  conditions with users who represent those expected to operate and  maintain the system when it is fielded or deployed. The Army conducted a  type of operational test called a limited user test in 2012 to evaluate the  Patriot system with PDB-7 software, the modern command and control  processor, and the PAC-3 MSE with the launcher upgrade against  requirements defined in the program’s capability development and  production documents. The Director of Operational Test and  Evaluation’s (DOT&E) report on the results of the limited user test is  classified, but it generally found that Patriot’s performance improved  against some threats compared to prior versions but had degradations in  system effectiveness against other threats. An unclassified summary of  Patriot performance shortfalls, as identified by DOT&E and the Army, is  shown in table 2. Some of the performance shortfalls can be attributed to  the radar’s limited sensing abilities. While the PAC-3 MSE missile has an  expanded battlespace over the PAC-3 missile, the radar is not able to  sense and support the full range and capabilities of PAC-3 MSE. In  addition, since experiencing fratricides during Operation Iraqi Freedom in  2003, the program has been working on upgrades to the system’s ability  to more accurately classify, identify, and discriminate threat objects.  While significant enhancements have been made since that time, the  program requires additional capabilities to meet requirements. The risk of  these performance shortfalls, left unaddressed, range from erroneous  engagements and missile wastage to mission failure or fratricide.
In addition, DOT&E’s limited user test report found that the Patriot system  as a whole did not meet the reliability requirement, but would have if the  Patriot radar had achieved its reliability goal. The metric for determining  reliability is an average of the number of hours between critical failures  that place the system out of service and into a state of repair. Although  the system is required to run at least 20 hours on average between  critical failures, during the limited user test, the Patriot fire unit fell short by  demonstrating an average time of around 11 hours. More than 70 percent  of the critical mission failures during the test were experienced by the  radar. Had the radar achieved its requirement of at least 38 hours, the fire  unit would have exceeded the 20 hour requirement. Army officials  attribute the radar reliability problems to a number of parts including  obsolete technology, which require high levels of maintenance. Too  frequent critical failures can create vulnerabilities for the system and  defended assets when the equipment is taken offline for maintenance  actions.
section_title: Current Patriot System Does Not Meet All Warfighter Needs
paragraphs: The warfighter has identified several capability needs for the Patriot  system that are currently unmet. One of the ways that warfighters in  various combatant commands express their capability needs is through  memos known as operational needs statements. The warfighter has  identified an operational need for capabilities to address many of the  same air and missile defense capability gaps for performance and  communications previously identified in table 1. While the shift to Army’s  IBCS, planned for initial fielding in fiscal year 2018, is designed to  address the capability need for joint integration below the battalion, the  warfighter has requested this new capability be fielded sooner.
Warfighters have also identified a need for reconfigurable training assets  and simulations for training in a variety of settings to operate and maintain  the system. See table 3 for current operational needs statements.
section_title: Multibillion Dollar Upgrade Strategy Awaits Key Testing Results to Determine the Extent to Which Upgrades Address Capability Needs or Require Additional Development
paragraphs: To address a diverse set of capability needs to mitigate evolving threats,  the Army is planning to field a number of upgrades, as well as a long-term  radar solution, projected to cost $2.9 billion through fiscal year 2021 with  additional costs needed for its long-term solutions. The program  successfully completed developmental testing on near and mid-term  upgrades in 2016. However, two operational test campaigns, consisting of  multiple ground and flight tests, currently planned to begin in late fiscal  year 2016 and 2019 should demonstrate how well the near and mid-term  upgrades work as intended and identify any performance shortfalls that  may require additional development.
subsections: section_title: Strategy to Address Patriot’s Capability Needs Is Projected to Cost $2.9 Billion through Fiscal Year 2021 with Additional Costs for Long-Term Solutions
paragraphs: The Army is fielding a number of upgrades in order to address divergent  needs identified by the Army, the program office, independent test  officials, and warfighters as discussed previously and summarized below  in table 4.
The Army has budgeted $2.9 billion in three budget lines for development  and procurement between fiscal years 2013 and 2021 for various  upgrades and a long-term radar solution. Specifically, the Army is  budgeting for three ongoing upgrades to address obsolescence issues,  four near-term hardware upgrades that begin fielding prior to fiscal year  2017, six mid-term upgrades and supporting equipment that will begin  fielding between fiscal years 2017 and 2021, and long-term upgrades— including a long-term radar solution, the details for which are still being  determined. Costs are expected to continue beyond fiscal year 2021 to  finish purchasing the necessary number of modifications already in  production as well as to develop and procure long-term solutions required  to address some of the capability needs. See Figure 3 for more details on  how costs are allocated among the obsolescence, near-term, mid-term,  and long-term upgrades.
Additional details on the upgrades including planned cost and schedule  are included below.
subsections: section_title: Ongoing Upgrades to Address Obsolescence Issues
paragraphs: The Army has spent nearly $306.3 million since fiscal year 2013 and  plans to spend an additional $361.5 million through fiscal year 2021 for  various obsolescence upgrades that have been ongoing in the program  for years and are planned to continue. These upgrades improve  readiness and reduce future operation and sustainment costs for Patriot  components. Additional details on these upgrades and the Patriot  capability needs they plan to address are included in table 5.
Requests for funding for these three ongoing upgrades to address  obsolescence issues are expected to continue beyond fiscal year 2021.  See figure 4 for planned costs between fiscal years 2013 and 2021.
The Army has spent nearly $273.9 million since fiscal year 2013 and  plans to spend an additional $553.7 million through fiscal year 2021 for  near-term upgrades that begin fielding prior to fiscal year 2017 to address  critical communication needs, ensure legacy components are sustainable,  and address warfighter needs for system capability and training. For  details on the near-term upgrades and the Patriot capability needs they  plan to address, see table 6.
The fielding schedule for Patriot near-term upgrades is included in figure  5 along with the total planned costs from fiscal years 2013-2021.  However, the program will need to request additional funds beyond fiscal  year 2021 to complete the purchase of launcher upgrades. Fielding for  some of the training software and hardware devices began prior to fiscal  year 2013.
The Army has spent nearly $553.1 million since fiscal year 2013 and  plans to spend an additional $437.3 million for mid-term upgrades and  supporting test equipment that begin fielding between fiscal years 2017  and 2021. Among the mid-term upgrades is the remaining hardware  needed—a radar digital processor—to prepare the system for integration  with IBCS. Also key among these upgrades is a major software upgrade  called Post Deployment Build-8 (PDB-8), which, in addition to a second  software upgrade called PDB-8.1, is intended to improve communications  and system capabilities against threats. Together, these mid-term  upgrades, along with a test detachment, are intended to improve system  performance, address warfighter needs, reduce obsolescence, and  support Patriot testing needs. For details on the near-term upgrades and  test detachment and the Patriot capability needs they plan to address,  see table 7.
The fielding schedule and total planned costs for Patriot mid-term  upgrades between fiscal years 2013 and 2021 are included in figure 6.  Costs for PDB-8 and PDB-8.1 software-related tasks are estimated based  on software-related tasks in the budget. Congress recommended  reductions in requested development funding for software-related efforts  by 50 percent or more each year between fiscal year 2013 and 2015,  amounting to nearly $200 million in reductions. According to program  officials, these reductions caused the program to delay some planned  capabilities from PDB-8 until PDB-8.1. Officials explained that software  capabilities currently planned for PDB 8.1 could be affected by available  funding in any given year and may lead to deferring capability into future  software upgrades. The program has already planned to continue  software capability costs beyond fiscal year 2019 for future software  improvements in the missile, launcher, or radar components following  PDB-8.1.
Additional details on the status of the development and procurement of  Patriot’s near and mid-term upgrades is included in appendix III.
section_title: Long-term Upgrades
paragraphs: The Army has spent around $8.5 million since fiscal year 2013 and plans  to spend an additional $437.8 million between fiscal years 2017 and 2021  for long-term software and radar solutions to continue to address  capability needs. Of the planned $437.8 million, the program has initially  budgeted around $74 million in fiscal years 2020 and 2021 for future  software improvements in the missile, launcher, or radar components  beyond PDB-8.1, with plans to continue software investments beyond  2021. The remaining $364.1 million is planned through fiscal year 2021  as a portion of total expected costs for a long-term radar solution. These  costs are part of a program funding line established in the 2017  president’s budget that the Army plans to manage as a new major  defense acquisition program, known as the LTAMD sensor, beginning in  fiscal year 2016. This long-term LTAMD sensor solution will be selected  based on the findings in the ongoing LTAMD AOA that is being conducted  as a result of concerns over the current Patriot radar’s high obsolescence  and sustainment costs as well as issues with performance and reliability.  For additional information on the AOA, see appendix II. There are many  radar options being considered in the AOA, from the current Patriot radar  with some modifications all the way up to a brand new radar  development. Officials estimate that fielding for the selected radar  solution could begin in the fiscal year 2028 time frame, with tactical  fielding to be completed within 7 years. Depending on the Army’s  selected radar solution, costs could increase and continue well beyond  fiscal year 2021 for additional development as well as for procurement  costs, which have not yet been determined. A breakdown of total planned  costs from fiscal years 2013 to 2021 for long-term upgrades as well as a  long-term radar solution is included in figure 7.
section_title: Patriot Program Successfully Completed Developmental Testing on Near and Mid-term Upgrades in Fiscal Year 2016
paragraphs: The Patriot program successfully completed developmental testing on the  system configured with near and mid-term upgrades in addition to  completing some limited developmental testing on the current PDB-7  version integrated with IBCS. Test and evaluation activities are an integral  part of developing and producing weapon systems, as they provide  knowledge of a system’s capabilities and limitations as it matures and is  eventually delivered for use by the warfighter. Developmental testing,  which is conducted by contractors, university and government labs, and  various DOD organizations, is intended to provide feedback on the  progress of a system’s design process and its combat capability as it  advances toward initial production or deployment.
The Patriot program successfully completed developmental testing in  fiscal year 2016 for the system configured with near and mid-term  hardware upgrades. The Army Test and Evaluation Center conducted  system-level developmental testing for Patriot configured with PDB-8  software in addition to other hardware upgrades, including modernized  displays in the command and control stations, the PAC-3 MSE with the  supporting launcher upgrades, and the radar digital processor. As part of  this test, the program successfully conducted four flight tests. These flight  tests demonstrated the system’s ability to intercept targets using a variety  of Patriot missiles, including the PAC-3 MSE. The Army Test and  Evaluation Command also performed testing on individual hardware  upgrades with favorable results. For example, the command conducted  some limited testing on the program’s new communication terminals and  found that the upgrades generally work as intended. However, additional  testing to evaluate the full functionality of the terminals is required prior to  full material release.
The IAMD program conducted two developmental flight intercept tests in  2015 of the PDB-7 version of Patriot integrated with IBCS, which also met  main objectives. During one of these tests, IBCS was able to command a  Patriot launcher to launch a missile and destroy a target using tracking  data from another Army system radar.
section_title: Two Operational Tests Will Determine How Well Near and Mid-Term Upgrades Address Patriot Capability Needs and Identify Any Performance Shortfalls That May Require Further Development
paragraphs: The program currently has two operational tests planned through 2020  that will test the system configured with upgraded software PDB-8 and  PDB-8.1 as well as with assorted near-term and mid-term hardware  upgrades as seen in table 8. Operational test and evaluation is intended  to evaluate a system’s effectiveness and suitability under realistic combat  conditions before full-rate production or deployment occurs. Operational  testing for PDB-8 is planned to begin in the fourth quarter of fiscal year  2016 and complete in the fourth quarter of fiscal year 2017. Operational  testing for PDB-8.1 is planned to begin in the fourth quarter of fiscal year  2019 and complete in the third quarter of fiscal year 2020. While  developmental testing thus far has been successful, the results of  operational test and evaluation will reveal the extent to which many of the  upgrades work as intended to address some of Patriot’s diverse capability  needs.
For example, operational testing for PDB-8 will evaluate how well the  software and hardware upgrades address the previously identified  performance shortfalls from PDB-7—including issues with the radar’s  reliability. In addition, the test will also evaluate the effectiveness and  efficiency of training aids and devices that are being procured to address  warfighter needs. Operational testing for PDB-8.1 is planned to evaluate  how well PDB-8.1 software capability upgrades effectively address  remaining system performance needs. According to Army Test and  Evaluation Command officials, upgrades that have not yet begun  production, like the global positioning anti-jamming hardware upgrade  and the radar anti-jamming upgrade, have not yet been incorporated into  testing plans.
However, near- and mid-term upgrades aren’t expected to fully address  all of the Patriot capability needs, which will require long-term upgrade  solutions. For example, the program plans for its near and mid-term  upgrades to provide significant enhancements to radar reliability and  sensing range to support the PAC-3 MSE missile’s mission against  stressing threats, but does not expect them to fully address the  performance needs without the long-term radar solution. In addition,  currently planned software upgrades are intended to provide capabilities  to help address tactical ballistic missile threats and electronic attacks, but  additional long-term software—and potential additional hardware— investments are needed to continue improving capabilities against the  evolving threat, which continues to create new gaps in the system’s  capabilities.
Operational testing results could identify unexpected performance  shortfalls in the near and mid-term upgrades that require additional  development. In the case of PDB-7, for example, operational test results  identified unexpected performance shortfalls in system reliability that  required additional development in the latest near and mid-term upgrades  to address. Operational testing for PDB-8 or PDB-8.1 could also identify  unexpected performance shortfalls that require additional development to  insert capabilities into future software or hardware upgrades for Patriot  components.
section_title: The Army Lacks an Oversight Mechanism to Track Progress and Ensure Accountability of Near and Mid-term Patriot Upgrades If Additional Development Is Needed
paragraphs: Oversight of Patriot upgrades has been limited because of how the Army  chose to define and manage them, including not establishing oversight  mechanisms similar to those generally applicable to major defense  acquisition programs. The Army chose to incorporate the Patriot upgrade  efforts into the existing Patriot program which made certain oversight  mechanisms inapplicable. While it would not be productive for the  program to go back and establish these mechanisms from development  start, upcoming operational tests provide the Army with an opportunity to  provide oversight and ensure accountability for the cost, schedule, and  performance of near- and mid-term upgrades, tested along with PDB-8  and PDB-8.1, if further development is needed.
subsections: section_title: Congressional Oversight of Near and Mid-term Upgrades Has Been Limited By DOD’s Decision Not to Track and Report Cost, Schedule, or Performance Progress
paragraphs: Up to this point, the Patriot program has not put a mechanism in place to  track or report progress against cost, schedule, or performance baselines  of its upgrade efforts, similar to those generally required of multibillion  dollar DOD acquisition programs. Under DOD instruction 5000.02 and  related statutes, major defense acquisition programs (MDAPs) are  subject to a number of oversight mechanisms that provide transparency  into program plans and progress. Although the Army’s 2013 cost  estimate for all the Patriot upgrades met the threshold to be considered a  separate MDAP, the Army chose not to define the upgrade efforts as  such. Instead, the upgrades were incorporated into the existing Patriot  program, which resulted in the upgrade efforts not being separately  subject to statutory and regulatory reporting requirements that generally  apply to MDAPs. In addition, the program did not establish any oversight  mechanisms for the upgrades that were similar to those generally  required of MDAPs.
For example, new MDAPs are generally required to establish an  approved program baseline that includes initial estimates for key cost,  schedule, and performance metrics at the beginning of system  development, at the start of production, and before the start of full rate  production. Information about these baselines is reported to Congress in  a standardized format through Selected Acquisition Reports. On a  periodic basis, programs update the information in these reports by  comparing the latest cost, schedule, and performance estimates against  the initial estimates and providing explanations for any major deviations.  Establishing reliable cost and schedule estimates are best practices that  we have found go hand-in-hand as fundamental management tools that  can help all government programs use public funds effectively. Further,  as we demonstrate each year in special annual reports assessing DOD’s  acquisition of selected weapon programs, and in related testimonies  before congressional committees, regular comparisons of program cost,  schedule, and performance against initial estimates is an essential  oversight tool. Such data, when maintained and reported on a regular  basis, help the decisionmakers who oversee program progress  understand the significance of any increases or decreases in cost or  schedule as a program evolves, provide transparency, and give Congress  and the Office of the Secretary of Defense a mechanism to hold the  program accountable for its intended results. As we reported in our March  2016 assessment, programs that do not uniformly implement these and  other best practices tend to realize significant cost growth and delays in  delivering needed capabilities.
Army officials explained that the existing Patriot program’s 2002  acquisition strategy provided approval for the Army to execute Patriot  upgrades as part of this program, which was defined as an MDAP, and  the Office of the Secretary of Defense had no objection. However, the  requirement for MDAPs to continue reporting Selected Acquisition  Reports ceases after 90 percent of the program’s items are delivered or  90 percent of planned expenditures under the program have been made.  The Patriot program submitted its final Selected Acquisition Report in  2004 when the program was considered more than 90 percent complete.  Absent the requirement to do so, the program has not provided  decisionmakers with similar information. As a result, there has been no  mechanism for DOD and congressional decisionmakers to monitor  performance of the approximately $1 billion spent on Patriot upgrades  since 2013 and to ensure that efforts have resulted in progress toward  meeting the program’s goals.
section_title: Upcoming Tests of PDB-8 and PDB-8.1 Provide Opportunity to Increase Oversight of Near and Mid-term Upgrades If Further Development Is Needed
paragraphs: While it would not be productive for DOD to go back and track cost or  schedule changes from the start of the Patriot upgrade efforts (see  appendix III), in the event that upcoming operational tests reveal the need  for further development of PDB-8 and PDB-8.1 and other near- and mid- term upgrades tested along with that software, the department will have  an opportunity to provide increased oversight of those upgrades. As  noted above, DOD already plans to define the long-term LTAMD sensor  solution as a separate MDAP, which indicates the program would be  subject to the oversight requirements applicable to MDAPs, such as those  discussed above.
Without estimated costs and schedule needed to complete the  development of upgrades for essential Patriot capabilities, similar to those  generally required of new major defense acquisition programs, DOD and  congressional decisionmakers will lack an essential oversight tool. In  addition, unless, at the same time, DOD provides Congress with an  estimate of the amount of development costs it has incurred since 2013  for near- and mid-term Patriot upgrades operationally tested along with  PDB-8 and PDB-8.1, Congress will not have a basis from which to  understand the significance of any increases or decreases as the  program evolves. Finally, without annual reporting mechanisms that  enable comparisons between subsequent cost and schedule estimates  and initial estimates, along with periodic explanations for any major cost  or schedule deviations, Congress will lack critical information it needs to  evaluate future program budget requests.
section_title: The Army’s Plan to Complete Patriot Modernization Efforts Synchronizes Fielding with Training and Brings Benefits and Challenges That the Army Continues to Manage
paragraphs: The Army selected a plan to synchronize its fielding of upgraded versions  of the Patriot system during its transition to the Integrated Air and Missile  Defense Battle Command System (IBCS) that allows it to meet  operational demands. Integrating Patriot battalions with IBCS can provide  organizational and personnel flexibility in the future. However, the process  of fielding these upgrades over the course of the 8-year transition to IBCS  amplifies some of the challenges the Army is already facing with training  complexity and maintenance schedules for the Patriot system. The Army  is taking steps to mitigate these challenges.
subsections: section_title: The Army’s Plan for Fielding Modernized Patriots to Combatant Commands Synchronizes with Testing and Training Requirements to Meet High Operational Demands
paragraphs: The Army has a plan for fielding modernized Patriots to Combatant  Commands. The process of modernizing a Patriot battalion—transitioning  it from its current PDB-7 software version into launchers and radars  integrated with IBCS, involves two phases. The first phase requires the  battalion to be upgraded to the PDB-8 software version. Once the  battalion receives PDB-8, it is ready for phase 2, which consists of a  second software update to integrate the system components with IBCS.  In some cases, a battalion can undergo both modernization phases  consecutively, but, in other cases, a battalion can complete phase 1 and  then wait a number of years to complete phase 2. The fielding plan the  Army selected completes phase 2 of integrating the battalion into IBCS at  a rate of approximately two Patriot battalions per year. By fiscal year 2022  the Army plans to have completed phase 1 for all 15 battalions with 9  battalions completing phase 2 and being IBCS compatible. IBCS  integration continues through fiscal year 2025, as seen in figure 8.
To synchronize fielding with testing, the Army removed a Patriot battalion  from the operational deployment rotation and assigned it solely to  modernization testing. Army officials told us this is a key enabler of the  fielding strategy—without it the plan becomes unworkable. Specifically,  the amount of time required to begin and complete IBCS integration  testing exceeds the amount of time that any one Patriot battalion is  available to perform that testing. Therefore, the Army would have to start  with one battalion and complete the testing with a second battalion— which would add an extra 6 to 9 months to train the second battalion on  how to use the new equipment. After completing the United States / North  Atlantic Treaty Organization mission in Turkey, the Army was able to  adjust its Patriot unit rotation schedule which enabled the Army to assign  a battalion to support Patriot modernization testing. The battalion’s test  assignment began in April 2016 and the Army plans to keep the battalion  solely for testing into fiscal year 2018. Army officials also told us that the  Vice Chief of Staff for the Army recently approved increased funding for  the Army Air and Missile Defense test detachment to increase the  manning from 35 to over 140. Increasing the size of the detachment will  allow the Patriot test battalion to rejoin the operational rotation in fiscal  year 2018, providing the combatant commands with more available  Patriot battalions.
The Army considered four alternative plans for how and when to field  these 2 phases of modernization to the 15 Patriot battalions. The baseline  plan would have upgraded three battalions per year to PDB-8 and one  per year to IBCS. Another plan would have upgraded two or three  battalions per year to PDB-8 and two per year to IBCS, while focusing on  upgrading units in Europe first. A third plan would have upgraded three  battalions per year to PDB-8 and two to IBCS, and would have upgraded  units in the Pacific first. A fourth alternative, which the Army selected,  completes phase 1 of the upgrades for the nine Patriot battalions that are  not being upgraded directly to IBCS compatibility by fiscal year 2022 and  completes phase 2 of the upgrades to make all 15 Patriot battalions IBCS  compatible by 2025.
The Army prioritized meeting training requirements and operational  demands when selecting its plan for completing Patriot modernization  efforts. The Army used five criteria to evaluate the four alternative plans.  The Army’s evaluation criteria included maximizing the number of Patriot  battalions available at any given time to support operations, maintaining  the same software version for all Patriot battalions under a particular  brigade to make training consistent, and meeting these and other  competing needs within funding constraints. Table 9 below provides a  description of the criteria, the weighting the Army assigned to it, and how  well the plan the Army selected optimized the criteria. Based on the  Army’s analysis, the selected plan did the best job of balancing all of the  key considerations reflected in these criteria.
section_title: Patriot’s Inclusion in IBCS Provides Operational and Personnel Benefits
paragraphs: Army officials told us that moving Patriot to IBCS provides benefits in  meeting Combatant Command operational needs more flexibly because  the system can be reorganized so that it no longer has to be deployed as  a complete battalion. IBCS-compatible Patriot components can be  deployed as individual radars and launchers, networked through IBCS.  Army officials told us that instead of having 15 Patriot battalions, the Army  will have 60 fire units’ worth of radars and launchers that can be deployed  more flexibly to meet combatant command operational demands.
Transitioning Patriot to IBCS compatibility can potentially lead to  organizational changes that reduce the number of personnel required to  operate and maintain the radars and launchers. The Army plans to use  this streamlined organizational structure as an opportunity to create a  more even distribution of tasks. As part of its findings during the PDB-7  Limited User Test, DOT&E reported that Patriot personnel currently  performing the job of operator/maintainers are required to perform many  complex tasks, resulting in poor operator performance. Army officials told  us they expect to realign the current number of personnel specialties  within Patriot from nine specialties down to four. In addition, these  specialties will no longer be Patriot specific—rather they will cut across  the integrated air and missile defense community, allowing the Army to  address some challenges with the relatively low number of personnel in  some specialties. Army officials told us that the realignment would also  allow the Army to alter the skillset of personnel who are currently  operators/maintainers of the equipment into purely operators, while  maintainers would take on some additional responsibilities. Further, by  2025 the Army plans for current Patriot operators and maintainers to  maintain and operate a variety of Army air and missile defense systems,  as opposed to being assigned solely to Patriot.
section_title: Patriot’s Inclusion in IBCS Amplifies Challenges with Training Complexity and Maintenance Schedules, but the Army Has Mitigation Plans in Place
paragraphs: Migrating Patriot to IBCS amplifies training challenges by adding new  training into the Army’s Patriot training schedule. Further, for a period of  time the Army will be training personnel on three different versions of  Patriot—PDB-7, PDB-8, and IBCS. Army officials told us that due to the  high deployment frequency of the Patriot force, the current training  schedule does not completely prepare Patriot operators on all tasks  before deployments. To address this, the Army revised the training  certification progression so that high priority training is completed before  deployment, and less important training can occur after deployment.  However, to prepare for the transition to IBCS, the warfighter requires  additional training on how to effectively operate the equipment under an  airspace complicated with data from multiple sensors. This increasingly  complex training required by Patriot operators could cause further issues  with the Patriot training schedule in the future. Over the long term,  officials told us that the Army plans to address some of these challenges  by updating the training certification program to match up with the  changes to the Patriot system (for instance more emphasis on joint  operations) and by adding more advanced certification levels that would  include skills not currently included as part of the certification process.  While Army officials told us they are in the initial stages of implementing  changes to the training program, they expect it to be implemented by  2025 when the Army completes the transition of all Patriot units to IBCS.
The modernization fielding plan the Army is pursuing also poses a near-  to mid-term maintenance challenge. The Army currently plans to perform  comprehensive maintenance on only one Patriot battalion per year  through fiscal year 2021 in order for battalions to be available for  modernization, training, and operations. However, Army officials told us  they will not be able to complete maintenance on all 15 Patriot battalions  within the expected 10-year life cycle at that rate. As a result, officials  confirmed that the Army is assuming an elevated risk of equipment  breakdown. To mitigate this challenge in the short term, the Army is  performing less comprehensive maintenance after every deployment and  maintaining a sizable inventory of spares for those parts that have high  failure rates. As more Patriot battalions become IBCS-compatible, the  Army is considering ways to schedule comprehensive maintenance on  more than one battalion per year. However, the officials were unsure if  they would be able to have two battalions worth of equipment offline for  maintenance and still have enough availability to meet training and  operational demands.
section_title: The Army Holds Regular Synchronization Meetings to Manage Challenges It Encounters during Patriot’s Transition to IBCS
paragraphs: The Army regularly coordinates on the status of doctrine, organization,  training, materiel, leadership, personnel, and facilities implications of  Patriots transition to IBCS through the use of quarterly transformation  summits. These summits are internal meetings that include  decisionmakers from all of the key domains within the Army that need to  synchronize on integrated air and missile defense issues, including  training, doctrine, leader development, and facilities. Briefings from these  summits show that the Army officials discuss modernization and  maintenance schedules, training strategy, and facility needs, among other  topics at these summits. Army officials told us that as a result of these  meetings, the Army decided to alter the Patriot deployment duration from  12 months to 9 months, concluding that this change would have a  minimal impact on the modernization and training schedules, while  providing the same operational support to combatant commands. In  implementing the deployment duration change the Army will keep five  battalions over the next 5 years on the 12-month deployment schedule,  while all other Patriot deployments will last for 9 months. Army officials  said that this fluctuation was necessary in order to allow enough time for  other Patriot battalion modernization, testing, and training to occur— information they were aware of because of the summit discussions.
section_title: Conclusions
paragraphs: As a cornerstone of the Army’s air and missile defense architecture, the  Patriot system is deployed worldwide in defense of the United States and  its allies. The program faces multiple challenges to overcome the  obsolescence of a system that has been fielded for decades, improve  capabilities to address ever-evolving threats, and complete its transition  from a stand-alone system to an integrated component of the Army’s  Integrated Air and Missile Defense. The Army has spent approximately  $1.1 billion since 2013 to develop and procure Patriot upgrades and has  requested another $1.8 billion, which includes funding for a long-term  radar solution, over the next five years. A modernization program of this  magnitude and complexity demands high-level oversight to ensure that  the upgrades are completed on time, within planned cost, and that they  provide the intended capabilities. In the long term, the Patriot system will  no longer be Patriot as we know it but will be broken down into its major  components—a radar, launcher, and a missile—integrated with Army’s  Integrated Air and Missile Defense System of Systems. Of the three  remaining components, the Army has already defined the missile as a  separate major defense acquisition program and currently plans to do the  same for the LTAMD sensor solution, which accounts for $364 million of  the requested $1.8 billion over the next five years. Continuing to  separately manage and track progress for these components should help  provide Congress with the oversight and accountability it needs to make  important investment decisions. Although the Army estimated in 2013 that  costs for Patriot upgrades would meet the threshold to be considered a  major defense acquisition program (MDAP), the Army chose to  incorporate the Patriot upgrade efforts into the existing Patriot program  which made certain oversight mechanisms inapplicable. The Army would  have put itself in a much better position to oversee its Patriot upgrade  efforts had it made the decision in 2013 to manage Patriot upgrades as a  separate major defense acquisition program. Should operational testing  for PDB-8 and PDB-8.1 reveal performance shortfalls in the near and  mid-term upgrades tested, the additional development required could  present an opportunity for DOD to provide a level of oversight and  accountability not seen by the Patriot upgrade efforts so far. Beginning  any additional development with cost, schedule, and performance  estimates—informed by an estimate of the amount of development costs  the upgrade effort has incurred since 2013—would provide DOD and  congressional decisionmakers an essential oversight tool, particularly  when considering future budget requests. Further, regular comparisons of  program cost, schedule, and performance against initial estimates  enhance decisionmakers’ understanding of the significance of any  increases or decreases in cost or schedule as a program evolves.
section_title: Recommendations for Executive Action
paragraphs: In the event that operational test results for PDB-8 and PDB-8.1 reveal  performance shortfalls that require additional development of the near  and mid-term upgrades tested, we recommend that the Secretary of  Defense direct the Secretary of the Army to establish mechanisms for  overseeing those upgrades commensurate with other major defense  acquisition programs, to include:  1.  An initial report—similar to a Selected Acquisition Report—as soon as  practical following operational testing for both PDB-8 and PDB-8.1, on  the near and mid-term upgrades evaluated during these tests,  including:  cost, schedule, and performance estimates for any additional  development that is needed; and an estimate of the amount of development costs it has incurred  since 2013 for near- and mid-term Patriot upgrades operationally  tested along with PDB-8 and PDB-8.1.  2.  Annual updates to Congress comparing the latest cost and schedule  estimates against the initial estimates and providing explanations for  any major deviations until development is complete.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOD for comment. DOD provided us  with written comments which are reprinted in appendix IV. DOD also  provided technical comments, which were incorporated as appropriate.
DOD partially concurred with our recommendations to provide an initial  report—similar to a Selected Acquisition Report—and to provide annual  updates to Congress in an effort to establish oversight mechanisms  commensurate with other major defense acquisition programs for  upgrades operationally tested with PDB-8 and PDB-8.1 in the event that  operational test results reveal performance shortfalls that require  additional development. In its response, DOD stated that system software  updates currently being performed for Patriot, such as PDB-8 and PDB- 8.1, will cease with updates transitioning to IBCS. It also noted that future  post deployment build updates will be developed and tested for IBCS as  part of the Army’s IAMD program, which is subject to acquisition oversight  and reporting required by law and regulation. Further, DOD noted that  future development and testing of the LTAMD sensor will also be subject  to acquisition oversight and reporting required by law and regulation.  DOD stated that using existing oversight and reporting mechanisms for  these major defense acquisition programs would more accurately reflect  the development program and is more appropriate than introducing  additional non-standard reports.
DOD’s response focuses on tracking and reporting progress on other  MDAPs without clarifying how or if it will track progress on current PDB-8  and PDB-8.1 efforts. The IAMD program has already established its  planned content in a baseline, and details for the LTAMD sensor program  are still being determined. Regardless, tracking and reporting progress on  the pre-existing IAMD program or future development LTAMD sensor  program will not provide Congress with oversight and accountability on  the outcomes for current work on the near- and mid-term upgrades tested  with PDB-8 and PDB-8.1. As such, we maintain our position that the  Secretary of Defense should take the recommended actions to direct the  Army to establish mechanisms for overseeing any additional work on  those upgrades commensurate with other major defense acquisition  programs, by providing an initial report that is similar to a Selection  Acquisition Report and annual updates to Congress that compare the  latest cost and schedule estimates against the initial estimates for PDB-8  and PDB-8.1 upgrades.
We are sending copies of this report to the appropriate congressional  committees, the Secretary of Defense, and the Secretary of the Army.  The report is also available at no charge on the GAO website at  http://www.gao.gov.
Should you or your staff have any questions about this report, please  contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff who made major contributions to  this report are listed in appendix V.
section_title: Appendix I: Scope and Methodology
paragraphs: To determine the current status of the Patriot system’s performance and  the extent to which it addresses warfighter needs, we did the following:  1.  To determine the current status of the Patriot system’s performance,  we reviewed briefings from the Lower Tier Project Office in Huntsville,  AL and from the Capabilities Development and Integration Directorate  at Fort Sill in Lawton, OK on the current system’s performance  specifications. To determine the extent to which the current version is  meeting its performance requirements, we reviewed 2013 limited user  test results from the Director, Operational Test and Evaluation  (DOT&E) to see how well the Patriot system performed against its  performance parameters as defined in the capabilities development  and production requirements documents. In addition, we obtained the  Patriot’s Post Development Build-7 (PBD-7) conditional material  release “get well” plans, which outline the performance shortfalls of  PDB-7 that need to be mitigated. We also discussed these shortfalls  with officials from DOT&E in Arlington, VA; the Army Test and  Evaluation Command at Fort Bliss in El Paso, TX who conducted the  PDB-7 limited user test; the Lower Tier Project Office; and the  Capabilities Development and Integration Directorate.  2.  To determine the extent to which the current version of the Patriot  system is meeting warfighter needs to address the growing threat, we  reviewed warfighter operational needs statements, which document  requests from the warfighter to the Army for urgent, real-time Patriot  capabilities and other needed upgrades. We assessed the reliability of  the currently open Patriot-related operational needs statements from  2013 by comparing the list of operational needs statements obtained  from the Capabilities Development and Integration Directorate to  those received from the Capabilities Integration Division of the  Department of the Army Military Operations in Arlington, VA. Based  on our review of the data and interviews with officials at both  locations, we determined that the data were sufficiently reliable for the  purposes of our reporting objectives. We also held discussions with  these officials about the unfulfilled operational needs statements and  the Army’s plan for addressing them. In addition, we interviewed  combatant command officials from the Pacific Command in Honolulu,  HI; the European Command in Stuttgart, Germany; and the Central  Command in Tampa, FL to obtain views on Patriot performance  needs from various combatant commands.
To assess the extent to which the Patriot system upgrades will address  capability needs and describe the cost, schedule, and testing plans  associated with those upgrades we did the following:  1.  To determine the various Patriot capability needs, we began by  reviewing the validated air and missile defense capability gaps, which  the program used as a foundation for its 2013 requirements  documents. We examined these Patriot-related gaps listed in the  2011 Army Functional Concept for Fires Capability-Based Need  Assessment Functional Needs Analysis and Functional Solution  Analysis reports. Based on our analysis of these documents and  additional Army briefings and plans, we identified a selection of high- priority critical air and missile defense gaps that were related to the  Patriot program. We also reviewed requirements in the Patriot  Increment 3 Capability Development Document related to training and  obsolescence and sustainment and discussed these requirements  with Army officials at the Lower Tier Project Office and the Air  Defense Artillery School at Fort Sill in Lawton, OK.  2.  To understand the evolving threat and how it is driving capability  needs for the Patriot system, we reviewed the 2011 and 2015 System  Threat Assessment Reports and discussed the Patriot-related threat  assessment findings with officials from the Missile and Space  Intelligence Center in Huntsville, AL and the Capabilities Development  and Integration Directorate.  3.  To describe the cost, schedule, and testing plans for the Patriot  upgrades, we obtained and analyzed detailed cost data derived from  program budgets, program schedules for testing and fielding, and test  and evaluation master plans. We discussed these plans with officials  from DOT&E; the Capabilities Development and Integration  Directorate; and the Lower Tier Project Office. We focused our cost  review on two Patriot program budget lines, which detail the U.S.  contribution to development and procurement costs for planned  upgrades, and a third budget line providing initial development funding  for the Lower Tier Air and Missile Defense (LTAMD) sensor solution.  Planned costs for fiscal years 2017 through 2021 are based on  detailed Army planning budget data supporting the President’s budget  for fiscal year 2017. We deflated these budget numbers to base year  2017 dollars.  4.  To determine the extent to which planned upgrades will address  capability needs, we obtained detailed information from the  Capabilities Development and Integration Directorate officials  mapping each of the planned upgrades to the capability need it is  intended to help address. We also obtained and reviewed the  schedule and scope of planned operational testing in the System  Evaluation Plan to determine when the upgrades would be evaluated.  Further, we reviewed the scope of the analysis of alternatives  currently underway to determine what capability needs the radar and  launcher alternatives being considered are intended to address and  discussed these needs with Army officials from the Capabilities  Development and Integration Directorate and the Lower Tier Project  Office.
To determine the level of oversight and accountability provided for the  upgrades, we received information regarding how and why the upgrades  were executed under the existing Patriot program from Army officials. We  reviewed prior legislation and related reports since 2012 to understand  Congress’s concerns on oversight and accountability for the latest Patriot  upgrades. We then reviewed DOD guidance documents and briefings to  determine the level of oversight planned for the long-term radar solution.  We also reviewed DOD acquisition regulations and related statutes to  determine the typical requirements for facilitating Congressional oversight  and accountability of major defense acquisition programs.
To assess the extent to which the Army’s plan for fielding modernized  Patriots synchronizes with training schedules and operational demands,  we analyzed the Army’s fielding plan as well as operational and training  schedules. We also interviewed knowledgeable Army officials to discuss  how the fielding plan was chosen, the benefits and challenges associated  with the chosen plan, as well as any effects of the plan on operations,  personnel, doctrine, organization, testing, and training.
To assess the extent to which DOD’s guidance for conducting its LTAMD  analysis of alternatives (AOA) meets GAO best practices, we obtained  Department of Defense AOA guidance documents. These documents  consist of a directive from the Army Headquarters directing the Army  Training and Doctrine Command Analysis Center to conduct the LTAMD  AOA study, a study plan developed by the Army Training and Doctrine  Command Analysis Center, and guidance from the Office of the Secretary  of Defense for Cost Assessment and Program Evaluation (CAPE). We  compared the processes outlined in the guidance documents to the 22  best practices GAO identified in GAO-16-22. We also met with officials  from CAPE to discuss GAO best practice processes that were not  documented in the guidance documents and supplemented our analysis  with some of this information. We used a five-point scoring system to  evaluate how well the LTAMD AOA guidance documents conformed to  each of the 22 best practices. We then used the average of the scores for  the best practices under each of the four characteristics—well- documented, comprehensive, unbiased, and credible—to determine an  overall score for each characteristic. The results of GAO’s analysis  underwent four separate levels of internal review to ensure accuracy as  well as cross-checking the scores throughout the analysis for  consistency. In addition, we provided the initial results of our analysis to  officials in the CAPE and Army Training and Doctrine Command Analysis  Center for review and received technical comments, which we  incorporated, as appropriate, into our final analysis. To characterize our  final results, if the average score for each characteristic was “met” or  “substantially met,” we concluded that the AOA process conformed to  best practices and could therefore be considered reliable.
We conducted this performance audit from June 2015 to August 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: The Lower Tier Air and Missile Defense Analysis of Alternatives Guidance Compared to GAO’s Best Practices for an Analysis of Alternatives Process
paragraphs: As part of our review of the Patriot system, we assessed the extent to  which the Department of Defense’s (DOD) guidance for conducting its  Lower Tier Air and Missile Defense (LTAMD) analysis of alternatives  (AOA), which is evaluating material modernization solutions for the  current Patriot radar and launcher for use with the Integrated Air and  Missile Defense (IAM) Battle Command System (IBCS), meets GAO best  practices and found that the guidance documents substantially met GAO  standards to be considered reliable. We compared the processes outlined  in the LTAMD AOA guidance documents to GAO best practices because  the LTAMD AOA report was not available at the time of our review.
The LTAMD AOA guidance documents provide the AOA study team with  a high-level roadmap for how to conduct the LTAMD AOA by outlining  processes to identify and select the alternatives, metrics, models, and  scenarios for use throughout the AOA process. While we cannot make  conclusions about the final AOA report until it is finalized and released, by  comparing the processes described in the LTAMD AOA guidance  documents to the 22 GAO best practices, we can make conclusions on  the quality of the processes used to develop it. If the processes are of  high quality, then the AOA study team has a good roadmap, which, if  followed, could produce a high-quality, reliable AOA. Based on our  analysis, the LTAMD AOA process described in its guidance met or  substantially met the criteria to be considered well-documented,  comprehensive, unbiased, and credible.
While we found that the LTAMD AOA guidance documents met or  substantially met 18 of the 22 best practices GAO established for the  AOA process to be considered reliable, our review also found that  contrary to GAO best practices, the final AOA report will not select a  preferred solution. Specifically, the LTAMD AOA guidance did not instruct  the study team to assign relative importance to the criteria that are used  to compare the options or to select a preferred solution for a modernized  radar and launcher as part of the final AOA report. According to CAPE  officials involved in the LTAMD AOA efforts, the purpose of this AOA is to  provide an analytic comparison of the options based on the criteria but to  then allow external decisionmakers to determine the relative importance  of each criterion and derive their own preferred solution. CAPE’s position  is that GAO’s best practice of assigning relative importance to criteria is  not appropriate for strategic investment decisions such as this. In  contrast, GAO best practices recommend that solutions be compared  based on pre-established criteria that reflect the relative importance of the  criteria because not reflecting its relative importance up front can  oversimply results and potentially mask important information leading to  an uninformed decision. In addition, GAO best practices state that a  preferred alternative should be identified and a rationale for that decision  be included as part of an AOA report. While a recommended solution in  the AOA report does not have to be binding, without one, decisionmakers  outside of the AOA process may misinterpret the analysis within the AOA  report and potentially come to a biased decision.
subsections: section_title: GAO’s 22 Best Practices for the AOA Process: Background and Introduction
paragraphs: In October 2015, GAO identified 22 best practices to provide a framework  for conducting an AOA and help ensure that entities consistently and  reliably select a preferred solution that best meets mission needs. To  identify a high-quality, reliable AOA process, GAO grouped the 22 best  practices under four characteristics. These characteristics evaluate  whether the AOA process is well-documented, comprehensive, unbiased,  and credible.  “Well-documented” means that the AOA process is thoroughly  described in a single document, including all source data, has clearly  detailed methodologies, calculations and results, and that selection  criteria are explained.  “Comprehensive” means that the AOA process ensures that the  mission need is defined in a way to allow for a robust set of  alternatives, that no alternatives are omitted, and that each alternative  is examined thoroughly for the project’s entire life-cycle.  “Unbiased” means that the AOA process does not have a  predisposition toward one alternative or another; it is based on  traceable and verifiable information.  “Credible” means that the AOA process thoroughly discusses the  limitations of the analyses resulting from the uncertainty that  surrounds both the data and the assumptions for each alternative.
Table 10 provides an explanation of how individual best practices are  grouped under each characteristic.
section_title: LTAMD AOA Guidance Results
paragraphs: Overall, the DOD’s LTAMD AOA guidance documents met or  substantially met the four characteristics of a high-quality and reliable  AOA process. To make this determination, we reviewed and scored how  well the guidance documents addressed each of the 22 best practices.  We scored the 22 best practices using a five-point system as follows:  “met” means the LTAMD AOA guidance documentation demonstrated  that it completely met the best practice; “substantially met” means that it  met a large portion of the best practice; “partially met” means that it met  about half of the best practice; “minimally met” means that it met a small  portion of the best practice; and “did not meet” means that it did not meet  the best practice. We found that the LTAMD AOA guidance documents  met or substantially met 18 of the 22 best practices. We then took the  average of those best practice scores that aligned with each of the four  characteristics, as shown above in Table 9, to derive a final score for  each characteristic. Table 11 provides the average score of the best  practices under each characteristic.
section_title: Appendix III: Status of Development and Procurement for Near and Mid-Term Upgrades
paragraphs: The Patriot program has made notable progress in the development and  procurement of near and mid-term upgrades since the upgrade efforts  began in 2013. Up to this point, significant costs for development and  procurement have already been incurred, costs and activities are winding  down, and the program plans to release the first of two major hardware  and software upgrades next year. In sum, the Army has spent about $1.1  billion of the $2.9 billion planned between fiscal years 2013 and 2021 to  address Patriot capability needs, as seen in figure 9.
Of the $1.8 billion currently planned between fiscal years 2017 and 2021,  $645 million is for development. Of those development funds, the  majority, $364 million, is allotted to developing the future radar solution,  the Lower Tier Air and Missile Defense (LTAMD) sensor, which the Army  currently plans to define as a separate major defense acquisition program  (MDAP).  Further, of the $645 million in development, only about $280  million is currently planned between fiscal years 2017 and 2021 for  developing software and hardware upgrades. The program has already  spent about $210 million for the development of near and mid-term  software and hardware upgrades between fiscal years 2013 and 2016.
Aside from the future radar development, there does not appear to be a  new wave of development activities beginning in the near future. Funding  for PDB-8 was already completed in fiscal year 2016 with fielding planned  for fiscal year 2017. Further, as seen in figure 10, costs planned for  software development appear to be tapering off toward the end of the  Future Years’ Defense Program in fiscal year 2021 when the program  expects to release PDB-8.1.
Near-term and mid-term upgrade procurement activities also appear to be  winding down. Most of the defined hardware upgrades are already in  production. Further, many of these upgrades were already mature with  relatively little being spent on hardware development for the purposes of  adapting them for Patriot or maximizing their benefit to the system.
Although the program is still planning to spend $1.15 billion in  procurement between fiscal years 2017 and 2021, which includes  ongoing upgrades to address obsolescence issues, six of the nine near- term and mid-term hardware upgrades and supporting equipment have  begun production, as seen in figure 11.
Lastly, costs planned for procurement upgrades appear to be tapering-off  toward the end of the Future Years’ Defense Program in fiscal year 2021,  as seen in figure 12. Currently, funds planned to continue beyond fiscal  year 2021 are for ongoing upgrades to address obsolescence issues, for  completing the purchase of launcher modifications, and for continuing  investments in training upgrades.
section_title: Appendix IV: Comments from the Department of Defense
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, LaTonya D. Miller, Assistant  Director; Kevin L. O’Neill; James P. Haynes II; Meredith Allen Kimmett;  Randy F. Neice; Jenny Shinn; David L. Richards; Jennifer V. Leotta;  Karen A. Richey; Alyssa B. Weir; Katherine Shea Lenane; Stephanie M.  Gustafson; Oziel A. Trevino; and Joseph W. Kirschbaum made key  contributions to this report.