Patriot Modernization: Oversight Mechanism Needed to Track Progress and Provide Accountability
Released: 2016-08-25T00:00:00

Letter

Background

Patriot is a mobile Army surface-to-air missile system designed to counter tactical ballistic missiles; cruise missiles; and other threats such as airplanes, helicopters, and unmanned aerial vehicles. Patriot was first deployed in the early 1980s and since that time has received a number of substantial updates to keep pace with the growing threat. Patriot is deployed worldwide in defense of the United States and its allies’ key national interests, ground forces, and critical assets.

A Patriot fire unit is made up of four basic components: (1) a ground- based radar to detect and track targets; (2) launchers; (3) interceptor missiles; and (4) a command, control, and communication station. Patriot fire units are organized to fight in groups known as battalions. Each battalion is controlled by its own command and control station and can manage up to six fire units, although a battalion is typically deployed with four. For a notional configuration of a Patriot battalion, see figure 1. Several battalions can be commanded by an Army brigade. Brigades are also responsible for certifying that the equipment can be employed as required and for training the battalions. The brigade manages battalion personnel under its command, with the ability to transfer personnel among battalions to fill personnel gaps as needed.

The air and missile defense architecture consists of several systems deployed together to provide a layered defense against various threats in a range of battlespaces. Other air and missile defense systems can contain, like Patriot, a sensor, a launcher, and a system-centric command and control station. These systems’ command and control stations can share information with other air and missile defense systems or with other joint systems through external communication links, as seen in figure 1. The air and missile defense architecture includes systems designed to counter threats at a low altitude—such as rockets, artillery, and mortar— as well as systems designed to defeat high-altitude threats intercepted above the earth’s atmosphere. Patriot serves as the Army’s primary element deployed to intercept targets in this middle range of battlespace—above the range of rockets, artillery, and mortar, but within the earth’s atmosphere.

To Address Evolving Threats and Related Capability Gaps, the Army Adopted a New Integrated Air and Missile Defense Architecture

The Army has identified a number of air and missile defense communication and performance capability gaps in its ability to address evolving global threats. Over the last decade, adversaries have acquired more robust, diverse, and complex threats. According to a 2010 Ballistic Missile Defense Review Report, ballistic missiles are more technically sophisticated, more proliferated, include more advanced countermeasures, and continue to challenge U.S. ballistic missile defense system capabilities. Cruise missiles have also become relatively simple to develop, are cheaper than ballistic missiles or aircraft, and are easy to export. Additionally, advanced electronic attacks, such as jamming or spoofing, have become more widespread and easier to effectively produce. Sophisticated enemies also have the ability to use a combination of integrated attacks including electronic and cyber warfare, a variety of inbound ballistic and cruise missiles, special operation forces, and other methods to complicate the battlespace. The Army has identified some high-priority air and missile defense gaps in its ability to respond to the growing threats, as seen in table 1.

The Army announced an Air and Missile Defense Strategy in 2012 to address communication and performance capability gaps by integrating its current air and missile defense system components (e.g. sensors and launchers), including Patriot, under a central network and command and control system and linking them with joint and potential coalition allies. The Integrated Air and Missile Defense (IAMD) program is currently developing the IAMD Battle Command System (IBCS) that plans to connect Patriot radars and launchers into IBCS’s central network and command and control stations. By connecting these components directly with IBCS, the Army intends to divest air and missile defense systems of their system-specific command and control stations and allow them to become network enabled sensors and launchers. See figure 2 below for a notional representation of the future Integrated Air and Missile Defense architecture.

The Army intends for the integrated air and missile defense architecture to address communication and performance capability gaps by allowing IBCS to collect information from a variety of sensors, fuse that data into a single battlespace picture, and use that information to engage targets. Receiving sensor data from a range of sensors could enable longer- distance engagements and provide commanders with more decision time to select the appropriate response, prevent fratricide, and allow any joint sensor to pair with the best available launcher. In addition, by integrating several individual sensors’ data, IBCS could compare and resolve conflicts within the individual systems’ abilities to accurately classify, identify, and discriminate potential threat objects to provide more accurate data back to the systems. IBCS could also help mitigate the risk of electronic attack since additional sensor data could help confirm where targets are when individual radars are being jammed or spoofed. In addition, because launchers would have access to additional sensor data, they could see more of the battlespace and use that information to more effectively engage threats. IBCS is intended to multiply the performance capabilities of the individual sensors and launchers connected to its network. Therefore, the capability of the networked architecture relies upon the ability of Patriot, as well as other air and missile defense systems, to connect with IBCS and provide the needed quality data for enhanced performance capabilities. Similar endeavors to create a system-of-systems architecture with an extensive communication and information network have proved challenging for DOD in the past. For example, prior work on the Army’s Future Combat Systems, a multibillion dollar development program originally consisting of 18 manned and unmanned systems tied together by an extensive communications and information network, faced rising costs and technical challenges that eventually led to its cancellation.

The Army Is Conducting an Analysis of Alternatives for a Patriot Radar and Launcher

In 2014, DOD provided guidance to the Army for conducting its LTAMD analysis of alternatives (AOA) to explore options for an efficient and cost- effective long-term radar and launcher solution—with considered alternatives ranging from the current Patriot assets with modifications up to total replacements—that will be able to connect with IBCS and address capability needs related to radar reliability, range, and 360-degree surveillance. The AOA results will support a decision for a new radar acquisition program, known as the LTAMD sensor, that will require a significant long-term financial investment. Issues with the Patriot radar have been raised in the past. For example, the Director of Operational Test and Evaluation identified performance and reliability issues with the current Patriot radar in its annual report since 2013. In addition, the Army conducted a business case analysis in 2013 and found that upgrades to the Patriot radar could result in operations and support savings, performance improvements, and reliability enhancements. An AOA is a key first step in the acquisition process, intended to assess alternative solutions for addressing a validated need. AOAs are generally performed or updated to support key acquisition decision points. During the course of our audit, an official in the Office of the Secretary of Defense for Cost Assessment and Program Evaluation (CAPE) stated that he expected the final LTAMD AOA report to receive approval in the third quarter of fiscal year 2016. As of August 2016, the report was still under independent review with the CAPE.

The Patriot Program Has Identified a Need for Training and Obsolescence and Sustainment Upgrades

To prepare the warfighter for the transition from the current, or legacy, Patriot system to IBCS-integrated Patriot radars and launchers, the Patriot program identified a need for training upgrades. Upgraded training aids and devices are necessary because transitioning to IBCS changes the way the warfighter employs the Patriot equipment.

The Patriot program has also identified a need to continue substantial investments to address obsolescence and sustainment issues. For example, the process of upgrading all of the legacy Patriot battalions to IBCS-integrated radars and launchers is an 8-year process that officials expect to begin in fiscal year 2017 and complete in fiscal year 2025. The legacy Patriot system components need ongoing obsolescence and sustainment improvements to improve reliability and availability, remain affordable, and be compatible with the different versions of operational Patriot battalions during that time. In addition, the program intends to continue obsolescence and sustainment investments to maintain readiness, improve reliability, and lower sustainment costs to support deployed forces with legacy radars until the legacy radar is fully replaced. Officials estimate that a new radar development could begin fielding in the fiscal year 2028 time frame with tactical fielding completing within 7 years. However, these plans are still preliminary and the milestone approval process is still underway. Lastly, obsolescence and sustainment improvements support legacy versions of Patriot systems, which foreign military partners continue to buy and operate. Patriots have been sold world-wide to 12 foreign military partners who share costs for sustainment and capability improvements in addition to investing in development to mitigate system obsolescence.

Current Version of Patriot Has Capability Improvements, Performance Shortfalls, and Does Not Yet Meet All Warfighter Needs

The currently fielded version of Patriot represents an improvement over prior versions through upgraded software, a more capable missile, and increased processor capabilities. However, the current version demonstrated a number of performance shortfalls against its documented requirements. In addition, warfighters from various combatant commands have expressed critical needs for additional performance capabilities and training equipment for the Patriot system that are currently unmet.

Current Patriot System Includes Capability Improvements over Prior Version but Operational Testing Revealed Some Performance Shortfalls

The current version of the Patriot system added performance capabilities through a software and processor upgrade in 2013 and an upgraded missile and launcher that began fielding in fiscal year 2016. In 2013, the Patriot program released its current system software upgrade known as Post Deployment Build-7 (PDB-7) that provided improvements in threat tracking, debris mitigation, and user interface. The software is supported by a new modern processor in the command and control station. This new processor provides Patriot with the ability to process more complex algorithms that improve the system’s capabilities against advanced threats. It also provides a platform for future capability improvements. Lastly, a launcher upgrade allows the system to launch and support use of the new Patriot Advanced Capability-3 (PAC-3) Missile Segment Enhancement (MSE) missile. The PAC-3 MSE, budgeted for and managed under a separate acquisition program, was fielded in the first quarter of fiscal year 2016 and is an upgrade to the predecessor PAC-3 missile by providing better lethality and a longer range—flying approximately 50 percent higher in altitude and 100 percent farther downrange.

While the system has made improvements, operational testing revealed that the system requires significant upgrades to the radar and software to bring the system up to the level of capabilities required. Operational testing is a field test of a system or item under realistic operational conditions with users who represent those expected to operate and maintain the system when it is fielded or deployed. The Army conducted a type of operational test called a limited user test in 2012 to evaluate the Patriot system with PDB-7 software, the modern command and control processor, and the PAC-3 MSE with the launcher upgrade against requirements defined in the program’s capability development and production documents. The Director of Operational Test and Evaluation’s (DOT&E) report on the results of the limited user test is classified, but it generally found that Patriot’s performance improved against some threats compared to prior versions but had degradations in system effectiveness against other threats. An unclassified summary of Patriot performance shortfalls, as identified by DOT&E and the Army, is shown in table 2. Some of the performance shortfalls can be attributed to the radar’s limited sensing abilities. While the PAC-3 MSE missile has an expanded battlespace over the PAC-3 missile, the radar is not able to sense and support the full range and capabilities of PAC-3 MSE. In addition, since experiencing fratricides during Operation Iraqi Freedom in 2003, the program has been working on upgrades to the system’s ability to more accurately classify, identify, and discriminate threat objects. While significant enhancements have been made since that time, the program requires additional capabilities to meet requirements. The risk of these performance shortfalls, left unaddressed, range from erroneous engagements and missile wastage to mission failure or fratricide.

In addition, DOT&E’s limited user test report found that the Patriot system as a whole did not meet the reliability requirement, but would have if the Patriot radar had achieved its reliability goal. The metric for determining reliability is an average of the number of hours between critical failures that place the system out of service and into a state of repair. Although the system is required to run at least 20 hours on average between critical failures, during the limited user test, the Patriot fire unit fell short by demonstrating an average time of around 11 hours. More than 70 percent of the critical mission failures during the test were experienced by the radar. Had the radar achieved its requirement of at least 38 hours, the fire unit would have exceeded the 20 hour requirement. Army officials attribute the radar reliability problems to a number of parts including obsolete technology, which require high levels of maintenance. Too frequent critical failures can create vulnerabilities for the system and defended assets when the equipment is taken offline for maintenance actions.

Current Patriot System Does Not Meet All Warfighter Needs

The warfighter has identified several capability needs for the Patriot system that are currently unmet. One of the ways that warfighters in various combatant commands express their capability needs is through memos known as operational needs statements. The warfighter has identified an operational need for capabilities to address many of the same air and missile defense capability gaps for performance and communications previously identified in table 1. While the shift to Army’s IBCS, planned for initial fielding in fiscal year 2018, is designed to address the capability need for joint integration below the battalion, the warfighter has requested this new capability be fielded sooner.

Warfighters have also identified a need for reconfigurable training assets and simulations for training in a variety of settings to operate and maintain the system. See table 3 for current operational needs statements.

Multibillion Dollar Upgrade Strategy Awaits Key Testing Results to Determine the Extent to Which Upgrades Address Capability Needs or Require Additional Development

To address a diverse set of capability needs to mitigate evolving threats, the Army is planning to field a number of upgrades, as well as a long-term radar solution, projected to cost $2.9 billion through fiscal year 2021 with additional costs needed for its long-term solutions. The program successfully completed developmental testing on near and mid-term upgrades in 2016. However, two operational test campaigns, consisting of multiple ground and flight tests, currently planned to begin in late fiscal year 2016 and 2019 should demonstrate how well the near and mid-term upgrades work as intended and identify any performance shortfalls that may require additional development.

Strategy to Address Patriot’s Capability Needs Is Projected to Cost $2.9 Billion through Fiscal Year 2021 with Additional Costs for Long-Term Solutions

The Army is fielding a number of upgrades in order to address divergent needs identified by the Army, the program office, independent test officials, and warfighters as discussed previously and summarized below in table 4.

The Army has budgeted $2.9 billion in three budget lines for development and procurement between fiscal years 2013 and 2021 for various upgrades and a long-term radar solution. Specifically, the Army is budgeting for three ongoing upgrades to address obsolescence issues, four near-term hardware upgrades that begin fielding prior to fiscal year 2017, six mid-term upgrades and supporting equipment that will begin fielding between fiscal years 2017 and 2021, and long-term upgrades— including a long-term radar solution, the details for which are still being determined. Costs are expected to continue beyond fiscal year 2021 to finish purchasing the necessary number of modifications already in production as well as to develop and procure long-term solutions required to address some of the capability needs. See Figure 3 for more details on how costs are allocated among the obsolescence, near-term, mid-term, and long-term upgrades.

Additional details on the upgrades including planned cost and schedule are included below.

Ongoing Upgrades to Address Obsolescence Issues

The Army has spent nearly $306.3 million since fiscal year 2013 and plans to spend an additional $361.5 million through fiscal year 2021 for various obsolescence upgrades that have been ongoing in the program for years and are planned to continue. These upgrades improve readiness and reduce future operation and sustainment costs for Patriot components. Additional details on these upgrades and the Patriot capability needs they plan to address are included in table 5.

Requests for funding for these three ongoing upgrades to address obsolescence issues are expected to continue beyond fiscal year 2021. See figure 4 for planned costs between fiscal years 2013 and 2021.

The Army has spent nearly $273.9 million since fiscal year 2013 and plans to spend an additional $553.7 million through fiscal year 2021 for near-term upgrades that begin fielding prior to fiscal year 2017 to address critical communication needs, ensure legacy components are sustainable, and address warfighter needs for system capability and training. For details on the near-term upgrades and the Patriot capability needs they plan to address, see table 6.

The fielding schedule for Patriot near-term upgrades is included in figure 5 along with the total planned costs from fiscal years 2013-2021. However, the program will need to request additional funds beyond fiscal year 2021 to complete the purchase of launcher upgrades. Fielding for some of the training software and hardware devices began prior to fiscal year 2013.

The Army has spent nearly $553.1 million since fiscal year 2013 and plans to spend an additional $437.3 million for mid-term upgrades and supporting test equipment that begin fielding between fiscal years 2017 and 2021. Among the mid-term upgrades is the remaining hardware needed—a radar digital processor—to prepare the system for integration with IBCS. Also key among these upgrades is a major software upgrade called Post Deployment Build-8 (PDB-8), which, in addition to a second software upgrade called PDB-8.1, is intended to improve communications and system capabilities against threats. Together, these mid-term upgrades, along with a test detachment, are intended to improve system performance, address warfighter needs, reduce obsolescence, and support Patriot testing needs. For details on the near-term upgrades and test detachment and the Patriot capability needs they plan to address, see table 7.

The fielding schedule and total planned costs for Patriot mid-term upgrades between fiscal years 2013 and 2021 are included in figure 6. Costs for PDB-8 and PDB-8.1 software-related tasks are estimated based on software-related tasks in the budget. Congress recommended reductions in requested development funding for software-related efforts by 50 percent or more each year between fiscal year 2013 and 2015, amounting to nearly $200 million in reductions. According to program officials, these reductions caused the program to delay some planned capabilities from PDB-8 until PDB-8.1. Officials explained that software capabilities currently planned for PDB 8.1 could be affected by available funding in any given year and may lead to deferring capability into future software upgrades. The program has already planned to continue software capability costs beyond fiscal year 2019 for future software improvements in the missile, launcher, or radar components following PDB-8.1.

Additional details on the status of the development and procurement of Patriot’s near and mid-term upgrades is included in appendix III.

Long-term Upgrades

The Army has spent around $8.5 million since fiscal year 2013 and plans to spend an additional $437.8 million between fiscal years 2017 and 2021 for long-term software and radar solutions to continue to address capability needs. Of the planned $437.8 million, the program has initially budgeted around $74 million in fiscal years 2020 and 2021 for future software improvements in the missile, launcher, or radar components beyond PDB-8.1, with plans to continue software investments beyond 2021. The remaining $364.1 million is planned through fiscal year 2021 as a portion of total expected costs for a long-term radar solution. These costs are part of a program funding line established in the 2017 president’s budget that the Army plans to manage as a new major defense acquisition program, known as the LTAMD sensor, beginning in fiscal year 2016. This long-term LTAMD sensor solution will be selected based on the findings in the ongoing LTAMD AOA that is being conducted as a result of concerns over the current Patriot radar’s high obsolescence and sustainment costs as well as issues with performance and reliability. For additional information on the AOA, see appendix II. There are many radar options being considered in the AOA, from the current Patriot radar with some modifications all the way up to a brand new radar development. Officials estimate that fielding for the selected radar solution could begin in the fiscal year 2028 time frame, with tactical fielding to be completed within 7 years. Depending on the Army’s selected radar solution, costs could increase and continue well beyond fiscal year 2021 for additional development as well as for procurement costs, which have not yet been determined. A breakdown of total planned costs from fiscal years 2013 to 2021 for long-term upgrades as well as a long-term radar solution is included in figure 7.

Patriot Program Successfully Completed Developmental Testing on Near and Mid-term Upgrades in Fiscal Year 2016

The Patriot program successfully completed developmental testing on the system configured with near and mid-term upgrades in addition to completing some limited developmental testing on the current PDB-7 version integrated with IBCS. Test and evaluation activities are an integral part of developing and producing weapon systems, as they provide knowledge of a system’s capabilities and limitations as it matures and is eventually delivered for use by the warfighter. Developmental testing, which is conducted by contractors, university and government labs, and various DOD organizations, is intended to provide feedback on the progress of a system’s design process and its combat capability as it advances toward initial production or deployment.

The Patriot program successfully completed developmental testing in fiscal year 2016 for the system configured with near and mid-term hardware upgrades. The Army Test and Evaluation Center conducted system-level developmental testing for Patriot configured with PDB-8 software in addition to other hardware upgrades, including modernized displays in the command and control stations, the PAC-3 MSE with the supporting launcher upgrades, and the radar digital processor. As part of this test, the program successfully conducted four flight tests. These flight tests demonstrated the system’s ability to intercept targets using a variety of Patriot missiles, including the PAC-3 MSE. The Army Test and Evaluation Command also performed testing on individual hardware upgrades with favorable results. For example, the command conducted some limited testing on the program’s new communication terminals and found that the upgrades generally work as intended. However, additional testing to evaluate the full functionality of the terminals is required prior to full material release.

The IAMD program conducted two developmental flight intercept tests in 2015 of the PDB-7 version of Patriot integrated with IBCS, which also met main objectives. During one of these tests, IBCS was able to command a Patriot launcher to launch a missile and destroy a target using tracking data from another Army system radar.

Two Operational Tests Will Determine How Well Near and Mid-Term Upgrades Address Patriot Capability Needs and Identify Any Performance Shortfalls That May Require Further Development

The program currently has two operational tests planned through 2020 that will test the system configured with upgraded software PDB-8 and PDB-8.1 as well as with assorted near-term and mid-term hardware upgrades as seen in table 8. Operational test and evaluation is intended to evaluate a system’s effectiveness and suitability under realistic combat conditions before full-rate production or deployment occurs. Operational testing for PDB-8 is planned to begin in the fourth quarter of fiscal year 2016 and complete in the fourth quarter of fiscal year 2017. Operational testing for PDB-8.1 is planned to begin in the fourth quarter of fiscal year 2019 and complete in the third quarter of fiscal year 2020. While developmental testing thus far has been successful, the results of operational test and evaluation will reveal the extent to which many of the upgrades work as intended to address some of Patriot’s diverse capability needs.

For example, operational testing for PDB-8 will evaluate how well the software and hardware upgrades address the previously identified performance shortfalls from PDB-7—including issues with the radar’s reliability. In addition, the test will also evaluate the effectiveness and efficiency of training aids and devices that are being procured to address warfighter needs. Operational testing for PDB-8.1 is planned to evaluate how well PDB-8.1 software capability upgrades effectively address remaining system performance needs. According to Army Test and Evaluation Command officials, upgrades that have not yet begun production, like the global positioning anti-jamming hardware upgrade and the radar anti-jamming upgrade, have not yet been incorporated into testing plans.

However, near- and mid-term upgrades aren’t expected to fully address all of the Patriot capability needs, which will require long-term upgrade solutions. For example, the program plans for its near and mid-term upgrades to provide significant enhancements to radar reliability and sensing range to support the PAC-3 MSE missile’s mission against stressing threats, but does not expect them to fully address the performance needs without the long-term radar solution. In addition, currently planned software upgrades are intended to provide capabilities to help address tactical ballistic missile threats and electronic attacks, but additional long-term software—and potential additional hardware— investments are needed to continue improving capabilities against the evolving threat, which continues to create new gaps in the system’s capabilities.

Operational testing results could identify unexpected performance shortfalls in the near and mid-term upgrades that require additional development. In the case of PDB-7, for example, operational test results identified unexpected performance shortfalls in system reliability that required additional development in the latest near and mid-term upgrades to address. Operational testing for PDB-8 or PDB-8.1 could also identify unexpected performance shortfalls that require additional development to insert capabilities into future software or hardware upgrades for Patriot components.

The Army Lacks an Oversight Mechanism to Track Progress and Ensure Accountability of Near and Mid-term Patriot Upgrades If Additional Development Is Needed

Oversight of Patriot upgrades has been limited because of how the Army chose to define and manage them, including not establishing oversight mechanisms similar to those generally applicable to major defense acquisition programs. The Army chose to incorporate the Patriot upgrade efforts into the existing Patriot program which made certain oversight mechanisms inapplicable. While it would not be productive for the program to go back and establish these mechanisms from development start, upcoming operational tests provide the Army with an opportunity to provide oversight and ensure accountability for the cost, schedule, and performance of near- and mid-term upgrades, tested along with PDB-8 and PDB-8.1, if further development is needed.

Congressional Oversight of Near and Mid-term Upgrades Has Been Limited By DOD’s Decision Not to Track and Report Cost, Schedule, or Performance Progress

Up to this point, the Patriot program has not put a mechanism in place to track or report progress against cost, schedule, or performance baselines of its upgrade efforts, similar to those generally required of multibillion dollar DOD acquisition programs. Under DOD instruction 5000.02 and related statutes, major defense acquisition programs (MDAPs) are subject to a number of oversight mechanisms that provide transparency into program plans and progress. Although the Army’s 2013 cost estimate for all the Patriot upgrades met the threshold to be considered a separate MDAP, the Army chose not to define the upgrade efforts as such. Instead, the upgrades were incorporated into the existing Patriot program, which resulted in the upgrade efforts not being separately subject to statutory and regulatory reporting requirements that generally apply to MDAPs. In addition, the program did not establish any oversight mechanisms for the upgrades that were similar to those generally required of MDAPs.

For example, new MDAPs are generally required to establish an approved program baseline that includes initial estimates for key cost, schedule, and performance metrics at the beginning of system development, at the start of production, and before the start of full rate production. Information about these baselines is reported to Congress in a standardized format through Selected Acquisition Reports. On a periodic basis, programs update the information in these reports by comparing the latest cost, schedule, and performance estimates against the initial estimates and providing explanations for any major deviations. Establishing reliable cost and schedule estimates are best practices that we have found go hand-in-hand as fundamental management tools that can help all government programs use public funds effectively. Further, as we demonstrate each year in special annual reports assessing DOD’s acquisition of selected weapon programs, and in related testimonies before congressional committees, regular comparisons of program cost, schedule, and performance against initial estimates is an essential oversight tool. Such data, when maintained and reported on a regular basis, help the decisionmakers who oversee program progress understand the significance of any increases or decreases in cost or schedule as a program evolves, provide transparency, and give Congress and the Office of the Secretary of Defense a mechanism to hold the program accountable for its intended results. As we reported in our March 2016 assessment, programs that do not uniformly implement these and other best practices tend to realize significant cost growth and delays in delivering needed capabilities.

Army officials explained that the existing Patriot program’s 2002 acquisition strategy provided approval for the Army to execute Patriot upgrades as part of this program, which was defined as an MDAP, and the Office of the Secretary of Defense had no objection. However, the requirement for MDAPs to continue reporting Selected Acquisition Reports ceases after 90 percent of the program’s items are delivered or 90 percent of planned expenditures under the program have been made. The Patriot program submitted its final Selected Acquisition Report in 2004 when the program was considered more than 90 percent complete. Absent the requirement to do so, the program has not provided decisionmakers with similar information. As a result, there has been no mechanism for DOD and congressional decisionmakers to monitor performance of the approximately $1 billion spent on Patriot upgrades since 2013 and to ensure that efforts have resulted in progress toward meeting the program’s goals.

Upcoming Tests of PDB-8 and PDB-8.1 Provide Opportunity to Increase Oversight of Near and Mid-term Upgrades If Further Development Is Needed

While it would not be productive for DOD to go back and track cost or schedule changes from the start of the Patriot upgrade efforts (see appendix III), in the event that upcoming operational tests reveal the need for further development of PDB-8 and PDB-8.1 and other near- and mid- term upgrades tested along with that software, the department will have an opportunity to provide increased oversight of those upgrades. As noted above, DOD already plans to define the long-term LTAMD sensor solution as a separate MDAP, which indicates the program would be subject to the oversight requirements applicable to MDAPs, such as those discussed above.

Without estimated costs and schedule needed to complete the development of upgrades for essential Patriot capabilities, similar to those generally required of new major defense acquisition programs, DOD and congressional decisionmakers will lack an essential oversight tool. In addition, unless, at the same time, DOD provides Congress with an estimate of the amount of development costs it has incurred since 2013 for near- and mid-term Patriot upgrades operationally tested along with PDB-8 and PDB-8.1, Congress will not have a basis from which to understand the significance of any increases or decreases as the program evolves. Finally, without annual reporting mechanisms that enable comparisons between subsequent cost and schedule estimates and initial estimates, along with periodic explanations for any major cost or schedule deviations, Congress will lack critical information it needs to evaluate future program budget requests.

The Army’s Plan to Complete Patriot Modernization Efforts Synchronizes Fielding with Training and Brings Benefits and Challenges That the Army Continues to Manage

The Army selected a plan to synchronize its fielding of upgraded versions of the Patriot system during its transition to the Integrated Air and Missile Defense Battle Command System (IBCS) that allows it to meet operational demands. Integrating Patriot battalions with IBCS can provide organizational and personnel flexibility in the future. However, the process of fielding these upgrades over the course of the 8-year transition to IBCS amplifies some of the challenges the Army is already facing with training complexity and maintenance schedules for the Patriot system. The Army is taking steps to mitigate these challenges.

The Army’s Plan for Fielding Modernized Patriots to Combatant Commands Synchronizes with Testing and Training Requirements to Meet High Operational Demands

The Army has a plan for fielding modernized Patriots to Combatant Commands. The process of modernizing a Patriot battalion—transitioning it from its current PDB-7 software version into launchers and radars integrated with IBCS, involves two phases. The first phase requires the battalion to be upgraded to the PDB-8 software version. Once the battalion receives PDB-8, it is ready for phase 2, which consists of a second software update to integrate the system components with IBCS. In some cases, a battalion can undergo both modernization phases consecutively, but, in other cases, a battalion can complete phase 1 and then wait a number of years to complete phase 2. The fielding plan the Army selected completes phase 2 of integrating the battalion into IBCS at a rate of approximately two Patriot battalions per year. By fiscal year 2022 the Army plans to have completed phase 1 for all 15 battalions with 9 battalions completing phase 2 and being IBCS compatible. IBCS integration continues through fiscal year 2025, as seen in figure 8.

To synchronize fielding with testing, the Army removed a Patriot battalion from the operational deployment rotation and assigned it solely to modernization testing. Army officials told us this is a key enabler of the fielding strategy—without it the plan becomes unworkable. Specifically, the amount of time required to begin and complete IBCS integration testing exceeds the amount of time that any one Patriot battalion is available to perform that testing. Therefore, the Army would have to start with one battalion and complete the testing with a second battalion— which would add an extra 6 to 9 months to train the second battalion on how to use the new equipment. After completing the United States / North Atlantic Treaty Organization mission in Turkey, the Army was able to adjust its Patriot unit rotation schedule which enabled the Army to assign a battalion to support Patriot modernization testing. The battalion’s test assignment began in April 2016 and the Army plans to keep the battalion solely for testing into fiscal year 2018. Army officials also told us that the Vice Chief of Staff for the Army recently approved increased funding for the Army Air and Missile Defense test detachment to increase the manning from 35 to over 140. Increasing the size of the detachment will allow the Patriot test battalion to rejoin the operational rotation in fiscal year 2018, providing the combatant commands with more available Patriot battalions.

The Army considered four alternative plans for how and when to field these 2 phases of modernization to the 15 Patriot battalions. The baseline plan would have upgraded three battalions per year to PDB-8 and one per year to IBCS. Another plan would have upgraded two or three battalions per year to PDB-8 and two per year to IBCS, while focusing on upgrading units in Europe first. A third plan would have upgraded three battalions per year to PDB-8 and two to IBCS, and would have upgraded units in the Pacific first. A fourth alternative, which the Army selected, completes phase 1 of the upgrades for the nine Patriot battalions that are not being upgraded directly to IBCS compatibility by fiscal year 2022 and completes phase 2 of the upgrades to make all 15 Patriot battalions IBCS compatible by 2025.

The Army prioritized meeting training requirements and operational demands when selecting its plan for completing Patriot modernization efforts. The Army used five criteria to evaluate the four alternative plans. The Army’s evaluation criteria included maximizing the number of Patriot battalions available at any given time to support operations, maintaining the same software version for all Patriot battalions under a particular brigade to make training consistent, and meeting these and other competing needs within funding constraints. Table 9 below provides a description of the criteria, the weighting the Army assigned to it, and how well the plan the Army selected optimized the criteria. Based on the Army’s analysis, the selected plan did the best job of balancing all of the key considerations reflected in these criteria.

Patriot’s Inclusion in IBCS Provides Operational and Personnel Benefits

Army officials told us that moving Patriot to IBCS provides benefits in meeting Combatant Command operational needs more flexibly because the system can be reorganized so that it no longer has to be deployed as a complete battalion. IBCS-compatible Patriot components can be deployed as individual radars and launchers, networked through IBCS. Army officials told us that instead of having 15 Patriot battalions, the Army will have 60 fire units’ worth of radars and launchers that can be deployed more flexibly to meet combatant command operational demands.

Transitioning Patriot to IBCS compatibility can potentially lead to organizational changes that reduce the number of personnel required to operate and maintain the radars and launchers. The Army plans to use this streamlined organizational structure as an opportunity to create a more even distribution of tasks. As part of its findings during the PDB-7 Limited User Test, DOT&E reported that Patriot personnel currently performing the job of operator/maintainers are required to perform many complex tasks, resulting in poor operator performance. Army officials told us they expect to realign the current number of personnel specialties within Patriot from nine specialties down to four. In addition, these specialties will no longer be Patriot specific—rather they will cut across the integrated air and missile defense community, allowing the Army to address some challenges with the relatively low number of personnel in some specialties. Army officials told us that the realignment would also allow the Army to alter the skillset of personnel who are currently operators/maintainers of the equipment into purely operators, while maintainers would take on some additional responsibilities. Further, by 2025 the Army plans for current Patriot operators and maintainers to maintain and operate a variety of Army air and missile defense systems, as opposed to being assigned solely to Patriot.

Patriot’s Inclusion in IBCS Amplifies Challenges with Training Complexity and Maintenance Schedules, but the Army Has Mitigation Plans in Place

Migrating Patriot to IBCS amplifies training challenges by adding new training into the Army’s Patriot training schedule. Further, for a period of time the Army will be training personnel on three different versions of Patriot—PDB-7, PDB-8, and IBCS. Army officials told us that due to the high deployment frequency of the Patriot force, the current training schedule does not completely prepare Patriot operators on all tasks before deployments. To address this, the Army revised the training certification progression so that high priority training is completed before deployment, and less important training can occur after deployment. However, to prepare for the transition to IBCS, the warfighter requires additional training on how to effectively operate the equipment under an airspace complicated with data from multiple sensors. This increasingly complex training required by Patriot operators could cause further issues with the Patriot training schedule in the future. Over the long term, officials told us that the Army plans to address some of these challenges by updating the training certification program to match up with the changes to the Patriot system (for instance more emphasis on joint operations) and by adding more advanced certification levels that would include skills not currently included as part of the certification process. While Army officials told us they are in the initial stages of implementing changes to the training program, they expect it to be implemented by 2025 when the Army completes the transition of all Patriot units to IBCS.

The modernization fielding plan the Army is pursuing also poses a near- to mid-term maintenance challenge. The Army currently plans to perform comprehensive maintenance on only one Patriot battalion per year through fiscal year 2021 in order for battalions to be available for modernization, training, and operations. However, Army officials told us they will not be able to complete maintenance on all 15 Patriot battalions within the expected 10-year life cycle at that rate. As a result, officials confirmed that the Army is assuming an elevated risk of equipment breakdown. To mitigate this challenge in the short term, the Army is performing less comprehensive maintenance after every deployment and maintaining a sizable inventory of spares for those parts that have high failure rates. As more Patriot battalions become IBCS-compatible, the Army is considering ways to schedule comprehensive maintenance on more than one battalion per year. However, the officials were unsure if they would be able to have two battalions worth of equipment offline for maintenance and still have enough availability to meet training and operational demands.

The Army Holds Regular Synchronization Meetings to Manage Challenges It Encounters during Patriot’s Transition to IBCS

The Army regularly coordinates on the status of doctrine, organization, training, materiel, leadership, personnel, and facilities implications of Patriots transition to IBCS through the use of quarterly transformation summits. These summits are internal meetings that include decisionmakers from all of the key domains within the Army that need to synchronize on integrated air and missile defense issues, including training, doctrine, leader development, and facilities. Briefings from these summits show that the Army officials discuss modernization and maintenance schedules, training strategy, and facility needs, among other topics at these summits. Army officials told us that as a result of these meetings, the Army decided to alter the Patriot deployment duration from 12 months to 9 months, concluding that this change would have a minimal impact on the modernization and training schedules, while providing the same operational support to combatant commands. In implementing the deployment duration change the Army will keep five battalions over the next 5 years on the 12-month deployment schedule, while all other Patriot deployments will last for 9 months. Army officials said that this fluctuation was necessary in order to allow enough time for other Patriot battalion modernization, testing, and training to occur— information they were aware of because of the summit discussions.

Conclusions

As a cornerstone of the Army’s air and missile defense architecture, the Patriot system is deployed worldwide in defense of the United States and its allies. The program faces multiple challenges to overcome the obsolescence of a system that has been fielded for decades, improve capabilities to address ever-evolving threats, and complete its transition from a stand-alone system to an integrated component of the Army’s Integrated Air and Missile Defense. The Army has spent approximately $1.1 billion since 2013 to develop and procure Patriot upgrades and has requested another $1.8 billion, which includes funding for a long-term radar solution, over the next five years. A modernization program of this magnitude and complexity demands high-level oversight to ensure that the upgrades are completed on time, within planned cost, and that they provide the intended capabilities. In the long term, the Patriot system will no longer be Patriot as we know it but will be broken down into its major components—a radar, launcher, and a missile—integrated with Army’s Integrated Air and Missile Defense System of Systems. Of the three remaining components, the Army has already defined the missile as a separate major defense acquisition program and currently plans to do the same for the LTAMD sensor solution, which accounts for $364 million of the requested $1.8 billion over the next five years. Continuing to separately manage and track progress for these components should help provide Congress with the oversight and accountability it needs to make important investment decisions. Although the Army estimated in 2013 that costs for Patriot upgrades would meet the threshold to be considered a major defense acquisition program (MDAP), the Army chose to incorporate the Patriot upgrade efforts into the existing Patriot program which made certain oversight mechanisms inapplicable. The Army would have put itself in a much better position to oversee its Patriot upgrade efforts had it made the decision in 2013 to manage Patriot upgrades as a separate major defense acquisition program. Should operational testing for PDB-8 and PDB-8.1 reveal performance shortfalls in the near and mid-term upgrades tested, the additional development required could present an opportunity for DOD to provide a level of oversight and accountability not seen by the Patriot upgrade efforts so far. Beginning any additional development with cost, schedule, and performance estimates—informed by an estimate of the amount of development costs the upgrade effort has incurred since 2013—would provide DOD and congressional decisionmakers an essential oversight tool, particularly when considering future budget requests. Further, regular comparisons of program cost, schedule, and performance against initial estimates enhance decisionmakers’ understanding of the significance of any increases or decreases in cost or schedule as a program evolves.

Recommendations for Executive Action

In the event that operational test results for PDB-8 and PDB-8.1 reveal performance shortfalls that require additional development of the near and mid-term upgrades tested, we recommend that the Secretary of Defense direct the Secretary of the Army to establish mechanisms for overseeing those upgrades commensurate with other major defense acquisition programs, to include: 1. An initial report—similar to a Selected Acquisition Report—as soon as practical following operational testing for both PDB-8 and PDB-8.1, on the near and mid-term upgrades evaluated during these tests, including: cost, schedule, and performance estimates for any additional development that is needed; and an estimate of the amount of development costs it has incurred since 2013 for near- and mid-term Patriot upgrades operationally tested along with PDB-8 and PDB-8.1. 2. Annual updates to Congress comparing the latest cost and schedule estimates against the initial estimates and providing explanations for any major deviations until development is complete.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD for comment. DOD provided us with written comments which are reprinted in appendix IV. DOD also provided technical comments, which were incorporated as appropriate.

DOD partially concurred with our recommendations to provide an initial report—similar to a Selected Acquisition Report—and to provide annual updates to Congress in an effort to establish oversight mechanisms commensurate with other major defense acquisition programs for upgrades operationally tested with PDB-8 and PDB-8.1 in the event that operational test results reveal performance shortfalls that require additional development. In its response, DOD stated that system software updates currently being performed for Patriot, such as PDB-8 and PDB- 8.1, will cease with updates transitioning to IBCS. It also noted that future post deployment build updates will be developed and tested for IBCS as part of the Army’s IAMD program, which is subject to acquisition oversight and reporting required by law and regulation. Further, DOD noted that future development and testing of the LTAMD sensor will also be subject to acquisition oversight and reporting required by law and regulation. DOD stated that using existing oversight and reporting mechanisms for these major defense acquisition programs would more accurately reflect the development program and is more appropriate than introducing additional non-standard reports.

DOD’s response focuses on tracking and reporting progress on other MDAPs without clarifying how or if it will track progress on current PDB-8 and PDB-8.1 efforts. The IAMD program has already established its planned content in a baseline, and details for the LTAMD sensor program are still being determined. Regardless, tracking and reporting progress on the pre-existing IAMD program or future development LTAMD sensor program will not provide Congress with oversight and accountability on the outcomes for current work on the near- and mid-term upgrades tested with PDB-8 and PDB-8.1. As such, we maintain our position that the Secretary of Defense should take the recommended actions to direct the Army to establish mechanisms for overseeing any additional work on those upgrades commensurate with other major defense acquisition programs, by providing an initial report that is similar to a Selection Acquisition Report and annual updates to Congress that compare the latest cost and schedule estimates against the initial estimates for PDB-8 and PDB-8.1 upgrades.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, and the Secretary of the Army. The report is also available at no charge on the GAO website at http://www.gao.gov.

Should you or your staff have any questions about this report, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Scope and Methodology

To determine the current status of the Patriot system’s performance and the extent to which it addresses warfighter needs, we did the following: 1. To determine the current status of the Patriot system’s performance, we reviewed briefings from the Lower Tier Project Office in Huntsville, AL and from the Capabilities Development and Integration Directorate at Fort Sill in Lawton, OK on the current system’s performance specifications. To determine the extent to which the current version is meeting its performance requirements, we reviewed 2013 limited user test results from the Director, Operational Test and Evaluation (DOT&E) to see how well the Patriot system performed against its performance parameters as defined in the capabilities development and production requirements documents. In addition, we obtained the Patriot’s Post Development Build-7 (PBD-7) conditional material release “get well” plans, which outline the performance shortfalls of PDB-7 that need to be mitigated. We also discussed these shortfalls with officials from DOT&E in Arlington, VA; the Army Test and Evaluation Command at Fort Bliss in El Paso, TX who conducted the PDB-7 limited user test; the Lower Tier Project Office; and the Capabilities Development and Integration Directorate. 2. To determine the extent to which the current version of the Patriot system is meeting warfighter needs to address the growing threat, we reviewed warfighter operational needs statements, which document requests from the warfighter to the Army for urgent, real-time Patriot capabilities and other needed upgrades. We assessed the reliability of the currently open Patriot-related operational needs statements from 2013 by comparing the list of operational needs statements obtained from the Capabilities Development and Integration Directorate to those received from the Capabilities Integration Division of the Department of the Army Military Operations in Arlington, VA. Based on our review of the data and interviews with officials at both locations, we determined that the data were sufficiently reliable for the purposes of our reporting objectives. We also held discussions with these officials about the unfulfilled operational needs statements and the Army’s plan for addressing them. In addition, we interviewed combatant command officials from the Pacific Command in Honolulu, HI; the European Command in Stuttgart, Germany; and the Central Command in Tampa, FL to obtain views on Patriot performance needs from various combatant commands.

To assess the extent to which the Patriot system upgrades will address capability needs and describe the cost, schedule, and testing plans associated with those upgrades we did the following: 1. To determine the various Patriot capability needs, we began by reviewing the validated air and missile defense capability gaps, which the program used as a foundation for its 2013 requirements documents. We examined these Patriot-related gaps listed in the 2011 Army Functional Concept for Fires Capability-Based Need Assessment Functional Needs Analysis and Functional Solution Analysis reports. Based on our analysis of these documents and additional Army briefings and plans, we identified a selection of high- priority critical air and missile defense gaps that were related to the Patriot program. We also reviewed requirements in the Patriot Increment 3 Capability Development Document related to training and obsolescence and sustainment and discussed these requirements with Army officials at the Lower Tier Project Office and the Air Defense Artillery School at Fort Sill in Lawton, OK. 2. To understand the evolving threat and how it is driving capability needs for the Patriot system, we reviewed the 2011 and 2015 System Threat Assessment Reports and discussed the Patriot-related threat assessment findings with officials from the Missile and Space Intelligence Center in Huntsville, AL and the Capabilities Development and Integration Directorate. 3. To describe the cost, schedule, and testing plans for the Patriot upgrades, we obtained and analyzed detailed cost data derived from program budgets, program schedules for testing and fielding, and test and evaluation master plans. We discussed these plans with officials from DOT&E; the Capabilities Development and Integration Directorate; and the Lower Tier Project Office. We focused our cost review on two Patriot program budget lines, which detail the U.S. contribution to development and procurement costs for planned upgrades, and a third budget line providing initial development funding for the Lower Tier Air and Missile Defense (LTAMD) sensor solution. Planned costs for fiscal years 2017 through 2021 are based on detailed Army planning budget data supporting the President’s budget for fiscal year 2017. We deflated these budget numbers to base year 2017 dollars. 4. To determine the extent to which planned upgrades will address capability needs, we obtained detailed information from the Capabilities Development and Integration Directorate officials mapping each of the planned upgrades to the capability need it is intended to help address. We also obtained and reviewed the schedule and scope of planned operational testing in the System Evaluation Plan to determine when the upgrades would be evaluated. Further, we reviewed the scope of the analysis of alternatives currently underway to determine what capability needs the radar and launcher alternatives being considered are intended to address and discussed these needs with Army officials from the Capabilities Development and Integration Directorate and the Lower Tier Project Office.

To determine the level of oversight and accountability provided for the upgrades, we received information regarding how and why the upgrades were executed under the existing Patriot program from Army officials. We reviewed prior legislation and related reports since 2012 to understand Congress’s concerns on oversight and accountability for the latest Patriot upgrades. We then reviewed DOD guidance documents and briefings to determine the level of oversight planned for the long-term radar solution. We also reviewed DOD acquisition regulations and related statutes to determine the typical requirements for facilitating Congressional oversight and accountability of major defense acquisition programs.

To assess the extent to which the Army’s plan for fielding modernized Patriots synchronizes with training schedules and operational demands, we analyzed the Army’s fielding plan as well as operational and training schedules. We also interviewed knowledgeable Army officials to discuss how the fielding plan was chosen, the benefits and challenges associated with the chosen plan, as well as any effects of the plan on operations, personnel, doctrine, organization, testing, and training.

To assess the extent to which DOD’s guidance for conducting its LTAMD analysis of alternatives (AOA) meets GAO best practices, we obtained Department of Defense AOA guidance documents. These documents consist of a directive from the Army Headquarters directing the Army Training and Doctrine Command Analysis Center to conduct the LTAMD AOA study, a study plan developed by the Army Training and Doctrine Command Analysis Center, and guidance from the Office of the Secretary of Defense for Cost Assessment and Program Evaluation (CAPE). We compared the processes outlined in the guidance documents to the 22 best practices GAO identified in GAO-16-22. We also met with officials from CAPE to discuss GAO best practice processes that were not documented in the guidance documents and supplemented our analysis with some of this information. We used a five-point scoring system to evaluate how well the LTAMD AOA guidance documents conformed to each of the 22 best practices. We then used the average of the scores for the best practices under each of the four characteristics—well- documented, comprehensive, unbiased, and credible—to determine an overall score for each characteristic. The results of GAO’s analysis underwent four separate levels of internal review to ensure accuracy as well as cross-checking the scores throughout the analysis for consistency. In addition, we provided the initial results of our analysis to officials in the CAPE and Army Training and Doctrine Command Analysis Center for review and received technical comments, which we incorporated, as appropriate, into our final analysis. To characterize our final results, if the average score for each characteristic was “met” or “substantially met,” we concluded that the AOA process conformed to best practices and could therefore be considered reliable.

We conducted this performance audit from June 2015 to August 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: The Lower Tier Air and Missile Defense Analysis of Alternatives Guidance Compared to GAO’s Best Practices for an Analysis of Alternatives Process

As part of our review of the Patriot system, we assessed the extent to which the Department of Defense’s (DOD) guidance for conducting its Lower Tier Air and Missile Defense (LTAMD) analysis of alternatives (AOA), which is evaluating material modernization solutions for the current Patriot radar and launcher for use with the Integrated Air and Missile Defense (IAM) Battle Command System (IBCS), meets GAO best practices and found that the guidance documents substantially met GAO standards to be considered reliable. We compared the processes outlined in the LTAMD AOA guidance documents to GAO best practices because the LTAMD AOA report was not available at the time of our review.

The LTAMD AOA guidance documents provide the AOA study team with a high-level roadmap for how to conduct the LTAMD AOA by outlining processes to identify and select the alternatives, metrics, models, and scenarios for use throughout the AOA process. While we cannot make conclusions about the final AOA report until it is finalized and released, by comparing the processes described in the LTAMD AOA guidance documents to the 22 GAO best practices, we can make conclusions on the quality of the processes used to develop it. If the processes are of high quality, then the AOA study team has a good roadmap, which, if followed, could produce a high-quality, reliable AOA. Based on our analysis, the LTAMD AOA process described in its guidance met or substantially met the criteria to be considered well-documented, comprehensive, unbiased, and credible.

While we found that the LTAMD AOA guidance documents met or substantially met 18 of the 22 best practices GAO established for the AOA process to be considered reliable, our review also found that contrary to GAO best practices, the final AOA report will not select a preferred solution. Specifically, the LTAMD AOA guidance did not instruct the study team to assign relative importance to the criteria that are used to compare the options or to select a preferred solution for a modernized radar and launcher as part of the final AOA report. According to CAPE officials involved in the LTAMD AOA efforts, the purpose of this AOA is to provide an analytic comparison of the options based on the criteria but to then allow external decisionmakers to determine the relative importance of each criterion and derive their own preferred solution. CAPE’s position is that GAO’s best practice of assigning relative importance to criteria is not appropriate for strategic investment decisions such as this. In contrast, GAO best practices recommend that solutions be compared based on pre-established criteria that reflect the relative importance of the criteria because not reflecting its relative importance up front can oversimply results and potentially mask important information leading to an uninformed decision. In addition, GAO best practices state that a preferred alternative should be identified and a rationale for that decision be included as part of an AOA report. While a recommended solution in the AOA report does not have to be binding, without one, decisionmakers outside of the AOA process may misinterpret the analysis within the AOA report and potentially come to a biased decision.

GAO’s 22 Best Practices for the AOA Process: Background and Introduction

In October 2015, GAO identified 22 best practices to provide a framework for conducting an AOA and help ensure that entities consistently and reliably select a preferred solution that best meets mission needs. To identify a high-quality, reliable AOA process, GAO grouped the 22 best practices under four characteristics. These characteristics evaluate whether the AOA process is well-documented, comprehensive, unbiased, and credible. “Well-documented” means that the AOA process is thoroughly described in a single document, including all source data, has clearly detailed methodologies, calculations and results, and that selection criteria are explained. “Comprehensive” means that the AOA process ensures that the mission need is defined in a way to allow for a robust set of alternatives, that no alternatives are omitted, and that each alternative is examined thoroughly for the project’s entire life-cycle. “Unbiased” means that the AOA process does not have a predisposition toward one alternative or another; it is based on traceable and verifiable information. “Credible” means that the AOA process thoroughly discusses the limitations of the analyses resulting from the uncertainty that surrounds both the data and the assumptions for each alternative.

Table 10 provides an explanation of how individual best practices are grouped under each characteristic.

LTAMD AOA Guidance Results

Overall, the DOD’s LTAMD AOA guidance documents met or substantially met the four characteristics of a high-quality and reliable AOA process. To make this determination, we reviewed and scored how well the guidance documents addressed each of the 22 best practices. We scored the 22 best practices using a five-point system as follows: “met” means the LTAMD AOA guidance documentation demonstrated that it completely met the best practice; “substantially met” means that it met a large portion of the best practice; “partially met” means that it met about half of the best practice; “minimally met” means that it met a small portion of the best practice; and “did not meet” means that it did not meet the best practice. We found that the LTAMD AOA guidance documents met or substantially met 18 of the 22 best practices. We then took the average of those best practice scores that aligned with each of the four characteristics, as shown above in Table 9, to derive a final score for each characteristic. Table 11 provides the average score of the best practices under each characteristic.

Appendix III: Status of Development and Procurement for Near and Mid-Term Upgrades

The Patriot program has made notable progress in the development and procurement of near and mid-term upgrades since the upgrade efforts began in 2013. Up to this point, significant costs for development and procurement have already been incurred, costs and activities are winding down, and the program plans to release the first of two major hardware and software upgrades next year. In sum, the Army has spent about $1.1 billion of the $2.9 billion planned between fiscal years 2013 and 2021 to address Patriot capability needs, as seen in figure 9.

Of the $1.8 billion currently planned between fiscal years 2017 and 2021, $645 million is for development. Of those development funds, the majority, $364 million, is allotted to developing the future radar solution, the Lower Tier Air and Missile Defense (LTAMD) sensor, which the Army currently plans to define as a separate major defense acquisition program (MDAP). Further, of the $645 million in development, only about $280 million is currently planned between fiscal years 2017 and 2021 for developing software and hardware upgrades. The program has already spent about $210 million for the development of near and mid-term software and hardware upgrades between fiscal years 2013 and 2016.

Aside from the future radar development, there does not appear to be a new wave of development activities beginning in the near future. Funding for PDB-8 was already completed in fiscal year 2016 with fielding planned for fiscal year 2017. Further, as seen in figure 10, costs planned for software development appear to be tapering off toward the end of the Future Years’ Defense Program in fiscal year 2021 when the program expects to release PDB-8.1.

Near-term and mid-term upgrade procurement activities also appear to be winding down. Most of the defined hardware upgrades are already in production. Further, many of these upgrades were already mature with relatively little being spent on hardware development for the purposes of adapting them for Patriot or maximizing their benefit to the system.

Although the program is still planning to spend $1.15 billion in procurement between fiscal years 2017 and 2021, which includes ongoing upgrades to address obsolescence issues, six of the nine near- term and mid-term hardware upgrades and supporting equipment have begun production, as seen in figure 11.

Lastly, costs planned for procurement upgrades appear to be tapering-off toward the end of the Future Years’ Defense Program in fiscal year 2021, as seen in figure 12. Currently, funds planned to continue beyond fiscal year 2021 are for ongoing upgrades to address obsolescence issues, for completing the purchase of launcher modifications, and for continuing investments in training upgrades.

Appendix IV: Comments from the Department of Defense

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, LaTonya D. Miller, Assistant Director; Kevin L. O’Neill; James P. Haynes II; Meredith Allen Kimmett; Randy F. Neice; Jenny Shinn; David L. Richards; Jennifer V. Leotta; Karen A. Richey; Alyssa B. Weir; Katherine Shea Lenane; Stephanie M. Gustafson; Oziel A. Trevino; and Joseph W. Kirschbaum made key contributions to this report.