Defense Acquisitions: Analysis of Processes Used to Evaluate Active Protection Systems
Published: 2007-06-08T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Active Protection Systems (APS) protect vehicles from attack by detecting and intercepting missiles or munitions. In 2005, the lead systems integrator for the Army's Future Combat Systems (FCS) program sought proposals for an APS developer and design and to deliver APS prototypes on vehicles by fiscal year 2009. Raytheon was chosen the APS developer. At the same time, the Department of Defense's Office of Force Transformation (OFT) evaluated near-term APS for potential use in Iraq. GAO was asked to review the Army's actions on APS/FCS: (1) the process for selecting the subcontractor to develop an APS for FCS and if potential conflicts of interest were avoided; (2) the timing of the trade study and if it followed a consistent methodology to evaluate alternatives, and the results; (3) the role the Army and Boeing played in selecting the developer; and (4) the process followed to provide a near-term APS solution for current forces.
section_title: What GAO Found
paragraphs: In selecting the APS developer, the Army and Boeing--the FCS lead systems integrator--followed the provisions of the FCS lead systems integrator contract, as well as the Federal Acquisition Regulation, in addressing organizational conflicts of interest. No officials from the offering companies participated in the evaluation and all offerors were evaluated based on the same criteria. Four proposals were evaluated and three were determined to be comparable in terms of cost and schedule. The winner--Raytheon--was chosen on technical merit, as being more likely to meet APS requirements although its design had less mature technology. The APS development contract required the source selection winner to perform a trade study to assess alternatives and select the best design for development, and the Raytheon design was chosen. The trade study applied a consistent methodology to all alternatives before selecting Raytheon's vertical launch design. While the role played by Raytheon in the trade study was in accordance with its contract, the rationale for having the trade study follow the source selection is not entirely clear. The purpose of the trade study was to select the best concept; yet the source selection process that preceded it had, in fact, chosen Raytheon primarily on the technical merits of its vertical launch design concept. Although the vertical launch technology is not mature, the Army estimated that it could be available for prototype delivery to current force vehicles in fiscal year 2009 and tested on a FCS vehicle in 2011. This may be an optimistic estimate, as the FCS vehicle is yet to be fully developed. The Army and Boeing were extensively involved in APS source selection and the trade study. FCS officials actively participated and concurred in the final selection of the APS developer. FCS officials and technical experts from Army research centers took part in the trade study and helped choose the vertical launch design. Boeing officials took part in various ways and, with the Army's concurrence, selected Raytheon as the APS developer, participated in the trade study, and recommended the vertical launch approach. In its pursuit of a different APS concept, OFT was responding to an urgent need statement issued by the Central Command with potential for near-term fielding. This evaluation centered on the results of physical testing of the most technically mature candidate system, the Trophy. Decisions on how to proceed with Trophy involved disagreement between OFT and the Army. While the Trophy tests were successful, the Joint Rapid Acquisition Cell decided to defer fielding the APS system, based in part on the recommendation of Army officials, who believed that testing had not been realistic and integrating it on the platform would delay fielding other useful capabilities. OFT officials proposed additional testing of Trophy to answer these questions, but funding for further OFT testing of this system was discontinued after the Joint Rapid Acquisition Cell's decision.

section_title: Letter
paragraphs: An Active Protection System (APS) is intended to protect a vehicle from  An Active Protection System (APS) is intended to protect a vehicle from  attack by detecting and intercepting missiles or munitions before they hit  attack by detecting and intercepting missiles or munitions before they hit  the vehicle. It has significant possibilities for offering light vehicles some  the vehicle. It has significant possibilities for offering light vehicles some  of the protection that armor gives, without the extra weight. In April 2005,  of the protection that armor gives, without the extra weight. In April 2005,  BAE, a subcontractor for the Army’s Future Combat System (FCS)  BAE, a subcontractor for the Army’s Future Combat System (FCS)  program, issued a draft request for proposals to choose the developer for  program, issued a draft request for proposals to choose the developer for  an APS for FCS and current force manned ground vehicles. To avoid an  an APS for FCS and current force manned ground vehicles. To avoid an  organizational conflict of interest, the lead systems integrator, the Boeing  organizational conflict of interest, the lead systems integrator, the Boeing  Company, ultimately reissued the draft request for proposals, after BAE  Company, ultimately reissued the draft request for proposals, after BAE  decided to submit its own bid on this contract. Raytheon was eventually  decided to submit its own bid on this contract. Raytheon was eventually  chosen as the APS developer and a subsequent trade study reached the  chosen as the APS developer and a subsequent trade study reached the  conclusion that Raytheon’s vertical launch concept was the optimal  conclusion that Raytheon’s vertical launch concept was the optimal  solution to meet the APS requirements. Also, in April 2005, the U.S. Central  solution to meet the APS requirements. Also, in April 2005, the U.S. Central  Command issued an urgent operational need statement to field a  Command issued an urgent operational need statement to field a  combination of near-term technologies, including a different APS, on a  combination of near-term technologies, including a different APS, on a  modified Stryker vehicle, the Full-Spectrum Effects Platform, for  modified Stryker vehicle, the Full-Spectrum Effects Platform, for  demonstration and potential use in Iraq. The effort made in response to  demonstration and potential use in Iraq. The effort made in response to  this need was managed by the Office of Force Transformation (OFT),  this need was managed by the Office of Force Transformation (OFT),  which reported to the Secretary of Defense. After evaluating six candidate  which reported to the Secretary of Defense. After evaluating six candidate  APS, the OFT chose to test the Trophy APS because they considered it to  APS, the OFT chose to test the Trophy APS because they considered it to  be the most technically mature. In June 2006, the Joint Rapid Acquisition  be the most technically mature. In June 2006, the Joint Rapid Acquisition  Cell decided to defer fielding the APS system.  Cell decided to defer fielding the APS system.
Both the Army’s FCS program and the OFT effort were looking for APS  Both the Army’s FCS program and the OFT effort were looking for APS  applications for current Army forces, but with somewhat different  applications for current Army forces, but with somewhat different  objectives in mind. Although the primary focus of the APS subcontractor  objectives in mind. Although the primary focus of the APS subcontractor  selection and trade study was to pick a developer and a design for the FCS  selection and trade study was to pick a developer and a design for the FCS  APS, another objective was to field an APS on current force combat  APS, another objective was to field an APS on current force combat  vehicle platforms in the 2009 time period. On the other hand, the OFT  worked on a separate objective aimed at providing these capabilities  sooner to forces currently in theater, to provide a nearer-term  demonstration of APS.
This letter is to respond to your request for a review of the process that led  to the Army’s decision to pursue a new APS system under the FCS  program. You asked us to review this process because of uncertainty  surrounding the facts that led to the Army’s decision to pursue a vertical  launch APS concept. Specifically, this report addresses:  1.  The process for selecting the subcontractor to develop an APS for FCS  and whether potential conflicts of interest were avoided.  2.  The timing of the trade study, whether it followed a consistent  methodology to evaluate all the alternatives, and the results.  3.  The role the Army and the lead systems integrator played in selecting  the APS developer.  4.  The process followed for evaluating a near-term APS solution for  current forces.
In conducting our work, we held discussions with FCS, Army, and other  DOD officials. We reviewed documents relevant to the decision the Army  made about which APS system to pursue and held discussions with key  officials about various decisions that have been made in pursuing this  technology. We conducted our work between October 2006 and June 2007  in accordance with generally accepted government auditing standards.  Appendix I further discusses our scope and methodology.
subsections: section_title: Background
paragraphs: The FCS concept is designed to be part of the Army’s Future Force, which  is intended to transform the Army into a more rapidly deployable and  responsive force that differs substantially from the large division-centric  structure of the past. The FCS family of weapons is now expected to  include 14 manned and unmanned ground vehicles, air vehicles, sensors,  and munitions that will be linked by an advanced information network.  Fundamentally, the FCS concept is to replace mass with superior  information—allowing soldiers to see and hit the enemy first rather than  to rely on heavy armor to withstand a hit. The Army envisions a new way  of fighting that depends on networking the force, which involves linking  people, platforms, weapons, and sensors seamlessly together in a system  of systems.
Within the FCS program, eight types of manned ground vehicles are being  developed, each having a common engine, chassis, and other components.  One of the other common components is a hit avoidance system that  features a set of capabilities to detect, avoid, and/or defeat threats against  the manned ground vehicles. One of its subsystems is the APS, which is  intended to protect a vehicle from attack by detecting a threat in the form  of an incoming round or rocket propelled grenade (threat) and launching  an interceptor round from the vehicle to destroy the incoming weapon. An  APS system consists of a radar to detect the incoming weapon, a launcher,  an interceptor or missile, and a computing system.
The Army has employed a management approach for FCS that centers on a  lead systems integrator to provide significant management services to help  the Army define and develop FCS and reach across traditional Army  mission areas. Boeing, along with its subcontractor, the Science  Applications International Corporation (SAIC), serves as the lead systems  integrator for the FCS system development and demonstration phase of  acquisition, which is expected to extend until 2014. The lead systems  integrator has a close partner-like relationship with the Army and its  responsibilities include requirements development, design, and source  selection of major system and subsystem subcontractors. In the case of  APS, the first-tier subcontractors are the manned ground vehicle  integrators, BAE and General Dynamics Land Systems, who are  responsible for developing individual systems. BAE was designated the hit  avoidance integrator, a role that covers more than active protection, and  was responsible for awarding the subcontract to the APS developer. This  subcontract has three elements: a base contract, option A to support the  current force (the short-range solution) and option B to support the FCS  manned ground vehicles (short- and long-range solution). Figure  1illustrates these relationships.
A separate initiative involving active protection resulted from a Joint  Urgent Operational Needs Statement, issued by Central Command and the  Multi-National Corps in Iraq in April of 2005, which requested 14 special- equipped vehicles with a host of distinctive capabilities, one of which was  an APS. The need statement called for a capability to field a combination  of near-term technologies that would be useful in conducting force  protection missions, reconnaissance and crowd control in Iraq and an  evaluation of an active protection capability against rocket-propelled  grenades as part of this suite of capabilities. To respond to this need  statement, the Joint Rapid Acquisition Cell, a group within the Office of  the Secretary of Defense (OSD) that seeks solutions to urgent needs and  focuses on near-term or off-the-shelf equipment to meet these needs,  provided funding to the Army, which worked with the OFT to evaluate  various technologies, including an APS, for inclusion on the vehicles. The  OFT was also an office within the OSD, and its role was to examine  unanticipated needs and experiment with innovative technologies that  could be used to meet warfighter needs.
Both the process for evaluating APS sources and concepts to meet FCS  needs and the urgent needs of the Central Command occurred nearly  simultaneously, as shown in figure 3.
As can be seen in figure 3, many events took place at the same time. The  lead systems integrator for FCS completed its subcontractor selection for  APS shortly before decisions were made on the near term system being  considered to meet the Central Command need. The Trophy system was  evaluated as a candidate system in both processes.
section_title: APS Source Selection Avoided Organizational Conflicts of Interest
paragraphs: In choosing the developer for the APS system, the FCS lead systems  integrator, with Army support and concurrence, conducted a source  selection and followed the FCS lead systems integrator subcontract  provisions for avoiding organizational conflicts of interest. The purpose  was to select the subcontractor for the APS that would be best able to  develop the overall APS architecture to address the FCS requirements to  defeat the short- and long-range antiarmor threats as well as meet the  current force needs for defeating short-range rocket-propelled grenade  attacks. The subcontractor selected would support the hit avoidance  integrator in integrating APS technology into the FCS manned ground  vehicles and also apply this architecture to the Army’s current force. The  contract included two options that were to supply the specific design for  the APS system: Option A for the short-range APS for the current force;  and Option B for the short- and long-range solution for the FCS. These  options would be awarded later, based on the results of trade studies  subsequently performed.
To protect against organizational conflicts of interest, contracts between  the FCS lead systems integrator and its subcontractors preclude a  subcontractor from conducting or participating in a source selection for  other FCS subcontracts if any part of its organization submits a proposal.  Under normal circumstances, since the APS would be part of the hit  avoidance system of the FCS manned ground vehicles, the hit avoidance  integrator, BAE, would have had the primary responsibility to issue the  requests for proposals, conduct the source selection evaluation, and award  the contract. In this capacity, BAE issued a draft request for proposals for  the APS in April 2005. When the firm subsequently decided to submit a  proposal on the APS subcontract, it was required, under the FCS lead  systems integrator subcontract organizational conflict of interest  provisions, to notify the lead systems integrator, Boeing, of its intention.  BAE did so and the lead systems integrator reissued the request for  proposals for APS in September 2005 and assumed the source selection  responsibilities. BAE submitted its proposal but then had no further role in  the evaluation of proposals or the actual source selection. After the source  selection was complete, the lead systems integrator transferred contract  responsibility to BAE, and BAE assumed the responsibility for awarding  and administering the APS contract.
From our review, the documentation from the APS source selection  process shows that (1) no officials from the offering companies  participated in the source selection process, and (2) all offerors were  evaluated based on the same criteria contained in the request for  proposals. In response to this request for proposals, four proposals were  received. Three proposals were considered competitive, while the fourth  was eliminated from consideration as it was considered “unsatisfactory” in  technical merit and its architectural approach did not meet the  requirements. Proposals from the remaining three companies—BAE,  Raytheon, and General Dynamics Land Systems—were evaluated in the  source selection process and no officials from these companies were on  the evaluating or selecting teams. The source selection evaluation team  consisted of 53 members, with 27 lead systems integrator representatives  and 26 government representatives, including personnel from the FCS  program manager’s office, Army research centers, and the Defense  Contract Management Agency. After evaluating each of the proposals  against the criteria spelled out in the request for proposals, the source  selection evaluation team made its recommendation to the lead systems  integrator source selection executive, who accepted itsrecommendation.
Our review of the documentation shows that the criteria were ranked in  order of importance, with technical merit considered most important, then  cost, management/schedule and finally past performance. The technical  merit criteria were divided into six sub-factors: systems engineering and  architecture; expertise in APS technologies; simulation, modeling and test;  fratricide and collateral damage; specialty engineering; and integration  capability. Cost criteria were based on the realism, reasonableness,  completeness, and affordability of the proposal. Management/ schedule  criteria included such areas as expertise and experience in key positions.  The past performance risk rating category was based on whether the  respondents’ past performance raised doubts about their being able to  perform the contract.
Since all three proposals were deemed comparable in the areas of cost,  management/schedule, and past performance, the primary discriminating  factor became technical merit. According to the evaluation  documentation, the technical merit scores were assessed based on  whether the proposal demonstrated that the contractor understood the  requirements and on its approach to meeting these requirements in each of  the six technical merit sub-factors. Also, part of the technical score was a  proposal risk evaluation, defined as the degree any proposal weaknesses  could cause disruption of schedule, increase in cost, or degradation in  performance.
While the source selection’s stated purpose was to choose the company  best able to develop the APS and not a specific design, each proposal used  a specific APS system as an “artifact” to illustrate how they intended to  meet the requirements. Even though, in theory, one company could have  been chosen as the APS developer while another company’s preferred  design could have been selected for development, much of the source  selection assessment of technical merit was based on the “artifact” used  for illustration. For example, in the technical merit category of APS  expertise, the source selection evaluation of Raytheon states that “the  vertical launch concept solves several design and integration problems.”  Similarly, the BAE evaluation in the criteria of APS expertise states that  “the proposed long-range countermeasure…design has effectiveness  against the full spectrum of threats.” The General Dynamics Land System’s  evaluation discusses the relatively high technology readiness level (TRL)  of the “proposed Trophy system.” Therefore, while each company’s  proposed solution was not the only aspect of the proposals to be  evaluated, the evaluation documentation shows that the technical merit  category was a key factor in the evaluation.
The source selection evaluation team decided that the BAE and Raytheon  proposals had the highest technical merit. BAE had a lower-risk approach  and its solution had been tested in a relevant environment: however, the  source selection evaluation team stated that this low-risk approach could  prevent BAE from considering higher-risk options that would enable them  to meet the full range of the performance requirements, such as protection  from top-attack weapons. In addition, the source selection evaluation team  determined that, while both Raytheon and BAE could develop the design  presented in the BAE proposal, Raytheon would have the advantage if the  vertical launch design was chosen. The evaluation team concluded that the  Raytheon approach would have the best chance of meeting all the  requirements. Based on the team’s recommendation, the lead systems  integrator selected Raytheon. The integrator accepted the higher risk  because it concluded that the Raytheon proposal had excellent technical  merit and the firm would be better able to develop the vertical launch  technology, if that were the design decided upon in the trade study.
section_title: Trade Study Used Consistent Method in Reaffirming Vertical Launch Concept
paragraphs: The APS development contract required the winner of the source selection  to perform a trade study identifying and assessing competing APS  alternatives. The trade study used a methodology consistent with Army  guidance to evaluate all alternatives, ultimately selecting Raytheon’s  vertical launch as the best design. According to the Army and the lead  systems integrator, conducting the trade study after choosing the APS  subcontractor could have resulted in selecting a different concept than  Raytheon’s vertical launch design. However, in our view, this possibility  appears remote given the selection of Raytheon as APS developer was  based largely on the technical merits of its vertical launch design and the  fact that it would be best able to develop that design.
The development contract’s terms required the source selection winner to  perform a trade study that would identify and assess APS alternatives and  select an APS design from among competing alternatives. Therefore, once  Raytheon won the development contract in March 2006, it was required to  conduct the trade study rather than simply develop its own design. Since  the trade study was not a source selection, FAR contract provisions  regarding organizational conflicts of interest did not apply and Raytheon  was free to participate in the study as the responsible contractor. The  trade study’s specific objective was to choose a single short-range APS  architecture (launcher and interceptor) that best met active protection  requirements for FCS manned ground vehicles, with consideration for  application to the current force. The study was conducted in May 2006 and  Raytheon’s vertical launch concept was selected as the design.
Based on the trade study documentation, the study was conducted using a  methodology prescribed by Army guidance and this methodology was  applied consistently to all APS alternatives. Seven alternatives survived a  screening process and were then evaluated against a set of weighted  criteria. The study concluded that Raytheon’s vertical launch was the best  design approach.
According to general Army guidance for trade studies, steps in the trade  study process should include such elements as incorporating stakeholders,  identifying assumptions, determining criteria, identifying alternatives, and  conducting comparative analyses. The APS trade study process  consistently applied such methodology to all APS alternatives by using  separate, independent roles for a technical team and stakeholders;  operating under a set of assumptions; using validated, protected technical  data on each alternative; having a screening process to filter out non- viable alternatives; and using a set of weighted criteria to assess  alternatives that survived the screening process.
The trade study was performed by a technical team and stakeholders— each having separate roles and operating independently from one another.  The technical team provided technical input and expertise to the  stakeholders, who were the voting members of the study and made the  final selection. The technical team, 21 members from industry and  government as shown in table 1, included individuals who were subject  matter experts as well as those from organizations participating in  development of the short-range APS. Raytheon had 11 members on the  technical team—the most from any single organization. The Army stated  that this representation included administrators and observers and  occurred because Raytheon had been designated APS developer, was thus  required to conduct the trade study, and could gain knowledge from  attending subject matter experts.
The stakeholders made the final selection. The composition and number of  stakeholders are shown in table 2. The stakeholders were program leads  from the Army, lead systems integrator, and subcontractors responsible  for integrating the FCS manned ground vehicles. According to the Army,  Raytheon’s APS program manager was included as a stakeholder because  Raytheon as developer had responsibility for developing the design chosen  by the trade study process.
The technical team and stakeholders operated the trade study under  assumptions that set parameters for screening and evaluating each  alternative. These assumptions were tied to such areas as performance  and threat. Additionally, they conducted the study using data that was  previously validated and remained protected throughout the study’s  course. The primary source of the data was the Army Research,  Development, and Engineering Command’s APS database, which  contained data gathered and validated by the Command’s subordinate  labs. This data was protected by third parties, including the Department of  Energy’s Idaho National Lab, to ensure it was not changed during the  study.
The technical team used initial screening processes to eliminate four  alternatives and identify seven viable alternatives for further assessment.  The screening process filtered out the four alternatives that could not  meet one or both of two criteria: (1) ability to grow to meet 360-degree  hemispherical requirements, and (2) ability to be procured within a  program schedule that would meet the need for prototype delivery of a  short-range solution to the current force in fiscal year 2009. The seven  alternatives that survived the screening process are shown in table 3, along  with the respective government organizations and industry associated with  each.
The technical team assessed the seven alternatives against a set of five  weighted criteria. According to the Army, these were the same top-level  criteria mandated in all FCS trade studies, and their weights were assigned  by FCS chief engineers. Table 4 defines each of the criteria and provides  information on respective weights.
The vertical launch concept scored highest in every category of criteria  except risk. The Army indicated that the concept had about one-third  better overall weighted performance than the other alternatives. Army  officials described the vertical launch design as having technical  advantages over the other alternatives—including the need for less space,  weight, and power—as well as cost benefits.
The Army and lead systems integrator officials told us that the trade study  could have resulted in the selection of a design other than Raytheon’s.  They also stated that, had this occurred, Raytheon as APS developer  would have been required to develop this design rather than the vertical  launch. While in theory the APS source selection chose a developer and  the trade study chose the design to develop, in reality it is difficult to  separate the trade study results and the source selection decision. In our  view, in both the source selection and trade study, criteria related to  technical aspects of the designs were deciding factors. Considering that  the source selection evaluation relied on artifacts representing specific  systems—and Raytheon won the source selection based in large part on  the technical merit of its artifact—it seems unlikely that the APS trade  study would have resulted in the selection of any system other than  Raytheon’s vertical launch.
Although the trade study concluded that vertical launch was a high-payoff  approach, it also noted that it was a high risk due to its low technology  maturity. At the time of the trade study, as shown in table 5, the vertical  launch was less technologically mature than the other alternatives except  for one.
The Army expects the design to reach TRL 6 (system model or prototype  demonstration in a relevant environment) by August or September 2007.  The Army expects the vertical launch concept to be available for prototype  delivery to current force combat vehicles in fiscal year 2009 and for testing  on a FCS vehicle in 2011. These estimates appear optimistic. At a TRL 5,  the vertical launch will require additional technology development and  demonstration before it is ready for either application. Also, the FCS  vehicles have not been fully developed yet. Assuming all goes as planned,  most FCS vehicle prototypes are expected to be available in 2011 for  developmental testing. As we noted in our March 2007 report, the Army  has in general been accepting significant risks with immature technologies  for the FCS program, coupled with compressed schedules for testing and  evaluating prototypes.
section_title: Army and Lead Systems Integrator Had Extensive Roles in APS Subcontractor Selection and Trade Study
paragraphs: The Army and the lead systems integrator were both extensively involved  in preparing for and conducting the APS subcontractor selection and the  trade study. Prior to the selection, FCS program officials assisted in APS  requirements development and reviewed and approved the scope of work,  schedule, and evaluation criteria for the request for proposals. After the  proposals were received, FCS program officials, technical experts from  various Army research centers, representatives of the Tank-Automotive  and Armaments Command and the Training and Doctrine Command were  active participants in the selection evaluation team and reviewed the  proposals along with the lead systems integrator members. The Source  Selection Advisory Council, who advise the Source Selection Executive,   provided oversight to the evaluation team and also had representatives  from the FCS program manager’s office and the Army research  community. Similarly, Army FCS officials, as well as technical experts  from Army research centers, were members of the trade study technical  team and also concurred in the choice of the vertical launch concept. The  co-lead of the trade study was an FCS official. The lead systems  integrator’s office assumed responsibility for the selection process, was  the selection executive, and made the final choice of an APS developer. In  addition to its lead role in the APS subcontractor selection, the lead  systems integrator was represented on the trade study technical team and  was one of the stakeholders.
As our previous body of work on the FCS program has shown, the Army’s  participation in the APS subcontractor selection and trade study is  consistent with the Army’s general approach to FCS. Army leadership set  up the FCS program in such a way that it would create more competition  and have more influence over the selection of suppliers below the lead  systems integrator. In setting up FCS, Army leadership noted that  traditionally, once the Army hired a prime contractor, that contractor  would bring its own supplier chains. The Army was not very involved in  the choice of the suppliers. In FCS, the Army called for the lead systems  integrator to hold a competition for the next tier of contractors. The Army  had veto power over these selections. In addition, the Army directed that  the lead systems integrator employ integrators at lower levels in the  program, for high-cost items such as sensors and active protection  systems and the Army has been involved with these selections. These  integrators were also to hold competitions to select suppliers for those  systems. This strategy was designed to keep the first tier of contractors  from bringing their own supplier chains and pushed competition and Army  visibility down lower in the supplier chain. The fact that the decisions on  the APS subcontractor selection and trade study lend themselves to after- the-fact examination is due in part to the Army’s focus on competition at  lower supplier levels on FCS.
section_title: OFT Process for Evaluating APS Was More Test-Based and Near-Term Oriented
paragraphs: The process followed by OFT to meet the urgent needs of the Central  Command was characterized by a simpler evaluation of active protection  systems with potential for near term fielding, followed by actual physical  testing of the  APS candidate system that the OFT considered most  technically mature, the Trophy. The Army’s Program Manager’s Office for  Close Combat Systems was also involved in this evaluation. While the  testing of Trophy had a high success rate, the Joint Rapid Acquisition Cell  decided to defer fielding the Trophy based, at least in part, on the  recommendations of the Army that the testing was not realistic and the  Trophy’s integration on the platform would delay fielding of other useful  capabilities. OFT officials did not agree with the Army’s position and  thought the system’s success in testing indicated it should be further  evaluated.
To meet the Central Command’s need, OFT began an effort, the Full- Spectrum Effects Platform, to incorporate and test various improvements  for potential application to existing military vehicles such as the Stryker.  The platform itself is a modified Stryker vehicle. The program was divided  into spirals: spiral 0 was to evaluate the synergy of the different systems,  including the APS, on the vehicle and to compile lessons learned to aid in  future concepts of operations, development and integration. Spiral 1 was  intended to field a limited number of such systems to current forces in- theater in 2007, for purposes of an operational assessment of the various  capabilities. The Full Spectrum Effects Platform is not part of or  associated with FCS.
OFT, in association with the Naval Surface Warfare Center, evaluated six  candidate APS systems. Army representatives from the Program Manager,  Close Combat Systems were also involved in this evaluation. The six  candidate systems evaluated are shown in table 6.
These systems were evaluated because the OFT and Navy and Army  officials considered them to be the most promising APS solutions available  within the required schedule. They evaluated each system based on such  criteria as the feasibility of the operational concept, its cost and schedule  factors, as well as its weight, size, and power requirements. Trophy was  selected as the most promising system because it was the most technically  mature system and was being developed by Israeli defense forces that had  done initial work to integrate it on a light armored vehicle.
OFT subsequently sponsored tests of the Trophy APS as part of the Full- Spectrum Effects Platform at Naval Surface Warfare Center in Dahlgren,  Virginia. A representative from the Army’s Program Manager, Close  Combat Systems, was part of the oversight team for these tests. In these  test firings, the Trophy APS did well, destroying 35 of 38 incoming rocket- propelled grenades. However, the process for deciding how to proceed  based on the test results was not agreed to in advance. A disagreement  subsequently arose between OFT and the Army Close Combat System  officials on how best to proceed from the testing.
Although the tests were not designed to represent the Trophy’s capabilities  in a realistic operational environment, OFT officials concluded that  Trophy showed enough promise that they recommended continued testing  to demonstrate its capabilities under various conditions. These officials  estimated that an additional $13 million would cover the cost for this  testing. They believed that Trophy could be integrated in the near term on  existing light-armored vehicles and meet the urgent need for an immediate  APS capability.
The Army officials disagreed with OFT’s assessment that further testing of  Trophy for inclusion on the Full Spectrum Effects Platform was justified.  According to the Army officials, Trophy was not tested in a realistic  environment for collateral damage or effectiveness. They believed that it  would not be sufficiently tested for operational and safety issues within  the time period required for the first spiral of the Full Spectrum Effects  Platform. A delay in its integration on the Platform would delay, by at least  6 to 14 months, demonstration of other potentially useful capabilities,that  could be immediately incorporated. Further, the Army estimated that it  would take 5 years to integrate and field Trophy on other current force  manned ground vehicles. The Army recommended to the Joint Rapid  Acquisition Cell that the Trophy APS be excluded from Spiral 1 of the Full- Spectrum Effects Platform. In lieu of putting this technology in the field,  the Army recommended that slat armor be incorporated on Spiral 1, since  it has been effective in defeating the current rocket-propelled grenade  threat. OFT officials disagreed, reasoning that although the use of slat  armor on the current force has seemed to mitigate the effects of the  rocket-propelled grenades currently in use, improved munitions will soon  be available, and the slat armor will no longer be effective against these  threats. They believed that the Trophy should be tested further in order to  answer the questions raised by the Army and to provide insight into its  capabilities. OFT officials based their position on the Trophy’s success in  these tests, its high level of technical maturity when compared to other  active protection systems, and the criticality of the need.
The Joint Rapid Acquisition Cell presented this information to Central  Command and recommended slipping the active protection capability to a  later platform spiral, once it was more mature. Currently, there are no  plans for further evaluation of active protection for future platform spirals.  Upon the removal of the Trophy APS system from the Full-Spectrum  Effects Platform vehicle, the Joint Rapid Acquisition Cell discontinued  funding for further testing and evaluation of the Trophy. The disagreement  between Army and OFT officials notwithstanding, we did not find  information that would challenge the decision to defer the introduction of  the Trophy on light-armored vehicles. On the other hand, the 5 years the  Army estimated would be needed to integrate the comparatively mature  Trophy system on the existing Stryker vehicle does not appear consistent  with its estimates that the less mature vertical launch system could be  ready for prototype delivery on Strykers in 2 years and on the yet-to-be  developed FCS prototypes in 3 years.
section_title: Conclusions
paragraphs: The FCS lead systems integrator, with support from the Army, followed a  consistent and disciplined process in both selecting Raytheon to develop  the APS for FCS and in conducting the trade study and followed the lead  systems integrator subcontract and FAR provisions for avoiding  organizational conflicts of interest. While the role played by Raytheon in  the trade study was in accordance with its contract and thus not improper,  the rationale for having the trade study follow the source selection is not  entirely clear. The purpose of the trade study was to select the best  concept; yet, the source selection process that preceded it had, in fact,  chosen Raytheon primarily on the technical merits of its vertical launch  design concept. It was thus improbable that the trade study would reach a  different conclusion. Both the Army and the lead systems integrator were  closely involved throughout the source selection and trade study  processes and concurred in the selection of Raytheon’s APS concept.
The process for evaluating the Trophy system to meet the urgent needs of  the Central Command was different. It centered more directly on the  results of physical testing, followed a less-disciplined decision-making  process, and was characterized by considerable disagreement between  OFT and the Army. While the decision to defer the use of the Trophy on  fielded vehicles appears prudent in light of the limited realism of the  testing, the promising results of the testing likewise appeared to warrant  additional testing of the Trophy system to either confirm or dispel  potential risks in the use of APS capabilities. Discontinuing all testing of  the Trophy systems may thus have been premature, particularly in light of  the need to better understand tactics, techniques and procedures and  concepts of operations for both near-term and long-term applications.
section_title: Recommendations for Executive Action
paragraphs: Because of the likelihood that the Army will introduce APS into its forces,  we recommend that the Secretary of Defense support additional testing  and demonstration of near-term APS systems on the Full Spectrum Effects  Platform or similar vehicles to, at a minimum, help develop tactics,  techniques, procedures, and concepts of operations for both near-term and  long-term active protection systems.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided us with written comments on a draft of this report. The  comments are reprinted in appendix II. DOD did not concur with our  recommendation. DOD also provided technical comments, which we  incorporated where appropriate.
DOD did not concur with our recommendation that the Secretary of  Defense support additional testing and demonstration of near-term active  protection systems on the Full Spectrum Effects Platform that could  respond to the Central Command’s need.  It stated that the original  decision in May 2006 that delayed delivering Full Spectrum Effects  Platform capabilities due to technical development and performance risks  remains true today.  DOD added that there are no active protection  systems mature enough at this time to integrate on a Full Spectrum Effects  Platform regardless of any additional testing and demonstration efforts.   This represents a much more decided opinion than was rendered at the  time of the OFT tests.  At that time, Army officials believed that the Trophy  would not be sufficiently tested for operational and safety issues in time  for the first spiral of the Full Spectrum Effects Platform.  OFT officials  believed that the Trophy should be tested further to answer the questions  raised by the Army and to provide insight into its capabilities.  Ultimately,  the Joint Rapid Acquisition Cell recommended slipping the active  protection capability to a later spiral of the Full Spectrum Effects  Platform.  This was the basis for our recommendation for additional  testing of near-term active protection systems on the Full Spectrum  Effects Platform.
DOD stated that it continues to pursue active protection, citing the Army’s  vertical launch system for FCS.  As stated in our report, this system is  technically immature and the Army’s estimates for testing it appear  optimistic.  According to the Institute of Defense Analysis, the vertical  launch system is ambitious, with much enabling technology not yet  demonstrated.  Given the criticality of active protection for the FCS  manned ground vehicles, additional testing of near-term active protection  systems could provide valuable insights into operations and tactics that  would benefit future applications, such as FCS.  DOD noted that the  Trophy system is being tested on the Wolf Pack Platoon Project, an OSD  Rapid Reaction Technology Office (formerly OFT) effort.  However, this  project is not directed toward development of APS tactics, techniques,  procedures, or concepts of operations.  In addition, it will not include  testing against live targets.
Testing near-term active protection systems on the Full Spectrum Effects  Platform or similar vehicles is valuable for answering remaining questions  about such systems and to provide insights for the employment of future  systems.  This is particularly important given the likelihood that the Army  will field some form of APS to its forces. We have broadened our  recommendation to capture the value of continued testing of near-term  APS for tactics, techniques and procedures and concepts of operations.
Please contact me on (202) 512-4841 if you or your staff has any questions  concerning this report. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.
section_title: Appendix I: Scope and Methodology
paragraphs: To develop the information on the U.S. Army’s decision to pursue a new  APS system under the FCS program, we interviewed officials of the Office  of the Assistant Secretary of the Army (Acquisition, Logistics and  Technology); the Tank-Automotive and Armaments Command; the Joint  Rapid Acquisition Cell; the Office of Force Transformation; the Naval  Surface Warfare Center (Dahlgren Division); the Program Manager for the  Future Combat System (Brigade Combat Team); and the Future Combat  System Lead Systems Integrator.
We reviewed the APS subcontractor selection documentation, including  the APS request for proposal, current force and FCS operational  requirements documents, subcontract proposals, criteria used to rate  those proposals, and the APS development contract to determine if  procedures for avoiding organizational conflicts of interest were followed  and how the APS subcontractor was selected. In addition, we held  discussions with key Army officials and lead systems integrator  representatives regarding this process and their roles in it. To determine  why the trade study was conducted after source selection, we reviewed  the trade study process and results and Army guidelines for conducting  trade studies. To identify the roles played by both the Army and lead  systems integrator in the selection of an APS, we reviewed documentation  concerning their roles in these processes. We also reviewed these  materials to determine whether consideration was given to a separate APS  solution for current forces and, in conjunction with this issue, we  reviewed test reports and other documentation and discussed the testing  of an alternative APS system, the Trophy, with the parties involved.
In evaluating the APS subcontractor selection and trade study processes,  we did not attempt to determine if the best technical solution was chosen,  but only if these processes followed lead systems integrator provisions for  organizational conflicts of interest and used a consistent methodology for  the trade study. We conducted our work between October 2006 and June  2007 in accordance with generally accepted government auditing  standards.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: Other contributors to this report were Assistant Director William R.  Graveline, Marie P. Ahearn, Beverly Breen, Tana Davis, Letisha Jenkins,  Kenneth E. Patton, and Robert Swierczek.