The F/A-18E/F is currently undergoing development flight testing as part of its engineering and manufacturing development (EMD) phase of the acquisition cycle. The development flight test program is under the responsibility of the Integrated Test Team, which consists of Navy and contractor personnel. The team also receives support from the Navy’s Operational Test and Evaluation Force. The F/A-18E/F development flight test program began in February 1996 at the Naval Air Warfare Center, Patuxent River Naval Air Station, Lexington Park, Maryland. The Integrated Test Team is using the seven test aircraft provided by Boeing (formerly McDonnell Douglas) under the EMD contract. The seven aircraft consist of five single-seat E models and two 2-seat F models. Boeing has also built 3 ground test article aircraft to use in conducting tests at its St. Louis, Missouri, facility, and General Electric Corporation, Lynn, Massachusetts, has delivered 21 engines for flight testing. The Navy plans to procure 62 low-rate initial production aircraft in 3 separate procurement lots. In March 1997, the Navy received approval to procure 12 aircraft under the first low-rate initial production lot. The decision to approve the procurement of the next 20 aircraft under the second low-rate initial production lot was scheduled for the end of 1997,and the decision to approve the procurement of the final 30 aircraft under the third low-rate initial production lot is scheduled for late 1998 or early 1999. The primary purpose of the development test program is to identify system deficiencies so they can be corrected and have a production representative aircraft ready to begin Operational Test and Evaluation in May 1999. As the flight test program progressed, delays were encountered due to events that normally occur during testing, such as inclement weather conditions and required equipment maintenance. Testing delays were also caused by unanticipated events. For example, in the summer of 1996, a 3-month machinist strike at the airframe contractor’s plant delayed the delivery of the last three EMD aircraft and, in turn, delayed the testing that was to be done on these aircraft. Also, in November 1996, an in-flight engine failure occurred at the Patuxent River test range, which stopped flight testing for 2 months on all but one EMD aircraft—an F model that was being prepared for initial carrier qualification flights. F/A-18E/F program management developed a revised flight test plan that will help cope with the delays in the original flight test program. Development of the revised plan began with an Integrated Test Team meeting in September 1996. According to the minutes of that meeting, the team reviewed flight test data and revised the original flight test plan by identifying areas in which testing could be reduced but essential program requirements and goals could still be met. At the time of our review, however, the revised flight test program was about 4 weeks behind schedule. Program documents predict that, although flight testing is behind schedule, decisions to reduce test points will enable the Navy to regain lost time. The documents state that the Navy anticipates completing development testing in November 1998 and begin operational testing in May 1999, as originally planned. In the meantime, program officials plan to conduct monthly reviews to identify additional areas that can be deleted from the flight test program. The Integrated Test Team and F/A-18E/F program management officials stated that, while the elimination of some data collection requirements might add some risk to the E/F program, the risk is at an acceptable level. The Navy’s F/A-18E/F Integrated Test team established a system for identifying deficiencies during the development program. That system, which is described in appendix II, identified over 400 deficiencies as of December 1997. The number of deficiencies changes constantly as some are resolved and others are identified. The deficiencies include problems with E/F flying qualities, structural concerns that could have a negative impact on the aircraft’s service life, engine deficiencies that could impact aircraft performance and engine life, and weapon separation problems that cause bomb-to-bomb collisions that require additional testing. The Navy also established a Program Risk Advisory Board. The Board identifies deficiencies from flight or ground test data and assesses the risk that the deficiencies represent to the program. Boeing also identifies deficiencies during flight or ground tests that it believes represent a risk to the program and develops mitigation plans for resolving these risks. As of September 1997, the Board and Boeing had identified 33 and 38 program risks, respectively. A listing of risk items and their assigned level of risk by the Board and Boeing is in appendix III. Although many of the deficiencies have not been resolved, Navy program management continues to project that the F/A-18E/F will be ready for operational testing as scheduled in May 1999 and that the aircraft will meet all operational performance requirements. On the other hand, the Navy’s Program Risk Advisory Board stated in July 1997 that the Navy’s Operational Test and Evaluation Force may find that the E/F is not operationally effective or suitable. According to program officials who are members of the Board, the Board’s assessment reflects the realization that the F/A-18E/F may not be as capable in a number of operational performance areas as the most recently procured C model aircraft, which are equipped with an enhanced performance engine. This issue was addressed in a classified December 1997 Operational Test and Evaluation Force report. That report was requested by the F/A-18E/F program office. The report is referred to as a Quick Look Report because it represents the Operational Test and Evaluation Force’s preliminary conclusions based on a limited analysis of data collected during its operational assessment completed in November 1997. The Quick Look Report identified 16 major deficiencies with the E/F, such as air-to-ground sensor performance, air-to-ground weapons, air-to-air sensor performance, and survivability. However, the report concluded that the F/A-18E/F is potentially operationally effective and potentially operationally suitable. The report also confirmed the Program Risk Advisory Board’s concerns regarding certain classified operational performance characteristics of the E/F compared with the operational capabilities of the F/A-18C. In addition, the report indicated that the Operational Test and Evaluation Force’s final report, scheduled to be issued in March 1998, will be based on more detailed analysis of available data and may contain modified conclusions. The following section discusses selected risk items that were identified by program officials and documents as significant concerns, including items discussed in our previous report on the F/A-18E/F program. These items are wing problems, new technology advances, engine challenges, weapons separation problems, and horizontal and vertical tail problems. In March 1996, during flight testing at the Patuxent River Naval Air Station, the F/A-18E/F experienced wing drop. The Navy and Boeing describe the phenomenon as an unacceptable, uncommanded abrupt lateral roll that randomly occurs at the altitude and speed at which air-to-air combat maneuvers are expected to occur. A joint Navy/Boeing team concluded that wing drop was caused by a loss of lift on one of the outer wing panels during maneuvering. According to Navy and Boeing officials, wing drop is the most challenging technical risk to the F/A-18E/F program. The deficiency has been classified by Boeing and the Program Risk Advisory Board as a medium technical, schedule, and cost risk to the low-rate initial production phase of the E/F program. Program officials consider wing drop to be a high-risk deficiency. The F/A-18E/F Integrated Test Team concluded that if wing drop is not corrected, it will prevent or severely restrict the performance of the F/A-18E/F during air-to-air combat maneuvering. The F/A-18E/F Program Risk Advisory Board concluded that this deficiency would cause the aircraft to be unacceptable for operational test and evaluation and could result in a schedule slip. Boeing and the Navy have continued their attempts to define the cause of wing drop and identify potential solutions. For example, 25 potential wing modifications have been tested in a wind tunnel. Flight hardware to test two leading-edge wing modifications have been designed and fabricated, and flight testing of the modifications has begun. One of the leading-edge wing modifications provided no improvement. The other provided improvement for turns above 20,000 feet, but improvements are still needed for air-to-air tracking tasks and turns at 15,000 feet and below. In September 1997, a Blue Ribbon panel concluded that an intermediate solution to wing drop would be to fix both the leading and trailing edges of the wing. The Blue Ribbon panel further proposed that a total wing redesign should be considered as the long-term solution to wing drop. In November 1997, the Assistant Secretary of the Navy for Research, Development, and Acquisition advised the Secretary of the Navy that the low-cost, quick fixes have improved aircraft performance but have not completely resolved the wing drop issue. The Assistant Secretary also stated that the best and worst case scenarios for resolving the problem ranged from a combination of software changes with simple wing modifications, which should not impact production and acquisition plans, to a more complex and lengthy wing redesign, which would impact production and acquisition of the aircraft. In January 1998, program officials told us that the F/A-18E/F will not require a major wing redesign. This assessment is based on their assumption that although wing modifications that are currently under investigation might not entirely eliminate the possible occurrence of wing drop, the modifications would reduce wing drop effects to an acceptable level. Until the Navy identifies and completes its flight testing of these wing modifications, their impact on such things as the F/A-18E/F’s speed and maneuverability, range, weight, and the planned reduced radar cross section of the aircraft to increase its survivability, will not be known. Program officials estimated that they will be able to quantify these performance impacts and decide on the best solution to the wing drop problem by March 1998. This plan coincides with the next major funding decision for the F/A-18E/F program, which will be a decision by the Assistant Secretary of the Navy for Research, Development, and Acquisition on whether to approve full funding of the next 20 aircraft under the second of three low-rate initial production decisions. New technology features (the details of which are classified) have been incorporated into the F/A-18E/F to improve its survivability by reducing the aircraft’s susceptibility to being detected by enemy radar. The Integrated Test Team has documented new technology anomalies that could negatively affect the new technology features to be incorporated into the aircraft. In September 1997, Boeing and the Navy’s Program Risk Advisory Board listed new technology concerns as a high risk to the F/A-18E/F program. The new technology anomalies include such things as seal failures, damage to special coatings, door latches, wing delaminations, and the aircraft’s wind screen. Efforts to correct these problems are ongoing. For example, Boeing has been training its maintenance crews on the proper cleaning and application methods of seals to reduce the failures that have occurred. Longer term production fixes call for redesigning such things as doors and hinges. Further, the test aircraft have received structural repairs to address large delaminations that have occurred on the underside of the aircraft from blown tires. However, these repairs used protruding fasteners that would be unacceptable in operational aircraft because they would negatively impact aircraft signature. Efforts are underway to develop better repair procedures for aircraft to be produced under the second and third low-rate initial production phases of the program. Boeing and the Navy have stated that there is currently no definitive answer as to the impact these changes will have on the reduced radar cross section of the E/F. They believe that the F/A-18E/F will have unacceptable operational test and evaluation results if the fixes do not work. However, if the fixes do work, they need to be included on the aircraft being produced under the first lot of low-rate initial production, because these aircraft will be used for Operational Test and Evaluation. If these fixes are not included, it is likely that operational evaluation will be unacceptable. The Program Risk Advisory Board has identified engine-related issues, including engine warm-up time required before carrier launch, partial engine flameouts during some test flights, visible engine smoke, and engine failures during flight and ground tests. In addition, high-pressure engine turbine blades that had been redesigned to reduce heat to achieve the required engine service life caused an in-flight engine failure. Consequently, the Navy decided to revert to the original turbine blade design. The Navy generally views the engine anomalies as a medium risk to the program. The engine contractor, on the other hand, is redesigning certain portions of the engine and views the engine as a low-risk component of the program. The engine contractor stated that engine anomalies and component redesign have delayed the EMD schedule by 6 to 8 months and increased cost by 4 percent. However, the contractor believes that it will meet the low-rate initial production schedule by extending the work schedule as required. The Navy, however, has expressed concern over engine problems. For example, the Integrated Test Team stated that (1) stalls that occur prior to engine warm-up will preclude the performance of the deck launch intercept mission, which is defined as 5 minutes from engine start to launch; (2) visible engine smoke would increase the overall visibility of the aircraft, which may result in earlier visual acquisition of the aircraft by adversary pilots; and (3) engine flameouts and stalls could result in the destruction of the engine. The Program Risk Advisory Board stated that these engine deficiencies may make the F/A-18E/F unacceptable for operational evaluation or may jeopardize successful operational evaluation. The F/A-18E/F is designed to have more payload capacity than current F/A-18C/Ds as a result of adding two new wing stations to carry external stores. Early wind tunnel tests conducted in July and August 1993 showed that some stores would hit the side of the aircraft or other stores when released. The Navy and Boeing identified the cause of weapon separation problems as the adverse air flow created by the E/F airframe. Boeing spent about 1 year developing and testing several improvement concepts before selecting a redesigned pylon as the intended fix to the stores separation problem. Weapon separation testing with the redesigned pylon began in February 1997 and is expected to continue through November 1998. As of September 1997, the weapon separation problem was classified by Boeing and the Navy Program Risk Advisory Board as a medium technical risk to the EMD phase of the E/F program. In its risk assessment, Boeing stated that if stores separation problems continue to occur during testing, additional changes would be required. In recent flight tests during November and December 1997, bomb-to-bomb collisions occurred when releasing certain weapons. In addition to the weapon separation problems, recent tests have revealed that noise and vibration may cause structural damage to stores being carried under the wing. Currently, this problem is resulting in speed limitations on the aircraft when carrying certain weapons. The F/A-18E/F experienced delaminations, or peeling, in its horizontal tail stabilator. This deficiency was first identified during pre-production ground testing of the EMD aircraft design at the contractor plant in July 1995. The testing showed small areas where the metal substructure and the composite skin did not bond. The contractor used fasteners to ensure that any delaminations of the horizontal stabilator that occurred would not cause any in-flight failures. The contractor also initiated an inspection program every 25 flight hours for the problem area. All seven EMD test aircraft have been equipped with the redesigned horizontal stabilator. According to Boeing, no significant delaminations were occurring, therefore, the inspection frequency is being raised to 50 flight hours. A redesign of the horizontal stabilator for the low-rate initial production aircraft was completed in October 1996 and is currently undergoing testing. In November 1997, delamination occurred during testing of the redesigned stabilator. This resulted in a decision to stop production pending completion of a review of the delamination problem. In commenting on a draft of this report, the Department of Defense (DOD) stated that additional testing and analysis since November 1997 led to the conclusion that the original EMD stabilator design with fasteners is acceptable. The EMD aircraft are in the process of testing this design and, according to DOD, the low-rate initial production aircraft that will have this design will have the stabilators tested prior to delivery. DOD also stated that a slightly redesigned stabilator, to be used in aircraft that will be produced subsequent to the first lot of low-rate production aircraft, is undergoing testing that is scheduled to be completed this summer. The F/A-18E/F vertical tail has not been certified because it experienced deficiencies during testing early in the test cycle. This deficiency has been classified by both Boeing and the Program Risk Advisory Board as a medium technical risk to the low-rate initial production phase of the F/A-18E/F program. According to Boeing, all vertical tail design changes will be incorporated in the aircraft to be procured during low-rate initial production. However, the design changes resulted in a vertical tail weight increase of 20 pounds. An additional vertical tail redesign plan is in process. The purpose of the second redesign is to incorporate weight savings of 29 pounds and improve the tail’s producibility. The redesign is intended to provide a fully certified vertical tail at the start of the third low-rate initial production lot. Testing of the redesigned vertical tail is scheduled to be completed in late 1999. The Navy has consistently maintained that the F/A-18E/F will be developed and produced within the cost estimates established for the program. However, certain key assumptions on which the F/A-18E/F cost estimates were based have been overcome by events. These assumptions relate to such things as: no unanticipated issues during the development program; the number of aircraft to be bought, in total and on an annual basis; the ratio of the E and F models to the total number of aircraft to be bought; and inflation factors to be used in projecting future years’ costs. Adjusting these assumptions to reflect recent events will likely result in higher F/A-18E/F development and unit production costs than the Navy currently estimates. The development cost for the F/A-18E/F program has been capped by the Congress at $4.88 billion (1990 base year dollars). It will be a challenge for the Navy to stay within this cost because, according to Navy documents, that amount is adequate to fund the program based on the assumption that problems would not occur during testing. However, the program has experienced deficiencies; the development flight test program still has about 1 more year, and additional deficiencies may be identified during that time; and EMD funding reserves have nearly all been used. The Navy’s Program Executive Officer for tactical aircraft has raised concerns about the ability of the F/A-18E/F development effort to fund the correction of these deficiencies because the program’s EMD management reserves have diminished significantly. For example, Boeing’s EMD airframe management reserve has decreased from $256 million when the program began to $56.7 million in October 1997. This reserve was used to correct deficiencies as they developed. Of the $56.7 million, $50.9 million has been targeted for known deficiencies that have not yet been corrected, leaving a balance of $5.8 million. In addition, the $28 million EMD engine management reserve at General Electric has been depleted. According to an October 1997 F/A-18E/F program management status report, the lack of engine management reserve is a real concern considering that engine problems need to be corrected. According to the report, General Electric has not yet quantified the full cost impact, but future overruns are expected. The development flight test program will not be completed for another year. Program management has stated that the development flight test program is normally the most risky portion of the development effort. Therefore, if changes to correct known deficiencies fail or if additional deficiencies develop, the cost of correcting them will likely cause the $4.88 billion development cost estimate to be exceeded. The Navy also faces a challenge in procuring the F/A-18E/F within the unit cost originally estimated. Its unit procurement cost estimates have been based on what has become unrealistically high quantities of E/F aircraft that will be bought, a lack of factoring in the cost effect of the Navy’s decision to buy more of the higher cost F models than was factored into the original cost estimates, and an unrealistically low inflation factor for purchases in later years of the program. Originally, Navy projections of F/A-18E/F unit procurement costs were based on procuring 1,000 aircraft at a peak annual production rate of 72 aircraft. Neither of these assumptions are likely to be realized. The assumption that 1,000 E/F aircraft will be procured is not consistent with the outcome of the Quadrennial Defense Review and current Defense Planning Guidance. In May 1997, the Quadrennial Defense Review recommended that, due to funding constraints, the total procurement of F/A-18E/Fs should be reduced to 548 aircraft. The October 1997 Defense Acquisition Executive Summary Report revised the total F/A-18E/F procurement to 548 aircraft. In terms of the Navy’s assumed annual production rate of 72 aircraft, in March 1997 the Under Secretary of Defense for Acquisition indicated the annual E/F production rates would be lower. He directed that he be given the opportunity to review any plan to acquire production tooling that would support producing more than 48 aircraft per year. The May 1997 Quadrennial Defense Review report also recommended an annual production rate of 48 aircraft. According to information provided to you in July 1997 by the Director of Strategic and Tactical Systems, Office of the Secretary of Defense, the lower total buy will decrease the total procurement cost but increase the E/F’s unit procurement cost from $57 million to $64 million (fiscal year 1997 dollars). When the F/A-18E/F program was approved in 1992, the procurement plan called for the majority (820, or 82 percent) of the F/A-18E/F buy to be single-seat E models. Only 180, or 18 percent, of the 1,000 aircraft buy would be two-seat F models to be used for training purposes. However, the Navy has since decided that the majority of the total buy will now be two-seat F models that will require the crew members in the second seat to perform operational as well as training functions. According to program documents, the Navy is using a buy of 548 aircraft, as recommended in the Quadrennial Defense Review, for planning purposes. This buy will consist of 288 (about 53 percent) F model aircraft and 260 (about 47 percent) E model aircraft. This revised acquisition strategy has significant cost implications because, according to program officials, the two-seat F model will cost about $1.5 million more per aircraft than the single-seat E model. However, this cost differential is expected to increase. According to program documents, the back seat of the F will have to be upgraded to accomplish the operational missions that will now be assigned to that model. The cost of this upgrade, which is expected to be accomplished by 2005, has not been estimated. Navy unit procurement cost estimates for the 15-year F/A-18E/F acquisition program assume an annual inflation rate that is provided by the Office of the Secretary of Defense. The unit procurement cost estimates in the Navy’s F/A-18E/F Selected Acquisition Reports from program approval in 1992 through December 1995 were based on a 3-percent annual inflation factor, which measures the general inflation of the U.S. economy rather than the inflation rate for the aerospace industry. The December 1996 Selected Acquisition Report stated a lower projection of E/F unit procurement cost based on a lower 2.2-percent annual inflation factor. According to program documents, the inflation rates provided by the Office of the Secretary of Defense for budget estimating are lower than escalation indexes developed from historical escalation data published by the Bureau of Labor Statistics, which uses the Data Resources Incorporated econometric forecasting model for the aerospace industry. According to E/F program management, the escalation factors generated by the model will be used as a baseline to negotiate E/F procurement cost. Table 1 compares the DOD annual inflation rates with aerospace industry annual inflation rates. Using the higher aerospace industry inflation rates would substantially increase the F/A-18E/F unit procurement cost estimate. The use of understated inflation rates to estimate unit cost is not unique to the F/A-18E/F program. We have issued reports that discuss the impact of understated inflation rates. The ongoing test program has identified numerous deficiencies with the F/A-18E/F aircraft. The Navy’s system for identifying the program risk associated with these deficiencies indicates that several of them are significant. As of March 1998, the Navy had not decided how to resolve some of the deficiencies or predicted the costs involved in resolving them. A Navy board established to identify risks to the F/A-18E/F program has stated that, until several of the deficiencies have been resolved, the Operational Test and Evaluation portion of the F/A-18E/F program, scheduled to begin in May 1999, might slip or that the F/A-18E/F will have an unsuccessful Operational Test and Evaluation. We recognize that the F/A-18E/F development test program has nearly 1 year remaining before it is scheduled to be completed. Therefore, the Navy still has time to try to resolve the deficiencies being identified during the test program. However, additional deficiencies may be identified before the test program is completed. The issue is how much time and money will be required to satisfactorily resolve these deficiencies. This will not be known until the E/F has completed its Operational Test and Evaluation. The deficiencies discussed in this report were identified prior to DOD’s March 1997 decision to approve the E/F program to enter low-rate initial production. DOD’s approval to advance the program into production indicates its optimism and willingness to accept the risk that these deficiencies, and any additional deficiencies that might arise, will be resolved with little or no cost, schedule, or performance impact on the program. Program documents indicate, however, that correcting some of these deficiencies, such as the wing drop problem, could have significant cost, schedule, and performance impacts on the F/A-18E/F program. We believe that DOD and the Navy need to adopt a more cautious approach as they make funding decisions for the E/F program and prepare for Operational Test and Evaluation of the aircraft. Therefore, we recommend that the Secretary of Defense direct the Secretary of the Navy to not approve contracting for any additional F/A-18E/F aircraft beyond the 12 aircraft contracted for during the first low-rate production phase of the program until the Navy demonstrates through flight testing that identified aircraft deficiencies have been corrected. This will still provide the Navy with the necessary aircraft to conduct operational testing of the F/A-18E/F. We also recommend that the Navy not begin Operational Test and Evaluation of the F/A-18E/F until corrections of deficiencies are incorporated in the aircraft that will be used for the evaluation. In commenting on a draft of this report, DOD partially concurred with both of our recommendations. Regarding our recommendation that no additional aircraft be contracted for until flight testing has demonstrated that aircraft deficiencies have been corrected, DOD stated that its testing to date has not identified any specific deficiencies that are predicted to prevent achieving an operationally effective level of performance. DOD also stated that it would ensure that the solution to the wing drop problem has been demonstrated before proceeding with full funding of the second low-rate production lot of the aircraft. DOD further stated that the Secretary of Defense has said that these funds would not be released until he is satisfied that the wing drop problem has been corrected. We believe the same level of commitment is needed relative to the other deficiencies that the F/A-18E/F Integrated Test Team has identified, such as the engine and weapon separation problems. Regarding our recommendation that Operational Test and Evaluation of the F/A-18E/F not begin until correction of deficiencies are incorporated in the aircraft to be used for operational evaluation, DOD stated that it agreed that operational evaluation should begin in May 1999 with production representative aircraft that have incorporated needed corrections. The underlying basis of our recommendation is that the Navy needs to demonstrate through flight testing that all the required fixes have been made and incorporated in the test aircraft before beginning Operational Test and Evaluation, even if the schedule needs to slip beyond May 1999. This approach would provide a sound basis for evaluating and quantifying the capabilities of the aircraft that will be provided to the fleet. This evaluation is particularly important because the F/A-18E/F will be the Navy’s primary fighter aircraft until the Joint Strike Fighter becomes available. A realistic comparison on the operational capabilities of the E/F with the newest F/A-18C/Ds currently in the fleet would provide the basis for a decision on how many E/F aircraft the Navy should ultimately procure as replacements for the C/D aircraft. In addition to its comment on our recommendations, DOD provided specific comments on other portions of our draft report. DOD’s comments and our response appear in appendix IV. As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies to interested congressional committees; the Secretaries of Defense and the Navy; and the Director, Office of Management and Budget. We will also make copies available to others upon request. Please contact me at (202) 512-4841 if you or your staff have any questions concerning this report. Major contributors to this report are listed in appendix V. To evaluate the status of the test program, we gathered and evaluated all F/A-18E/F flight test deficiency reports prepared as of December 4, 1997, by the F/A-18E/F Integrated Test Team. We interviewed the team’s management, the F/A-18E/F contractors (Boeing Corporation, St. Louis, Missouri, and General Electric Corporation, Lynn, Massachusetts), E/F program management, and the Navy’s Operational Test and Evaluation Force’s test personnel about the implications of documented program deficiencies on program cost, schedule, and performance. To determine which deficiency areas the Navy and the Program Risk Advisory Board determined to be risks to the F/A-18E/F program, we obtained Program Risk Advisory Board risk assessments and interviewed Board officials. We interviewed Navy program management and contractor officials about the implications of these risks on program cost, schedule, and performance. We discussed with the contractors their identified F/A-18E/F engineering and manufacturing development (EMD) and low-rate initial production program risks and the implications on program cost, schedule, and performance. We obtained detailed information on the potential cost, schedule, and performance impact of medium- to high-risk areas. We interviewed Defense Contract Management Command officials at Boeing and General Electric Corporations about their role in on-site monitoring and evaluation of the contractors’ F/A-18E/F development efforts, E/F deficiencies, and development risks facing the contractors. We also obtained documents in which the Command formally reported its findings to Navy headquarters. We interviewed Operational Test and Evaluation Force officials about their role in evaluating the E/F and plans for conducting future operational testing. To address F/A-18E/F development and procurement cost issues, we interviewed program and contractor officials responsible for financial matters and received briefings and answers to our questions concerning program cost. We conducted our review from May 1997 to January 1998 in accordance with generally accepted government auditing standards. The Integrated Test Team categorizes deficiencies it identifies during flight testing in either watch item, white sheet, or deficiency reports. Watch item reports document deficiencies that require design or software changes that need management attention. White sheet reports document deficiencies for which no fix has been identified, a fix has failed re-evaluation, or a fix impacts significant test events. Deficiency reports are submitted when an identified fix fails a second retest or time is needed to develop a plan of action. Initially, deficiencies are documented in watch item reports. If not corrected, they are sequentially escalated to a white sheet report and finally to a deficiency report. Therefore, the number of deficiencies in each of these categories changes continually as new deficiencies are identified, resolved, and moved among the categories. As of October 1997, the Integrated Test Team had categorized 370 deficiencies in watch item reports, 88 deficiencies in white sheet reports, and 30 deficiencies in deficiency reports. Deficiencies within each of these categories are also classified by their severity. The most severe of these classifications is a deficiency in which there is a high probability that it will cause aircraft control loss, equipment destruction, or injury to flight test personnel. Table III.1 shows the risks identified by Boeing in the F/A-18E/F EMD and low-rate initial production (LRIP) program and by the Navy’s F/A-18E/F Program Risk Advisory Board (PRAB) at the September 1997 Program Management Review. Blank cells indicate that Boeing or PRAB did not identify these as risk items as of September 1997. EMD risk (Boeing) LRIP risk (Boeing) Program risk (PRAB) Contractor quality assurance inspection transition Environmental control system aft center fuselage overheating Bleed cell 4 heat exchanger leak detection system Drift-free pressure transmitter set sensors Antenna producibility (Boeing) Antenna producibility (Northrop) Proposed specification change notice impact (Northrop) Engine full production qualification schedule Engine exhaust smoke at LRIP Engine bay fire extinguisher system Follow-on test and evaluation program definition (continued) EMD risk (Boeing) LRIP risk (Boeing) Program risk (PRAB) Ground station automated maintenance environment Multipurpose color display/up-front control display New technology producibility and performance Operational test requirements versus expected performance Operational test requirements versus specification performance Engine mounts (spares) The following are GAO’s comments to DOD’s letter dated February 9, 1998. 1. The first operational assessment, during which operational testers flew the E/F aircraft, was conducted by the Operational Test and Evaluation Force in November 1997. The preliminary report on that assessment, referred to as a Quick Look Report, identified 16 major deficiencies that must be corrected prior to the commencement of Operational Test and Evaluation. Further, the statements in our report concerning the possibility that the E/F might not achieve an operationally effective level of performance until identified deficiencies are corrected were taken directly from documents and reports prepared by the F/A-18E/F Integrated Test Team. 2. DOD’s comments stated that our final report should compare the consequences of not providing full funding for the second lot of LRIP aircraft because this would result in a production break and involve considerable costs. The Navy has not delivered any of the 12 aircraft being built under the first LRIP contract. The first aircraft is scheduled to be delivered in 1999, or about 20 months from the time of initial low-rate production decision, followed by the production of 1 aircraft per month until all 12 aircraft are completed. This schedule gives the Navy time to reassess its F/A-18E/F production plans. This reassessment should consider the cost and schedule tradeoffs of stretching out the production of the first 12 aircraft compared with proceeding with the current production schedule and accepting the potential for costly modifications and retrofits that may be required to correct current and future deficiencies. 3. We have revised the wording of our recommendation to clarify that we were referring to delaying Operational Test and Evaluation until corrections of deficiencies are incorporated in the aircraft that will be used for the evaluation. 4. DOD’s comments addressed the original test plan. Our report addressed the revised test plan. The point we make in our report is that the revised development test plan is focused on maintaining a development test schedule that will not cause delays in beginning the next phase of testing—Operational Test and Evaluation. Maintaining the test schedule will be a challenge because program documents state that E/F management anticipates that the remainder of the flight test program will experience an increase in testing requirements similar to what DOD’s comments stated has already occurred. This issue was addressed in an August 1997 flight test program review. The result of that review was that further increases in test requirements will have to be offset with corresponding reductions in the baseline test program. 5. We agree that finding discrepancies from predicted performance is the purpose of flight testing. However, inherent in the flight test program should be quantifying the effect that the correction of deficiencies will have on the E/F’s ability to meet its Key Performance Parameters. That is the underlying basis for our recommendation that no additional aircraft be produced until flight testing has validated the Navy’s predictions that the deficiencies being identified by the Integrated Test Team are resolved. 6. Our report addresses the need to determine the operational performance of the E/F after the correction of deficiencies have been incorporated in the aircraft. For example, the Blue Ribbon Panel that studied the wing drop problem stated that proposed fixes are expected to increase drag on the airplane, which could degrade the aircraft’s range. This finding is significant because range is one of the E/F’s Key Performance Parameters and one of the key improvements over the existing F/A-18C/D that the Navy cited in justifying the procurement of the E/F. Program management range estimates in January 1998 show that the F/A-18E has a slight range margin compared with F/A-18E/F threshold requirements (400 nautical miles versus 390 nautical miles with 2 external fuel tanks and 450 nautical miles versus 430 nautical miles with 3 external fuel tanks, respectively). The F/A-18F, which is heavier and has less internal fuel capacity than the E model, will have less range than the E model. The final operational performance of the E/F’s range and other Key Performance Parameters will not be known until all deficiencies have been corrected and their impact on the aircraft has been quantified. 7. We recognize that the March 1997 Operational Requirements Document contains the Key Performance Parameters that will be measured when evaluating the operational capabilities of the E/F. However, that document stipulates that the aerodynamic performance of the E/F is required to be as good as Lot XII F/A-18C/Ds. These C/D aircraft were built in the late 1980s and early 1990s. They are not as operationally effective as the more currently procured C/Ds that have been equipped with enhanced performance engines. 8. We reviewed the Operational Test and Evaluation Force’s Quick Look Report on the November 1997 operational assessment and could not verify DOD’s statement that the assessment found that the slight reduction in acceleration and maneuvering energy of the E/F had no significant tactical impact. Therefore, we discussed DOD’s statement with Operational Test and Evaluation Force officials who conducted the operational assessment. According to those officials, the Quick Look Report did not contain the cited conclusion. The officials cautioned, however, that they did not disagree with DOD’s comment because the operational impact of the E/F’s slight reduction in acceleration and maneuvering energy will depend on the specific mission profile (e.g., altitude and speed) and aircraft configuration (e.g., weapons being carried) that is being flown. In some cases, the C/D will out perform the E/F and vice versa. The officials also cautioned that its Quick Look Report was based on its preliminary analysis of limited data and stated that its evaluation of E/F operational capabilities might be modified after additional analyses are conducted. 9. We discussed the Cost Analysis Improvement Group’s March 1997 cost estimate with group members who prepared the estimate. These officials told us that the estimate was based on a total E/F buy of 1,000 aircraft and an annual peak production of 72 aircraft. The estimate was not based on the currently planned procurement of 548 aircraft and an annual peak production of 48 aircraft. Additionally, the officials told us that they did not factor in the increased development and procurement costs of upgrading the back seat of the F model to enable it to perform its assigned missions because the cost of the upgrade has not been determined. Furthermore, the March 1997 estimate, like the E/F program management estimate, used DOD-directed annual inflation rates and not the higher aerospace industry inflation rates that we discussed in our report. All of these factors understate the E/F cost estimates. 10. DOD’s December 1996 Selected Acquisition Report (the most currently available) shows that operation and support costs for a 12-aircraft E/F squadron will be about $3.2 million greater per year than a similar-sized F/A-18C/D squadron. This estimate represents an increase of over $1 billion when extrapolated over the E/F fleet and a 20- to 30-year service life. Therefore, we disagree with DOD’s comment that lower E/F operation and support costs will lower the E/F cost estimate. 11. In addition to the statements from the wing drop Blue Ribbon Panel that DOD included in its comments, the Panel stated that more flight test points are required in order to optimize the combination of fixes and to confirm the fixes at all points in the flight envelope. The Panel stated that this flight test approach was necessary because the underlying flow mechanisms of wing drop are not well understood due to the lack of adequate wind tunnel test techniques and practical computational procedures. In addition, the Panel stated that, although it is optimistic that an acceptable combination of fixes can be found, some of the more promising fixes will increase drag to some extent, may impact the observability characteristics, and may alter the design loads on the wing and flap components. The Panel further stated that these impacts must be quantified, and appropriate tradeoffs must be made to determine the optimum configuration and to assess the performance impacts. The Panel’s statements are consistent with the recommendations in our report. 12. DOD’s comment that the E/F program is committed to implementing all required fixes on the aircraft prior to Operational Test and Evaluation is based on DOD’s confidence that predictive tools will help resolve any radar cross section issues that might arise as a result of incorporating solutions to deficiencies. Our position is that solutions will not be known until they are assessed during flight testing rather than through simulation and modeling. Our position was substantiated by the Fiscal Year 1997 Annual Report of the Director, Operational Test and Evaluation, dated February 1998. The report stated that a challenge to the operational test program will be to design a strategy that will be able to determine if the F/A-18E/F will be more survivable than the F/A-18C/D, which is a key requirement of the E/F program. According to the report, existing models have many limitations in the ability to make this determination, and efforts to improve these predictive tools will not likely be mature in time to support the E/F program. 13. The engine fixes discussed in DOD’s comments have not yet been demonstrated and validated during flight testing, and DOD’s statement that the visible engine exhaust issue has been resolved for some time is not supported by program documents. In December 1997, the PRAB listed engine smoke as a medium-risk item that, if not corrected, will make the aircraft unacceptable for or jeopardize successful Operational Test and Evaluation. In addition, a December 1997 F/A-18E/F Propulsion and Power Program status report raised a number of recent engine concerns. The report stated that the major concern is keeping the engine development on schedule. Engine schedule slips to date could affect delivery of engines for the LRIP aircraft. Also, the engine is experiencing potentially problematic weight growth. The engine has reached its specification weight, and redesign changes to address a blade containment failure will cause the engine to exceed its specification weight. The program office has initiated a weight reduction study to identify ways to reduce engine weight by more than 56 pounds. In addition, the status report raised concerns about the engines inability to accept the growth necessary to accommodate the electronically scanned array radar that is a pre-planned product improvement for the E/F. According to the status report, a conscious decision was made to not design the engine for additional growth capability to avoid a major redesign of the back end of the aircraft to relocate the vertical tail. Taken in combination, these factors portray a less optimistic engine situation than indicated in DOD’s comments. 14. We discussed DOD’s statements with officials in the F/A-18E/F program office. The officials told us that the modification of the bomb release interval has not yet been flight tested. Also, weapon separation test data show that only about 21-percent of the testing has been done. It will not be known whether the weapon separation problem has been corrected until the testing has been completed. 15. We have revised our report to incorporate this information. 16. DOD’s comments discussed components testing. However, the vertical tail cannot be certified until the completion of tests of the tail attached to the aircraft. These tests are not scheduled to be completed until late 1999. 17. In a January 1998 program status report, program funding was listed as one of the major challenges facing the E/F program. The report stated that the EMD program is still funded at the “nothing goes wrong” level. Whether the EMD program will be completed within the congressional cost cap is not currently known. Steven F. Kuhta Jerry W. Clark William E. Petrick, Jr. Lawrence A. Dandridge Lillian I. Slodkowski The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Summary: Pursuant to a congressional request, GAO reviewed the F/A-18E/F development program, focusing on the: (1) status of the E/F development flight test program; (2) deficiencies that have been identified to date and corrective actions planned; and (3) current cost estimate for the program. GAO noted that: (1) the Navy has revised the F/A-18E/F flight test program by decreasing the data collection requirements that were originally planned; (2) program documents state that, although flight testing is behind schedule, program decisions to reduce test points will enable the Navy to regain lost time and complete development testing in November 1998, as originally planned; (3) F/A-18E/F program documents identified numerous deficiencies relative to the aircraft's operational performance; (4) the most challenging technical issue is wing drop; (5) until these issues are resolved through software or hardware changes that have been adequately tested, the cost, schedule, and operational performance impact of resolving these deficiencies cannot be determined; (6) the Navy remains confident that it can correct these deficiencies; (7) in addition, a Navy board that assesses risk areas in the E/F program stated in July 1997, that operational testing may determine that the aircraft is not operationally effective or suitable; (8) a December 1997 preliminary operational assessment report, which is classified and based on limited data and analysis, identified 16 major deficiencies with the E/F aircraft but concluded that the F/A-18E/F is potentially operationally effective and suitable; (9) the Navy has consistently stated that the F/A-18E/F will be developed and produced within the cost estimates established for the program; (10) certain key assumptions on which the cost estimate was made have been overtaken by events; (11) program documents state that the current development effort is funded based on the assumption that problems would not occur during testing; (12) unanticipated aircraft deficiencies have occurred, and most of the program's management reserve has been depleted; (13) since the flight test program has about 1 year remaining, it is probable that additional deficiencies will develop; (14) correcting current and potential future deficiencies could result in the development effort exceeding the congressional cost cap; (15) the Navy's F/A-18E/F unit procurement cost estimates are understated; (16) these cost estimates were based on what has become unrealistically high quantities of E/F aircraft that will be bought; and (17) more realistic assumptions indicate that, although the total procurement cost will decrease, the F/A-18E/F unit cost will be more than the Navy currently estimates.