KC-46 Tanker Aircraft: Program Generally Stable but Improvements in Managing Schedule Are Needed
Released: 2013-02-27T00:00:00

Letter

Background

In February 2011, Boeing won the Air Force competition to develop the next generation aerial refueling tanker aircraft. Boeing received a FPIF development contract with incentives to control cost while limiting the government’s financial liability. The development contract is to design, manufacture, and deliver four KC-46 tankers for flight testing. The Air Force expects to exercise contract options for the first production lot in 2015 and the second production lot in 2016, both needed for Boeing to produce and deliver 18 operational aircraft in the final production configuration by August 2017. In addition, all required training must be complete, and the required support equipment, and sustainment support in place by August 2017. Then the acquisition strategy calls for Boeing to produce the remaining aircraft through year 2027 at a target rate of 15 aircraft per year. Separate competitions may occur for later acquisitions, nominally called the KC-Y and KC-Z, to replace the rest of the KC-135 fleet and the KC-10 fleet (the Air Force’s other large tanker).

The KC-46 program’s acquisition strategy is to convert a commercial Boeing 767 airframe into a militarized aerial refueling tanker in two phases. In the first, Boeing is modifying their 767 airframe with a cargo door and an advanced flight deck display borrowed from the new Boeing 787 aircraft and calling this modified version the 767-2C. In the second, the 767-2C airframe will be further militarized by adding the air refueling capabilities, an air refueling operator station that includes panoramic three-dimensional displays, and threat detection and avoidance systems. Figure 1 depicts how the 767-2C aircraft is to be configured into the KC- 46 tanker.

The new KC-46 tanker is expected to be more capable than the KC-135 it replaces in several respects. It will have a modernized KC-10 tanker refueling boom integrated with a fly-by-wire (computer assisted) control system and a permanent hose and drogue refueling system that will enable both types of refueling to be employed on the same mission.KC-135 has to land and switch equipment to transition from one mode to another. Also, the KC-46 is expected to be able to refuel in a variety of night-time and covert mission settings and will have countermeasures to protect it against infrared missile threats. Designed with more refueling capacity, improved efficiency, and increased cargo and medical evacuation capabilities than its predecessor, the KC-46 is intended to provide aerial refueling to Air Force, Navy, Marine Corps, and allied aircraft. Appendix II compares, in more detail, the current capabilities of the KC-135 with the planned capabilities of the new KC-46 tanker.

The KC-46 program is one of only a few major weapon system programs in recent years to employ a fixed price development contract. In the past, DOD has typically used cost-reimbursement contracts in which the government pays all allowable costs incurred by the contractor. Recent legislation and defense policy now emphasize the use of fixed price development contracts, where warranted, to limit the government’s exposure to cost increases. Defense officials believe that a fixed price development contract is appropriate for this program because KC-46 development is considered to be a relatively low-risk effort to integrate mostly mature military technologies onto a well-defined commercial derivative aircraft.

KC-46 Program Cost, Schedule, and Performance Estimates Remain Unchanged but There Are Some Concerns as Development Progresses

The KC-46 program estimates for cost, schedule, and performance are essentially unchanged from last year. While the current cost estimate shows an increase of about $217 million for development and procurement combined, program officials explained this increase was unneeded funding that will be returned to DOD. The contractor is running very close to its budget and schedule. Development work is more than one-fourth complete and the program successfully accomplished its preliminary design review (PDR) on schedule. Also, the program has now implemented metrics, in response to our prior recommendation, to measure the development progress toward achieving its nine key performance parameters and projects it will meet those requirements by the end of development. However, there are two areas of concern at this point. First, both the contractor and the government estimate that Boeing’s development cost will exceed the contract ceiling price of $4.9 billion, and second, the contractor has already allocated about 80 percent of the management reserves budget set aside for known and unknown development risks with about 5 years of work remaining.

Quantities, Schedule, Cost, and Expected Performance Unchanged Since Program Start

Since the start of the KC-46 program about two years ago, planned aircraft quantities and key schedule events remain unchanged. The program’s latest cost estimate shows a total increase of $217 million in development and procurement, but Air Force officials told us that this is due to unneeded funding from DOD budget adjustments and the Tanker Replacement Transfer Fund. Officials said that the February 2011 cost estimate remains the program of record and that these funding additions will be returned to DOD. The program also completed a major schedule milestone in April 2012 with a successful PDR that determined no major design changes were needed. The PDR is an important risk reduction activity as the program moves into integration and manufacture of the KC- 46. Table 1 summarizes the planned quantities, costs, and milestone dates approved when the program began in February 2011 and the most current estimates in October 2012.

The current development cost estimate of $7.2 billion as reported in October 2012 includes $4.9 billion for the aircraft development contract and 4 test aircraft, $0.3 billion for the aircrew and maintenance training systems, and $2 billion for other government costs to include program office support, government test and evaluation support, contract performance risk, and other development risks associated with the aircraft and training systems. The total procurement cost estimate of $40.4 billion is to procure 175 production aircraft, initial spares, and other support items as priced in contract options. The military construction estimate of $4.3 billion includes the projected costs to build aircraft hangers, maintenance and supply shops, and other facilities to house and support the KC-46 fleet.

Through December 2012, Boeing has accomplished approximately $1.4 billion (28 percent) in development work and has more than $3.5 billion (72 percent) in estimated work to go over the next 5 years. Boeing reports it is running very close to its budget for cost and schedule. Minimal schedule delays to this point are attributed primarily to design difficulties with the aerial refueling system, hardware deliveries for the system integration labs, and configuration changes to the commercial platform needed to accommodate military specific hardware. DOD officials do not expect these delays to affect the completion of critical tasks that ultimately determine whether Boeing can meet the required delivery date in the development contract. While the schedule delays are relatively small, studies of more than 700 defense programs have determined there is limited opportunity for a program to get back on schedule once they are more than 15 to 20 percent complete. Also, the pace of development work is expected to accelerate. So far, on average, about $60 million worth of work has been completed per month, but over the next year more than $100 million worth of work per month is planned.

The program office currently projects that the KC-46 aircraft will meet the requirements of all nine key performance parameters by the end of development. These parameters are system characteristics considered critical or essential to developing an effective military capability. Satisfying these parameters will ensure that the KC-46 will be able to accomplish its primary mission of providing worldwide, day and night, adverse weather aerial refueling as well as its secondary missions. Several of these parameters address performance characteristics that are limited or do not exist in the current tanker fleet. For example, only 8 KC-135 aircraft have the capability to receive fuel from another aerial refueling tanker while airborne. At times this limits the range of the tanker force and lessens the efficient use of assets. The Air Force hopes to address this by enabling the entire KC-46 fleet to receive fuel from other tankers. Appendix III describes the key performance parameters for the KC-46 program.

Our report on the KC-46 last year noted that the program had yet to fully implement specific metrics needed to measure progress against these parameters. As a result, we recommended it do so as soon as possible to help ensure that progress toward meeting these parameters can be appropriately measured. The program has now fully implemented metrics to help measure progress toward achieving these parameters. Now that these metrics have been fully established we plan to track their status in our subsequent annual reviews of the KC-46 program.

Development Contract Cost Estimates Exceed the Contract Ceiling Price and the Majority of Funding to Alleviate Program Risk Has Been Allocated

The KC-46 development contract is designed to hold Boeing accountable for cost, limit the government’s financial liability, and provide Boeing incentives to reduce costs in order to earn more profit. At this point in the program, both the contractor and the government estimate that development costs will exceed the contract ceiling price of $4.9 billion and Boeing has already allocated about 80 percent of the contract’s management reserves, which are set aside for known and unknown development risks, with about five years of development work remaining.

Barring any changes to KC-46 requirements by the Air Force, the contract specifies a target price of $4.4 billion and a ceiling price of $4.9 billion at which point Boeing must assume responsibility for all additional costs.Currently, both Boeing and the Air Force project that the development effort will exceed the $4.9 billion ceiling price, with the Air Force concluding that the primary reason is the schedule risk associated with the remainder of the development effort. If so, the contractor will have to absorb all costs above this amount. Table 2 provides development contract details, current contractor and government estimates to complete, and the projected amounts over ceiling to be absorbed by the contractor.

The development contract performance baseline set aside $354 million in the management reserves account, about 7 percent of the contract ceiling price. As of December 2012, less than $72 million in unallocated reserves remain. About $282 million had been allocated, the majority to non- commercial militarization requirements, including: $94 million for increased system engineering and program $72 million for design and integration of military equipment with the $52 million for construction of system integration labs to reduce $42 million for additional test and evaluation challenges as well as cost growth for training activities, support equipment, and operational site activation.

Figure 2 shows the use of management reserves to date and future projections based on past use. Since the start of performance reporting in May 2011, the contractor has allocated an average of about $15 million in reserves each month.

The quick rate of depletion of Boeing’s management reserves raises concerns. Two years into a 7-year development program, the contractor has already allocated about 80 percent of the total available. Less than $72 million is available for future contingencies related to the more than $3.5 billion in government funded contract work remaining. DOD anticipates this negative trend will continue, since Boeing has told them design and technical issues driving the allocation of management reserves are not fully resolved.

At the current allocation rate, our analysis shows that management reserves will be depleted in May 2013, prior to the critical design review in July 2013 and more than 4 years before the contractually required delivery date for 18 operationally ready KC-46 aircraft in August 2017.

According to GAO’s Cost Estimating and Assessment Guide, significant use of management reserves early in a program may indicate contract performance problems and decreases the amount of reserves available for future risks, particularly during the test and evaluation phase when demand may be the greatest. At the current rate, none of the reserves will be available to complete the bulk of development work, as well as the entire period of development testing. Even though Boeing is contractually liable for all costs above the $4.9 billion ceiling price, unanticipated design changes, deficiencies discovered in testing, and other risks encountered that might require management reserves funding could place added pressures on cost and schedule as the development program moves forward. The program has not yet evaluated how the significant use of these funds early in development could impact future milestones.

Program Is Working On Some Testing, Design, Manufacturing, and Technical Challenges

With development generally proceeding as planned, the program is addressing, in varying degrees, some key challenges. All major stakeholders in the program have identified concerns about the degree of risk in the KC-46 flight test plans and an integrated test team is evaluating and adjusting test plans. Also, in preparing for the program’s critical design review, completing extensive engineering drawings on time will be challenging and some lower level subsystem design reviews are behind schedule. Furthermore, Boeing changed its plans and location for manufacturing and assembling military equipment and is still in the process of relocating key personnel and establishing needed facilities. While not as complex as a new fighter aircraft, the tanker program still needs to integrate critical technologies, develop and test software, keep the aircraft within its target weight, and at the same time, navigate risks posed by the concurrency, or overlap, between testing and production activities.

Flight Test Plans a Concern among Program Stakeholders

The Air Force, DOD, Boeing, and the FAA have all identified the aggressive KC-46 flight test schedule as a risk. Developmental flight tests, to prove aircraft design and demonstrate the aircraft will perform as expected, are set to occur within a 15-month window starting in early 2015 and ending in 2016. Concerns include the relatively short time for flight testing and fixing any deficiencies, plans for aircraft flying rates and personnel resources, and the time needed for air worthiness flight certifications.

To achieve developmental flight test plans, the contractor has proposed a 5- to 6-day-a-week flight test approach based on its commercial aircraft test practices. As part of this plan, Boeing intends to fly KC-46 test aircraft 5 days a week with a scheduled make-up day on day 6, and possibly an additional day, if necessary. This is a more aggressive pace than the two to three test missions per week for each test aircraft typically assumed by the Air Force for other aircraft programs. Air Force officials said their typical approach enables execution of test plans at a steady rate and includes sufficient time for data analysis, aircraft maintenance, aircrew training, and test planning between test flights. This issue and other flight test issues are being addressed by officials on the Integrated Test Team (ITT). The ITT was concerned about the Air Force’s access to contractor test documents and data but now has resolved this issue with Boeing. Also, the ITT is currently in the process of developing formal agreements with the Air Force, Navy, Boeing, and a foreign partner regarding the use of 18 military receiver aircraft to be used to certify the KC-46’s aerial refueling capabilities. Appendix IV provides a list of the top 8 flight test risks identified by the ITT and plans to mitigate those risks.

GAO-12-366. scheduled to begin in May 2016, be delayed by at least 6 months to allow additional time for completion of developmental testing and initial aircrew and maintenance training due to the aggressive flight test schedule. Their assessment also cites a concern that the current schedule for military flight testing may need to be extended by 4 to 7 months. In addition, the DOD development testing office has stated that the proposed flight test plan allots little time for correction of deficiencies discovered during development testing prior to the planned start of operational testing. In January 2013, both offices approved the KC-46 overall test strategy, but each office still has concerns regarding the program’s detailed test plans, including the training of aircrew and maintenance personnel overlapping with the completion of developmental testing.

The FAA still has to certify airworthiness for both the 767-2C and then the KC-46. The first FAA certification must approve the modifications being made to the original baseline 767 design and the second certification must approve the installation of military aerial refueling equipment for the KC-46 tanker configuration. Boeing established plans for the FAA to accomplish part of both of these certifications concurrently rather than consecutively which is the typical procedure. FAA officials said that such a condensed timeframe will require additional agency resources which the Air Force has to pay under a program services agreement. According to DOD risk assessments, if problems arise during this concurrent certification process, little time will be left for Boeing to recover from delays.

Completing Required Engineering Drawings for the Upcoming Critical Design Review May Be Challenging

The KC-46 critical design review (CDR) is scheduled for July 2013. The CDR is a major milestone that assesses the system’s final design so that the product can move into fabrication, demonstration, and test. It also verifies whether performance requirements can be met within cost and schedule constraints. To prepare for the complete system CDR, the contractor has been conducting individual subsystem design reviews during 2012, a few of which were delayed. The design review on the aerial refueling boom hardware, for example, has been rescheduled 5 months later than planned due to design problems. A design review of the software associated with the aerial refueling operator station has also been rescheduled. Also, an analysis to validate the software design was delayed from June 2012 until February 2013. Despite these slips, program officials told us subsystem reviews will still occur in time to support the July 2013 CDR. During the upcoming CDR, Air Force officials indicated they will also review and approve the contractor’s plans for system specifications, flight testing, and supplier management.

An important contractual requirement (and best practice) is for Boeing to release 90 percent of the total engineering design drawings by the CDR. At this point, Boeing data shows that, as of early December 2012, Boeing has completed the expected amount of drawings and about 60 percent of design drawings were complete, which is about 9,000 out of nearly 15,000 total drawings. However, given the time remaining, reaching the 90 percent requirement by CDR could be challenging. Drawings still to be completed include much of the more complex, new design efforts to integrate the military technologies on the commercial derivative airframe. Figure 3 shows that as of December 2012, Boeing is adhering to its schedule to complete the required number of drawings before CDR.

The location, facilities, and some personnel for militarizing the commercial-derivative aircraft have changed since the contract was awarded. The Air Force originally expected Boeing to do most of the work at its long-standing Wichita, Kansas, facility. In January 2012, however, Boeing announced plans to close its Wichita plant and move all military modification assembly work to Seattle (home to its commercial manufacturing operations) to achieve greater cost efficiencies and accommodate defense cuts. The company has relocated key personnel, and plans are in place for the remaining relocations. A temporary facility was opened in Seattle in October 2012 to begin production of the first refueling aircraft boom, and the company is also examining what additional facilities may be needed for future production and aircraft militarization. Air Force officials stated that Boeing must still meet all contractual requirements on-time and within cost, regardless of where development has been taking place, and has identified this transition as a watch item going forward.

Program Has Some Remaining Technical Challenges

The KC-46 program plans to integrate three critical technologies—a Three Dimensional Display, Airborne ESTAR, and Threat Correlation Software—needed to achieve the tanker’s capability requirements. These technologies have each been demonstrated in a relevant environment in accordance with DOD and statutory requirements, but have not yet been demonstrated in a realistic environment. Demonstrating technology in a realistic environment offers a higher level of maturity and is considered a best practice. We have previously reported that programs that began development with technologies demonstrated to this level experienced less cost growth than programs with less mature technologies.However, the program does not plan to demonstrate these critical technologies in a realistic, operational environment before production starts. Boeing is required to submit an update to its technology maturation plans prior to the CDR in July 2013 and the start of low-rate initial production in 2015. Appendix V describes the KC-46 program’s three critical technologies in detail.

Other technical challenges include:

Software development. Software development plans are still evolving and not complete at this time. While the total amount of software under development has been reduced by 40 percent since the start of development, increased amounts of certain types of software are now anticipated and the planned mix of software has also changed. Boeing now estimates they will be not be able to reuse as much existing software as they thought, and must instead develop more new and modified software. Table 3 shows the estimated changes to the KC-46 software plan. While needing less software overall is a positive, the need for more new and modified software typically requires more testing than software being reused. Growth in these two software classifications is not a favorable trend. However, at this point, officials do not expect that this will affect the software test schedule.

Aircraft weight. Projected weight increased since last year more than anticipated and is now expected to exceed the KC-46’s target weight. If the target weight is not achieved, the aircraft will not be able to carry as much fuel as required. Essentially, every one pound in excess weight equals one pound less fuel that can be carried to accomplish its primary mission of refueling other aircraft. Extra weight could also affect operating requirements for takeoff, mission radius, and landing. The program does have a mitigation strategy in place to help control weight. Historically, weapon system acquisition programs can experience weight gains during development. With about 5 years of development remaining, including the entire flight test program, additional weight reduction activities may be necessary.

Wing aerial refueling pod. Problems with buffeting, or instability, of the aircraft’s wing experienced on another Boeing tanker, led to the introduction of a new wing aerial refueling pod design for the KC-46. The new design also required changes in the way the refueling hose exits the pod, raising concerns whether the hose will be stable in flight. If the new design has technical shortcomings, additional cost and time for unplanned design changes and subsequent testing may be needed.

Boom refueling system. Some changes to the boom refueling system design have been determined necessary, possibly delaying the development of boom hardware. Boeing has added manpower to help manage its suppliers and is addressing risk. According to the program office, the boom refueling system is still on schedule, but if hardware is delayed, boom testing could be as well.

Concurrency Increases Schedule Risk

Successful and timely resolution of design, manufacturing, testing, and technical challenges could lessen and help manage cost and schedule risks posed by the concurrency, or overlap, of development, testing, and production activities. As we reported last year, funding commitments and the start of low rate production is scheduled before significant development and testing activities are completed. Currently, about 60 percent of the dedicated KC-46 development flight testing is planned for completion by the start of production. While not as extensive or potentially costly as we have reported elsewhere, KC-46 concurrency can have cost and schedule consequences. Development flight testing is supposed to demonstrate the maturity of the design and to fix design and performance problems during the development phase. Discovering and fixing such problems during production may require modifications to aircraft already built and, as a result, affect schedule. Figure 4 shows the program’s current schedule with concurrency between planned development, testing, and production.

Program Has Effective Mechanisms for Mitigating Risks and Conducting Program Oversight

The KC-46 program continues to mitigate risks principally through its acquisition strategy and contract mechanism. The use of a fixed price incentive contract limits government cost risk and the plan to convert a commercial derivative aircraft into the KC-46 tanker lessens technology risk. Our assessments of the KC-46 master schedule, the acquisition plan, and management framework find that they favorably compare with best practices and acquisition reform legislation, with some exceptions.

Program Contracting Method and Acquisition Strategy Mitigate Risks

The KC-46 fixed price development contract helps control costs by placing more responsibility on the contractor and limiting government liability for cost increases. Specific contract provisions ensure that Boeing must correct any deficiencies and bring them to the final aircraft configuration at no additional cost to the government. Program officials have also taken steps with the KC-46 acquisition strategy to limit changes to requirements similar to those that have caused problems in many previous acquisition programs. Now, an engineering or contract change affecting system requirements or possibly impacting program cost, schedule, and performance baselines must be approved by top Air Force officials. Program officials also maintain the program is being managed in an event-based manner—meaning the start of low-rate initial production is not driven by a certain schedule date, but instead will not be approved until Boeing demonstrates the knowledge and readiness required for production.

As DOD seeks to employ more fixed price development contracts, where appropriate, and acquire more evolutionary weapon system capabilities, different acquisition strategies and contract provisions come into play. At times, these factors, and others, can change the responsibilities among key stakeholders. For example, when using a strategy that couples a fixed price contract with a largely commercial-derivative system, the government generally has and needs less access to the contractor’s activities compared to when a purely military system is being acquired. KC-46 program officials told us the majority of preliminary and critical design reviews for components and subsystems used on the commercial 767-2C aircraft are conducted as Boeing internal events, with the Air Force participating but not leading these reviews. Also, officials told us they do not receive regular status updates for some types of contractor data, including engineering design drawings. Officials from the Defense Contract Management Agency, which oversees the KC-46 development contract, also told us their typical oversight of defense contracts involving commercial items is more constrained compared to its oversight of unique military weapon systems.

Program Development Schedule Mostly Meets Best Practices for Schedule Estimating

The KC-46 program’s schedule substantially meets the 10 best practices we have identified as being associated with effective schedule estimating. Our analysis found that while the schedule does not fully meet any of the 10 best practices, it is generally comprehensive, well- constructed with a logical sequence of activities, controlled, and credible. Table 4 summarizes these best practices and our assessment of the degree to which the KC-46 program has met them.

Overall, the evidence suggests that the KC-46 program office and Boeing are working to develop and maintain a healthy master schedule. Still, our assessment identified three areas that could be improved where the current schedule partially meets the best practices.

First, our analysis found that not all activities are reflected in the schedule. The schedule primarily contains Boeing’s activities but does not fully include government activities. To better satisfy best practices, a program’s schedule should reflect all efforts necessary for successful completion regardless of who performs them.

Second, we found a relatively large number of schedule lags and date constraints that may negatively impact the ability to predict how the delay or early completion of scheduled activities could affect the KC- 46 major milestones and planned finish date. While lags are included in a schedule to denote the passage of time, the KC-46 schedule provided few details on the reasons why such a large number of lags are included in their schedule. We also found a large number of constraints built into the schedule that are being used to control the timing of activities. Program officials stated these time constraints represent the contractor’s resource availability but they also prevent activities from starting sooner than planned to take advantage of time savings realized on earlier activities. It also requires constant manual upkeep of the schedule, increasing the likelihood of errors.

Third, while the program office adhered to the best practice that a risk analysis be conducted on its planned schedule and even conducted a second revised schedule risk analysis after our initial assessment, we report this best practice as partially met. Our initial assessment found that the original program office schedule risk analysis may not have factored in enough risk into the contractor’s activity duration estimates, and assumptions used to conduct the risk assessment were not fully documented. Nor did it account for the correlation between activities, that is, the degree to which the duration of some related activities may vary together. Program officials reported that they revised their original analysis during October 2012 to research differences and record assumptions between their schedule risk analysis and the contractor’s. According to the program office, the differences were resolved and assumptions are now documented, which addressed some of our earlier concerns. While the program office reported these constructive efforts, the revised schedule risk analysis still does not account for the correlation between activities, and we did not have the time or complete information to accomplish an independent assessment of the revised analysis.

Program Has Generally Adhered to Acquisition Best Practices and Reform Legislation

On the whole, the KC-46 program’s acquisition plan and management framework continue to favorably compare with the standards and requirements in GAO’s best practices work on acquisition development and reform legislation. The program has utilized: a time-defined development approach of about 7 years; a process requiring consultation with top Air Force officials before manufacturing readiness levels designed to provide a common measure and vocabulary for assessing manufacturing maturity and risk; and a knowledge-based acquisition approach, in which knowledge is acquired at key decision points, by ensuring requirements and resources match, the product design is stable, and manufacturing processes are mature.

As we reported last year, while the program has implemented many acquisition best practices, the Air Force did not conduct a technology development phase and instead proceeded directly into development. The program’s three critical technologies were assessed as approaching maturity and meeting internal defense policy, but below the fully mature level associated with best practices. Our prior work consistently shows that programs going directly into development before fully maturing all critical technologies typically incur additional costs and take longer to complete.

The program reports that it is also meeting many of the requirements of the Weapon Systems Acquisition Reform Act of 2009 (Reform Act) that encourages DOD to engage in a more robust discussion of trade-offs among cost, schedule, and performance. legislation, the program has implemented cost and schedule management requirements and is measuring and reporting on operation and sustainment costs. Further, they have also implemented cost effective measures including plans for future competitions for aircraft subsystems and several software integration laboratories. The Air Force is also employing an incremental acquisition approach, as mentioned in the Reform Act, to replacing the current refueling fleet by procuring the KC-46, and then later the KC-Y and KC-Z. This approach could leverage competition for future tanker aircraft development and procurement.

Pub. L. No. 111-23, § 201. technical, cost, and schedule delivery information to mitigate risk. Appendix VII provides a list of key Boeing suppliers for the KC-46 program. In addition, the Air Force has approved the contractor’s manufacturing program plan that leverages military modifications on the commercial 767 assembly line with Boeing planning to use a manufacturing readiness assessment and production reviews for quality assurance.

Conclusions

Entering its third year, the KC-46 development program is, for the most part, progressing as planned even though some concerns exist. The program has an ambitious schedule, particularly with regard to flight testing. While program estimates are essentially unchanged, the development contract cost estimate continues to be above the contract ceiling price, making it essential the government not change KC-46 requirements. Boeing has also allocated management reserves at a high rate which raises concerns because doing so early in a program is often an indicator of future contract performance problems. While the fixed price development contract caps the government’s cost liability, it would still behoove the Air Force to fully understand the causal factors driving the accelerated use of management reserves in order to recognize risks, consider potential trade-offs, and better understand circumstances that could impact on-time delivery to the warfighter. Also, improvements to a few aspects of the program’s master schedule could make it more complete and robust to further help ensure program success.

Recommendations for Executive Action

We are recommending that the Secretary of Defense take the following two actions on the KC-46 program.

To help understand and monitor the causes of the majority of contractor management reserves being allocated two years into development, the Secretary of Defense should direct the Air Force, after Boeing has fully resolved the relevant design and technical issues, to analyze the root causes for the rate of expenditure of reserves in order to help the Air Force fully recognize and mitigate risk areas.

To help maintain a more thorough and insightful KC-46 development schedule, the Secretary of Defense should direct the Air Force to address our concerns related to three schedule best practices (capturing all activities, sequencing all activities, and conducting a schedule risk analysis), where we concluded the program’s master schedule had only partially met best practice criteria.

Agency Comments

DOD provided us with written comments on a draft of this report which are reprinted in appendix VIII. DOD concurred with our two recommendations. We also incorporated technical comments from DOD as appropriate.

Regarding the recommendation to analyze the use of management reserves, DOD stated that the contractor has performed a root cause analysis and that the program office will monitor, analyze, and report on the use of management reserves. Regarding the recommendation to improve the master schedule, DOD stated that the Air Force is taking action to address each of the 3 schedule best practices we cited.

We are sending copies of this report to the Secretary of Defense; the Secretary of the Air Force; and the Director of the Office of Management and Budget. The report also is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff has any questions concerning this report, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff contributing to this report are listed in appendix IX.

Appendix I: Scope and Methodology

We interviewed officials from the KC-46 program, Air Force, and Office of the Secretary of Defense (OSD) to obtain their views on KC-46 development progress, ongoing concerns and actions taken to address program technical risks, and future plans to complete KC-46 flight testing and manufacturing. We also reviewed program documentation and plans for compliance with current Department of Defense (DOD) policy, acquisition reform legislation, and GAO best practices for weapon system development.

To determine the extent the KC-46 program is meeting cost, schedule, and performance goals in the calendar year of this review (2012), we reviewed briefings by program and contractor officials, financial management documents, defense acquisition executive summary reports, selected acquisition reports, monthly activity reports, technical performance indicators, risk assessments, and other documentation. To evaluate cost information, we reviewed program office documentation on what actions are currently being taken in the areas of earned value management and contractor performance. We also reviewed contractor use of management reserves funding to project when this funding will likely be depleted. To assess the program’s development schedule progress, we reviewed the program’s latest master schedule and compared it against previous master schedules and reviewed monthly Defense Contract Management Agency (DCMA) reports for information relating to a 14-point schedule risk assessment they conduct as part of their ongoing oversight to identify changes that could impact key program milestone events. Regarding performance goals, we reviewed key performance parameters and progress in fully implementing technical performance measures used to evaluate whether program performance parameters are being achieved.

To identify the design, manufacturing, testing plan, and technology challenges, we met with contractor, Air Force, and DOD officials and examined program documentation such as the post preliminary design review report, manufacturing program plan, software block functionality plan, reliability growth curve charts, and critical technology element maturation plan. To measure progress regarding the relocation of military modification work on the KC-46 tanker, we met with Boeing officials and reviewed briefings on their transition plans from Wichita, Kansas to Everett, Washington and the implications on expertise, knowledge, personnel relocation, and associated costs and impact on program development and production. To assess progress toward test plans, we compared the revised KC-46 test and evaluation master plan to the original and against recent reviews completed by the Air Force and DOD test offices to evaluate whether changes have been made to mitigate flight testing concerns. Specifically, we analyzed contractor and program office mitigation efforts planned to deal with identified flight test challenges of the commercial derivative 767-2C and KC-46 tanker and also Federal Aviation Administration (FAA) air worthiness certification requirements. We also discussed related software development, test, and integration with the DCMA, Director, Operational Test, and Evaluation (DOT&E) officials, and OSD Developmental Test and Evaluation (DT&E) officials and reviewed DOT&E and DT&E annual assessments on the KC- 46 program, integrated test team minutes, program assessment reports, and contractor and program office risk summary charts to identify risk areas and what actions are being taken to address challenges to the program. Finally, we leveraged information received as part of a related GAO review on weapon system assessments, which includes the KC-46 program.

To assess the extent the program has developed effective, appropriate methods to mitigate challenges, we analyzed contractor and program office plans for risk mitigation contained on their respective risk and technical watch lists. We reviewed a preliminary design report and briefing and subsystems critical design review results. We concentrated especially on follow-up actions (redesigns) the program has identified as a need to address, the program’s critical technology maturation plan, key performance parameter achievement concerns and operational assessment criteria used to note significant trends in development efforts, programmatic voids, risk areas, and operational testing plans. We compared these risk mitigation plans to GAO’s commercial best practices work on weapon system programs, focusing on knowledge points, technology readiness levels, software development, and testing to identify processes and trends that provide a framework for improving weapon system development outcomes. Finally, to analyze risk in the program’s development schedule, we reviewed the integrated master schedule and determined the extent to which the program’s development schedule was prepared in accordance with best practices that GAO has identified as fundamental to having a reliable schedule. We then characterized the extent to which each of the 10 scheduling best practices were met; that is, we rated each characteristic as being either: not met, minimally met, partially met, substantially met, or fully met. We could not assess the contractor’s manufacturing processes because the program is only in its second year of development and it is too early for this assessment.

In performing our work, we interviewed officials from Air Mobility Command, Scott Air Force Base, Illinois; Air Force Operational Test and Evaluation Center, Detachment 5, Edwards Air Force Base, California; 412th Test Wing, Edwards Air Force Base, California; KC-46 program office, Wright-Patterson Air Force Base, Ohio; Defense Contract Management Agency, Seattle, Washington; and Federal Aviation Administration, Wichita, Kansas. We also met with and obtained information from the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics, in Washington, D.C; the Director of Operational Test and Evaluation, Washington, D.C.; and the Deputy Assistant Secretary of Defense for Developmental Test and Evaluation, Washington, D.C.

We conducted this performance audit from June 2012 to February 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comparison of Current KC-135 versus Planned KC-46 Performance Capabilities

Not capable of both on same mission.

Appendix III: Description of KC-46 Key Performance Parameters

Description Aircraft shall be capable of accomplishing air refueling of all Department of Defense current and programmed (budgeted) receiver aircraft. The aircraft shall be capable of conducting both boom and drogue air refueling on the same mission.

Aircraft shall be capable of carrying certain amounts of fuel (to use in air refueling) certain distances.

Aircraft shall be capable of worldwide flight operations in all civil and military airspace.

Aircraft shall be capable of transporting certain amounts of both equipment and personnel.

Receiver Air Refueling Capability Aircraft shall be capable of receiving air refueling from any compatible tanker aircraft.

Aircraft shall be able to operate in chemical and biological environments.

Aircraft must be able to have effective information exchanges with many other Department of Defense systems to fully support execution of all necessary missions and activities.

Aircraft shall be capable of operating in hostile threat environments.

Aircraft shall be capable of conducting drogue refueling on multiple aircraft on the same mission.

Appendix IV: Top Risks Identified by the KC- 46 Integrated Test Team and Proposed Mitigation Efforts

Risk Risk issue 1

Military Type Certification Schedule and Flight Test Rate: Planned military-specific test schedule is more aggressive than historical experience. Planned flight hours per aircraft per month average (50 hours vs. approximately 30 hours) and test efficiency exceeds that for similar aircraft (85 percent vs. 55 percent).

Air Force mitigation efforts Integrated Test Team (ITT) clarification: Approximately 90 percent of aircraft testing will be achieved under the Federal Aviation Administration (FAA) process. Previous commercial 767 aircraft test programs have significantly exceeded 30 flight hours per aircraft per month and 55 percent efficiency.

Receiver Certification Planned Flight Test Hours: Aerial refueling certification testing of required Initial Operational Test and Evaluation (IOT&E) receiver aircraft will take longer (approximately 400 flight hours) than the hours and time in the current schedule. This is estimated to extend the test schedule by almost 6 to 8 months.

ITT clarification: Boeing flight test hours are indicative of actual receiver time on station, not total flight time. The method of calculating test hours is now understood correctly.

Access to Boeing Data and Personnel: Integrated Test Team requires access to Boeing proprietary commercial test data and commercial division personnel. Air Force Flight Test Center is concerned about Boeing proprietary Test Planning, Execution, and Reporting Tool interfacing with government tracking tools.

ITT resolution: Responsible Test Organization (RTO) and Boeing reached agreement that test planning will be conducted by Boeing. A common test and evaluation database is being implemented through Boeing’s Integrated Digital Environment (IDE) database system.

Responsible Test Organization (RTO) Crew Staffing: RTO crew personnel could impact test operations schedule due to availability of RTO personnel. There are no contractual Type 1 (one-time or limited training) slots for replacement training and there have been issues with retention of crews due to permanent change of station and deployments.

ITT resolution: Boeing has agreed to include a 10 percent replacement training factor (by specialty) for all Type 1 training. An executable training plan has been built within the fixed-price construct. Increases in Type 1 training allocations for the 412th Test Wing is to be offset by personnel sharing.

Developmental Testing (DT) / Operational Testing (OT) Maintenance Staffing: Insufficient number of qualified tanker maintenance personnel to support DT and IOT&E. Air Mobility Command and Air Force Materiel Command (AFMC) planning and programming effort required to meet requirement of 100 maximum slots to support IOT&E and 15 additional slots for DT.

ITT resolution: Initial staffing assumptions by the RTO and Air Force Operational Test and Evaluation Center (AFOTEC) did not support the pace of test outlined by Boeing’s Stage 2 flight test plans. RTO has since added additional manning slots for DT and will share resources with AFOTEC to meet OT demands. The KC-46 Program Office has adjusted Type 1 training requirements to the new demand.

DT/OT Aerial Refueling Operator (ARO) Personnel: Insufficient number of qualified ARO personnel to support DT and IOT&E activities. Potential AFMC programming and budgeting input required for additional ARO positions for DT and OT activities.

ITT resolution: Initial staffing assumptions by the RTO and AFOTEC did not support the pace of test outlined by Boeing’s Stage 2 test plans. RTO has since added additional manning slots for DT and will share resources with AFOTEC to meet OT demands. The KC-46 Program Office has adjusted Type-I training requirements to the new demand. Additionally, the 370 Flight Test Squadron remained activated to support KC-46 testing.

Schedule Sufficiency for Deficiency Correction: Flight- test schedule does not reflect sufficient time to correct discrepancies identified prior to start of operational testing. There is a lack of Integrated Test Team insight into schedule and contract and concerns whether the Joint Deficiency Reporting System can support pace of test operations.

ITT clarification: Although a Correction-of-Deficiency period is not identified on the current KC-46 Test and Evaluation Master Plan (TEMP) test schedule due to a lack of available space, there is a 90-day aircraft refurbishment period of time to bring the four test aircraft to production configuration. This time is annotated on the detailed test schedules provided by Boeing.

Risk Risk issue 8

Receiver Aircraft Availability: Lack of military receiver aircraft availability to support aerial refueling certification testing. Air Force will have to maximize flexibility of receiver aircraft for air refueling certification testing by possibly utilizing Navy, Boeing, and foreign partner F/A-18 aircraft.

Air Force mitigation efforts ITT clarification: Memorandums of Agreement / Memorandums of Understanding (MOAs/MOUs) will be implemented between organizations to facilitate support of test activity. MOAs are expected with owning aircraft using commands to support receiver certifications.

Appendix V: KC-46 Critical Technology Elements

Appendix V: KC-46 Critical Technology Elements Description The display screens at boom operator stations inside the aircraft provide the visual cues needed for the operator to monitor the aircraft being refueled before and after contact with the refueling boom or drogue. The images of the aircraft on the screens are captured by a pair of cameras outside that aircraft that are meant to replicate the binocular aspect of human vision by supplying an image from two separate points of view, replicating how humans see two points of view, one for each eye. The resulting image separation provides the boom operator with greater fidelity and a more realistic impression of depth, or a 3rd dimension.

Testing to Date Similar technology has been used on two foreign-operated refueling aircraft and a representative model in tests with other Boeing tankers.

This software module is planned to have an algorithm that allows for automatically re-routing and constructing new flight paths for the aircraft that are safe, flyable, and avoid potential threats. The algorithm is new and novel technology, critical to meeting operational requirements.

Airborne ESTAR has been tested in a simulation that provided data on its performance, interfaces, and functionality.

Somewhat similar to Airborne ESTAR, this new software module serves to correlate tracks from multiple potential threats and automatically help re-route the tanker’s flight path to avoid them.

The integration of software algorithms with the associated processor has been laboratory tested in a relevant environment.

Appendix VI: Description of GAO Scheduling Best Practices

Criterion (1) Capturing all activities

Explanation The schedule should reflect all activities as defined in the project’s work breakdown structure (WBS), which defines in detail the work necessary to accomplish a project’s objectives, including activities both the owner and contractor are to perform.

(2) Sequencing all activities

The schedule should be planned so that critical project dates can be met. To do this, activities need to be logically sequenced-that is, listed in the order in which they are to be carried out. In particular, activities that must be completed before other activities can begin (predecessor activities), as well as activities that cannot begin until other activities are completed (successor activities), should be identified. Date constraints and lags should be minimized and justified to help ensure that the interdependence of activities that collectively lead to the completion of events or milestones can be established and used to guide work and measure progress. (3) Assigning resources to all activities The schedule should reflect the resources (labor, materials, overhead) needed to do the work, whether they will be available when needed, and any funding or time constraints.

(4) Establishing the duration of all activities

The schedule should realistically reflect how long each activity will take. When the duration of each activity is determined, the same rationale, historical data, and assumptions used for cost estimating should be used. Durations should be reasonably short and meaningful and allow for discrete progress measurement. Schedules that contain planning and summary planning packages as activities will normally reflect longer durations until broken into work packages or specific activities.

(5) Verifying that the schedule can be traced horizontally and vertically

The detailed schedule should be horizontally traceable, meaning that it should link products and outcomes associated with other sequenced activities. These links are commonly referred to as “hand-offs” and serve to verify that activities are arranged in the right order for achieving aggregated products or outcomes. The integrated master schedule (IMS) should also be vertically traceable-that is, varying levels of activities and supporting subactivities can be traced. Such mapping or alignment of levels enables different groups to work to the same master schedule.

(6) Confirming that the critical path is valid

The schedule should identify the program critical path-the path of longest duration through the sequence of activities. Establishing a valid critical path is necessary for examining the effects of any activity’s slipping along this path. The program critical path determines the program’s earliest completion date and focuses the team’s energy and management’s attention on the activities that will lead to the project’s success.

(7) Ensuring reasonable total float

The schedule should identify reasonable float (or slack)-the amount of time by which a predecessor activity can slip before the delay affects the program’s estimated finish date- so that the schedule’s flexibility can be determined. Large total float on an activity or path indicates that the activity or path can be delayed without jeopardizing the finish date. The length of delay that can be accommodated without the finish date’s slipping depends on a variety of factors, including the number of date constraints within the schedule and the amount of uncertainty in the duration estimates, but the activity’s total float provides a reasonable estimate of this value. As a general rule, activities along the critical path have the least amount of float. (8) Conducting a schedule risk analysis A schedule risk analysis uses a good critical path method (CPM) schedule and data about project schedule risks and opportunities as well as statistical simulation to predict the level of confidence in meeting a program’s completion date, determine the time contingency needed for a level of confidence, and identify high-priority risks and opportunities. As a result, the baseline schedule should include a buffer or reserve of extra time.

Explanation Progress updates and logic provide a realistic forecast of start and completion dates for program activities. Maintaining the integrity of the schedule logic at regular intervals is necessary to reflect the true status of the program. To ensure that the schedule is properly updated, people responsible for the updating should be training in critical path method scheduling.

(10) Maintaining a baseline schedule

A baseline schedule is the basis for managing the project scope, the time period for accomplishing it, and the required resources. The baseline schedule is designated the target schedule, subject to a configuration management control process, against which project performance can be measured, monitored, and reported. The schedule should be continually monitored so as to reveal when forecasted completion dates differ from planned dates and whether schedule variances will affect downstream work. A corresponding baseline document explains the overall approach to the project, defines custom fields in the schedule file, details ground rules and assumptions used in developing the schedule, and justifies constraints, lags, long activity durations, and any other unique features of the schedule.

Appendix VII: Key Boeing Suppliers for the KC-46 Program

Supplier and location Cobham (Davenport, Iowa)

DRS Laurel Technologies Inc. (Johnstown, Pa.) Eaton Aerospace (Grand Rapids and Jackson, Mich.)

GE Aviation Systems (Grand Rapids, Mich. and Clearwater, Fla.)

Goodrich (Colorado and Ontario, Canada) Honeywell (Phoenix and Tucson, Ariz.; Coon Rapids, Mich., and Urbana, Ohio)

Moog Inc. (East Aurora, N.Y., Torrence, Calif., and Wolverhampton, UK)

Northrop Grumman (Rolling Meadows, Ill.) Parker Aerospace (Ariz., Calif., Florida, Ga., Mich., N.Y., N.C., Ohio, Tex., and Utah)

Pratt & Whitney (Middletown, Conn.) Raytheon Company (El Segundo, Calif.) Rockwell Collins (Cedar Rapids, Iowa)

Spirit (Wichita, Ks. and Prestwick, Scotland)

Triumph Group Inc. (Berwyn, Pa.)

Appendix VIII: Comments from the Department of Defense

Appendix IX: GAO Contact and Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact name above, the following staff members made key contributions to this report: Bruce Fairbairn, Assistant Director; Tina Cheng; Matt Drerup; Keith Hudson; John Krump; Don Springman; and Robert Swierczek.