Defense Management: Opportunities to Enhance the Implementation of Performance-Based Logistics
Published: 2004-08-16T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD) is pursuing a policy that promotes performance-based logistics at the platform level as the preferred product support strategy for its weapon systems, based in part on DOD's perception that this is an industry best practice. GAO was asked to compare industry practices for activities using complex and costly equipment with life-cycle management issues similar to those of military systems to identify lessons learned that can be useful to DOD. This is the first of two reports addressing DOD's implementation of performance-based logistics and is intended to facilitate DOD's development of new guidance on the use of this approach.
section_title: What GAO Found
paragraphs: DOD's current policy for implementing performance-based logistics as a preferred support approach at the weapon system platform level does not reflect the practices of private-sector companies that support expensive and complex equipment with life-cycle management issues. The companies GAO interviewed use performance-based contracting as a tool rather than as a preferred support concept at the weapon system platform level. While 7 of the 14 companies GAO interviewed use some type of performance-based contracting, they use it at the subsystem or component level--for commodities such as engines, wheels, and brakes--when it is cost-effective and reduces risk in a noncompetitive environment. DOD's proposed policy of pursuing performance-based logistics as the preferred support approach at the platform level results in contracting out the program-integration function--a core process the private-sector firms consider integral to successful business operations. Further, this proposed policy could limit opportunities to take advantage of competition when it is available for subsystems or components as well as limit opportunities to gain purchasing power from volume discounts on components across an entire fleet and avoid the administrative costs charged by a prime integrator. While DOD is proposing the aggressive use of performance-based logistics on both older and new weapon system platforms, the companies GAO interviewed use performance-based contracting at the subsystem or component level when it is cost-effective--often in a noncompetitive environment when the manufacturer controls expensive repair parts, such as engines. In general company officials said they rely more widely on other contracting vehicles, such as time and material contracts, particularly for new systems. Company officials noted that in the absence of accurate and reliable information on system performance to establish a baseline for evaluating the cost-effectiveness of a performance-based contract for new systems, the risk of the negotiated price's being excessive is increased. The companies GAO interviewed also emphasized the importance of having rights to the technical data--such as maintenance drawings, specifications, and tolerances--needed to support the management of all logistics contracts and, should the service provider arrangements fail, to support competition among alternate providers. In contrast, DOD program managers often opt to spend limited acquisition dollars on increased weapon system capability rather than on rights to the technical data--thus limiting their flexibility to perform work in-house or to support alternate source development should contractual arrangements fail.

section_title: Letter
paragraphs: In the past 4 to 5 years, the Department of Defense (DOD) has taken steps  to manage the total life-cycle support costs of its weapon systems and to  improve logistics support to the warfighter by reengineering its processes  for both acquiring and sustaining weapon systems. As part of these  reengineered processes, DOD has directed weapon system program  managers to develop acquisition strategies that maximize competition,  innovation, and interoperability and to capitalize on commercial  technologies to reduce costs. Within the area of weapon system  sustainment, DOD is pursuing a policy to implement a concept it calls  performance-based logistics as the preferred support strategy for DOD  weapon systems. This concept is a variation on other contractor logistics  support strategies calling for long-term support of military systems by  the systems’ manufacturers. The concept involves defining a level of  performance for a weapon system already fielded or about to be fielded  that is to be achieved over a fixed period of time for a fixed level of annual  funding. More recently, in February 2004, the Deputy Secretary of Defense  issued a memorandum promoting a more aggressive implementation of  performance-based logistics that was in part based on the perception that  this is an industry best practice.
As requested, we are reviewing DOD’s process of implementing  performance-based logistics as the preferred support strategy for its  weapon systems. As a part of this review we determined what types of  contractor logistics support arrangements the private sector uses for  activities that have complex and costly equipment with life-cycle  management issues similar to those of military systems, and what  potential lessons can be learned from a comparison between private- sector contractor support practices and the contractor logistics support  practices DOD is urging the services to implement. While conducting our  work, we learned that DOD soon will be issuing additional policy guidance  on its use of performance-based logistics. This is the first of two GAO  reports addressing DOD’s implementation of performance-based logistics  and is intended to provide the Secretary of Defense with  recommendations that should facilitate DOD’s development of new  guidance. Our follow-on report in early 2005 will determine similarities  and differences in the way the identified DOD programs are structured  and managed, identify approaches that appear to offer the greatest  opportunities to achieve cost effectiveness, and evaluate the demonstrated  cost savings or improved responsiveness of the new DOD concept.
As a part of this review, we examined Office of the Secretary of Defense  and service policies and guidance; collected data on performance-based  logistics programs identified by the services; and conducted case studies  on a limited number of the programs. We also reviewed the logistics- contracting practices of 14 private-sector companies from the air carrier,  maritime shipping, energy exploration, mining, and entertainment  industries—companies that use complex and costly equipment with  life-cycle issues similar to those of military weapon systems and that are  motivated by the desire to minimize costs and maximize profits to choose  the most cost-effective option. We held group discussions covering  standard questions about the industries’ contractor logistics support  practices, and we compared the results of these interviews with the  preliminary information obtained from our analyses of DOD policies and  programs. We reviewed the reliability of the projected cost and savings  data used in this report and determined that it was sufficient for our  purposes. We performed our work from September 2003 through June  2004 in accordance with generally accepted government auditing  standards. The scope and methodology section contains more detailed  information about the work we performed.
subsections: section_title: Background
paragraphs: Performance-based logistics is the DOD term for the process of  (1) identifying a level of performance required by the warfighter and  (2) negotiating a performance-based contract between the government  and the product support integrator—that is generally the original  equipment manufacturer of the total system—to provide long-term total  system support for a weapon system at a fixed level of annual funding.  Instead of buying spares, repairs, tools, and data in individual transactions,  the method in a performance-based logistics arrangement is to buy a  predetermined level of availability that meets the warfighter’s objectives.  To implement performance-based logistics, DOD selects a product support  integrator to serve as the single point of accountability, integrating support  from all sources to achieve the performance outcome metrics specified  in the performance-based support agreement. The metrics used include  operational availability (a measure of the degree to which an item is in an  operable state and can be committed at the start of a mission when the  mission is called for at an unknown point in time); mission capability  (the material condition, indicating that it can perform at least one and  potentially all of its designated missions); and customer wait time  (the total elapsed time between issuance of a customer order and  fulfillment of that order). For example, the Navy now uses two metrics for  its performance-based contract for the T-45 aircraft system—“ready for  training,” which requires that the contractor have a minimum number of  aircraft ready for training at 7:00 AM each business day in order to achieve  a 57 percent aircraft availability; and “sortie completion,” which requires  that the contractor meet 98 percent of the requirements for the scheduled  training flights. As an incentive, the contract pays a performance bonus  (maximum of $5 million annually) if the contractor exceeds the  performance metrics. If the contractor only meets—or fails to meet—the  minimum metrics, the contractor then receives none of the annual  performance bonus.
DOD Directive 5000.1, the Defense Acquisition System, highlights the  department’s preference for using performance-based logistics at the  platform level, stating, “Program Managers shall develop and implement  performance-based logistics strategies that optimize total system  availability while minimizing cost and logistics footprint.” As part of its  implementation of this strategy, in 2003 DOD proposed that the Congress  adopt legislative changes that would allow the services to increase the  appropriations allocation flexibility within a weapon system program,  allowing the program manager to use funds from different accounts  (such as operation and maintenance; research, development, test, and  evaluation; and procurement) to pay for system support costs. Although  this proposal was not adopted, DOD continues to pursue various avenues  that would support the overall objective of having greater flexibility by  using a single line of support funding managed by the program office for  total system operation and maintenance costs. Most recently, on  February 4, 2004, the Deputy Secretary of Defense (1) directed the  Under Secretary of Defense (Acquisition, Technology, and Logistics) in  conjunction with the Under Secretary of Defense (Comptroller) to issue  clear guidance on purchasing using performance criteria; and (2) directed  each service to provide a plan to aggressively implement performance- based logistics, including transferring appropriate funding as needed, on  current and planned weapon system platforms for fiscal years 2006–2009.
While this directive does not preclude the services from using  performance-based logistics contracts below the platform level, it does  express DOD’s intent to apply the concept at the platform level as a  preferred practice. As we discuss in the next section, DOD has established  separate goals for implementing performance-based service contracts, and  the services have identified many contracts as performance-based logistics  arrangements that are, in fact, below the platform level. However,  according to Office of Secretary of Defense officials, DOD would like to  implement performance-based logistics at the platform level to move from  contracting for material availability to weapon system availability. DOD  considers that the platform level offers the metrics needed to implement a  true performance-based logistics arrangement.
subsections: section_title: DOD Performance-Based Logistics Evolving from Its Use of Performance-Based Service Contracting
paragraphs: The Office of Management and Budget indicates that performance-based  service contracting, from which performance-based logistics has evolved,  has been referenced in regulation, guidance, and policy for more than two  decades, and federal agencies have used performance-based contracting to  varying degrees for acquiring a range of services. In 1991 the Office of  Management and Budget issued a policy letter establishing the use of a  performance-based approach for service contracting, and in 1994 it  initiated a governmentwide pilot project to encourage the use of  performance-based service contracts in federal agencies, including DOD.
The use of performance-based service contracts to acquire services offers  a number of potential benefits, particularly when services are acquired  by means of a fixed price agreement. Performance-based contracts can  encourage contractors to be innovative and to find cost-effective ways of  delivering services for a fixed level of funding. By shifting the focus from  process to results, these contracts can potentially produce better  outcomes and reduced costs.
In view of the potential benefits, Congress has been encouraging greater  use of performance-based service contracting. In an August 2003  memorandum to the military departments, the Under Secretary of Defense  (Acquisition, Technology and Logistics) stated that DOD should continue  to increase its use of performance-based service acquisitions. He noted  that DOD has a goal to award 50 percent of contract actions and dollars  using performance-based specifications by fiscal year 2005.
The more specific concept of performance-based logistics as an approach  for supporting military systems emerged from DOD’s 1999 study, Product  Support for the 21st Century, which identified 30 pilot programs (10 in  each military department) to test logistics support reengineering concepts  that placed greater reliance on the private sector. Many of the pilots  involved various types of contractor logistics support, prime vendor  support, or performance-based type arrangements. Others focused on  including reduced operation and support costs and improved readiness  as performance requirements for new system development. The  September 30, 2001, Quadrennial Defense Review advanced DOD’s  move toward this concept by advocating the implementation of  performance-based logistics with appropriate metrics that would be  designed to compress the supply chain and improve the readiness of  major weapon systems and commodities.  A November 2001 Office of the  Deputy Under Secretary of Defense document, Product Support for the  21st Century: A Program Manager’s Guide to Buying Performance,  intended as a guide for program managers, stated that program managers  will implement performance-based logistics on all new systems and on  acquisition category I and II fielded systems selected on the basis of a  sound business case.
It is unclear how many performance-based logistics programs the  services have implemented. In response to our inquiries, the Army  identified 74 performance-based logistics programs, the Navy identified  106, the Air Force 4, and the Marine Corps 1. We noted a broad range of  contract arrangements is identified under the performance-based logistics  umbrella, with many of them initiated under a different name, such as  contractor logistics support or total systems support responsibility and  later identified as performance-based logistics arrangements. Most of the  DOD performance-based logistics arrangements currently identified by the  services are used for subsystems or components rather than for weapon  system platforms.
Fiscal years 2003 to 2007 Defense Planning Guidance required the  services to submit plans that identified their implementation schedules for  performance-based logistics to all new weapon systems and acquisition  category I and II fielded systems. Similarly, a February 13, 2002, letter from  the Under Secretary of Defense (Acquisition, Technology, and Logistics) to  the services emphasized the need for the plans required by the Defense  Planning Guidance and directed that the plans be issued by May 1, 2002.
But although the services issued plans, they did not take an aggressive  approach toward adopting this concept, according to Office of Secretary  of Defense logistics officials. An October 2003 Defense Business Board  report encouraged the department to move more quickly in adopting the  performance-based logistics, stating, “Performance-based logistics is an  industry best practice and a DOD best practice. DOD should consider  using it for all its weapon systems, new and legacy, provided it is  supported by a business-case analysis.” This task force was chartered by  the Under Secretary of Defense (Comptroller) and Chief Financial Officer  to describe private-sector best practices used in managing supply chain  partnering arrangements and to propose how to apply such practices to  the supply chain processes used by DOD. Citing this task force report, the  aforementioned February 2004 Deputy Secretary of Defense memorandum  to the military departments stated, “Delay in implementing this practice  complicates our funding, limits industry flexibility, and increases DOD  inventory. We must streamline our contracting and financing mechanisms  aggressively to buy availability and readiness measured by performance  criteria.”
section_title: DOD’s Current Preferred Approach to Performance-Based Logistics May Limit Opportunities for Competition and Savings
paragraphs: Because DOD proposes using performance-based logistics at the platform  level as the predominant support strategy for its military systems, it may  limit opportunities for savings from competition, volume discounts, and  reduced administrative costs. Also, by often not contracting for long-term  access to technical data, programs officials are further limiting their  support options.
In the private sector, performance-based contracting is a tool used  according to the applicability of subsystem or component and  circumstance, when it is cost-effective and reduces risk in a  noncompetitive environment. DOD, by contrast, proposes using it as the  predominant product support strategy for its military systems. Further,  when private-sector companies use performance-based contracting, they  use it at the subsystem or component level, retaining the program  integration function themselves as a core business function essential to  successful business operations. Conversely, DOD policy memoranda  support using performance-based contracting at the platform level and  using the contractor as the support integrator. Moreover, private sector  companies emphasize the importance of having the rights to contracts and  competition. DOD, in contrast, is frequently not acquiring the same level of  technical data in its acquisition of new programs.
subsections: section_title: Performance-Based Contracting for Logistics Used as a Tool in Industry Rather Than a Preferred Support Concept as Proposed by DOD
paragraphs: While our review of private sector companies did find that half of those  we interviewed are using performance-based contracting, the industry  approach is much different from DOD’s preferred approach for  performance-based logistics. As previously discussed, Office of the  Secretary of Defense guidance has over the past several years encouraged  the services to use performance-based logistics at the weapon system level  as the preferred approach for life-cycle management of military systems.  DOD officials have stated that this is an industry best practice and should  be adopted more aggressively, but in 7 of 14 companies we interviewed  that used some type of performance-based contracting, this agreement  was used at the subsystem or component level—that is, for engines,  auxiliary power units, wheels, or brakes— and it was generally used for  older systems.
The following chart characterizes the companies we interviewed by  industry type, by the extent to which they outsource logistics support  activities, by the predominant contracting practices used, and by the types  of subsystems or components outsourced using performance-based  contracting. Pseudonyms are used rather than the actual company names.  These companies generate annual revenue generally exceeding $1 billion,  and they use complex and expensive equipment for which they require  high levels of availability and reliability as well as efficiency in managing  lifecycle costs. The life-cycle management issues are comparable to those  of DOD in managing its weapon system sustainment programs.
As shown above, performance-based contracting in the companies we  interviewed is most widely used in the air carrier industry, and it also has  limited use in the energy exploration and mining industry. According to air  carrier officials, time and material contracts are more prevalent than  performance contracts, because industry prefers to use short-term (2 to  3 years) competitive contracts when possible. In a sole-source  environment companies sometimes use longer-term (10 to12 years)  performance-based contracts for supporting some subsystems such as  engines, if they have sufficient historical data to establish an accurate  baseline. For example, all but one of the air carrier industry companies  had performance-based contracts for one or more engines. The amount of  engine workload managed by performance-based contracts varied from  company to company. For example, Company C, which outsourced  38 percent of its total maintenance workload, used performance-based  contracts for one-fifth of its outsourced engine work; while Company A,  which outsourced 65 percent of its maintenance workload, used  performance-based contracts for all of its engine work. The air carrier  companies did not use performance-based contracts for contracted work  on airframes, work that generally comprises about 30 percent of the  commercial aviation maintenance and repair market. Table 2 provides  information regarding the percentage of dollars spent on the repair of each  type of subsystem or component managed using performance-based  contracts by the air carrier companies and the one non-air carrier  company that used performance-based contracts. The subsystems or  components for which the companies used performance-based contracts  most widely were auxiliary power units and wheels and brakes.
Company officials noted that performance-based contracts are a tool most  often used selectively in a noncompetitive environment in an effort to  control cost and reduce risk. Additionally, they said that performance- based contracting works better for subsystems and components where  available cost and performance data are sufficient to establish a good  business case analysis, noting that this is more difficult to accomplish for  new systems, where performance data are uncertain. Performance-based  contracts differ from traditional logistics contracts by focusing on the  purchase of weapon system sustainment as an integrated package based  on output measures—such as a predetermined level of system availability.  In contrast, traditional transaction-based time and material contracts are  used to purchase logistics inputs—such as quantities of spare parts,  specific repair tasks, and engineering studies. Under transaction-based  contracts, the government pays for each transaction as a separate  deliverable; whereas under a performance-based contract, the contractor  is being paid for achieving an outcome performance metric, regardless of  what he does to achieve that performance.
In concept, performance-based contracts encourage the contractor to  achieve a high level of performance at a fixed cost. However, air carrier  industry officials we interviewed said that entering into a performance- based contract without good baseline data introduces a higher level of risk  that the arrangement may not be cost-effective. For example, officials  from one company said they used a performance-based contract for the  older of the two types of engines in the company’s inventory. Officials said  they would wait to collect sufficient performance data on the newer  engine before considering a performance-based contract. The officials  noted that they had originally used a performance-based contract on the  newer engine, but found that, because the reliability of the engine was  greater than expected, the contract arrangement was not cost-effective.  The company was able to change the contract to a time and material  contract, to allow time to collect sufficient performance data to support a  fact-based business case analysis to determine the company’s “should”  cost amount for a performance-based contract.
Performance-based contracting offers DOD opportunities to  provide contractors incentives to achieve desired levels of operational  performance at a fixed cost when the department has historical  performance information. But in the absence of reliable and  complete performance data as a baseline, the adoption of this approach  as the preferred support strategy for new weapon systems could  undermine DOD’s ability to negotiate cost-effective terms—particularly  since the performance-based contracts at the weapon system level have  cost-reimbursement elements, while the private-sector companies  generally used fixed-price agreements. Private-sector officials noted that it  is important to use fixed prices for materials, since the high price of  materials is a key factor driving the companies to use performance-based  contracts.
section_title: Private Sector Applies Performance-Based Logistics at the Subsystem or Component Level Rather Than at the Platform Level and Retains System Integrator Function
paragraphs: DOD policy promotes using performance-based contracting differently  from the way private-sector firms use it in supporting complex and  expensive systems. The companies we reviewed generally used  performance-based contracting at the subsystem level for engines and  certain other components rather than at the platform level, as proposed by  DOD. Furthermore, when using performance-based contracting, these  companies do not contract out the program integration function, as the  military services are doing.
We found no performance-based contracts for maintenance of airframes  or maintenance of any equipment platform among the private-sector  companies we reviewed. Industry officials cited three reasons why they  believe the use of performance contracts is more advantageous at the  subsystem or component level. First, they prefer to take advantage of  competition whenever it is available and to manage support contracts  through the use of competitive procedures. For example, because airframe  maintenance support is available from a competitive market, the  companies generally use a combination of fixed price and time and  material contracts for this category of service. Conversely, performance- based contracts are often used for engine repair because of the high cost  of spare and repair parts that are available only from the original  equipment manufacturer. Officials said that there are too few third-party  repair vendors to foster competition. Second, company officials  emphasized the importance of gaining purchasing power from volume  discounts on subsystems or components across their entire fleet of  systems as a reason for not implementing performance contracting at the  platform level. Finally, by having contracts at the subsystem or component  level, companies can avoid the administrative costs that would be charged  by a prime integrator.
Similar to the approach used by the companies we reviewed, we noted  that the Navy has used performance-based contracts primarily at the  subsystem or component level. Navy officials said that implementation at  this level is easier because the service could implement this concept more  readily under DOD’s current funding structure. The funding is handled  through the working capital fund, with reimbursement to the fund coming  from the sale of subsystems or components to the fleet. Navy officials also  noted that by implementing performance-based logistics at this level, they  can save money by competing subsystems or components where a  competitive market exists, consolidating the requirements of multiple  programs and leveraging their buying power to obtain a pricing advantage,  and reducing administrative costs—advantages also recognized by the  private sector.
The Navy’s history of using a performance-based contract for logistics  support of the T-45 trainer aircraft illustrates how savings may be achieved  by implementing the concept at the subsystem level rather than the  weapon system level. The program office originally had a performance- based contract for the entire weapon system with the original equipment  manufacturer. The contract was a 5-year firm-fixed price with an option  for a sixth year period. Program office officials said the sole metric used,  ready-for-training aircraft, resulted in there being an insufficient number  of aircraft available to fly scheduled training sorties. Additionally, because  actual flying hours were fewer than forecasted, the Navy was paying for  flying hours it was not flying. Concluding that benefits weren’t as  expected, that the costs were too high, and that savings were achievable  by negotiating separate contracts for the airframe and engine, the program  office chose not to exercise the option. The new engine contract is a  performance-based contract awarded on a sole-source basis to the engine  manufacturer, and the airframe performance-based contract was awarded  competitively. According to Navy program office officials, the revised  approach resulted in a projected savings of $37 million in the first year and  projected savings of $144 million at the end of a 5-year period. The  savings are being achieved through elimination of the administrative costs  charged by the prime contractor for the engine work and through  competition for the aircraft system.
Another potential adverse effect of awarding a performance-based  contract at the weapon system level is the loss of management control and  expertise over the system that private-sector firms said was essential to  the success of their business operations. Industry officials said that  managing their supplier base and ensuring the availability of their  equipment to generate revenue is too critical to entrust to a second party.  Further, they believe that contracting out support at a platform level by  using a system integrator limits the potential to optimize savings through  competition and volume discounts and adds administrative costs charged  by the prime integrator for managing subcontractors.
section_title: Not Obtaining Sufficient Rights to Technical Data Could Limit Long-Term Support Options and Could Increase Long-Term Costs
paragraphs: The spokespersons for every company we visited told us that when they  purchase equipment they make sure to acquire the technical data  necessary to support it, regardless of whether the company intends to  support the equipment in-house or outsource some of its support  operations. Company officials said that this data was essential to their own  management and oversight functions. For example, officials from a  company that outsources most of its repairs pointed out that its engineers  use the data to perform such tasks as establishing reliability metrics,  evaluating performance, and revising repair standards. Additionally,  officials stated that owning the technical data afforded their companies  the flexibility that enabled them either to perform the work in-house or to  offer the work up for competition. Several company officials said that it is  best to obtain the technical data at the time the equipment is purchased,  when the buyer has the most leverage in its negotiations with the  manufacturer. Trying to obtain the technical data at a later time is difficult  to negotiate and more expensive. These companies do not price their  technical data items separately. DOD program offices, however, negotiate  a price for maintenance-and-repair technical data separately from the  price of the military hardware systems. According to service competition- advocate officials, program managers faced with limited acquisition  dollars often make trade-off decisions to buy increased weapon system  capability in lieu of technical data.
We reported in 2002 that DOD program offices have often failed to put  adequate emphasis on obtaining needed technical data during the  acquisition process. We recommended that DOD emphasize the  importance of obtaining technical data and consider including a priced  option for the purchase of technical data when considering proposals for  new weapon systems or modifications to existing systems. DOD concurred  with our recommendation, noting that there was a requirement in DOD  5000.2R for program offices to provide long-term access to data required  for the competitive sourcing of systems support throughout the life cycle.  Additionally, by implementing total life-cycle systems management, DOD  would strengthen its emphasis on acquiring technical data when  negotiating support agreements with logistics providers. Nonetheless, the  DOD has further diminished the emphasis it places on the need to acquire  rights to technical data. For example, in May 2003, DOD replaced its  acquisition regulation with a streamlined instruction, which eliminated  the prior regulation’s requirement for the program manager to provide for  long-term access to data required for the competitive sourcing of weapon  system support throughout the life cycle of the system. This language is  now provided as guidance in the Interim Defense Acquisition Guidebook,  but it is not mandatory that this guidance be followed.
According to DOD and service logistics officials, program managers  should develop strategies that provide the government with sufficient and  affordable technical data rights to enable them to put the work out for  competition or develop alternate public or private sources for weapon  system support if performance-based logistics arrangements fail or  become too expensive. Logistics officials recognize that program  managers who implement performance-based logistics contracts on new  weapon systems may wish to delay taking delivery of technical data early  in the life of the system, because unlike the stable designs of commercial  equipment purchased in the private sector, the data for cutting edge  technology lacks maturity and is frequently changed. Alternatively,  program managers sometimes pay the original equipment manufacturers  both to maintain the technical and weapon system configuration data and  to provide the program managers with sufficient access to enable them to  manage and oversee the performance-based logistics contract. However,  logistics officials agree that the product support strategy should clearly  provide for the future delivery of the technical data when required to  support competition or alternative source development.
Service logistics and competition-advocate officials said that it is critical  that this strategy be developed during the weapon system acquisition  phase, when the program office has its greatest leverage in negotiating  the price of the technical data and the conditions under which the  manufacturer must deliver the data. For example, in the course of the  acquisition of the V-22 aircraft engine, the Navy program office obtained a  technical data license agreement, according to which the manufacturer  agreed to deliver a complete data package if it failed to perform in  compliance with the statement of work at the agreed-to price and  schedule. Conversely, when the program office does not obtain the  technical data at the time of purchase, the future costs for obtaining these  data are not knowable and, without the leverage of the original package  purchase, could be prohibitively expensive. In our review of data collected  from DOD’s performance-based logistics program offices, we noted that  DOD had not negotiated for the maintenance drawing packages for the  Javelin missile, F-117 aircraft, and TOW missile improved target  acquisition system, and DOD would have to purchase them at a later date  at a price to be negotiated.
In April 2004, the Logistics Management Institute reported in a review of  performance-based logistics arrangements that it found no evidence to  indicate either the quantity or the quality of logistics management data— including technical data—available to the government was compromised  by the use of performance-based logistics arrangements. This report also  noted, however, that the acquisition guidance published by the Office of  the Secretary of Defense does not address strategies for terminating  interim contractor support or performance-based logistics contracts. The  Logistics Management Institute report recommended that the Defense  Acquisition Working Group include performance-based logistics “exit  strategy” guidance in the defense acquisition guidebook. Nonetheless, as  we have previously noted, guidance in this handbook is not mandatory.
section_title: Conclusions
paragraphs: The use of performance–based contracting for the support of complex  and costly military systems offers opportunities for military program  managers to incentivize contractors to achieve desired levels of weapon  system performance. However, our review of the use of the practice in  private-sector firms indicates that DOD’s proposed guidance to adopt  performance-based logistics aggressively at the platform level could limit  competition, and such guidance might not be the most cost-effective  approach for using this concept. Additionally, although DOD based its  rationale for using performance-based logistics at least partially on the  perception that this is an industry best practice, it appears that perception  is not the case. DOD’s approach toward implementing the concept appears  inconsistent with the way private-sector companies we interviewed use  performance-based contracting in acquiring support for their equipment,  and DOD’s approach has risks that should be addressed as it develops its  guidance for using performance-based logistics.
Using performance-based logistics as the preferred approach for managing  the support of major weapon system programs—even though private- sector company officials use performance-based contracting selectively,  when appropriate and cost-effective—carries the risk of increasing life- cycle cost. Both private- and public-sector experiences with performance- based contracting illuminate the challenges involved in developing a  meaningful baseline for establishing a performance-based arrangement for  new systems, because not enough is known early in the program about  performance characteristics and because there is risk to both the program  office and the contractor that may translate into high cost. Additionally,  the use of performance-based logistics can limit the competition that  would be available for providing logistics support when support decisions  are made at the subsystem or component level rather than at the platform  level. Using performance-based logistics at the platform level also creates  risk by contracting out the program integration function—a core function  that private contractors consider essential for the cost-effective  management of costly and complex systems over their life cycle.
Finally, adopting performance-based logistics at the weapon system  platform level may be influencing program offices to obtain access only to  technical data necessary to manage the performance-based contract  during the acquisition phase—and not to provide a strategy for the future  delivery of technical data in case the performance-based arrangement  fails. In such a case, the program manager would have limited flexibility in  choosing whether to perform maintenance in-house, select an alternative  vendor, or offer the work for competition.
section_title: Recommendations for Executive Action
paragraphs: In order for the department to improve the implementation of  performance-based logistics, we recommend that the Secretary of Defense  direct that the Under Secretary of Defense (Acquisition, Technology and  Logistics) and the Under Secretary of Defense (Comptroller) implement  the following two recommendations:  1.  Incorporate in DOD’s guidance to the services the private sector’s  practice of using performance-based logistics as a tool to achieve  economies at the subsystem or component level, rather than as a  preferred practice at the platform level. Also, incorporate the private  sector’s practice of using it when sufficient performance data are  available to establish a meaningful cost baseline and  2.  Consider requiring program offices, during weapon system acquisition,  to develop acquisition strategies that provide for the future delivery of  sufficient technical data to enable the program office to select an  alternate source—public or private—or to offer the work out for  competition if the performance-based arrangement fails or becomes  prohibitively expensive.
section_title: Agency Comments and Our Evaluation
paragraphs: In commenting on a draft of this report, DOD concurred with our  recommendations to enhance the implementation of performance-based  logistics.
Regarding our recommendation to incorporate in its performance-based  logistics guidance to the services the private sector’s practice of using  performance-based logistics as a tool to achieve economies at the  subsystem or component level, DOD’s response stated that the department  recognizes the need to re-emphasize the use of performance-based  logistics for subsystems and components in its policy memorandum and  guide books. Nonetheless, the response noted that the department  believes that it is still prudent to pursue performance-based logistics  strategies at the platform level where supported by a business case  analysis. The private sector companies we interviewed noted that the  more cost effective alternative is to use competitive procedures where  practicable at the subsystem or component level supported by a cost  analysis using reliable performance data.
Regarding our comment that DOD also incorporate in its guidance the  private sector practice of using performance-based logistics when  sufficient performance data are available to establish a meaningful cost  baseline, DOD stated that its policy is that a business case analysis should  be performed to help make the determination to use performance-based  logistics or traditional logistics support arrangement, and that the business  case analysis incorporate the use of performance data, if available, in  establishing a meaningful cost baseline. DOD stated that it will emphasize  the use of performance data in a revised policy memorandum on  performance-based logistics. However, based on information we obtained  from the private sector companies we interviewed, developing reliable  cost and performance data to support a valid cost analysis at the platform  level for a new system will be a challenge and may not be reliable in  identifying the most cost-effective support option over the life cycle of the  system. As we noted in our report, one company tried a performance- based contract for a new engine but found that because the reliability of  the engine was greater than expected, this contract management was not  cost-effective. Company officials said they preferred to collect reliable  performance data over a period of time to support negotiations for a  performance-based contract.
In response to our recommendation to consider requiring program offices  to develop acquisition strategies that provide for the future delivery of  sufficient technical data to select an alternate source—public or private— or to offer the work out for competition if the performance-based  arrangement fails or becomes prohibitively expensive, DOD stated that it  will take steps to address this issue in the next iteration of the DOD  Directive 5000.1 and DOD Instruction 5000.2 acquisition regulation policy.  According to the response, the new policy will require the program  manager to establish a data management strategy that requires access to  the minimum data necessary to sustain the fielded system, recompete or  reconstitute sustainment if necessary, promote real time access vice  delivery of the data, and provide for the availability of quality data at the  point of need for the intended user. According to DOD, for performance- based logistics arrangements, these actions will include acquiring the  appropriate technical data to support an exit strategy should the  arrangement fail or become too expensive.
section_title: Scope and Methodology
paragraphs: The objectives of our review were to determine (1) what types of  contractor support practices the private sector used to support complex  and costly equipment that have life-cycle management issues similar to  military weapons systems and (2) what potential lessons could be learned  through a comparison of private sector contractor logistics support  practices that DOD currently uses, or plans to use, under its  implementation of performance-based logistics.
To identify commercial industries that use complex and costly equipment  with life-cycle management issues similar to military weapon systems, we  interviewed DOD depot maintenance and logistics policy officials. We also  conducted a literature search to identify appropriate industry groups and  interviewed officials from the Industrial College of the Armed Forces, the  Aerospace Industries Association, the American Association of Port  Authorities, International Council of Cruise Lines, the Society for Mining  Metallurgy and Exploration, the Construction Industry Institute, and the  Council of Logistics Management to validate and refine the identified  industries and to identify appropriate candidate companies within the  industry groups. Within the air carrier, maritime shipping, energy  exploration, mining, and entertainment industries, we identified over 250  companies and selected 67 companies based on sales/revenues,  production rankings, and management awards that might be good  candidates for our study. We eliminated three companies that did not  outsource significant amounts of logistics support, and 50 companies  either did not respond to our initial inquires or declined to participate in  the study. Fourteen companies agreed to participate and completed our  interviews and follow-up questions. Thirteen of the 14 companies we  interviewed agreed to be identified and are listed below by industry group:  Air carriers (Continental Airlines, Houston, Texas; Delta Air Lines, Atlanta,  Georgia; FedEx Corp., Memphis, Tennessee; Southwest Airlines, Dallas,  Texas; and United Airlines, San Francisco, California); Energy Exploration  and Mining (British Petroleum, Houston, Texas; Diamond Offshore,  Houston, Texas; Phelps Dodge, Phoenix, Arizona; and Vulcan Material,  Birmingham, Alabama); Maritime (Carnival Cruises, Miami, Florida;  Conoco Philips Polar Tanker, Long Beach, California; and Disney Cruise,  Orlando, Florida); and Entertainment (Disney World, Orlando, Florida).
To identify private sector support practices, including performance-based  logistics, we conducted group discussions with respective company  officials responsible for maintenance and support operations, budgeting,  and contracting. To collect consistent information among the companies,  we developed standard group discussion questions based on our literature  search and discussions with industry experts. We also included questions  to determine how the companies addressed logistics and contracting  issues similar to those that DOD faced in implementing performance- based logistics. We analyzed the responses to identify the prevailing  industry practices in supporting complex and costly equipment, especially  focusing on the contracting approaches and practices used to outsource  support functions and activities.
We reviewed and discussed with Office of the Secretary of Defense and  military department officials at the headquarters and major acquisition  commands the department’s plans, policies, and procedures for using  performance-based logistics. We also collected policy and guidance  (published and under development) by the Office of the Secretary of  Defense as well as the military departments’ policies and implementation  plans.
To assess what lessons could be drawn from the private sector companies’  experiences to guide DOD’s logistics support efforts, we interviewed  DOD officials and reviewed ongoing logistics programs. We assessed the  reliability of the projected cost and savings data we used in this report by  reviewing supporting documentation and interviewing knowledgeable  personnel; and we determined that it was sufficient for our purposes. We  compared and contrasted the contract logistic approaches and practices  used by private sector activities with those currently used by DOD and  envisioned under its plans for implementing performance-based logistics.  This comparison included such elements as the (1) use of performance- based contracting and the extent of its application, (2) assigning a single  integrator for equipment or weapons system maintenance and logistics  support on a platform level, (3) management and oversight including the  importance of technical data, and (4) the degree of competitive sourcing  and the importance of leveraging purchasing power. As part of our  continuing review we are also conducting case studies on DOD  performance-based logistics weapon systems to further compare the new  DOD approach and practices with those of the private sector. This work is  continuing and we expect to complete our final report early in 2005.
We performed our work from September 2003 through June 2004 in  accordance with generally accepted government auditing standards.
We are sending copies of this report to the appropriate congressional  committees, and it will be available at no charge on GAO’s Web site at  http://www.gao.gov. We are continuing with our review of performance- based logistics in the private sector and in DOD and plan to report the  results early in 2005.
If you or your staff have any questions on the matters discussed in this  letter, please contact me at (202) 512-8412 or solisw@gao.gov or my  assistant director, Julia Denman, at (202) 512-4290 or denmanj@gao.gov.  Larry Junek, Thom Barger, Pamela Valentine, Judith Collins, and Cheryl  Weissman were major contributors to this report.
section_title: Appendix I: Comments from the Department of Defense
subsections: section_title: GAO’s Comment
paragraphs: 1.  Only 7 of the 14 companies we interviewed use some type  of performance-based contracting arrangements. None of the  performance-based arrangements in the seven companies using them  were at the platform level.
section_title: GAO’s Mission
paragraphs: The Government Accountability Office, the audit, evaluation and  investigative arm of Congress, exists to support Congress in meeting its  constitutional responsibilities and to help improve the performance and  accountability of the federal government for the American people. GAO  examines the use of public funds; evaluates federal programs and policies;  and provides analyses, recommendations, and other assistance to help  Congress make informed oversight, policy, and funding decisions. GAO’s  commitment to good government is reflected in its core values of  accountability, integrity, and reliability.
section_title: Obtaining Copies of GAO Reports and Testimony
paragraphs: The fastest and easiest way to obtain copies of GAO documents at no cost  is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts  newly released reports, testimony, and correspondence on its Web site. To  have GAO e-mail you a list of newly posted products every afternoon, go to  www.gao.gov and select “Subscribe to Updates.”
subsections: section_title: Order by Mail or Phone
section_title: To Report Fraud, Waste, and Abuse in Federal Programs
section_title: Congressional Relations
section_title: Public Affairs