Canceled DOD Programs: DOD Needs to Better Use Available Guidance and Manage Reusable Assets
Published: 2014-03-27T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In the face of increasing budgetary pressures, it is important that DOD continue to find ways to manage its portfolio of major defense acquisition programs (MDAP) more efficiently. In 2008, GAO concluded that program cancellations can be a valuable portfolio management tool, and found that DOD can take various steps to retain value from the work completed. The Senate report accompanying the National Defense Authorization Act for 2013 mandated GAO to review issues associated with cancellations of MDAPs. This report assesses (1) the effects of program cancellation, (2) the adequacy of DOD guidance on program cancellations, (3) the extent to which DOD leveraged or transferred to other programs various types of assets from canceled programs, and (4) the usefulness of contract termination cost estimates. To do this work, GAO reviewed the cancellation of five major DOD programs from across the military services; interviewed officials at the Office of the Secretary of Defense, the military services, and others; and reviewed current guidance related to program cancellation.
section_title: What GAO Found
paragraphs: Cancellation of a major Department of Defense (DOD) weapon program can have broad effects. Cancellation of one program can affect the schedules or budgets for related programs as well as the industrial base and local economies. For example, the cancellation of one program has impacted the schedule for the Army's network modernization efforts. DOD stakeholders can provide input to program officials on the potential effects of a cancellation.
DOD has developed an online resource, Smart Shutdown, to offer tools for program managers related to program cancellations. In 2013, a guidebook was added that addresses the spectrum of cancellation effects. However, current efforts to publicize this online resource may not be sufficient and program officials may lack the knowledge necessary to leverage investments as effectively as possible. Furthermore, while this resource is designed to allow officials to share lessons learned about program cancellation, there have been few contributions to date. Without increased sharing of lessons learned, DOD will miss opportunities to build and share knowledge on program shutdown.
GAO could not determine the extent to which DOD leveraged the assets in its case studies because DOD does not have a way to track all types of assets from canceled programs.
DOD relies on a government-wide process for tracking disposal of government property, such as computer hardware and equipment, but this process is not tailored to weapon systems and was not designed to track other types of assets such as rights to technical data and software, or partially-developed technologies. Because there is no department-wide process for disseminating information about all assets available for reuse, DOD cannot ensure that technologies go to parties who can best use or develop them further.
Estimates of contract termination costs are sometimes perceived as a useful tool in managing a portfolio of investments and informing contract termination and budgeting decisions. However, the usefulness of these estimates is limited by, among other factors, inherent uncertainties about costs that cannot be addressed until a program is actually terminated.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOD direct program officials to refer to Smart Shutdown guidance and to provide lessons learned, and that DOD develop department-wide processes to improve tracking and dissemination of information about assets available. DOD partially concurred with these recommendations, but stated use of and contribution to Smart Shutdown should be encouraged rather than directed. GAO maintains direction would be more effective.

section_title: Letter
paragraphs: The federal government has faced increasing budgetary pressures over  the last few years, and measures like sequestration have forced agencies  across the government to scale back and reduce spending. To reduce  costs in the future, it will be important for the Department of Defense  (DOD) to make difficult trade-off decisions, including whether to continue  investing in weapon programs when requirements change or programs  are not meeting goals. Decisions to cancel major programs outright or  terminate large contracts have not been common in the past. However, in  recent years, several major defense acquisition programs have been  canceled—for example, the VH-71 Presidential Helicopter Replacement  Program and the Future Combat System—prompting Congress and the  public to ask questions about the potential savings and impacts of  canceling programs relative to continuing to fund troubled or obsolete  programs. Given that fiscal constraints may require more frequent  cancellations, having the appropriate tools, guidance, and processes to  make effective decisions is vital. Moreover, having a sound understanding  of the costs, benefits, and legal requirements involved in cancellations  and terminations is important.
The Senate report accompanying the National Defense Authorization Act  for Fiscal Year 2013 (NDAA) mandated GAO to review and report on  issues associated with contract terminations and program cancellations of  major defense acquisition programs (MDAP). This report examines (1)  the effects of program cancellation, (2) the adequacy of DOD guidance on  program cancellations, (3) the extent to which DOD leveraged—by  transferring for further development or use on other programs—various  types of assets from canceled programs, and (4) the usefulness of  contract termination cost estimates.
In order to do our work, we reviewed relevant laws, regulations, and  guidance related to contract terminations, financial management, cost  estimating, and government property, including rights to technical data.  We also reviewed our prior reports on contract terminations and program  cancellations, other related reports, and other literature.
We interviewed officials at DOD—including former key staff of canceled  programs—at the Office of the Secretary of Defense (OSD), Army, Air  Force, and Navy to discuss contract terminations and program  cancellations. In addition, we met with Defense Contract Management  Agency (DCMA) Terminations Center and Plant Clearance officials to  discuss contract termination procedures, and obtained relevant  information. We also interviewed officials at the Defense Acquisition  University (DAU) and the DOD Office of Cost Assessment and Program  Evaluation (CAPE) to discuss, respectively, issues to consider in shutting  down a program and termination cost estimates.
Our work included five non-generalizable case studies of canceled  programs:
Aerial Common Sensor (ACS),
Comanche Helicopter,
Future Combat System (FCS),
Transformational Satellite Communications System (TSAT), and
VH-71 Presidential Helicopter Replacement Program (VH-71).
The NDAA Senate report mandated that we review the FCS and VH-71  cancellations. In addition, we selected FCS and VH-71 for review  because these were recent cancellations of expensive and complex DOD  programs, and thus illustrative of current issues faced by program  managers in cancellation of major defense acquisition programs. To  select the other cases, we obtained DCMA data on the highest dollar  contract terminations since those covered in our 2008 report that were  associated with canceled programs. In addition, we considered  cancellations reviewed for our 2008 report where there was potential to  assess additional activity since that report. We then selected cases with  completed settlements and the potential for follow up on leveraged  assets, and also included in our total sample at least one case each for  the Army, Navy, and Air Force. At the time they were active, all of these  programs met the expected investment threshold to be designated  Acquisition Category I—the current equivalent of which means the  program would require more than $480 million in fiscal year 2014 dollars  to complete development, testing, and evaluation. In several cases,  investment in these programs had reached into the billions at the time of  contract termination. Although the results of these case studies cannot be  generalized, we determined the evidence we obtained from the case  studies was sufficient to provide illustrative information on cases from  across the Army, Navy, and Air Force. When feasible, we reviewed prior  GAO work addressing the canceled programs in our sample to garner any  insights. The scope of this review did not include interviewing contractors.
In addition, we supplemented our work by incorporating information  gathered for a recently issued report on DOD estimates of contract  termination liability. Section 812 of the National Defense Authorization  Act for Fiscal Year 2013 mandated that we report to the congressional  defense committees on the extent to which, among other things, DOD  considered estimates of potential termination liability as a factor in  entering into and terminating certain contracts. Our work in response to  that mandate included reviewing a DOD assessment of the extent to  which the department considers potential termination liability as a factor in  entering into and terminating contracts. We also selected a non- generalizable sample of seven current weapon programs in development  and production and collected information on whether these programs  have been requiring contract termination liability estimates before award  and during the course of a contract. These programs were selected to  represent a range of weapon platforms and military services, and to  include both development and production programs. Although the  responses received from these programs cannot be generalized, we  determined the evidence we obtained from these programs was sufficient  to provide illustrative information on DOD’s use of termination liability  estimates.
We conducted this performance audit from October 2012 to March 2014  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: DOD weapon programs range from highly complex and sophisticated  aircraft, missile interceptors, submarines, and space-based sensors, to  new communication and ground control systems that support and  interconnect this equipment, to smaller, less complex systems that  support the warfighter. Increasingly, individual weapon system  investments are getting more complex and more dependent on software.  Moreover, in many cases, weapon systems are also expected to  incorporate technologies that push the state-of-the-art while operating in  harsh and even untested environments—adding daunting technical  challenges that increase the cost and complexity of development.
DOD typically executes several primary contracts over the life cycle of a  weapon system program. These contracts generally cover development,  production, or maintenance efforts.
Although a program and its contracts are related, they are separate,  distinct efforts that can end separately. For the purposes of this review,  we refer to the end of a program as a cancellation, and the end of a  contract as a termination. For example, DOD may cancel a program but  continue a contract related to that program. Alternatively, DOD can  terminate a contract but continue the program. In some cases, programs  may not be formally canceled, but instead restructured. For example,  DOD could continue a program, but change requirements and modify a  contract or even change contractors to better align the program with  available resources.
Cancellation of major programs has not been common in the past, and  we have noted in prior work that the acquisition of weapon systems  involves strong incentives to reduce the risk that a program will be  interrupted or called into question. When programs have been canceled,  cost, schedule, and performance problems have often been cited as  reasons for this decision, and officials have noted that program  cancellation is perceived as failure. However, in some circumstances,  program cancellation may be the best choice. As we reported in March  2013, DOD’s 2012 portfolio of 86 major defense acquisition programs is  estimated to cost a total of $1.6 trillion. This represents a decrease of  about $44 billion from the previous year, due primarily to program  cancellations and restructurings. However, compared to original  estimates, the total cost of the portfolio has increased by over $400 billion  with an average delay of 27 months in the delivery of initial operating  capability, which suggests that opportunities exist to improve efficiencies  in DOD’s portfolio. In 2008, we concluded that from an investment  portfolio perspective, cancellation can be a valuable tool in responding to  long-term fiscal imbalances. In 2009, Congress acted to make  cancellation the default for programs that experience a breach of a critical  cost growth threshold. In recent years, DOD has cited renewed efforts to  end troubled programs and to achieve a better capabilities balance, and  to that end DOD’s annual budget submissions in recent years have each  reflected major program cancellations—at least three per year and as  many as eight.
In response to potential cancellations, defense stakeholders have  sometimes expressed concerns that terminating a contract will cost more  than completing it. In our 2008 report, we reviewed DOD’s past  experience with program cancellations and contract terminations and  found that there are limited circumstances in which it could cost the  government more to terminate a contract than to complete it. Of the  eight contracts we reviewed, we did not find any cases in which the total  amount paid on a terminated contract exceeded the estimated contract  price. We recommended DOD review, and as needed amend, termination  guidance to ensure it consistently identified the conditions under which it  is appropriate to end programs or contracts, and provided the knowledge  needed to use terminations as an investment portfolio tool. DOD  concurred with this recommendation. However, DOD subsequently  determined that no changes to guidance were needed. We also  highlighted important lessons for DOD in making decisions to cancel  individual programs as well as in managing its broader investment  portfolio. For example, when considering cancellation of individual  programs, contract termination costs are generally not a compelling  reason to continue programs or contracts that otherwise warrant ending.  Moreover, while incurred or “sunk” costs in programs being considered for  cancellation may be substantial, they must be paid regardless of whether  or not a contract is terminated. Therefore, the decision to terminate a  contract or cancel a program should not be driven by sunk costs. In  addition, when a contract warrants termination, the decision should be  made as soon as possible because delaying a termination almost always  results in higher settlement costs to the government. Finally, we noted  when a program or contract ends, DOD can make choices that would  allow the government to retain value from the work completed.
subsections: section_title: Weapon Programs Acquire Various Assets That May Be Leveraged
paragraphs: Weapon programs acquire various types of assets throughout a  program’s life cycle; assets that may include property, including computer hardware, equipment, material, special  tooling, facilities, and real property; and  software and technical data, which can include manuals, engineering  drawings, specifications, data rights, and data delivery.
When a program ends, activities related to shutting down the program  include determining how to retain the most value from the government’s  investment, which, for the purposes of this report, is the government’s  expenditures on the program up to the date of cancellation. This can  include terminating or otherwise restructuring contracts, and assessing  how best to leverage the program’s assets. For example, program  officials might identify technologies and facilities that can be realigned for  use on other programs. Alternately, the government may ultimately sell  assets or determine that they are not suitable for continued development.  In some cases, assets that are not reused may be destroyed or  dismantled. Figure 1 illustrates some examples of program assets.
The types of assets a program owns will depend, in part, on the phase of  the program. A program in production is more likely to own hardware and  other types of tangible assets than a program in development. For this  report, our sample included a range of programs in the development  phase.
section_title: Effects of Weapon Program Cancellation Extend beyond a Program and Its Contractors
paragraphs: Program cancellations often result in contract terminations, creating  immediate impacts on programs and their contractors as program offices  work with the prime contractor to end work under the contract and  determine the costs associated with contract termination. However, the  impact of a weapon program cancellation can extend beyond the  immediate program and its contractors and ultimately affect a broader  industrial base and local economies, as well as other programs and  agencies. Consideration of these various effects and consultation with  various stakeholders enables officials to have a better understanding of  the costs, benefits, and legal requirements involved in a particular  cancellation and to take steps that can help mitigate negative effects. In  addition, shutdown activities can take years. While the immediate task of  shutting down the program falls to the program office, there are other  necessary tasks, such as reallocating program funds, that fall beyond the  program’s purview. Figure 2 illustrates the potential effects to be  considered when a program is canceled.
subsections: section_title: Program Costs
paragraphs: A canceled program does not stop incurring costs immediately upon  cancellation. Once a program has been canceled, the program office  must undertake shutdown activities, which include determining whether to  terminate or otherwise restructure contracts and determining how best to  leverage the program’s assets. Conducting tasks such as negotiating with  contractors and disposing of inventory requires program staff to remain  on duty and can incur cost to the government. When cancellation leads to  the termination of a contract for convenience, the Federal Acquisition  Regulation (FAR) directs the contractor to submit a proposal to the  government for the amount of reimbursement it believes it is entitled to  within one year of the official date of the termination. This may include  termination costs, the incurred costs for work already performed by the  contractor, and possibly a reasonable profit or fee on completed work.  Termination costs, which represent costs that the government would not  have otherwise incurred if it did not terminate the contract, can include  expenses such as preparing the settlement proposal, negotiating with  subcontractors, and disposing of inventory. With assistance from the  Defense Contracting Audit Agency to audit the settlement proposals,  DOD then enters into negotiations with the contractor to determine a  settlement. Ultimately, DOD will likely be required to reimburse the  contractor for termination costs, which can reach up to the hundreds of  millions of dollars for a large program, though these costs can be small  relative to the full cost of the program. The final amount of termination  cost owed will not be determined until final settlement. For canceled  programs, the extent of work associated with a shutdown is sometimes  not immediately clear. One case study program official told us decision  makers did not initially understand the need to continue to dedicate  resources to a canceled program to fund shutdown activities.
section_title: Agency Budget
paragraphs: When DOD cancels a program, it must address not only the costs of  terminating any related contracts but the broader impact across the  agency, including deciding on how to reallocate funds originally budgeted  for the canceled program and determining when those funds will be  available. This task extends beyond the canceled weapon system’s  program office. For example, when the Army’s Comanche Helicopter  program was canceled in 2004, the Army proposed an investment  strategy that would redistribute $14.6 billion of planned Comanche  funding through fiscal year 2011 to enhance a broad range of Army  aviation modernization efforts. After securing approval from the Secretary  of Defense and the President, the Army briefed Congress on its plan to  redistribute the remaining Comanche funds within the Army aviation  portfolio. Based on those discussions, the Army developed a plan to  redistribute this funding to 30 different projects and other weapon  programs.
section_title: Capabilities Delivery
paragraphs: Cancellation of a program can lead to a delay in providing capabilities to  the warfighter. As a result, DOD may need to increase funds for other  programs to extend the life of existing or legacy systems. For example,  the Navy’s VH-71 program was initiated to build a replacement for the  current presidential helicopters with improved transportation,  communication, and security capabilities. The Navy’s original plan was to  field the first of these new helicopters by 2008. However, after the  program experienced schedule slips and continued cost increases, the  program was canceled in 2009. The Navy has since initiated a successor  VXX Presidential Helicopter Replacement Program, which is still in the  early stages of development and not expected to have initial operational  capability until fiscal year 2020. Until that time, the Navy plans to continue  to maintain the legacy helicopters that it had otherwise planned to replace  with the VH-71. Additionally, in May 2007, we found that the ACS contract  termination resulted in about an additional $900 million investment  beyond what had been previously planned to maintain the legacy  systems. Another example is the Army’s canceled FCS, a multibillion  dollar development program originally consisting of 18 manned and  unmanned systems tied together by an extensive communications and  information network. Initiated in 2003, the first Army combat teams were  planned to be FCS-equipped in 2011, while the rest would be fully  equipped by 2032. When the Manned Ground Vehicle portion of the  program was canceled in 2009, the Army altered its modernization  strategy but continued to develop unmanned vehicles and improved  network and technology sensors in order to deliver them to selected  combat teams starting in fiscal year 2011. Faced with disappointing test  results and high costs, FCS was fully canceled in 2011. The Army has  since incorporated its FCS efforts into a new acquisition strategy to  modernize its network, but it does not plan to include the canceled FCS  components, and other technologies to modernize the network will be  delivered later than originally planned through FCS.
section_title: Industrial Base
paragraphs: DOD relies on a broad industrial base that is capable of producing  complex products that can be unique to defense applications. When a  major weapon program is canceled, it can impact sectors within the  industrial base and potentially diminish the capacity to produce key  capabilities. The Weapon System Acquisition Reform Act of 2009  mandated that DOD add the impact of the cancellation of major weapon  systems in its assessment of the national technology and industrial base  that it provides annually to Congress. For example, the 2012  assessment reported on the impact of the 2009 cancellation of the  Manned Ground Vehicle portion of the Army’s FCS, which was supposed  to replace current Army vehicles such as the M1 Abrams Tank and M-2  Bradley Fighting Vehicle. In our prior work, we found that smaller  companies that rely on government contracts can be particularly affected  by program cancellations. In addition, program cancellation can impact  a community that is economically dependent on a contractor facility. In  the example noted above, changes to the Army’s ground vehicle program  can have an economic impact in cities and towns within the Upper  Midwest and Great Lakes region of the United States, where a large  portion of military ground vehicle production is based.
section_title: Staff and Personnel
paragraphs: When a program is canceled, personnel working in the program are  immediately affected. One of the challenges program managers face is  addressing potential deflated worker morale as well as the logistics of  transferring personnel while maintaining essential employees to  effectively shut down the program. DOD officials have noted that their  goals in addressing personnel issues resulting from canceled programs  are to maintain open communications and minimize disruptions. For  example, one senior official suggested that during a shutdown, the  program office could offer training to remaining program staff in order to  maintain skills that were lost due to the departure of staff that had already  left the program.
section_title: Additional Program Partners
paragraphs: For some programs, cancellation may impact additional stakeholders. For  example, the Army’s Medium Extended Air Defense System (MEADS),  which DOD proposed to no longer fund after fiscal year 2013, was being  developed by the Army in collaboration with two other countries. DOD’s  cancellation of MEADS would leave the two partners with considerably  fewer resources than anticipated to complete development.
Consulting the broad range of stakeholders who will be affected by, or  have knowledge of the effects of, a program cancellation, can enable  officials to be aware of important effects of program cancellation. For  example, one former Comanche official stated that the Army would have  had more accurate knowledge of what funds were available for programs  after cancellation and avoided delays during the contract termination  process if it had involved legal advisors and the Comptroller’s office  earlier in the process. As the effects of a major weapon program  cancellation extend far beyond the program office, there are multiple  stakeholders representing various interests that can assist in enabling  program officials to fully assess these effects as shown in table 2.
section_title: New Guidance Requires Input and Awareness to Work as Intended
paragraphs: DOD has recently developed guidance for shutting down canceled  programs. In the absence of guidance, officials from our case study  programs reported relying heavily on experienced staff to navigate the  shutdown process. The new online guidance addresses the spectrum of  effects of cancellation and offers a variety of tools for program managers.  However, current plans for notifying program managers of this resource  may not be sufficient. Furthermore, the guidance was designed to serve  as a lessons learned mechanism but there have been few actual  contributions of lessons learned from the acquisition community.
subsections: section_title: Without Formal Guidance Available, Program Officials Have Relied on Personnel with Cancellation Experience
paragraphs: The decision to cancel a program is often communicated by an  acquisition decision memorandum, which documents the decision to  cancel and provides objectives for program officials to accomplish during  the shutdown process. However, at the time the programs we selected for  case study were canceled, DOD did not have department-wide guidance  on how to achieve those objectives. In 2008 we recommended that DOD  review, and as needed amend, guidance on terminations across the  military services and DOD entities to ensure that guidance consistently  identified the conditions under which it is appropriate to end programs or  contracts, and that guidance provided the necessary knowledge for using  terminations as an investment portfolio management tool. Following our  report, DOD conducted a review and determined that no changes to DOD  guidance were needed. The FAR and Defense Federal Acquisition  Regulation Supplement (DFARS) contain guidance on the process of  terminating contracts; however, contract termination is only one of several  aspects of program cancellation that program officials must consider  when shutting down a program. As discussed earlier, other areas of  consideration include disposing of program assets and overseeing the  budget during shutdown of the program. Officials from most of our case  study programs stated that at the time they were shutting down their  canceled programs, they were not aware of department-wide or service  specific guidance addressing these other aspects of program shutdown.  Army officials were able to identify some Army guidance on shutting down  programs, although it was unclear whether officials from our Army case  study programs referred to this guidance during the shutdown process for  those programs. During the course of this review, Air Force officials  provided documentation of new draft Air Force guidance, which was not  yet in effect when the TSAT program was canceled. Navy officials were  unable to identify Navy guidance on the subject. DOD officials confirmed  that there was no department-wide guidance on program cancellation and  shutdown.
Officials from most of our case study programs told us that without  department-wide guidance, they learned about cancellation and shutdown  by working with personnel who had been through cancellations before.  For example, FCS officials told us that a number of their program office  staff, including those working on disposition of assets, had worked on the  Crusader cancellation. In addition, the DCMA termination contracting  officer working on FCS had also handled the Crusader cancellation.  Similarly, a former VH-71 official told us that he was brought on as  program manager for VH-71 when officials began to think about  cancellation, because he had been through cancellation of a major  program before. The Comanche contracting officer stated that he had  been through several cancellations before helping to shut down the  Comanche program. ACS officials reported relying heavily on the DCMA  termination contracting officer for guidance. This continuity seems to have  worked well for our case study programs. However, several of the experts  in this area from our case study programs have retired or will soon be  eligible to do so, and DOD may not be able to continue to make this  knowledge accessible to program managers without a more  institutionalized mechanism for doing so.
section_title: Realizing Full Potential of New Resource Depends on Awareness of This Resource and Input from Experienced Program Officials
paragraphs: Recognizing the need for guidance on shutting down programs, in 2009  the Defense Acquisition University (DAU) began development of a special  interest website called Smart Shutdown, which offers a variety of  resources related to program shutdown. In December 2013, DAU  finalized a guidebook geared toward program managers which is now  available through the website. The guidebook is constructed to make  program officials aware of various areas to be considered in program  cancellation, and to help them shut down a program as efficiently as  possible, while maintaining the best value for the government. These  areas for consideration include
Personnel,
Operational Capability/Requirement,
Technology,
Facilities/Hardware/Software/Program Security,
Contracts, and
Budget.
DAU officials told us that because the specific issues faced by a program  manager during shutdown will vary substantially by program, the  guidebook is intended to provide insight on common issues to be  considered throughout the process, rather than detailed prescriptive  guidance. Other materials available through the website include a sample  briefing to leadership, and a database of considerations when disposing  of certain assets.
This guidance is intended to be advisory in nature. DAU disseminates  information about this resource through a required program manager  course; however, program managers are required to take this course  once before becoming program managers of high value programs.  Therefore, program managers who have taken the course before this  resource was developed will not have received this information.  Furthermore, it may be many years between a program manager’s  completion of this course and the point at which he or she is actually  faced with the prospect of shutting down a program. Therefore,  mentioning the Smart Shutdown materials in a one-time program  manager course does not ensure that a program manager faced with  cancellation will be aware of this resource. Internal control standards call  for agencies to undertake control activities, which include mechanisms to  help ensure that management’s directives are carried out. In several  cases, acquisition decision memorandums directed program officials to  retain maximum value from DOD’s investment in canceled programs.  However, without awareness of DAU’s Smart Shutdown resources,  program officials may lack the knowledge necessary to leverage  investments effectively as directed. Inclusion of information on Smart  Shutdown in an acquisition decision memorandum communicating  cancellation of the program, for example, would be one mechanism to  better ensure program managers have information on this resource when  they need it.
To develop this guidance, DAU interviewed individuals who had  experienced program cancellation. DAU officials stated that the resource  will benefit from more input from experienced members of the acquisitions  community. The website is constructed to allow input from the acquisition  community in identifying goals, processes, issues, best practices, and  considerations in all aspects of program cancellation activities. In  particular, its database is designed to allow officials who have  experienced program cancellation to share lessons learned. According to  the Defense Acquisition Guidebook, which provides acquisition best  practices, program managers can gain insight into risks, uncertainties,  and opportunities that their programs may encounter by reviewing  experiences of other programs described in lessons learned. The  guidebook states that program managers should take advantage of these  types of resources when available. However, DAU officials told us that  they have received very few contributions from the acquisition community  to date. In fact, we found that several of our case study programs had  written compilations of lessons learned during the cancellation process,  but DAU officials were not aware of most of these lessons learned  documents. Without continued contribution of lessons learned from  program officials with experience shutting down canceled programs, DOD  will be missing opportunities to build and share knowledge on program  shutdown.
section_title: Information about Whether Assets Are Being Leveraged Is Limited
paragraphs: We could not determine the extent to which DOD leveraged the assets in  our case studies because DOD does not have a way to track all types of  assets. DOD uses a basic, federal government-wide process and IT  system for reutilizing, disposing of, and tracking government property.  However, both the process and the system are designed to track specific  pieces of equipment, materials, and tools and are not designed to track  intellectual property, such as rights to technical data and software,  partially developed technologies, or even other types of assets such as  real property and hazardous materials. Because there is no department- wide process or requirement for disseminating information about all  assets available for reuse, DOD cannot help ensure that technologies,  hardware, and other potentially useful assets can be delivered to parties  who can best develop them further. There are inherent difficulties in  transferring the full range of assets on any program, given the uniqueness  of a program’s requirements. Nevertheless, given that DOD may spend  billions of dollars before deciding to cancel a program, it is in the  government’s interest to ensure these investments are leveraged to the  maximum extent possible.
subsections: section_title: DOD Follows a Federal Process to Track Some Assets
paragraphs: The FAR specifies a government-wide process, which DOD refers to as  plant clearance, for reutilizing and disposing of property no longer  required for continued performance of a contract. Figure 3 illustrates the  basic process.
The plant clearance process applies to property, such as material,  equipment, special tooling, and special test equipment, which may be  government-furnished or contractor-acquired in performance of a  contract. However, the process does not apply to software and technical  data or types of assets such as real property or partially developed  technologies. Table 3 provides examples of property processed through  and outside of the process.
The Plant Clearance Automated Reutilization Screening System  (PCARSS) automates the process for reporting, screening, requisitioning,  and disposing of excess property. When a contract is terminated or  completed, contractors are to enter inventories of excess property directly  into PCARSS. DOD components and federal agencies have access to  PCARSS, in which they can review available property and requisition that  in which they are interested. The FAR identifies reutilization priorities for  plant clearance of property. In descending order, these priorities are  reuse within the owning agency, transfer of educationally useful  equipment to schools and nonprofit organizations, and reuse within the  federal government or donation as surplus property.
Some case study program officials found PCARSS cumbersome.  According to DCMA plant clearance officials, a contractor may submit  over 1,000 inventory schedules, including those of subcontractors, for a  contract termination. These inventory schedules can contain many  thousands of line items, making it time-consuming and difficult to identify  property for potential reuse. Also, many of the items listed tend to be  smaller components, such as cables and screws. Table 4 cites examples  of items listed in a PCARSS inventory schedule.
section_title: Many Assets Are Processed Outside of PCARSS and Are Not Tracked
paragraphs: For our five case study programs, which were all canceled during  program development, the most valuable program assets for transfer  were often partially developed systems or technologies, which can be a  combination of software, technical data, or property. These types of  assets were generally not processed through PCARSS nor tracked in any  other systematic way. Table 5 lists examples of major assets transferred  from some of our canceled case study programs, but does not represent  a comprehensive accounting of all major assets for the programs.
Former officials of our case study programs were not able to provide  comprehensive lists of asset disposition because they did not have  systems to track this information. They tracked transfers of assets  processed outside of the plant clearance process mostly through  program-developed spreadsheets. The FCS program also used  contracting officer letters to document the transfers. When an item was  transferred for use on another contract, the transfer was documented by  contract modifications to both the gaining and losing contracts.
Though they could not provide comprehensive lists of asset disposition,  program officials reported that a substantial amount of assets was  transferred outside the PCARSS system. The Comanche program, for  instance, reported reutilizing about two-thirds of its assets; less than 1  percent was processed through PCARSS. Officials of the VH-71 and ACS  programs, both of whose acquisition strategies were to integrate existing  or developing systems onto a base platform, told us they expected that  program assets would be used on successor or related programs and  therefore bypassed the plant clearance process. In addition, PCARSS is  not used for assets that the government allows the contractor to continue  to use after a program is terminated, or items sold back to the contractor  or to third parties. For example, the VH-71 program contractor, with  program office and DCMA termination contracting officer concurrence,  negotiated a sale to Canada of nine prototype helicopters. This  transaction did not go through PCARSS.
The acquisition decision memorandums cancelling several of our case  study programs or terminating related contracts instructed program  officials to leverage DOD’s investment in the programs. But there is no  DOD requirement to track all program assets. Moreover, DOD has no  comprehensive mechanisms in place to facilitate activities, such as  sharing information on available assets, valuing these assets, and  tracking transfer, which would help to ensure these assets are effectively  leveraged as directed. Without systems to track disposition of assets, the  government may not be able to ensure it is getting the most value for  assets.
section_title: Rights to Technical Data and Software Present Unique Challenges
paragraphs: Technical data and software, which are also dealt with outside the  disposition process, present unique challenges to tracking. According to a  DOD report, information technology represents a considerable proportion  of defense weapon programs underway—a proportion that is likely to  increase in the future. Some DOD program developments have  extensively relied on software and technical data. For example, the FCS  development effort projected that 95 percent of FCS functionality would  be controlled by software. However, technical data rights are not tracked  centrally, but rather, according to DOD officials, only as a component of  contracts that involve such rights.
DOD and its contractors are to follow the FAR and DFARS procedures  that outline the government’s rights to acquire, use, and disclose  technical data. Defense procedures require contractors to identify in the  contract the technical data rights to be granted to the government and  delivered under the contract.  Contractors generally identify the  government’s rights to technical data at the beginning of the contract  process and prior to delivery of the technical data using the Contract Data  Requirements List (CDRL)—an exhibit to the contract that contains the  technical data deliverables. The technical data rights assigned to the  government can vary depending on the nature of the data, source of the  funding and negotiated terms of the contract. The standard data rights  range from the least restrictive, or unlimited and government purpose  rights, to more restrictive rights, such as limited or special rights.
Although rights to data are usually defined when a contract is executed,  the government may adjust or renegotiate those rights during the life of a  contract. DOD policy requires that prior to issuing a contract solicitation,  program managers for major acquisition programs must assess the merits  of including a priced contract option for the future delivery of technical  data and intellectual property rights that are not acquired in the initial  contract.
DOD officials mentioned several reasons the tracking and transfer of data  rights can be problematic and challenging when a contract is terminated.  Specifically, the following:
Limited recordkeeping for unrestricted data rights—The CDRL  does not contain all technical data items delivered to the government.  The DFARS requires contractors to identify technical data rights with  restrictions in the contracts. According to an Army intellectual  property attorney, the CDRL generally contains the restricted items  and not the items that are unrestricted such as those assigned  government purpose or unlimited rights. Consequently, the  unrestricted rights are not evident at contract termination and it can be  difficult to track all technical data rights that should be accounted for  when a contract is terminated.
Complications transferring a combination of software under  contract—When a system under development includes a  combination of software products, each with different types of rights  assigned, the government may need to identify all of the software  under the contract and renegotiate the data rights. For example, when  the FCS program was canceled, the Army planned to transfer the  System of Systems Common Operating Environment (SOSCOE) to  the Army Aviation and Missile Research Development and  Engineering Center for further development and distribution to  program offices that had use for it. However, SOSCOE contained a  combination of software that was DOD-developed, developed by FCS  subcontractors, developed for commercial purposes, and some that  was open source. Before transferring SOSCOE, the Army worked with  the subcontractor to develop a single distribution agreement for the  government purpose rights and open source software, and a special  license agreement for the portion of software developed for  commercial purposes.
Difficulty in determining the value of technical data and software  before transferring data rights—According to an Army intellectual  property attorney, the assignment of value to technical data and  software rights is not clearly understood and these values are  assigned, in most cases, by the contractor. Further, some contractors  are more sophisticated than others in determining the value of data  rights and maintaining good documentation. In some cases, the  government and contractors have had difficulties valuing rights in  technical data even if line items in the contract identify the technical  data to be delivered and specify the assigned value for each item. The  government may need to contest the contractors’ assigned values to  technical data rights before it can transfer those rights, particularly  when there are unknowns with a program, such as the case with  research and development programs where the development  approach may not be stable and could necessitate changes to the  contract.
Loss of knowledgeable personnel for tracking transfers— According to DOD officials, staff familiar with the contract and the  government’s technical data rights would be the best source for  tracking their transfer. However, when a program is canceled,  valuable contract knowledge can be lost when staff resign, are  reassigned, or are transferred to another program.
section_title: Uniqueness of Weapon System Requirements Makes It Inherently Difficult to Transfer All Assets
paragraphs: The uniqueness of weapon system requirements presents inherent  challenges to transferring assets. Because technologies meet very  specific needs and are usually costly to develop, there may be a very  narrow set of organizations interested and able to afford continuing the  development effort. Some former program officials told us that  transferring existing program assets depends on knowledge of other  programs that might be able to use them, which often happens through  networking. For example, the Comanche program had staff with aircraft  technology expertise who helped find appropriate recipients for  equipment. A former FCS official told us that he tried—by speaking at  trade shows, trying to publish technical papers, and networking with  people—to make the user community aware that SOSCOE was available  for distribution. However, he commented that these efforts were not  effective, and noted that the lack of an Army infrastructure for sharing  information hampered the efforts. After spending approximately $900  million under the FCS contract to develop the SOSCOE, the Army  transferred it to a research and development center for further  development. However, while the center initially funded additional  development, the former FCS official indicated that the SOSCOE will no  longer be funded.
Additionally, a technology may become irrelevant before it is fully  developed. For example, a former ACS official told us that technologies  under development for ACS were of little use to the eventual replacement  program because it had different requirements. Lastly, while not citing  specific cases, other program and Office of the Secretary of Defense  officials commented that it is not unusual to be ultimately unable to  transfer a technology, given the uniqueness of the technologies being  developed.
section_title: Contract Termination Cost Estimates Have Limitations
paragraphs: Estimates of contract termination costs are sometimes perceived by  program officials as a useful tool in managing a portfolio of investments  and informing contract termination planning and budgeting decisions.  However, there are some uncertainties about costs that cannot be  addressed until a program is actually terminated, making estimates less  useful for the purpose of informing the decision to terminate a contract.  Thus, while DOD generally has access to contract termination cost  estimates, they have limitations due to issues such as limited visibility into  cost assumptions.
subsections: section_title: Some Elements of Termination Costs Are Inherently Uncertain
paragraphs: According to DOD officials, estimates of termination costs can be useful  for contract termination planning and budgeting decisions. However,  some issues involving termination costs may not be resolved until after  the termination settlement process begins. Contractors have a year from  the time of termination to produce a settlement proposal and, according to  a senior contracting termination official, contractors often request  extensions of this deadline for large and complex programs. During this  time, the contractor is to resolve areas of uncertainty and take a position  as to which costs can be claimed as termination costs. For example,  former contracting officials for the VH-71 program noted that there was  uncertainty during contract execution regarding whether the cost of  certain contractor investments, such as facilities and test vehicles, could  be claimed by the contractor as termination costs. Contractors and  program offices must also agree on the price of any contract changes that  had been unpriced at the time of termination. In the case of FCS, a former  program official said that a series of partial contract terminations and  changes to the contract led to frequent fluctuations in cost estimates that  made it more difficult to make an accurate estimate of the cost of  terminating the contract. DCMA officials told us final settlement for the  FCS contract is not expected until 2016. In two of the five canceled  programs we reviewed, a major obstacle to settling the terminated  contract was determining the fees due to the contractors for their work. A  senior contract termination official said that there are a number of other  termination costs that can be significant but difficult to calculate until after  the termination process has begun, including the costs of stopping work  until the contract is terminated and costs associated with the Worker  Adjustment and Retraining Notification Act, which requires contractors to  provide advance notice in the event of a mass termination or plant  closure. As a result, there will always be a level of uncertainty in cost  estimates prepared before termination, which limits the utility of those  estimates.
section_title: Contract Termination Cost Estimates Are Generally Available to DOD but Have Limitations
paragraphs: Generally, DOD weapon programs receive regular estimates of  termination costs from their contractors in contract funds status reports  (CFSR). Termination cost estimates are included among other cost data  in these reports, which contractors are required to regularly provide the  program office to help forecast and plan funding for the program. The  CFSRs we reviewed included individual estimates of the cost of  terminating the contract during each of the immediate months or quarters,  then estimates for termination costs in subsequent quarterly or annual  periods thereafter. In response to our survey of seven major weapon  programs currently in development or production, conducted as part of  our related work, the five programs that reported a source for information  used to develop their termination estimates identified their contractor,  usually through CFSRs. Six of the respondents reported considering  termination liability estimates during contract execution while one  reported considering termination liability estimates when awarding the  contract.
Some program officials we spoke with were uncertain of how contractors  arrived at these estimates, and were not aware of any additional  information provided by contractors to support these estimates. For two of  the canceled programs we reviewed, estimated termination costs  provided by the contractor increased as terminations became more likely,  in one example by almost 50 percent from the previous quarter. A  former program official that we spoke with said that these increases  suggest that when producing their termination cost estimates, the  contractors were including all costs that could conceivably be allowable  under the FAR. In at least one case, uncertainty about costs included in  the contractor estimate led to considerable uncertainty in the program  office estimate that was submitted to decision makers.
DOD also has access to its own independent termination cost estimates  from the Office of Cost Assessment and Program Evaluation (CAPE);  however, their usefulness to DOD during the consideration or planning of  a program cancellation is limited because they are usually developed  before contracts are awarded. At major milestone decisions in which  weapon programs enter the next acquisition phase, CAPE produces  independent costs assessments for DOD that include estimated  termination costs for the program. While these estimates are produced  independently within DOD, unlike those in the CFSRs, CAPE might not  provide estimates between milestone reviews, which can be many years  apart. For example, the VH-71 was canceled in 2009, while its last  milestone review was in 2005. Furthermore, when CAPE produces  estimates at the milestone phase, it is generally before a contract is  awarded, so the estimates do not reflect any data from contract  execution. According to a CAPE cost assessor, contract termination  estimates are generally not a major point of consideration during DOD  milestone reviews.
Though it would not have applied to the canceled programs we reviewed,  the recently updated weapons acquisition instruction released by DOD in  November 2013 requires a termination liability estimate for all large  weapon acquisitions prior to contract award. Specifically, the  instructions require weapon programs for which termination liability can  be reasonably expected to exceed $100 million to include an estimate of  the potential termination liability in the program’s acquisition strategy as  well as how the liability is likely to increase or decrease during execution  of the contract.
In limited instances, contractors may provide estimates of some  termination costs prior to contract execution. Contracts for weapon  programs may contain a special termination cost (STC) clause, which  reflects the program and the contractor’s estimation of certain termination  costs prior to award of the contract. However, according to officials, these  clauses are rare and do not necessarily represent all types of allowable  termination costs. Specifically, these clauses define a maximum amount  of specific termination costs that will be payable to the contractor if the  contract is terminated. DFARS states that this clause may be used when  approved by an agency head, when the contract meets certain duration  and investment thresholds, and when adequate funds are available to  cover potential liability.
Although contracts for two of the programs we reviewed contained such  clauses, FCS and Comanche, a senior DCMA contract termination official  said that he has rarely encountered these clauses in the terminated  contracts he has reviewed. For those two programs, termination cost  uncertainty remained despite the existence of the clause. In the case of  the Comanche program, the STC clause set the special termination cost  limit for the contract at $123.2 million for fiscal year 2004, the year in  which it was canceled. When the Army requested an estimate from the  contractor for the final termination costs, the contractor responded with a  rough order of magnitude of $640 million, noting that the estimate  included other termination costs not covered by the STC clause. The STC  clause did not provide a basis for the program office to evaluate these  other potential termination costs included in the contractor’s rough order  of magnitude termination cost estimate. When the contract termination  was settled, the final termination costs were around $143 million after  program officials made efforts to minimize these costs. In the case of  FCS, officials were unclear how the STC clause in the primary  development contract would be applied since the program was canceled  through a series of partial contract terminations, and they thought costs  for each partial termination were unlikely to reach the limit in the STC  clause. The government and the prime contractor have not yet reached a  final termination settlement on that contract.
section_title: Conclusions
paragraphs: The magnitude of DOD’s investments in major acquisition programs  demands good stewardship when it is decided a program should be  canceled. Though cancellations of major programs to date have been  relatively rare, a single one may represent a multibillion dollar proposition  that can have far-reaching effects on DOD, the industrial base, and even  local economies. DAU has developed guidance that addresses the  spectrum of issues faced in a cancellation, though current efforts may be  insufficient to ensure that program managers who are faced with a  program cancellation will be aware of this guidance. Without this  awareness, program officials may lack the knowledge necessary to  manage the shutdown process and leverage related investments as  effectively as possible. Furthermore, without the continued input of  lessons learned from past cancellations, DOD will miss opportunities to  continue to build and share knowledge regarding program shutdown.  Leveraging investments in canceled programs is complicated because  there is no DOD process to track all program assets or disseminate  information about assets available for reuse. DOD uses a process that  provides visibility over specific tools, materials, and equipment that  support a program being canceled, but it has little visibility over perhaps  more valuable, but less tangible, assets such as software or partially  developed technology, as our case studies demonstrated. Leveraging  these investments is further complicated by the fact that the technologies  may be unique and expensive to continue to develop, thereby limiting the  pool of organizations that want partially developed technologies,  specialized hardware or other property. Without more comprehensive  processes to track and disseminate information about all available assets,  DOD’s efforts to ensure that it gets the most value for its investments will  continue to be hampered.
section_title: Recommendations for Executive Action
paragraphs: To improve DOD’s ability to ensure it is fully leveraging investments made  in canceled programs, we recommend that the Secretary of Defense  direct the Office of Acquisition, Technology, and Logistics to take the  following three actions:  (1) Direct program officials to refer to DAU’s Smart Shutdown guidance  throughout the process of shutting down the program. This could be  done, for example, through acquisition decision memorandums.  (2) Direct program officials to provide lessons learned, if applicable,  related to program cancellation and shut down to DAU to enhance its  Smart Shutdown toolkit.  (3) Develop department-wide processes to improve tracking of assets,  including technical data and software, and dissemination of information  about assets available for reuse after programs are canceled.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided us with written comments on a draft of this report, which  are reprinted in appendix III.
DOD partially concurred with our recommendation to direct program  officials to refer to DAU’s Smart Shutdown guidance. DOD agreed that  Smart Shutdown is a beneficial tool, but stated that it planned to highly  encourage, rather than direct, reference to this resource. While we  support DAU’s efforts to further promote the toolkit, we question whether  these efforts will be sufficient to ensure that all program officials faced  with cancellation have knowledge about cancellation and shutdown  procedures to leverage investments effectively. Thus, we continue to  believe that direction to program officials to refer to the Smart Shutdown  toolkit would be a more effective mechanism to ensure that  management’s goals for the shutdown would be achieved. As noted in  this report, officials from most of our case study programs told us that  without department-wide guidance, they learned about cancellation and  shutdown by working with personnel who had been through cancellations  before. However, several of the experts in this area from the case study  programs have retired or soon will be eligible to do so. When a major  program is canceled, acquisition decision memorandums which contain  instructions for the cancellation process are sometimes issued to program  officials, and these may direct program officials to work to achieve DOD  goals such as retaining maximum value from DOD’s investment in the  program. Including direction in this acquisition decision memorandum for  program officials to refer to the Smart Shutdown resource during this  process would be one way to ensure program officials are aware of a  resource that will help them to best achieve management’s objectives.
DOD also partially concurred with our recommendation to direct program  officials to submit lessons learned after cancellation, again stating that it  would highly encourage, rather than direct, this action. We continue to  believe that direction to submit lessons learned, where applicable, is the  most effective way to ensure knowledge about cancellation is captured  and shared. As discussed in this report, we found that DAU had received  few contributions of lessons learned. Without such direction, there may be  little incentive for program officials to take the time to share lessons  learned before moving on to a new assignment.
DOD concurred with our recommendation to develop department-wide  processes to improve tracking of assets, including technical data and  software, and dissemination of information about assets available for  reuse after programs are canceled.  However, DOD did not provide  information on any specific actions it planned to take to address this  recommendation.
We are sending copies of this report to the Secretary of Defense and  interested congressional committees. In addition, the report is available at  no charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix IV.
section_title: Appendix I: Scope and Methodology
paragraphs: The Senate report accompanying the National Defense Authorization Act  for Fiscal Year 2013 mandated GAO to review and report on issues  associated with contract terminations and program cancellations of major  defense acquisition programs (MDAP). This report examines (1) the  effects of program cancellation, (2) the adequacy of current Department  of Defense (DOD) guidance on program cancellations, (3) the extent to  which DOD leveraged—by transferring for further development or use on  other programs—various types of assets from canceled programs, and  (4) the usefulness of contract termination cost estimates. We reviewed  relevant laws, regulations, and guidance related to contract terminations,  financial management, cost estimating, and government property,  including rights to technical data. We also reviewed our prior reports on  contract terminations and program cancellations, other related reports,  and other literature.
We interviewed DOD officials—including former key staff of canceled  programs—at the Office of the Secretary of Defense (OSD), Air Force,  Army, and Navy to discuss contract terminations and program  cancellations. In addition, we met with Defense Contract Management  Agency (DCMA) Terminations Center and Plant Clearance officials to  discuss contract termination procedures, and obtained relevant  information. We also interviewed officials at the Defense Acquisition  University (DAU) and the DOD Office of Cost Assessment and Program  Evaluation (CAPE) to discuss, respectively, issues to consider in shutting  down a program and termination cost estimates.
Our work included five non-generalizable case studies of programs  canceled during development:
Aerial Common Sensor (ACS),
Comanche Helicopter,
Future Combat System (FCS),
Transformational Satellite Communications System (TSAT), and
VH-71 Presidential Helicopter Replacement Program (VH-71).
The Senate report mandated that we review the FCS and VH-71  cancellations. In addition, we selected FCS and VH-71 for review  because these were recent cancellations of expensive and complex DOD  programs, and thus illustrative of current issues faced by program  managers in cancellation of major defense acquisition programs. To  select the other cases, we obtained DCMA data on the highest dollar  contract terminations since our 2008 report associated with canceled  programs. In addition, we considered cancellations reviewed for our 2008  report where there was potential to assess additional activity since that  report. We then selected cases with completed settlements and the  potential for follow-up on leveraged assets, and also included in our total  sample at least one case each for the Army, Navy, and Air Force. At the  time they were active, all of these programs met the expected investment  threshold to be designated Acquisition Category I – the current equivalent  of which means the program would require more than $480 million in  fiscal year 2014 dollars to complete development, testing and evaluation.  In several cases, investment in these programs had reached into the  billions at the time of contract termination. Although the results of these  case studies cannot be generalized, we determined the evidence we  obtained from the case studies was sufficient to provide illustrative  information on cases from across the Army, Navy and Air Force. We did  not review any classified aspects of these programs. We also reviewed  prior GAO work addressing the canceled programs to garner any insights.
To examine the impacts of program cancellation, we reviewed Federal  Acquisition Regulation (FAR) and Defense Federal Acquisition Regulation  Supplement (DFARS) parts covering contract termination, contract cost  principles and procedures, and government property to identify,  respectively, what steps to take when a contract is terminated, what  termination costs are allowable, and what issues arise in disposing of  program assets. We also looked at specific FAR and DFARS contract  clauses related to these areas. In addition, we obtained termination  settlement memorandums for the three of our case study programs for  which these were available. Settlement memorandums were not yet  available for FCS or VH-71.
In addition, we discussed potential impacts with DAU staff, who had  considered the broader effects of program cancellations as part of an  effort to develop guidance on shutting down program efforts after a  contract termination or program cancellation. They shared with us the  results of their data gathering, including comments culled from interviews  with DOD senior acquisition officials. Lastly, we asked former officials of  our case study programs about specific impacts of their program  cancellations.
To determine the adequacy of current DOD guidance on program  cancellations, we reviewed guidance obtained from our prior work. Then  we met with OSD, Army, Air Force, and Navy officials to determine  whether any additional guidance had been issued subsequently, and  whether they considered available guidance adequate for ending a  contract or program. We also discussed the availability of guidance with  DCMA officials, including any changes to the FAR since our prior work.  We reviewed new resources offered by DAU, including a guidebook  published in December 2013.
Furthermore, we obtained views about the adequacy of guidance from  DAU staff, who had identified a need for department-wide guidance for  program officials, especially program managers, on how to shut down a  canceled program. Finally, we solicited views on the adequacy of  available guidance from former case study program officials, though in  many cases these officials were unable to identify guidance they relied on  in addition to the FAR, if any.
To look at the extent to which DOD leveraged various types of assets  from canceled programs, we reviewed FAR and DFARS parts for contract  termination, government property, patents, data, and copyrights, and their  associated clauses to determine if they specified how the various types  are to be disposed when they are no longer needed for contract  performance. We discussed the process for disposing of government  property with DCMA Plant Clearance and Terminations Center officials,  and obtained information on property reutilization, when available. We  also met with an Army intellectual property law attorney to learn about the  treatment of intellectual property assets.
We specifically discussed transfer and leveraging of various program  assets with former officials for our case study programs, and obtained  available documentation. However, officials told us that contract files for  the Comanche and ACS programs were no longer available because the  cases had been settled beyond the federal record retention period. For  these two programs, we had to rely mostly on the memories of former  program officials for information on leveraged assets. However, for both  programs we had other documentation available from our prior work.
To consider the usefulness of contract termination cost estimates, we  primarily asked former program officials whether estimates were  prepared, by whom, and how they were used. We also obtained  documentation, when available, of termination cost estimates from  Contract Funds Status Reports submitted by contractors. These  estimates were no longer available for ACS or Comanche. In addition, we  discussed termination cost estimates with an official from the CAPE. The  scope of this review did not include interviews with contractors.
We supplemented our termination cost estimate work by incorporating  information gathered for a recently issued report on DOD estimates of  contract termination liability. That work included reviewing a DOD  assessment of the extent to which the department considers potential  termination liability as a factor in entering into and terminating contracts.  We also interviewed DOD officials responsible for conducting the  assessment.
In addition to our five case studies of canceled programs, we also  selected a non-generalizable sample of seven current weapon programs  in development and production to ask further questions about DOD’s use  of termination liability estimates. We selected these programs from  among those participating in our annual assessment of weapon systems,  and they included Integrated Air and Missile Defense and Excalibur  Precision 155mm Projectiles (Army), AIM-9X Block II Air-to-Air Missile  and Ship to Shore Connector (Navy), and Family of Advanced Beyond  Line-of-Sight Terminals, GPS III OCX Ground Control Segment, and MQ- 9 Unmanned Aircraft System Reaper (Air Force). These programs were  selected to represent a range of weapon platforms and military services,  and to include both development and production programs. We asked  officials at these programs a standard set of questions to determine how  DOD officials consider termination liability costs prior to contract award  and during contract execution, and to identify guidance used in making  those decisions. Although the responses received from these programs  cannot be generalized, we determined the evidence we obtained from  these programs was sufficient to provide illustrative information on DOD’s  use of termination liability estimates.
We conducted this performance audit from October 2012 to March 2014  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Plant Clearance Process
paragraphs: Description  Within 120 days from effective termination date.
Plant Clearance Officer (PLCO) reviews and accepts, or returns for correction,  within 10 days of receipt from contractor. PLCO verifies within 20 days of  acceptance.
Standard screening period of 46 days begins upon Plant Clearance Officer’s  acceptance of inventory schedule. Agency has 20 days to screen property for  other use within agency, transfer educationally useful equipment to other federal  agencies, and to schools and non-profits.
Disposal in accordance with agency procedures  without report to General Services Administration  (GSA)
1) property for abandonment or destruction  2) property furnished to nonappropriated fund activities  3) foreign excess personal property  4) scrap, except scrap condition aircraft  5) perishables  6) trading stamps and bonus goods  7) hazardous waste or toxic and hazardous materials  8) controlled substances  9) property dangerous to public health and safety  10) classified items or national security-sensitive property  Also, nuclear materials in accordance with Nuclear Regulatory Commission,  applicable state licenses, applicable federal regulations, and agency regulations  No later than 21st day, Plant Clearance Officer is to submit revised schedules and  report to General Services Administration for reuse within government or donation  as surplus property.
Screening by other federal agencies, and for  donations  21 days concurrent screening plus 5 days donation processing. Transfer requests  on first-come, first-served basis; General Services Administration (GSA) transmits  approval to Plant Clearance Officer. Property not available to donees until  screening completion. Days 42-46 reserved for GSA to make allocation.
Abandonment, destruction, or sale after screening  Surplus property having completed screening shall be sold in accordance with  Federal Management Regulation policy. Agencies may specify implementing  procedures.  If inventory unsuccessfully screened, has no commercial value, does not require  demilitarization, and does not constitute a danger to public health or welfare, the  Plant Clearance Officer can direct the contractor to destroy property, or abandon it  at the contractor or subcontractor premises (with contractor consent for sensitive  property).
Plant Clearance Officer shall promptly prepare report following disposition of  property identified on inventory schedule and crediting of any related proceeds.  The report is to identify any lost or otherwise unaccounted for property and  changes in quantity or value made by the contractor after submission of initial  schedule. The report is to be provided to the termination contracting officer.
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Katherine Trimble (Assistant  Director), Ann Marie Udale (Analyst-in-Charge), Peter Anderson, Virginia  Chanley, Maria Durant, Kristine Hassinger, John Krump, Carol Petersen,  Raffaele Roffo, Matthew Shaffer, and Roxanna Sun made key  contributions to this report.