DOD weapon programs range from highly complex and sophisticated aircraft, missile interceptors, submarines, and space-based sensors, to new communication and ground control systems that support and interconnect this equipment, to smaller, less complex systems that support the warfighter. Increasingly, individual weapon system investments are getting more complex and more dependent on software. Moreover, in many cases, weapon systems are also expected to incorporate technologies that push the state-of-the-art while operating in harsh and even untested environments—adding daunting technical challenges that increase the cost and complexity of development. DOD typically executes several primary contracts over the life cycle of a weapon system program. These contracts generally cover development, production, or maintenance efforts. Although a program and its contracts are related, they are separate, distinct efforts that can end separately. For the purposes of this review, we refer to the end of a program as a cancellation, and the end of a contract as a termination. For example, DOD may cancel a program but continue a contract related to that program. Alternatively, DOD can terminate a contract but continue the program. In some cases, programs may not be formally canceled, but instead restructured. For example, DOD could continue a program, but change requirements and modify a contract or even change contractors to better align the program with available resources. Cancellation of major programs has not been common in the past, and we have noted in prior work that the acquisition of weapon systems involves strong incentives to reduce the risk that a program will be interrupted or called into question. When programs have been canceled, cost, schedule, and performance problems have often been cited as reasons for this decision, and officials have noted that program cancellation is perceived as failure. However, in some circumstances, program cancellation may be the best choice. As we reported in March 2013, DOD’s 2012 portfolio of 86 major defense acquisition programs is estimated to cost a total of $1.6 trillion. This represents a decrease of about $44 billion from the previous year, due primarily to program cancellations and restructurings. However, compared to original estimates, the total cost of the portfolio has increased by over $400 billion with an average delay of 27 months in the delivery of initial operating capability, which suggests that opportunities exist to improve efficiencies in DOD’s portfolio. In 2008, we concluded that from an investment portfolio perspective, cancellation can be a valuable tool in responding to long-term fiscal imbalances. In 2009, Congress acted to make cancellation the default for programs that experience a breach of a critical cost growth threshold. In recent years, DOD has cited renewed efforts to end troubled programs and to achieve a better capabilities balance, and to that end DOD’s annual budget submissions in recent years have each reflected major program cancellations—at least three per year and as many as eight. In response to potential cancellations, defense stakeholders have sometimes expressed concerns that terminating a contract will cost more than completing it. In our 2008 report, we reviewed DOD’s past experience with program cancellations and contract terminations and found that there are limited circumstances in which it could cost the government more to terminate a contract than to complete it. Of the eight contracts we reviewed, we did not find any cases in which the total amount paid on a terminated contract exceeded the estimated contract price. We recommended DOD review, and as needed amend, termination guidance to ensure it consistently identified the conditions under which it is appropriate to end programs or contracts, and provided the knowledge needed to use terminations as an investment portfolio tool. DOD concurred with this recommendation. However, DOD subsequently determined that no changes to guidance were needed. We also highlighted important lessons for DOD in making decisions to cancel individual programs as well as in managing its broader investment portfolio. For example, when considering cancellation of individual programs, contract termination costs are generally not a compelling reason to continue programs or contracts that otherwise warrant ending. Moreover, while incurred or “sunk” costs in programs being considered for cancellation may be substantial, they must be paid regardless of whether or not a contract is terminated. Therefore, the decision to terminate a contract or cancel a program should not be driven by sunk costs. In addition, when a contract warrants termination, the decision should be made as soon as possible because delaying a termination almost always results in higher settlement costs to the government. Finally, we noted when a program or contract ends, DOD can make choices that would allow the government to retain value from the work completed. Weapon programs acquire various types of assets throughout a program’s life cycle; assets that may include property, including computer hardware, equipment, material, special tooling, facilities, and real property; and software and technical data, which can include manuals, engineering drawings, specifications, data rights, and data delivery. When a program ends, activities related to shutting down the program include determining how to retain the most value from the government’s investment, which, for the purposes of this report, is the government’s expenditures on the program up to the date of cancellation. This can include terminating or otherwise restructuring contracts, and assessing how best to leverage the program’s assets. For example, program officials might identify technologies and facilities that can be realigned for use on other programs. Alternately, the government may ultimately sell assets or determine that they are not suitable for continued development. In some cases, assets that are not reused may be destroyed or dismantled. Figure 1 illustrates some examples of program assets. The types of assets a program owns will depend, in part, on the phase of the program. A program in production is more likely to own hardware and other types of tangible assets than a program in development. For this report, our sample included a range of programs in the development phase. Program cancellations often result in contract terminations, creating immediate impacts on programs and their contractors as program offices work with the prime contractor to end work under the contract and determine the costs associated with contract termination. However, the impact of a weapon program cancellation can extend beyond the immediate program and its contractors and ultimately affect a broader industrial base and local economies, as well as other programs and agencies. Consideration of these various effects and consultation with various stakeholders enables officials to have a better understanding of the costs, benefits, and legal requirements involved in a particular cancellation and to take steps that can help mitigate negative effects. In addition, shutdown activities can take years. While the immediate task of shutting down the program falls to the program office, there are other necessary tasks, such as reallocating program funds, that fall beyond the program’s purview. Figure 2 illustrates the potential effects to be considered when a program is canceled. A canceled program does not stop incurring costs immediately upon cancellation. Once a program has been canceled, the program office must undertake shutdown activities, which include determining whether to terminate or otherwise restructure contracts and determining how best to leverage the program’s assets. Conducting tasks such as negotiating with contractors and disposing of inventory requires program staff to remain on duty and can incur cost to the government. When cancellation leads to the termination of a contract for convenience, the Federal Acquisition Regulation (FAR) directs the contractor to submit a proposal to the government for the amount of reimbursement it believes it is entitled to within one year of the official date of the termination. This may include termination costs, the incurred costs for work already performed by the contractor, and possibly a reasonable profit or fee on completed work. Termination costs, which represent costs that the government would not have otherwise incurred if it did not terminate the contract, can include expenses such as preparing the settlement proposal, negotiating with subcontractors, and disposing of inventory. With assistance from the Defense Contracting Audit Agency to audit the settlement proposals, DOD then enters into negotiations with the contractor to determine a settlement. Ultimately, DOD will likely be required to reimburse the contractor for termination costs, which can reach up to the hundreds of millions of dollars for a large program, though these costs can be small relative to the full cost of the program. The final amount of termination cost owed will not be determined until final settlement. For canceled programs, the extent of work associated with a shutdown is sometimes not immediately clear. One case study program official told us decision makers did not initially understand the need to continue to dedicate resources to a canceled program to fund shutdown activities. When DOD cancels a program, it must address not only the costs of terminating any related contracts but the broader impact across the agency, including deciding on how to reallocate funds originally budgeted for the canceled program and determining when those funds will be available. This task extends beyond the canceled weapon system’s program office. For example, when the Army’s Comanche Helicopter program was canceled in 2004, the Army proposed an investment strategy that would redistribute $14.6 billion of planned Comanche funding through fiscal year 2011 to enhance a broad range of Army aviation modernization efforts. After securing approval from the Secretary of Defense and the President, the Army briefed Congress on its plan to redistribute the remaining Comanche funds within the Army aviation portfolio. Based on those discussions, the Army developed a plan to redistribute this funding to 30 different projects and other weapon programs. Cancellation of a program can lead to a delay in providing capabilities to the warfighter. As a result, DOD may need to increase funds for other programs to extend the life of existing or legacy systems. For example, the Navy’s VH-71 program was initiated to build a replacement for the current presidential helicopters with improved transportation, communication, and security capabilities. The Navy’s original plan was to field the first of these new helicopters by 2008. However, after the program experienced schedule slips and continued cost increases, the program was canceled in 2009. The Navy has since initiated a successor VXX Presidential Helicopter Replacement Program, which is still in the early stages of development and not expected to have initial operational capability until fiscal year 2020. Until that time, the Navy plans to continue to maintain the legacy helicopters that it had otherwise planned to replace with the VH-71. Additionally, in May 2007, we found that the ACS contract termination resulted in about an additional $900 million investment beyond what had been previously planned to maintain the legacy systems. Another example is the Army’s canceled FCS, a multibillion dollar development program originally consisting of 18 manned and unmanned systems tied together by an extensive communications and information network. Initiated in 2003, the first Army combat teams were planned to be FCS-equipped in 2011, while the rest would be fully equipped by 2032. When the Manned Ground Vehicle portion of the program was canceled in 2009, the Army altered its modernization strategy but continued to develop unmanned vehicles and improved network and technology sensors in order to deliver them to selected combat teams starting in fiscal year 2011. Faced with disappointing test results and high costs, FCS was fully canceled in 2011. The Army has since incorporated its FCS efforts into a new acquisition strategy to modernize its network, but it does not plan to include the canceled FCS components, and other technologies to modernize the network will be delivered later than originally planned through FCS. DOD relies on a broad industrial base that is capable of producing complex products that can be unique to defense applications. When a major weapon program is canceled, it can impact sectors within the industrial base and potentially diminish the capacity to produce key capabilities. The Weapon System Acquisition Reform Act of 2009 mandated that DOD add the impact of the cancellation of major weapon systems in its assessment of the national technology and industrial base that it provides annually to Congress. For example, the 2012 assessment reported on the impact of the 2009 cancellation of the Manned Ground Vehicle portion of the Army’s FCS, which was supposed to replace current Army vehicles such as the M1 Abrams Tank and M-2 Bradley Fighting Vehicle. In our prior work, we found that smaller companies that rely on government contracts can be particularly affected by program cancellations. In addition, program cancellation can impact a community that is economically dependent on a contractor facility. In the example noted above, changes to the Army’s ground vehicle program can have an economic impact in cities and towns within the Upper Midwest and Great Lakes region of the United States, where a large portion of military ground vehicle production is based. When a program is canceled, personnel working in the program are immediately affected. One of the challenges program managers face is addressing potential deflated worker morale as well as the logistics of transferring personnel while maintaining essential employees to effectively shut down the program. DOD officials have noted that their goals in addressing personnel issues resulting from canceled programs are to maintain open communications and minimize disruptions. For example, one senior official suggested that during a shutdown, the program office could offer training to remaining program staff in order to maintain skills that were lost due to the departure of staff that had already left the program. For some programs, cancellation may impact additional stakeholders. For example, the Army’s Medium Extended Air Defense System (MEADS), which DOD proposed to no longer fund after fiscal year 2013, was being developed by the Army in collaboration with two other countries. DOD’s cancellation of MEADS would leave the two partners with considerably fewer resources than anticipated to complete development. Consulting the broad range of stakeholders who will be affected by, or have knowledge of the effects of, a program cancellation, can enable officials to be aware of important effects of program cancellation. For example, one former Comanche official stated that the Army would have had more accurate knowledge of what funds were available for programs after cancellation and avoided delays during the contract termination process if it had involved legal advisors and the Comptroller’s office earlier in the process. As the effects of a major weapon program cancellation extend far beyond the program office, there are multiple stakeholders representing various interests that can assist in enabling program officials to fully assess these effects as shown in table 2. DOD has recently developed guidance for shutting down canceled programs. In the absence of guidance, officials from our case study programs reported relying heavily on experienced staff to navigate the shutdown process. The new online guidance addresses the spectrum of effects of cancellation and offers a variety of tools for program managers. However, current plans for notifying program managers of this resource may not be sufficient. Furthermore, the guidance was designed to serve as a lessons learned mechanism but there have been few actual contributions of lessons learned from the acquisition community. The decision to cancel a program is often communicated by an acquisition decision memorandum, which documents the decision to cancel and provides objectives for program officials to accomplish during the shutdown process. However, at the time the programs we selected for case study were canceled, DOD did not have department-wide guidance on how to achieve those objectives. In 2008 we recommended that DOD review, and as needed amend, guidance on terminations across the military services and DOD entities to ensure that guidance consistently identified the conditions under which it is appropriate to end programs or contracts, and that guidance provided the necessary knowledge for using terminations as an investment portfolio management tool. Following our report, DOD conducted a review and determined that no changes to DOD guidance were needed. The FAR and Defense Federal Acquisition Regulation Supplement (DFARS) contain guidance on the process of terminating contracts; however, contract termination is only one of several aspects of program cancellation that program officials must consider when shutting down a program. As discussed earlier, other areas of consideration include disposing of program assets and overseeing the budget during shutdown of the program. Officials from most of our case study programs stated that at the time they were shutting down their canceled programs, they were not aware of department-wide or service specific guidance addressing these other aspects of program shutdown. Army officials were able to identify some Army guidance on shutting down programs, although it was unclear whether officials from our Army case study programs referred to this guidance during the shutdown process for those programs. During the course of this review, Air Force officials provided documentation of new draft Air Force guidance, which was not yet in effect when the TSAT program was canceled. Navy officials were unable to identify Navy guidance on the subject. DOD officials confirmed that there was no department-wide guidance on program cancellation and shutdown. Officials from most of our case study programs told us that without department-wide guidance, they learned about cancellation and shutdown by working with personnel who had been through cancellations before. For example, FCS officials told us that a number of their program office staff, including those working on disposition of assets, had worked on the Crusader cancellation. In addition, the DCMA termination contracting officer working on FCS had also handled the Crusader cancellation. Similarly, a former VH-71 official told us that he was brought on as program manager for VH-71 when officials began to think about cancellation, because he had been through cancellation of a major program before. The Comanche contracting officer stated that he had been through several cancellations before helping to shut down the Comanche program. ACS officials reported relying heavily on the DCMA termination contracting officer for guidance. This continuity seems to have worked well for our case study programs. However, several of the experts in this area from our case study programs have retired or will soon be eligible to do so, and DOD may not be able to continue to make this knowledge accessible to program managers without a more institutionalized mechanism for doing so. Recognizing the need for guidance on shutting down programs, in 2009 the Defense Acquisition University (DAU) began development of a special interest website called Smart Shutdown, which offers a variety of resources related to program shutdown. In December 2013, DAU finalized a guidebook geared toward program managers which is now available through the website. The guidebook is constructed to make program officials aware of various areas to be considered in program cancellation, and to help them shut down a program as efficiently as possible, while maintaining the best value for the government. These areas for consideration include Personnel, Operational Capability/Requirement, Technology, Facilities/Hardware/Software/Program Security, Contracts, and Budget. DAU officials told us that because the specific issues faced by a program manager during shutdown will vary substantially by program, the guidebook is intended to provide insight on common issues to be considered throughout the process, rather than detailed prescriptive guidance. Other materials available through the website include a sample briefing to leadership, and a database of considerations when disposing of certain assets. This guidance is intended to be advisory in nature. DAU disseminates information about this resource through a required program manager course; however, program managers are required to take this course once before becoming program managers of high value programs. Therefore, program managers who have taken the course before this resource was developed will not have received this information. Furthermore, it may be many years between a program manager’s completion of this course and the point at which he or she is actually faced with the prospect of shutting down a program. Therefore, mentioning the Smart Shutdown materials in a one-time program manager course does not ensure that a program manager faced with cancellation will be aware of this resource. Internal control standards call for agencies to undertake control activities, which include mechanisms to help ensure that management’s directives are carried out. In several cases, acquisition decision memorandums directed program officials to retain maximum value from DOD’s investment in canceled programs. However, without awareness of DAU’s Smart Shutdown resources, program officials may lack the knowledge necessary to leverage investments effectively as directed. Inclusion of information on Smart Shutdown in an acquisition decision memorandum communicating cancellation of the program, for example, would be one mechanism to better ensure program managers have information on this resource when they need it. To develop this guidance, DAU interviewed individuals who had experienced program cancellation. DAU officials stated that the resource will benefit from more input from experienced members of the acquisitions community. The website is constructed to allow input from the acquisition community in identifying goals, processes, issues, best practices, and considerations in all aspects of program cancellation activities. In particular, its database is designed to allow officials who have experienced program cancellation to share lessons learned. According to the Defense Acquisition Guidebook, which provides acquisition best practices, program managers can gain insight into risks, uncertainties, and opportunities that their programs may encounter by reviewing experiences of other programs described in lessons learned. The guidebook states that program managers should take advantage of these types of resources when available. However, DAU officials told us that they have received very few contributions from the acquisition community to date. In fact, we found that several of our case study programs had written compilations of lessons learned during the cancellation process, but DAU officials were not aware of most of these lessons learned documents. Without continued contribution of lessons learned from program officials with experience shutting down canceled programs, DOD will be missing opportunities to build and share knowledge on program shutdown. We could not determine the extent to which DOD leveraged the assets in our case studies because DOD does not have a way to track all types of assets. DOD uses a basic, federal government-wide process and IT system for reutilizing, disposing of, and tracking government property. However, both the process and the system are designed to track specific pieces of equipment, materials, and tools and are not designed to track intellectual property, such as rights to technical data and software, partially developed technologies, or even other types of assets such as real property and hazardous materials. Because there is no department- wide process or requirement for disseminating information about all assets available for reuse, DOD cannot help ensure that technologies, hardware, and other potentially useful assets can be delivered to parties who can best develop them further. There are inherent difficulties in transferring the full range of assets on any program, given the uniqueness of a program’s requirements. Nevertheless, given that DOD may spend billions of dollars before deciding to cancel a program, it is in the government’s interest to ensure these investments are leveraged to the maximum extent possible. The FAR specifies a government-wide process, which DOD refers to as plant clearance, for reutilizing and disposing of property no longer required for continued performance of a contract. Figure 3 illustrates the basic process. The plant clearance process applies to property, such as material, equipment, special tooling, and special test equipment, which may be government-furnished or contractor-acquired in performance of a contract. However, the process does not apply to software and technical data or types of assets such as real property or partially developed technologies. Table 3 provides examples of property processed through and outside of the process. The Plant Clearance Automated Reutilization Screening System (PCARSS) automates the process for reporting, screening, requisitioning, and disposing of excess property. When a contract is terminated or completed, contractors are to enter inventories of excess property directly into PCARSS. DOD components and federal agencies have access to PCARSS, in which they can review available property and requisition that in which they are interested. The FAR identifies reutilization priorities for plant clearance of property. In descending order, these priorities are reuse within the owning agency, transfer of educationally useful equipment to schools and nonprofit organizations, and reuse within the federal government or donation as surplus property. Some case study program officials found PCARSS cumbersome. According to DCMA plant clearance officials, a contractor may submit over 1,000 inventory schedules, including those of subcontractors, for a contract termination. These inventory schedules can contain many thousands of line items, making it time-consuming and difficult to identify property for potential reuse. Also, many of the items listed tend to be smaller components, such as cables and screws. Table 4 cites examples of items listed in a PCARSS inventory schedule. For our five case study programs, which were all canceled during program development, the most valuable program assets for transfer were often partially developed systems or technologies, which can be a combination of software, technical data, or property. These types of assets were generally not processed through PCARSS nor tracked in any other systematic way. Table 5 lists examples of major assets transferred from some of our canceled case study programs, but does not represent a comprehensive accounting of all major assets for the programs. Former officials of our case study programs were not able to provide comprehensive lists of asset disposition because they did not have systems to track this information. They tracked transfers of assets processed outside of the plant clearance process mostly through program-developed spreadsheets. The FCS program also used contracting officer letters to document the transfers. When an item was transferred for use on another contract, the transfer was documented by contract modifications to both the gaining and losing contracts. Though they could not provide comprehensive lists of asset disposition, program officials reported that a substantial amount of assets was transferred outside the PCARSS system. The Comanche program, for instance, reported reutilizing about two-thirds of its assets; less than 1 percent was processed through PCARSS. Officials of the VH-71 and ACS programs, both of whose acquisition strategies were to integrate existing or developing systems onto a base platform, told us they expected that program assets would be used on successor or related programs and therefore bypassed the plant clearance process. In addition, PCARSS is not used for assets that the government allows the contractor to continue to use after a program is terminated, or items sold back to the contractor or to third parties. For example, the VH-71 program contractor, with program office and DCMA termination contracting officer concurrence, negotiated a sale to Canada of nine prototype helicopters. This transaction did not go through PCARSS. The acquisition decision memorandums cancelling several of our case study programs or terminating related contracts instructed program officials to leverage DOD’s investment in the programs. But there is no DOD requirement to track all program assets. Moreover, DOD has no comprehensive mechanisms in place to facilitate activities, such as sharing information on available assets, valuing these assets, and tracking transfer, which would help to ensure these assets are effectively leveraged as directed. Without systems to track disposition of assets, the government may not be able to ensure it is getting the most value for assets. Technical data and software, which are also dealt with outside the disposition process, present unique challenges to tracking. According to a DOD report, information technology represents a considerable proportion of defense weapon programs underway—a proportion that is likely to increase in the future. Some DOD program developments have extensively relied on software and technical data. For example, the FCS development effort projected that 95 percent of FCS functionality would be controlled by software. However, technical data rights are not tracked centrally, but rather, according to DOD officials, only as a component of contracts that involve such rights. DOD and its contractors are to follow the FAR and DFARS procedures that outline the government’s rights to acquire, use, and disclose technical data. Defense procedures require contractors to identify in the contract the technical data rights to be granted to the government and delivered under the contract. Contractors generally identify the government’s rights to technical data at the beginning of the contract process and prior to delivery of the technical data using the Contract Data Requirements List (CDRL)—an exhibit to the contract that contains the technical data deliverables. The technical data rights assigned to the government can vary depending on the nature of the data, source of the funding and negotiated terms of the contract. The standard data rights range from the least restrictive, or unlimited and government purpose rights, to more restrictive rights, such as limited or special rights. Although rights to data are usually defined when a contract is executed, the government may adjust or renegotiate those rights during the life of a contract. DOD policy requires that prior to issuing a contract solicitation, program managers for major acquisition programs must assess the merits of including a priced contract option for the future delivery of technical data and intellectual property rights that are not acquired in the initial contract. DOD officials mentioned several reasons the tracking and transfer of data rights can be problematic and challenging when a contract is terminated. Specifically, the following: Limited recordkeeping for unrestricted data rights—The CDRL does not contain all technical data items delivered to the government. The DFARS requires contractors to identify technical data rights with restrictions in the contracts. According to an Army intellectual property attorney, the CDRL generally contains the restricted items and not the items that are unrestricted such as those assigned government purpose or unlimited rights. Consequently, the unrestricted rights are not evident at contract termination and it can be difficult to track all technical data rights that should be accounted for when a contract is terminated. Complications transferring a combination of software under contract—When a system under development includes a combination of software products, each with different types of rights assigned, the government may need to identify all of the software under the contract and renegotiate the data rights. For example, when the FCS program was canceled, the Army planned to transfer the System of Systems Common Operating Environment (SOSCOE) to the Army Aviation and Missile Research Development and Engineering Center for further development and distribution to program offices that had use for it. However, SOSCOE contained a combination of software that was DOD-developed, developed by FCS subcontractors, developed for commercial purposes, and some that was open source. Before transferring SOSCOE, the Army worked with the subcontractor to develop a single distribution agreement for the government purpose rights and open source software, and a special license agreement for the portion of software developed for commercial purposes. Difficulty in determining the value of technical data and software before transferring data rights—According to an Army intellectual property attorney, the assignment of value to technical data and software rights is not clearly understood and these values are assigned, in most cases, by the contractor. Further, some contractors are more sophisticated than others in determining the value of data rights and maintaining good documentation. In some cases, the government and contractors have had difficulties valuing rights in technical data even if line items in the contract identify the technical data to be delivered and specify the assigned value for each item. The government may need to contest the contractors’ assigned values to technical data rights before it can transfer those rights, particularly when there are unknowns with a program, such as the case with research and development programs where the development approach may not be stable and could necessitate changes to the contract. Loss of knowledgeable personnel for tracking transfers— According to DOD officials, staff familiar with the contract and the government’s technical data rights would be the best source for tracking their transfer. However, when a program is canceled, valuable contract knowledge can be lost when staff resign, are reassigned, or are transferred to another program. The uniqueness of weapon system requirements presents inherent challenges to transferring assets. Because technologies meet very specific needs and are usually costly to develop, there may be a very narrow set of organizations interested and able to afford continuing the development effort. Some former program officials told us that transferring existing program assets depends on knowledge of other programs that might be able to use them, which often happens through networking. For example, the Comanche program had staff with aircraft technology expertise who helped find appropriate recipients for equipment. A former FCS official told us that he tried—by speaking at trade shows, trying to publish technical papers, and networking with people—to make the user community aware that SOSCOE was available for distribution. However, he commented that these efforts were not effective, and noted that the lack of an Army infrastructure for sharing information hampered the efforts. After spending approximately $900 million under the FCS contract to develop the SOSCOE, the Army transferred it to a research and development center for further development. However, while the center initially funded additional development, the former FCS official indicated that the SOSCOE will no longer be funded. Additionally, a technology may become irrelevant before it is fully developed. For example, a former ACS official told us that technologies under development for ACS were of little use to the eventual replacement program because it had different requirements. Lastly, while not citing specific cases, other program and Office of the Secretary of Defense officials commented that it is not unusual to be ultimately unable to transfer a technology, given the uniqueness of the technologies being developed. Estimates of contract termination costs are sometimes perceived by program officials as a useful tool in managing a portfolio of investments and informing contract termination planning and budgeting decisions. However, there are some uncertainties about costs that cannot be addressed until a program is actually terminated, making estimates less useful for the purpose of informing the decision to terminate a contract. Thus, while DOD generally has access to contract termination cost estimates, they have limitations due to issues such as limited visibility into cost assumptions. According to DOD officials, estimates of termination costs can be useful for contract termination planning and budgeting decisions. However, some issues involving termination costs may not be resolved until after the termination settlement process begins. Contractors have a year from the time of termination to produce a settlement proposal and, according to a senior contracting termination official, contractors often request extensions of this deadline for large and complex programs. During this time, the contractor is to resolve areas of uncertainty and take a position as to which costs can be claimed as termination costs. For example, former contracting officials for the VH-71 program noted that there was uncertainty during contract execution regarding whether the cost of certain contractor investments, such as facilities and test vehicles, could be claimed by the contractor as termination costs. Contractors and program offices must also agree on the price of any contract changes that had been unpriced at the time of termination. In the case of FCS, a former program official said that a series of partial contract terminations and changes to the contract led to frequent fluctuations in cost estimates that made it more difficult to make an accurate estimate of the cost of terminating the contract. DCMA officials told us final settlement for the FCS contract is not expected until 2016. In two of the five canceled programs we reviewed, a major obstacle to settling the terminated contract was determining the fees due to the contractors for their work. A senior contract termination official said that there are a number of other termination costs that can be significant but difficult to calculate until after the termination process has begun, including the costs of stopping work until the contract is terminated and costs associated with the Worker Adjustment and Retraining Notification Act, which requires contractors to provide advance notice in the event of a mass termination or plant closure. As a result, there will always be a level of uncertainty in cost estimates prepared before termination, which limits the utility of those estimates. Generally, DOD weapon programs receive regular estimates of termination costs from their contractors in contract funds status reports (CFSR). Termination cost estimates are included among other cost data in these reports, which contractors are required to regularly provide the program office to help forecast and plan funding for the program. The CFSRs we reviewed included individual estimates of the cost of terminating the contract during each of the immediate months or quarters, then estimates for termination costs in subsequent quarterly or annual periods thereafter. In response to our survey of seven major weapon programs currently in development or production, conducted as part of our related work, the five programs that reported a source for information used to develop their termination estimates identified their contractor, usually through CFSRs. Six of the respondents reported considering termination liability estimates during contract execution while one reported considering termination liability estimates when awarding the contract. Some program officials we spoke with were uncertain of how contractors arrived at these estimates, and were not aware of any additional information provided by contractors to support these estimates. For two of the canceled programs we reviewed, estimated termination costs provided by the contractor increased as terminations became more likely, in one example by almost 50 percent from the previous quarter. A former program official that we spoke with said that these increases suggest that when producing their termination cost estimates, the contractors were including all costs that could conceivably be allowable under the FAR. In at least one case, uncertainty about costs included in the contractor estimate led to considerable uncertainty in the program office estimate that was submitted to decision makers. DOD also has access to its own independent termination cost estimates from the Office of Cost Assessment and Program Evaluation (CAPE); however, their usefulness to DOD during the consideration or planning of a program cancellation is limited because they are usually developed before contracts are awarded. At major milestone decisions in which weapon programs enter the next acquisition phase, CAPE produces independent costs assessments for DOD that include estimated termination costs for the program. While these estimates are produced independently within DOD, unlike those in the CFSRs, CAPE might not provide estimates between milestone reviews, which can be many years apart. For example, the VH-71 was canceled in 2009, while its last milestone review was in 2005. Furthermore, when CAPE produces estimates at the milestone phase, it is generally before a contract is awarded, so the estimates do not reflect any data from contract execution. According to a CAPE cost assessor, contract termination estimates are generally not a major point of consideration during DOD milestone reviews. Though it would not have applied to the canceled programs we reviewed, the recently updated weapons acquisition instruction released by DOD in November 2013 requires a termination liability estimate for all large weapon acquisitions prior to contract award. Specifically, the instructions require weapon programs for which termination liability can be reasonably expected to exceed $100 million to include an estimate of the potential termination liability in the program’s acquisition strategy as well as how the liability is likely to increase or decrease during execution of the contract. In limited instances, contractors may provide estimates of some termination costs prior to contract execution. Contracts for weapon programs may contain a special termination cost (STC) clause, which reflects the program and the contractor’s estimation of certain termination costs prior to award of the contract. However, according to officials, these clauses are rare and do not necessarily represent all types of allowable termination costs. Specifically, these clauses define a maximum amount of specific termination costs that will be payable to the contractor if the contract is terminated. DFARS states that this clause may be used when approved by an agency head, when the contract meets certain duration and investment thresholds, and when adequate funds are available to cover potential liability. Although contracts for two of the programs we reviewed contained such clauses, FCS and Comanche, a senior DCMA contract termination official said that he has rarely encountered these clauses in the terminated contracts he has reviewed. For those two programs, termination cost uncertainty remained despite the existence of the clause. In the case of the Comanche program, the STC clause set the special termination cost limit for the contract at $123.2 million for fiscal year 2004, the year in which it was canceled. When the Army requested an estimate from the contractor for the final termination costs, the contractor responded with a rough order of magnitude of $640 million, noting that the estimate included other termination costs not covered by the STC clause. The STC clause did not provide a basis for the program office to evaluate these other potential termination costs included in the contractor’s rough order of magnitude termination cost estimate. When the contract termination was settled, the final termination costs were around $143 million after program officials made efforts to minimize these costs. In the case of FCS, officials were unclear how the STC clause in the primary development contract would be applied since the program was canceled through a series of partial contract terminations, and they thought costs for each partial termination were unlikely to reach the limit in the STC clause. The government and the prime contractor have not yet reached a final termination settlement on that contract. The magnitude of DOD’s investments in major acquisition programs demands good stewardship when it is decided a program should be canceled. Though cancellations of major programs to date have been relatively rare, a single one may represent a multibillion dollar proposition that can have far-reaching effects on DOD, the industrial base, and even local economies. DAU has developed guidance that addresses the spectrum of issues faced in a cancellation, though current efforts may be insufficient to ensure that program managers who are faced with a program cancellation will be aware of this guidance. Without this awareness, program officials may lack the knowledge necessary to manage the shutdown process and leverage related investments as effectively as possible. Furthermore, without the continued input of lessons learned from past cancellations, DOD will miss opportunities to continue to build and share knowledge regarding program shutdown. Leveraging investments in canceled programs is complicated because there is no DOD process to track all program assets or disseminate information about assets available for reuse. DOD uses a process that provides visibility over specific tools, materials, and equipment that support a program being canceled, but it has little visibility over perhaps more valuable, but less tangible, assets such as software or partially developed technology, as our case studies demonstrated. Leveraging these investments is further complicated by the fact that the technologies may be unique and expensive to continue to develop, thereby limiting the pool of organizations that want partially developed technologies, specialized hardware or other property. Without more comprehensive processes to track and disseminate information about all available assets, DOD’s efforts to ensure that it gets the most value for its investments will continue to be hampered. To improve DOD’s ability to ensure it is fully leveraging investments made in canceled programs, we recommend that the Secretary of Defense direct the Office of Acquisition, Technology, and Logistics to take the following three actions: (1) Direct program officials to refer to DAU’s Smart Shutdown guidance throughout the process of shutting down the program. This could be done, for example, through acquisition decision memorandums. (2) Direct program officials to provide lessons learned, if applicable, related to program cancellation and shut down to DAU to enhance its Smart Shutdown toolkit. (3) Develop department-wide processes to improve tracking of assets, including technical data and software, and dissemination of information about assets available for reuse after programs are canceled. DOD provided us with written comments on a draft of this report, which are reprinted in appendix III. DOD partially concurred with our recommendation to direct program officials to refer to DAU’s Smart Shutdown guidance. DOD agreed that Smart Shutdown is a beneficial tool, but stated that it planned to highly encourage, rather than direct, reference to this resource. While we support DAU’s efforts to further promote the toolkit, we question whether these efforts will be sufficient to ensure that all program officials faced with cancellation have knowledge about cancellation and shutdown procedures to leverage investments effectively. Thus, we continue to believe that direction to program officials to refer to the Smart Shutdown toolkit would be a more effective mechanism to ensure that management’s goals for the shutdown would be achieved. As noted in this report, officials from most of our case study programs told us that without department-wide guidance, they learned about cancellation and shutdown by working with personnel who had been through cancellations before. However, several of the experts in this area from the case study programs have retired or soon will be eligible to do so. When a major program is canceled, acquisition decision memorandums which contain instructions for the cancellation process are sometimes issued to program officials, and these may direct program officials to work to achieve DOD goals such as retaining maximum value from DOD’s investment in the program. Including direction in this acquisition decision memorandum for program officials to refer to the Smart Shutdown resource during this process would be one way to ensure program officials are aware of a resource that will help them to best achieve management’s objectives. DOD also partially concurred with our recommendation to direct program officials to submit lessons learned after cancellation, again stating that it would highly encourage, rather than direct, this action. We continue to believe that direction to submit lessons learned, where applicable, is the most effective way to ensure knowledge about cancellation is captured and shared. As discussed in this report, we found that DAU had received few contributions of lessons learned. Without such direction, there may be little incentive for program officials to take the time to share lessons learned before moving on to a new assignment. DOD concurred with our recommendation to develop department-wide processes to improve tracking of assets, including technical data and software, and dissemination of information about assets available for reuse after programs are canceled. However, DOD did not provide information on any specific actions it planned to take to address this recommendation. We are sending copies of this report to the Secretary of Defense and interested congressional committees. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV. The Senate report accompanying the National Defense Authorization Act for Fiscal Year 2013 mandated GAO to review and report on issues associated with contract terminations and program cancellations of major defense acquisition programs (MDAP). This report examines (1) the effects of program cancellation, (2) the adequacy of current Department of Defense (DOD) guidance on program cancellations, (3) the extent to which DOD leveraged—by transferring for further development or use on other programs—various types of assets from canceled programs, and (4) the usefulness of contract termination cost estimates. We reviewed relevant laws, regulations, and guidance related to contract terminations, financial management, cost estimating, and government property, including rights to technical data. We also reviewed our prior reports on contract terminations and program cancellations, other related reports, and other literature. We interviewed DOD officials—including former key staff of canceled programs—at the Office of the Secretary of Defense (OSD), Air Force, Army, and Navy to discuss contract terminations and program cancellations. In addition, we met with Defense Contract Management Agency (DCMA) Terminations Center and Plant Clearance officials to discuss contract termination procedures, and obtained relevant information. We also interviewed officials at the Defense Acquisition University (DAU) and the DOD Office of Cost Assessment and Program Evaluation (CAPE) to discuss, respectively, issues to consider in shutting down a program and termination cost estimates. Our work included five non-generalizable case studies of programs canceled during development: Aerial Common Sensor (ACS), Comanche Helicopter, Future Combat System (FCS), Transformational Satellite Communications System (TSAT), and VH-71 Presidential Helicopter Replacement Program (VH-71). The Senate report mandated that we review the FCS and VH-71 cancellations. In addition, we selected FCS and VH-71 for review because these were recent cancellations of expensive and complex DOD programs, and thus illustrative of current issues faced by program managers in cancellation of major defense acquisition programs. To select the other cases, we obtained DCMA data on the highest dollar contract terminations since our 2008 report associated with canceled programs. In addition, we considered cancellations reviewed for our 2008 report where there was potential to assess additional activity since that report. We then selected cases with completed settlements and the potential for follow-up on leveraged assets, and also included in our total sample at least one case each for the Army, Navy, and Air Force. At the time they were active, all of these programs met the expected investment threshold to be designated Acquisition Category I – the current equivalent of which means the program would require more than $480 million in fiscal year 2014 dollars to complete development, testing and evaluation. In several cases, investment in these programs had reached into the billions at the time of contract termination. Although the results of these case studies cannot be generalized, we determined the evidence we obtained from the case studies was sufficient to provide illustrative information on cases from across the Army, Navy and Air Force. We did not review any classified aspects of these programs. We also reviewed prior GAO work addressing the canceled programs to garner any insights. To examine the impacts of program cancellation, we reviewed Federal Acquisition Regulation (FAR) and Defense Federal Acquisition Regulation Supplement (DFARS) parts covering contract termination, contract cost principles and procedures, and government property to identify, respectively, what steps to take when a contract is terminated, what termination costs are allowable, and what issues arise in disposing of program assets. We also looked at specific FAR and DFARS contract clauses related to these areas. In addition, we obtained termination settlement memorandums for the three of our case study programs for which these were available. Settlement memorandums were not yet available for FCS or VH-71. In addition, we discussed potential impacts with DAU staff, who had considered the broader effects of program cancellations as part of an effort to develop guidance on shutting down program efforts after a contract termination or program cancellation. They shared with us the results of their data gathering, including comments culled from interviews with DOD senior acquisition officials. Lastly, we asked former officials of our case study programs about specific impacts of their program cancellations. To determine the adequacy of current DOD guidance on program cancellations, we reviewed guidance obtained from our prior work. Then we met with OSD, Army, Air Force, and Navy officials to determine whether any additional guidance had been issued subsequently, and whether they considered available guidance adequate for ending a contract or program. We also discussed the availability of guidance with DCMA officials, including any changes to the FAR since our prior work. We reviewed new resources offered by DAU, including a guidebook published in December 2013. Furthermore, we obtained views about the adequacy of guidance from DAU staff, who had identified a need for department-wide guidance for program officials, especially program managers, on how to shut down a canceled program. Finally, we solicited views on the adequacy of available guidance from former case study program officials, though in many cases these officials were unable to identify guidance they relied on in addition to the FAR, if any. To look at the extent to which DOD leveraged various types of assets from canceled programs, we reviewed FAR and DFARS parts for contract termination, government property, patents, data, and copyrights, and their associated clauses to determine if they specified how the various types are to be disposed when they are no longer needed for contract performance. We discussed the process for disposing of government property with DCMA Plant Clearance and Terminations Center officials, and obtained information on property reutilization, when available. We also met with an Army intellectual property law attorney to learn about the treatment of intellectual property assets. We specifically discussed transfer and leveraging of various program assets with former officials for our case study programs, and obtained available documentation. However, officials told us that contract files for the Comanche and ACS programs were no longer available because the cases had been settled beyond the federal record retention period. For these two programs, we had to rely mostly on the memories of former program officials for information on leveraged assets. However, for both programs we had other documentation available from our prior work. To consider the usefulness of contract termination cost estimates, we primarily asked former program officials whether estimates were prepared, by whom, and how they were used. We also obtained documentation, when available, of termination cost estimates from Contract Funds Status Reports submitted by contractors. These estimates were no longer available for ACS or Comanche. In addition, we discussed termination cost estimates with an official from the CAPE. The scope of this review did not include interviews with contractors. We supplemented our termination cost estimate work by incorporating information gathered for a recently issued report on DOD estimates of contract termination liability. That work included reviewing a DOD assessment of the extent to which the department considers potential termination liability as a factor in entering into and terminating contracts. We also interviewed DOD officials responsible for conducting the assessment. In addition to our five case studies of canceled programs, we also selected a non-generalizable sample of seven current weapon programs in development and production to ask further questions about DOD’s use of termination liability estimates. We selected these programs from among those participating in our annual assessment of weapon systems, and they included Integrated Air and Missile Defense and Excalibur Precision 155mm Projectiles (Army), AIM-9X Block II Air-to-Air Missile and Ship to Shore Connector (Navy), and Family of Advanced Beyond Line-of-Sight Terminals, GPS III OCX Ground Control Segment, and MQ- 9 Unmanned Aircraft System Reaper (Air Force). These programs were selected to represent a range of weapon platforms and military services, and to include both development and production programs. We asked officials at these programs a standard set of questions to determine how DOD officials consider termination liability costs prior to contract award and during contract execution, and to identify guidance used in making those decisions. Although the responses received from these programs cannot be generalized, we determined the evidence we obtained from these programs was sufficient to provide illustrative information on DOD’s use of termination liability estimates. We conducted this performance audit from October 2012 to March 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Description Within 120 days from effective termination date. Plant Clearance Officer (PLCO) reviews and accepts, or returns for correction, within 10 days of receipt from contractor. PLCO verifies within 20 days of acceptance. Standard screening period of 46 days begins upon Plant Clearance Officer’s acceptance of inventory schedule. Agency has 20 days to screen property for other use within agency, transfer educationally useful equipment to other federal agencies, and to schools and non-profits. Disposal in accordance with agency procedures without report to General Services Administration (GSA) 1) property for abandonment or destruction 2) property furnished to nonappropriated fund activities 3) foreign excess personal property 4) scrap, except scrap condition aircraft 5) perishables 6) trading stamps and bonus goods 7) hazardous waste or toxic and hazardous materials 8) controlled substances 9) property dangerous to public health and safety 10) classified items or national security-sensitive property Also, nuclear materials in accordance with Nuclear Regulatory Commission, applicable state licenses, applicable federal regulations, and agency regulations No later than 21st day, Plant Clearance Officer is to submit revised schedules and report to General Services Administration for reuse within government or donation as surplus property. Screening by other federal agencies, and for donations 21 days concurrent screening plus 5 days donation processing. Transfer requests on first-come, first-served basis; General Services Administration (GSA) transmits approval to Plant Clearance Officer. Property not available to donees until screening completion. Days 42-46 reserved for GSA to make allocation. Abandonment, destruction, or sale after screening Surplus property having completed screening shall be sold in accordance with Federal Management Regulation policy. Agencies may specify implementing procedures. If inventory unsuccessfully screened, has no commercial value, does not require demilitarization, and does not constitute a danger to public health or welfare, the Plant Clearance Officer can direct the contractor to destroy property, or abandon it at the contractor or subcontractor premises (with contractor consent for sensitive property). Plant Clearance Officer shall promptly prepare report following disposition of property identified on inventory schedule and crediting of any related proceeds. The report is to identify any lost or otherwise unaccounted for property and changes in quantity or value made by the contractor after submission of initial schedule. The report is to be provided to the termination contracting officer. In addition to the contact named above, Katherine Trimble (Assistant Director), Ann Marie Udale (Analyst-in-Charge), Peter Anderson, Virginia Chanley, Maria Durant, Kristine Hassinger, John Krump, Carol Petersen, Raffaele Roffo, Matthew Shaffer, and Roxanna Sun made key contributions to this report.

Summary: In the face of increasing budgetary pressures, it is important that DOD continue to find ways to manage its portfolio of major defense acquisition programs (MDAP) more efficiently. In 2008, GAO concluded that program cancellations can be a valuable portfolio management tool, and found that DOD can take various steps to retain value from the work completed. The Senate report accompanying the National Defense Authorization Act for 2013 mandated GAO to review issues associated with cancellations of MDAPs. This report assesses (1) the effects of program cancellation, (2) the adequacy of DOD guidance on program cancellations, (3) the extent to which DOD leveraged or transferred to other programs various types of assets from canceled programs, and (4) the usefulness of contract termination cost estimates. To do this work, GAO reviewed the cancellation of five major DOD programs from across the military services; interviewed officials at the Office of the Secretary of Defense, the military services, and others; and reviewed current guidance related to program cancellation. Cancellation of a major Department of Defense (DOD) weapon program can have broad effects. Cancellation of one program can affect the schedules or budgets for related programs as well as the industrial base and local economies. For example, the cancellation of one program has impacted the schedule for the Army's network modernization efforts. DOD stakeholders can provide input to program officials on the potential effects of a cancellation. DOD has developed an online resource, Smart Shutdown, to offer tools for program managers related to program cancellations. In 2013, a guidebook was added that addresses the spectrum of cancellation effects. However, current efforts to publicize this online resource may not be sufficient and program officials may lack the knowledge necessary to leverage investments as effectively as possible. Furthermore, while this resource is designed to allow officials to share lessons learned about program cancellation, there have been few contributions to date. Without increased sharing of lessons learned, DOD will miss opportunities to build and share knowledge on program shutdown. GAO could not determine the extent to which DOD leveraged the assets in its case studies because DOD does not have a way to track all types of assets from canceled programs. DOD relies on a government-wide process for tracking disposal of government property, such as computer hardware and equipment, but this process is not tailored to weapon systems and was not designed to track other types of assets such as rights to technical data and software, or partially-developed technologies. Because there is no department-wide process for disseminating information about all assets available for reuse, DOD cannot ensure that technologies go to parties who can best use or develop them further. Estimates of contract termination costs are sometimes perceived as a useful tool in managing a portfolio of investments and informing contract termination and budgeting decisions. However, the usefulness of these estimates is limited by, among other factors, inherent uncertainties about costs that cannot be addressed until a program is actually terminated. GAO recommends that DOD direct program officials to refer to Smart Shutdown guidance and to provide lessons learned, and that DOD develop department-wide processes to improve tracking and dissemination of information about assets available. DOD partially concurred with these recommendations, but stated use of and contribution to Smart Shutdown should be encouraged rather than directed. GAO maintains direction would be more effective.