DOD’s acquisition of weapon system programs has been on our High Risk List since 1990 because DOD programs consistently fall short of cost, schedule, and performance expectations. Congress and DOD have long explored ways to curtail these cost, schedule, and performance problems, and both took related actions about a decade ago, with Congress passing the Weapon Systems Acquisition Reform Act of 2009 and DOD implementing its “Better Buying Power” initiatives. The Weapon Systems Acquisition Reform Act of 2009 aimed to improve the organization and procedures of DOD for the acquisition of major weapon systems, for example by revising the certifications that programs were expected to complete before approval for system development start. The new certifications included the need to conduct trade-offs among cost, schedule, and performance objectives and for independent verification of technology maturity. In 2010, DOD started its own acquisition reform initiatives through “Better Buying Power.” These reforms required DOD programs to conduct analyses of program affordability and set cost targets, among other things, which placed cost constraints on programs and encouraged programs to find cost improvements during program execution. These and other reforms championed sound management practices, such as realistic cost estimating, increased use of prototyping, and systems engineering. In 2016, we found that DOD was beginning to decrease the amount of cost growth in its major defense acquisition program portfolio. Despite DOD’s improvements in cost control, however, members of Congress remained concerned that the DOD acquisition process was overly bureaucratic and too slow to deliver capability to the warfighter. Congress enacted numerous additional acquisition-related provisions in the National Defense Authorization Acts for Fiscal Year 2016 and subsequent years that addressed the processes with which DOD and the military departments acquire goods and services and encourage innovation. These provisions addressed a wide range of acquisition issues, such as the: creation of new processes for oversight of major defense acquisition programs; development of streamlined alternative acquisition paths; and changes to DOD’s other transaction authority, which allows DOD to enter into agreements that generally do not follow a standard format or include terms and conditions required in traditional mechanisms, such as contracts or grants. Congress also required that DOD establish a panel in the National Defense Authorization Act for Fiscal Year 2016, referred to as the “Section 809 Panel,” to identify ways to streamline and improve the defense acquisition system. The panel issued its final report in January 2019, which, together with its earlier reports, included a wide range of recommendations aimed at changing the overall structure and operations of defense acquisition. DOD acquisition policy defines an acquisition program as a directed, funded effort that provides a new, improved, or continuing materiel, weapon, or information system, or a service capability in response to an approved need. DOD Directive 5000.01, The Defense Acquisition System, provides management principles and mandatory policies and procedures for managing all acquisition programs. Oversight levels and procedures for DOD’s acquisition programs are outlined in DOD Instruction 5000.02, Operation of the Defense Acquisition System. Traditionally, defense acquisition programs are classified into acquisition categories based on the value and type of acquisition. DOD’s most costly programs have historically been referred to as major defense acquisition or Acquisition Category I programs. Programs with lower costs are categorized as Acquisition Category II or III programs. The acquisition category of a program can affect oversight levels and procedures, such as what program information and documents are required and who is designated as the milestone decision authority. Among other responsibilities, the milestone decision authority approves entry of an acquisition program into the next phase of the acquisition process and is accountable for cost, schedule, and performance reporting. DOD’s acquisition process includes three major milestones at which program offices provide information to or receive a waiver from the milestone decision authority. The milestone decision authority then makes a decision on whether the program is ready to transition to the next acquisition phase. The milestones normally represent transition points in programs at which there is a marked increase in the funding required for the program. Milestone A is the decision for an acquisition program to enter into the technology maturation and risk reduction phase. Milestone B is the decision to enter the engineering and manufacturing development phase. Milestone C is the decision to enter the production and deployment phase. Programs may start at different milestones depending on the circumstances of the particular program, such as whether the technologies the program plans to use are mature. Some major defense acquisition programs, such as the Marine Corps’ Amphibious Combat Vehicle program and the Navy’s Next Generation Jammer-Mid Band program, entered the acquisition system at milestone A. Other programs, such as the Air Force’s Combat Rescue Helicopter program and the Army’s Armored Multi-Purpose Vehicle program, entered directly at milestone B without having a milestone A because technologies were considered mature by the Office of the Secretary of Defense and an independent review team, respectively. Figure 1 illustrates the key milestones associated with the defense acquisition system. DOD’s acquisition policy encourages tailoring the acquisition process, including tailoring of documentation or information requirements. In previous work, we identified opportunities for DOD to tailor the documentation and oversight needed for major defense acquisition programs. In 2015, we found that 24 acquisition programs we surveyed spent, on average, over 2 years completing up to 49 information requirements for their most recent milestone decision. We found that DOD’s review process was a key factor that influenced the time needed to complete the information requirements. In total, the requirements averaged 5,600 staff days to document, yet acquisition officials considered only about half of the requirements as high value. We recommended that DOD eliminate reviews and information requirements that do not add value or are no longer needed. DOD agreed with both recommendations and took some actions through its Better Buying Power initiatives to streamline documentation and staff reviews. Among the information requirements that acquisition officials considered most valuable were those that support a sound business case. A solid, executable business case provides credible evidence that (1) the warfighter’s needs are valid and that they can best be met with the chosen concept, and (2) the chosen concept can be developed and produced within existing resources—such as technologies, design knowledge, funding, and time. Establishing a sound business case for individual programs depends on disciplined requirements and funding processes, and calls for a realistic assessment of risks and costs; doing otherwise undermines the intent of the business case and increases the risk of cost and schedule overruns and performance shortfalls. The program’s business case typically includes documentation of the capabilities required of the weapon system, the strategy for acquiring the weapon system, sound cost estimates based on independent assessments, and a realistic assessment of technical and schedule risks. Several entities at the enterprise level (meaning the Office of the Secretary of Defense, Joint Chiefs of Staff, and Joint Staff) and the military department level play a role in the oversight and budgeting for DOD weapon system acquisition programs. In general, at the enterprise level, the acquisition and budgeting processes are managed by subordinate offices within the Office of the Secretary of Defense. More specifically: The Under Secretary of Defense for Research and Engineering is responsible for establishing policies on and supervising all aspects of defense research and engineering, technology development, technology transition, prototyping, experimentation, and developmental testing activities and programs, including the allocation of resources for defense research and engineering. This organization has a significant role in activities prior to milestone B, but also interacts with major defense acquisition programs throughout their life cycles with regard to technical risks. For major defense acquisition programs, the Under Secretary conducts assessments in areas such as technology maturity, interoperability, and cyber security. The Under Secretary of Defense for Acquisition and Sustainment is responsible for establishing policies on and supervising all matters relating to acquisition (including (1) system design, development, and production; and (2) procurement of goods and services) and sustainment (including logistics, maintenance, and materiel readiness). This organization has certain oversight responsibilities for major defense acquisition programs throughout the acquisition process, such as collecting and distributing performance data. The Under Secretary is the Defense Acquisition Executive and serves as the milestone decision authority for certain major defense acquisition programs. The Director, Cost Assessment and Program Evaluation and the Under Secretary of Defense (Comptroller) manage the annual budget preparation process for acquisition programs. These organizations have cost assessment and budgetary responsibilities, respectively, for major defense acquisition programs leading up to each milestone and once these programs have been fielded. At the military department level, the service acquisition executive, also known as the component acquisition executive, is a civilian official within a military department who is responsible for all acquisition functions within the department and can serve as the milestone decision authority. The following officials serve as the service acquisition executive for the military departments: the Assistant Secretary of the Air Force (Acquisition, Technology, and Logistics) for the Air Force; the Assistant Secretary of the Army (Acquisition, Logistics and Technology) for the Army; and the Assistant Secretary of the Navy (Research, Development and Acquisition) for the Navy and the Marine Corps. We focused our review on five selected reforms from the National Defense Authorization Acts for Fiscal Years 2016 and 2017. Three of the reforms affect the processes related to DOD’s oversight of major defense acquisition programs, the fourth restructured acquisition oversight functions in the Office of the Secretary of Defense, and the fifth provides alternative acquisition pathways for programs that are not considered major defense acquisition programs and have an objective of being completed within 5 years. Table 1 identifies the source of the five reforms that we reviewed and provides a brief summary of each reform. For additional detail on the statute, amendments, and related DOD guidance we reviewed, see appendix II. We found that DOD has made progress implementing reforms that have affected the oversight of major defense acquisition programs. Decision- making authority for these programs has been realigned between the Office of the Secretary of Defense and the military departments. In addition, new processes are in place to improve DOD’s consideration of program cost, fielding, and performance goals and assessment of technical risk although questions remain about how they will be implemented. The Office of the Secretary of Defense has also restructured in an effort to increase innovation in the earlier stages of the acquisition process and reduce cost, schedule, and performance risks in later stages. While the restructure has begun to take shape, additional steps remain to be completed, including developing charters and fully staffing new offices. These steps are important to determining how acquisition oversight roles within the Office of the Secretary of Defense— which had been executed by a single office for decades—will be divided and how new offices will be structured to effectively carry out their work. Milestone decision authority for most major defense acquisition programs now resides with the military departments, a reform generally required for programs starting after October 1, 2016 by section 825 of the National Defense Authorization Act for Fiscal Year 2016. According to data from DOD’s Defense Acquisition Visibility Environment system, as of March 2019, milestone decision authority was at the military department level for 80 of 89 major defense acquisition programs. The 80 programs include all six programs that started at milestone B or an equivalent milestone since this reform became effective on October 1, 2016, and 74 other programs that started before the reform became effective. The nine programs retained by the Office of the Secretary of Defense all began prior to the reform becoming effective and include programs that are high risk, joint, or have had significant cost or schedule growth, such as the F-35 Joint Strike Fighter program and the Army’s Integrated Air and Missile Defense program. See appendix III for more information about milestone decision authority, including a list of the major defense acquisition programs as of March 2019 and the milestone decision authority for each. Prior to this reform going into effect, the Under Secretary of Defense for Acquisition, Technology and Logistics within the Office of the Secretary of Defense typically was the milestone decision authority for major defense acquisition programs until they entered the production and deployment phase—that is, for the milestone A, B, and C decisions. The Under Secretary then typically delegated milestone decision authority to the military departments after the milestone C decision. Under the new reform, the Secretary of Defense may designate an alternate milestone decision authority under certain circumstances. For example, the Secretary may determine that the program meets one of several characteristics outlined in statute, such as addressing a joint requirement or the program being critical to a major interagency requirement. There are now substantially more major defense acquisition programs with decision authority at the military department level. This change resulted from both the statutory reform for newly started programs and changes to milestone decision authority for existing programs resulting from a separate review conducted by the Office of the Secretary of Defense after the reform became effective, wherein the military department was designated the milestone decision authority for approximately 20 programs. See figure 2 for trends in the level of milestone decision authority from 2012 to 2019. major defense acquisition program will provide an options matrix to stakeholders including the Under Secretaries of Defense for Research and Engineering and Acquisition and Sustainment, Cost Assessment and Program Evaluation, and the Joint Staff, which must include at least three options that represent differing assumptions about possible solutions, technical risks, cost, schedule, and affordability. gathering and distributing data and lessons learned, and conducting or approving independent cost estimates. These stakeholders must be granted the access necessary to complete independent analysis in their area of responsibility. This analysis will consider aggregated risk regarding technical feasibility, cost, schedule, and affordability, and will be submitted to the milestone decision authority. A goal establishment meeting will be held within 30 days of the program’s analysis of alternatives outbrief and will be co-chaired by the milestone decision authority and Vice Chief of the pertinent military service(s) and supported by the stakeholders identified above. As of March 2019, no programs have held a milestone A since the reform became effective, and no programs have had goals established under the new process. establish a process to develop program cost, fielding, and performance goals for major defense acquisition programs that reach milestone A after October 1, 2017. The statute described the goals as follows: (1) the cost goal is for both procurement unit cost and sustainment cost, (2) the fielding goal is the date for initial operational capability, and (3) the performance goal is for technology maturation, prototyping, and a modular open system approach. DOD issued a policy for the process in November 2018, stating that stakeholders will complete independent analyses in their areas of responsibility to consider the aggregated risk regarding technical feasibility, cost, schedule, and affordability, which will be submitted to the milestone decision authority (typically at the military department level). The policy stated that it applies to all major defense acquisition programs that enter the acquisition process after October 1, 2017, without regard to what milestone initiates the program. The policy also stated that the Office of the Secretary of Defense will have the opportunity to consult with the milestone decision authority on revised goals if the program exceeds its initial cost or fielding goals prior to the next milestone or production decision. DOD acquisition policy already required programs to document objectives for system cost, schedule, and performance in an acquisition program baseline at milestone B and affordability cost goals were to be set at milestone A. Under the new process, fielding and performance goals are established earlier and all three goals (cost, fielding, and performance) are required to be established before funds are obligated for technology development, systems development, or production, rather than being set at specific program milestones. The new process also adds a meeting to review and discuss the goals before they are approved by the milestone decision authority. Officials from the Office of the Under Secretary of Defense for Acquisition and Sustainment told us that this new process is intended to consolidate existing information to inform earlier decisions on which investments the department wants to make. As of March 2019, no major defense acquisition programs have held a milestone A since the statutory requirement became effective, and no major defense acquisition programs have had goals established under the new process. According to officials from the Office of the Under Secretary of Defense for Acquisition and Sustainment, no new programs have been required to have goals established since DOD’s policy for the process was issued in November 2018. These officials told us they rely on the milestone decision authority to notify them when goals need to be established and that the first programs expected to have goals established under the new policy are the Army’s Gator Landmine Replacement Program and the Air Force’s Mk21A Reentry Vehicle. Both programs are slated to go through the process in mid-2019. Independent technical risk assessments. The Under Secretary of Defense for Research and Engineering is now responsible for conducting or approving independent technical risk assessments for major defense acquisition programs prior to milestones A and B and before production decisions. According to DOD’s December 2018 independent technical risk assessment policy, the assessments will consider the full spectrum of technology, engineering, and integration risk, including critical technologies and manufacturing processes, and the potential impacts to cost, schedule, and performance. The reform required the assessments for major defense acquisition programs reaching milestone A after October 1, 2017; no major defense acquisition programs have held a milestone A since that date. DOD policy issued in December 2018 implementing the statute states that the assessments will be conducted for all major defense acquisition programs at each upcoming milestone throughout the acquisition process, effective December 3, 2018. As a result, the assessments will be conducted regardless of whether the program reached milestone A after October 1, 2017. As of March 2019, the Office of the Under Secretary of Defense for Research and Engineering had conducted eight independent technical risk assessments on major defense acquisition programs. One additional assessment on the Infrared Search and Track Block II program was delegated to the Navy to conduct, although the Office of the Under Secretary of Defense for Research and Engineering still approved the assessment. While DOD acquisition guidance previously provided for similar types of assessments, they were not always required to be conducted or approved at the Office of the Secretary of Defense level for all major defense acquisition programs. DOD acquisition guidance previously provided for the Office of the Secretary of Defense to request broad program assessments related to systems engineering, including risk areas, at all milestones for major defense acquisition programs with milestone decision authority at the Office of the Secretary of Defense level. Additionally, all major defense acquisition programs were required to have a separate assessment of critical technology elements prior to entering the system development phase or the production and deployment phase if the system enters the acquisition life cycle after system development. DOD’s December 2018 policy requires that independent technical risk assessments be conducted or approved at the Office of the Secretary of Defense level by the Office of the Under Secretary of Defense for Research and Engineering unless this responsibility is delegated, regardless of the level of the milestone decision authority. The Office of the Secretary of Defense officially reorganized its acquisition organization on January 31, 2018, in response to Section 901 of the Fiscal Year 2017 National Defense Authorization Act. Under the reorganization, responsibilities of the former Under Secretary of Defense for Acquisition, Technology and Logistics were divided between two new offices—the Under Secretary of Defense for Research and Engineering and the Under Secretary of Defense for Acquisition and Sustainment (see fig. 3 and app. IV for organizational charts). According to the conference report accompanying the legislation, the priorities framing the conference discussions on reorganization included elevating the mission of advancing technology and innovation within DOD, and fostering distinct technology and acquisition cultures. The report further states that the conferees expect that the Under Secretary of Defense for Research and Engineering would take risks, test, and experiment, and have the latitude to fail, as appropriate. Additionally, the report states that the conferees expect the Under Secretary of Defense for Acquisition and Sustainment to focus on timely, cost-effective delivery and sustainment of products and services, and seek to minimize any risks to that objective. It is too early to say whether the goals of the reorganization have been realized. In July 2018, the Deputy Secretary of Defense issued a memorandum outlining the overall organizational structures, roles, and responsibilities of the two new Under Secretary offices. Responsibilities of many prior subordinate offices were realigned to one of the two new Under Secretary offices as part of the reorganization. For example, systems engineering falls under the Under Secretary of Defense for Research and Engineering and contracting policy and oversight falls under the Under Secretary of Defense for Acquisition and Sustainment. New offices or positions were also created during the reorganization. For example, the Office of the Under Secretary of Defense for Research and Engineering created eight assistant director positions to serve as resident experts in strategic technology areas, such as cyber, quantum science, and hypersonics. Similarly, the Office of the Under Secretary of Defense for Acquisition and Sustainment created an Assistant Secretary of Defense for Sustainment. Previously, sustainment activities were spread across several organizations headed by two Assistant Secretaries of Defense. While foundational steps to stand up the two new Under Secretary offices have been taken, as of March 2019, reorganization actions were ongoing in two major areas: completing chartering directives that define the scope of responsibilities for the two new offices and hiring additional people for the new offices, including for several senior leadership positions. Chartering directives: Officials from the Office of the Chief Management Officer originally expected charters for the two offices to be completed by January 2019, but progress has been delayed. According to DOD policy, chartering directives are required to define the scope of functional responsibilities and identify all delegated authorities for the chartered organizations. According to a July 2018 memorandum issued by the Deputy Secretary of Defense, the Chief Management Officer is to oversee the development of the charters. Officials from the Office of the Chief Management Officer stated that they are doing so with significant input from the Under Secretaries of Defense for Research and Engineering and Acquisition and Sustainment. These officials told us that the development of the charters has taken longer than expected because redistributing the responsibilities of a single office into two new offices was complicated due to the number of shared or partially overlapping interests. Officials from the Offices of the Under Secretaries of Defense for Research and Engineering and Acquisition and Sustainment now estimate the charters will be completed in July 2019 after department- wide coordination, though they said that time frame may be optimistic given the challenges to date. These officials also told us they expect that they will need to make additional changes to other existing acquisition policies and guidance to incorporate the new content of the chartering directives once complete. Hiring additional employees: In order to stand up the two newly- created organizations, on February 1, 2018, 516 civilian and military positions from the former Office of the Under Secretary of Defense for Acquisition, Technology and Logistics were divided between the two new Under Secretary offices. Finalizing staffing for both offices has been a gradual process that will not be completed until at least fiscal year 2020 because of the need to: (1) reduce positions to meet statutorily-directed cost-savings objectives; (2) realign positions between the two offices; and (3) hire additional staff. Table 2 provides additional detail on past and expected changes to authorized positions. Both Under Secretaries are still working to staff their offices, with approximately 30 percent of current positions vacant in the Office of the Under Secretary of Defense for Research and Engineering, and 8 percent of current positions vacant in the Office of the Under Secretary of Defense for Acquisition and Sustainment. See figure 4 for the current status of staffing within both offices. Both Under Secretaries have experienced challenges while staffing their offices. For example: The Office of the Under Secretary of Defense for Acquisition and Sustainment has experienced challenges stemming from needing to meet required personnel reductions while also hiring staff to align with the revised priorities from the reorganization. As part of the restructuring, the office will absorb all of the 57 remaining civilian and military position reductions that were originally assigned to the former Office of the Under Secretary of Defense for Acquisition, Technology and Logistics. These reductions will occur during both fiscal years 2019 and 2020. At the same time, officials said they are still working to hire staff with skills in needed areas such as data analytics. Officials said they are leveraging existing authorities such as voluntary early retirement authority and voluntary separation incentive payments to meet their targeted number of authorized positions by the end of fiscal year 2020. Officials from the Office of the Under Secretary of Defense for Research and Engineering said their challenges have primarily been negotiating the appropriate number of positions for the organization and staffing the organization in a timely manner. For example, 13 positions are not currently available to be filled because they will not be transferred from the Office of the Under Secretary of Defense for Acquisition and Sustainment until fiscal year 2020. The officials also stated that there have been delays related to developing new position descriptions, revalidating existing position descriptions, and finding individuals with the right skill sets for positions. Both offices have been delayed in filling key leadership positions. According to officials from these offices, vacant positions include the Deputy Director of Mission Engineering and Integration, the Director of Systems Engineering, and the Principal Director of Defense Pricing and Contracting. Senior officials from both offices told us that they have been unable to fill some vacant senior executive positions since the most recent Secretary of Defense resigned on December 31, 2018. The inability to fill these positions is due to the Office of Personnel Management’s general policy to suspend processing for senior executive service career appointments when an agency head leaves, until a successor is appointed at the agency. As of March 2019, a new Secretary of Defense had yet to be confirmed. Senior level officials also told us that some decisions about structure and staffing may be held up until after these executive positions are filled, but that in the interim, they are moving forward with daily operations and in some instances have other employees acting in those roles. As of March 2019, the military departments had begun using middle-tier acquisition pathways for over 35 rapid prototyping and rapid fielding programs under interim guidance issued by the Under Secretary of Defense for Acquisition and Sustainment and the military departments. However, DOD has yet to determine certain aspects of program oversight, including what information military departments should consider in selecting programs and what metrics and data the Office of the Secretary of Defense and military department leaders should use to assess performance. The Departments of the Air Force, Army, and Navy have begun to execute over 35 unclassified and classified acquisition programs using new acquisition pathways distinct from the traditional DOD acquisition process. Section 804 of the National Defense Authorization Act for Fiscal Year 2016 required DOD to issue guidance establishing two new streamlined acquisition pathways for DOD—rapid prototyping and rapid fielding—under the broader term “middle tier of acquisitions.” According to the Joint Explanatory Statement accompanying the National Defense Authorization Act, the guidance was to create an expedited and streamlined “middle tier” of acquisition programs intended to be completed within 5 years. The Joint Explanatory Statement noted that middle-tier programs would be distinctive from rapid acquisitions that are generally completed within 6 months to 2 years and traditional acquisitions that last much longer than 5 years. Statute lays out more specific intended time frames and expectations for programs using these two pathways: The rapid prototyping pathway is to provide for the use of innovative technologies to rapidly develop fieldable prototypes to demonstrate new capabilities and meet emerging needs. The objective of a rapid prototyping program is to field a prototype that can be demonstrated in an operational environment and provide for a residual operational capability within 5 years of the development of an approved requirement. The rapid fielding pathway is to provide for the use of proven technologies to field production quantities of new or upgraded systems with minimal development required. The objective of a rapid fielding program is to begin production within 6 months and complete fielding within 5 years of the development of an approved requirement. Middle-tier acquisition pathways are distinct from the traditional acquisition system for major defense acquisition programs. These pathways allow for programs to be exempted from the acquisition and requirements processes defined by DOD Directive 5000.01 and the Manual for the Operation of the Joint Capabilities Integration and Development System. The statute does not identify a dollar limit for programs using middle-tier acquisition pathways. Middle-tier programs are typically approved for initiation by the service acquisition executive, although Air Force policy also allows for smaller programs to be initiated by the program executive officer. Table 3 shows the number of unclassified programs initiated by the military departments as of March 2019. The middle-tier programs initiated to date represent a range of products, dollar amounts, and complexity. For example, one of the smaller dollar value programs is an approximately $30 million Navy effort to develop a prototype rocket motor that would support extended ranges for an existing missile. One of the larger dollar value programs is a multibillion dollar Army effort to develop the next generation combat vehicle. The military departments generally require funding these programs through the traditional budget process, using DOD’s existing planning, programming, budgeting, and execution process. Based on estimated program costs reported by the military departments, we found that approximately half of the programs initiated to date would be categorized as major defense acquisition programs if they were not being pursued under a middle-tier pathway. In some cases, such as the Army’s Lower Tier Air and Missile Defense Sensor program, an existing program planned as a major defense acquisition program shifted to a middle-tier acquisition pathway. Appendix V includes a list of middle-tier acquisition programs started by the military departments as of March 2019. Although DOD and the military departments have issued interim guidance for using middle-tier acquisition pathways, we found that DOD has not provided department-wide guidance on how certain aspects of program oversight will be conducted. DOD has yet to determine what types of business case information should be submitted to decision makers to help ensure well-informed decisions about program initiation and how program performance will be measured consistently. Section 804 of the National Defense Authorization Act for Fiscal Year 2016 required the Under Secretary of Defense for Acquisition, Technology and Logistics to establish guidance for middle-tier acquisitions. In response, the Under Secretary of Defense for Acquisition and Sustainment issued interim guidance in April 2018 that provided the military departments and other DOD components with the authority to implement middle-tier acquisition programs on an interim basis through September 30, 2019. The guidance laid out the broad purposes and requirements of middle-tier acquisition authorities, and encouraged the military departments and other DOD components using middle-tier acquisition pathways to develop specific implementation processes and procedures to implement the interim authority. Between April 2018 and September 2018, the military departments each issued their own implementing guidance, which provided additional details on how middle- tier programs would be selected and overseen within their department during the period of the interim authority. Subsequently, the Under Secretary of Defense for Acquisition and Sustainment issued two additional interim guidance memorandums: the first in October 2018, which described how the Office of the Secretary of Defense and the Joint Staff would conduct oversight of the military departments’ use of middle-tier acquisition pathways, and the second in March 2019, which addressed sustainment planning considerations for programs using the rapid fielding pathway. The Director, Cost Assessment and Program Evaluation, also issued guidance in April 2019 that included a life-cycle cost estimating policy for programs using the rapid fielding pathway. Statute requires that the guidance from the Office of the Secretary of Defense include a “merit-based process” for considering potential middle- tier programs, although the interim guidance does not describe what the process should include or what information should be considered by decision makers to assess merit other than meeting the needs communicated by the Joint Chiefs of Staff and the combatant commanders. Guidance from each of the military departments provides additional detail on the program selection process, to include describing generally the type of information decision makers should consider when selecting programs. Neither the Office of the Secretary of Defense’s guidance nor the military departments’ guidance fully identifies key elements of a business case to be provided as part of the program initiation process. Our past work has shown that in order to make sound decisions about initiating acquisition programs, it is important that decision makers have the information they need to assess the business case, including that (1) the warfighter need exists and that it can best be met with the chosen concept and (2) the concept can be developed and produced within existing resources. Information needed to establish a business case for a traditional acquisition program typically includes a requirements document (which provides information on the capabilities required of the weapon system); the strategy for acquiring the weapon system; sound cost estimates based on independent assessments; and a realistic assessment of risks, including those risks related to technology and schedule. For a middle-tier acquisition program, business case information would help decision makers make well-informed decisions, to include assessing whether the program is likely to meet objectives established in statute to complete a prototype with a residual operational capability (in the case of a rapid prototyping program) or complete fielding (in the case of a rapid fielding program) within 5 years of an approved requirement. Programs using a middle-tier pathway are intended to be completed within 5 years, and guidance may provide for expedited and streamlined procedures. As a result, the appropriate documents to provide business case information for a middle-tier acquisition program may not need to be as detailed as those for a major defense acquisition program. These documents may also vary to some extent depending on whether a program is a rapid prototyping or a rapid fielding program. However, having this type of information available in some form at program initiation can help decision makers to assess the soundness of a program’s business case at the time a decision is made to start a new program. Oversight at this time is critical because, as we have previously reported, program initiation presents the greatest point of leverage in the program life cycle for decision makers. Table 4 provides additional detail about certain types of business case documentation that are to be considered at program initiation for middle-tier acquisition programs according to the Office of the Secretary of Defense and the military departments’ guidance. Section 804 of the National Defense Authorization Act for Fiscal Year 2016 directed the Under Secretary of Defense for Acquisition, Technology and Logistics to establish guidance for middle-tier acquisitions within 180 days of enactment of the statute (which would have been May 2016), but guidance was not issued until April 2018. According to officials who were involved in efforts to develop the guidance, the Office of the Secretary of Defense circulated multiple iterations of draft guidance, but was unable to reach agreement with the military departments because of concerns that the guidance was too burdensome. These officials told us that as a result, the Under Secretary of Defense for Acquisition and Sustainment decided instead to issue broad interim guidance and allow each of the military departments and other DOD components to develop processes and procedures to implement the interim authority. As stated in the Under Secretary of Defense for Acquisition and Sustainment’s April 2018 interim guidance, the Under Secretary of Defense for Acquisition and Sustainment would develop final guidance for the department in 2019 based on lessons learned from the military departments and other DOD components. According to officials from the Office of the Under Secretary of Defense for Acquisition and Sustainment, the Under Secretary began the process in February 2019 to develop this final guidance. The process was in its initial stages as of March 2019 and officials involved told us they hope to complete the final guidance by September 2019. The business case information programs provided to decision makers at initiation varied widely for nine middle-tier acquisition programs we reviewed. We found that certain types of business case information, such as an assessment of schedule risk that would indicate whether a program could realistically be expected to be completed within time frame objectives in statute, were often not completed at the time of program initiation. For example: Six programs had approved requirements at program initiation. Three of these programs had requirements validated through DOD’s traditional requirements process prior to the decision to start under a middle-tier pathway. Two of these programs, both of which were Air Force programs, had previously planned to start as major defense acquisition programs. The third program, an Army program, had requirements based on those approved for an existing major defense acquisition program. The other three of these programs, all of which were Navy programs, had high-level requirements that described, for example, what environments the system should be tested in or what quantity should be fielded. These requirements were approved by the Navy’s Accelerated Acquisition Board of Directors, which includes the Chief of Naval Operations and the Assistant Secretary of the Navy for Research, Development and Acquisition, among other officials. Three programs were still in the process of developing requirements at the time of program initiation. Only one of the nine programs had an approved acquisition strategy at the time of program initiation. Officials from the other programs told us they planned to develop an acquisition strategy or were in the process of developing or updating one. While all nine of the programs had developed at least a draft cost estimate at program initiation, only one of the nine programs had an assessment of its program cost estimate completed by the military department cost agency at the time of program initiation. Officials from three other programs said that an assessment by the military department cost agency was in progress or planned. Officials from the other five programs told us they had developed a draft cost estimate at program initiation that in some cases was still expected to change and that they did not plan for an assessment by the military department cost agency. The programs varied in the extent to which they assessed risk at program initiation. Four programs had risk assessments that addressed schedule and technology risks, which are types of risks we have identified in our previous work as important to understanding a program’s business case. Two other programs had risk assessments that included either schedule or technology risks but not both. Officials from the other three programs stated that they were still in the process of assessing risks and had yet to assess risks related to meeting statutory schedule objectives at the time of program initiation. Without the Office of the Under Secretary of Defense for Acquisition and Sustainment identifying in its final guidance the minimum program information needed to help decision makers evaluate the program’s business case, DOD cannot ensure that the military departments are consistently considering these types of information. As a result, DOD is not well positioned to ensure that approved middle-tier acquisition programs represent sound investments and are likely to meet the objective of delivering prototypes or capability to the warfighter within 5 years. The Office of the Secretary of Defense and the military departments generally collect program data but we found that neither the Office of the Secretary of Defense nor the military departments has identified metrics that would allow them to use that data to measure and report on program performance in a consistent manner. Developing such metrics would allow senior leaders in the Office of the Secretary of Defense and the military departments to monitor and assess performance across the portfolio of middle-tier programs during program execution, including whether programs are on track to meet statutory objectives for rapid prototyping and rapid fielding. Table 5 provides additional detail on the extent to which guidance addresses the collection of program data and identification of metrics to measure program performance. The Office of the Secretary of Defense began collecting middle-tier program data from the military departments in November 2018 as part of an effort to ensure that middle-tier authority was being used appropriately within the department. However, the office has yet to determine what metrics it will use to measure program performance consistently across the portfolio. Officials within the Office of the Secretary of Defense who are involved with collecting the data told us that they are still refining what data should be collected, determining how to standardize definitions to improve the consistency of data, and considering how to use the data collected to monitor program execution. For example, they are still trying to determine the appropriate triggers that would allow them to know that a middle-tier program may be experiencing cost or schedule challenges. Similarly, guidance from two of the three military departments requires the collection of program data, but the military departments also have not identified metrics to consistently measure performance across programs. The Navy’s guidance does not require the collection of program data or identify metrics to measure program performance. Interim guidance from the Air Force and the Army requires the collection of program data and also requires programs to develop metrics to measure performance, but these metrics are not required to be consistent across programs. Decisions about specific metrics to be reported are left to the discretion of the decision authority for each program, who is typically the service acquisition executive or a program executive officer. As a result, these metrics may not allow consistent measurement of performance across programs because, for example, programs may have a different starting point for reporting data, or may change the metrics that are being assessed at different points within the life of a program. According to federal internal control standards, the ability of agency management to compare actual performance to planned or expected results throughout the organization and analyze significant differences is important to help ensure that the agency is meeting objectives and addressing risks appropriately. These standards also state that agency management should define objectives in quantitative or qualitative terms to permit reasonably consistent measurement of performance toward those objectives. For middle-tier acquisition programs, statute includes objectives related to fielding time frames for both rapid prototyping and rapid fielding programs. Additionally, for rapid prototyping, part of the objective is that the prototype fielded can be demonstrated in an operational environment and provide for residual operational capability. Middle-tier acquisition programs are to be provided streamlined processes, including for program oversight. Decisions about how to measure program performance therefore should be considered in light of how to facilitate oversight without losing the benefits of the flexibilities offered by middle-tier pathways. However, without the Office of the Under Secretary of Defense for Acquisition and Sustainment identifying in its final guidance a minimum set of metrics that can be used to measure performance of programs across the military departments, DOD risks not knowing how the department’s portfolio of middle-tier programs is progressing, including whether programs are on track to meet statutory objectives for rapid prototyping and rapid fielding. As a result, senior leaders in the Office of the Secretary of Defense and the military departments may lack insight needed to identify and address emerging challenges in a timely manner. This is particularly important given that the portfolio includes complex, costly programs that address important capability gaps for the department. While DOD has made progress implementing individual reforms, it continues to face challenges that affect the implementation of the reforms we reviewed. First, we found that senior DOD leadership has not fully addressed disagreements about the division of acquisition oversight roles and responsibilities between the Office of the Secretary of Defense and the military departments. As a result, there have been continuing differences of opinion about how to implement specific reforms. Second, DOD has yet to address persistent portfolio management challenges that affect its ability to effectively manage its portfolio of weapon system investments. Lastly, DOD has yet to develop processes to assess the effectiveness of recent reforms. Without developing such processes, DOD officials will not be well positioned to assess whether reforms are having the intended effects, such as improving innovation and delivering capability to the warfighter more quickly, or if additional changes are necessary to achieve such outcomes. Top DOD leadership has not fully addressed disagreements that remain about the division of acquisition oversight responsibilities between the Office of the Secretary of Defense and the military departments. Our past work has shown that in times of significant organizational transformation, top leadership must set the direction, pace, and tone for the transformation. Personal involvement of these leaders in driving change, including the Secretary and Deputy Secretary, helps provide stability. Internal control standards for federal agencies also emphasize the importance of management communicating information down and across organizational levels in order to enable personnel to perform key roles in achieving objectives and addressing risks. The Deputy Secretary of Defense has weighed in on the division of acquisition oversight responsibilities within the Office of the Secretary of Defense and has addressed specific roles and responsibilities for certain reforms. However, despite continuing disagreements about the division of oversight roles and responsibilities between the Office of the Secretary of Defense and the military departments, DOD’s top leadership has not provided a detailed framework addressing the appropriate roles of each party for acquisition oversight. Officials from the Office of the Secretary of Defense and the military departments we met with expressed different opinions on the appropriate oversight role of the Office of the Secretary of Defense. For example, the Under Secretaries of Defense for Research and Engineering and Acquisition and Sustainment both stated that in cases where the milestone decision authority is at the military department level, the military departments do not see the value in having the Office of the Secretary of Defense involved. This is consistent with concerns that officials from all three military departments have raised in speaking with us. Specifically, officials from all three departments raised concerns that the Office of the Secretary of Defense is overreaching on its oversight responsibilities in some cases, and creating new oversight processes that contradict the intent of recent reforms to speed up the acquisition process. Implementation of several of the reforms we reviewed has resulted in disagreements between the Office of the Secretary of Defense and the military departments that have yet to be resolved. For example: Cost, fielding, and performance goals. Despite the issuance of policy by the Deputy Secretary of Defense in November 2018 on the establishment of cost, fielding, and performance goals, military department officials have continued to express concerns that the process is too burdensome and involves too many stakeholders from the Office of the Secretary of Defense. These officials stated that Office of the Secretary of Defense involvement in programs with decision authority at the military departments, such as participation in meetings with the milestone decision authority to provide advice on cost, fielding, and performance goals, would slow down programs that other reforms were intended to accelerate. They added that they had expressed these concerns to the Office of the Secretary of Defense during the drafting of the policy, but they did not feel that their input was appropriately considered in the final policy. Officials from the Office of the Under Secretary of Defense for Acquisition and Sustainment stated that the analysis and meetings that involve the Office of the Secretary of Defense are ways for stakeholders to advise the milestone decision authority on program decisions based on information from existing oversight mechanisms, such as independent cost estimates and analyses of alternatives. Previously this type of oversight was conducted via multiple meetings leading up to program milestones. The policy states that the policy procedures will be revisited in 6 months and lessons learned incorporated where needed. Independent technical risk assessments. Debates about who should conduct independent technical risk assessments were elevated to the Deputy Secretary of Defense. Subsequently, the Deputy Secretary issued guidance in December 2018 to reiterate that the Under Secretary of Defense for Research and Engineering would conduct or approve these assessments for all major defense acquisition programs, although that responsibility may be delegated. However, despite the issuance of new guidance, there continue to be ongoing debates about when assessments will be delegated to the military departments. The December 2018 guidance does not include criteria for when responsibility for the assessments may be delegated. Officials from the Office of the Under Secretary of Defense for Research and Engineering said that decisions about whether to delegate assessments should be based primarily on the risk level of the program, but officials from military departments stated that these assessments should be conducted within the military department. Officials from the Office of the Under Secretary of Defense for Research and Engineering told us that they had convened a joint working group with the military departments in February 2019 to address this and other implementation issues related to independent technical risk assessments. In the meantime, nearly all assessments continue to be conducted by the Office of the Under Secretary of Defense for Research and Engineering. Middle-tier acquisition. Office of the Secretary of Defense and military department officials also disagree on the extent to which the Office of the Secretary of Defense should weigh in on the appropriateness of a program using a middle-tier pathway. DOD’s October 2018 interim governance guidance provided that the Office of the Secretary of Defense may determine that specific programs were not appropriate for a middle-tier pathway. However, officials from the Air Force and the Army expressed concerns to us about whether that determination was appropriate to be made by the Office of the Secretary of Defense since, from their perspective, programs should be selected at the military department level. Office of the Secretary of Defense officials also told us that there are differences of opinion between them and the military departments on the appropriate amount of information that programs should report to the Office of the Secretary of Defense, including whether the same information should be provided by all middle-tier programs, regardless of expected program cost. As stated earlier, DOD is in the process of finalizing guidance for middle-tier acquisition programs, which could address these issues. Documents that could outline roles and responsibilities of the various parties for acquisition oversight are still being developed. For example, as discussed earlier, officials from the Offices of the Under Secretaries of Defense for Research and Engineering and Acquisition and Sustainment told us that chartering directives for these offices, which are expected to be completed in July 2019, may address to some extent how the offices should work together and with the military departments and other external organizations. In addition, officials from the Office of the Under Secretary of Defense for Acquisition and Sustainment told us that while reforms are currently being implemented under multiple different polices and guidance documents, DOD Instruction 5000.02 will be substantially revised, including to reflect the latest reforms. When completed, the instruction is expected to provide further detail on how oversight activities will be carried out by various acquisition entities. Officials stated that they hoped to complete a version of the revision of DOD Instruction 5000.02 by the end of 2019, but they acknowledged that this estimate was optimistic and that it might take longer than expected to come to agreement on this policy. Without a comprehensive framework from top leadership in the near term that addresses acquisition oversight roles and responsibilities in detail, DOD’s ability to continue with reform implementation, including its ability to finalize policies that could clarify roles and responsibilities, may be slowed by ongoing disagreements. In the longer term, without resolving these issues, DOD cannot ensure that it is achieving the balance between oversight and accountability and efficient program management that senior leadership expects as an outcome of acquisition reform. With too little oversight, acquisition programs may not be properly scrutinized before they are started, which could lead to poor program cost and schedule outcomes. Alternatively, if new oversight processes are too burdensome, DOD may not achieve the expected benefits of streamlining its acquisition processes. As part of this review, we also assessed DOD’s efforts to implement our previous portfolio management recommendations and identified opportunities and challenges related to portfolio management that DOD may face as it continues to implement acquisition reforms. Our past work has shown that when investments are not managed as a portfolio at the enterprise level (meaning at the level of the Office of the Secretary of Defense, Joint Chiefs of Staff, and Joint Staff), the military departments plan to acquire more weapons than DOD can afford, sometimes develop potentially duplicative solutions to common needs, and do not always choose an optimal mix of investments to ensure the department can maintain its technological edge in the future. Realigning roles and responsibilities for decisions related to weapon system programs between the Office of the Secretary of Defense and the military departments could lead to further questions about who is ultimately responsible and accountable for portfolio management decisions if leadership roles are not clearly defined. Officials we met with from the Office of the Secretary of Defense told us that questions remain about the division of responsibilities between the Office of the Secretary of Defense and the military departments for making these types of portfolio management decisions. They told us that concerns we had previously identified about the division of decision-making authority for portfolio management had yet to be addressed during the implementation of recent acquisition reforms, and that in some cases, the reforms had led to additional questions. For example, the Under Secretary of Defense for Research and Engineering told us that while the statute that created his position as part of the restructuring of the former Office of the Under Secretary of Defense for Acquisition, Technology and Logistics assigns him responsibility for allocating resources for defense research and engineering, because he does not have control over the research and engineering budget, in actuality the military departments decide how to prioritize their investments. We found in August 2015 that DOD has had difficulty implementing portfolio management at the enterprise level in part due to diffuse decision-making responsibilities that make it difficult to determine who is empowered to make enterprise-level weapon system investment decisions. At that time, we recommended that DOD revise its portfolio management directive in accordance with portfolio management best practices. We also recommended that the Secretary of Defense designate the Deputy Secretary of Defense or some appropriate delegate responsible for the directive’s implementation, among other recommendations. DOD partially concurred with the recommendations, but the planned actions DOD identified at the time of our report did not fully address the issues we identified. For example, DOD stated that it did not plan to revise its portfolio management directive as we recommended, but instead planned to rescind it and direct stakeholders to participate in portfolio management through the requirements, acquisition, and budget processes. In response, we expressed concern that this approach could reinforce the stove-piped governance structure that we found to be an impediment to integrated portfolio management. As of March 2019, DOD had yet to implement our recommendations. An official from the Office of the Under Secretary of Defense for Acquisition and Sustainment told us that DOD is revising its portfolio management directive, but that there was not yet an estimated completion date. We are not making new recommendations on portfolio management in this report, but we continue to believe that DOD should implement our prior recommendations in order to improve its portfolio management capabilities. See appendix VI for additional details on our assessment of DOD’s progress in this area. DOD is beginning to monitor the implementation of certain reforms, but has yet to establish processes to assess the overall effectiveness of its reform efforts. Collectively, the reforms offer the potential for DOD to significantly reduce the time needed to approve and field acquisition programs by allowing the military departments additional opportunities to tailor documents needed for approval and limiting oversight by the Office of the Secretary of Defense. Ultimately, DOD anticipates that this opportunity will improve the speed at which new capabilities are delivered to the warfighter. Our prior work has identified steps that agencies, such as DOD, can take to help ensure successful implementation of reform efforts, including establishing clear outcome-oriented goals and performance measures putting in place processes to collect the needed data and evidence to effectively measure the reforms’ outcome-oriented goals. The Office of the Secretary of Defense has taken some initial steps to collect data that may help to measure the outcomes of a few reforms, but has yet to determine goals or processes for assessing the overall effectiveness of the reforms. For example, as previously discussed, the Office of the Under Secretary of Defense for Acquisition and Sustainment began initial efforts to collect middle-tier acquisition program data, such as cost and schedule data, from the military departments in November 2018. Officials from that office told us that once they address reliability concerns with the data they are receiving, such as ensuring that programs report schedule data in a consistent fashion, they anticipate that they will be able to use the data to better understand the military departments’ use of middle-tier acquisition pathways. However, according to officials we spoke with from the offices of the Under Secretaries of Defense for Research and Engineering and Acquisition and Sustainment, DOD has not determined how it will assess whether the reforms are collectively resulting in an acquisition process that is more efficient or how it will measure their effect on cost and schedule outcomes. These officials told us that it is important to have data to assess the effect of recent acquisition policy and organizational changes, but they have not determined specifically who will do the assessment, how it will be done, and what data will be needed. They told us that as a part of the reorganization of the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics, they are still in the process of assessing data gaps and needs within the newly-formed organizations and that this type of analysis needs to be completed before they determine how they will assess recent reforms. We recognize that assessing the cumulative effect of recent acquisition reforms on the acquisition process and on the cost and schedule performance of the major defense acquisition program portfolio could take several years because a critical mass of programs will need to go through the new acquisition processes. In the interim, however, determining how an assessment of reforms will be conducted is an important first step in determining whether the reforms are having their intended effect. If DOD officials wait too long to plan for how the department will assess the effect of recent acquisition reforms, including identifying who will be responsible for the assessment and what data will be needed, they may miss the opportunity to collect data from the beginning of implementation needed to measure progress. As a result, they may not be informed about early indications of improvements or problems in the cost, schedule, and performance of programs. Recent acquisition reforms have given DOD significant opportunity to focus on delivering innovative capability to the warfighter more quickly and reduce bureaucratic processes that had built up over time. While DOD has made progress in implementing these reforms, continued attention from top leadership would help ensure that the progress the department has made is not unnecessarily slowed or halted. Middle-tier acquisition will require careful consideration as the department proceeds with the development of final guidance. Middle-tier programs are generally exempt from traditional acquisition and requirements processes, but they may still be large, expensive programs critical to the department’s ability to meet its mission. Identifying the types of business case elements decision makers should consider when initiating programs would improve the department’s ability to ensure that the programs the military departments select are sound investments and likely to succeed using a middle-tier acquisition pathway. Identifying metrics to track performance consistently across the portfolio of middle-tier programs will provide necessary information to senior leaders once programs have been started to assess the performance of middle-tier acquisition programs, including whether they are well positioned to meet statutory objectives. The department also faces challenges that affect the implementation of the reforms we reviewed. These sweeping changes have resulted in some disagreements about oversight roles and responsibilities between the Office of the Secretary of Defense and the military departments that have not been fully resolved. Clear communication from top leadership of a framework for oversight roles and responsibilities that is detailed enough to address areas of continued disagreement would help the department to move forward with effective implementation of the reforms. Developing an approach to assess the effects of recent acquisition reforms is also critical so that DOD can monitor whether reforms are collectively having the effect of speeding up the acquisition process without unintended negative consequences on cost and performance of acquisition programs. We also continue to believe that DOD should address our past recommendations to clarify and strengthen roles and responsibilities at the enterprise level for making portfolio management decisions to make sure that its investments are strategy-driven, affordable, and balance near- and long-term needs. In fact, these recommendations may take on more importance for DOD in light of the implementation of acquisition reforms that will further diffuse responsibility for initiating and overseeing acquisition programs. We are making the following four recommendations to DOD: The Secretary of Defense should direct the Under Secretary of Defense for Acquisition and Sustainment to identify in final guidance the types of business case elements potential middle-tier acquisition programs should develop and decision makers should consider at program initiation to assess the soundness of programs’ business cases, including whether programs are well positioned to meet statutory objectives. (Recommendation 1) The Secretary of Defense should direct the Under Secretary of Defense for Acquisition and Sustainment to determine and identify in final guidance for middle-tier acquisition programs the metrics that will be used to assess the performance of middle-tier acquisition programs across the military departments, including whether programs are meeting statutory objectives. (Recommendation 2) The Secretary of Defense should ensure that a comprehensive framework that clarifies the roles and responsibilities of the Office of the Secretary of Defense and the military departments for acquisition oversight is communicated by senior leadership. This framework should be detailed enough to address areas of continued disagreement among key stakeholders and serve to inform the department’s revisions of other acquisition policies such as DOD Instruction 5000.02. (Recommendation 3) The Secretary of Defense should develop a plan for how the department will assess the effect of recent acquisition reforms, including identifying who will be responsible for the assessment and what data will be needed. (Recommendation 4) We provided a draft of this product to DOD for comment. In its comments, reproduced in appendix VII, DOD concurred with our four recommendations. DOD also provided technical comments with regard to improving the clarity of the discussion of certain reforms and providing additional context about military departments’ oversight practices for middle-tier acquisition programs, among other issues. We incorporated DOD’s technical comments as appropriate. In its written comments, DOD described planned actions to address our recommendations. Specifically, in response to our first recommendation, to identify the types of business case elements that should be considered by decision-makers for middle-tier programs at program initiation, DOD stated that it expects to identify these business case elements in its final guidance on middle-tier programs, which it expects to complete in September 2019. In response to our second recommendation, to identify metrics that will be used to assess the performance of middle-tier programs, DOD stated that it plans to determine performance metrics in coordination with its release of its final guidance on middle-tier programs. DOD expects to release this guidance in late 2019. In response to our third recommendation, for senior leadership to clarify acquisition oversight roles and responsibilities, DOD stated that these roles and responsibilities will be finalized through the issuance of chartering directives and updated acquisition policy; issuance is expected by the end of 2019. Finally, in response to our fourth recommendation, to plan for assessing the effects of acquisition reforms, DOD stated that it has included a division in the Office of the Assistant Secretary of Defense for Acquisition to analyze and assess this and other high-level oversight and policy issues. DOD’s planned actions to address our first, second, and fourth recommendation, if implemented effectively, should address the intent of our recommendations. With regard to our third recommendation, however, we do not believe that the steps outlined in DOD’s written comments are likely to fully address the disagreements about acquisition oversight roles and responsibilities that we identified in the report. We acknowledge in the report that DOD plans to issue chartering directives and re-issue DOD Instruction 5000.02 as part of its efforts to outline the roles and responsibilities of various parties for acquisition oversight, as DOD reiterated in its written comments. However, without a comprehensive framework to inform the revisions of acquisition policies, such as DOD Instruction 5000.02, DOD’s ability to finalize these policies may be hindered by the disagreements between the Office of the Secretary of Defense and the military departments that we identified in our report. These disagreements are persistent and focused on fundamental acquisition oversight issues. Simply issuing chartering directives and finalizing policy as planned may not be enough to ensure that areas of disagreement are resolved and that officials within the Office of the Secretary of Defense and the military departments have a shared understanding of an acquisition oversight framework for the entire Department that will serve as the basis for any policy. Furthermore, without senior leadership within DOD communicating this framework to the Office of the Secretary of Defense and the military departments in sufficient detail to address areas of disagreement among key stakeholders, disagreement will likely persist and the intended impacts of reforms could be stymied. We are sending copies of this report to the appropriate congressional committees and the Acting Secretary of Defense. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-4841 or oakleys@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VIII. This report addresses (1) the progress the Department of Defense (DOD) has made to implement selected oversight reforms for major defense acquisition programs; (2) how DOD has used middle-tier acquisition pathways and the extent to which DOD has developed guidance on middle-tier program oversight; and (3) challenges DOD faces related to reform implementation. The conference report for the National Defense Authorization Act for Fiscal Year 2018 and the Senate Armed Services Committee report accompanying a bill for the National Defense Authorization Act for Fiscal Year 2018 also contained provisions for GAO to review project, program, and portfolio management standards within DOD. Appendix VI of this report includes our assessment of DOD’s efforts to implement our previous portfolio management recommendations and identifies opportunities and challenges related to portfolio management that DOD may face as it continues to implement acquisition reforms. We focused our review on five selected reforms from the National Defense Authorization Acts for Fiscal Years 2016 and 2017 that we determined substantially affected DOD’s oversight of acquisition programs. Our selections were informed by our analysis of the National Defense Authorization Acts for Fiscal Years 2016 and 2017 and our past work on factors affecting the oversight of major defense acquisition programs. We also interviewed officials from the Office of the Secretary of Defense and the military departments to obtain their perspectives on the most significant reforms to acquisition oversight and considered those perspectives when we made our selections. For the purposes of our report, when we refer to a reform, we are referring to a specific change to DOD’s acquisition oversight processes or roles and responsibilities. Two of the reforms we reviewed align with sections of the National Defense Authorization Act for Fiscal Year 2016, and the other three align with one or more sections from the National Defense Authorization Act for Fiscal Year 2017. Table 6 identifies the specific sections or subsections that we reviewed for each reform. We also reviewed related amendments to these sections from National Defense Authorization Acts for subsequent years to determine whether the National Defense Authorization Act sections we reviewed, or sections of the U.S. Code that were added by sections we reviewed, had been modified since being signed into law. When we identified amendments, we assessed DOD’s progress in implementing the statute as amended. Appendix II provides additional details about the original legislative requirements and amendments, if any, to each of the reforms we selected. To identify the progress DOD has made to implement selected oversight reforms for major defense acquisition programs, we analyzed three selected reforms that affect processes related to DOD’s oversight of major defense acquisition programs: designating military departments to be the milestone decision performing independent technical risk assessments; and establishing cost, fielding, and performance goals. We also analyzed one reform that restructured acquisition oversight functions in the Office of the Secretary of Defense. We analyzed the associated National Defense Authorization Act sections and reviewed related acquisition policies and guidance from the Office of the Secretary of Defense and the military departments (see app. II for a list of key guidance we reviewed for each reform). For each reform, we analyzed DOD and military department policies and guidance to determine steps DOD and the military departments had taken to implement the reforms. We also compared new or updated policies and guidance, when available, with prior policies and guidance to determine how oversight roles, responsibilities, and processes had changed for DOD’s major defense acquisition programs. To obtain additional insight into how designation of milestone decision authority had changed as a result of recent reforms, we requested and analyzed data provided by DOD about the milestone decision authority levels for the major defense acquisition program portfolio. To assess the reliability of these data, we discussed the data and sources used to compile them with DOD officials, reviewed the data for errors, reviewed related documentation on programs with milestone decision authority at the military department level, and compared the data when possible to other sources, such as publicly available lists of major defense acquisition programs. On the basis of these steps, we determined that the data we used were sufficiently reliable to identify changes in the level of milestone decision authority over time for major defense acquisition programs. To assess changes resulting from the reorganization of Office of the Under Secretary of Defense for Acquisition, Technology and Logistics, we also reviewed updated organizational charts and staffing and vacancy data for the successor offices (the Office of the Under Secretary of Defense for Research and Engineering and the Office of the Under Secretary of Defense for Acquisition and Sustainment) and compared these to past organizational charts and staffing data for the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics. To determine the current percentage of positions vacant in each office, we compared actual data for filled positions as of March 2019 to the total number of vacant positions as of the same point in time. The vacancy numbers do not include vacant positions that are slotted for future reduction or transfer. To assess the reliability of these data, we requested and reviewed written responses from DOD officials on the reliability of the data and sources used to compile it, reviewed the data for logical inconsistencies, and compared the data when possible to other sources, such as related data provided for other time frames. On the basis of these steps, we determined that the data we used were sufficiently reliable to identify the current staffing status for the two new Under Secretary offices. To determine how DOD has used middle-tier acquisition pathways, we reviewed the relevant statute and guidance, and obtained information from the military departments about the number and types of programs using middle-tier acquisition pathways as of March 2019. We analyzed the guidance from the Office of the Under Secretary of Defense for Acquisition and Sustainment and the military departments to determine how they were implementing the statute with regard to selection of programs and program oversight. We also compared the guidance with our past work on elements of business cases that should be completed at program initiation to determine what elements were addressed by DOD guidance. At each of the military departments, we judgmentally selected three middle-tier programs to review in additional detail. We selected programs to obtain a range of program costs (including programs that were above the equivalent threshold cost for designation as a major defense acquisition program if the program were not using a middle-tier acquisition pathway, as well as those below that threshold) and types of programs being executed under middle-tier acquisition pathways (such as space, artillery, software, missile, and ground vehicle programs). Programs we selected include: Air Force: Hypersonic Conventional Strike Weapon; Next Generation Overhead Persistent Infrared Space; Protected Tactical Enterprise Service; Army: Extended Range Cannon Artillery; Optionally Manned Fighting Vehicle; Rapid Opioid Countermeasures System; and Navy: STANDARD Missile-2 Block IIIC; STANDARD Missile-6 Block IB Phase IA Rocket Motor; STANDARD Missile-6 Block IB Phase IB All Up Round. For these programs, we collected and analyzed additional information such as acquisition decision memorandums, acquisition strategies, program cost and schedule estimates, and risk assessments. We also interviewed or received detailed written responses from program officials that addressed issues such as how decisions were made to execute programs under middle-tier acquisition pathways and how oversight for programs was being conducted. Further, we reviewed interim guidance from the Office of the Under Secretary of Defense for Acquisition and Sustainment and the military departments to determine how DOD planned to measure middle-tier program performance. We compared DOD and the military departments’ guidance on developing metrics and collecting data to assess middle-tier program performance to relevant internal controls related to consistent measurement of program performance. To assess the challenges DOD faces with regard to reform implementation, we reviewed policy and guidance issued by top DOD leadership that outlined roles and responsibilities for the Office of the Secretary of Defense and the military departments with regard to acquisition oversight and compared them to leading practices for leadership involvement in agency transformations that we had identified in prior work. We also collected and analyzed information about DOD’s actions taken to implement prior recommendations we have made to improve portfolio management at DOD and analyzed the acquisition oversight reforms we included in this review to identify opportunities and challenges related to portfolio management that DOD may face as it continues to implement acquisition reforms. Lastly, we reviewed DOD’s plans and ongoing efforts to develop performance measures and collect data to assess the effects of acquisition reforms and compared these efforts with success factors for reform implementation identified in our past work. For all objectives, we also conducted interviews with officials from the Office of the Secretary of Defense, the Joint Staff, and the military departments to obtain additional insight into implementation status, implementation challenges, and future plans, including: Office of the Secretary of Defense: The Office of the Under Secretary of Defense for Research and Engineering, the Office of the Under Secretary of Defense for Acquisition and Sustainment, the Office of the Under Secretary of Defense (Comptroller), the Office of the Chief Management Officer, the Office of the Director of Operational Test and Evaluation, the Office of the Director of Cost Assessment and Program Evaluation, and the Office of the General Counsel. Joint Staff: Force Structure, Resource and Assessment Directorate, J-8. Military departments: For each of the three military departments (Air Force, Army, and Navy) we interviewed acquisition officials from the Service Acquisition Executive’s office, requirements officials supporting the Chief of Staff of the respective armed force, and officials from the military department cost agencies. At the Air Force we also interviewed officials from the Office of the General Counsel. We conducted this performance audit from March 2018 to June 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix III: Milestone Decision Authority for Major Defense Acquisition Programs as of March 2019 Programs with milestone decision authority at the Air Force level (26) Advanced Extremely High Frequency Satellite AIM-120 Advanced Medium Range Air-to-Air Missile Air Force Intercontinental Ballistic Missile Fuze Modernization Airborne Warning and Control System Block 40/45 Upgrade B61 Mod 12 Life Extension Program Tailkit Assembly F-15 Eagle Passive Active Warning Survivability System Family of Advanced Beyond Line-of-Sight Terminals Global Positioning System III Follow-On Production Military Global Positioning System User Equipment Increment 1 MQ- 9 Reaper Unmanned Aircraft System Small Diameter Bomb Increment II Space Based Infrared System High Space Fence Ground-Based Radar System Increment 1 Wideband Global SATCOM Programs with milestone decision authority at the Army level (18) AH-64E Apache New Build Airborne & Maritime/Fixed Station Joint Tactical Radio System AN/TPQ-53 Counterfire Target Acquisition Radar Common Infrared Countermeasure Programs with milestone decision authority at the Army level (18) Guided Multiple Launch Rocket System/Guided Multiple Launch Rocket System Alternative Warhead Handheld, Manpack, and Small Form Fit Radios M88A2 Heavy Equipment Recovery Combat Utility Lift Evacuation System MQ-1C Gray Eagle Unmanned Aircraft System Patriot Advanced Capability-3 Missile Segment Enhancement RQ-7B Shadow Tactical Unmanned Aircraft System Warfighter Information Network-Tactical Increment 2 Programs with milestone decision authority at the Navy level (36) Advanced Arresting Gear AGM-88E Advanced Anti-Radiation Guided Missile Air and Missile Defense Radar Amphibious Combat Vehicle Phase 1 Increment 1 CVN 78 Gerald R. Ford Class Nuclear Aircraft Carrier DDG 1000 Zumwalt Class Destroyer DDG 51 Arleigh Burke Class Guided Missile Destroyer H-1 Upgrades (4BW/4BN) Joint Precision Approach and Landing System LHA 6 America Class Amphibious Assault Ship Littoral Combat Ship Mission Modules LPD 17 San Antonio Class Amphibious Transport Dock MQ-4C Triton Unmanned Aircraft System Programs with milestone decision authority at the Navy level (36) MQ-8 Fire Scout Unmanned Aircraft System Offensive Anti-Surface Warfare Increment 1 (Long Range Anti-Ship Missile) The conference report for the National Defense Authorization Act for Fiscal Year 2018 and the Senate Armed Services Committee report accompanying a bill for the National Defense Authorization Act for Fiscal Year 2018 included provisions for GAO to review project, program, and portfolio management standards within DOD. This appendix includes our assessment of DOD’s efforts to implement our previous portfolio management recommendations and identifies opportunities and challenges related to portfolio management that DOD may face as it continues to implement acquisition reforms. Portfolio management is used by leading commercial companies to help ensure their investments are optimized to meet customer needs within available resources. Portfolio management focuses on products collectively at an enterprise level and involves evaluating, selecting, prioritizing, and allocating limited resources to projects that best accomplish strategic or organizational goals. It is also a vehicle to make a wide variety of decisions, including capability and funding trade-offs, to achieve the optimal capability mix for a given level of investment. For DOD, effective portfolio management can help to ensure that weapon system investments are strategy-driven and affordable and balance near- and long-term needs. Take a hypothetical example in which DOD starts with 10 programs and $50 billion to invest. Without portfolio management, program managers may seek to get the most that they can out of each of the 10 programs, without assessing their aggregate contributions to defense. Using portfolio management, DOD executives would look at different combinations of and approaches to the 10 programs to determine what, collectively, would provide the best capabilities for $50 billion. This would enable executives to decide, for example, whether it is better to concentrate more investment in seven programs rather than fund all 10 as best as possible. In another example, if a program began to have cost or performance problems, portfolio management would consider whether the other programs in the portfolio could address the requirements of the problematic program rather than just putting more money into it. Portfolio management activities at DOD are carried out at both the enterprise level and the military department level and responsibilities are divided among the requirements community, the acquisition community, and the budget community. At the enterprise level, the primary offices responsible for portfolio management are the Joint Staff (representing the requirements community), the Under Secretary of Defense for Research and Engineering and the Under Secretary of Defense for Acquisition and Sustainment (representing the acquisition community), and the Director of Cost Assessment and Program Evaluation (representing the budget community). In 2007, we identified several best practices for portfolio management, which we revalidated in 2015. These leading practices encourage organizations to assess product investments collectively from an enterprise level, rather than as independent and unrelated initiatives; continually make go/no-go decisions through a gated review process to rebalance portfolios based on investments that add the most value; use an integrated approach to prioritize needs and allocate resources in accordance with strategic goals; rank and select investments using a disciplined process to assess the costs, benefits, and risks of alternative products; empower leadership to make investment decisions and hold leadership accountable for investment outcomes; and provide sustained leadership for portfolio management. Portfolio management best practices and the Project Management Institute’s portfolio management standards also state that organizations should conduct regular reviews to adjust to strategic changes or changes in the mix of products within a portfolio, among other reasons. From a DOD perspective, portfolio reviews can help increase return on taxpayers’ investments in weapon systems in a number of ways, such as: helping to ensure investments align with national security and military strategies; prioritizing the most important investments; selecting the optimum mix of investments; identifying and eliminating unwarranted duplication; monitoring programs’ health to determine whether changes to the portfolio are warranted; and determining whether investments are affordable. We have previously reported that DOD was not effectively using portfolio management to optimize its weapon system investments. In 2015, we identified several factors that inhibited DOD’s ability to do so, including fragmented governance, a lack of sustained leadership, and a perceived lack of decision-making authority at the enterprise level. We also found that DOD’s portfolio management policy was dated, not fully consistent with best practices, and was not being implemented by the Department, in part due to changes in leadership priorities. Further, DOD’s enterprise- level requirements, acquisition, and budgeting communities—meaning those at the Office of the Secretary of Defense, Joint Chiefs of Staff, and Joint Staff level—were not consistently conducting portfolio reviews or collaborating to integrate key information. As a result, we reported that DOD may have been missing opportunities to better leverage its resources and identify investment priorities that best reflect DOD-wide needs. We recommended that DOD update its portfolio management policy; designate a senior official responsible for its implementation; conduct annual portfolio reviews that integrate key information from the requirements, acquisition, and budget processes; and invest in analytical tools to support its portfolio management efforts. DOD partially concurred with the recommendations, but the planned actions DOD identified at the time of our report did not fully address the issues we identified. As of March 2019, DOD had yet to implement our recommendations from 2015 (see table 16 for details of implementation status). It is too soon to assess the effect of the acquisition reforms we reviewed on DOD’s portfolio management efforts because a critical mass of programs has not yet gone through the new acquisition processes. Depending on how the department implements these reforms, some aspects of these reforms could help to address previously-identified deficiencies in portfolio management in the department. For example: Officials in the Office of the Under Secretary of Defense for Acquisition and Sustainment told us that now that milestone decision authority for major defense acquisition programs has largely shifted from the Office of the Secretary of Defense to the military departments, they expect that they will have more time to focus on portfolio-level issues such as identifying how systems need to work together to fill capability gaps since they are less involved in the details of individual programs. We previously reported that DOD’s processes were too focused on optimizing individual investments rather than considering investments across the department. The process developed by DOD to establish cost, fielding, and performance goals brings together officials from DOD’s acquisition, requirements, and budget communities, the three key entities with responsibility for portfolio management, to provide advice on the establishment of program goals. We previously reported that DOD’s enterprise-level processes, organizations, and decision makers oversee weapon system investments generally as stove-pipes and not as an integrated whole. While the process assesses programs on an individual basis rather than collectively from an enterprise level as called for by portfolio management best practices, it may still provide additional shared insight across the acquisition, requirements, and budget communities to help assess portfolios in a more integrated fashion at an enterprise level. However, other aspects of certain reforms have the potential to exacerbate challenges we have previously identified with DOD’s portfolio management approach if not actively managed. For example: Realigning roles and responsibilities for decisions related to weapon systems programs could lead to further questions about who is ultimately responsible and accountable for portfolio management decisions if leadership roles are not clearly defined. We previously reported that DOD’s governance structure for portfolio management was fragmented in part as a result of widely-dispersed decision making responsibilities for weapon system investments. We found that this dispersion of responsibility made it difficult to determine who was empowered to make enterprise-level weapon system investment decisions and who can be considered portfolio managers. According to portfolio management best practices, leadership should be clearly defined and held accountable for outcomes. Programs under middle-tier acquisition pathways have fewer requirements to report program information to offices within the Office of the Secretary of Defense and the Joint Staff than major defense acquisition programs. For example, middle-tier acquisition programs are generally exempted from the Joint Capabilities Integration and Development System for requirements development. Therefore Joint Staff officials may have less information about program requirements than for a major defense acquisition program. Office of the Secretary of Defense officials told us they are working with the military departments and other stakeholders to determine what information is needed for oversight and portfolio management for middle-tier acquisition programs. Office of the Secretary of Defense and Joint Staff officials told us that guidance issued by the Under Secretary of Defense for Acquisition and Sustainment in October 2018 that gives the Office of the Secretary of Defense and Joint Staff formal roles in a governance process may help to ensure sufficient insight. DOD’s ability to develop a common set of portfolios to facilitate integrated portfolio analysis may be more difficult. We previously reported that the requirements, acquisition, and budget communities at DOD were using different portfolio constructs, meaning that they defined their portfolios differently and did not use a standard approach to group investments into portfolios. We identified the use of different approaches as a barrier to taking an integrated approach to prioritize needs and allocate resources in accordance with strategic goals, as called for by portfolio management best practices. For example, the requirements community uses eight joint capability areas for examining warfighter needs, acquisition portfolios vary by military department, and budget data are organized into 11 major force programs. In our prior work, many officials at DOD said that using a wide variety of portfolio constructs is necessary and sometimes beneficial given the different roles and perspectives of the organizations involved. However, as notionally illustrated in figure 8, the different communities need to go through an extensive mapping exercise when they want to analyze their portfolios from another perspective—for example, examining funding associated with joint capability areas. With the reorganization of the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics, officials from the Offices of the Under Secretaries of Defense for Research and Engineering and Acquisition and Sustainment told us that portfolio management activities that used to be conducted by the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics are now split between their offices. Officials from the Office of the Under Secretary of Defense for Research and Engineering told us that they were still in the process of determining what portfolio construct they would use to group investments for portfolio management purposes. If that office decides to use a different portfolio construct than other entities, that decision will increase the already complex process of mapping together portfolios in order to perform an integrated portfolio analysis. Officials from both offices told us that they were working on a pilot effort to conduct portfolio management by focusing on DOD’s missions rather than programs, which could help to standardize the portfolio constructs if the approach is accepted on a wider scale. In addition to the contact named above, Cheryl Andrew (Assistant Director), Marie Ahearn, Peter W. Anderson, David Dornisch, Anne McDonough, Melissa Pope, Scott Purdy, Juli Steinhouse, Sara Sullivan, Anne Louise Taylor, Alyssa Weir, and David Wishard made key contributions to this report.