Acquisition Reform: DOD Should Streamline Its Decision-Making Process for Weapon Systems to Reduce Inefficiencies
Published: 2015-02-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DOD has long sought to improve the efficiency of its weapon system acquisition process, including the time and effort needed to complete the milestone decision process.
The National Defense Authorization Act for Fiscal Year 2014 mandated GAO to review DOD's weapon system acquisition process. This report examines (1) the effort and value involved in the preparation for a milestone decision; (2) factors that influence the time needed to complete the milestone decision process; and (3) alternative processes used by some DOD programs and leading commercial firms.
To perform this work, GAO examined the levels of review and information requirements that are part of DOD's process. GAO surveyed 24 program managers and 40 other DOD officials on the value and the time to complete milestone documentation. For 15 program offices, we gathered data on the time to complete the entire milestone decision process. GAO discussed with DOD officials the factors that lead to inefficiencies. GAO also examined practices used by some classified DOD programs and five commercial firms generally recognized as leaders in product development.
section_title: What GAO Found
paragraphs: The acquisition programs GAO surveyed spent, on average, over 2 years completing numerous information requirements for their most recent milestone decision, yet acquisition officials considered only about half of the requirements as high value. The requirements, in total, averaged 5,600 staff days to document.
The Department of Defense's (DOD) review process is a key factor that influences the time needed to complete information requirements. The process in some instances can include up to 56 organizations at 8 levels and accounts for about half of the time needed to complete information requirements. Most program managers felt that these reviews added high value to only 10 percent of the documents.
DOD's F-16 aircraft program, some classified programs, and five commercial firms GAO visited use streamlined processes with fewer documents and reviews and offer alternatives to the traditional DOD process. Establishing an efficient process for documentation and oversight is a key internal control to avoid wasteful spending. The challenge is to find the right balance between effective oversight and the competing demands on programs. DOD, however, has not yet identified ways to achieve the right balance by minimizing the time spent on information requirements and reviews that contribute to its inefficient milestone decision process.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOD identify and potentially eliminate reviews and information requirements that are no longer needed and select programs to pilot more streamlined approaches to provide only the most essential information to decision makers. DOD concurred with both recommendations.

section_title: Letter
paragraphs: The Department of Defense (DOD) has repeatedly delivered the most  capable weapon systems in the world, but with consistent schedule  delays and at significant cost to taxpayers. The process used to manage  the acquisition of these systems has been characterized by organizations  both internal and external to DOD as one that is inefficient, cumbersome,  and bureaucratic. A contributing factor to this inefficient process is the  significant time and effort required to complete information requirements  before an acquisition program can proceed through a milestone to the  next phase in the weapon system acquisition process. DOD leadership  has acknowledged that too much time is invested in preparing for key  milestones, including the documentation and oversight of information  required by statutes and policy, which takes time away from conducting  day-to-day core program management tasks such as contractor  oversight, engineering, and risk management.
There is a natural tension between oversight and accountability on one  hand and efficient program management on the other. Oversight and  accountability add process, which is justifiable given the high cost and  risk associated with major weapons and the inherent incentives for  program managers and contractors to be optimistic about what they can  accomplish. The additional process does come at a cost of time and other  resources that a program manager could potentially employ better. If it  were clearly demonstrable that program cost and schedule outcomes  have improved as oversight has intensified, then the additional process  would be easy to accept. But, as we have reported over the years,  program outcomes have not significantly improved. Thus, it is an  appropriate time to assess the value of the documents and reviews  demanded of today’s acquisition process. We are not suggesting that  individual actions to improve oversight were not justifiable, but rather  questioning whether their cumulative demand is in balance with their  desired effects.
According to federal internal control standards, agencies should develop  effective and efficient processes to ensure that actions are taken to  address requirements, such as in this case, completing the information  required to aid in milestone decisions. DOD has embarked on some  initiatives aimed at streamlining different aspects of its weapon system  acquisition process, but acknowledges more needs to be done. In the  National Defense Authorization Act for Fiscal Year 2014, Congress  mandated that GAO review DOD’s weapon systems acquisition process,  with an objective of identifying processes or procedures with little or no  value added. This report examines (1) the effort and value involved in  DOD’s preparation for a milestone decision, (2) the factors that influence  the time needed to complete the milestone decision process, and           (3) alternative processes used by some DOD programs and leading  commercial firms.
In conducting our review, we examined relevant statutes, DOD policies,  and military service guidance for DOD acquisitions. To help us gather  data on the effort and value involved with preparing for a milestone  review, we used two surveys on nongeneralizable samples of programs.  Fifteen of 19 relevant major defense acquisition programs’ offices  completed the first survey. These programs each had a milestone  decision since January 2011 and provided data on the time involved with  completing the milestone decision process. Respondents for the other  survey were 24 major defense acquisition program managers, 25  program executive officers, 3 military service acquisition executives, and  12 Office of the Secretary of Defense organizations identified as key  stakeholders in the acquisition milestone decision process. These  respondents provided the value they place on the statutory and policy  information requirements. We received the completed surveys between  July and October 2014. In addition, we selected four of these programs— two Army, one Air Force, and one Navy, that recently each had a  milestone decision—as case studies to gain more in-depth knowledge  about the milestone decision process. To understand the milestone  decision process, key participants’ roles, and factors influencing the  efficiency of the process, we met with acquisition executives and  functional leaders from disciplines such as testing, systems engineering,  and cost within the Office of the Secretary of Defense and the military  services. We also met with five leading commercial firms to examine their  milestone decision processes. We discussed their review process,  participants’ roles, and key practices that enable a more streamlined  approach. Finally, we reviewed several DOD classified programs to learn  more about other DOD acquisition models. A more detailed discussion of  our objectives, scope, and methodology is included in appendix I.
We conducted this performance audit from January 2014 to February  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: As of December 2014, DOD’s portfolio of major defense acquisition  programs included 78 programs with a total estimated acquisition cost of  roughly $1.4 trillion. The Under Secretary of Defense for Acquisition,  Technology and Logistics is the defense acquisition executive, and for 38  of these 78 programs, the Under Secretary is the milestone decision  authority, responsible for making decisions at major program milestones.  These programs are referred to as Acquisition Category (ACAT) ID  programs. For the remaining 40 programs, most of which are in  production, the Under Secretary has delegated milestone decision  making authority to the cognizant military service acquisition executive;  these programs are referred to as ACAT IC programs. DOD also has  programs that have not entered the engineering and manufacturing  development phase. These programs are not yet part of the portfolio, but  are expected to enter soon. For these programs, the Under Secretary is  normally the milestone decision authority.
In DOD’s acquisition process, weapon system programs typically proceed  through three major milestones—A, B, and C—where program offices  provide information to the milestone decision authority in order to make a  decision on whether the program is ready to transition to the next  acquisition phase. The milestones normally represent transition points in  the overall acquisition process where there is a marked increase in the  resources required for the program. Milestone A is the decision for an  acquisition program to enter into the technology maturation and risk  reduction phase; Milestone B is the decision to enter the engineering and  manufacturing development phase; and Milestone C is the decision to  enter the production and deployment phase. Figure 1 depicts DOD’s  acquisition process.
DOD’s acquisition process is managed and supported by officials at  different hierarchical levels. Weapon system program managers typically  report to program executive officers in each military service who are  charged with overseeing the execution of a portfolio of related systems  such as fighter aircraft or ships. Program executive officers, in turn,  typically report to a military service acquisition executive, who reports to  the defense acquisition executive. As part of the milestone decision  process, programs are reviewed at each level before reaching the  milestone decision authority. Figure 2 shows the different levels.
Statutes and DOD policy require the documentation of specific  information on major defense acquisition programs at each acquisition  milestone. Our review focused on the information required at Milestone B,  most of which is also expected at Milestone C. Appendix II includes a list  and description of these information requirements. While several different  Office of the Secretary of Defense organizations and other organizations  have responsibility for compiling and documenting the information, the  majority of the responsibility rests with the program office managing the  acquisition.
For nearly 20 years, GAO has examined the best practices for product  development from over 40 commercial firms to identify potential  opportunities for DOD to adopt and implement those practices. For this  review, we visited five leading commercial firms that follow a gated or  milestone process in developing new products. While their business  models are different than DOD’s, and often their products are less  technically complex, commercial firms share a common goal with DOD in  delivering their products to their customer on time and within cost  estimates. Leading commercial firms can provide alternative approaches  for milestone decision processes.
section_title: Information Required for Milestone Decisions Takes Significant Time and Effort to Document, but Does Not Always Add High Value
paragraphs: Programs we surveyed spent on average over 2 years completing the  steps necessary to document up to 49 information requirements for their  most recent acquisition milestone. This includes the time for the program  office to develop the documentation and for various stakeholders to  review and approve the documentation. These 49 information  requirements also took, in total, on average 5,600 staff days for programs  to document. However, on average, almost half of these requirements, 24  of the 49, were not highly valued by the acquisition officials we surveyed.  Four major defense acquisition programs we examined illustrate the  challenges in completing the milestone decision process.
subsections: section_title: Programs Spend Considerable Time and Resources Documenting the Information Required at Milestones, but the Majority Are Not Highly Valued By Acquisition Officials
paragraphs: Programs can spend a significant amount of time documenting up to 49  information requirements in advance of a Milestone B or C review. The  requirements cover a vast array of program information, such as  information on the overall acquisition strategy to justify the business case  for a program; detailed implementation plans, such as those for systems  engineering and testing; informational reports, analysis, and  assessments; and decisions and certifications.
We surveyed 24 program managers that held a milestone B or C decision  since 2010 and found that it took them over 2 years on average to  complete the entire set of documents needed for the milestone decision.  The program managers, as well as other acquisition officials we  surveyed, considered on average about half of the information  requirements as not highly valued. Figure 3 provides a summary of this  information. More details about the survey results are presented in  appendix III.
Programs spent an average of about 1 year to complete each information  requirement. However, as shown in figure 3, there was a wide range in  the length of time it took to complete documentation, as some took almost  2 years to complete and some took less than 6 months. About half of the  time for each information requirement was spent documenting the  information and the other half for review. These 49 requirements also  took, in total, on average 5,600 staff days for programs to document. We  did not ask programs to provide data on the staff days needed to review  and approve the documentation because they do not have access to data  on the amount of time officials at levels above them spend completing this  process.
As shown in figure 3, acquisition officials on average considered
24 requirements as providing high value to their organization’s role in  the milestone decision process,
20 requirements as providing moderate value, and
5 requirements as providing less than moderate value.
Information requirements considered high value by stakeholders include a  program’s acquisition strategy, sustainment plan, and information related  to planned technologies, cost, and testing. Several senior acquisition  officials we met with considered many of these requirements as critical to  the program’s business case,the capabilities required of the weapon system, the strategy for acquiring  the weapon system, and the cost, schedule, and performance baselines.   which typically includes documentation on  Information requirements valued the least (less than moderate value), on  the other hand, include such documentation as the benefit analysis and  determination for potentially bundling contract requirements; the Clinger- Cohen certification for information technology investments; the corrosion  prevention control plan to assess the impact of corrosion on cost,  availability, and safety of equipment; the item unique identification  implementation plan for managing assets; and the replaced system  sustainment plan for documenting the estimated cost to sustain a system  until the new program is fielded. One service acquisition executive, for  example, stated that program managers should not have to develop an  item unique identification implementation plan because government  contractors put the unique identification numbers on parts. Another senior  official stated that the Clinger-Cohen Act requirements are geared  towards the acquisition environment of the 1990s. This official believes  the requirements should be updated to reflect the current environment for  procuring information systems.
As part of the process of documenting the information required at the  milestones, program officials brief cognizant officials responsible for the  different functional areas, such as test or systems engineering, as well as  senior leadership and the milestone decision authority on specific aspects  of the program’s overall plans. The briefings, done in parallel with the  actual process of documenting the required information, are used as a  forum for DOD to discuss the information and to determine a program’s  readiness for the milestone decision. Program offices can spend a great  deal of time and effort briefing the different officials and senior leaders in  advance of the milestone decision. Data provided by 9 of the programs  we surveyed that recently had a milestone B decision showed that  programs provided an average of 55 briefings over a period of just over a  year and a half leading up to the milestone.
section_title: Four Acquisition Programs Illustrate the Significant Time and Effort to Complete the Information Requirements
paragraphs: We examined four major defense acquisition programs, at least one from  each military service that recently held a milestone decision, to get more  specific details of the time and effort expended by programs to complete  the milestone decision process. All four programs needed about 24  months to complete the process. While the number of documents varied  for each program, it took an average of over 13 months to complete each  document based on three programs that could provide data. Two of these  programs used contractors to provide assistance in completing the  documents. Figure 4 provides a summary of the overall effort required of  these four programs—two of which were preparing for milestone B and  the other two were preparing for milestone C.
Two of the programs that tracked the staff days required to prepare the  milestone documents told us they spent 3,800 and 9,867 staff days,  respectively. These same programs also used contractors to assist with  the documents.
section_title: The Number of Stakeholder Reviews at Each Milestone Is a Large Contributor to the Effort Needed to Complete Information Requirements
paragraphs: A primary reason it takes over 2 years to complete the information  required for a milestone decision is the large number of stakeholders that  review the documents at the many organizational levels above the  program office. We found that stakeholders in many different offices  among 8 different levels can review the information and documentation  needed to support a milestone decision. According to the program offices  we surveyed, these reviews added only moderate or less value to most  documents. DOD recognizes that it has too many levels of review and  has several initiatives to eliminate the acknowledged bureaucracy, but  has had limited success implementing changes to reduce the time and  effort needed to review documentation.
subsections: section_title: Reviews Are Conducted at Multiple Levels by Different Functional Organizations
paragraphs: The information and documentation required at milestones can be  reviewed by as many as eight different organizational levels before a  decision is reached on whether a program is ready for the next acquisition  phase. In general, the information is reviewed at each level to gain  approval before the program provides the information to the next level.  This is done serially, which takes more time. Eventually, the defense  acquisition executive and other senior executives review the information  and determine whether the program is ready to proceed to the next  acquisition phase. Figure 5 shows the multiple levels of reviews.
Many different functional organizations within each level review the  information before the document is approved. The number of  organizations conducting reviews varies depending on the information  included in each document. A few documents that include a wide breadth  of information can be reviewed by many offices at each level. For  example, Air Force acquisition strategies, that on average took over 12  months to complete for the programs we surveyed, can be reviewed by  56 offices, some more than once, before being approved. Figure 6 lists  the organizations involved in this review process.
The reviews of more narrowly focused documentation also go through  different levels, but may be reviewed by fewer organizations at each level.  As one example, offices at as many as four levels took an average of 7  months to review a program’s Technology Readiness Assessment, based  on responses to our survey. This assessment is prepared prior to  Milestone B to show the results of an assessment of the maturity levels of  the critical technologies planned to be integrated onto the program.  Initially, the program office prepares an assessment of the different  technologies’ maturity levels, taking into account the conclusions reached  by a panel of independent subject matter experts. Then, the program  executive officer reviews and approves the assessment. Next, a service  level expert, with possible assistance from a science and technology  expert, reviews the assessment. After that, officials from the Office of the  Secretary of Defense, Assistant Secretary for Research and Engineering  office evaluate the assessment and make their own independent  assessment. Finally, the milestone decision authority certifies whether to  approve the program to enter engineering and manufacturing  development or defer this decision until technologies are mature. Each  of these four levels of review, done serially, can present new questions  and comments that need to be resolved before the program can satisfy  the information requirement.
GAO, Defense Acquisitions: Assessments of Selected Weapon Programs,  GAO-08-467SP (Washington, D.C.: Mar. 31, 2008); and Defense Acquisitions:  Assessments of Major Weapon Programs, GAO-04-248 (Washington, D.C.: Mar. 31,  2004). program results is related instead to incentives. We have reported  previously on several factors that create incentives for DOD to deviate  from sound acquisition practices and reform initiatives. These factors  include (1) mismatches between capability requirements and the  knowledge, funding, and time planned to develop a new system, (2)  programs being started to fill voids in military capability but quickly  evolving to address other, conflicting demands, and (3) programs being  funded in a way where there are few consequences if funding is not used  efficiently.
DOD has recognized that its extensive review process is a challenge. A  DOD study in 2011 highlighted the many organizational levels of  oversight and said DOD has a “checkers checking checkers” system,  which contributes to inefficiencies that can undermine program managers’  execution of programs because they spend too much time complying with  the oversight process, including documenting the information  requirements.time and resources addressing conflicting comments/concerns expressed  by the functional offices at the different levels during the review process.  Officials also told us the functional staff conducting reviews typically  wanted significantly more information than their superiors want or need  and this often leads to multiple revisions. For example, the Deputy  Assistant Secretary for Systems Engineering has indicated he wants  limited, specific information in a systems engineering plan and even  issued guidance to promulgate this direction. Despite this direction, we  were told the systems engineering plan for one Navy program grew from  100 pages to 243 pages in length, because staff wanted additional  information added as it went through the review process. In contrast,  however, one of the three service acquisition executives we surveyed and  some senior level officials within the Office of the Secretary of Defense  stated that staff reviews are helpful as they ensure the documentation is  sufficient before executives at each level perform their review.
Several program officials told us they spend extensive  Service officials also told us that while it is important to get input from  functional staffs on their areas of expertise, these staffs can have “tunnel  vision,” or focus only on their respective area and do not adequately  consider whether their recommended changes to documentation might  add schedule time, additional costs, or have other effects on a program.  Officials expressed frustration that functional staffs are not held  accountable for the potential effect on a program as a result of their  recommended changes. Recently, the Under Secretary of Defense for  Acquisition, Technology and Logistics has tried to clarify the role of some  staff, stating in a memorandum that the service acquisition executives,  the program executive officers, and program managers are responsible  and accountable for the programs they manage; everyone else (i.e., staff  supporting the Office of the Secretary of Defense staff) has a supporting  or advisory role.
section_title: Documentation Reviews Are Not Highly Valued by Program Officials
paragraphs: While there are multiple levels and many organizations involved in  reviews, overall the 24 program managers we surveyed did not think  these reviews added significant value to the documentation. The program  managers considered the value added to 10 percent of the  documentation to be high. However, for the remaining 90 percent of the  documents, the officials believed the reviews did not add high value—61  percent were moderate and 29 percent less than moderate. Figure 7  provides a summary of the program offices’ assessment.
Of the 14 documentation reviews that were considered to add less than  moderate value, 2 documents were reviewed for an average of 10 months  each and the others ranged between 2.5 and 8.5 months. Other service  level officials we surveyed—program executive officers and the three  service acquisition executives—had views similar to the program  managers; they considered the value added to be high for less than 10  percent of the documentation.
section_title: DOD Has Had Limited Success in Streamlining Its Milestone Decision Process
paragraphs: DOD has acknowledged that too much time is spent on reviews and  preparing documents and has taken some steps over the past several  years to address some of the unproductive steps identified in its  milestone decision processes. For the most part, however, efforts to  date have been limited in scope and have not had a significant effect on  the amount of time and effort program offices spend on documentation  required at milestones. One has even stalled. Examples of these efforts  include:  In 2011, the Under Secretary for Acquisition, Technology and  Logistics delegated the approval authority for three milestone  documents from the Office of the Secretary of Defense level to the  service level. This reduced the number of levels of review and  reviewers of these documents. A DOD official told us the approval  authority for additional documents could be delegated in the future,  but no additional documents are currently being considered.
In 2013, the Under Secretary for Acquisition, Technology and  Logistics asked the service acquisition executives to identify programs  where the milestone decision authority could be potentially delegated  from the Office of the Secretary of Defense to a lower level.  Delegation to the lower level also reduces the number of levels of  review and reviewers. The services identified 18 programs: 7  programs from the Air Force; 5 programs from the Navy; and 6  programs from the Army. In September 2014 the Under Secretary of  Defense for Acquisition, Technology and Logistics delegated the  authority to act as the milestone decision authority to the Secretary of  the Air Force for 3 programs, the Secretary of the Navy for 1 program,  and the Secretary of the Army for 1 program.
In April 2013, the Under Secretary of Defense for Acquisition,  Technology and Logistics issued guidance that included a potential  pilot test of a “skunkworks” process for major defense acquisition  programs. The Under Secretary requested that each service  recommend one candidate program for a pilot test by July 2013. As of  October 2014, programs had not been identified and the effort has  been placed on hold. Office of the Secretary of Defense officials  stated it has been difficult to identify programs that meet the Under  Secretary’s expected preconditions—namely to identify programs that  have well defined requirements, a strong relationship with industry,  and a highly qualified and appropriately staffed government team that  can remain with the program until it is delivered.
In 2014, DOD began using an Electronic Coordination Tool designed  to electronically disseminate and track the progress of documentation  being reviewed. The tool is used to enforce time limits for the review  of documents and provide near real-time views of all comments made  during the review process to promote greater efficiency across the  department. DOD officials have begun using this tool with the  Acquisition Strategy and hope to add more documents over time.
DOD is currently assessing many of the documents it develops in  response to statutory information requirements and plans to propose  legislative modifications to Congress in the spring of 2015 to help  streamline documentation while still meeting the intentions of the  statutes.
DOD’s revised acquisition policy has also placed greater emphasis on  “tailoring,” which means modifying the traditional acquisition process,  including documentation and reviews, to best suit a program’s needs.
However, a few program officials told us that trying to tailor by obtaining  waivers for milestone requirements involves significant time and effort,  and that it is often easier to simply complete the requirements rather than  try to obtain waivers. While we did not examine the overall use of tailoring  by DOD programs during our review, we examined two programs that  attempted tailored documentation and reviews, but in the end, neither  was able to make significant changes. Specifically, the F-22 Increment  3.2B program told us they requested waivers for 17 requirements, but  ultimately only 2 were waived. In addition, the Long Range Strike– Bomber, in direction provided by the former Secretary of Defense, was to  be managed with a streamlined approach. The program was initially  allowed the flexibility to tailor many of the needed documents and  reviews. However, over time, these flexibilities have been scaled back.
section_title: DOD Has Experience Using a Streamlined Process, and Commercial Firms Also Offer Alternative Approaches
paragraphs: DOD has proven it can streamline its process. Several past programs,  like the F-16 and F-117, were managed successfully with a more  streamlined approach and DOD is currently using a more streamlined  milestone decision process for some classified programs. Commercial  companies we examined—Boeing, Caterpillar, Cummins, Honda, and  Motorola Solutions—also use processes that minimize the levels of  review resulting in a quicker, more efficient milestone decision process.
subsections: section_title: DOD’s Initial Acquisition Policy Provided a Framework for a More Streamlined Process
paragraphs: In 1971, DOD issued its initial 5000 acquisition policy. The policy, which  totaled seven pages, provided for minimum formal reporting and more  streamlined layers of authority than the complex process in place today.  Specifically, the original guidance provided for (1) minimal layers of  authority above the program office; (2) few demands on programs for  formal reporting; (3) minimal demands for non-recurring information and  for responding to these requests informally; and (4) the development of a  single, key document to support program management and milestone  decision making. Over time, a large, bureaucratic process has supplanted  these elements. For example, requirements have been added to improve  cost estimating, logistics planning, design reviews, and technology  maturity assessments. Each of these areas has been in great need of  improvement and individual documentation and review requirements were  aimed at addressing known shortfalls.
Several studies by acquisition experts over the past decade have  highlighted the need for DOD to again streamline its process. For  example, a Defense Acquisition Performance Panel stated in its 2006  report that complex acquisition processes do not promote program  success, but increase cost, add to schedules, and obfuscate  accountability. The Panel recommended that DOD create a streamlined  acquisition organization with accountability assigned and enforced at  each level. In 2009, the Defense Science Board reported that DOD’s  milestone decision process should take a few days of preparation, not the  months and months currently required. The report describes a process  with too much bureaucracy, overlap and diffusion of responsibilities, and  a need for excessive coordination among acquisition organizations. The  report recommended that DOD streamline the acquisition process.
section_title: DOD Has Used a More Streamlined Process with Success
paragraphs: The F-16 program, developed in the 1970s, was managed under a  streamlined process laid out in the early acquisition guidance. DOD  officials have often stated that one contributing factor to the F-16  program’s success was the use of a more streamlined approach, where  the number of levels of review and reviewers was minimized, and  emphasis was placed on real-time program reviews in lieu of preparing  formal reports and documents. According to one former DOD senior  official, the program office was staffed with an experienced program  manager and functional experts, which worked closely and collaborated  with functional offices to achieve a common goal of fielding a usable  combat capability as quickly as possible. The F-16 program also operated  with different incentives than most programs, which enabled a more  streamlined approach. For example, the F-16 was developed as a low- cost fighter with a strategy that involved making incremental technology  improvements and incorporating performance trades by the customer to  keep costs down.
The F-117 aircraft, which was largely developed in the early 1980s in a  classified security environment, was managed with a “skunkworks”  approach. According to a RAND study, central to the F-117 program approach was its flexibility and responsiveness in decision-making.DOD leadership delegated more decision-making to the program office,  with an associated reduction in detailed, document oversight by higher  levels. RAND stated that the willingness to delegate decision making  authority to lower levels enabled a quicker response to problems. A  former Air Force senior leader during the program’s development, who  later served as the Under Secretary of Defense (Acquisition and  Technology), stated that the program held monthly meetings between the  functional managers and program management. The meeting participants  were empowered to make decisions and did not need to seek approval  from their senior leadership after the meetings. It was expected that any  issues would be addressed with their leadership prior to the meetings.  The frequent interactions reduced the need for reports, documents, and  reviews. In another report, a former senior Air Force acquisition officer,  who also served as program director for the F-117, reported that the  ability to have a quicker process comes from pushing decision-making to  the lowest levels without having to proceed up the chain of command for  approval to implement decisions.
DOD is using a more streamlined approach for some of its current  classified programs that may have the potential to make the milestone  decision process more efficient. A few classified programs we reviewed  are managed with a process that includes fewer levels and reviewers  between the program office and decision authority. the program manager reports to the program executive officer who  reports directly to a Board of Directors comprised of the service  acquisition executive, service secretary and chief, and the defense  acquisition executive. The Board of Directors serves as the milestone  decision authority for the programs. Decisions by the Board are  unanimous agreements by all members. Leading up to the Board  meeting, programs have separate, focused interactions with a small  number of key functional offices as necessary. According to service  officials, establishing this short, narrow chain of command allows for a  more expedited decision-making process that requires less time and  resources. Figure 8 shows the levels of review for the milestone decision  process.
Section 2430 of title 10, U.S. Code, specifically excludes highly sensitive classified  programs (as determined by the Secretary of Defense) from the definition of a major  defense acquisition programs. Therefore, statutes governing major defense acquisition  programs generally do not apply to classified programs.
section_title: Commercial Firms Provide Alternative Approaches for Milestone Decisions
paragraphs: Commercial companies we examined—Boeing, Caterpillar, Cummins,  Honda, and Motorola Solutions—use a more streamlined process than  DOD traditionally uses for its major defense acquisition programs.  Companies prepare similar documents as DOD acquisition programs, but  only a few of the most critical ones, the business case documents, require  senior management approval. A key enabler to this approach is the  establishment of frequent, regular interactions between program officials  and decision makers. Companies minimize the levels of review needed to  determine whether a program is ready to advance to the next acquisition  phase, resulting in a quicker, more efficient process.
The companies prepared documents similar to those of DOD such as  development, test, engineering, and manufacturing plans. Officials at  Motorola Solutions, Cummins, and Boeing stated that most documents  are prepared and approved by functional managers assigned to the  program office core team. Programs prepare an integrated document that  summarizes key program information for the decision makers to review  and approve. Figure 9 illustrates the levels at which documents are  generally prepared and approved for commercial companies we visited.
According to company officials, the integrated document may include  information on customer requirements, resources, schedules, risks,  technical data, and market launch plans. As part of the milestone decision  process, program managers also provide evidence that the other program  documents have been completed and approved by the appropriate  official.
For the companies we visited, ensuring that the program management  team has a strong link to decision makers was a critical factor to their  streamlined approach. Several companies held meetings between  program officials and senior managers at frequent intervals to assess  progress towards the next milestone decision. Officials stated that  frequent, regular interactions enable senior managers to stay informed of  program issues and plans, allowing the decision meeting to focus on  making a well-informed decision, instead of spending time bringing  decision makers up to date (see fig. 10).
Cummins functional managers, for example, meet one-on-one with  senior program managers on a monthly basis to review program  progress. Officials stated that about 2 weeks prior to the decision  meeting, a comprehensive review is conducted with each program’s  functional area manager, supporting team, and senior functional  manager to ensure required activities have been completed before a  milestone review and the plan going forward is sound.
Boeing officials stated that they conduct a series of monthly meetings  between program functional area managers and senior managers to  assess whether a program is meeting the criteria needed for moving  into the next phase. According to officials, the results provide support  for the milestone decision.
Honda has established an environment that encourages frequent,  direct interaction between program participants. Senior managers,  program managers, and staff work in an open bullpen environment,  rather than offices. This layout facilitates real-time discussions across  organizational levels, multiple programs, and functional areas. Issues  can be quickly discussed and resolved as they arise so only the most  important ones need to be addressed at the milestone decision  meetings.
Companies we visited told us it typically takes only a few months or  sometimes even a few weeks to complete the milestone decision  process. The process for these companies included one or two levels of  review to assess whether a program is ready to advance to the next  phase. For example, Motorola Solutions and Cummins use a process in  which programs proceed directly to the decision maker after they have  packaged together information needed to support a milestone decision.  Motorola Solutions program officials provided information to their decision  makers about a week in advance of the decision meeting. During that  week, program officials meet individually with principal members on the  decision-making committee. The purpose of these meetings is not to  present the program’s plans but to address any last minute concerns.  According to Motorola officials, the decision meeting typically lasts about  30 minutes because issues are usually resolved in these earlier meetings.  Boeing and Honda generally include one additional level of review.
The commercial model, in which good program outcomes can be  achieved with a more streamlined oversight process, includes a natural  incentive that engenders efficient business practices. Market imperatives  incentivize commercial stakeholders to keep a program on track to meet  business goals. In addition, awards and incentives for managers are often  tied to the company’s overall financial success. As a result, commercial  managers are incentivized to raise issues early and seek help if needed.  They know if the program fails, everyone involved fails because market  opportunity is missed and business revenues will be impacted.  Commercial product development cycle times are relatively short (less  than 5 years), making it easier to minimize management turnover and to  maintain accountability. DOD’s acquisitions occur in a different  environment in which cycle times are long (10 to 15 years), management  turnover is frequent, accountability is elusive, and cost and schedules are  not constrained by market forces. Seen in this light, DOD must have an  oversight process that substitutes discipline for commercial market  incentives. Several industry officials stated that companies often add  oversight levels or reviews as a first reaction after failures or problems  occur. However, the officials further stated that this does not solve the  root problems and often it makes the process less efficient. Two  companies we visited highlighted an inspection-intensive oversight  process they implemented as a deliberate attempt to address problems  that had occurred but found that it led to an adversarial environment and  an inefficient process. Both companies eventually abandoned this  approach and replaced it with an approach where program officials are  incentivized to reach out to recognized experts within the company for  assistance when needed.
section_title: Conclusions
paragraphs: Over time, DOD has essentially tried to overcome a legacy of negative  cost and schedule weapon system program outcomes by requiring  extensive documentation to support program strategies, plans, and other  information prior to a milestone decision. Much of the information required  in this documentation was added by policy as well as statute and these  requirements likely represented legitimate reactions to problems.  However, the consequence of this approach is that an extensive process  has built up, in which program offices and other DOD organizations spend  an enormous amount of time and effort preparing and reviewing  documentation. Given the persistence of weapon system acquisition  problems over decades, especially schedule delays and cost overruns,  the effort involved with documenting and reviewing information  requirements does not appear to correspond to the value gained.  Programs we surveyed spent over 2 years completing information  requirements that in some instances can be reviewed by as many as 56  organizations at eight levels. In the end, program officials felt almost half  of these information requirements were not of high value. Further,  program managers did not highly value the reviews by higher level DOD  organizations for 90 percent of the documentation.
The need to document information about essential aspects of a program  and for an appropriate level of review and approval is legitimate.  However, over time, the outcomes of weapon system programs have  proven resistant to the oversight process. At the same time, the process  has become bloated, time-consuming, and cumbersome to complete. The  challenge is to find the right balance between having an effective  oversight process and the competing demands such a process places on  program management. Meeting the challenge will depend on DOD’s  ability to identify the key problem areas in weapon system acquisitions  and the associated root causes that exist today and whether information  requirements and reviews are linked to addressing these problems. As  we have noted in prior work, the most important information  requirements—those that enable a program to establish a sound  business case—include well-defined requirements, reasonable life-cycle  cost estimates, and a knowledge-based acquisition plan. If information  requirements and reviews are not clearly linked with the elements of a  sound business case and/or the key issues facing acquisitions today,  then they can be streamlined or even eliminated. If they are linked, but  are not working well, then they warrant re-thinking. While the data support  that change is needed, change does not mean weakening oversight, as  unsatisfactory outcomes from acquisition programs may persist. Rather,  the goal of change is to perform effective oversight more efficiently, and  to recognize problems or incentives that require remedies and not just  more information requirements.
In this time of decreasing defense budgets, where every dollar spent on  inefficient activities is one less dollar available for modernizing our future  force, a close look at the review process is warranted to provide  stakeholders needed information in a more efficient and cost effective  manner. The surveys of DOD acquisition officials we conducted, the  results of which are shown in figure 3 and appendix III, highlight  information requirements that provide less than moderate value to  acquisition officials. These requirements, as well as ones that take a year  or more to complete, could serve as a starting point for discussions on  what documentation is really needed for weapon acquisition programs  and how to streamline the review process. Officials within the Office of the  Secretary of Defense believe the Electronic Coordination Tool shows  promise for reducing review times on documents. Currently, it is being  used to reduce review times for acquisition strategies, and other  documents may be added in the future. Automating the document review  process, however, is relatively easy compared to potentially eliminating  levels of review because that will require DOD to move away from its  “checkers checking checkers” culture and make tough choices as to  which levels of review do not add value and are not necessary. If DOD  does not eliminate levels of review, inefficiencies are likely to continue.  According to federal internal control standards, agencies should develop  effective and efficient processes to ensure that actions are taken to  address requirements, such as in this case, completing the required  information to aid in milestone decisions. In other words, DOD should be  striving to make its process more efficient.
Selecting pilot test programs to experiment with streamlined acquisition  processes, while capturing lessons learned from the pilot, would be steps  in the right direction. The pilot programs could rely on practices used by  some DOD classified programs and private industry companies we  visited—namely, fewer information requirements and levels of review and  more frequent interaction between the program office and actual decision  makers.
section_title: Recommendations for Executive Action
paragraphs: To help improve DOD’s milestone decision process, we recommend that  the Secretary of Defense direct the Under Secretary of Defense for  Acquisition, Technology and Logistics in collaboration with the military  service acquisition executives, program executive officers, and program  managers take the following two actions:  In the near term, identify and potentially eliminate (1) reviews  associated with information requirements, with a specific focus on  reducing review levels that do not add value, and (2) information  requirements that do not add value and are no longer needed. For the  remaining reviews and information requirements, evaluate and  determine different approaches, such as consolidating information  requirements and delegating approval authority, which could provide  for a more efficient milestone process. This effort should also include  a re-examination of the reason(s) why an information requirement was  originally considered necessary in order to determine what information  is still needed and if a more efficient approach could be used.  Findings and survey responses included in this report could be used  as a starting point for this examination.
As a longer-term effort, select several current or new major defense  acquisition programs to pilot, on a broader scale, different approaches  for streamlining the entire milestone decision process, with the results  evaluated and reported for potential wider use. The pilot programs  should consider the following:
Defining the appropriate information needed to support milestone  decisions while still ensuring program accountability and  oversight. The information should be based on the business case  principles needed for well-informed milestone decisions including  well defined requirements, reasonable life-cycle cost estimates,  and a knowledge-based acquisition plan.
Developing an efficient process for providing this information to  the milestone decision authority by (1) minimizing any reviews  between the program office and the different functional staff  offices within each chain of command level and (2) establishing  frequent, regular interaction between the program office and  milestone decision makers, in lieu of documentation reviews, to  help expedite the process.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided us with written comments on a draft of this report. DOD  concurred with both of our recommendations. DOD’s comments are  reprinted in appendix IV.
DOD concurred with our first recommendation, indicating that the  Department’s Better Buying Power initiative contains efforts to streamline  documentation requirements and staff reviews and its recent (February  2015) set of legislative proposals to Congress seeks to reduce some  DOD reporting requirements. We acknowledge these efforts as steps in  the right direction. We believe DOD can and should do more to eliminate  reviews and information requirements that do not add value and are no  longer needed. For the most part, efforts to date have been limited in  scope and have not yet had a significant impact on the amount of time  and effort program offices spend on documentation required at  milestones. The Under Secretary of Defense for Acquisition, Technology  and Logistics acknowledged in April 2014 that DOD has not had  significant success in eliminating unproductive processes and  bureaucracy. We also note that DOD’s recent set of legislative proposals  to Congress for inclusion into the National Defense Authorization Act for  Fiscal Year 2016 primarily seek to reduce reporting requirements, but do  not address streamlining the many levels of review. As we reported, a  primary reason it takes over 2 years to complete the information required  for a milestone decision is the large number of stakeholders that review  the documents at the many organizational levels above the program  office. While it will take a coordinated effort on the part of the Department,  we believe DOD can reduce the many levels of review.
DOD also concurred with our second recommendation, indicating that  while not yet fully implemented, it has a Better Buying Power initiative to  identify appropriate programs to pilot test a streamlined acquisition  approach. As we reported, however, DOD has not yet identified candidate  programs even though the initiative was proposed in April 2013 and was  supposed to begin in July 2013. DOD officials told us it has been difficult  to identify programs that meet the preconditions for the pilot set by the  Under Secretary of Defense for Acquisition, Technology and Logistics— namely programs that have well defined requirements, a strong  relationship with industry, and a highly qualified government team that  can remain with the program until it is delivered. We encourage DOD to  initiate the pilot, specifically on some current programs that have recently  held a milestone B or will be approaching this milestone soon (e.g.  Presidential Helicopter, Armored Multi-Purpose Vehicle, Joint Air-to- Ground Missile, Next Generation Jammer, Amphibious Combat Vehicle),  as long as the aforementioned criterion of well defined requirements is  considered. Almost two years have passed since the initiative was first  proposed and even after DOD decides on the specific programs for the  pilot, it will most likely be several years until lessons learned can be  documented and potentially applied to other programs. We reiterate that,  when implemented, each pilot should examine different approaches for  streamlining the entire milestone decision process, including defining the  appropriate information needed to support milestone decisions, such as  business case principles like well defined requirements, reasonable life- cycle cost estimates, and a knowledge-based acquisition plan. Pilot  programs should also strive to develop a more efficient process for  providing this information to the milestone decision authority, which would  most likely include minimizing reviews between the program office and  the different functional staff offices within each chain of command level  and establishing frequent, regular interaction between the program office  and milestone decision makers, in lieu of documentation reviews.
We are sending copies of this report to interested congressional  committees; the Secretary of Defense; the Under Secretary of Defense  for Acquisition, Technology and Logistics; and the Secretaries of the Air  Force, Army, and Navy. This report also is available at no charge on  GAO’s website at http://www.gao.gov.
Should you or your staff have any questions on the matters covered in  this report, please contact me at (202) 512-4841 or sullivanm@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report examines the Department of Defense’s (DOD) weapon  system acquisition process. Specifically we examined (1) the effort and  value involved in DOD’s preparation for a milestone decision, (2) the  factors that influence the time needed to complete the milestone decision  process, and (3) alternative processes used by some DOD programs and  leading commercial firms.
To determine the value and effort involved in DOD’s preparation for a  milestone decision, we collected data from current and future major  defense acquisition programs.
First, we distributed two questionnaires by email in an attached Microsoft  Word file asking program managers of current major defense acquisition  programs to (1) provide a value for each information requirement  applicable at either Milestone B or Milestone C; (2) provide a value for the  review of each information requirement for either Milestone B or  Milestone C; (3) provide the length of time required to develop each  information requirement; (4) provide the number of staff days spent by the  program office to develop each information requirement; (5) provide the  length of time it took each information requirement to get through the  review and approval process; and (6) identify the primary users and  customers of each information requirement. One questionnaire was sent  to 11 program managers of current major defense acquisition programs  identified in the Defense Acquisition Management Information Retrieval  system as having completed a Milestone B decision review since January  1, 2010, and a different questionnaire to 15 program managers of current  major defense acquisition programs identified in the Defense Acquisition  Management Information Retrieval system as having completed a  Milestone C decision review since January 1, 2010. We received  responses from 24 program managers, between July and October 2014,  11 program managers from Milestone B programs and 13 program  managers from Milestone C programs. Because there is a slight variation  in the number of information requirements applicable at Milestone B  verses at Milestone C, in our analysis we excluded 2 information  requirements applicable at Milestone C—the Capability Production  Document and the General Equipment Valuation.
We took a number of steps to ensure reliability of the data collected  through our questionnaires, including reviewing responses to identify  obvious errors or inconsistencies and conducting follow-up to clarify  responses when needed.
Second, in a separate data collection effort to determine the number of  briefings and the length of time needed to complete the milestone  decision process, we submitted questions for an electronic questionnaire  distributed to 55 programs, as part of GAO’s Annual Weapons System  Assessment. We asked programs if they had completed a milestone  decision review as of January 1, 2011, and if yes, to provide additional  information regarding that milestone review. Twenty-four programs out of  55 responded that they had completed a milestone decision review in that  time frame; however, not all programs provided information on the review.  Four programs were excluded from our analysis because they were  unable to provide the additional data. Another 5 programs were excluded  because we determined they have been designated as an Acquisition  Category IC program; our analysis was limited to Acquisition Category ID  programs. Of the 15 programs in our analysis, 11 are current programs  and 4 are future programs. Our results are not intended to be  generalizable and as such, results from nongeneralizable samples cannot  be used to make inferences about all major defense acquisition  programs.
To better understand DOD’s milestone process, we selected 4 major  defense acquisition programs to use as case studies to gain more in- depth knowledge about the milestone decision process: the Air Force’s F- 22 Increment 3.2B Modernization program, the Navy’s P-8A program; and  the Army’s Joint Light Tactical Vehicle and Paladin Integrated  Management programs. We used a data collection instrument to ensure  we received similar information for all 4 case study programs in our  review. We collected data on the number of briefings the program office  held with program executive officers, service-level officials, and Office of  the Secretary of Defense-level officials; the number of documents the  program completed for the milestone decision review; and a timeline of  their milestone decision review. In addition, we asked programs to  provide detailed information related to the information requirements they  prepared for the milestone, including the length of time spent  documenting each information requirement; length of time it took the  documentation to make it through the review process; the number of staff  days the program office spent documenting each information  requirement; and the cost to document each information requirement. We  also reviewed milestone documents that programs prepared for the  milestone in order to better understand what information is contained  within the documents. Finally, we met with program officials from each  case study program to obtain additional information on the milestone  decision process.
We selected our case studies based on input from officials with the  military services using the criterion that the program had been through  either Milestone B or Milestone C since January 1, 2010. Further, the  programs we selected for review represent each of the military services.  Two programs—F-22 Increment 3.2B Modernization and Joint Light  Tactical Vehicle—completed a Milestone B review and 2 programs—P-8A  and Paladin Integrated Management—completed a Milestone C review.  While our sample of four case studies allowed us to learn about  inefficiencies with the milestone decision process, it was designed to  provide anecdotal information, not findings that would be representative  of all of the department’s major defense acquisition programs.
To determine the factors that influence the time needed to complete the  milestone decision process, we met with officials and functional leaders  and reviewed documents from several organizations within the Office of  the Under Secretary of Defense for Acquisition, Technology and  Logistics, including the Under Secretary. Specifically, we met with officials  from the offices of (1) Acquisition Resources and Analysis; (2) Defense  Procurement and Acquisition Policy; (3) Deputy Assistant Secretary of  Defense for Systems Engineering; and (4) Deputy Assistant Secretary of  Defense for Developmental Test and Evaluation. We also met with  officials from the Office of the Director, Cost Assessment and Program  Evaluation; and the Office of the Director, Operational Test and  Evaluation.
We also met with officials and reviewed documents from the military  services, including the Department of the Air Force, Department of the  Army, and the Department of the Navy, including the service acquisition  executives. Within each military service, we also met with officials from  functional offices including the (1) Air Force Director of Test and  Evaluation; (2) Deputy Assistant Secretary of the Air Force for Science,  Technology, and Engineering; (3) Deputy Assistant Secretary of the Navy  for Research, Development, Test & Evaluation; (4) Deputy Under  Secretary of the Army for Test and Evaluation; and the (5) Director of  Army System of Systems Engineering and Integration.
In order to capture the views of officials at the different levels involved in  the milestone decision process, we also sent a questionnaire to program  executive officers with responsibility for defense acquisition programs, all  3 military service acquisition executives, and 13 Office of the Secretary of  Defense organizations identified as key stakeholders in the acquisition  milestone decision process. We received responses from 25 program  executive officers, all 3 military service acquisition executives, and 12  Office of the Secretary of Defense organizations.
We analyzed the data provided by program managers, program executive  officers, military service acquisition executives, and Office of the  Secretary of Defense officials to determine the overall value of the  milestone information requirements and the overall value of the review of  the information requirements to the various groups involved in the  milestone decision process. Our results are not intended to be  generalizable and as such, results from nongeneralizable samples cannot  be used to make inferences about all major defense acquisition  programs.
Further, we reviewed relevant statutes, DOD policies, and military service  guidance for DOD acquisitions.
To examine alternative processes used by some DOD programs, we  reviewed the processes used by some current classified programs. We  also reviewed reports and studies done by acquisition experts that  examined past programs, including the F-117 and F-16, which  successfully used a more streamlined process. In addition, we examined  acquisition policies that were in place at the time of these programs  development.
To identify practices used by leading commercial firms that might be used  to improve DOD’s acquisition process, we visited five companies to learn  more about how they manage their product development processes. We  selected these companies, in part, based on our previous GAO best  practices work. These companies are recognized leaders in their industry,  and are recognized for having successful, proven product development  processes.
The companies selected for use in our review include:
Boeing, a leading aerospace company and a manufacturer of  commercial jetliners. We met with officials and discussed their  practices for managing the development of commercial aircraft in  Seattle, Washington.
Caterpillar Inc. (Caterpillar), a leading manufacturer of construction  and mining equipment, diesel and natural gas engines, and industrial  gas turbines. We met with officials in Peoria, Illinois.
Cummins Inc. (Cummins), a leading manufacturer of diesel and  natural gas-powered engines for on-highway and off-highway use. We  met with officials in Columbus, Indiana.
Honda of America Manufacturing, Inc. (Honda), a leading  manufacturer of motorcycles and automobiles. We met with officials at  their location in Raymond, Ohio.
Motorola Solutions, a leading manufacturer of data capture devices  such as professional and commercial radios and communication  systems. We met with officials in Schaumburg, Illinois.
At each company we discussed the new product development process  from concept to full production; the methods, tools, measures and metrics  used by leadership in monitoring and overseeing product development  execution progress; and roles and responsibilities of the product  development manager.
We conducted this performance audit from January 2014 to February  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Information Requirements for Major Defense Acquisition Programs at Milestone B and C Identified in Interim DOD Instruction 5000.02
paragraphs: Enclosure 1 of the Interim Department of Defense (DOD) Instruction  5000.02 identifies several information requirements which must be  documented for the milestones of the DOD acquisition process, as well as  the source of each requirement in statute, a DOD directive, instruction,  and/or manual, or a regulation. Not all of the information requirements are  applicable at every milestone and not all of the requirements equate to a  separate document.
Our review focused on documentation related to 49 statutory and policy  information requirements that is expected to be completed at Milestone  B, most of which is also expected at Milestone C. Of the 49 requirements  applicable at Milestone B, 44 are also applicable at Milestone C. Two  other information requirements are only applicable at Milestone C and not  at Milestone B. We did not include these two requirements in our review,  but they are listed below for a total of 51 requirements. The number of  documents a program has to complete will vary depending on the type of  program. For example, space programs have to complete an Orbital  Debris Mitigation Risk Report, which is not required for non-space  programs, and some requirements apply only to programs acquiring  information technology.
Description  Memorandum reflecting the milestone decision authority’s certification, prior to granting  milestone approval, as to certain program matters.
Documents the decisions and direction resulting from each milestone and other major  decision point reviews.
Agreement between the milestone decision authority and the program manager and  his/her acquisition chain of command that will be used for tracking and reporting for the  life of a program or program increment; contains schedule, performance, and cost  parameters that are the basis for satisfying an identified mission need.
Describes the overall strategy for managing an acquisition program, including the program  manager’s plan to achieve programmatic goals and summarizes the program planning  and resulting program structure.
Provides a design constraint on the product DOD will build, procure, and sustain based  upon the budgets DOD expects to have for the product over its life cycle.
Summarizes an analytical comparison of the operational effectiveness, suitability, and life- cycle cost (or total ownership cost, if applicable) of alternatives that satisfy established  capability needs.
Documents the bandwidth requirements needed to support a program and how they will  be met.
For bundled acquisitions, an analysis to determine the relative benefit to the government  among two or more alternative procurement strategies and a determination of whether  consolidation of the requirements is necessary and justified.
Describes how to redesign the way work is done to improve performance in meeting the  organization’s mission while reducing costs.
Defines authoritative, measurable, and testable parameters across one or more  increments of a material capability solution, by setting key performance parameters, key  system attributes, and additional performance attributes necessary for the acquisition  community to design and propose systems and to establish programmatic baselines.
Provides authoritative, testable capability requirements, in terms of key performance  parameters, key system attributes, and additional performance attributes, for the  production and deployment phase of an acquisition program.
For programs that acquire IT, documents compliance with the various requirements of the  Clinger-Cohen Act of 1996, subtitle III of title 40, U.S. Code.
For programs containing information technology, documents a program’s plan for ensuring  cybersecurity.
Promotes, monitors, and evaluates programs for the communication and exchange of  technological data among defense research facilities, combatant commands, and other  organizations involved in developing technological requirements for new items.
Documents the contract type selected by the milestone decision authority for a major  defense acquisition program that is consistent with the level of program risk.
Ensures that opportunities to conduct cooperative research and development projects are  considered at an early point during the formal development review process by indicating  whether or not a project similar to the one under consideration by DOD is in development  or production by another country or organization.
Description  Determination of whether the weapon system or military equipment being acquired is  necessary to enable the armed forces to fulfill the strategic and contingency plans  prepared by the Chairman of the Joint Chiefs of Staff. If the determination is positive, then  an estimate of those core capability requirements and sustaining workloads are provided,  organized by work breakdown structure and expressed in direct labor hours.
Documents the plan to prevent and control corrosion from impacting the availability, cost,  and safety of military equipment.
Cost Analysis Requirements Description  Describes formally an acquisition program for purposes of preparing both the DOD  Component Cost Estimate and the cost assessment and program evaluation independent  cost estimate.
Cost analysis to support the Development Request for Proposal Release (RFP) decision  point, which will vary depending on the program and information needed to support the  decision to release the RFP.
Cost analysis conducted by the service cost agency.
Cost position established by the DOD component that is derived from the DOD  Component Cost Estimate and the program office estimate per DOD component policy,  and signed by the DOD component Deputy Assistant Secretary for Cost and Economics.
System-specific criteria which normally track progress in important technical, schedule, or  management risk areas.
For systems that use the electromagnetic spectrum while operating in the United States  and its possessions, a certification by the National Telecommunications and Information  Administration (NTIA) that a candidate system conforms to the spectrum allocation  scheme of the United States and its possessions.
Certifies that the DOD component will fully fund the program to the DOD Component Cost  Position (CCP) in the current Future Years Defense Program (FYDP), or will commit to full  funding of the CCP during the preparation of the next FYDP, with identification of specific  offsets to address any funding shortfalls that may exist in the current FYDP.
Program description identifying contract-deliverable military equipment, non-military  equipment, and other deliverable items and plans to ensure that all deliverable equipment  requiring capitalization is serially identified and valued.
Cost estimate covering the full life-cycle cost of a program including all costs of  development, procurement, military construction, and operations and support, without  regard to funding source or management control, prepared or approved by the Director of  Cost Assessment and Program Evaluation.
Analysis of a program’s supportability planning that assesses the program office’s product  support strategy and how this strategy leads to successfully operating a system at an  affordable cost.
Analysis that the skills and knowledge, processes, facilities, and equipment necessary to  design, develop, manufacture, repair, and support a program are available and affordable.
Documents a program’s information-related needs in support of the operational and  functional capabilities that the program either delivers or contributes.
Documents a program’s strategy to identify and manage the full spectrum of intellectual  property (IP) and related issues throughout the program’s life-cycle describing, at a  minimum, how the program will assess program needs for, and acquire competitively  whenever possible, the IP deliverables and associated license rights necessary for  competitive and affordable acquisition and sustainment over the product life-cycle.
Description  Documents the program manager’s and product support manager’s plan for implementing  item unique identification (IUID) as an integral activity within MIL-STD-130N item  identification processes to identify and track applicable major end items and configuration- controlled items.
For programs that are dependent on intelligence mission data, defines specific intelligence  mission data requirements for a program and becomes more detailed as the system  progresses towards initial operational capability.
A living document describing a program manager’s approach and resources necessary to  develop and integrate sustainment requirements into the systems design, development,  testing and evaluation, fielding and operations.
Documents the quantity of the product needed to provide production representative test  articles for operational test and evaluation and efficient ramp up to full production.
Provides out-year projections of active-duty and reserve end-strength, civilian full-time  equivalents, and contractor support work-years for a major defense acquisition program.
Provides information on, among other things, whether there are commercial off-the-shelf  products that meet the defined requirements in the business case, could be modified to  meet requirements, or could meet requirements when it is necessary to modify those  requirements to a reasonable extent.
Describes the operational tasks, events, durations, frequency and environment in which  the materiel solution is expected to perform each mission and each phase of the mission.
Assessment of debris generation risk during launch, on-orbit operations, and end-of-life  disposal, and compliance with the U.S. Government Orbital Debris Mitigation Standard  Practices.
Documents the comprehensive approach to system security engineering analysis and the  associated results to ensure that programs adequately protect their technology,  components, and information throughout the acquisition process during design,  development, delivery and sustainment.
Describes the strategy for integrating environment, safety, and occupational health  considerations into the systems engineering process, how they are managed, and how  they are integrated with human systems integration efforts.
For a program that will replace another program, documents the budget estimates  required to sustain the existing system until the new system assumes responsibility; the  milestone schedule for developing and fielding the new system; and an analysis of the  ability of the existing system to maintain mission capability against relevant threats.
Communicates government requirements to prospective contractors and solicits  proposals; defines the government’s expectations in terms of the performance and  functional specifications, program planning, program process, risks, and assumptions; and  reflects the program’s plans articulated in the draft Acquisition Strategy and other draft,  key planning documents such as the Systems Engineering Plan, Program Protection Plan,  Test and Evaluation Master Plan, and Life-Cycle Sustainment Plan.
Documents stretch goals for costs that DOD expects its leaders to do their best to reach,  which are based on real opportunities, but challenging to execute.
Documents the program manager’s plan for the use of small business innovation research  and small business technology transfer program technologies and associated planned  funding profile.
Information requirement  Spectrum Supportability Risk Assessment  For spectrum-dependent systems, identifies and mitigates regulatory, technical, and  operational spectrum supportability risks.
Documents key technical risks, processes, resources, metrics, systems engineering  products, and completed and scheduled system engineering activities to help the program  manager develop, communicate, and manage the overall systems engineering approach  that guides all technical activities of a program.
Addresses projected adversary capabilities at system initial operating capability (IOC) and  IOC plus 10 years; should be system specific, to the degree that the system definition is  available at the time the assessment is being prepared.
Assesses the maturity of, and the risk associated with, critical technologies, to assist in  the determination of whether the technologies of a program have acceptable levels of risk,  based in part on the degree to which they have been demonstrated, and to support risk- mitigation plans.
Estimate of DOD’s potential liability if it terminates a contract for a program; the estimate  must include how such termination liability is likely to increase or decrease over the period  of performance.
Documents the overall structure and objectives of the test and evaluation program. It  provides a framework within which to generate detailed test and evaluation plans and  documents schedule and resource implications associated with the test and evaluation  program.
section_title: Appendix III: Value of Information Requirements Assessed by DOD Organizational Levels
section_title: Appendix IV: Comments from the Department of Defense
section_title: Appendix V: GAO Contact and Acknowledgments
subsections: section_title: GAO Contact
paragraphs: Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov.
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Cheryl K. Andrew, Assistant  Director; Don M. Springman, Analyst-in-Charge; Julie C. Hadley; Matthew  B. Lea; Brian T. Smith; Kristine R. Hassinger; Kenneth E. Patton; Laura  S. Greifner; Oziel A. Trevino; and Nathaniel O. Vaught made key  contributions to this report.