Defense Intelligence: Additional Steps Could Better Integrate Intelligence Input into DOD's Acquisition of Major Weapon Systems
Published: 2016-11-01T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DOD has reported that it expects to invest $1.6 trillion on acquiring 80 major defense acquisition programs, many of which depend on intelligence input both during the acquisition process and to effectively perform missions once deployed. The complexity of advanced weapon systems, such as the F-35, is creating increasing demand for intelligence mission data—such as radar signatures—for sensors and processes supporting warfighters.
The National Defense Authorization Act for Fiscal Year 2016 includes a provision that GAO review intelligence integration into DOD acquisitions. This report evaluates, for major defense acquisition programs, the extent to which DOD has (1) processes and procedures for certifying and training personnel providing intelligence input into acquisition programs; (2) efforts to improve processes and procedures for integrating intelligence into its acquisitions; and (3) efforts to develop tools to integrate intelligence into its acquisitions. GAO compared certification and training to relevant guidance; reviewed relevant documents to identify intelligence inputs and the provision of intelligence input into acquisition programs; and interviewed cognizant officials.
section_title: What GAO Found
paragraphs: The Department of Defense (DOD) has developed certifications and training for acquisition and intelligence personnel, but it does not have certifications for certain personnel who provide intelligence support to acquisition programs. These personnel help integrate threat information on foreign capabilities and intelligence mission data—technical intelligence such as radar signatures and geospatial mapping data—into acquisition programs. DOD uses certifications to determine that an employee has necessary education, training, and experience. The lack of certifications for personnel providing intelligence support to acquisition programs has led to the services developing varying levels of training: the Air Force certifies and requires training specific to providing intelligence support, the Army offers training that is not required, and the Navy has no formal training. Without certifications for personnel providing intelligence support to acquisition programs, DOD does not have assurance that these personnel are prepared to carry out their duties.
DOD has multiple efforts underway to improve processes and procedures for integrating intelligence into its acquisitions but does not require prioritization of intelligence mission data, which would identify those data most needed for a weapon system to perform its mission. A task force DOD created in 2015 to better integrate intelligence into acquisition programs identified the need for prioritization and proposed processes and procedures for doing so. Without department-wide requirements to prioritize intelligence mission data, new processes and procedures such as those developed by the task force may not be fully implemented and weapon systems could be deployed without the intelligence mission data they need to perform their missions.
DOD is developing two tools for integrating intelligence into major defense acquisition programs. One tool to share threat information lacks a communication plan to obtain feedback from users to better ensure its effectiveness. Without user feedback, DOD may not receive useful information to develop the tool. The other tool is for acquisition programs to communicate their intelligence needs to the intelligence community, though the services did not identify a need for the tool and there is no mechanism to fund its implementation and operation. Without assessing the need for such a tool or plans or funding for implementation and operation, DOD may be using funds unnecessarily to develop an unneeded tool.
section_title: What GAO Recommends
paragraphs: GAO recommends DOD create certifications and training for intelligence support personnel, require that intelligence mission data be prioritized, develop a communication plan for a threat information tool, and determine the need to develop another tool. DOD concurred with GAO's recommendations.

section_title: Letter
paragraphs: Congress and the Department of Defense (DOD) have long sought to  improve the acquisition of major weapon systems, an area that we first  placed on our High Risk List in 1990. DOD reported in March 2016 that it  expects to invest $1.6 trillion on the development or procurement of its  portfolio of 80 major defense acquisition programs, many of which  depend on intelligence input both during the acquisition process and  subsequently, to effectively perform their missions once deployed. The  intelligence community provides this information not only when systems  are deployed in support of the warfighter, but also as the platforms are  developed and procured as a defense acquisition program. This latter  role of the intelligence community, known as intelligence support to  acquisition, has become increasingly important as weapon systems have  become more advanced and dependent on processing data from on- board sensors to perform their mission.
The intelligence community provides support to acquisition programs in  two broad categories: threat intelligence on foreign capabilities, and  intelligence mission data. Threat intelligence identifies the capability and  strategy of an adversary’s system that could defeat or reduce the  effectiveness of a friendly military mission. Intelligence mission data refer  to technical intelligence information such as radar signatures or  geospatial mapping data that weapon systems use to carry out a mission  once deployed.
Further, according to DOD, adversaries are developing new threats more  rapidly than the department can develop new systems to counter them.  There is a growing concern that the technological superiority of the United  States over foreign adversaries has been decreasing, thereby  representing a threat to national security and military capabilities. The  confluence of these trends has created recognition within DOD of the  need for more effective collaboration across the defense acquisition and  intelligence communities. To address this issue, among others, the Under  Secretary of Defense for Acquisition, Technology, and Logistics  (USD(AT&L)) has implemented a series of reform efforts known as Better  Buying Power to improve various aspects of the defense acquisition  enterprise, such as improving collaboration between the acquisition and  intelligence communities. In the third phase of this effort, known as Better  Buying Power 3.0, USD(AT&L) identified a need for stronger partnerships  among the acquisition and intelligence communities in order to maintain  knowledge of current and potential threats.
The National Defense Authorization Act for Fiscal Year 2016 includes a  provision that we review the processes and procedures for the integration  of intelligence into the defense acquisition process. This report  evaluates, for major defense acquisition programs, the extent to which  DOD has (1) processes and procedures for certifying and training  personnel assigned to provide intelligence support to acquisition  programs; (2) efforts to improve processes and procedures for integrating  intelligence into its acquisition programs; and (3) efforts to develop new  tools for integrating intelligence into its acquisition programs. We also  collected information related to DOD’s efforts to identify opportunities for  weapon systems to collect intelligence even when unrelated to their  primary mission, which is presented in appendix I.
To address each of our three objectives, we reviewed written directives,  instructions, publications, guides, and briefs related to the role of  intelligence in acquisitions. We also interviewed officials from the  acquisition, intelligence, and requirements components of DOD, including  the offices of the USD(AT&L) and the Under Secretary of Defense for  Intelligence (USD(I)); the Joint Staff; the Army, Navy, Air Force, and  Marine Corps; and the Defense Intelligence Agency (DIA). We  interviewed members of the Acquisition Intelligence Requirements  Executive Steering Group and Task Force to assess their efforts to better  integrate intelligence into DOD acquisitions. We also interviewed officials  from the offices of the Director of National Intelligence; Cost Assessment  and Program Evaluation; Director of Operational Test and Evaluation; and  USD(AT&L) Office of Performance Assessments and Root Cause  Analyses.
To address the first objective we reviewed DOD guidance governing the  management of intelligence and acquisition personnel. We interviewed  officials from USD(AT&L), USD(I), the Joint Staff, and the Army, Navy, Air  Force, and Marine Corps intelligence and acquisition communities who  participate in the development of policy and management of personnel  regarding staffing, qualifications, certification, and training of personnel  providing intelligence support to acquisition programs. We also  interviewed and received written responses from Defense Acquisition  University officials regarding changes to the acquisition curriculum that  included additional intelligence material. We analyzed the certifications  and qualifications established by DOD for implementing the Defense  Acquisition Workforce Improvement Act; DOD guidance for acquisition  personnel; and certifications and qualifications for intelligence personnel  under USD(I) guidance, to determine whether any certifications apply to  personnel who provide intelligence support to acquisition programs.
To address the second objective we interviewed or requested information  from acquisition program offices, as well as from staff who provide  intelligence support to those programs. We selected a non-generalizable  sample of six Acquisition Category I programs from the four services— including two each from the Navy and Air Force, and one each from the  Army and Marine Corps—to obtain an understanding of how intelligence  is integrated into acquisition programs. We discussed the questions  orally or received written responses from each program. While the  responses we obtained are not generalizable to all major defense  acquisition programs, the information learned from program officials  provided context and important insights for our understanding of the  interaction between acquisition and intelligence personnel. To determine  current intelligence inputs and processes for major defense acquisition  programs, we reviewed both department-wide and service-level guidance,  including Department of Defense Instruction 5000.02, Operation of the  Defense Acquisition System, and analyzed the guidance to determine  where DOD guidance indicated that intelligence inputs into acquisition  programs are to occur. Due to the evolving nature of the role of  intelligence in acquisitions, we also sent a set of structured questions to  officials from the Office of USD(I), Joint Staff, DIA, and the intelligence  organizations of the services to confirm the key intelligence inputs into  major defense acquisitions as of June 1, 2016.
To address the third objective, we identified two tools that DOD is  currently developing through discussions with Acquisition Intelligence  Requirements Task Force officials. We verified that these tools were in  development through interviews with officials responsible for oversight of  acquisitions and intelligence, including officials at USD(AT&L), USD(I),  and DIA. We interviewed DOD officials and viewed demonstrations of the  developmental versions of these tools. We compared the developmental  plans and information provided to us by DIA and Performance  Assessments and Root Cause Analyses officials against standards  developed by professional organizations, such as the Project  Management Institute’s A Guide to the Project Management Body of  Knowledge, federal standards for internal controls, and key practices  for collaboration among federal agencies. Further details of our  objectives, scope, and methodology are presented in appendix II.
We conducted this performance audit from December 2015 to November  2016 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: DOD manages the acquisition of weapon systems through the Defense  Acquisition System, which is an event-based process. Acquisition  programs proceed through a series of milestone reviews and other  decision points that may authorize entry into the next program phase (see  figure 1). Based upon DOD’s acquisition-related guidance, for major  defense acquisition programs that begin during the materiel solution  analysis phase, intelligence inputs into the acquisition process are  expected to be provided prior to the Milestone A review, the point at  which approval is sought to proceed to the next phase in the process.  Further, most of the intelligence inputs are to be verified or updated at  points prior to the Milestone B decision and again prior to the Milestone C  decision, the point at which approval is sought in order to progress to the  production and deployment phase. We describe the intelligence inputs  and key DOD guidance for providing intelligence support to acquisition  programs in appendix III.
The USD(AT&L) is responsible for acquisition policy and oversight, and  as the Defense Acquisition Executive has responsibility for supervising  the Defense Acquisition System. The milestone decision authority is the  designated individual with overall responsibility for a program and has the  authority to approve its progression to the next phase of the acquisition  process. The milestone decision authority is accountable for cost,  schedule, and performance reporting. The service acquisition  communities are led by Service Acquisition Executives who are assistant  secretaries within their respective military departments. For example, the  Assistant Secretary of the Army (Acquisition, Logistics, and Technology)  serves as the Army acquisition executive, while the Assistant Secretary of  the Navy (Research, Development, & Acquisition) serves as the Navy  acquisition executive. The program manager is the designated individual  with responsibility for individual acquisition programs who has the  authority to accomplish that program’s development, production, and  sustainment objectives to meet the user’s operational needs, and is  accountable for cost, schedule, and performance reporting to the  milestone decision authority.
At the Chairman of the Joint Chiefs of Staff level, the J-8 Directorate  provides support to the Joint Staff for evaluating and developing force  structure requirements, and its director serves as Secretary of the Joint  Requirements Oversight Council and as Chairman of the Joint  Capabilities Board. In these capacities, the director orchestrates Joint  Staff support of the capabilities development process through the Joint  Capabilities Integration and Development System. One of the J-8  Directorate’s objectives is to provide early capability development  guidance to the services. The services have different organizational  structures that define their respective requirements communities. For  example, according to the Air Force, via its Major Commands, the Air  Force has personnel responsible for 12 capability portfolios such as Air  Superiority and Global Precision Attack that manage Air Force capability  requirements.
According to DOD, the speed of technical innovation and the complexity  of advanced weapon systems, such as the F-35, are creating an  increasing demand for specialized intelligence mission data to provide  information for sensors and automated processes supporting the  warfighter. There are several types of intelligence mission data, each  used by weapon systems in different ways, including signatures,  electronic warfare integrated reprogramming data, characteristics and  performance, order of battle, and geospatial intelligence (see figure 2).  Signatures are distinct, repeating characteristics, such as radio  frequencies or acoustic characteristics, which are associated with a  particular type of equipment, materiel, activity, individual, or event. For  example, a weapon system may associate a specific signature with an  enemy system, and identify it as such. Electronic warfare integrated  reprogramming data also describe radio frequencies, and are typically  used to attack or control other electronic systems—for example, jamming  enemy radar capabilities. Characteristics and performance data describe  the abilities of a particular foreign military system, while order of battle  describes the strength and structure of armed forces. These can assist a  weapon system and its operator to prioritize and determine appropriate  actions against the enemy.
The title, specific duties, and organizational structure for the personnel  providing intelligence support to acquisition programs vary by service  (see table 1). Personnel who provide intelligence support to acquisition  programs may also coordinate, and in some cases create, the completion  of key intelligence products that accompany the acquisition process  through documented processes such as Threat Steering Groups, which  assemble intelligence and acquisition representatives with knowledge of  systems that are specific to the acquisition program.
In the Army, during the acquisition lifecycle, the threat integration staff  officers assigned to Army Intelligence coordinate intelligence support to  acquisition programs through the Threat Steering Group, in which  Training and Doctrine Command threat managers and Army Materiel  Command foreign intelligence officers participate. Threat assessments  before Milestone B are generally managed by Training and Doctrine  Command threat managers. Threat assessments from Milestone B and  beyond are typically managed by Army Materiel Command foreign  intelligence officers.
In the Navy, intelligence support to acquisition programs is provided by  intelligence personnel in the Office of Naval Intelligence, which is  responsible for the production and validation of intelligence inputs to Navy  acquisition programs. The acquisition programs are supported by  scientific and technical intelligence liaison officers who are hired and  funded by the Navy entities responsible for management of assigned  acquisition programs, called system commands, and are responsible for  coordinating between the system command and the intelligence  community. For example, the scientific and technical intelligence liaison  officer is responsible for requesting the production and validation of  intelligence inputs such as threat assessments, which are used to obtain  the threat intelligence required to inform acquisition cost, schedule, and  performance decision making by program managers.
For programs that are managed by the Marine Corps acquisition agencies  (Marine Corps System Command and Program Executive Officer Land  Systems), intelligence support is provided by military and civilian  intelligence analysts at Marine Corps Intelligence Activity, the service  intelligence center. Marine Corps System Command’s intelligence  support is coordinated by a scientific and technical intelligence liaison  officer. Marine Corps-funded programs at other Navy system commands  follow the intelligence support processes for the hosting organization. For  example, the Marine Corps version of the F-35 or helicopter acquisitions  would both follow processes for intelligence support to acquisition  programs used by Naval Air Systems Command.
The Air Force materiel commands (Air Force Space Command and Air  Force Materiel Command) use acquisition intelligence specialists to  support acquisition programs identified as intelligence sensitive. These  specialists, along with intelligence analysts at the National Air and Space  Intelligence Center, provide intelligence products and input based on their  individual levels of experience and training.
section_title: DOD Has Processes and Procedures for Certifying Both Intelligence and Acquisition Personnel but Not for Personnel Providing Intelligence Support to Acquisition Programs, and Services’ Training Varies
paragraphs: DOD has processes and procedures for the certification of both  intelligence and acquisition personnel, but it has not established  certifications for personnel providing intelligence support to acquisition  programs. Though DOD has not developed certifications specific to  personnel who provide intelligence support to acquisition programs, the  Air Force and the Army have each developed certifications for these  personnel. In the absence of department-wide certifications, the services  have developed varying levels of training for personnel providing  intelligence support to acquisition programs, and this training may not be  specific to providing intelligence support to acquisition programs.
subsections: section_title: DOD Has Certifications for Intelligence and Acquisition Personnel and the Air Force and Army Have Developed Certifications for Personnel Who Provide Intelligence Support to Acquisition Programs
paragraphs: Neither USD(I) certifications for the defense intelligence workforce nor  USD(AT&L) certifications for the defense acquisition workforce include a  certification specific to those personnel providing intelligence support to  acquisition programs. The Defense Acquisition Workforce Improvement  Act generally requires DOD to establish policies and procedures for the  management of DOD’s acquisition workforce, including education,  training, and career development. USD(AT&L) subsequently organized  certain acquisition-related positions into 14 career fields and established  a certification process by which DOD components determine that  employees have met standard requirements for education, training, and  experience for each field. However, personnel who provide intelligence  support to acquisition programs are not included in the 14 career fields  with established certifications. According to service officials, acquisition  certifications for personnel who provide intelligence support to acquisition  programs have not been developed because there is no career field for  intelligence support to acquisition. As a result, personnel providing  intelligence support to acquisition programs are not required to obtain  certification for any of the acquisition-related career fields. Officials from  the Air Force stated that the lack of certification has resulted in critical skill  gaps for personnel providing intelligence support to acquisition programs.
Similarly, USD(I) is responsible for establishing a department-level  certification program for the defense intelligence workforce. As a result,  certifications for 15 different intelligence disciplines have been developed,  such as geospatial intelligence and collection management, and several  other certifications are currently being developed for disciplines such as  all-source analysis—an intelligence activity that involves the integration,  evaluation, and interpretation of information from all available data  sources and types. Intelligence officials stated that personnel providing  intelligence support to acquisition programs may become eligible for  fundamental intelligence certifications, such as all-source analysis  certification. However, these officials stated that the certification is  designed to certify fundamental competencies for all intelligence analysts,  and is not specific to providing intelligence support to acquisition  programs.
Though DOD has not developed certifications specific to personnel who  provide intelligence support to acquisition programs, the Air Force and the  Army have each developed certifications for these personnel. The Air  Force has established a certification for personnel who provide  intelligence support to acquisition programs via both service-wide  guidance and guidance from organizations involved in acquisition, such  as Air Force Materiel Command. The Air Force requires that, for initial  certification, personnel assigned to positions providing intelligence  support to acquisition programs must complete certain training, including  Air Force and Defense Acquisition University classes, and have 1 year of  experience in a designated “acquisition intelligence” position.  Additionally, individual acquisition organizations, such as the Air Force  Life Cycle Management Center, require additional training and also  require that intelligence managers certify that personnel providing  intelligence support to acquisition programs have completed the required  training, met the experience requirements, and can complete a list of  unique tasks specific to the performance of their individual job.
The Army developed an optional certification for civilian personnel in  intelligence positions, including intelligence support to acquisition  programs. A 2001 training plan describes a process for individuals to  document competency in different specialty areas based on the job duties  and seniority of the position. The plan shows that the intelligence support  to acquisition specialty requires competency in areas such as threat  intelligence and technical knowledge of acquisition organizations.  Personnel may achieve these competencies through any combination of  previous experience, classroom, and on-the-job training. Subsequently,  they may request an optional certification from their command  organization if a supervisor certifies the individual’s qualifications.
section_title: The Services and USD(AT&L) Have Developed Some Training for Personnel Providing Intelligence Support to Acquisition Programs, but Cannot Ensure Personnel Are Trained to Carry Out Duties
paragraphs: The services have developed varying levels of training for the personnel  who provide intelligence support to acquisition programs in the absence  of certifications required by DOD for these personnel. Air Force officials  stated that training for personnel providing intelligence support to  acquisition programs is accomplished through their certification process,  which requires these personnel to complete a series of classroom and on- the-job training units, including Defense Acquisition University classes in  acquisition management fundamentals, among others, and an Air Force  4-day training course called the Acquisition Intelligence Formal Training  Unit. Individual acquisition organizations such as the Air Force Life Cycle  Management Center require additional training, including courses on  intelligence acquisition life-cycle management and the Joint Capabilities  Integration and Development System; a review of key acquisition  documents; and on-the-job training based on unit-specific missions. Air  Force officials also stated that experience as an active-duty intelligence  officer and an acquisition program manager all also provided training for  the position.
Army officials stated that threat managers and foreign intelligence  officers, two of the three groups that provide intelligence support to  acquisition programs, are required to take several courses through the  Defense Acquisition University and Defense Security Service. In addition,  the threat intelligence branch of Army Intelligence has an annual training  course that includes training in subjects specific to providing intelligence  support to acquisition programs, such as critical threats and technology  protection. However, according to Army officials, Army personnel  providing intelligence support to acquisitions are not required to take this  course. An Army intelligence official stated that the course is optional  because of a lack of travel and training funds.
The Navy and Marine Corps have identified and required different levels  of training relevant for their personnel. Navy officials stated that, as of  June 2016, there was no formal training across the department for  personnel providing intelligence support to acquisition programs, although  some Navy organizations have developed training policies specific to their  organizations. For example, according to Navy officials, Naval Air  Systems Command, the Navy acquisition organization generally  responsible for naval aircraft, weapons, and systems, has a training  program for its scientific and technical intelligence liaison officers. This  training includes intelligence community and Defense Acquisition  University courses, computer-based training, and a certification exam,  which requires its liaison officers to attend the Air Force’s Acquisition  Intelligence Formal Training Unit. According to Navy officials, personnel  providing intelligence support to acquisition programs at other naval  system commands, such as those responsible for sea and space  systems, receive primarily ad hoc and on-the-job training.
Marine Corps officials stated that, as of June 2016, personnel who  provide intelligence support to acquisition programs were required to take  an online Defense Acquisition University course on acquisition  management fundamentals that is not specific to providing intelligence  support to acquisition programs, in addition to on-the-job training in order  to perform their job duties. These officials also stated that in 2016  personnel were required to attend a version of the Air Force Acquisition  Intelligence Formal Training Unit, and that although the Marine Corps is  exploring the use of the Army Intelligence training course, it is not  required for personnel to attend the training.
USD(AT&L), working with the Defense Acquisition University, established  training related to the integration of intelligence and acquisition. In May  and June 2015, Defense Acquisition University increased the integration  of intelligence and acquisition in its curriculum. For example, the  university added a case study regarding critical intelligence parameters to  the program manager’s course, and it also added discussion topics about  the need for intelligence in acquisition programs to several courses  intended for acquisition executives and senior officials. Both entry-level  and advanced courses were modified to include content on the  relationship between intelligence and acquisition organizations. For  example, officials stated that they invited a speaker from the Joint Staff to  an advanced course to speak about the relationship between intelligence  and acquisition.
While this training is intended to address the identified need for greater  intelligence training for acquisition personnel, the training may not be  accessible to personnel providing intelligence support to acquisition  programs. Service intelligence officials stated that because positions for  providing intelligence support to acquisition programs are not designated  as acquisition-related for the purposes of Defense Acquisition Workforce  Improvement Act certification, some courses are available to these  personnel only on a space-available basis. Some other courses, such as  those for program managers, require Defense Acquisition Workforce  Improvement Act certification in designated career fields as a pre- requisite. As described above, DOD has not required certification for  personnel providing intelligence support to acquisition programs. As a  result, these personnel may be unable to access these courses.
Without requiring certifications for personnel who provide intelligence  support to acquisition programs, DOD has no assurance that these  personnel are qualified and prepared to carry out their duties. The  department has established certifications for both acquisition and  intelligence positions in order to ensure that those respective workforces  are qualified to carry out their duties. Key principles for the management  of federal employees state that agencies should develop training  strategies and tools that, among other things, can be aligned to improve  the critical skills needed in their workforce. We previously found that  when intelligence training is not fully implemented or required, programs  and organizations may be unable to fully succeed in their goals. The  DOD Inspector General has also found that the lack of common training  standards has resulted in difficulties for personnel in performing common  tasks and in a critical skills gap across military intelligence services and  agencies. While all four services have established or identified training  for personnel providing intelligence support to acquisition programs,  without department-wide required certifications that include training  standards, there may be inconsistent levels of expertise and skill among  these personnel. Without a certification process that includes required  training for personnel who provide intelligence support to acquisition  programs, DOD may not be able to ensure that all personnel who provide  intelligence support to acquisition programs are familiar with and able to  provide intelligence inputs to their assigned acquisition programs.
section_title: DOD Has Multiple Efforts Underway to Improve Processes and Procedures for Integrating Intelligence into Major Defense Acquisition Programs but Faces Challenges Prioritizing Intelligence Mission Data Needs
paragraphs: As of July 2016, DOD had multiple efforts underway to improve  processes and procedures for integrating intelligence into major defense  acquisition programs. For example, USD(AT&L) had identified several  intelligence-related tasks in its Better Buying Power 3.0 initiative. Further,  USD(AT&L), USD(I), and the Joint Staff had created an executive  steering group and a task force—the Acquisition Intelligence  Requirements Task Force—to improve the integration of intelligence into  major defense acquisition programs. This task force has identified the  need for intelligence mission data to be prioritized, but DOD has not  required such prioritization.
subsections: section_title: USD(AT&L) Has Identified Several Intelligence- Related Tasks to Address Challenges Integrating Intelligence into Acquisitions
paragraphs: In order to increase the productivity, efficiency, and effectiveness of  DOD’s acquisition, technology, and logistics efforts, USD(AT&L) issued  the Better Buying Power 3.0 initiative in January 2015. This initiative  contains nine tasks related to integrating intelligence into acquisitions,  which are described in table 2.
Among the nine tasks, USD(AT&L) describes the use of critical  intelligence parameters as a key aspect of the linkage among the  acquisition, intelligence, and requirements communities. Critical  intelligence parameter thresholds, if breached, indicate an adversary’s  potential ability to substantially reduce the performance or even defeat  the capability of the weapon system undergoing acquisition. The  intelligence community monitors foreign threat capabilities and informs  the acquisition community of a breach, which triggers a review process to  resolve or mitigate the breach.
Other intelligence-related tasks under Better Buying Power 3.0 include  direction to the Assistant Secretary of Defense for Acquisitions to work  with the Office of the USD(I) to review DOD Directive 5250.01,  Management of Intelligence Mission Data (IMD) in DOD Acquisition. As of  July 2016, this review is being facilitated by the Acquisition Intelligence  Requirements Task Force, described below, which is coordinating a  revised draft among stakeholder entities.
section_title: USD(AT&L), USD(I), and Joint Staff Created an Executive Steering Group and Task Force to Improve the Integration of Intelligence into Acquisition Programs
paragraphs: DOD created an executive steering group and task force to better  integrate intelligence into acquisition programs. On December 4, 2015,  the offices of USD(AT&L) and USD(I), along with the Joint Staff, created  the Acquisition Intelligence Requirements Executive Steering Group and  the Acquisition Intelligence Requirements Task Force through a joint  memorandum to better integrate, coordinate, and prioritize intelligence  processes and procedures for providing intelligence support to acquisition  programs. The steering group is co-chaired by senior level members of  the offices of USD(AT&L) and USD(I), and the Joint Staff, and it is  composed of representatives from the Office of the Director of National  Intelligence, Director of Operational Test and Evaluation, Office of Cost  Assessment and Program Evaluation, service acquisition executives,  military service intelligence staffs, and DIA, among other stakeholders.
The memorandum states that the steering group replaces the Intelligence  Mission Data Oversight Board and the Intelligence Mission Data Senior  Steering Group that were previously established in DOD Directive  5250.01, issued in January 2013. A DIA official explained that the  Intelligence Mission Data Senior Steering Group never met and that the  Acquisition Intelligence Requirements Task Force was created to address  intelligence support to acquisition, including intelligence mission data  issues.
Both the executive steering group and the task force began to meet prior  to their formal creation in December 2015. The task force initially met in  October 2015 and subsequent to January 2016 has generally held weekly  meetings, while the executive steering group initially met in August 2015  and met quarterly subsequent to December 2015. Since February 2016, a  senior executive service-level director has led the task force composed of  O-6 level representatives from the organizations that form the steering  group. According to task force officials, early efforts of the task force  included engaging major defense acquisition programs that the task force  identified as intelligence mission data-dependent in order to identify and  summarize intelligence supportability issues and raise them to decision  makers prior to milestone reviews for the acquisition programs.
section_title: DOD’s Acquisition Intelligence Requirements Task Force Identified the Need for, but DOD Has Not Required Prioritization of, Intelligence Mission Data
paragraphs: Officials from USD(AT&L), USD(I), and Joint Staff stated that while there  have previously been other weapon systems with intelligence mission  data shortfalls, such as the F-22 and F-18G, the F-35’s greater reliance  on intelligence mission data and concerns regarding the service  intelligence centers’ ability to produce the needed data brought the  problem to the forefront. For example, DOD reported in 2013 that the  initial release of intelligence mission data requirements for the F-35 in  2008 presented a unique challenge with regard to the amount and  breadth of intelligence requirements for the intelligence community, and  for the service intelligence centers specifically. These officials stated  that the main impetus for creating the Acquisition Intelligence  Requirements Executive Steering Group and the Acquisition Intelligence  Requirements Task Force concerned the shortfall in providing intelligence  mission data to the F-35 program—data needed for the F-35 to perform  its mission once it became an operational weapon system. Task force  officials stated that prioritizing intelligence mission data will ensure that  the data provided are sufficient to meet the requirements of advanced  weapon systems, such as the F-35.
DOD has processes and procedures related to intelligence mission data,  such as those in DOD Instruction 5000.02 and DOD Directive 5250.01,  but they do not require prioritization of the data. For example, DOD  guidance requires DOD’s intelligence mission data-dependent acquisition  programs to develop a Lifecycle Mission Data Plan to identify anticipated  intelligence mission data needs over the life of a weapon system, from  program start through the life-cycle of the program to disposal. However,  the plan categorizes the intelligence mission data needs by means of a  spectrum arranged by data availability—that is, showing data ranging  from those that are most available to those that are least available. That  presentation of information does not convey a prioritization of what the  weapon system most needs to perform its mission.
DOD Directive 5250.01 directs DIA to establish the Intelligence Mission  Data Center, which is to serve as the focal point for intelligence mission  data development, production, and sharing, but it does not assign the  agency the role of prioritizing intelligence mission data needs. Task force  officials stated that a DIA working group for intelligence mission data was  created to oversee and coordinate intelligence mission data production  across the defense intelligence enterprise, and that the Intelligence  Mission Data Center will support the working group by facilitating the  discovery and sharing of existing intelligence mission data. However,  although it may be helpful in preventing the duplication of efforts in the  collection of intelligence mission data, the Intelligence Mission Data  Center’s work does not constitute a means for prioritizing mission data by  need for individual acquisition programs.
Officials from USD(AT&L), USD(I), Joint Staff, and the task force  described a lack of prioritization at multiple levels, to include within the  individual acquisition programs, as well as at the service and department  levels. Officials from USD(AT&L), USD(I), and Joint Staff, as well as  service-level officials on the task force, stated that there were currently no  required processes or procedures for prioritizing intelligence mission data  needs at any of these levels. For example, at the acquisition program  level, an F-22 will have different intelligence mission data needs and  priorities from those of a Navy submarine. At the service level, each  service will have intelligence mission data needs based on the types of  weapon systems it is developing and has already deployed. At the  department level, the Air Force and the Navy may have similar  intelligence mission data needs for their respective fighter aircraft, but the  Army’s intelligence mission data needs will likely differ greatly from those  of its sister services based on the respective threats each faces.
As of July 2016, no requirements existed within DOD guidance to  prioritize intelligence mission data, though there were efforts underway in  2016 by the task force and within the Air Force to develop processes and  procedures for intelligence mission data prioritization. According to task  force and Air Force officials, the Acquisition Intelligence Requirements  Task Force worked in parallel with an Air Force effort to identify potential  processes to prioritize intelligence mission data at the acquisition program  and service levels, respectively. These officials presented proposals in  June and July 2016 for potential processes and procedures to prioritize  intelligence mission data at the acquisition program level. They also  described how prioritization at the service and department levels may be  accomplished, as shown below:
Acquisition program prioritization: The task force proposal would  involve prioritization of intelligence mission data into the following four  levels of impact on the acquisition program’s capabilities were the  data not acquired or unavailable:  1.  Level I denotes unacceptable degradation: intelligence mission  data requirements that, if not satisfied, would result in  unacceptable mission task degradation with no work-around  possible;  2.  Level II denotes significant degradation: intelligence mission  requirements that, if not satisfied, would result in a significant  mission task degradation that is unacceptable to the operator but  for which a work-around is available, acceptable to the operator,  and must be applied;  3.  Level III denotes partial degradation: intelligence mission data  requirements that, if not satisfied, would result in a partial or  minimal degradation that is acceptable to the operator and for  which a work-around is optional; and,  4.  Level IV denotes little to no impact: intelligence mission data  requirements that, if not satisfied, would result in little to no  degradation to the mission.
Service prioritization: Air Force officials described an effort undertaken  in May 2016 to apply the individual program approach using levels 1  to 4 described above to categorize 150 intelligence mission data  needs at the service level. Task force officials stated that the Air Force  effort undertaken in fiscal year 2016 would use a cost-capability  approach to better inform the fiscal year 2017 service-wide  prioritization effort.
Department prioritization: Task force officials indicated that  implementing prioritization at the individual program and service levels  would be required prior to developing an enterprise-wide capability to  prioritize intelligence mission data. Officials from USD(AT&L) stated  that the enterprise-wide prioritization of intelligence mission data could  also be informed by efforts related to developing Integrated DOD  Intelligence Priorities. Furthermore, officials from the DIA stated that  there is a lack of coordination regarding how the service intelligence  centers conduct their business, and that the centers currently were  not prioritizing, verifying, or balancing the work related to producing  intelligence mission data. Task force officials stated that DIA was  developing an intelligence mission data production prioritization  process that would respond to enterprise-wide intelligence mission  data priorities.
Though DOD has taken efforts to identify and develop processes and  procedures to prioritize intelligence mission data, previous efforts have  not succeeded in implementing intelligence mission data prioritization.  Federal internal control standards state that management should  establish an organizational structure, assign responsibility, and delegate  authority to achieve objectives. Per the joint memorandum that  established the Acquisition Intelligence Requirements Executive Steering  Group, the steering group and associated task force were created to  integrate, coordinate, and prioritize intelligence support functions and  processes. However, previous efforts have not succeeded in  implementing a system to prioritize these data at any level. For example,  the Intelligence Mission Data Senior Steering Group never met, and the  DIA’s standing working group efforts may lead to a prevention of  duplication of intelligence mission data production efforts, but would not  lead to prioritization of the data. The Acquisition Intelligence  Requirements Task Force was created to address intelligence support to  acquisition, including intelligence mission data issues, and it has  developed some proposed processes and procedures for prioritization of  intelligence mission data. Without specific DOD guidance requiring  intelligence mission data prioritization, new processes and procedures  such as those developed by the task force and the services may not be  fully implemented. With no required process to prioritize intelligence  mission data, the intelligence community may continue to process  requests for intelligence mission data as they are received, and thus  weapon systems may not have the intelligence mission data they need to  successfully perform their missions once operational.
section_title: DOD Is Developing New Tools for Integrating Intelligence into Acquisition Programs but Faces Challenges in Development and Implementation
paragraphs: As of July 2016, DOD was developing new tools to better integrate  intelligence into acquisition programs. DIA was developing the Validated  Online Lifecycle Threat, an online tool to provide threat information to  acquisition programs in a more timely and effective manner than the  current manually generated process in use. However, DIA had not  effectively communicated with stakeholders about the tool or sought  feedback from its intended users. Separately, officials from Performance  Assessments and Root Cause Analyses, an office within USD(AT&L),  were developing a tool for the acquisition community to communicate  intelligence needs from individual acquisition programs to the intelligence  community. However, intelligence community users had not expressed a  need or defined requirements for this tool. If the tool does not meet the  user’s needs, or will not be used, moving forward with its development  could use funds unnecessarily.
subsections: section_title: DOD Is Developing the Validated Online Lifecycle Threat Tool but Has Not Effectively Communicated with Stakeholders and Intended Users of the Tool.
paragraphs: DOD began developing a new tool in fiscal year 2015 to better report  threat information from the intelligence community to acquisition  programs, but it has not effectively communicated with stakeholders  about the tool or sought feedback from its intended users. As reported by  DIA and described in USD(AT&L)’s Better Buying Power 3.0 initiative, the  Validated Online Lifecycle Threat is a new tool that DIA began developing  in fiscal year 2015 to develop and report threat information to acquisition  programs. DOD guidance describes the System Threat Assessment  Report as the primary threat document for supporting the Defense  Acquisition Board’s milestone reviews. DOD officials described the  System Threat Assessment Report as a primary intelligence input into  major DOD acquisitions.
DOD officials described challenges regarding the timeliness and  usefulness of the System Threat Assessment Report. Specifically,  officials from USD(AT&L), Joint Staff, and DIA, as well as service  intelligence officials, stated that the System Threat Assessment Reports  historically arrived at acquisition program offices late, not until after the  requirements for a new weapon system had been identified and approved  as part of the Joint Capabilities Integration Development System process  well after the designing of the weapon system had begun. Furthermore,  officials from DIA, USD(AT&L), and the service intelligence and  acquisition communities stated that these reports are often several  hundred pages in length, take as long as 9 months to produce, and are  not substantively used by acquisition program managers. According to  these officials, as well as officials from the Acquisition Intelligence  Requirements Task Force, program managers simply used System  Threat Assessment Reports to check off a box on a list of required  documents for the next acquisition milestone decision meeting.  Specifically, Air Force officials from the Joint Surveillance Target Attack  Radar System Recapitalization acquisition program stated that the  system threat assessment reports were not usable because they did not  contain the level of relevance and specificity needed by the acquisition  program, and because they were too long, fragmented, and difficult to  navigate. Lastly, DIA officials stated they had determined that as much as  80 percent of all System Threat Assessment Reports are repetitive with  each other and are not program specific.
According to DIA, the Validated Online Lifecycle Threat is a planned  system-specific threat tool created by selecting relevant modules from a  library of threat information. DIA and officials from USD(AT&L) described  the planned threat library as consisting of dynamic modules based upon  threat category, such as fighter aircraft, that would be updated by the  analyst at the services’ intelligence centers with new threat information as  it is produced within the intelligence community. DIA officials reported that  as part of a broader piloting effort the agency completed in 2015, they  were able to develop a Validated Online Lifecycle Threat in 3 months for  the Joint Surveillance Target Attack Radar System recapitalization  program. According to DIA officials, the agency will have spent nearly  $2.5 million from fiscal year 2015 through the end of fiscal year 2016 to  begin developing the Validated Online Lifecycle Threat and the  associated threat library, and it plans to have the tool completed by the  end of fiscal year 2017.
DIA officials reported that they had not effectively “marketed” the  Validated Online Lifecycle Threat tool to its intended stakeholders and  users. While Marine Corps officials stated that the threat modules can be  updated in a shorter timeframe than a System Threat Assessment  Report, officials from the Navy and the Army did not know that DIA  intended the Validated Online Lifecycle Threat to be a dynamic system,  and they stated that they believed the planned tool to be a static, online  version of the System Threat Assessment Report. For example, Army  officials stated that the tool may be less useful than its predecessor  because it is composed of static modules that may not provide the same  level of individualized detail. Though some Navy officials who provide  intelligence support to acquisition programs indicated that they had  received briefs and other information about the new tool, other Navy  officials expressed concerns that the Validated Online Lifecycle Threat  would not be customizable to programs and could, similar to the System  Threat Assessment Report, include extensive information that was not  program-specific and could therefore be as inefficient as the System  Threat Assessment Reports. Air Force officials from the Joint Surveillance  Target Attack Radar System Recapitalization acquisition program stated  that the Validated Online Threat Assessment tool was intended to  alleviate resource constraints in the intelligence community, but that the  tool was being implemented without input from the acquisition community  and individual program management offices.
The Project Management Institute’s A Guide to the Project Management  Body of Knowledge states that managing stakeholders’ engagement  helps to increase the probability of project success by ensuring that  stakeholders clearly understand the project goals, objectives, benefits,  and risks. This enables stakeholders to be active supporters of the  project and to help guide activities and project decisions. Federal internal  control standards state that management should communicate quality  information externally so that external parties can help the entity achieve  its objectives and address related risks. We found that some potential  users of the Validated Online Lifecycle Threat report have not received  information regarding the intended capabilities of the new system  because DIA has not effectively communicated information about the tool  with stakeholders and intended users. A communication plan would  include processes for communicating the intended capabilities of the  Validated Online Lifecycle Threat tool to stakeholders such as  USD(AT&L) and USD(I) and users such as personnel who provide  intelligence support to acquisition programs. Without effectively  communicating such information to potential users, DIA may not receive  useful feedback as it develops the tool, and concerns regarding  timeliness, usability, and redundancy may not be effectively addressed.
section_title: Users Have Not Expressed a Need or Defined Requirements for a Tool for Acquisition Programs to Communicate Their Intelligence Needs to the Intelligence Community
paragraphs: Officials from Performance Assessments and Root Cause Analyses, an  office within USD(AT&L), are developing a tool for communication of  intelligence needs from acquisition programs to the intelligence  community, but intended users have not expressed a need or defined  requirements for the tool. The office is responsible for conducting root- cause analyses of acquisition programs that encounter Nunn-McCurdy  breaches, among other things. According to these officials, the Assistant  Secretary for Defense (Acquisitions) requested that the office perform an  analysis of the root causes of challenges faced by the integration of  intelligence into acquisitions. This analysis identified issues with threat  intelligence and intelligence mission data and resulted in pilot projects of  new tools such as the Validated Online Lifecycle Threat.
According to Performance Assessments and Root Cause Analyses  officials, acquisition programs request intelligence both through informal  means, such as conversations and emails, and through formal means via  information systems, such as the Community On-Line Intelligence System  for End Users and Managers. These officials reported that the formal  requests for intelligence often contain vague or inaccurate information  and do not allow the intelligence community to prioritize or fulfill requests  efficiently. Officials told us that to resolve this issue they are developing  an online tool called the Acquisition Intelligence Support Assessment that  would allow acquisition personnel to communicate intelligence needs to  the intelligence community over an online system.
According to Performance Assessments and Root Cause Analyses  officials, personnel providing intelligence support to acquisition programs  will be able to access the online tool and determine whether particular  threat intelligence is currently available. If it is not, requests can be made  via the tool to the intelligence community, which would then use the  tracking capabilities planned for the tool to monitor requests from multiple  programs, and assign staff and resources as necessary. Performance  Assessments and Root Cause Analyses officials stated that this tool  could also be useful for personnel with limited knowledge or familiarity  with acquisition programs due to time and resource constraints. They also  stated that it would be useful for the intelligence community to manage  and prioritize intelligence requests from the acquisition community.
Officials from the office of Performance Assessments and Root Cause  Analyses stated that they were independently developing the tool for the  acquisition and intelligence communities before integrating it into existing  processes. These officials stated that they chose this approach after  conducting their root cause analysis and identifying challenges related to  integrating intelligence into acquisition programs. DOD awarded contracts  in August and December 2015 for the initial analysis and for commencing  development of the communication tool for a cost of approximately $1.1  million. Officials stated that they expected to spend in total about $1.2  million, sourced from available operational funding within the Office of  Performance Assessments and Root Cause Analyses.
While Performance Assessments and Root Cause Analyses has funded  the development of the Acquisition Intelligence Support Assessment tool,  these officials stated that there is currently no mechanism to fund future  implementation and operation of the tool once fully developed, and they  estimated that the system will cost $3 million to $5 million per year to  operate. These officials reported that another office must be tasked to  oversee the implementation of the tool, and suggested that the  Acquisition Intelligence Requirements Task Force or a Joint Staff office  might assume responsibility for the system. Officials from the task force  have recommended that the task force evaluate whether the Acquisition  Intelligence Support Assessment tool should be used or merged with  existing tools, but no decision regarding the planned implementation or  operation of the tool had been made as of July 2016.
Performance Assessments and Root Cause Analyses officials described  several steps they had taken to introduce the tool to potential  stakeholders, including holding demonstration events, working groups,  and briefings. Specifically, these officials stated that they had introduced  the Acquisition Intelligence Support Assessment tool to acquisition and  intelligence stakeholders in October 2015 and June 2016, but service  officials who are potential users of the tool told us that they had not  identified a need for the tool. Air Force officials stated that they already  track intelligence requests through existing information systems, and that  the new tool would likely duplicate existing processes. Officials from the  Navy stated that the developmental nature of the tool prevents a full  assessment of its strengths and weaknesses. Officials from the Army  stated that they would wait until the tool is fully developed before deciding  whether to use it, and officials from the Marine Corps stated that their  input had not been solicited.
We have previously identified leading practices for increased  collaboration among agencies, including defining and articulating a  common outcome; agreeing on roles and responsibilities, and  establishing compatible policies, procedures, and other means to operate  across agency boundaries. Given that acquisition and intelligence  personnel have not identified requirements for the Acquisition Intelligence  Support Assessment tool, it may not fulfill the needs of acquisition  programs and the intelligence community or work as intended, and the  services may prefer to use existing systems. As a result, DOD may use  funds unnecessarily to develop a tool that is not needed. Further, without  plans or funding for implementation and operation, the Acquisition  Intelligence Support Assessment tool may not be fully implemented or  sustained once operational.
section_title: Conclusions
paragraphs: DOD has long recognized the need to improve its process for the  acquisition of major weapon systems, and it has recently undertaken  efforts to improve intelligence input both during the acquisition process  and, subsequently, to help enable weapon systems to more effectively  perform their missions once deployed. For example, the department has  worked to integrate intelligence into its acquisition program manager  courses and has developed potential processes for prioritizing intelligence  mission data needs. Addressing gaps we identified in several key areas  will enable DOD to better leverage its efforts. First, without a department- wide certification process that includes training standards, DOD may not  be able to ensure that all personnel who provide intelligence support to  acquisition programs are familiar with and able to provide intelligence  inputs to their portfolios of acquisition programs. Second, without specific  requirements for intelligence mission data prioritization in DOD guidance,  DOD may not be able to ensure that weapon systems have the data they  need to successfully perform their missions once operational. Third,  potential users of DOD’s planned Validated Online Lifecycle Threat report  have not received information or provided feedback regarding the  intended capabilities of the new tool because the DIA has not effectively  communicated the intended capabilities to stakeholders and potential  users. Without a communication plan, DIA may not receive useful  feedback as it develops the system, and ongoing concerns regarding  timeliness, usability, and redundancy of threat information may not be  effectively addressed. Fourth, without conducting an assessment of the  need for and defining requirements for development of its proposed  Acquisition Intelligence Support Assessment tool, DOD may be using  funds to unnecessarily develop a tool that is not needed or, if needed,  may not be fully implemented or sustained once operational.
section_title: Recommendations for Executive Action
paragraphs: To enhance DOD’s efforts to better integrate and improve intelligence  support to major defense acquisition programs, we recommend that the  Secretary of Defense direct—as appropriate—the Under Secretary of  Defense for Acquisition, Technology, and Logistics; the Under Secretary  of Defense for Intelligence; and/or the Secretaries of the military  departments, to take the following four actions in coordination with one  another:
To better enable personnel to provide intelligence inputs to their  portfolios of acquisition programs, establish certifications that include  having these personnel complete required training.
To facilitate implementation of improved processes and procedures  developed by the Acquisition Intelligence Requirements Task Force  and by the Air Force for the integration of intelligence into major  defense acquisition programs, revise relevant guidance and  procedures—including DOD Instruction 5000.02 and DOD Directive  5250.01—to require that intelligence mission data at the acquisition  program, service, and department levels be prioritized.
To better ensure that DOD obtains useful feedback from stakeholders  and the intended users of the Validated Online Lifecycle Threat tool,  instruct the Director of the Defense Intelligence Agency to develop a  communication plan for the tool that includes plans for communicating  with and obtaining feedback from stakeholders and intended users  such as acquisition program offices and personnel providing  intelligence support to acquisition programs.
To ensure that it fulfills the needs of acquisition programs and the  intelligence community and works as intended, assess the need for  the Acquisition Intelligence Support Assessment tool and, if validated  by this assessment, define this tool’s requirements for development  and identify the entity responsible for providing oversight and funding  for its continued development, implementation, and operation.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOD for review and comment. DOD  provided technical comments, which we incorporated as appropriate.  DOD concurred with all four of our recommendations and the responses  are reprinted in their entirety in appendix IV. Based on discussions with  the department, we also revised our recommendations to more accurately  characterize the relevant DOD organizations and offices.
We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; the Under Secretary of Defense  for Acquisition, Technology, and Logistics; the Under Secretary of  Defense for Intelligence; the Secretaries of the Air Force, Army, and  Navy; and other interested parties. In addition, this report is available at  no charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-9971 or kirschbaumj@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix V.
section_title: Appendix I: Information on DOD’s Efforts to Assess the Intelligence-Gathering Capability of Weapon Systems beyond the Scope of Their Mission
paragraphs: We reviewed relevant acquisition-related processes and procedures and  found that the Department of Defense (DOD) has no requirement to  assess a weapon system’s ability to gather intelligence beyond or outside  the scope of its mission.
DOD officials we spoke with from Joint Staff, the Under Secretary of  Defense for Acquisition, Technology, and Logistics (USD(AT&L)), and the  services indicated that there are no requirements for them to consider the  ability of a system to gather intelligence beyond or outside the scope of its  mission during the acquisition process, or otherwise, and that currently  there were no plans to consider performing such assessments. Officials  from the Army and Marine Corps indicated that this had not been  considered because their services were generally focused on non- advanced weapons such as tanks that were ill-suited for this purpose. Air  Force officials told us that current advanced weapon systems such as the  B-2 and F-22, and future systems such as the F-35 have or will have the  capability to gather intelligence outside and beyond the scope needed to  perform their missions, but that there were not currently any plans to  assess this capability during the acquisition process. These officials told  us that these assessments may occur, such as for the non-traditional use  of intelligence, surveillance, and reconnaissance systems, in the post- deployment period as opportunities and capabilities arise.
Officials from some of the service intelligence communities as well as  from USD(AT&L) indicated that one of the challenges faced by current  and future advanced weapon systems is the ability to store and then  offload intelligence data in such a way as to be immediately useful to  analysts in the intelligence community. Air Force officials we spoke with  indicated that there have been efforts to make use of non-traditional  intelligence, surveillance, and reconnaissance systems in the field as a  way of gathering signals intelligence information, but that assessing this  capability was not considered during the acquisition process.
section_title: Appendix II: Objectives, Scope, and Methodology
paragraphs: The National Defense Authorization Act for Fiscal Year 2016 includes a  provision that we review the processes and procedures for the integration  of intelligence into the defense acquisition process. This report  evaluates, for major defense acquisition programs, the extent to which  DOD has (1) processes and procedures for certifying and training  personnel assigned to provide intelligence support to acquisition  programs; (2) efforts to improve processes and procedures for integrating  intelligence into its acquisition programs; and (3) efforts to develop new  tools for integrating intelligence into its acquisition programs. We also  collected information related to DOD’s efforts to identify opportunities for  weapon systems to collect intelligence even when unrelated to their  primary mission, which is presented in appendix I.
To determine the extent to which DOD has processes and procedures for  certifying and training of personnel assigned to provide intelligence  support to acquisition programs, we reviewed DOD guidance governing  the management of intelligence and acquisition personnel. We  interviewed officials from the offices identified in this appendix who  participate in the development of guidance and management of personnel  providing intelligence support to acquisition. We submitted written  requests for guidance regarding staffing, qualifications, certification, and  training of personnel to these officials, and we reviewed their responses.  We also interviewed and received written responses from officials from  the Defense Acquisition University regarding changes to the acquisition  curriculum that included additional intelligence material. We reviewed the  certifications and qualifications DOD has established in implementing the  Defense Acquisition Workforce Improvement Act, and other DOD  guidance for the training and management of acquisition personnel, and  Under Secretary for Intelligence (USD(I)) guidance related to certifications  and qualifications for intelligence personnel. We reviewed training and  certification guidance for both acquisition personnel, as administered by  USD(AT&L), and intelligence personnel, as administered USD(I),  because interviews with DOD officials indicated that personnel who  provide intelligence support to acquisition programs are managed by  acquisition and intelligence components, depending on the service.
To determine the extent to which DOD has processes and procedures for  the integration of intelligence into the acquisition of weapon systems, we  reviewed department-level directives, instructions, and other guidance  that governs intelligence input into the acquisition process. To identify  additional processes specific to the military services, we interviewed  officials from the offices identified in this appendix, and we submitted  written requests for information in order to obtain the documents identified  by these officials. To determine the validity of the document sources used  to identify the intelligence inputs, we reviewed written responses from  service acquisition and intelligence officials at the Army, Navy, Air Force,  and Marine Corps to verify that the documents were current and in use by  the respective services. To determine the extent to which ongoing DOD  initiatives will address identified issues with acquisition and intelligence  integration, we interviewed officials from offices identified in this appendix  and requested documentation on the progress of intelligence-related  tasks identified in USD(AT&L)’s Better Buying Power 3.0 initiatives. We  also observed meetings of the Acquisition Intelligence Requirements  Task Force, and we observed briefings from the task force to the  Acquisition Intelligence Requirements Executive Steering Group. We also  compared the proposed intelligence mission data prioritization processes  with Standards for Internal Control in the Federal Government, which  states that management should establish an organizational structure,  assign responsibility, and delegate authority to achieve objectives.
After receiving the documents, an analyst reviewed each document and  identified actions (such as formation of a working group or certification),  products (such as reports or data), or processes (such as an intelligence  parameter breach or formal review) that could be considered as  intelligence inputs required for an acquisition program classified by DOD  as Acquisition Category I. We defined an intelligence input as any action,  process, or product that involved or included the participation of an  intelligence professional and was provided for a specific acquisition  program. The analyst categorized those inputs with similar names or  document sources and then entered each category and input onto a  spreadsheet. The intelligence inputs are provided in appendix III.
To verify our identification of intelligence inputs, we created a standard  data collection instrument based on the spreadsheet of identified  intelligence inputs. The data collection instrument asked respondents to  identify, for Acquisition Category I programs initiated as of June 1, 2016,  (1) when, if at all, each of the identified intelligence inputs would be used  throughout an acquisition lifecycle; (2) whether their office would provide  input into the item; (3) whether these inputs were required to be provided;  and (4) to provide any comments or additional inputs, if necessary. We  sent this data collection instrument to officials of the intelligence  components of the Army, Navy, Air Force, and Marine Corps; the DIA; the  Office of the Under Secretary of Defense for Intelligence; and the Joint  Staff Directorate for Intelligence, J-2, for a total of seven responses. We  selected these officials and organizations because document sources  identified these organizations as providers of intelligence inputs, and thus  the most likely to identify intelligence inputs. Data were reviewed by two  analysts to ensure that all data were fully extracted and correctly  tabulated.
To provide illustrative examples and determine how processes and  procedures are implemented for individual acquisition programs, we  selected a nongeneralizable sample of six major defense acquisition  programs. We used a stratified purposeful sampling procedure in which  we intentionally chose acquisition programs with particular characteristics  to capture both important similarities and variations. We selected from a  population of acquisition programs identified by the Acquisition  Intelligence Requirements Task Force as having significant intelligence  needs. Two analysts then classified each program with characteristics  using information from a GAO assessment of major defense acquisition  programs, including whether the program was focused on warfare  domains of land, air, or sea; and what service was primarily responsible  for the program. We excluded space and satellite programs from  selection due to the unique differences and higher security classifications  of these programs, as compared with other major defense acquisition  programs. Based on these characteristics, we then selected two Air Force  programs, two Navy programs, one Army program, and one Marine Corps  program. The programs included the following:
Army, Armored Multi-Purpose Vehicle
Navy, Ohio-Class Replacement
Navy, Air and Missile Defense Radar
Air Force, F-22 Increment 3.2B Modernization
Air Force, Joint Surveillance Target Attack Radar System
Marine Corps, CH-53K Heavy Lift Replacement Helicopter  We selected this number and distribution of acquisition programs  because officials from the Air Force and Navy stated that they had many  programs that used intelligence mission data, and officials from the Army  and Marine Corps stated that they did not have many intelligence mission  data-dependent programs. We submitted identical questions and  requests for information to officials from each program management  office, as well as individuals identified by the program as being personnel  who provide intelligence input into the acquisition program. We discussed  the questions orally or received written responses from officials and  intelligence personnel from each program. While the responses we  obtained are not generalizable to all major defense acquisition programs,  the information obtained from program officials provided context and  important insights for our understanding of the interaction of acquisition  and intelligence personnel.
To examine the extent to which DOD has efforts to develop new tools for  integrating intelligence into acquisitions we identified two tools that were  currently in development through discussions with Acquisition Intelligence  Requirements Task Force officials. We verified that these tools were in  development through interviews with officials involved in oversight of  acquisitions and intelligence, including officials at USD(AT&L), USD(I),  and DIA. We conducted a site visit to DIA’s Technology and Long-Range  Assessment offices in Charlottesville, Virginia, where we interviewed  officials and observed a demonstration of a developmental version of the  Validated Online Lifecycle Threat tool. We also interviewed officials from  Performance Assessments and Root Cause Analyses, and we viewed a  presentation and demonstration of a developmental version of the  Acquisition Intelligence Support Assessment tool. We collected  developmental plans and briefings for both of these tools, and we  compared our observations and statements made by DIA and  Performance Assessments and Root Cause Analyses officials against the  documents, and against statements from acquisition, intelligence, and  program management office officials. We compared the developmental  plans and information provided to us by DIA and Performance  Assessments and Root Cause Analyses officials against standards in the  Project Management Institute’s A Guide to the Project Management Body  of Knowledge, federal standards for internal controls, and key practices  for collaboration among federal agencies.
To examine DOD’s processes and procedures for assessing during the  acquisition process a weapon system’s ability to gather intelligence when  unrelated to its primary mission, we reviewed DOD reports and guidance  for acquisition management identified for the previous objectives. We  systematically reviewed the content of these documents for any  information relevant to assessing during the acquisition process a  system’s ability to gather intelligence. We were unable to identify any  process or procedure relevant to this objective. To confirm this finding, we  interviewed acquisitions, intelligence, and requirements professionals  from offices identified in this appendix. Further details are provided in  appendix I.
We obtained relevant documentation and interviewed officials from the  following organizations:
Office of Under Secretary of Defense for Intelligence;
Office of Under Secretary of Defense for Acquisition, Technology, and
Office of Performance Assessments and Root Cause Analyses;
Director of Operational Test and Evaluation;
Cost Assessment and Program Evaluation;
Acquisition Intelligence Requirements Task Force and Executive
Defense Intelligence Agency;
Defense Acquisition University;
U.S. Marine Corps;
U.S. Air Force;
Office of the Director of National Intelligence  We conducted this performance audit from December 2015 to November  2016 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix III: Intelligence Inputs in the Defense Acquisition Process
paragraphs: We defined intelligence input in this report as any action, process, or  product that involved or included the participation of an intelligence  professional and was provided for a specific acquisition program, and we  then grouped the intelligence inputs we identified by inputs with similar  names, document sources, or categories. To confirm our appropriate  identification of intelligence inputs, we created a data collection  instrument based on the spreadsheet of identified intelligence inputs. The  instrument asked respondents to identify, for an Acquisition Category I  program initiated as of June 1, 2016, (1) when, if at all, each of the  identified intelligence inputs would be used throughout its acquisition  lifecycle; (2) whether their office would provide input for each entry we  identified; (3) whether these inputs are required to be provided; and (4) to  provide any comments or additional inputs, if necessary.
For programs begun prior to Milestone A, the intelligence inputs that we  identified from a review of DOD’s acquisition-related guidance are to be  provided prior to Milestone A review, with several updated at points prior  to subsequent milestone reviews. The responses to our data collection  instrument from the Joint Staff, USD(I), DIA, and the service intelligence  community indicated that each intelligence input we identified had one or  more respondent reporting that the respondent’s office would make that  input, which verified the individual inputs we identified (see table 3). DIA,  and Army also remarked that the Validated Online Lifecycle Threat report  will replace the Capstone Threat Assessment, and DIA, the Air Force,  and the Army remarked that the Validated Online Lifecycle Threat report  will replace the System Threat Assessment Report. We have noted that  these inputs are to be phased out once the Validated Online Lifecycle  Threat is operational.
The responses regarding when the identified intelligence inputs would be  used throughout the lifecycle varied among the respondents, but every  input had one or more respondent reporting that the input would be made  prior to Milestone A. We attribute this variance to different interpretations  of inputs that do not align with every milestone, such as threat-related  inputs that occur prior to Milestone A and then are continuously monitored  for changes; or to inputs that are made through groups that meet on a  schedule independent of acquisition milestones. The responses to the  data collection instrument regarding whether each input is required to be  provided for every Acquisition Category I also varied.
The threat assessment and validation category of inputs represent direct  inputs from the intelligence community into acquisition programs. Others,  such as Critical Intelligence Parameter and Critical Program Information,  are foreign threat factors monitored by the intelligence community for  changes that may impact an acquisition program. For the other categories  of input, the intelligence community provides varying degrees of direct  and indirect inputs into the acquisition process. On some of the  responses to our data collection instrument, the respondents provided  comments that indicated they were not familiar with the input, and on the  others, we attribute the variance to different understandings of guidance  among the respondents.
During our review of DOD guidance to identify intelligence inputs into  acquisition programs throughout the acquisition lifecycle we identified the  following key guidance documents for providing intelligence support to  acquisition programs:
DOD Directive 5000.01: The Defense Acquisition System, issued May  12, 2003, provides management principles and mandatory policies  and procedures for managing all acquisition programs, along with  DOD Instruction 5000.02. This directive notes that intelligence and the  understanding of threat capabilities are integral to system  development and acquisitions decisions, and that program managers  are to keep threat capabilities current and validated in program  documents throughout the acquisition process.
DOD Instruction 5000.02: Operation of the Defense Acquisition  System, issued January 7, 2015, provides the detailed procedures  that guide the operation of the Defense Acquisition System.  Regarding intelligence inputs into the acquisition process, among  others, the guidance identifies the requirement for a Lifecycle Mission  Data Plan for acquisition programs dependent upon intelligence  mission data. Additionally, the guidance describes the need to  consider threat projections in the context of Analyses of Alternatives.  It further notes that affordability analysis should involve a DOD  component’s intelligence and acquisition communities. Finally, DOD  Instruction 5000.02 lists requirements for a number of intelligence  inputs, such as Capstone Threat Assessments, Initial Threat  Environment Assessments, System Threat Assessment Reports, and  Technology Targeting Risk Assessments. According to DOD officials,  several of these inputs are being phased out.
DOD Directive 5250.01: Management of Intelligence Mission Data  (IMD) in DOD Acquisition, issued January 22, 2013, establishes  policies and assigns responsibilities to provide linkages between the  management, production, and application of DOD intelligence mission  data and accommodation of intelligence mission data in the  acquisition process. It helps to synchronize the acquisition,  intelligence, and requirement communities regarding intelligence  integration into the requirements process and acquisition life cycle.  According to Acquisition Intelligence requirements Task Force  officials, this directive is currently under revision.
Chairman of the Joint Chiefs of Staff Instruction 5123.01G: Charter of  the Joint Requirements Oversight Council (JROC), issued February  12, 2015, implements the Joint Requirements Oversight Council,  established by statute, which supports the Chairman of the Joint  Chiefs of Staff in carrying out the duties of the principal military  advisor to the President, National Security Staff, and Secretary of  Defense, among other functions. This instruction notes that the  Secretary of Defense has designated the Under Secretary of Defense  for Intelligence as one of the advisors to the Council, and identifies the  Director of the Joint Staff Directorate for Intelligence as an advisor on  intelligence supportability and intelligence interoperability issues,  among other things.
Manual for the Operation of the Joint Capabilities Integration and  Development System (JCIDS), issued February 12, 2015 (including  changes through December 18, 2015): The manual provides detailed  guidelines and procedures for the Joint Capabilities Integration and  Development System, and describes interactions of that process with  several other departmental processes. Among other things, the  manual contains a content guide for intelligence supportability,  providing general descriptions of categories of intelligence support, to  assist with the identification of intelligence support requirements and  sufficiency or risk of shortfalls in intelligence infrastructure required to  support a proposed potential acquisition program throughout its  lifecycle. The manual indicates that in cases where the intelligence  support requirements exceed the intelligence community’s ability to  provide support, resources required to augment the intelligence  support must be accounted for in program affordability documentation.  Categories of intelligence support listed in the manual include  intelligence manpower support, intelligence resource support,  intelligence planning and operations support, targeting support, and  intelligence mission data support, among others. Intelligence  manpower support is to be addressed where the proposed acquisition  will require intelligence personnel for development, testing, training, or  operation. In some circumstances, the category may address  necessary manpower changes or specific required skills. Intelligence  resource support is to be addressed if the proposed acquisition or  supporting efforts will require or depend upon intelligence funding.
Defense Acquisition Guidebook: The Defense Acquisition University  maintains this DOD best practice guide, which complements DOD  Directive 5000.01 and DOD Instruction 5000.02. Chapter 8 –  Intelligence Analysis Support to Acquisition—describes various  aspects of providing intelligence support to acquisition programs such  as threat intelligence support and signature and other intelligence  mission data support. The Defense Acquisition Guidebook is currently  under revision, according to Acquisition Intelligence Requirements  Task Force officials.
Defense Intelligence Agency Instruction 5000.002, Intelligence Threat  Support for Major Defense Acquisition Programs, issued February 1,  2013: Referenced in guidance such as DOD Instruction 5000.02, the  DIA instruction assigns responsibilities and establishes procedures for  DIA and DOD components to provide intelligence threat support for  major defense acquisition programs.
section_title: Appendix IV: Comments from the Department of Defense
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, GAO staff who made  contributions to this report include Brian Mazanec, Assistant Director;  Scott Behen, Pat Donahue, Ben Emmel, Amie Lesser, Jason Lyuke, C.  James Madar, Ronald Schwenn, Michael Shaughnessy, and Cheryl  Weissman.