Defense considered appropriate. Since that time, the joint non-lethal weapons program, with the Commandant of the Marine Corps assigned executive agent, has assumed a role in the requirements development process as well as research, development, test and evaluation. DOD Directive 3000.3 assigns the Under Secretary of Defense for Acquisition, Technology, and Logistics principal oversight for th Non-Lethal Weapons Program, including joint service program coordination to help highlight and prevent duplication of program development, while the Assistant Secretary of Defense for Special Operations and Low- Intensity Conflict under the Under Secretary Defense for Policy, has policy oversight for the development and employment of non-lethal weapons. The Commandant of the Marine Corps is the executive agent for the DOD Non-Lethal Weapons Program. The executive agent serves as the prim DOD/U.S. Coast Guard point of contact for non-lethal weapons and is tasked to coordinate, integrate, review, and recommend, the Join Lethal Weapons program to the Under Secretary of Defense for Acquisition, Technology, and Logistics; and coordinate requirements among the services. Such action is to be taken in consultation with the services, combatant commanders, DOD agencies, and U.S. Coast Guard. Within the Commandant of the Marine Corps’ command, the Joint Non- Lethal Weapons Directorate (JNLWD) was established to perform the day- to-day management of the NLW program. The JNLWD’s director repor the executive agent and, in addition to overseeing joint development efforts which are led by a designated service, monitors the status of service-unique NLW programs. The directorate consists of three divisio — Concepts and Requirements, Technology, and Acquisition — and a support branch. The JNLWD provides program research and de funds but the services are responsible for acquisition program management in accordance with applicable instructions and regulations. The services also established a joint coordination and integration groupcomprised of representatives from each service as well as U.S. Special Operations Command and the U.S. Coast Guard, whose principal role is to advise on and assist in NLW sy structure of the program is illustrated in figure 1. Joint planning to a capabilities-based model, DOD implemented JCIDS in 2 as the department’s principal process for identifying, assessing, and prioritizing joint military capabilities. The requirements and acquisition systems interlock to create products that are intended to meet DOD’s needs. DOD’s oversight of its systems acquisitions is described in a set of documents that provide the policies and guidance for departmental efforts to acquire service capabilities and systems. As figure 2 illustrates, the acquisition process consists of five phases, and at certain points the designated individual with overall responsibility for the program (known as the milestone decision authority) reviews the status of the effort and decides whether to approve entry into the next phase of the acquisition process. The materiel solution analysis phase begins with the Materiel Development Decision review, “at which point the Joint Staff shall present the JROC recommendations and the DOD component presents the including: the preliminary concept of operations, a description of the needed capability, the operational risk, and the basis for determining that non- materiel approaches will not sufficiently mitigate the capability gap.” The Technology Development phase begins at Milestone A, when the Milestone Decision Authority has approved a material solution and a Technology Development Strategy and has documented the decision in an Acquisition Decision Memorandum. Its purpose is to “reduce technology risk, determine and mature the appropriate set of technologies to be integrated into a full system, and to demonstrate on prototypes.” The Technology Development Strategy documents a number of things, including “a preliminary acquisition strategy, including overall cost, schedule, and performance goals for the total research and development program,” and exit criteria for the Technology Development phase. The Engineering and Manufacturing Development phase begins at Milestone B, when the Milestone Decision Authority approves the Acquisition Strategy and the Acquisition Program Baseline and documents the decision in an Acquisition Decision Memorandum. Its purposes include: to develop a system or an increment of capability; complete full system integration… develop an affordable and executable manufacturing process; ensure operational supportability with particular attention to minimizing the logistics footprint… ensure affordability… and demonstrate system integration, interoperability, safety, and utility. This phase includes a System Capability and Manufacturing Process Demonstration, which ends when “the system meets approved requirements and is demonstrated in its intended environment … manufacturing processes have been effectively demonstrated in a pilot line environment; industrial capabilities are reasonably available; and the system meets or exceeds exit criteria and Milestone C entrance requirements.” Successful developmental test and evaluation is also required during this effort. Test and evaluation are used to assess improvements to mission capability and operational support based on user needs. This phase concludes with Milestone C, where the Milestone Decision Authority must commit to the program or decide to end the effort. The purpose of the Production and Deployment phase is to “achieve an operational capability that satisfies mission needs,” utilizing operational test and evaluation to determine the effectiveness and suitability of the system. Criteria for entrance into this phase include: “acceptable performance in developmental test and evaluation and operational assessment… an approved Initial Capabilities Document (if Milestone C is program initiation); an approved Capability Production Document (CPD)… acceptable interoperability; acceptable operational supportability; and demonstration that the system is affordable throughout the life cycle, fully funded, and properly phased for rapid acquisition.” The Operations and Support phase is used “to execute a support program that meets materiel readiness and operational support performance requirements, and sustains the system in the most cost-effective manner over its total life cycle.” Criteria for entrance into this phase include “an approved CPD; an approved ; and a successful Full-Rate Production Decision.” Life-cycle sustainment includes considerations such as “supply; maintenance; transportation; sustaining engineering… environment, safety, and occupational health; supportability; and interoperability.” The fundamental purpose of testing and evaluation is the same for military and commercial products. Testing is the main instrument used to gauge the progress being made when an idea or concept is translated into an actual product. According to DOD guidance, a test is any procedure designed to obtain, verify, or provide data for the evaluation of research and development; progress in accomplishing development objectives; or performance and operational capability of systems, subsystems, components, and equipment items. Evaluation refers to what is learned from a test. The test and evaluation process provides an assessment of the attainment of technical performance, specifications, and system maturity to determine whether systems are operationally effective, suitable, and survivable for intended use. Testing and evaluation is used at a variety of levels, including basic technology, components and subsystems, and a complete system or product. The fundamental purpose of testing and evaluation is to provide knowledge to assist in managing the risks involved in developing, producing, operating, and sustaining systems and capabilities. In both DOD and commercial firms, product testing is conducted by organizations separate from those responsible for managing product development. Standard fielding processes require extensive testing for new programs. For example, the Army’s process to certify weapons for normal fielding requires several different kinds of assessment before a weapon can be fielded. Environmental and air worthiness statements are required. Results of user safety reviews, inspections, and analyses are also required, in addition to a safety confirmation from the Army Test and Evaluation Command. The Army Test and Evaluation Command must also provide an operational test report. Operational test and evaluation is conducted to estimate a system’s operational effectiveness and operational suitability. The testing agency will identify needed modifications; provide information on tactics, doctrine, organizations and personnel requirements; and evaluate the system’s logistic supportability. Thorough and complete testing not only provides assurance that weapons achieve the desired results intended by design, but also allows decision makers in charge of fielding determinations some level of confidence that their selections will perform as advertised. By contrast, an urgent or abbreviated fielding decision allows much of this testing to be bypassed when there is an established and immediate operational need, and then usually only user safety testing is required. For example, one Marine Corps policy states that abbreviated acquisition programs—which the policy identifies as generally small, low cost and low risk programs—are not required to undergo operational testing. Furthermore, another Marine Corps policy states that with appropriate commanding general authority, all testing can be waived, allowing weapons to be fielded in limited quantities to meet urgent operational requirements. Testing for commercial-off-the-shelf items can be even more limited than testing for those that are urgently fielded. DOD guidance states that—in order to take advantage of reduced acquisition time and to ensure that testing is not redundant and is limited to the minimum effort necessary to obtain the required data—“testing can be minimized by 1) obtaining and assessing contractor test results; 2) obtaining usage and failure data from other customers of the item; 3) observing contractor testing; 4) obtaining test results from independent test organizations (e.g., Underwriter’s Laboratory); and 5) verifying selected contractor test data.” Agency officials must determine that a contractor’s test results are sufficient before making the decision to use those test results instead of conducting their own tests. In 1998 the JNLWD contracted The Pennsylvania State University to convene the Human Effects Advisory Panel, a group of scientists who provide assessment of NLW. The panel issued recommendations on the following subjects: A quantitative definition of “non-lethal” and other associated terms, including incapacitation. An assessment of DOD’s methods to generate and verify human effects. An evaluation of DOD’s methodology to generate and validate data. An evaluation of data to support NLW effect analysis. The Human Effects Advisory Panel report concluded that there was a knowledge gap between the expectations of the warfighter and the information provided by the scientific community’s simulation tools. In response to the panel’s recommendations, the JNLW Integrated Process Team Chairman directed that the Human Effects Process Action Team be formed and requested membership from all of the services. The Human Effects Process Action Team was chartered to study the deficiencies in the process of understanding NLW human effects and to recommend policy changes that will help resolve these issues. The team examined current processes for evaluating NLW human effects and made three primary recommendations to DOD: (1) establish an independent board to review the human effects assessments accompanying NLW systems and to ensure that all reasonable assessments have been performed based on available technology and resources; (2) create a NLW Human Effects Center of Excellence to serve as the NLW program managers one-stop resource for information on human effects testing; and (3) adopt a risk assessment approach to evaluating the NLW human effects data due to the uncertainties involved with the science of human effects characterization. The first two recommendations have been implemented. In 2001, the Human Effects Center of Excellence was created via a memorandum of agreement between the Air Force Research Laboratory and the Joint Non-Lethal Weapons Program. The center was founded to provide assistance and advice to program managers concerning likely effects of non-lethal technologies and the risks associated with those effects. The center also serves as a central location for non-lethal human effects data and provides recommendations on which laboratories or field activities can collect scientifically derived information when such information is not already available. The Human Effects Review Board was established in 2000 to independently review non-lethal human effects research and analyses associated with specific NLW systems or technologies. The board consists of representatives from the services’ offices of the Surgeons General, the Medical Officer of the Marine Corps, and the services’ Safety Officers and includes legal and DOD policy participation. The board provides NLW program managers and milestone decision authorities with an independent measure of health risks and recommendations for mitigating potential risks. The Joint Non-Lethal Weapons Program has conducted more than 50 research and development efforts and spent at least $386 million since 1997, but it has not developed any new weapons, and the military services have fielded 4 items stemming from these efforts that only partially fill some capability gaps identified since 1998. Among the contributing factors, we found that DOD did not prioritize departmentwide non-lethal capability gaps until 2007 and still does not have efforts under way to fully address these gaps, that DOD did not give consistent consideration to logistics and supportability in its NLW development process; and that DOD exercises limited general oversight of the program. The Joint Non-Lethal Weapons Program is sponsoring efforts that address about two-thirds of DOD’s NLW capability gaps, but even those efforts provide incomplete solutions, according to the current joint capability assessment. Under the JCIDS process, formal capability assessments are to be used for identifying gaps in military capabilities and potential material and nonmateriel solutions for filling those gaps. Using this approach, the JNLWD, which sponsored the 2008 Joint Capabilities Document for Joint Non-Lethal Effects, identified 36 capability gaps that represented specific tasks where needs were not met by existing or planned systems. The tasks were categorized as either counter-personnel or counter-materiel, and included numerous variations, such as stopping a vehicle or vessel, suppressing individuals, and denying individuals access to an area, all under varying conditions. The gaps were then prioritized by service and combatant command representatives. The resulting list represented the areas in which Joint Non-Lethal Weapons Program research and development initiatives, service acquisition decisions, and other related resource investments are most needed to satisfy the needs of joint force commanders. Table 1 shows the 36 tasks that were analyzed and found to represent gaps in DOD’s NLW capability, as well as their relative priorities. While DOD is now building on the results of this process to determine how to fill the capability gaps, most of the gaps were already broadly identified 11 years ago. The list of 36 gaps is consistent with needs that were acknowledged in DOD’s 1998 Joint Concept for Non-Lethal Weapons as well as its 2002 Mission Needs Statement for a Family of Non-Lethal Capabilities. Though the JNLWD and the services have been working on non-lethal capabilities since 1997, most of these gaps in non-lethal capabilities still exist today. Table 2 compares non-lethal capability needs that DOD has identified prior to the ongoing capability-based assessment process. During the recent capability-based assessment, several NLW efforts were examined, including the 4 programs that have completed the development process and been fielded by one or more of the military services. These programs are 40 mm non-lethal crowd dispersal cartridge, modular crowd control munition, portable vehicle arresting barrier, and vehicle lightweight arresting device. Of these, 3 are variations of or munitions for existing weapons, and the portable vehicle arresting barrier was in an early stage of development when the JNLWD began funding the program. Even when combined with the 12 additional efforts that are ongoing as of March 2009, these programs will not completely eliminate the capability gaps they were designed to address. Existing joint efforts will not fully satisfy all of the tasks, conditions, and standards that DOD analyzed in the process of identifying NLW capability gaps. Based on our analysis of the JNLWD’s program information worksheets and other documents, we found that there are efforts under way to address the top two-thirds of the list of 36 gaps, although we note that there was no comprehensive source that identified each ongoing effort and linked it to the capability gap(s) it addressed. Appendix III provides further detail on the gaps that lack a corresponding effort to address them. Table 3 shows a list of current NLW programs and the priorities of the gaps they are supposed to address. Even though the programs listed in table 3 are intended to address about two-thirds of the capability gaps, they (along with systems already fielded) still only partially meet DOD’s NLW needs, based on the latest joint capability assessment. For example, a vehicle stopper that uses spikes and netting may not cause a quickly moving car to come to a complete stop before it reaches a checkpoint. Therefore, those capability gaps will not be fully addressed and will remain identified gaps. By not assessing and describing the extent to which efforts are expected to satisfy capability gaps, for example, in forums where information on ongoing and proposed programs is presented, the JNLWD has missed an opportunity to fully meet the warfighters’ highest-priority needs for non-lethal capabilities. The military services may also fund their own separate development efforts to address service-unique needs, but with few exceptions have not done so. According to both service and joint program officials, this reflects the low priority that services place on funding non- lethal weapons development. In addition, service NLW proponents have said that the existence of joint funding has made service funding more difficult to obtain. With little progress made toward filling the capability gaps with fielded equipment, joint force commanders continue to lack sufficient non-lethal capabilities. The Joint Coordination and Integration Group, by joint service agreement, is responsible in coordination with the JNLWD for cataloging and tracking progress of programs to include logistics sustainment and logistics requirements planning and ensuring that program managers in the military services conduct appropriate integrated logistic support planning and execution. However, DOD has not given timely and consistent consideration to NLW logistics and supportability because fielded NLW items, which have generally been urgently requested, commercially purchased, or both, have not been subject to the logistics requirements of the normal acquisition process. Moreover, the JNLWD does not make the best use of its own tools for assessing the status and progress of NLW efforts. Specific logistics planning procedures vary by service. For example, under Navy acquisitions policies, program managers are required to complete an independent logistics assessment before a research and development effort may advance through the acquisition process to the point, known as Milestone B, at which an acquisition program is formally initiated. However, only 6 ongoing joint directorate-funded NLW efforts have passed Milestone B, of which 4 have reached Milestone C, by which point operational supportability with particular attention to minimizing the logistics footprint should be ensured. Another 18 efforts were terminated for various reasons (one after passing milestone B), and two were advanced concept technology demonstrations, which were not required to follow the normal acquisition process while the demonstrations were under way. One of the advanced concept technology demonstrations pursued directed-energy technology research to develop a NLW that uses millimeter waves to produce an intense heating sensation on the surface of skin, causing an immediate response and movement by target personnel. This effort, which cost about $35.5 million, yielded two prototypes known as Active Denial Systems 1 and 2. The second prototype weighs more than 9 tons, and has been mounted on a heavier vehicle than the first prototype to accommodate additional armor and air-conditioning (see fig. 3). Because of its weight, it is not easily used for missions requiring mobility. This system also needs about 16 hours to cool down to its operating temperature of 4 degrees Kelvin (-452 degrees Fahrenheit), making it difficult to use on short notice unless the compressor is kept continuously running. In addition, the Marine Corps considered this system’s gyrotron, waveguides, super-conducting magnets, antenna, and some other major subsystems too complex to allow extensive field repair, so its utility could be further reduced. Combat damage to the antenna could create a logistics problem as it is a large item making storage and replacement difficult. The Joint Non-lethal Weapons Program sponsored a Concept Exploration Program for crowd-control technologies which published an analysis of multiple concepts in 2003. The report evaluated eight systems using a variety of criteria, including logistics, and found that the Active Denial System received the lowest benefit:cost score of these. JNLWD officials have made multiple attempts to field the Active Denial System under the rubric of an urgent operational need despite the logistics problems noted above and even though, if it is deployed, its mobility could be further limited where highway overpasses are present. In December 2008, the joint NLW program executive agent terminated efforts to deploy Active Denial System 2 overseas. The manner of purchase and fielding can also affect whether NLW undergo full suitability and supportability evaluations. The fielding process normally provides an opportunity to scrutinize items that a service has procured and intends to provide to its personnel. They receive a formal certification that they are safe, meet performance requirements, and are logistically supportable when used within stated operational parameters. However, nine NLW systems were purchased from commercial vendors and fielded under urgent processes which allow services to certify materiel on a limited basis in order to rapidly support an operational need. In some cases, logistics weaknesses that might have been uncovered by the normal fielding process were not discovered during the abbreviated analysis that takes place prior to fielding. For example, the FN-303 Less- Lethal Launching System program, which DOD spent about $2 million to evaluate, was terminated because the weapon was too heavy and ergonomically cumbersome, the weapon and ammunition magazine was too fragile, and the weapon required compressed air canisters in order to launch its non-lethal munitions. However, several dozen FN-303s were fielded to units even though their utility was limited by the availability of the canisters and the infrastructure to replenish them (see fig. 4). In addition, human electro-muscular incapacitation is an ongoing program that was initiated in fiscal year 2005. One such device, the TASER® X-26 (see fig. 5), has already been fielded to units—both domestically and overseas—as part of the multiple-item Non-Lethal Capability Set. According to NLW training course materials, however, the TASER® X-26E will not be deployed near flammable materials or liquids, as the arcing from the probes could ignite flammable material. In addition, if it is exposed to significant moisture, operators should dry the weapon thoroughly and wait at least 24 hours before proceeding. We believe that these factors could limit the range of environments in which the X-26E (which, like the X-26, has already been fielded) could be employed. We believe that the JNLWD is missing the opportunity to provide sufficient visibility to logistics concerns in part because it does not make optimum use of available tools to catalog and track progress and in part because most of the efforts it funds have not advanced to the stage where these concerns are paramount. The directorate’s program information worksheet, for example, is one of the means that the directorate uses to gather information from program managers about ongoing and proposed efforts, but it does not include a specific space for the program manager to describe logistics supportability goals and how they will be met. The JNLWD uses the program information worksheet to develop an Investment Decision Support Tool, in which the directorate ranks proposed efforts overall and according to five subcategories: cost, schedule, operational contribution, and technology and human effects readiness levels. In our review of available worksheets, we found that there was not necessarily a direct link between the operational contribution score and logistics concerns. For example, the Active Denial System and Mobility Denial System have both received operational contribution scores of 100 despite their supportability problems. Further, requirements for logistics analysis in preparation for a Milestone B decision are often not yet applicable, since 10 of the efforts that the JNLWD was funding as of March 2009 had not yet advanced to that step. An Army official told us that his senior leadership is beginning to require this earlier in the process for Army programs. Without giving full and early consideration to logistics and supportability issues, DOD increases the risk that developmental efforts may not meet service requirements and obtain service funding beyond research and development into acquisition and fielding. While the joint program funds technology research, it is the services that pay to procure, operate, and maintain equipment. DOD also increases its risk of fielding items under urgent processes that are infeasible, difficult to sustain, or both. While DOD has procedures to try to field needed capabilities quickly, these procedures are designed to maximize utility to the warfighter. To the extent that these procedures result in fielding cumbersome or fragile equipment, they may not achieve that goal. Without building these considerations into the earliest stages of development or consideration of commercial off-the-shelf items, DOD may miss opportunities to allocate resources more effectively. DOD’s oversight of the Joint NLW Program, for which the Under Secretary of Defense for Acquisition, Technology, and Logistics (AT&L) has principal oversight responsibility, and for which the Commandant of the Marine Corps has been assigned as Executive Agent, has been limited. This has resulted in gaps in the timeliness and utility of key program guidance as well as limited measurement of progress and performance. A well-managed program, according to federal internal control standards, sets clear and consistent objectives, monitors performance, and ensures that findings of audits and other reviews are promptly resolved. Further complicating DOD’s oversight, no single organization has visibility over all spending categories and available budget information may not fully capture all spending associated with the development of non-lethal capabilities. Both AT&L and the Executive Agent have broad responsibilities for oversight and management of DOD’s NLW program. Although DOD’s NLW policy directive does not specify how AT&L should carry out its oversight of the NLW program, AT&L’s general oversight responsibilities, including the development of acquisition-related plans, strategies, guidance, and assessments, and the principles of good program management are delineated in other DOD directives. According to the 2002 joint service memorandum of agreement, meanwhile, the Executive Agent is supposed to draft, staff, publish, and maintain a master plan that defines the vision, goals, and objectives of the program and includes an overarching framework for research, development, and acquisition as well as modeling and simulation and experimentation plans. However, these plans, along with other key program documents, are outdated and some are currently being revised, as noted in table 4. The Memorandum of Agreement is an agreement among the service chiefs of staff, Commandant of the U.S. Coast Guard, and the Commander of the U.S. Special Operations Command to implement procedures for the NLW program. The 2002 memorandum is outdated, for example, in its lack of provision for oversight of science and technology programs, which the Joint Non-Lethal Weapons Program began to fund in fiscal year 2005. Army and Navy officials have identified science and technology oversight as an issue to be addressed. The Joint NLW Program Master Plan’s purpose is to define the vision, goals, and objectives of the program and it includes an overarching framework for research, development and acquisition as well as modeling and simulation and experimentation plans. It is supposed to be updated biennially. The JNLWD started to update this plan but decided to await the 2003 release of JCIDS to accommodate its requirements. During this same time frame, the directorate was tasked to develop the NLW Capabilities Roadmap, to which it has turned its efforts. The Roadmap is designed to assist in the planning process and to support DOD leaders in making informed decisions regarding resources, priorities, and policies for NLW capabilities. While the existing version describes current efforts and lists anticipated milestones for them, it lacks some elements that could be helpful to decision makers. For example, the Roadmap does not provide guidance on how to allocate resources among priority areas, nor does it relate funding to overall DOD policy and strategy or provide guidance about how to evaluate program performance. Program officials recognized that the Roadmap had limitations and began to revise the initial version as soon as it was approved. As of March 2009, the Roadmap is still being revised. A key AT&L official said that NLW program oversight is exercised through participation in the semiannual meetings of two departmentwide NLW groups: the Joint Coordination and Integration Group and the general officer-level Integrated Product Team. The former advises on and assists in NLW system acquisition while the latter coordinates and integrates joint requirement and priorities and approves consolidated plans and programs. In addition, AT&L officials meet quarterly with the JNLWD Director and participate in other activities, such as the development of the Roadmap, as necessary. Notwithstanding the participation in meetings, however, strategic program direction rests on documents that have been delayed or that lack important elements necessary to make effective decisions. As a result, there is limited strategic direction for the program to guide its day- to-day efforts. While DOD’s NLW program lacks the visibility of other programs with higher priorities and larger budgets, sound program management and oversight practices should apply. Without a greater degree of participation in setting program priorities and reviewing and reporting on performance by AT&L, DOD will not have the level of necessary information needed to make informed decisions about the effective and efficient management of the NLW program. We identified 6 cases in which the joint non-lethal weapons program did not make timely decisions about when to discontinue its research efforts when several years have passed without substantive progress. Although not designed specifically for NLW efforts, DOD’s Financial Management Regulation sets a goal of 6 years (within the future years defense program) for a program to advance through advanced technology development into the acquisition process. Based on our analysis of JNLWD programs, three active development efforts and three terminated efforts reached or exceeded this time frame (see table 5). For example, the Airburst Non-Lethal Munition has been under development for the Army since 1999 (at a cost of nearly $15 million) and has yet to be fielded. In another example, the Mobility Denial System— which relied on slippery foam to limit vehicle traction—continued for 8 years (at a cost of about $10 million) before being terminated because it did not meet combat developers’ needs and its extensive water requirement was considered a logistics burden. Although the Active Denial concept demonstration only lasted 5 years, active denial technology research projects have been underway since at least 1997, at a combined cost of $55.2 million. According to a JNLWD official, the criteria used to determine when to cancel a program are formulated by the program sponsor, program manager, and joint representatives. They reflect technical, programmatic, and policy objectives to be accomplished, and program decisions are made based on the program’s achievement of these tasks with recommendations from the Joint Coordinating and Integration Group. However, as each program is addressed individually, there are no standard termination criteria that are applied to all NLW programs. The directorate uses an Investment Decision Support Tool to evaluate proposed NLW programs as well as active development programs, scoring them on a scale of 0 (lowest) to 100 (highest). The directorate uses this tool to assist in deciding if certain programs are worthy of JNLWD attention and funding, and its results are briefed to service representatives. However, according to a JNLWD official, the scores programs receive on the tool do not directly correlate to the priority for investments because the tool does not incorporate all factors that decision makers need to consider. For instance, the tool does not include the relative degrees of service and combatant command support, past technical performance, or technological feasibility. Appendix IV presents further detail on all of the JNLWD-sponsored programs currently under development and their most recent total scores on the investment decision support tool. While such a tool may be useful to decision makers, a method that incorporates all of the factors needed to make an informed decision, such as logistics and supportability and exit timeframes and criteria, would be a more effective instrument in allocating limited resources. By continuing to fund over long periods of time programs that have not demonstrated their intended capability or have logistics and supportability challenges, the directorate is encumbering resources that might better be used toward the development of other non-lethal weapons programs and capabilities. Further complicating DOD’s ability to oversee its NLW program is the fact that no single organization has visibility over all spending categories, and available budget information may not fully capture all spending associated with the development of non-lethal capabilities. To identify funding for all DOD NLW programs, the JNLWD could only provide us full budgetary figures for its own programs for science and technology or research, development, test and evaluation. Directorate officials are not assigned oversight over service-unique programs and so could not determine exactly how much the services and other DOD components (such as U.S. Special Operations Command) are spending on their own NLW programs. Conversely, the interservice coordinating groups may not provide all necessary information, as the representatives review and approve the approximately 70 percent of the JNLWD budget that goes to research and development, but do not receive a detailed breakdown of the remaining costs. These include such items as studies and analysis, contract support, and salaries for liaisons to the services and combatant commands. We examined budget documents to ascertain what DOD invested for NLW and we found that—in addition to the JNLWD—all four military services, U.S. Special Operations Command, and the Office of the Secretary of Defense have spent money on NLW programs in some capacity during at least some of the last 12 years, but documentation did not always show which non-lethal programs were being funded, nor was it always evident what year the money was being spent. Based on this limited documentation, we identified funding for the military services and other organizations that totaled about $355 million from 1997 through 2008. The JNLWD provided data showing that funding for JNLWD programs totaled about $462 million, giving DOD a total of $817 million budgeted for that time period. The reliability of our estimate of total spending was also affected because DOD budgets do not isolate the portion of weapon procurement budgets that should be attributed to non-lethal effects. Several lethal weapon systems may have non-lethal capabilities. For example, the Army’s Spider Anti-Personnel Landmine Alternative system can use non-lethal munitions to deter, rather than destroy, enemy personnel. However, none of the $172.1 million budgeted for the Spider through fiscal year 2009 was listed as part of the NLW program. Table 6 lists several examples of normally lethal weapon programs that have the capability to be used in a non-lethal manner or use non-lethal munitions. We also found that part of what the military services categorized as non- lethal weapons spending included items that were not necessarily developed as NLW. For example, non-lethal capability sets, which account for about $122 million, are being procured and distributed to units overseas as well as to National Guard troops stationed in units in U.S. states and territories, and are packed in several modules that may be tailored according to mission, for example, checkpoint guards. However, the sets contain such items as riot gear (face shields, shin guards, flex cuffs, etc.) that are not, by DOD’s own definition, non-lethal weapons. Rather, they could more accurately be described as personal protective equipment. Because of definition considerations such as this, we found determining the exact amounts of NLW-related spending to be problematic. DOD plans to spend about $789 million on non-lethal weapons from fiscal years 2009 through 2013. The complex nature of categorizing lethal versus non-lethal weapons and programs makes it all the more important for DOD to have a much clearer understanding of all the programs and investments it is making in NLW. DOD officials told us that they are trying to reach consensus among the services on defining what constitutes a non-lethal weapon in order to more accurately categorize them for budgetary and other purposes. The inability to easily track all money spent specifically on non-lethal capabilities—be they lethal weapons that have non-lethal capabilities or programs that contain items that are not NLW by definition—puts JNLWD and service officials at a distinct disadvantage as they will not have all the information they need to make informed budget decisions. Without adequate oversight, including program direction and visibility of all costs and individual program efforts, the directorate, the services, and DOD at large lack assurance that they are making the most effective use of departmentwide resources and meeting warfighters’ needs. DOD has begun to incorporate ideas about non-lethal capabilities into policy, doctrine, and training, but gaps in key policy decisions limit the effectiveness of doctrine changes and subsequent training. DOD has not yet clearly defined the accepted level of risk for fatality, nor has it fully developed weapons employment policies for overseas warfighting or homeland applications or ensured that warfighters and domestic responders are fully trained in NLW use. Without resolving these policy problems, DOD’s ability to integrate NLW concepts into doctrine and subsequently train personnel in those operations is limited. DOD published a directive in 1996 establishing policy and assigning responsibilities for the development and employment of non-lethal weapons. According to this directive, non-lethal weapons, doctrine, and concepts of operation are to be designed to reinforce deterrence and expand the range of options available to commanders. Non-lethal weapons are also meant to enhance the capability of U.S. forces to discourage, delay, or prevent hostile actions; limit escalation; take military action in situations where use of lethal force is not the preferred option; better protect the forces; temporarily disable equipment facilities and personnel; and help decrease the postconflict costs of reconstruction. DOD has begun to incorporate non-lethal weapons into existing doctrine and concept publications. The joint staff and the services have issued several dozen doctrine publications that cited a need for non-lethal capabilities and began to discuss the importance of developing capabilities that may be applied across the range of military operations, such as for Peace Operations, Urban Operations, and Civil Support. These organizations have also updated publications that describe a need to include NLW as part of the overall use of force continuum within planning for such diverse missions as command and control for joint land and maritime operations and joint counterdrug operations. In addition to mentioning the need for NLW capabilities in policy and doctrine, the services and combatant commands also have begun to incorporate non-lethal weapons into their plans and procedures. U.S. Northern Command has developed concept of operations plans for Defense Support to Civil Authorities/Homeland Defense missions. The military services have issued a joint tactics, techniques and procedures manual and updated the manual in 2007. They have also issued service- specific guidance. For example, the Air Force has a manual tailored to the particular needs of Air Force security forces and the Army has published a field manual on civil disturbance operations. The Marine Corps has developed mission-essential tasks and individual training standards in support of NLW use and recently issued specific use policy for human electromuscular incapacitation devices such as TASER®. Beginning in 1998, the Marine Corps was designated as the lead service for the Interservice Non-lethal Individual Weapons Instructor Course—the source for formal NLW instructor training for all of the services—and has also deployed mobile training teams to help facilitate on-site NLW training in Iraq. Although DOD has begun to incorporate NLW concepts throughout its body of operational doctrine, our analysis indicated that most references are limited, recognizing the value of such a capability, the need for it, or both, but not providing additional guidance about how such capabilities should modify existing operational concepts. For example, the joint doctrine publication, Doctrine for Joint Urban Operations, includes two paragraphs that describe the potential flexibility offered by non-lethal weapons but offers no other operational perspective or guidance in the 150-page publication. This may limit the utility of information on non- lethal weapons in existing doctrine. One reason for this is that we found gaps in DOD’s policy and guidance for NLW. DOD recognizes that policy tends to drive doctrine and doctrine, in turn, influences training and the execution of operations. Therefore, weaknesses in policy make it difficult to effectively produce or augment the doctrine and training. While DOD has clearly articulated a policy that non-lethal weapons shall not be required to have a zero probability of producing fatalities or permanent injuries, it has not (1) fully articulated what constitutes acceptable risk, (2) fully explained how employment doctrine should vary by scenario, or (3) provided specialized training to enable operators to make effective use of NLW in various contingencies, particularly within the United States. DOD has not reached consensus on how to answer these questions. The directive that establishes DOD policies and assigns responsibilities for the development and employment of non-lethal weapons states that NLW are designed and employed to minimize, rather than eliminate, fatalities. As such, NLW may have lethal effects and therefore carry some level of risk that is not precisely defined. For example, non-lethal is described in a joint functional concept as the degree to which the joint force is able to create desired effects using incapacitating, nonfatal capabilities. In addition, DOD stated as part of the capabilities-based assessment that increasing non-lethality widens the range of effects the joint force is able to achieve without using deadly force, or in the case of defense support to civil authorities, that the avoidance of casualties is imperative. An early assessment by the Human Effects Advisory Panel posited that acceptable risk was 1 percent suffering permanent damage (of which half were lethal), 98 percent incapacitation, and no effect on 1 percent of the population, but senior NLW program officials said that those figures were never considered authoritative and that acceptable risk had not been quantified. DOD has also not fully clarified what constitutes acceptable risk short of fatality. Current DOD policy defines NLW as weapons that are explicitly designed and primarily employed so as to incapacitate, and are intended to have “relatively reversible” effects. Based on the results of the capability- based assessment, the JNLWD has modified its definition of NLW to include weapons, devices, and munitions that are explicitly designed and primarily employed to “immediately incapacitate” targeted personnel or material, and that are intended to have “reversible” effects. These changes have not yet been incorporated into overarching DOD policy, although JNLWD officials have said that they are understood within DOD. However, publications are not fully synchronized throughout the department; for example, Army doctrine states that the use of NLW should “temporarily incapacitate.” Furthermore, it states that NLW use must not result in unnecessary suffering” without defining this term in the context of NLW effects. Each of these definitions has different implications for NLW development and employment, because as the “dose” of a weapon increases, so does its potential both to achieve the desired effect and to produce permanent injury. Thus, a NLW that is more successful with respect to onset – that is, immediately incapacitates — could be less so with respect to duration – that is, proves irreversible. Once DOD does finalize a new common definition, though, uncertainty about acceptable results of the use of NLW may still create unrealistic expectations. For example, the Marine Corps prepared an urgent needs statement to support its request to field a laser dazzler, in which it noted that the flares then in use caused injury and one fatality. The Marine Corps requesters wanted a capability that would avoid that outcome. The uncertainty of outcome may lead to expectations that non-lethal actually means never lethal, and moreover may not even cause any kind of serious injury. According to a senior NLW program official, the term “non-lethal” itself sets up false expectations, and DOD should establish a concerted strategic communications program to disabuse those who may be the target of such weapons as well as military users of the idea that “non- lethal” is risk-free. DOD’s interservice NLW training course materials also cite the possibility that political or military leaders might form an incorrect perception that NLW will allow wars and military operations other than war to be prosecuted without casualties. Other federal agencies, whose personnel may use NLW within the United States, manage expectations differently. For example, the Department of Justice uses the term “less- lethal” so as not to create the expectation that certain weapons never produce fatal results. The Department of Homeland Security uses both terms. Congress also defined NLW in a way that may reinforce expectations. In the statute directing DOD to establish centralized responsibility for the establishment of NLW technology it defined a “non- lethal weapon” as a weapon or instrument the effect of which on human targets is less than fatal. DOD has not been able to provide further clarification on acceptable risk primarily because there is no departmentwide consensus on what constitutes acceptable risk. DOD officials have been discussing the development of a methodology for characterizing acceptable risk that can be applied more specifically to individual non-lethal weapons or devices. They told us that they are nearing agreement within the department on this methodology. However, as of February 2009, the methodology had not been formally approved. The lack of a consistent and clear methodology regarding risk levels can hinder efforts to write formal requirements for material solutions to identified capability gaps, without which new products cannot progress through the acquisition process, and can complicate efforts to field NLW that are purchased from commercial vendors, to use them, or both. Until NLW terminology is clarified and fully disseminated, it could continue to create unrealistic expectations that could complicate the efforts of material developers, result in inconsistent rules of engagement, and make operational commanders more hesitant to employ any available NLW. Moreover, until DOD clarifies its policy on how to assess the risk of fatality or permanent injury it is willing to accept, it will be very difficult to develop, deploy, train for, and use any NLW that have the potential either to be lethal or create detrimental political effects. Our analysis of DOD’s policy and doctrine showed that DOD personnel lack clear guidance about how to employ NLW across the range of military operations, both overseas and domestically. This could be relevant, for example, for a range of missions that involve force but occur in a nominally “peacetime” scenario, or in ambiguous situations across the spectrum of conflict. NLW may be used to determine intent: a warning NLW, such as the laser dazzler, might be used to induce an approaching vehicle, individual, or group to stop, and failure to stop is then assumed to mean hostile intent against which lethal force may be used. However, our review and analysis of existing doctrine showed that it does not provide adequate guidance for employing NLW in ambiguous situations, such as when a vehicle or pedestrian is approaching a checkpoint and intent is not obvious. In some cases, servicemembers might not be able to determine intent until an individual is within a short distance. While there are NLW such as the laser dazzler that can allow troops to attempt to provide warnings over long distances, the existing suite of NLW that can incapacitate an individual is only effective at close range, so servicemembers have limited options. They could use blunt-force munitions or electro-muscular incapacitation devices, both of which are generally ineffective beyond a short range; or wait for an individual to approach to within the range at which these work, at which point effective self-defense may no longer be possible. These policy and weapons employment considerations are important to the warfighter because they represent a balance between safety/risk for U.S. service personnel and safety/risk to individuals or groups targeted by NLW. They are also important to provide context for the application of the standing rules of engagement that apply to military operations, for use where U.S. forces face hostile forces, hostile acts, or demonstrated hostile intent. Concepts of operations for use of DOD forces within the United States consider the avoidance of civilian casualties imperative, and the standing rules for use of force state that, normally, force is to be used only as a last resort and the force used should be the minimum necessary. Deadly force is to be used only when all lesser means have failed or cannot reasonably be employed, which implies that all other options must be exhausted first. By contrast, overarching DOD non-lethal weapons policy states that the existence of NLW in no way constitutes an obligation their employment and that the United States retains the option for DOD has not issued new guidance or immediate use of lethal weapons. instituted training that reconciles these two stances; for example Nation Guard Bureau guidance for the domestic employment of NLW contains little more than a restatement of passages from the DOD directive on non- lethal weapons. DOD also has not directed that any specific pieces of al equipment that could produce lethal effects be excluded from the capability sets that have been fielded in the United States. For example, non-lethal capability sets containing TASER® have been fielded to National Guard units in every state. TASER® is controversial because of concerns about injuries and fatalities that occurred in the course of its u se in law enforcement. The Marine Corps proscribed use of these weaponsuntil it published a policy specific to TASER®, and the Army compone U.S. Central Command decided not to train troops in their use. Furthermore, the joint staff has issued doctrine that states the employment of non-lethal weapons in certain supporting operations will also be governed by their political impact, For such operations, weapons employment policies would need to be developed and disseminated so that the existing rules of engagement and rules for the use of force could be adequately tailored to minimize detrimental political effects resulting from the use of NLW. In particular, DOD policy for employment of directed-energy NLW such as the Active Denial System is incomplete. The Under Secretary for Policy deemed the system politically untenable for detainee operations but has not yet issued employment policy for other missions. While the office of the Under Secretary of Defense for Policy has approved the Active Denial System in principle, a former senior policy official wrote that DOD would continue to require the development of definitive concepts of operation; rules of engagement; and tactics, techniques, and procedures before the Active Denial System could be deployed. While the Joint Non-lethal Weapons Program executive agent in December 2008 terminated efforts to deploy the existing Active Denial System overseas, DOD continues to try to find ways to deploy the system in the United States, possibly at the southern border. Unresolved questions about acceptable risk and proper employment guidance can also have an impact on the quality of training that warfighters and domestic responders receive. DOD’s Interservice Non- Lethal Individual Weapons Instructor Course provides scenario-based training that can be applied under the standing Rules for the Use of Force and Rules of Engagement that apply to all services, but it cannot integrate realistic training for specific situations into its curriculum unless appropriate policy has been developed. Because currently available NLW have short ranges, reaction time is limited and warfighters will need to make quick decisions, possibly in rapidly changing circumstances. While DOD can and does produce mission-specific rules of engagement, gaps in policy and doctrine limit the training that can be provided prior to deployment. Until these issues are resolved, doctrine and training for non- lethal weapons may be limited, and the warfighter or domestic responder may have fewer options other than resorting to lethal force. Testing NLW for effects on targets and bystanders is a difficult technological undertaking, in part because human effects testing needs to be done using modeling and surrogates which may not accurately reflect human responses. Further, almost all NLW have been fielded to date using abbreviated processes because of urgent needs, and as a result, NLW have generally not undergone the same level of effects testing to meet standard fielding requirements. One of the components lacking in DOD’s approach to testing NLW is a consistent methodology for assessing the risks of various human effects. While DOD has begun to develop elements of a risk assessment methodology, this methodology cannot be completed until human effects testing requirements are standardized in DOD policy. Testing and evaluation that include human effects testing measures could improve planning and commanders’ ability to avoid unintended effects of NLW use. The complicated nature of testing the outcomes of NLW use centers on the testing of the effects of these weapons on targets and bystanders— typically referred to as human effects testing. The test and evaluation process provides an assessment of the attainment of technical performance, specifications, and system maturity to determine whether systems are operationally effective, suitable, and survivable for intended use. Unlike conventional lethal weapons that destroy their targets principally through blast, penetration and fragmentation, NLW are intended to prevent the target from functioning and their effects are intended to be reversible. Testing of human effects would measure the NLW’s likelihood and degree of causing irreversible effects on human targets and bystanders. There are, however, several limitations to human effects testing. Although technology is improving to better test for and predict NLW human effects, DOD policies limit the use of human subjects for testing and the nature of nonlethality, which is aimed at producing reversible effects, poses challenges to testing accuracy. DOD policy states that “the rights and welfare of human subjects in research supported or conducted by the DOD Components shall be protected.” Therefore, when possible, human effects are derived from animal and computer-based models in substitution for direct effects on human subjects. We acknowledge the importance of ensuring the protection of human test subjects but also recognize that the test measures designed and put in place to operate within these restrictions—such as surrogate (i.e., test dummies) and animal tests—face unique challenges to produce accurate and timely test results. Furthermore, according to a DOD human effects testing official, the confidence intervals associated with non-lethal effects testing are typically low. For example, the Human Effects Center of Excellence reports that testing non-lethal munitions requires accurate accounting for projectile properties such as its mass, impact velocity, shape, target size, and impact location. However, potential for injury on the target varies depending on the weight and size of the target and the accuracy of the NLW projectiles— which are often hard to predict and test for. According to the Human Effects Advisory Panel, there is a knowledge gap between the expectations of the warfighter and the information that is being provided by models and simulation tools from the scientific community. Testing accuracy is inherently limited when extrapolations are based on subjects other than human subjects. The Advanced Total Body Model is an example of a simulation tool used to predict the risks associated with blunt impact weapons and was designed to test for effects upon various body parts (e.g., ribs, abdomen, head-neck). However, simulation tools are not capable of testing for all possible effects derived from a NLW. A broken rib, for example, could result in a punctured lung which may cause death. According to officials at the Human Effects Center of Excellence, other sources used to test and collect data for non-lethal effects are animals that share similar human characteristics. For example, they said chinchillas have similar inner ear structures to humans and are used to test for ear damage caused from acoustic NLW such as flash bang grenades. However, functional and anatomical differences between human and animal subjects may limit the generalizability of test results to human populations. If suitable testing that models NLW effects on humans is not conducted, then it becomes unclear how and when to use non-lethal weapons given the lack of assurance concerning the effects on the targets and bystanders. Current DOD testing policies do not address testing of NLW effects on human targets and bystanders. DOD has started to draft a policy that— once approved—would establish guidance and procedures for the characterization of target human effects in support of the development of NLW acquisition programs, but this policy has not yet been agreed on within DOD, formally approved, or implemented. Therefore, with the exception of laser-related weapons, human effects testing is currently not required, including the use of simulation tools and other methods. JNLWD officials told us that all NLW programs that receive JNLWD funding must be reviewed by the Human Effects Review Board before every major acquisition milestone. Service-funded programs are not required to undergo the same review; however, JNLWD officials said they encourage it. Army officials told us that they conduct human effects testing for non- urgently fielded NLW. Further limiting the amount of human effects testing being conducted is the fact that, to date, almost all NLW have been fielded using abbreviated processes to meet urgent operational requirements. When there is an established and immediate operational need, an urgent or abbreviated fielding decision allows DOD to bypass most testing, other than user safety testing, that is normally conducted for weapons fielded through the standard process. For example, the Army’s Urgent Material Release process requires a safety assessment, but Army officials told us that this safety testing is for the user of the weapon only, and does not test for the safety of the target or bystanders. Marine Corps policy states that with appropriate commanding general authority, weapons may be fielded in limited quantities to meet urgent operational requirements, even if all safety requirements are not met. A Marine Corps official told us that a commander’s willingness to accept safety risks associated with an NLW’s rapid acquisition given the urgent need of the weapon in the field is ultimately what drives NLW deployment. He said that the most complicating factor of streamlining and regulating the acquisition process for NLW from beginning to end is managing this balance between urgency and safety. DOD testing for commercial-off-the-shelf items can be even more limited than for those urgently fielded because agency officials can use contractor test data instead of conducting their own tests. The Defense Acquisition Guidebook recognizes the importance of oversight and government involvement in testing performed by contractors. Anytime agency officials decide to use contractor test results without adequate oversight or involvement in the testing, there is an increased risk that the testing was biased, the testing environment was not relevant for the weapon’s intended operational use, or the test results were inaccurately represented. One example of a commercial item where use of contractor test data had the potential to lead to unpredictable results was the TASER®. In 2002, the Human Effects Review Board reviewed the TASER® M26 model and submitted an approved but limited fielding recommendation. The recommendation was largely based on anecdotal data and field experience gathered over the last 20 years from police enforcement activities where TASER® was predominantly used on male targets. The Human Effects Review Board was concerned by the lack of unbiased, peer-reviewed scientific evidence of TASER® effects and effectiveness that is necessary to support a stronger endorsement. In 2003, the Army’s safety office at Picatinny Arsenal issued a safety certification to support the urgent fielding of the TASER® M26 model. However, the Army did not field the M26 model and instead fielded an even more advanced version of TASER®—the X26E model—that produces a 5 percent increase in muscle contraction compared to the approved M26 because it uses a waveform that is different from that of any preexisting models. Although the Human Effects Review Board did evaluate testing results for the TASER® X26 model in 2008 and determined that the human effects research conducted was sufficient, DOD increased the risks of unintended effects by reviewing testing data after the weapon was already being used in the field. Although testing the effects of NLW is technologically challenging, JNLWD recognizes that human effects, effectiveness, and risk must be quantified in order to support legal, treaty, and policy reviews and to ensure warfighter confidence in new technologies. DOD has begun to develop elements of a risk assessment methodology, but the methodology will not be complete until human effects testing requirements are standardized in DOD policy. In its Risk Management Guide for DOD Acquisition, DOD recognizes that risk management is critical to acquisition program success. In particular, DOD notes the need to define a program by satisfying the user’s need within acceptable risk. According to this guide, the purpose of addressing risk in programs is to help ensure that program cost, schedule, and performance objectives are achieved at every stage in the program’s life cycle and to communicate to all stakeholders the process for uncovering, determining the scope of, and managing program uncertainties. Although DOD policy does state that NLW shall not be required to have a zero probability of producing fatalities or significant injuries, our review found that the policy does not articulate a methodology for what constitutes acceptable risk of fatality and significant injuries across DOD and the services. Without a better understanding of acceptable risk, NLW developers and designers have no way of knowing whether the risk levels associated with the effects produced from NLW are in compliance with standards and whether NLW developments are progressing sufficiently to meet the needs of the warfighter. Although DOD has not established standardized acceptable levels of risk of fatality and significant injuries, DOD has a draft policy in development for a Risk of Significant Injury scale that characterizes the amount of treatment necessary to reverse the effects of an NLW (see fig. 6). The Risk of Significant Injury scale broadly categorizes three levels of health care capabilities required to reverse the effects of NLW once they are used on targets, but it does not take into account the risk probabilities of injury for each category for a given weapon or target. In other words, the Risk of Significant Injury scale does not assess the likelihood that non-lethal effects could cause a score of 0, 1, or 2 within the scale, and thus does not provide information about what probability to expect for each category of injury. The Human Effects Process Action Team concluded in 2000 that the Human Effects Review Board should make adopting a risk assessment approach to evaluating the NLW human effects data a priority because of the uncertainties involved with the science of human effects characterization. The team—directed by a Marine Corps lieutenant general and composed of each service’s acquisition executive and surgeon general—stated that a risk assessment methodology would allow the human effects of NLW to be expressed along with a measure of the confidence in the data. Hypothetically, if testing showed that a NLW carries a 15 percent risk for permanent injuries, a risk assessment methodology would allow a risk level (e.g., low-to-moderate or moderate- to-high) to be assigned. A commander can then use this as a basis for a decision on whether to accept that risk. DOD has not yet established a risk assessment methodology for human effects testing that is capable of identifying the potential risks associated with the use of NLW. Without a risk assessment methodology, NLW human effects are not fully understood and cannot accurately be predicted, which may result in unexpected effects upon targets and bystanders and cause political consequences. DOD recognizes that part of a successful risk management strategy includes sufficient testing and evaluation measures, and DOD also recognizes the importance of assessing operational effectiveness. Nevertheless, testing and evaluation that include human effects testing measures could improve planning and commanders’ ability to avoid unintended effects of NLW use. DOD’s Non-Lethal Weapons Program is intended to provide U.S. armed forces with flexibilities for dealing with the rapidly changing threat environment, especially when using lethal force is undesirable. However, key aspects of this program, such as assessing the extent to which priority capability gaps will be addressed, focusing on supportability and operational utility in the field, and providing oversight and full funding visibility, have been limited. These problems have contributed to the program’s overall limited progress in fielding suitable NLW. New weapons requirements and development are often understandably affected by technology hurdles and a preference to field an item that will partially meet needs quickly rather than wait indefinitely for a perfect solution. However, we note that 12 years have passed since DOD established the JNLWD and that the services were working on NLW development efforts even before that. Better planning, management, and oversight of NLW developmental efforts to incorporate early consideration of technology readiness, suitability, and supportability could improve the rate of progress. While individual services may attempt to satisfy some service- unique gaps on their own, the measure of a successful joint program will be whether it can successfully foster joint development. Without clearer policy on acceptable risk to both warfighters and potential targets, in both overseas and domestic scenarios, doctrine and training for NLW will continue to be limited. Finally, conducting suitable testing and evaluation is complicated in an environment where acceptable alternatives to human testing – animal tests and modeling and simulation – are themselves both limited and inherently difficult to extrapolate to humans. Although DOD recognizes that it needs to develop a risk assessment methodology and has taken steps toward that end, it still lacks the means to predict levels of risk concerning non-lethal effects on targets and bystanders. As a result of all these factors, DOD’s NLW program has had limited success in planning, developing, overseeing, and testing effective and efficient weapons. Unless these factors are addressed, the ability of U.S. forces to conduct operations across the full range of potential lethality where and when needed will be hindered, and they will continue to lack the means to escalate force while still achieving non-lethal effects. We recommend that the Secretary of Defense take the following eight actions: To help DOD better match program priorities to identified capability gaps, the Secretary of Defense should direct the JNLWD, in consultation with the services and combatant commanders, to assess and document the extent to which NLW efforts at the technology development stage and beyond (including procurement and operations and maintenance) address the highest-priority Joint Staff-validated capability gaps. To help DOD better incorporate logistics and supportability considerations, the Secretary of Defense should direct the JNLWD, in consultation with the services and combatant commanders, to ensure that appropriate logistics and supportability planning is integrated into development efforts at the earliest possible stage, including both DOD- developed and commercial weapons and capabilities. Incorporating changes to—and using information already gathered for—the JNLWD’s Investment Decision Support Tool might assist the directorate and DOD in establishing clear criteria and ensuring progress in this area. To help the Under Secretary of Defense for Acquisition, Technology and Logistics in its role in overseeing DOD’s Joint Non-Lethal Weapons Program, the Secretary of Defense should take the following actions: Require the Under Secretary of Defense for Acquisition, Technology and Logistics, in consultation with the Executive Agent, to ensure that NLW strategic guidance that sets out goals, objectives, and a framework for research, development, and acquisition—including science and technology efforts—is established and routinely updated. Require the Under Secretary of Defense for Acquisition, Technology and Logistics to oversee the development of performance evaluation criteria to guide decisions on how and for how long to allocate resources to research and development efforts. In addition to established DOD financial management regulations, DOD could use existing tools, such as the Investment Decision Support Tool, to help develop and implement these measures. Direct the Under Secretary of Defense for Acquisition, Technology and Logistics to develop and execute a methodology for monitoring all NLW-related funding and programs across DOD and designate a central focal point within that office to coordinate the effort with the JNLWD. To help DOD more fully incorporate non-lethal concepts and capabilities into its existing and new policy and doctrine for operations overseas and in the homeland, the Secretary of Defense should direct the Under Secretary of Defense for Policy to articulate a methodology and develop a time frame for determining acceptable risk with respect to lethality and permanent injury for operators, targets, and bystanders due to the use of specific types of NLW, and the Secretary of Defense should direct the Joint Staff, in consultation with the Under Secretary of Defense for Policy and the Services, to provide clearer weapons employment guidance that can be used to modify or augment existing rules of engagement or rules for the use of force for both warfighters and domestic responders on how non-lethal weapons should be used under certain conditions, and incorporate this guidance into training curricula. To help DOD conduct more thorough testing and evaluation of non-lethal weapons and aid end users’ ability to plan by knowing what to expect from NLW before using the weapon, the Secretary of Defense should direct the JNLWD and the military services to finalize and implement a risk assessment methodology for human effects testing of NLW and develop a timeline for implementing the methodology. In written comments on a draft of this report, DOD concurred with five of our recommendations, partially concurred with the other three, and described actions it is taking or will take to implement all of the recommendations. DOD’s comments are reprinted in appendix II. DOD also provided technical comments, which we have incorporated into the draft as appropriate. The Departments of Justice and Homeland Security also reviewed a draft of this report and had no comments. DOD concurred with our recommendation that DOD assess and document the extent to which non-lethal weapons efforts at the technology development stage and beyond address the highest-priority capability gaps and stated that they will incorporate a methodology for accomplishing this into the NLW Capabilities Roadmap and into the overall Joint Non-lethal Weapons program management process. With respect to our recommendation that DOD integrate logistics and supportability planning into NLW development efforts at the earliest possible stage, DOD agreed and stated that it would elevate consideration of logistics and supportability during program reviews and by using other existing tools, as appropriate. DOD agreed with our recommendation that it ensure that NLW strategic guidance that sets out goals, objectives, and a framework for research, development, and acquisition is established and routinely updated. DOD stated that to implement this recommendation, in addition to completing updates to DOD Directive 3000.3 and the Joint Services Memorandum of Agreement, the Office of the Under Secretary of Defense (AT&L) is working with the JNLWD to develop a new version of the Non-lethal Weapon Capabilities Roadmap and plans to be more active in the NLW Joint Integrated Product Team. DOD agreed with our recommendation that it develop and execute a methodology for monitoring all NLW-related funding and programs across DOD and designate a central focal point to coordinate the effort with the JNLWD. DOD stated that AT&L will coordinate with the directorate to develop and implement a methodology for monitoring NLW and funding and progress across the department in order to provide a more effective foundation for decision making. DOD also agreed with our recommendation that DOD finalize and implement a risk assessment methodology for human effects testing of NLW and develop a timeline for implementing the methodology. DOD stated that in addition to implementing a risk assessment framework across technology development programs, the JNLWD has begun to develop a human effects characterization guidance document that will become standard across DOD. We believe that these steps will improve management and operations of the Joint Non-lethal Weapons Program and encourage DOD to fully implement them as soon as possible. DOD partially concurred with our recommendation that the Secretary of Defense require the Under Secretary of Defense for AT&L to oversee the development of performance evaluation criteria to guide decisions on how and for how long to allocate resources to research and development efforts. DOD agreed that enhanced performance evaluation criteria could better guide resource allocation decisions, but does not believe that new measures are needed. Nevertheless, DOD stated that the JNLWD, with oversight from AT&L and the Joint Integrated Product Team, would improve existing evaluation criteria to more effectively guide resource allocation decisions. DOD partially concurred with our recommendation that DOD articulate a methodology and develop a time frame for determining acceptable risk with respect to lethality and permanent injury for operators, targets, and bystanders because of the use of specific types of NLW. DOD agreed with the need for a methodology and time frame for assessing the risks inherent in employing non-lethal weapons. DOD stated that as we mentioned in our draft report, the Risk of Significant Injury methodology will help address our recommendation and that DOD intends for this methodology to be the basis for the human effects characterization guidance document in development. DOD stated that it does not believe that such a methodology should articulate thresholds for acceptable risk and that such determinations should be left to military commanders with the advice of legal advisors. DOD also stated that it does not believe an acceptable risk methodology should include the risk to operators of a weapon because such risks are already addressed in the existing acquisition process. We agree that military commanders (with the appropriate legal advice) should make the determination of acceptable risk when employing any weapon— including NLW. Our intent was to highlight that in order to make these decisions, military commanders require accurate information on what effects a NLW should be expected to have. By finalizing and fully incorporating the risk of significant injury methodology and guidance into NLW efforts and properly implementing them, DOD should be able to arrive at the kind of consistent and accurate information needed. To the extent the acquisition process includes risk to the operator of a NLW, we would expect that this information would be provided to commanders in the same context as risk to targeted individuals and bystanders. DOD partially concurred with our recommendation that DOD provide clearer weapons employment guidance that can be used to modify or augment existing rules of engagement or rules for the use of force for both warfighters and domestic responders on how NLW should be used, and incorporate this guidance into training curricula. DOD agreed with the necessity of doctrine that more clearly addresses NLW employment, but stated that such doctrine should be integrated into existing policy documents rather than creating separate employment guidance. To the extent that clear guidance on the employment of NLW overseas and in the United States can be incorporated into existing or supplemental documents, we agree this should allow DOD to clarify how NLW are intended to be employed in the wide range of operational circumstances, enhance the broad understanding of the use-of-force continuum, and facilitate the modification of training curricula. We continue to believe that, in whatever form it is presented, DOD should provide the clearest possible guidance. As we discussed in our report, policy and doctrine tend to drive training, and the clearer they are, the better the training that can be provided. Although not addressing a specific recommendation, DOD expressed concern that we did not sufficiently acknowledge the positive steps taken and important contributions made by its investments in NLW, and cited the contribution of the Non-Lethal Capability Sets in supporting Operations Enduring Freedom and Iraqi Freedom in providing U.S. forces with valuable escalation-of-force options. We acknowledged the fielding of the Non-Lethal Capability Sets in our draft report. We also acknowledge that DOD has made progress in adapting the way it conducts operations by expanding the use and potential use of NLW, and also that several of the commercial-off-the-shelf items fielded under urgent requests have proven valuable and timely to the warfighter. Moreover, we recognize that the Non-Lethal Capability Sets have been requested by Army units and that the Army Chief of Staff directed a requirement that all brigade combat teams be issued these sets. However, DOD officials, including some who are part of the Joint Non-lethal Weapons Program, have pointed out to us that range limitations of current munitions and other Non-Lethal Capability Set items are important factors driving the JNLWD’s and services’ current development efforts. We continue to believe that in order to achieve the kinds of operational flexibility DOD seeks, including saving lives, greater effort is required to align policies, doctrine, technology, and logistics. DOD also expressed the view that we had not accurately portrayed its efforts with respect to the Active Denial System. DOD states that it intended the Active Denial System to be a concept demonstration and did not intend to develop a fully integrated, production-ready system. Although we acknowledged in our draft report that the Active Denial System was a concept demonstration, we observed that the level of resources the JNLWD devoted to the Active Denial System in comparison to its other efforts indicated a significant investment in a capability that was intended to eventually meet warfighters’ needs. Our use of the Active Denial System in our report was primarily in the context of illustrating specific findings. For example, we noted missteps with regard to the effort to deploy Active Denial System 2 as a way of illustrating gaps in DOD’s emphasis on fully developing logistics and supportability plans at the earliest possible stage of development. We are sending copies of this report to the Secretary of Defense, the Secretary of Homeland Security, the Attorney General, and other interested parties. In addition, the report is available at no charge on GAO’s Web site at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-5431 or dagostinod@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix V. To identify the extent to which the Department of Defense (DOD) has developed or fielded non-lethal weapons (NLW) or capabilities since the NLW program’s inception, we obtained and analyzed the lists of developmental efforts and fielded items from the directorate and the services, and compared these with the lists in the joint services’ manual on NLW tactics, techniques, and procedures. We interviewed program management officials from the directorate as well as Marine Corps Systems Command, the Army Program Executive Office for Close Combat Systems, the office of the Chief of Naval Operations, the Under Secretary of the Air Force for Acquisition, and U.S. Coast Guard headquarters. We attended the spring and fall 2008 meetings of both the joint coordination and integration group and integrated product team group, and reviewed briefings prepared to support prior years’ meetings. We reviewed the products of the capabilities-based assessment conducted under JCIDS (for example the Functional Area Analysis, Functional Needs Analysis, and Joint Capabilities Document for Non-Lethal Capabilities) as well as the program information worksheets and investment decision support tool that the directorate uses to help it analyze priorities in light of identified gaps. To identify DOD non-lethal weapon program funding since 1997, we compiled and analyzed non-lethal weapon program budget information from the directorate and the services, reviewed DOD’s fiscal year 2009 budget submissions and future years defense program data. We also reviewed DOD and Joint Non-Lethal Weapons Directorate management guidance as well as DOD acquisition management criteria and federal internal control standards. Our use of the budget data was to provide context for our discussion. We concluded that the figures were sufficient to provide context for our discussion. However, since NLW funding information is not centralized, we were not assured that the identified funding amount allocated to NLW programs was comprehensive. To determine the extent to which DOD has established and implemented policy and doctrine, we reviewed and analyzed joint and service directives and other publications, and conducted interviews with cognizant officials in DOD, including the Office of the Under Secretary of Defense for Policy, Arlington, Virginia (both Office of the Assistant Secretary of Defense, Homeland Defense and America’s Security Affairs; and Assistant Secretary of Defense for Special Operations/Low Intensity Conflict and Interdependent Capabilities); National Guard Bureau, Operations (J34), Arlington, Virginia; Office of the Under Secretary of Defense, Acquisition, Technology, and Logistics, Arlington, Virginia; Office of the Deputy Commandant of the Marine Corps, Plans, Policies, and Operations, Arlington, Virginia; and Joint Non-Lethal Weapons Directorate, Quantico, Virginia. In addition, we held teleconferences with officials from U.S. Northern Command headquarters and U.S. Army Training and Doctrine Command Headquarters. We also interviewed Department of Homeland Security officials with the Science and Technology Division, Customs and Border Protection, and U.S. Coast Guard Headquarters, all in Washington, D.C. Also in Washington we interviewed Department of Justice officials within the National Institute of Justice. To determine the extent to which DOD has established and implemented NLW training, we also met with officials at the Army Non-lethal Scalable Effects Center at the U.S. Army Military Police School at Fort Leonard Wood, Missouri, and the Marine Corps, Inter-Service Non-lethal Individuals Weapons Instructor Course at Fort Leonard Wood, Missouri, and reviewed training materials including the training course manual. To determine the extent to which NLW have undergone testing and evaluation, we first reviewed overarching acquisition policy—which includes both DOD Directive 5000.1, The Defense Acquisition System and DOD Instruction 5000.2, Operation of the Defense Acquisition System—to ascertain test and evaluation guidelines for programs such as weapons that must be procured. We also reviewed the Risk Management Guide for DOD Acquisition and DOD test and evaluation guidance. We then compared the results of independent human effects assessment review panels with DOD test and evaluation guidance, compared DOD’s prefielding testing requirements with the documentation that recorded the tests actually performed, and compared Service urgent and standard fielding requirements. Service-specific fielding policy such as Army Regulation 700-142, Type Classification, Material Release, Fielding, and Transfer and Marine Corps Order 5000.23, Policy for the Fielding of Ground Weapon Systems and Equipment Policy provided information about what testing was required prior to fielding under various circumstances. For the few NLW fielded, we reviewed the status of test and evaluation master plans and relevant documentation as well as Human Effects Review Board assessments to determine if adequate testing was completed prior to fielding. In addition to meeting with DOD and services’ test and evaluation officials, we also interviewed officials at the Human Effects Center of Excellence to discuss NLW testing in detail. Except where noted, we limited our discussion of technology development to those items that were specifically designed to conform to the DOD definition of non-lethal weapons. To conduct our work, we interviewed officials in the following DOD organizations at the stated locations: Office of the Under Secretary of Defense for Policy in Arlington, Virginia Office of the Under Secretary of Defense, Acquisition Technology, and Logistics in Arlington, Virginia Office of the Assistant Secretary of Defense, Homeland Defense and America’s Security Affairs in Arlington, Virginia Office of the Assistant Secretary of Defense for Policy, Special Operations – Low Intensity Conflict in Arlington, Virginia Office of the Director, Operational Test & Evaluation in Arlington, Virginia Joint Staff, (J8) Force Application Engagement Division in Arlington, U.S. Special Operations Command in Tampa, Florida U.S. Central Command in Tampa, Florida U.S. Northern Command via teleconference National Guard Bureau, Operations (J34) in Arlington, Virginia Joint Non-Lethal Weapons Directorate in Quantico, Virginia Office of the Secretary of the Air Force, Assistant Secretary for Acquisition (Deputy Assistant Secretary for Science, Technology, and Engineering), Science and Technology Division, in Arlington, Virginia Air Force Security Forces Center in San Antonio, Texas Human Effects Center of Excellence personnel with the Air Force Research Laboratory in San Antonio, Texas Department of the Army Headquarters (G3/G8) in Arlington, Virginia We conducted our review from March 2008 through April 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix IV: Non-Lethal Programs under Development That Were Evaluated by the Investment Decision Support Tool Mk 19 Short Range Non-Lethal Munition Airburst Non-Lethal Munition, Low Velocity Vehicle Lightweight Arresting Device Single Net Solution / Remote Deployment Device Optical Warning Distraction and Suppression (OWDS) Mobile Active Denial System (ADS) 66mm Light Vehicle Obscuration Smoke System Grenades Human Electro Muscular Incapacitation X-26 (TASER®) uirement could be met by leveraging the U.S. Marine Corps’ Ocular Interruption and U.S. Navy’s Unambiguous Warning Device programs. OI has a schedule to reach a Milestone B decision in 2009 . uested in the latest Program Objectives Memorandum. In addition to the contact named above, Joseph Kirschbaum, Assistant Director; Sandra Burrell; Scott Clayton; Grace A. Coleman; James Driggins; David F. Keefer; Gregory Marchand; Sally Newman; Rae Ann Sapp; Rebecca Shea; and Jena Whitley made key contributions to this report.

Summary: Nonlethal weapons (NLW) provide an alternative when lethal force is undesirable. The Department of Defense (DOD) defines NLW as those that are explicitly designed and primarily employed to incapacitate personnel or materiel, while minimizing fatalities, permanent injury to personnel, and undesired damage to property and the environment. DOD created the Joint Non-Lethal Weapons Program in 1996 to have centralized responsibility for the development of NLW and coordinate requirements among the services. GAO was asked to review the status of NLW programs within DOD and the military services by identifying the extent to which (1) DOD and the Joint Non-Lethal Weapons Program have developed and fielded NLW since the program's inception; (2) DOD has established and implemented policy, doctrine, and training for NLW; and (3) DOD has conducted testing and evaluation prior to fielding NLW. GAO reviewed and analyzed DOD and service plans, guidance, and doctrine and interviewed officials associated with NLW development. The joint non-lethal weapons program has conducted more than 50 research and development efforts and spent at least $386 million since 1997, but it has not developed any new weapons and the military services have fielded 4 items stemming from these efforts that only partially fill some capability gaps identified since 1998. Three major factors contribute to the program's limited progress in fully addressing capability gaps. First, DOD did not prioritize departmentwide non-lethal capability gaps until 2007 and still does not fully address these gaps. Second, DOD has not consistently incorporated logistics and supportability considerations early in the development process. As a result, DOD may miss opportunities to allocate resources more effectively. Third, DOD has exercised limited general oversight of the NLW program which has resulted in gaps in key program guidance as well as limited measurement of progress and performance. For example, DOD's road map of ongoing and projected NLW capabilities and efforts could be used to discharge oversight responsibilities, but the road map lacks guidance about how to allocate resources and evaluate performance. Further, DOD has no single organization with visibility over all spending, and available budget information may not fully capture all spending associated with the development of non-lethal capabilities. DOD has begun to incorporate ideas about non-lethal capabilities into policy, doctrine, and training but has not yet clearly articulated what constitutes acceptable risk for fatality, fully developed weapons employment policies for the use of force in overseas warfighting or homeland applications, or ensured that warfighters and domestic responders are fully trained in NLW use. Until these issues are resolved, doctrine and training for non-lethal weapons may be limited, and the warfighter or domestic responder may have fewer options other than resorting to lethal force. DOD lacks a clear methodology for estimating the human effects of non-lethal weapons and does not fully test and evaluate many non-lethal weapons because they have been fielded under urgent operational requirements that abbreviate normal DOD testing standards. Testing can be bypassed for commercial items because DOD officials can use contractor test data instead of conducting their own tests. Therefore, when NLW are fielded, commanders are uncertain about acceptable risk on targets and bystanders and cannot accurately predict their effects. DOD has begun to develop elements of a risk assessment methodology to address human effects testing; for example, it has drafted a Risk of Significant Injury scale, which broadly categorizes levels of health care capabilities required to reverse NLW effects. However, DOD has not completed a risk assessment methodology that would provide information to commanders so that they may then make a determination about its acceptability in their operating environment.