The ACTD process is intended to be much more flexible and streamlined than DOD’s formal acquisition process and in turn to save time and money. Under the ACTD program, prototypes are developed and provide users with the opportunity to demonstrate and assess the prototypes’ capabilities in realistic operational scenarios. From these demonstrations, users can refine operational requirements, develop an initial concept of operations, and determine the military utility of the technology before deciding whether additional units should be purchased. Not all projects are selected for transition into the normal acquisition process. Specifically, potential users can conclude that the technology (1) does not have sufficient military utility and that acquisition is not warranted or (2) has sufficient utility but that only the residual assets of the demonstration are needed and no additional procurement is necessary. Separate technologies within one project may even have varied outcomes. DOD’s traditional approach to developing and buying weapons—which takes an average of 10 to 15 years—is marked by four phases: exploring various weapon concepts, defining what the specific weapon system will look like, refining plans through systems development and demonstration, and producing the equipment in larger-scale quantities and operating and supporting it in the field. Before a program can proceed to each phase, defense officials review its progress to evaluate the ability to meet performance goals and whether risk is under control. The ACTD process is marked by three phases: selection of the projects, demonstration of the technologies, and residual use of prototypes and/or the transition of them to acquisition programs if the services or defense agencies decide to acquire more. The selection process begins via a data call to both the research and development and warfighting communities. The “Breakfast Club,” a panel of technology experts from various organizations, reviews the potential candidates. Candidates selected by this panel are submitted to the Joint Requirements Oversight Council for prioritization and then to the Under Secretary of Defense for Acquisition, Technology and Logistics for a final selection. Decisions to move from stage-to-stage, are less formal than the formal acquisition process, and the process is managed by a set of Office of the Secretary of Defense (OSD) guidelines, which contain advice and suggestions, as opposed to formal directives and regulations. While ACTD teams are to prepare management plans for the projects that spell out roles and responsibilities, objectives, and approaches, these plans are supposed to be flexible, short (less than 25 pages), and high level. Figure 1 illustrates the major phases of the ACTD process. The ACTD demonstration phase typically lasts an average of 2 to 4 years, with an added 2-year residual phase. According to OSD, this provides ample time to develop fieldable prototypes and to allow users to evaluate them. For less complex systems or systems that are available quickly (e.g., commercial-off-the-shelf systems), the time line may be significantly shorter. Similarly, for very complex systems that require extensive integration and developmental testing, more time may be required. A key to keeping the time frame short, according to DOD, is beginning the demonstration with mature technology. This prevents delays associated with additional development and rework. The ACTD process places the highest priority on addressing joint military needs, although some ACTDs focus on service specific capabilities. For example, DOD has found that combat identification systems across the services needed to be enhanced to reduce fratricide so that systems belonging to individual services and components, and even allies, could work together more effectively. As a result, it undertook an ACTD project that tested new technology designed to improve the capability of combat forces to positively identify hostile, friendly, and neutral platforms during air-to-surface and surface-to-surface operations. Another ACTD project was designed to demonstrate the capability to conduct joint amphibious mine countermeasure operations. Recently, some ACTD programs have focused on enhancing homeland security with domestic agencies. For example, DOD is now testing a command and control system that will allow emergency personnel first responding to the scene of an attack to talk to each other and have a better situational awareness. ACTDs are funded by a variety of sources, including the office within OSD with the oversight responsibility for the ACTD program and the military services or defense agencies responsible for conducting the demonstrations and/or the transitions. In fiscal year 2001, a total of $546 million was budgeted for ACTDs—$120 million from OSD and $426 million from the services and agency partners. Participating combatant commands provide additional resources through their support of training, military exercises, and other resources. Funding to acquire and maintain additional units comes from service and agency budgets. Of the 24 projects we reviewed, 21 transitioned at least some technologies to users, meaning that users found that these had some level of military utility and that a military service or a defense agency chose to accept and fund their transition in the form of residual assets or as an acquisition. For 13 of these projects, the services or agencies decided to acquire more of the items tested, and as a result, transitioned the items into formal acquisition programs. Two of the 13 had no residual assets in use. For 8 projects, the services/agencies decided not to acquire additional items, but to continue using the residual assets. Three projects had no residual assets and no acquisition planned. However, some of these projects experienced mixed outcomes—e.g., some technologies may have ended up in residual use while others were acquired or rejected altogether or the lead military service may have rejected the technology while other components decided to acquire it. For example: The Counterproliferation I project consisted of a variety of technologies, including sensors, targeting systems, and advanced weapons, designed to find and destroy nuclear, biological, and chemical facilities. The technologies were used in military operations in Kosovo. For example, an improved infrared sensor that can assess bomb damage to facilities was accepted by the Air Force as an upgrade to its standard targeting pod. Two other technologies—a hard target- penetrating bomb and a fuzing system—have transitioned to production and are expected to achieve initial operational capability in fiscal year 2003. However, the project’s weapon borne sensor technology did not prove to be mature enough and was dropped from the ACTD prior to any demonstrations. The Link-16 project demonstrated an interoperability between the Link- 16 communications link and other variable message format systems to improve situational awareness, interdiction, surveillance, and close air support. No service has adopted it for formal acquisition, but some regional combatant commanders and lower-level commands have purchased additional systems. Since the system was not adopted across DOD, its utility could not be optimized. The Military Operations in Urban Terrain project field-tested 128 items designed to enhance operations in urban environments—such as attacking and clearing buildings of enemy troops. Of these, 32 technologies were determined to have merit and were kept as residual items to be further evaluated. Some of these have already transitioned or are planned for transition to acquisition programs, including a door- breaching round, a man-portable unmanned aerial vehicle, elbow and kneepads, explosive cutting tape, ladders, body armor, and flexible restraining devices. Though the majority of the projects we examined transitioned technologies to users, we identified a range of factors that hampered this process. Specifically: The technology has been too immature to be tested in a realistic setting, leading to possible cancellation of the demonstration. The military services and defense agencies have been reluctant to fund acquisition of ACTD-proven technologies, especially those focusing on joint requirements, because of competing priorities. Appropriate expertise has not been employed for demonstrations and transitions. Transition for software projects has not been adequately planned. DOD lacks specific criteria to evaluate demonstration results, which may cause acquisition decisions to be based on too little knowledge. At times, top-level support can overcome these barriers. But more systemic improvements focused on transition planning and funding commitment could reduce the need for high-level intervention. Figure 3 highlights the specific factors we identified. Because ACTDs are often conducted during large-scale, force-on-force military exercises, any new systems being tested must be dependable, able to perform as intended, and available on schedule in order not to negatively affect the exercises. As such, DOD has stressed that new technologies proposed for ACTDs should be “mature,” that is, they should have already been demonstrated to perform successfully at the subsystem or component level. The technology of the ACTDs in our sample was not always mature. In some cases, problems were fairly basic, such as a technology having inadequate power supply or being too heavy and bulky to carry out its intended operation. In other cases, technologies had not reached a point where they could be tested in a realistic setting, forcing users to forego certain parts of a test. For example: The Joint Countermine project tested 15 technologies, including detection systems and clearance/breaching systems. During demonstration, users found that detection technologies had unacceptably high false alarm rates and a mine and heavy obstacle clearing device was simply too heavy, bulky, slow and difficult to operate remotely. Moreover, several systems could not be demonstrated on their intended platforms, or even associated with a suitable substitute platform. Further, a number of critical operational sequences, such as launch/recovery, ordnance handling, and system reconfiguration, had not been demonstrated. As a result, only some technologies in this project have transitioned. The Consequence Management project examined 15 technologies designed to identify and respond to a biological warfare threat. During demonstration, users found that some of the items used to collect samples failed to operate and did not have sufficient battery capability and that switches broke. None of the other technologies performed flawlessly, and limitations such as size and weight made it apparent that they were not field ready. None of the technologies from this project entered into the acquisition process, nor did DOD continue to use any of the residual assets.