Land Mine Detection: DOD's Research Program Needs a Comprehensive Evaluation Strategy
Released: 2001-04-17T00:00:00

Letter

Background

Since the advent of modern warfare, the presence of mines and minefields has hampered the freedom of movement of military forces. The origins of mine warfare may be traced back to crude explosive devices used during the Civil War. Since that time, the use of land mines has increased to a point where there are now over 750 types of land mines, ranging in sophistication from simple pressure-triggered explosives to more sophisticated devices that use advanced sensors. It is estimated that there are about 127 million land mines buried in 55 countries.

Land mines are considered to be a valuable military asset since, by slowing, channeling, and possibly killing opponents, they multiply the combat impact of defending forces. Their attractiveness to smaller military and paramilitary organizations, such as those in the Third World, is further enhanced because they do not require complex logistics support and are readily available and inexpensive. Virtually every combatant can make effective mines, and they will continue to be a viable weapon for the future.

U.S. forces must be prepared to operate in a mined environment across the spectrum of military operations, from peacetime activities to large-scale combat operations. Detection is a key component of countermine efforts. In combat operations, the countermine mission revolves around speed and mobility. Mines hinder maneuver commanders’ ability to accomplish their missions because unit commanders need to know where mines are located so they can avoid or neutralize them. In peacekeeping operations, mines are used against U.S. forces to slow or stop daily operations. This gives insurgents a way to control traffic flow of defense forces and affect the morale of both the military and civilian population.

Since World War II, the U.S. military’s primary land mine detection tool has been the hand-held metal detector used in conjunction with a manual probe. This method is slow, labor intensive, and dangerous because the operator is in close proximity to the explosive. The Army has also recently acquired a small number of vehicle-based metal detectors from South Africa to be used in route clearing operations and to be issued to units, as needed, on a contingency basis.

Metal detectors are also sensitive to trace metal elements and debris, which are found in most soils. This limitation leads to a high level of false alarms since operators often cannot distinguish between a metal fragment and a mine. False alarms translate into increased workload and time because each detection must be treated as if it were an explosive. The wide use of mines with little to no metal content also presents a significant problem for metal detectors. For example, according to DOD intelligence reports, about 75 percent of the land mines in Bosnia are low-metallic and some former Yugoslav mines containing no metal were known to have been manufactured. In fact, the Army has stated that the inability to effectively detect low metal and non-metallic mines remains a major operational deficiency for U.S. forces.

Given the limitations of the metal detector, DOD has been conducting research and development since World War II to improve its land mine detection capability. For example, during the 1940s the United States began research to develop a detector capable of finding nonmetallic mines. Since then, DOD has embarked on a number of unsuccessful efforts to develop a nonmetallic detector and to field a vehicle-based land mine detector. DOD now has new programs to develop a vehicle-based detector and an improved hand-held detector. DOD expects to field these new systems, both with nonmetallic capability, within the next 3 years. Airborne detectors are also being developed by both the Army and the Marine Corps for reconnaissance missions to locate minefields.

Countermine research and development, which includes land mine detection, is funded by a number of DOD organizations and coordinated through a newly established Unexploded Ordnance Center of Excellence. The Army is designated as the lead agency for DOD’s countermine research, with most of its detection research funding being managed by the Night Vision and Electronic Sensors Directorate (NVESD) and the Project Manager for Mines, Countermine and Demolitions. The Marine Corps and the Navy are also supporting a limited number of land mine detection research efforts. Additionally, the Defense Advanced Research Projects Agency (DARPA) has been involved with a number of land mine detection programs throughout the years.

In fiscal years 1998 through 2000, DOD funded over $360 million in countermine-related research and development projects, of which approximately $160 million was aimed specifically toward land mine detection. DOD sponsored an additional $47 million in research during this period for unexploded ordnance detection (which includes land mines) in support of other DOD missions such as humanitarian demining and environmental cleanup. Because of the basic nature of detection, these other efforts indirectly supported the countermine mission. Overall, DOD funding levels for countermine research have been sporadic over the years. Major countermine research initiatives and fieldings of new detectors have coincided with U.S. military actions, such as the Korean War, the Vietnam War, Operation Desert Storm, and the recent peacekeeping operations in the Balkans. Following each influx of countermine research funding has been a corresponding lull in activity.

A countermine program assessment conducted for the Army in 1993 concluded that whereas mine developments have benefited from the infusion of leap ahead technologies, countermine tools have been essentially product improved counterparts of World War II ideas. However, according to DOD, countermine development is a slow process because of the technological challenges inherent to land mine detection. Not only must a detector be able to find mines quickly and safely through large variety of soils and at varying depths in battlefield conditions with clutter and even countermeasures, but it must also be able to discriminate between mines (which vary considerably in size, shape, and component materials) and other buried objects.

DOD Does Not Employ An Effective Strategy For Identifying Promising Land Mine Detection Technologies

DOD’s ability to develop meaningful land mine detection solutions is limited by the absence of an effective strategy to guide its research and development program. DOD maintains frequent contact with the external research community to constantly learn about new detection approaches and technologies. However, it has not developed a comprehensive set of mission needs to guide its research programs and does not systematically evaluate the broad range of potential technologies that could address those mission needs. In addition, its resources for conducting critical basic research for addressing fundamental science-based questions are threatened. Lastly, because DOD’s testing plans do not require adequate testing of land mine detectors in development, the extent of performance limitations in the variety of operating conditions under which they are expected to be used will not be fully understood.

DOD Has Not Adequately Specified Mission Needs to Guide Its Research

DOD has not developed a comprehensive and specific set of mission-based criteria that reflect the needs of U.S. forces, upon which to base its investments in new technologies in land mine detection. Although DOD’s overall acquisition process sets out a needs-based framework to conduct research and development, DOD has not developed a complete statement of needs at the early stages of research when technologies are first investigated and selected. The process calls for an evolutionary definition of needs, meaning that statements of needs start in very general terms and become increasingly specific as programs mature. Early stages of research are generated from and guided by general statements of needs supplemented through collaboration between the combat users and the research communities.

In the case of land mine detection, the Army stated a general need of having its forces be able to operate freely in a mined environment. This need has received a broad definition, as “capabilities for rapid, remote or standoff surveillance, reconnaissance, detection, and neutralization of mines.” Further specification of the need is left to representatives of the user community and researchers to determine. It is only with respect to specific systems at later stages of the acquisition cycle that more formalized and specific requirements were established to guide decisions about further funding.

Although we found that a comprehesive set of specific measurable criteria representing mission needs had not been developed, we did find some specific criteria in use to guide research efforts, such as rates of advance and standoff distances. However, a number of these criteria were established by DOD to reflect incremental improvements over the current capabilities of technologies rather than to reflect the optimal needs of combat engineers. For example, the Army was using performance goals to guide its forward looking mine detection sensors program. The objective of this program was to investigate and develop mine detection technologies to increase standoff and speed for route clearance missions beyond current capabilities. Performance goals included developing a system with a standoff of greater than 20 meters with a rate of advance of 20 kilometers per hour. However, these goals were primarily driven by the capabilities and limitations of the systems being considered. According to an Army researcher, they were based on what existing technologies could achieve in a limited time period (3 years) and not on what the combat engineers would ultimately need. During our assessment of technologies, which is described in the next section of this report, we found that the standoff desired by combat engineers was almost 50 meters for route clearance missions with a rate of advance of 40 kilometers per hour.

One barrier to DOD’s developing a comprehensive set of mission needs is large gaps in information about target signature characteristics and environmental conditions. For example, significant information gaps exist about the rate at which land mines leak explosive vapors and the environmental pathways that the vapors take once they are released. Also, knowledge gaps about soil characteristics in future battlefields limit DOD’s ability to fully specify mission needs and knowledgeably select among competing technologies. They also reduce the pace of technological innovation by hampering researchers from predicting how their devices will function. DOD is currently funding research to answer several important questions in these areas. But, as discussed below, continued DOD funding is threatened.

DOD Does Not Systematically Evaluate Technological Options for Land Mine Detection Solutions

Just as DOD has failed to adequately specify countermine mission needs for assessing promising technologies, we found that it had not systematically assessed the strengths and the limitations of underlying technologies to meet mission needs. DOD employs a number of mechanisms to obtain ideas for promising land mine detection solutions. These include attending and sponsoring technical conferences, arranging informal system demonstrations, convening workshops, and publishing formal solicitations for research proposals. However, DOD does not systematically evaluate the merits of the wide variety of underlying technologies against a comprehensive set of mission needs to identify the most promising candidates for a focused and sustained research program. Instead, it generally evaluates the merits of specific systems proposed by developers against time-driven requirements of its research programs.

One way DOD identifies land mine detection ideas is through sponsoring and attending international technical conferences on land mine detection technologies. For example, it sponsors an annual conference on unexploded ordnance detection and clearance, at which countermine related detection is a major focus. Additionally, DOD research officials have chaired mine detection conferences within annual sensing technology symposia of the International Society for Optical Engineering (SPIE) since 1995. The most recent SPIE conference on mine detection, held in April 2000, included over 130 technical presentations by researchers from DOD and other organizations worldwide. SPIE provides DOD land mine research officials an opportunity to network with researchers working in different areas of sensing technologies. DOD also identifies new technologies through reviewing researchers’ ideas outside of the formal solicitation process by occasionally allowing researchers to demonstrate their ideas at DOD facilities. Technical workshops are another mechanism used by DOD to identify new ideas. For example, DOD’s Unexploded Ordnance Center of Excellence held a workshop, in part, to identify new land mine detection technologies in 1998. This workshop, largely attended by DOD staff and contractors, explored technological approaches that were not receiving a lot of attention. The report of the workshop pointed out several potential paths for future investment for land mine detection.

Of all the mechanisms DOD uses to identify new technologies, issuing announcements in the Commerce Business Daily is its principal means for communicating its research needs to the outside research community and receiving ideas and approaches to improve land mine detection capabilities. In our interviews with non-government researchers, we found that they use DOD’s announcements as their principal means for familiarizing themselves about DOD’s needs. In connection with our efforts to identify candidate technologies for land mine detection, we searched databases, such as the Commerce Business Daily, containing DOD announcements. We found that the Army placed 20 of the 25 announcements we identified from 1997 through 2000. NVESD accounted for 17 of the solicitations.

Countermine research and development detection funding is concentrated on four primary technologies…There has been increasing emphasis on radar and active electromagnetics as the technologies showing the greatest short term promise for the reliable detection of land mines (emphasis added).

At NVESD, which has the largest share of countermine detection research, programs are generally time-limited. As a result, evaluations of proposals are largely based on the maturity of the idea. An example is the Future Combat Systems (FCS) Mine Detection and Neutralization program, which is funded at about $21 million over 3 years. This program is designed to have a system ready for testing by fiscal year 2002, only 3 years after the program started. This pace is necessary to meet the Army’s overall goals for fielding FCS. NVESD officials told us that this time constraint means they are more apt to fund the more mature ideas. This time constraint could therefore result in not selecting potentially promising technologies that might involve more risk. Although NVESD officials stated that they are receptive to less developed ideas that show promise, the requirements of the program may make this difficult to do.

We found that DOD did not supplement its frequent announcements with periodic reviews of the underlying technologies that the responses were based on. Such a review would evaluate their future prospects and could suggest a long-term sustained research program in a technological area that required several thrusts, whereas the individual project proposals might appear to have doubtful value in themselves. Along a similar vein, in 1998 a Defense Science Board task force that evaluated DOD’s efforts in a closely related area of research and development also recommended a two-track approach for research and development. The Board found that, “there has been too little attention given to some techniques which may provide capabilities important for particular sites” and recommended that DOD institute a program parallel to the “baseline” program that “would involve an aggressive research and development effort … to explore some avenues which have received too little attention in the past.”

DOD’s Productive Program of Fundamental Research is Threatened

Numerous questions about the physics-based capabilities of the various detection technologies make it difficult, if not impossible, to evaluate them against mission needs at the present time. Although DOD has invested funds in basic research to address some of its questions, its efforts are expected to end after fiscal year 2001. In addition to providing support to technology evaluations, a sustained basic research program is needed to support DOD’s ongoing efforts to develop better systems.

Independent evaluations, as well as our assessment of candidate land mine detection technologies, which is presented in the next section of this report, have revealed many uncertainties about the strengths and limitations of each of the applicable technologies with respect to addressing countermine mission needs. In addition, DOD has noted a number of fundamental science-based questions regarding detection technologies. For example, 3 years ago the Center of Excellence, through a series of workshops, identified 81 broad research needs critical to improving detection capabilities. Examples of research needs included an improved understanding of the impact of environmental conditions on many of the technologies examined and better characterization of clutter, which contributes to the problem of false alarms currently plaguing a number of technologies. Some of the needs have been addressed since the workshops. For example, the Center sponsored follow-on workshops and independent studies of radar and metal detectors to address research questions specific to these technologies. However, DOD officials told us that the broad set of needs has not been systematically addressed and that many questions still remain. Also, over the past 3 years, DOD has invested about $4 million annually in basic research directed at answering fundamental science-based questions supporting land mine detection. This work has been managed by the Army Research Office, with funding provided by both the Army and DOD through its Multidisciplinary University Research Initiative. However, this research program is expected to end after fiscal year 2001.

According to DOD, this basic research has been valuable to its land mine detection program. For example, the 1999 Center of Excellence annual report states that the basic research program has improved physics-based modeling so that it is now possible to examine realistic problems that include soil interactions with buried targets. The results of this modeling have yielded insights into limitations of sensor performance in various environments. The report concludes that this modeling work needs to be continued and expanded to systematically study soil effects. In fact, the report recommends continued investment in basic research to increase understanding of phenomenology associated with detection technologies, stating that the greatest value of basic research comes from a sustained effort.

Land Mine Detectors Are Not Subjected to Adequate Testing to Reduce Uncertainties

DOD’s policy is that systems be tested under those realistic conditions that most stress them. According to DOD, this testing is to demonstrate that all technical risk areas have been identified and reduced. However, because of questions about the physics-based strengths and weaknesses of land mine detection technologies, there is uncertainty about how well the detectors currently in development will function in the various environmental conditions expected in countermine operations. Some of these questions could be answered through thorough developmental testing. However, DOD’s testing plans do not adequately subject its detectors to the multitude of conditions necessary to address these performance uncertainties.

We reviewed the Army’s testing plans for two land mine detection systems currently in development to determine whether the test protocols were designed on a framework of identifying and minimizing technical risks stemming from the uncertainties detailed above. These are the Handheld Stand-off Mine Detection System (HSTAMIDS) hand-held detector and the Ground Stand-off Mine Detection System (GSTAMIDS) vehicle-based detector. We found that the testing plans were not designed around the breadth of environmental conditions expected for those systems or around anticipated limitations and uncertainties. Rather, testing is to be conducted at only a limited number of locations and under ambient climatic conditions. As such, knowledge about the performance of these detectors in the variety of soil types and weather conditions expected in worldwide military operations is likely to be limited.

For example, the performance of ground penetrating radar, a primary sensor in both the HSTAMIDS and the GSTAMIDS, is questionable in saturated soils, such as what might occur after a heavy rain. However, neither the HSTAMIDS nor GSTAMIDS testing plans specifically call for testing in wet conditions. The only way this condition would be tested is if there is heavy rain on or just before the days that testing is to occur. As such, knowledge about the performance of these detectors in a variety of conditions is likely to be limited.

It is Uncertain Whether DOD is Investing in the Most Promising Technologies For Land Mine Detection

Incomplete knowledge of the properties of candidate land mine detection technologies makes it difficult to assess whether DOD is investing in the most promising technologies to address countermine detection missions. Because DOD had not performed a systematic assessment of potentially applicable technologies against military countermine mission needs, we performed our own evaluation. Through a broad and systematic review of technological candidates, we identified nine technologies with potential applicability, five of which DOD is currently exploring. However, insufficient information about these nine technologies prevented us from definitively concluding that any could address any of the missions. Additionally, because of these uncertainties, we could not conclude whether a “sensor fusion” approach involving a combination of two or more of the technologies would yield an adequate solution.

We conducted a broad search for potential technological candidates for solutions to the countermine problem, and then evaluated the candidates against a set of mission-based criteria to determine which candidates were promising for further research. A more detailed description of our methodology is presented in appendix I. For criteria, we identified operational needs for each of five different types of critical countermine missions: (1) breaching, (2) route clearance, (3) area clearance, (4) tactical reconnaissance, and (5) reconnaissance supporting stability and support operations during peacetime. A more detailed description of these missions is presented in appendix II.

We then developed a set of technical criteria to specifically define detection requirements for each mission. The criteria we developed were based on target parameters, operational parameters,and environmental parameters. Target parameters describe the physical characteristics of land mines and the methods by which they are emplaced. These include such characteristics as land mine sizes and shapes, metallic content, explosive content, burial depths and the length of time mines have been buried. Operational parameters describe the operational needs of the military as they relate to countermine operations involving mine detection. These factors include speed of advance, detection distance from the mine (called stand-off),and the level of precision in identifying the exact position of hidden mines. Target and operational parameters can vary among the five types of missions. Environmental parameters, unlike target and operational parameters, do not vary based on the type of mission. Rather environmental parameters are site-specific. They are natural and man-made conditions in and around the battlefield that affect mine detection. These parameters cover a wide array of atmospheric, surface, and sub-surface environmental conditions, such as air temperature, dust or fog obscuration, surface snow, varying soil types and post-blast explosive residue. A more detailed description of the criteria used in our evaluation is presented in appendix II.

Our search yielded 19 technological candidates, which span a wide variety of different physical principles and are shown in figure 1.

As shown in figure 1, the majority (15) of the technologies use energy from the electromagnetic (EM) spectrum, either to detect emissions from the mine or to project energy at the mine and detect a reflection. The energies used in these technologies span the entire EM spectrum, from radio waves (characterized by long wavelengths/low frequencies) to gamma rays (short wavelengths/high frequencies). Of the remaining four technologies not directly utilizing EM energy, two (biosensors and trace vapor detectors) operate by using a chemical or biological reaction to detect explosive vapor that is emitted from mines into the surrounding soil or the air directly above the ground. Another one is based on sending neutrons toward the target. The last technology works by sending acoustic or seismic energy toward a target and receiving an acoustic or seismic reflection. A more detailed discussion of these 19 technologies is included in appendix III.

When we evaluated the 19 technologies against the operational parameters, we found that 10 had one or more physics-based limitations that would prevent them from achieving any of the five countermine missions by themselves (see table 1). As can be seen from table 1, standoff and speed are the most challenging attributes of a detection system that would meet DOD’s countermine mission needs. Nine technologies failed to meet the standoff criterion, and four failed to meet the speed criterion for any of the five missions.

We judged that the remaining nine technologies were “potentially promising” because we did not conclusively identify any definitive operational limitations to preclude their use in one or more countermine missions. For all of these nine technologies, our ability to determine their operational capabilities was reduced by significant uncertainty as to their capabilities. Some, such as ground penetrating radar and acoustic technologies, have been studied for many years. Yet continuing improvements to the sensors and the critical mathematical equations that interpret the raw data coming from the sensors made it difficult for us to predict the absolute limits of their capabilities. Our inability to draw a conclusion about these technologies is supported by reports from the Institute for Defense Analyses and other organizations that have found similar uncertainty about their prospects. The critical issue for radar is whether it will ever be capable of doing a good enough job discriminating between targets and natural clutter to allow an acceptable rate of advance. The issue of clutter is the fundamental problem for many sensor approaches.

Our uncertainty about three technologies, terahertz imaging, x-ray fluorescence and electromagnetic radiography was different because their capabilities were not as well-studied. As a result, there was not enough information for us to determine whether they could meet mission-based criteria. In addition, DOD officials told us that they believe that two of them (terahertz imaging and x-ray fluorescence) have fundamental limitations that rule them out for countermine missions. They claimed that terahertz energy is unable to penetrate deep enough through the soil and that x-ray fluorescence has inadequate standoff. However, we were not able to resolve these issues.

We believe that the lack of consensus about the capabilities of most of the nine technologies is due, in part, to a basic lack of knowledge about the upper limits of their capabilities. The only way to determine whether these technologies can be employed in a detector that meets countermine mission needs is through a systematic research program.

DOD is currently investing in five of the nine technologies (see table 2), and it recently stopped funding a project in one of them (passive millimeter wave).

In our review of the ability of the nine technologies to operate in different environmental conditions, we could not, with certainty, identify absolute limitations on the ability of four to operate in expected environmental conditions. However, all nine have uncertainties about the range of environmental conditions in which they can adequately perform. The most significant uncertainties relate to performance in various surface and subsurface conditions, such as water saturated soil and differing soil types. In most cases, these uncertainties have not been adequately studied. Examples of environmental limitations and uncertainties for the nine technologies are presented in table 3.

The uncertainties about the various detection technologies also prevented us from determining if the technologies could be combined to meet mission needs. While most of the 19 technologies cannot meet operational and environmental mission needs, in theory a combination of different sensors might solve the countermine problem. This type of arrangement, known as sensor fusion, combines different approaches to compensate for the limitations of them individually. Canada and the Army are developing systems that use some form of sensor fusion. Canada’s Defense Research Establishment in Suffield, Alberta, has produced a multisensor land mine detector that employs thermal neutron activation (TNA), a type of neutron activation analysis, as a confirmation detector in a system that also employs a metal detector, infrared (IR), and ground penetrating radar to scan for mines. The TNA sensor is used to confirm or reject suspect targets that the three scanning sensors detect. The Army is developing a detector (HSTAMIDS) that uses sensor fusion to take advantage of the strengths of both metal detector and radar approaches. In this configuration, the radar is used to improve the metal detector’s performance with mines that employ small amounts of metal. However, neither of these systems (Canada’s and the Army’s) will meet the countermine mission needs stated previously because their component sensors are limited. Any detection system utilizing sensor fusion would somehow need to overcome limitations, such as standoff and speed, in underlying technologies. As pointed out previously, the capability of the identified technologies to meet mission needs is uncertain. Another consideration in developing a sensor fusion solution is that it would require significant advances in signal processing.

Conclusions

It is unclear whether DOD’s research investments are in those technologies that, either individually or in combination, have the greatest chance of leading to solutions that address the U.S. military’s countermine mission needs given the lack of knowledge about the strengths and the limitations of the various detection technologies. DOD’s strategy of working toward incrementally improving capabilities over current detectors may result in improvements over current capabilities. However, without a systematic and comprehensive evaluation of potential technologies based on a complete set of mission-based needs, DOD does not know if it has invested its funds wisely to address the needs of the military.

DOD’s testing plans for its land mine detection systems in development do not provide assurance that these systems will perform adequately under most expected conditions. Demarcating the acceptable operating conditions of a system is a critical part of research and development. This is important not only for determining if developmental systems will meet mission needs but also for defining the operational limitations so that users can make informed decisions about their use. Therefore, systems should be tested under those conditions that most stress them. Given the numerous environmental and climatic conditions that can be expected to affect the performance of any land mine detector, a robust program of developmental testing is essential to fully understand the strengths and limitations in performance under realistic conditions. Failing to test under a plan specifically designed around the expected environmental and climatic conditions of use as well as the anticipated limitations of the technologies could increase the risk of fielding the system.

Recommendations For Executive Action

To improve the Department’s ability to identify and pursue the most promising technologies for land mine detection, we recommend that the Secretary of Defense (1) direct the establishment of a long-range research program to periodically evaluate all applicable land mine detection technologies against a complete set of mission-based criteria and (2) provide a sustained level of basic research to sufficiently address scientific uncertainties. Mission-based criteria could include target signatures, operational requirements, and expected environmental conditions. We also recommend that the Secretary of Defense require the services to provide adequate testing conditions for land mine detection systems in development that better reflect the operating environment in which they will likely have to operate.

Agency Comments And Our Evaluation

DOD provided written comments on a draft of this report (see app. IV). DOD concurred with each of our three recommendations and augmented its concurrence with additional comments. DOD’s comments describe and illustrate the lack of a focused and systematic approach underlying DOD’s research programs for land mine detectors. It is not clear from DOD’s response what, if any, measures it plans to take to implement our recommendations.

In responding to our first recommendation, DOD states that the Army pursues a systematic research, development, and acquisition program to address land mine detection needs. However, we found that its approach lacked elements critical to the success of this program, such as the use of a comprehensive set of mission-based criteria and a systematic evaluation of the capability of competing alternative technologies to address these criteria. In fact, the Army Science Board study cited by DOD in its comments to us also recommended that “operational needs and priorities need to be clearly thought through and quantified.” There is nothing in DOD’s comments that is directed toward bridging these gaps. Therefore, we continue to believe that the changes that we have recommended are required.

Regarding our second recommendation, DOD describes the benefits provided by its current basic research program, but does not commit to continuing funding for basic research for land mine detection after this fiscal year. As we discuss in this report, we believe it is extremely important for DOD to continue with a sustained program of basic research to support its land mine detection program given the extent of the uncertainties surrounding the various technologies. This point was also made by the Army Science Board panel.

In response to our third recommendation, DOD states that the testing plans we reviewed were not detailed enough to allow us to reach our conclusions, and it describes certain activities that it is engaged in to incorporate realistic environmental conditions into its testing programs for HSTAMIDS and GSTAMIDS. However, we believe that the described activities further illustrate the lack of a systematic strategy to guide testing during product development. DOD acknowledged the threat to the performance of metal detectors from soils that are rich in iron oxide and pointed out that it is seeking to identify a “suitable site to test the HSTAMIDS system in unique soil environments such as laterite.” We feel that this is an important step in the development of this system. But we believe that this step, along with tests in saturated soils and snowy conditions, should have been taken much earlier, before a large commitment had been made to this system. Testing programs should also be driven by a systematic mission-based evaluation framework. Such an approach should delineate at the earliest stages of development the expected environmental operating conditions based on mission needs. An analysis should then be made to identify for testing those conditions that pose substantial challenges or uncertainties for detector performance. Without such a framework, there is a risk that uncertainties about the performance of these systems will remain after they have been fielded and that significant testing will ostensibly be conducted by users rather than by testers.

We are sending a copy of this report to the Honorable Mitchell E. Daniels, Jr., Director, Office of Management and Budget; the Honorable Donald H. Rumsfeld, Secretary of Defense; the Honorable Joseph W. Westphal, Acting Secretary of the Army; the Honorable Robert B. Pirie, Jr., Acting Secretary of the Navy; General James L. Jones, Commandant of the Marine Corps; and other interested congressional committees and parties. We will also make copies available to others upon request.

Please contact me on (202) 512-2700 if you or your staff have any questions concerning this report. Major contributors to this report were Kwai-Cheung Chan, Dan Engelberg, Cary Russell, and John Oppenheim.

Appendix I: Scope And Methodology

To determine whether the Department of Defense (DOD) employs an effective strategy for identifying the most promising land mine detection technologies, we reviewed literature related to research program design and met with experts in this area. We interviewed officials from the Army, the Navy, the Marine Corps and the Defense Advanced Research Projects Agency (DARPA) responsible for running land mine detection research programs. We also reviewed DOD policy and doctrine related to this area including the Defense Technology Area Plan, the Army Science and Technology Master Plan, and Countermine Modernization Plans.

To determine whether DOD is investing in the most promising technologies to fully address mission needs, we evaluated the set of potential land mine detection technologies identified through a systematic search against a set of criteria derived from mission needs. We first designed a framework for evaluating potential technologies. This framework assisted in identifying the most promising technologies and research gaps for further investigation. Through our discussions with DOD, we found out that such a framework had not previously been created.

Because our framework was mission directed, we identified a set of critical countermine missions that involve detecting land mines by systematically interviewing Army and Marine Corps combat engineers to determine how countermine activities fit into a variety of combat scenarios and reviewing Army and Marine Corps doctrine that discuss mine threats to U.S. forces and corresponding countermine tactics. Next, through a review of documents and discussions with Army and Marine Corps combat engineers, we identified technical criteria that define detection requirements for each mission. Officials representing the two organizations responsible for combat engineer requirements, the Army Engineer School and the Marine Corps Combat Development Command, reviewed and agreed with the set of criteria we developed. The critical missions and the set of criteria we developed are discussed in appendix II.

We then identified conventional and alternative technologies that could have value in terms of performing these land mine detection missions. We distinguished between technologies and systems. “Technologies are approaches by which principles of physics are exploited to achieve tasks.”

Systems are implementations of technologies. By developing a methodology that was based on identifying and characterizing technologies, rather than systems, we sought to go beyond the strengths and limitations of current devices and thereby provide information on which to base a future-oriented research program. We identified candidate technologies in three ways: One way was to review literature on land mine detection and interview researchers and other experts in the land mine detection field. Another way was to interview experts in related fields, such as geophysics and civil engineering, that involve similar activities (i.e., looking for hidden subsurface objects). In this, our goal was to find out if those fields use any tools that have not been explored by DOD. The final way was to review proposals that had been submitted to DOD in response to recent solicitations for funding. The technologies we identified are presented in appendix III.

We evaluated each of the identified technologies against the set of mission criteria to determine which were promising for land mine detection. We identified “potentially promising” technologies by eliminating those that have limitations that would preclude their meeting mission goals. In performing this evaluation, we attended conferences and workshops, reviewed published and unpublished technical literature, interviewed developers of land mine detection systems, and contracted with an expert in the field of land mine detection technologies to review our conclusions. We also obtained comments from technical experts from the Army. Finally, we determined which of the “potentially promising” technologies DOD was exploring by reviewing agency documents and interviewing DOD officials.

We performed our work from November 1999 to February 2001 in accordance with generally accepted government auditing standards.

Appendix II: Land Mine Detection Mission Requirements

Using our methodology, we identified land mine detection requirements. The five critical countermine missions that involve land mine detection are (1) breaching, (2) route clearance, (3) area clearance, (4) tactical reconnaissance, and (5) stability and support operations (SASO) reconnaissance. Breaching is the rapid creation of safe paths through a minefield to project combat forces to the other side. This mission is usually conducted while the force is under enemy fire. Route clearance is the detection and removal of mines along pre-existing roads and trails to allow for the passage of logistics and support forces. Area clearance is the detection and removal of mines in a designated area of operations to permit use by military forces. Tactical reconnaissance is performed to identify mine threats just prior to and throughout combat operations. SASO reconnaissance is used to assist in making decisions about where to locate forces and for planning area clearance operations. A principal difference between tactical and SASO reconnaissance is the time required for performing the mission. Because SASO reconnaissance involves peacetime operations, the speed at which it is conducted is not as critical as that for tactical reconnaissance.

We developed a set of technical criteria to specifically define detection requirements for each mission and grouped the criteria into target parameters, operational parameters, and environmental parameters. Target parameters describe the physical characteristics of land mines and the way they are emplaced. Given that there are over 750 types of land mines available worldwide, the target characteristics vary considerably. The parameters we identified are presented in table 4.

Operational parameters describe the operational needs of the military as they relate to countermine operations involving mine detection. Our set of operational parameters are also presented in table 4. One critical operational criterion for a mine detector is speed of advance. For time critical missions, like breaching and route clearance, a detector needs to function effectively at the military forces’ operational speeds. The ability of a detector to keep up with the required rate of advance is dependent on two factors: its scanning speed (the time to search a given area for mines) and its false alarm rate, which is based on the number of times a detector indicates the presence of a mine where one does not exist. False alarms reduce the rate of advance because combat forces must stop to confirm whether an alarm is actually a mine.

Another key operational parameter is standoff, which is the distance a mine detector (and its operator) can be from a mine and still be able to detect it. The minimum standoff required is the lethal radius of a mine, which is about 35 meters (for an antitank sized mine). This distance requirement increases as speed increases to allow for reaction time once an alarm is sounded. In cases of minefield reconnaissance performed by airborne detectors, the standoff required is the minimum altitude necessary to provide safety for the aircraft from enemy ground fire. One final operational parameter is the ability of a detector to accurately locate the position of a buried mine. This is important for reducing the time necessary to remove or otherwise neutralize the mine and the safety risk associated with manually probing the ground to find the exact mine position.

The environmental parameters we identified are presented in table 5. These are natural and man-made conditions in and around the battlefield that affect mine detection and are grouped into atmospheric, surface, subsurface, and other environmental conditions. While the target and operational parameters can vary among the five mission types, the environmental parameters are not mission-specific. Rather environmental parameters are site-specific.

Appendix III: Land Mine Detection Technologies

In this appendix, we briefly describe the land mine detection technologies and projects that we identified through our methodology. We grouped the individual projects and lines of effort on the basis of their underlying technological approach. Our grouping resulted in 19 distinct approaches.

These technologies vary in their maturity. Some, such as metal detectors and radar, have been explored by many researchers for many years. Much less is known about others such as electromagnetic radiography and microwave enhanced infrared. Others, such as x-ray fluoresence, have been used in other applications but have received relatively little attention thus far in this application.

The technologies use different principles. Fifteen of the 19 technologies are based on receiving electromagnetic (EM) energy from the target.Eleven of the 15 EM technologies are based on sending energy (in one case energy in the form of neutrons) into the ground. The remaining four EM technologies are “passive electromagnetic”; they are based on receiving energy that is emitted by the land mine. These four technologies are similar in principle; their relative strengths and limitations with respect to addressing countermine missions arise from the different types of energy that they receive. The final 4 of the 19 technologies are primarily not electromagnetic. Two capture and analyze the explosive that the mine releases into the ground or air, one is based on acoustic or seismic energy reflected off of the target, and one is based on sending neutrons toward the target.

Technologies That Are Based on Sending Electromagnetic Energy

Eleven technologies use electromagnetic energy and operate under three different approaches (see fig. 2).

Those that react with the explosive Those that react with the explosive 1. Conductivity/resistivity 2. Metal detectors 1. Electromagnetic radiography 2. Gamma ray imaging 3. Microwave enhanced infrared 4. Quadrupole resonance 5. X-ray fluorescence Four operate by sending EM energy into the ground, reflecting off the mine.

Five operate by sending EM energy into the ground, creating an effect on the explosive substance. Whereas four of the five act on the explosive within the mine casing, one relies on detecting released explosive molecules.

Two operate by detecting differences in the low frequency electromagnetic field around the mine.

Technologies That Reflect Energy Off the Mine

Four of the 11 active EM technologies (radar, terahertz imaging, LIDAR, and x-ray backscatter) are based on projecting energy into the ground and reflecting off the land mine. The presence of a mine or other buried object is detected from differences in the electromagnetic properties of the target and those of the surrounding ground. The relative strengths and limitations of these technologies vary with their wavelengths. Managing the trade-off between depth of penetration and resolution is one of the central research concerns in this area. The choice of frequency is important; lower frequencies allow better ground penetration but will suffer from poor spatial resolution. Radar’s relatively long wavelength (it operates in the microwave part of the electromagnetic spectrum) allows it to penetrate the ground deeply enough to reach buried mines. This ability, along with the fact that it can detect plastic mines, has made radar the focus of much research and development in the United States and in other nations. For example, DOD has incorporated radar into its hand-held system, Handheld Stand-off Mine Detection System (HSTAMIDS). However, whether a system based on radar will meet countermine mission needs remains in dispute. The poor spatial resolution of radar, which makes it difficult at best to distinguish between buried mines and other objects of a similar size and shape, is the largest obstacle. Another issue is its inability to penetrate soils that are saturated with water.

The other technologies have greater resolution but have a corresponding loss of depth penetration. Because LIDAR has a shorter wavelength than radar, it has a limited ability to detect buried mines. X-ray backscatter can provide detailed images of shallowly buried mines due to the extremely short wavelength of the x-rays. It operates by detecting the difference in the atomic number between the ground and the mine target. However, the applicability of this technology is limited due to the limited penetration of the x-rays into the ground. In theory, terahertz imaging should have a similar limitation. However, a researcher studying the feasibility of creating images of mines in the terahertz part of the spectrum told us that his system might be able to penetrate more deeply by increasing the power of the energy.

Technologies that create a detectable reaction with the explosive

Another general approach involves projecting energy into the ground that reacts with the molecules of the explosive, which send a signal that is received by the detector. Because it reacts with the explosive, rather than the container, the approach has the advantage of more specifically targeting land mines and being less prone to the clutter problem that hinders other active electromagnetic approaches. However, technologies that adopt this approach tend to be more complex and expensive.

We identified five distinct technologies that have been advanced that utilize this general approach. One of them, quadrupole resonance, is a relatively mature technology in land mine applications and systems have been built around it. Less is known about the other four technologies and how to apply them to detect land mines and what their capabilities are for addressing countermine missions. These four are electromagnetic radiography, microwave enhanced infrared, x-ray fluorescence, and gamma ray imaging. Therefore, our assessments are less complete for these than for the other more well-studied approaches.

Quadrupole resonance has been explored for identifying explosives for several years. Much of the basic research was conducted at the Naval Research Laboratory. Quadrupole resonance detectors are also being developed to screen for explosives at airports. In quadrupole resonance, a pulse of long wavelength energy causes the nitrogen nuclei in the explosives to emit a pulse of energy that is characteristic of the molecule. For example, the nitrogen atoms in TNT emit a unique pulse that can be picked up by the detector. One limitation of quadrupole resonance with respect to countermine missions is that the detector head must be close to the target. The speed at which quadrupole resonance can operate is in question. Current systems are fairly slow. In addition, research questions currently exist in several areas, including how to overcome interference from other sources of energy and how to configure a quadrupole resonance detector to detect TNT. Despite these limitations and questions, DOD is developing systems that use this technology. The Marine Corps is developing a hand-held device that uses quadrupole resonance and the Army is developing a land mine detection vehicle that would use an array of quadrupole resonance detectors across the front to confirm targets presented by sensors that use either radar or metal detector.

In conversations with individual systems developers, we identified four other examples of this land mine detection approach. The first two technologies are based on scanning the ground with long wavelength microwaves. This energy excites the explosive molecules that emit a signal that is detected. The other two technologies using this approach send shorter wavelength energy toward the target.

Electromagnetic radiography operates by scanning the ground with long wavelength microwaves. According to one developer, when it is struck by this energy; the target radiates back in a particular way; exciting molecules at atomic levels. The molecules respond with spin effects that produce “a spectrographic signature of the target substance.” As noted previously, very little is known at the present time about what the limits are of this technology in terms of the operational requirements and environmental conditions for countermine applications.

Microwave enhanced infrared detection operates by sending long wavelength microwaves into the ground and then detecting a “unique thermal signature and infrared spectra of chemical explosives.” One limitation with this approach is it cannot be used to detect metallic mines because the microwave energy cannot penetrate metal. In addition, the speed at which it can operate and the standoff distance are both highly uncertain.

The third technology illuminates the ground with x-rays that causes a series of changes in the electron configuration of the target atoms that results in the release of an x-ray photon (x-ray fluorescence). Unlike the other technologies in this category x-ray fluorescence detects molecules of explosive that are emitted from the mine. The amount of fluorescence is dependent on the target molecule. A critical issue in dispute at the present time is whether x-ray fluorescence can work at the distances required to address countermine missions The short wavelength of the x-rays used has a corresponding high degree of scattering. Several experts we spoke to expressed reservations about standoff for this technology, although the system developer claims to have surmounted this limitation.

The fourth technology is gamma ray imaging. The basis of this technique is an electron accelerator that produces gamma rays that “interact with the chemical elements in explosives to generate a unique signature.” Because of the scattering of the (short wavelength), x-ray and gamma ray detectors operating on these principles must be in close proximity to the target.

According to a developer, the detector must be within one foot of the target. Another obstacle is that the detector would require an extremely large source of energy to create the gamma rays.

Technologies that detect an electromagnetic field

We identified two technologies that are based on detecting an electromagnetic field.

The first is electromagnetic induction. As discussed in the background section, metal detectors that utilize this approach are the principal means for detecting land mines at the present time. Metal detectors generate a magnetic field that reacts with electric and/or magnetic properties of the target. This reaction causes the generation of a second magnetic field, which is received by the detector. The restriction to metallic objects is a limitation given the increasing development of mines with extremely small amounts of metal. Increasing the sensitivity of a metal detector to detect extremely small amounts of metal in these mines leads to its detecting other objects in the ground. Metal detectors are also limited by the need to be relatively close to the mine target in order to operate effectively.

The second technology is conductivity/resistivity that involves applying current to the ground using a set of electrodes and measuring the voltage developed between other electrodes. The voltage measured at the electrodes would be affected by the objects in the ground, including land mines. The conductivity technique was originally developed to locate minerals, oil deposits, and groundwater supplies. The need to place the electrodes in or on the ground is a concern for land mine detection applications of this technology.

Passive Electromagnetic Technologies

We identified four technologies that have been proposed which do not actively illuminate the target, but are based on detecting energy emitted or reflected by the mine. Three detect the energy naturally released by objects. They are ostensibly cameras that operate in a very similar fashion to video cameras, although they view not red, green, and blue frequencies, but other parts of the spectrum. Land mine detectors that use passive sensing principles spot either (1) a contrast between the energy emitted or reflected from the mine and that of the background or (2) the contrast between the (disturbed) soil immediately surrounding a buried mine and the top layer of soil. They can be designed to pick up this energy difference in different wavelength bands. Passive detectors have been designed or proposed to operate in different parts of the EM spectrum. We identified technologies that operate using infrared, millimeter wave, and microwave principles. Infrared, millimeter, and microwave techniques have different strengths and limitations. The trade-offs between scattering and resolution that exist with the active backscatter approaches (radar and LIDAR) also exist for passive EM technologies. For example, the longer wavelengths of microwave and millimeter waves allow them to penetrate through clouds, smoke, dust, dry leaves, and a thin layer of dry soil but provide more limited resolution of targets.

These four technologies are capable of greater standoff than others. Several nations are developing systems that use IR detection to detect minefields (tactical reconnaissance). Systems are also being developed to gather information in several infrared wavelength bands at the same time (“multi-spectral infrared”). This approach increases the amount of information available to distinguish mine targets from the background. The Marine Corps is conducting research in this area.

One of the constraints with infrared detection systems is that the mines’ signature against the background will tend to be reduced at certain times during the day. To overcome this limitation, researchers funded by DOD’s Multidisciplinary University Research Initiative (MURI) recently investigated amplifying the infrared signal by heating the ground with microwave energy. Their early findings suggest that microwave heating enhances the infrared signature of objects buried under smooth surfaces. However, much work remains. Given continued funding, they plan to add increasing complexity to their experimentation by testing with rough surfaces, random shapes, and different mine and soil characteristics. They will need to conduct additional research to determine whether the rate of heating is consistent with the speed required to meet most countermine missions.

The fourth passive electromagnetic approach is based on detecting the energy produced by the circuitry of advanced mines that contain sophisticated fuses. DOD has recently funded work on this approach as part of the MURI initiative. Apart from the limited applicability of this technology, questions remain concerning how feasible it is and how easily a detector operating on these principles might be fooled with a decoy.

Other Technologies

We identified four technologies that are not based on electromagnetic principles. They are acoustic/seismic, neutron activation, trace vapor and biosensors. Sensors that utilize an acoustic/seismic approach operate by creating an acoustic or seismic wave in the ground that reflects off the mine. The energy can be delivered in a number of different ways such as a loudspeaker, a seismic source coupled with the ground, and a laser striking the ground over the mine. In addition, there are different ways of receiving the signal from the target (electromagnetically through a doppler radar or doppler laser device or acoustically through a microphone). Numerous questions remain about whether an acoustic/seismic approach can meet the operational needs for countermine missions and the environmental factors that would influence its employment.

Although we identified no certain, absolute limitations to an acoustic/seismic approach meeting countermine missions, we did identify significant concerns. Acoustic waves are capable of imaging buried land mines. However, clutter is a major concern with acoustic approaches. Interference from rocks, vegetation, and other naturally objects in the environment alter the waves as they travel in the ground. Additional work needs to be conducted to assess the limits of an acoustic/seismic approach for detecting land mines. An acoustic system is one of the technologies that the Army is currently exploring for the Ground Stand-off Mine Detection System (GSTAMIDS).

Neutron activation analysis techniques operate on the principle that mine explosives have a much higher concentration of certain elements like nitrogen and hydrogen than naturally occurring objects. There are several neutron-based techniques for detecting these explosive properties in bulk form. All systems are composed of at least a neutron source – continuous or pulsed, emitting in bursts – to produce the neutrons that have to be directed into the ground, and a detector to characterize the outgoing radiation, usually gamma rays, resulting from the interaction of the neutrons with the soil and the substances it contains (e.g. the explosive). Neutron activation analysis cannot be used as a standoff detector. Our review indicated that neutron activation analysis must operate directly over the mine target. The limited speed of this technology is another restriction for most missions. In addition, unanswered questions about this technology concern the depth of penetration and whether it can be used to detect smaller anti-personnel mines. Because of these limitations and questions, neutron activation analysis is currently envisioned as having a role as a confirmation detector alongside faster sensors on systems that are remotely piloted. For example, as described above, Canada’s military has developed a vehicle that incorporates thermal neutron activation as a confirmation sensor. The vehicle would need to stop only when one of the scanning sensors indicated a possible mine target.

The other two technologies are trace vapor and biosensors. Trace vapor detectors involve sensing molecules of the explosive that emanate from the buried mine and then analyzing them. There are several different approaches for capturing and analyzing these molecules. In 1997, DARPA initiated a research program aimed at detecting land mines via their chemical signatures, referred to as the “electronic dog’s nose” program. The program was established because DARPA believed that the technologies DOD was developing (metal detectors, radar and infrared) were limited in that they were not seeking features unique to land mines and were susceptible to high false alarm rates from natural and man made clutter. Through this program, DARPA hoped to change the overall philosophy of mine detection in DOD by detecting the explosive, a unique feature of land mines. This work has been transitioned over to the Army. However, the role of trace vapor detectors in most countermine missions is likely to remain limited due to the limited standoff that can be achieved. The central feature of the biosensor technology approach is a living animal. Current examples of biosensors are dogs, bees, and microbes that detect explosives. Many research questions remain with these approaches.

Appendix IV: Comments From the Department of Defense

Bibliography

Andrews, Anne et al., Research on Ground-Penetrating Radar for Detection of Mines and Unexploded Ordnance: Current Status and Research Strategy, Institute for Defense Analyses, 1999.

Bruschini, Claudio and Bertrand Gros. A Survey of Current Sensor Technology Research for the Detection of Landmines, LAMI-DeTeC. Lausanne, Switzerland, 1997.

Bruschini, Claudio and Bertrand Gros. A Survey of Research on Sensor Technology for Landmine Detection. The Journal of Humanitarian Demining, Issue 2.1 (Feb. 1998).

Bruschini, Claudio, Karin De Bruyn, Hichem Sahli, and Jan Cornelis. Study on the State of the Art in the EU Related to Humanitarian Demining Technology, Products and Practice. École Polytechnique Fédérale de Lausanne and Vrije Universiteit Brusel. Brussels, Belgium, 1999.

Carruthers, Al, Scoping Study for Humanitarian Demining Technologies, Medicine Hat, Canada: Canadian Centre for Mine Action Technologies, 1999.

Craib, J.A., Survey of Mine Clearance Technology, Conducted for the United Nations University and the United Nations Department of Humanitarian Affairs, 1994.

Evaluation of Unexploded Ordnance Detection and Interrogation Technologies. Prepared for Panama Canal Treaty Implementation Plan Agency. U.S. Army Environmental Center and Naval Explosive Ordnance Disposal Technology Division, 1997.

Garwin, Richard L. and Jo L. Husbands. Progress in Humanitarian Demining: Technical and Policy Challenges. Prepared for the Xth Annual Amaldi Conference. Paris, France, 1997.

Groot, J.S. and Y.H.L. Janssen. Remote Land Mine(Field) Detection, An Overview of Techniques. TNO Defence Research. The Hague, The Netherlands., 1994.

Gros, Bertrand and Claudio Bruschini. Sensor Technologies for the Detection of Antipersonnel Mines, A Survey of Current Research and System Developments. EPFL-LAMI DeTeC. Lausanne, Switzerland., 1996.

Havlík, Stefan and Peter Licko. Humanitarian Demining: The Challenge for Robotic Research. The Journal of Humanitarian Demining, Issue 2.2 (May 1998).

Healey, A.J. and W.T. Webber. Sensors for the Detection of Land-based Munitions. Naval Postgraduate School. Monterey, CA., 1995 Heberlein, David C., Progress in Metal-Detection Techniques for Detecting and Identifying Landmines and Unexploded Ordnance, Institute for Defense Analyses, 2000.

Horowitz, Paul, et al., New Technological Approaches to Humanitarian Demining, The MITRE Corporation, 1996.

Hussein, Esam M.A. and Edward J. Waller. Landmine Detection: The Problem and the Challenge. Laboratory for Threat Material Detection, Department of Mechnaical Engineering, University of New Brunswick. Fredericton, NB, Canada. 1999.

Janzon, Bo, International Workshop of Technical Experts on Ordnance Recovery and Disposal in the Framework of International Demining Operations (report), National Defence Research Establishment, Stockholm, Sweden, 1994.

Johnson, B. et al., A Research and Development Strategy for Unexploded Ordnance Sensing, Massachusetts Institute of Technology, 1996.

Kerner, David, et al. Anti-Personnel Landmine (APL) Detection Technology Survey and Assessment. Prepared for the Defense Threat Reduction Agency. DynMeridian. Alexandria, VA., 1999.

McFee, John, et al., CRAD Countermine R&D Study – Final Report, Defense Research Establishment Suffield, 1994.

Mächler, Ph. Detection Technologies for Anti-Personnel Mines. LAMI- DeTeC. Lausanne, Switzerland, 1995.

Scroggins, Debra M., Technology Assessment for the Detection of Buried Metallic and Non-metallic Cased Ordnance, Naval Explosive Ordnance Disposal Technology Center, Indian Head, MD, 1993.

Sensor Technology Assessment for Ordnance and Explosive Waste Detection and Location. Prepared for U.S. Army Corps of Engineers and Army Yuma Proving Ground. Jet Propulsion Laboratory, California Institute of Technology. Pasadena, CA. 1995.

Tsipis, Kosta. Report on the Landmine Brainstorming Workshop of August 25-30, 1996. Program in Science and Technology for International Security, Massachusetts Institute of Technology. Cambridge, MA., 1996.