Mine Detection: Army Detector's Ability to Find Low-Metal Mines Not Clearly Demonstrated
Released: 1996-08-28T00:00:00

Letter

Background

The military services use portable or handheld metal detectors as one of several devices to detect and clear hazards such as landmines. As we reported last year, the detection and clearance of buried explosives like landmines is very difficult, and no ideal solution has emerged.Low-metallic content landmines—generally plastic-encased explosives with some metal parts inside—are among the most difficult mines for a metal detector to find, especially when buried. Mines placed on or protruding above the ground surface do not pose the same detection problem as buried mines because it is possible that they could be detected visually.

The typical portable metal detector uses electromagnetic induction technologies to find metal objects at or below the ground surface. These detectors induce a magnetic field, which in turn causes a secondary magnetic field to form around nearby objects that have conductive properties such as the metal in landmines. An object’s detectability is a function of the induced magnetic field’s strength and an object’s conductive properties, size, shape, and position. For example, copper, aluminum, and ordinary steel are good conductors and relatively easy to detect. Stainless steel is harder to detect than an identical piece of ordinary steel because it offers more resistance to the induced magnetic field and thus produces a weaker or smaller secondary magnetic field. Portable metal detectors operate on either the continuous wave or pulse method of transmitting and receiving. Continuous wave detectors induce and monitor magnetic fields continuously to sense any disruptions caused by a conductive object’s secondary field; pulse detectors transmit and receive in alternating cycles in search of secondary magnetic fields.

In 1962, the Army fielded the AN/PSS-11, a continuous wave portable mine detector. The last AN/PSS-11s were purchased in 1972. In the late 1970s, the Army began a program to improve the AN/PSS-11’s durability and maintainability by replacing its outdated electronics. As the lead service for the Department of Defense (DOD), the Army developed such a detector, tested it successfully, and approved its production in 1984. However, separate attempts to produce the detector to Army specifications—in 1985 with one manufacturer and in 1987 with another—failed due to the manufacturers’ technical or financial problems. As the AN/PSS-11 became increasingly more difficult to support due to the unavailability of replacement parts, the Army was faced with a shortfall. In May 1990, the Army decided to forgo development of a new or improved detector and instead to purchase a commercially available detector as an interim solution to its immediate shortfall.

After screening 12 commercially available metal detectors for sensitivity, suitability, and availability, the Army narrowed the field to two candidates—the AN-19/2 pulse detector made by Schiebel GmbH of Austria and the Metex 4.125 continuous wave detector made by Foerster Instruments, Inc. In March and July 1991, the Army awarded contracts to the respective manufacturers for test articles, with options for future buys. In December 1991, the Army selected the Schiebel detector to replace the AN/PSS-11 and designated it as the AN/PSS-12 Handheld Metallic Mine Detector. By the time the contract expired in March 1996, 18,235 AN/PSS-12s had been ordered and all but a few hundred had been delivered. The total cost of the detectors, including support items, came to $21.9 million. Of the total, 15,553 are for the Army, 571 for the Marine Corps, 326 for the Air Force, 323 for the Navy, and 1,462 for foreign military sales. As of March 1996, the Army had sent 261 of the detectors to Bosnia. The Air Force has also sent AN/PSS-12s to Bosnia. Until 2001, when the Army plans to field a new portable detector it is developing for low-metallic and nonmetallic mines, the AN/PSS-12 will remain the Army’s primary portable mine detector.

Low-Metallic Content Mines Are a Known Threat

According to defense intelligence information, low-metallic content mines have been a recognized threat for the last 14 years and are a prevalent threat in Bosnia. Low-metallic mines have been represented in Army tests of portable and other detection systems since 1983 and were included in the performance specifications used for the 1991 procurement of the AN/PSS-12. Army officials informed us that a separate technology effort was underway before 1991 to address the low-metallic and nonmetallic threat. The Army plans to complete this effort by fiscal year 2001.

Low-Metallic Mines Are Prevalent in Bosnia

According to the National Ground Intelligence Center, mines with minimal metal content were first fielded in the early 1950s. For years, however, no criterion or standard existed for defining a mine as having low-metallic content. In the early 1980s, the Center established the U.S. M-19 mine, which contains 2.46 grams of metal, as the threshold for low-metallic mines. By this standard, only mines containing 2.46 grams of metal or less are considered as low-metallic threats.

According to intelligence reports, over half of the landmines in Bosnia are buried, and about 75 percent of them are low-metallic mines. The metal content of these mines is confined to the aluminum casing around their chemical action fuzes. About eight different types of Yugoslav landmines with this type of fuze have been identified. The Center reported that some former Yugoslav mines containing no metal were known to have been manufactured. These mines’ fuzes are wrapped in plastic and would not be detectable by the AN/PSS-12 or any other metal detector. However, the mines recovered so far have all contained aluminum-clad fuzes.

Fuzes used in some of these mines contain between 0.4 and 1.5 grams of aluminum. Depending on the type, these mines may contain from one to three fuzes, any one of which is capable of detonating the mine. Examples include the TMA-1 and TMA-5, which contain one fuze; the TMA-2, which contains two fuzes; and the TMA-3 and TMA-4, which contain three fuzes. The most difficult to detect are the PMA-1, which contains less than 0.4 grams of aluminum in its fuze, laid horizontally in the mine, and the PMA-2, which has a vertical fuze (a more difficult position for detection) containing 0.5 grams of aluminum. For detection purposes, the metallic content of multiple fuzes is not additive; according to Army officials, the fuzes are positioned far enough apart in the mine as to generally limit detection to one fuze at a time.

Low-Metallic Mine Targets Were Included in Requirements for the 1991 Procurement

According to Army officials, the purpose of the 1991 procurement was to buy a detector with performance equal to or better than the AN/PSS-11. The detection and other performance requirements for the 1991 procurement were contained in a modified military specification associated with the earlier attempt to develop an improved version of the AN/PSS-11. This specification required that the detector have a greater than 92-percent probability of detecting metallic mines and mines with small metallic content. The specification described the following targets to be detected in three different types of soils—sand, loam, and magnetite (an iron-based soil): a small steel pin, 4.5 millimeters long, to simulate the M-14 mine (detection of this pin was desired but not required in magnetite); a hollow aluminum tube, 44.5 millimeters long and 6.4 millimeters in a steel PMN-6 striker pin, 57 millimeters long, one-third of which was 4.8 millimeters in diameter and the remainder 9.5 millimeters in diameter; and a simulated M-16 mine.

According to the 2.46 gram standard, the M-14 pin and the aluminum tube represented low-metallic targets. The M-16 is a metal-clad mine. The PMN-6 striker pin falls somewhere between the M-16 and the low-metallic targets. The designation PMN-6 refers to a British-made training mine that is a replica of the Soviet PMN mine. Like the Soviet mine, the PMN-6 training mine has a nonmetallic case and contains several metal components in addition to the striker pin, which collectively amount to over 17 grams of metal. According to the National Ground Intelligence Center, the striker pin itself would not qualify as a low-metallic target because it contains several times the amount of metal as the M-19.

Army Program Is Aimed at Detecting Low-Metallic and Nonmetallic Mines

According to Army officials, the Army began developing other technologies in the mid-1980s to detect low-metallic and nonmetallic mines. Under a program now known as the Handheld Standoff Mine Detection System, a detector is being developed that integrates ground-penetrating radar, infrared, and metal detection technologies, along with electronics that are intended to synthesize and interpret the signals from the three sensors for the operator. The detector is now in competitive prototype testing and is slated for a production decision in fiscal year 2001. A gap in detection capability against low-metallic and nonmetal mines may remain until then. Our September 1995 report on unexploded ordnance provides additional information on these technologies.

To provide some additional capability for U.S. forces in Bosnia, the Army is evaluating commercially available detectors that combine technologies such as ground-penetrating radar with electromagnetic induction methods. These detectors do not possess all of the capabilities planned for the detector in prototyping. According to Army officials, recent tests of such systems demonstrated a 70-percent detection capability against low-metallic and nonmetallic mines. The Army does not consider this detection rate acceptable for use by troops in the field. Further testing is planned.

AN/PSS-12 Detector Performed Poorly Against Low-Metallic Mines in Operational Testing

The Army’s Test and Experimentation Command, under the auspices of the Operational Test and Evaluation Command, conducted two operational tests during 1991 to assess the performance of the candidate metal detectors in a field environment. Short of war, operational testing is the most realistic way of assessing a system’s effectiveness and suitability for fielding. However, the operational tests had several shortcomings that complicate the assessment of the comparative performance of the two detector candidates and the baseline AN/PSS-11 against low-metallic mines. In the first test, the Schiebel detector found 3.4 percent of the low-metallic targets, compared with 24.2 percent for the AN/PSS-11. Because the Foerster detector was not included in the first test and the low-metallic targets were excluded from the second test, the Foerster was not tested against these low-metallic targets and no comparison could be made. The Foerster detected twice as many of the lowest metal content targets present at the beginning of the second test, but the Army concluded the targets were not representative of the higher metallic content Soviet mine and ruled them invalid. As performance against higher metallic targets was equal, price became the deciding factor in the procurement decision.

March 1991 Operational Testing

The first operational test of the detector candidates was conducted during March 20-28, 1991. Prior to this test, the Army had screened 12 different commercial detectors and had eliminated all but one because of (1) technical, performance, or production shortcomings or (2) high prices. Two Foerster candidates were among the detectors eliminated on the basis of price. Thus, the Schiebel was the only detector forwarded for operational testing with the baseline AN/PSS-11 detector.

This test included four target types: metal-clad training M-15 and M-16 mines, the M-14 pin, and the PMN-6 striker pin. Given that the hollow aluminum tube described in the specification was not used in the test, the M-14 pin was the only low-metallic target. The results of the test are shown in table 1.

It is strongly recommended that the Government not purchase this mine detector as a replacement for the AN/PSS-11 at this time. Rather, another survey should be conducted to identify candidate mine detectors that meet military specifications outlined in the test and evaluation master plan. Further technical and operational testing should result in a more suitable replacement mine detector. against mines with small metallic content (i.e., the M-14 and the PMN-6), the AN-19/2 fell considerably short of the PS requirement. . .Indeed, its performance during test was distinctly inferior to that of the AN/PSS-11 under the same conditions, although the AN/PSS-11 itself did not meet the PS requirement either.

The Test and Evaluation Command did state that the procurement decision should not depend too heavily on the detectors’ inability to detect low-metallic mines because such mines were just a step away from nonmetal mines, which would render a metallic mine detector useless. Nonetheless, the Command recommended that the Army (1) not approve the Schiebel for fielding as the AN/PSS-12 and (2) reexamine the role of the mine detector in the Army and confirm that the detection of mines with small metallic content remained a valid need.

Ultimately, the Army decided to eliminate the M-14 target from further testing because it concluded that the target’s metal content was so low that it was essentially nonmetal. It was not replaced with another low-metallic target. Army officials informed us that the user representative at the time did not want to reject the Schiebel on the basis of its performance against less lethal mines such as the M-14—considered likely to injure, rather than kill—if it could detect more lethal mines that could kill several individuals. This was a significant decision because the M-14 pin had been cited in the performance specification and had been used in Army testing of portable mine detectors since 1983. Such testing included the attempted product improvements of the AN/PSS-11 and the original screening of the 12 commercial detector candidates.

The Army realized in 1992 that the M-14 pin contained only a portion— 0.29 grams—of the total metal in the M-14 mine. According to testing conducted in 1996, the actual mine is more detectable than the target used. Had the Army known this at the time of the 1991 testing, it may have been able to substitute a more authentic low-metallic mine target.

September-October 1991 Operational Testing

Following the filing of a bid protest, the Army decided to readmit one of Foerster’s two original candidates to the competition and therefore it participated in the second operational test. The second operational test was held from September 17, 1991, to October 4, 1991. It included three examples each of the Foerster, Schiebel, and AN/PSS-11. As in the previous test, this test used targets designated as PMN-6s to simulate low-metallic mines. However, the second test used PMN-6 training mine casings, which contained the steel striker pin, a spring, and a small washer. Shortly before the test began, representatives from the program manager’s office and the U.S. Army Engineer School (which represented the user) contended that the PMN-6 target did not contain as much metal as a real Soviet PMN mine. They stated that metal would have to be added to the PMN-6 test targets already buried to make them realistic.

However, the purpose of the target was not to replicate the Soviet mine. In fact, the test report indicated that the purpose of the PMN-6 striker pin was to simulate the M-19 mine. The Soviet mines that the PMN-6 was modeled after are not low-metallic mines. The National Ground Intelligence Center reports that no Soviet landmine contains less than 8 grams of metal, which is more than the 2.46-gram threshold. While it would have been reasonable to ensure that the target was a fair replica of either the M-19 low-metallic mine or the striker pin described in the specification, it was not reasonable to insist that the target replicate the Soviet mine. The test team maintained that adding metal to the PMN-6 target could make its detectability climb to 100 percent; thus, there would be no way to discriminate one detector’s performance from another’s. Ultimately, it was agreed that a 2-inch metal washer would be added to each PMN-6 target. This was done by inserting the washers beneath the surface and on top of the buried targets, without digging them up. Because the M-14 low-metallic target had already been eliminated, adding metal to the PMN-6 was a key decision because it effectively eliminated the only remaining target the test team considered to have a metal content low enough to differentiate the performance of one detector from another.

The Test and Experimentation Command had planned a 1-day pilot test to work out procedures and firm up preparations for the operational test. The Command decided to conduct the pilot test with the PMN-6 targets in their original condition—without the large washer. Table 2 shows the results of the pilot test.

The percentages shown above are the averages for the three detectors of each type used. The best performance by a Foerster was 76.67 percent; by a Schiebel, 43.33 percent; and by an AN/PSS-11, 43.33 percent. While these results were included in the test report, they were excluded in the analysis of operational test results for the procurement decision on the flawed basis that the target was unrepresentative of a Soviet mine.

After the pilot test, the second operational test was conducted with the PMN-6 targets augmented with the large metal washers. The other two mine targets included in the test were metal-clad mines and thus had high metal content. These were the M-8, a training version of the M-16 mine, and the TMN-46, a training version of a Soviet antitank mine. Table 3 shows the results of the test.

These results showed that against the high-metallic mine targets remaining in the operational test, all three detectors found virtually all the mines and passed the Army’s 92-percent detection requirement. The results also confirmed the test team’s concern that adding metal to the PMN-6 target could cause detection percentages to climb to 100 percent for all the detectors.

Army Decision to Procure Schiebel Detector Did Not Weigh Performance Against Low-Metallic Mines

The Army’s decision to procure the Schiebel was based on detection performance against only the high metal content mines. In a December 13, 1991, memorandum, the Chairman of the Source Selection Board in charge of selecting the best detector candidate concluded that the performance difference between the detectors was not significant and that the additional cost of the Foerster was not justified by any significant increase in technical or operational benefit.

Army officials informed us that because none of the detectors, including the AN/PSS-11, could meet the 92-percent requirement against low-metallic mines, they were equally unable to satisfactorily detect such mines. Therefore, the ability to detect low-metallic mines was no longer a discriminating factor in selecting a replacement for the AN/PSS-11. Nonetheless, in the only comparable operational test, the Foerster detector demonstrated a significantly better ability to detect the lowest metal mine target—the pilot test’s PMN-6 target—than the Schiebel detector. Whether the Foerster’s better performance in the pilot test was worth its higher price was not assessed because low-metallic mines had already been eliminated as a factor by the time the decision was made.

Varying Test Conditions Make Comparison of Test Results Difficult

Over the years, the Army has gathered performance data on portable mine detectors from a number of sources, including technical tests, operational tests, demonstrations, and from actual use in operations, such as in Somalia. Regardless of how data is gathered, the performance of portable mine detectors is affected by several factors that, if not controlled, make it difficult to compare one test or operation with another. In the numerous tests and demonstrations of portable mine detectors conducted since 1983, these factors have not been held to consistent, realistic, or technically sound standards. The factors include target type, target burial depth and position, soil type and moisture content, and the distance between the detector head and ground surface. Performance is also affected by the proficiency of the operator, including such factors as maintaining the correct height and speed of the detector head as it is swept back and forth in the search for targets, the level of training, and the operator’s ability to pick up audio and visual cues that can help indicate the presence of a mine. In addition, as suggested by test results, different detectors of the same model can vary in performance.

While tests, by their nature, are conducted under controlled conditions to provide for valid data collection and analysis, technical and operational tests are conducted under different circumstances and are interpreted differently. Technical testing is intended to determine the technical capabilities of a detector under ideal conditions. While such testing can eliminate detectors that do not have the ability to meet performance requirements, it is not intended to assess performance under field conditions. Operational testing is much more realistic than technical testing, as it can introduce more factors that affect performance results, most importantly, the operator-machine interface. The two operational tests of portable mine detectors the Army conducted in 1991 are illustrative of how difficult it is to isolate detector performance from other factors when comparing test results. Their test conditions are compared in table 4.

Some tests are actually demonstrations, which fall somewhere between technical and operational testing, although they do not necessarily provide the discipline or data to support statistically valid or independent data analysis. Demonstrations of portable mine detectors have been conducted in a field environment; however, the detectors have been operated by contractor personnel or Army civilian personnel. Again, as in operational testing, they must contend with a variety of factors that can affect detector performance. While demonstrations do not enable conclusions to be drawn about a detector’s ability to meet military requirements, they are a vehicle for quickly gauging a detector’s potential performance in the field.

While the use of portable mine detectors in actual operations provides realistic information on detection performance, the number of mines detected are not usually recorded, and the number of mines missed, absent maps and records, may not be known. Results can also be site specific as to soil type, moisture content, and temperatures. Thus, these operations do not lend themselves to quantification of a detector’s performance. Moreover, one mishap can prove fatal. The AN/PSS-12 was used by U.S. forces in Somalia and by U.S. contractors in Kuwait and is currently deployed with U.S. forces in Bosnia. An Army after-action report from operations in Somalia states that the AN/PSS-12 could not detect low-metallic mines, but it offers no specifics on the detection shortfalls. Although many landmines were reportedly found by U.S. contractors in Kuwait using the Schiebel and other metal detectors, the fact that they were buried in sand and in patterns made them easier to find than might be the case in other situations. These operations do not provide information on the percentage or number and types of mines that were found by the metal detectors, nor do they indicate what mines were not detected.

The performance of the AN/PSS-11 in several tests conducted since 1983 illustrates how the measured performance of a detector can vary from one test to the next. In a 1983 field test outdoors at Fort Belvoir, Virginia, the AN/PSS-11 detected 80 percent of M-14 mine targets. In 1985 testing at the Fort Belvoir indoor mine lane facility, prototypes of the product-improved version of the AN/PSS-11 detected none of the M-14 targets buried in sand and 67 percent of the M-14 targets on the surface. In a 1988 field test to establish the AN/PSS-11’s detection capabilities as a standard for an Army development of a vehicle-mounted detection system, the AN/PSS-11 detected 82.5 percent of buried M-19 mine targets. As stated previously, the AN/PSS-11 detected 24.2 percent of M-14 targets, 80.8 percent of PMN-6 striker pins, and 28.9 percent of PMN-6 targets (without the large washer) in the 1991 operational tests. The data from these various sources defy a definitive conclusion on the performance of a detector that has been in the Army’s inventory for 30 years.

No Problems Reported With the Limited Use of the AN/PSS-12 in Bosnia

According to the Army, U.S.troops have not experienced problems with the AN/PSS-12 in Bosnia. Army officials have cited the successful use of the detector by other countries and the detectability of low-metallic mines in Bosnia as further evidence of the AN/PSS-12’s potential for performance there. However, this information is not consistent with the Army’s 1991 test results and information from other sources. Consequently, we believe the potential effectiveness of the AN/PSS-12 against low-metallic mines in Bosnia is inconclusive. The steps the Army has taken to minimize the threats posed by landmines there and the resultant infrequent reliance on the AN/PSS-12 may help to explain why the detector’s poor performance against low-metallic targets in testing has not been exhibited in Bosnia.

Performance of the AN/PSS-12 in Bosnia Is Inconclusive

While the Army does not know the percentage of each type of mine detected by the AN/PSS-12 since deploying to Bosnia, officials said that when the detector has been used it has worked well. As of July 1996, they reported that no U.S. troop casualties had occurred as a result of the detector’s having failed to detect a mine in Bosnia. Army officials noted that several other countries purchased the Schiebel detector before the United States, including Germany, Canada, Israel, Sweden, and the United Kingdom. They said that Canada and Sweden successfully used the Schiebel in the former Yugoslavia before the U.S. troops deployed there. The Schiebel was the detector of choice by contractors that conducted mine-clearing operations in Kuwait and by the United Nations in several of its demining operations. According to Army officials, its division engineer in Bosnia was not interested in any performance enhancements as the AN/PSS-12 was performing fine.

The Army also believes that the mines found so far in Bosnia have had enough metal content to be detectable by the AN/PSS-12. While these mines are classified as low-metallic mines, they reportedly contain more metal than the M-14 target used in the March 1991 operational test. More importantly, the metal contained in the Bosnian mines is aluminum. Because aluminum is lighter than steel, a piece of aluminum that weighs the same as a piece of steel would be considerably larger. Thus, according to Army officials, the aluminum in the Bosnian mines not only weighs more than the M-14 test target—it would be physically larger as well.

Other information clouds an overall picture of the AN/PSS-12’s use in operations. During the course of our review, we learned that Germany has decided to replace its Schiebel detectors with a detector made by Vallon GmbH of Germany, and the Netherlands is using a Foerster detector in Bosnia. In 1993, the United Kingdom replaced its Schiebel detectors in Cambodia. A State Department official assisting with the international humanitarian demining effort in Bosnia informed us that the AN/PSS-12 is used only in conjunction with probes (pointed rods used by hand). The Marine Corps informed us that it prefers the technology of the AN/PSS-11 and currently uses the old detector in Guantanamo Bay, Cuba. In April 1996, the Air Force issued a message to its explosive ordnance technicians deployed in Bosnia to clear landmines and other explosives from airfields, cautioning them that the AN/PSS-12 does not have the sensitivity to detect low-metallic mines they may encounter. The Air Force is processing an urgent contracting action to purchase another metal detector to replace its AN/PSS-12s in Bosnia. This action is unrelated to the Army’s near-term effort to evaluate commercial detectors that combine technologies for potential application to Bosnia.

We attempted to verify that the aluminum found in mines in Bosnia was in fact more detectable than the steel targets used in the 1991 testing. We contacted several countermine, testing, and explosive ordnance organizations within the services and none reported that they had developed credible data on the comparative detectability of different metals. At our request, a manufacturer of measurement and detection equipment compared the detectability of an aluminum target approximating a 0.4-gram aluminum fuze found in Bosnian mines with a steel target approximating the M-14 pin used in Army’s tests. The comparison did not show that the aluminum target was unequivocally more detectable than the steel target. We did not attempt to assess how the detectability of a 1- to 1.5-gram piece of aluminum found in some mines in Bosnia would compare with the more substantial PMN-6 striker pin used in testing.

Army Has Taken Steps to Minimize the Threat Posed by Landmines in Bosnia

According to information we obtained from Department of State, Defense Intelligence Agency, and Army officials, several factors have minimized the risks landmines pose to U.S. troops in Bosnia. These steps include the following:

The former warring parties, who are responsible for removing landmines, have provided maps, when available, of mined areas so that these areas can be avoided.

For the most part, landmines are believed to be concentrated in known zones of separation that formerly existed between the warring factions. These zones are avoided when possible. However, a State Department official said less is known about the landmine threat outside these zones.

Because U.S. forces are not taking ground as they would in a combat situation, they can move along established routes or roads. This gives combat engineers the opportunity to run rollers down the routes several times to detonate mines before any attempts are made at dismounted mine detection. Most main routes are believed to be safe.

Some mine survey, route clearance assurance, and site clearance work has been contracted out.

According to the State Department, areas considered cleared by the former warring parties must still be verified by peacekeeping forces. This is because the warring parties (1) are responsible for clearing areas only within the first 30 days after turning the areas over, (2) do not necessarily have the best mine detection and clearance equipment or training, and (3) did not prepare many maps of mined areas.

Army officials have described their approach to mined areas in Bosnia as follows. All personnel are provided with extensive mine-awareness training before going into the theater. Before U.S. forces move into an area, an intelligence assessment is made. Discussions are held with the former warring parties to determine whether the area is mined and if so, what kinds of mines were used. At a more detailed level, some exploration may be done by engineers using probes to find sample mines. The troops can then verify whether the mines are consistent with the initial assessment. Data sheets on the threat mines are available that describe the characteristics of each mine and help make an accurate identification. If an area cannot be accessed by rollers, the combat engineers then assess whether the mines found can be detected with the AN/PSS-12. Army officials said they do this by actually checking the detector against the sample mines found in the ground. If the mines can be detected with the AN/PSS-12, then U.S. troops can go in dismounted. If the mines are not detectable, U.S. troops do not go in dismounted. As a last resort, probes could be used.

Recommendation

Had the 1991 operational testing properly portrayed low-metallic mines, the Army may have had greater assurance that the detector it selected as the AN/PSS-12 was the best choice at the time against the full range of landmines. The limitations of this testing are perhaps more apparent now than at the time; while the testing became focused on higher metallic content mines, low-metallic content mines are prevalent in Bosnia. Although testing may not be able to replicate all of the conditions expected in actual operations, it should provide a sound assessment of detection and other performance capabilities that can serve as a consistent baseline for comparing results from test to test. Because the 1991 testing did not provide such a foundation, an assessment of the AN/PSS-12’s performance in Bosnia or any operation is perhaps more subjective than it should be.

Accordingly, we recommend that the Secretary of Defense establish and enforce realistic and consistent test standards for testing countermine and mine detection systems that reflect known threat mines and the conditions under which they are likely to be encountered. Such standards should be applied not only to the acquisition of new systems but to the evaluation of near-term or experimental solutions as well.

Agency Comments and Our Evaluation

DOD concurred with our recommendation to establish realistic and consistent test standards. It also noted that the research, development, testing, and evaluation of countermine and mine detection systems were being reviewed by an unexploded ordnance clearance executive committee and steering group (see app. I).

Although DOD concurred with our recommendation, it stated that the soldiers in Bosnia are not in danger due to the performance of the AN/PSS-12 in the presence of low-metallic mines and disagreed with any implication to the contrary. DOD reiterated that U.S. forces avoid mines when possible, using devices such as rollers and probes in addition to the AN/PSS-12 when mines are encountered, and that other countries selected the same detector before the Army did. These points were covered in the draft report. The information available to date supports DOD’s characterization of the relative safety of U.S. forces operating in the presence of landmines in Bosnia. The analytical dilemma is in reconciling the poor performance of the AN/PSS-12 against low-metallic targets in operational testing with its reported satisfactory performance in Bosnia where low-metallic mines are prevalent. We believe it is the prudent steps taken by the Army to avoid and minimize the landmine threat in Bosnia—more so than the capability of the AN/PSS-12 or the detectability of the low-metallic content mines there relative to the test targets—that explains the difference between the detector’s performance in operational testing with its experience in Bosnia.

DOD also noted that an independent technical test conducted in June 1996 within DOD shows that the AN/PSS-12 can consistently detect M-14 low-metallic mines when inert mines are used instead of targets. The data from this test indicates that the inert M-14 mine is more detectable by the AN/PSS-12 than the M-14 firing pin first used and later removed as a target in the 1991 operational testing, although no detection percentages were obtained to measure consistency. The improvement is attributed to the fact that the inert mine contains more metal than the firing pin.

The June 1996 test does raise additional questions about the usefulness of the information obtained in Army testing since 1983 that used the M-14 firing pin as a target. However, it does not supplant the 1991 operational test results because it was a limited technical test and was not intended to replicate a realistic environment. In the June 1996 test, landmines were not buried but placed on the ground with the detectors held directly over them. The essence of the test was to lower the detector over the mine and record the distance at which the detection was made; no searching was involved. By comparison, in the pilot test phase of the September-October 1991 operational test, the Schiebel detector found only 32.2 percent of the PMN-6 targets, which contained significantly more metal than the inert M-14 mine. The need to put the June 1996 test results into the proper perspective underscores the value of establishing realistic and consistent test standards.

Scope and Methodology

To obtain information for this report, we reviewed numerous documents relating to the test and evaluation of portable mine detectors, including several military services’ test reports since 1983, the contract file on the AN/PSS-12 procurement, files from previous investigations of the AN/PSS-12 procurement conducted within DOD, the after-action report on Somalia, threat publications prepared by the National Ground Intelligence Center, Army, Navy, and U.S. Marine Corps evaluations, and evaluations conducted by the Naval Explosive Ordnance Disposal Technology Division.

We interviewed officials from the Office of the Secretary of Defense; the Departments of State, the Army, the Navy, and the Air Force; the U.S. Marine Corps; the Defense Intelligence Agency; the National Ground Intelligence Center; and the Joint Naval Explosive Ordnance Disposal Technology Division. We also interviewed current and former Army program officials, representatives from the Army contracting office at the Army Aviation and Troop Command, current and former Army user representatives from the Army Engineer School, representatives from the Army Test and Experimentation Command involved with the conduct of both operational tests, and a representative from the Army Waterways Experimentation Station that supplied PMN-6 mines for the second operational test. We did not visit Bosnia-Herzegovina, but information was obtained from Army officials in direct contact with units there and from other sources as indicated. We also interviewed representatives from detection equipment manufacturers and, at our request, the Canadian firm, Geonics, Ltd., conducted a laboratory test to compare the detectability of steel and aluminum targets.

We conducted our review from December 1995 to July 1996 in accordance with generally accepted government auditing standards.

We are sending copies of this report to other interested congressional committees and the Secretary of Defense. We will also make copies available to others upon request.

Please contact me at (202) 512-5140 if you or your staff have any questions concerning this report. Major contributors to this report were Paul L. Francis and James B. Dowd.

Comments From the Department of Defense

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.