NBSIR 75-719

A Compliance Testing System

for NI LEGJ

Robert Mills

Law Enforcement Standards Laboratory

Center for Consumer Product Technology

Institute for Applied Technology

National Bureau of Standards

Washington, D. C. 20234

February 1 976

Final

Prepared for National Institute of Law Enforcement and Criminal Justice

Law Enforcement Assistance Administration

U.S. Department of Justice

Washington, D. C. 20531

NBSIR 75-719

A COMPLIANCE TESTING SYSTEM

FOR NILECJ

Robert Mills

Law Enforcement Standards Laboratory

Center for Consumer Product Technology

Institute for Applied Technology

National Bureau of Standards

Washington, D. C. 20234

February 1 976

Prepared for National Institute of Law Enforcement and Criminal Justice

Law Enforcement Assistance Administration

U.S. Department of Justice

Washington, D. C. 20531

U.S. DEPARTMENT OF COMMERCE, Elliot L. Richardson, Secretary

James A. Baker, III. Under Secretary

Dr. Betsy Ancker-Johnson, Assistant Secretary for Science and Technology

NATIONAL BUREAU OF STANDARDS, Ernest Ambler, Acting Director

Final

ACKNOWLEDGMENTS

The author is a program manager in the Law Enforcement

Standards Laboratory (LESL) of the National Bureau of

Standards. This report was prepared under the direction of

Jacob J. Diamond, Chief of LESL. It was sponsored by the

Advanced Technology Division of the LEAA National Institute of

Law Enforcement and Criminal Justice, Lester D. Shubin, Program

Manager for Standards.

ii

A Compliance Testing System for NILECJ

CONTENTS

Page

Abstract v

Introduction .. 1

Compliance Testing Mechanisms «... 2

Laboratory Evaluation and Accreditation 6

Compliance Information System 8

Role and Qualifications of NBS 9

Who Pays? 10

Glossary 14

Index of Terms 19

iii
r, - - ii. /, «... \ c ^, '

si. * - . 1 * , ' " > » - * '

1*^ i .- ' c - : V s ^ . ..

J Ti ; • - . .
ABSTRACT

In order to assure that NILECJ equipment standards

have the impact intended, a Compliance Testing and

Laboratory Accreditation program is needed to

establish which items available on the market do, in

fact, meet the requirements of the standards. This

report contains recommendations for such a NILECJ

program. In brief, the proposed program would (a)

result in a body of qualification and acceptance test

data, (b) establish a list of testing laboratories

competent to perform these tests, and (c) set up a "compliance information system" for the dissemination

of this information to officials in the criminal justice

system.

Key words: Acceptance testing; compliance testing;

laboratory evaluation; performance testing; qualified

products lists; testing.

v
I .:}'.' : k> J

u

.; :) b>?r io. ri/'

i5 .

-v,

.'
A Compliance Testing System for NILECJ

INTRODUCTION

The National Institute of Law Enforcement and Criminal

Justice (NILECJ) of the Department of Justice is assisting law

enforcement agencies in the procurement and effective use of

the best equipment that present technology can offer. One

means to that end is the development of performance standards

for law enforcement equipment, a task assigned to the Law

Enforcement Standards Laboratory (LESL) of the National Bureau

of Standards. These performance standards can be referenced in

purchase documents; they serve as objective descriptions of

wha- the user wants and what the supplier agrees to give him.

They can also help guide manufacturers in the design of

equipment of high utility.

The development and dissemination of standards, however,

are only steps in the procurement process. The ultimate

benefits to the law enforcement community, to the public, and

to the manufacturers cannot be realized until a means is

established to ascertain which hardware available on the market

does, in fact, meet the requirements of the standards.

Lester C. Thurow, professor of economics and management at

the Massachusetts Institute of Technology, has written of this

need in a private communication.

"Without testing and inspection, there is the classic

economic problem of bad money driving out good. To obtain an

order, a company must submit the lowest bid. One of the

easiest ways to submit a lower bid is to lower the quality of

the equipment being sold. The initial quality cuts are small,

but competitive pressures force an escalation of this process

until the equipment no longer provides the services desired.

"With simple goods this problem can be avoided, since the

customer can quickly determine that he is being sold a good

without the desired performance characteristic. VJith

complicated equipment, there is no simple or easy way to

determine whether the good is up to standards. To know whether

the equipment is good or bad requires technical testing, both

initially to insure that equipment can meet performance

standards and subsequently to insure that it does.

1
"This testing is in the long-run interest of manufacturers

(although it may not be in the interest of any particular

manufacturer) since it allows high quality equipment

manufacturers to stay in business. Without testing, they will

be forced to lower quality standards by the competitive

pressures of the market place or go out of business."

This report makes recommendations to NILECJ for the

establishment of a compliance testing system for equipment used

by the criminal justice system. This "Compliance Testing

System" would (a) make available to law enforcement officials

impartial information concerning compliance with NILECJ

standards, and (b) enable manufacturers to avoid the time and

expense of convincing individually hundreds of police

departments that their products are capable of satisfying

minimum performance requirements.

Compliance* Testing Mechanisms

There are several alternatives, not necessarily exclusive

of one another, for ascertaining the compliance of law

enforcement equipment with the K'lLECJ standards. First, some

larger police departments have test and evaluation divisions

responsible for determining the utility of new products.

However, most departments do not have in-house capabilities for

performing compliance tests in accordance with the NILECJ

standards

.

Secondly, inspectors can monitor the manufacturing process

at the plant. This "on-site inspection" can be used to monitor

quality control. It is being used for some food inspection and

military procurement programs, but it is expensive and is not

considered to be a viable alternative for NILECJ.

Thirdly, manufacturers can be asked to certify that their

equipment meets the requirements of the appropriate standard,

and be held legally accountable for their certification.

However, reliance solely on manufacturer certification

generally is not satisfactory. Since transgressions must be

redressed after the fact, usually by time-consuming court

action, there is little protection against companies that have

no reputation to uphold and are able to go out of business

after a short period of operation. Even for more reputable

companies, the pressure of realizing a profit too frequently

tempts manufacturers into compromises of product quality if

*These and other terms relating to product standards,

product testing and laboratory accreditation are defined in

the glossary on pages 13 to 18.

2.
routine independent checks are not made. See, for example,

"The Aircraft Brake Scandal," in the April 1972 issue of Harper

Magazine, which relates a tale of falsified laboratory test

results and fraudulent claims by a manufacturer having trouble

fulfilling a government contract. Manufacturers may of course

do their own product testing. This certainly should be

encouraged, as long as the results are intended primarily for

the manufacturer's own use.

Another approach is to have the product manufacturer

certify and an independent testing laboratory verify. In this

way, the manufacturer retains the legal responsibility for the

quality of his product. Once the product is found by the

testing laboratory to be in compliance with the appropriate

standard, the name of both the product and its manufacturer can

be placed on a qualified products list (QPL) . As long as his

product remains on the published QPL, the manufacturer is

obligated to make that product in compliance with the product

standard.

Two types of product compliance testing are of interest.

Product qualification tests are performed in advance of and

independent of any specific procurement action, for the purpose

of establishing a QPL. The fact that a product has been tested

and placed on a QPL means that the particular model meets the

requirements of the standard. The inclusion of a product on a

QPL does not, however, guarantee the acceptability of every

individual lot or shipment. This assurance can be obtained

from product acceptance tests , which are performed to determine

the acceptability of delivered items which have been purchased

under a contract requiring compliance with the appropriate

standard.

The Compliance Testing System must be based on technical

competence and integrity if it is to enjoy any significant

degree of acceptance. For example, the testing laboratories

must be independent laboratories. Independent laboratories can

be commercial testing laboratories (there are several thousand

in the United States) , non-profit institution laboratories,

government laboratories, or university laboratories. By

definition, an independent laboratory cannot be connected with

the manufacturer of the product, cannot be involved in the

promotion of the product, and cannot have an inordinate (for

purposes considered here, greater than 10 percent) amount of

its income derived from a company which manufactures a product

3.
of the type under test. A corporate or captive laboratory

which is controlled by a manufacturer or distributor is

obviously not an unbiased, independent laboratory, even though

it may be a very competent one; however, the laboratory

accreditation program described later can benefit captive as

well as independent laboratories.

Other safeguards could readily be incorporated into the

program. Renewal qualification tests will be required, to

assure that changes in product design have not been made,

either intentionally or unintentionally. When possible,

manufacturers should be required to select different testing

laboratories for the original and the renewal qualification

testing of a product. In addition, the results of any

acceptance tests of a given product could be compared for

consistency with the qualification test results. Products for

which comparatively large amounts of acceptance test results

were available should require less frequent renewal testing to

stay on the QPL than those products for which few acceptance

test data were available.

The samples tested in qualification tests must be

representative of those produced in production runs. If sales

volume is large enough, this is best accomplished by having the

samples purchased anonymously through a regular retail

supplier. However, purchasing the test samples may, in some

cases, add significantly to the cost of the compliance testing.

There may also be some legal obligation to purchase and test

all available brands. This could lead to difficulties;

conceivably, a manufacturer could insist that MILECJ purchase a

product which obviously would not meet the requirements of the

standard, and he could demand an exorbitant price.

An alternative is to invite all manufacturers of law

enforcement equipment for which L'lLECJ standards have been

promulgated to submit their products for qualification testing,

rianufacturers would be required to certify that the submitted

products were representative of their production in all

significant respects. Significant differences between

qualification and acceptance test results would then indicate a violation of that product's certification or trouble with one

of the testing laboratories, either of which would require

remedial action.
Tlie methods used for qualification and acceptance testing

must be identical. However, economic considerations may

dictate that acceptance testing be less thorough than

qualification testing. In that case, the methods for

acceptance testing might consist of a subset of the test

methods for qualification testing. LESL could recommend the

appropriate subset. More than one subset may have to be

available to accommodate different local economic

circumstances. LESL should make certain that the purchaser is

properly advised by the qualified testing laboratory of the

reduced quality assurance to be expected for a given subset of

test methods. Advice should also be available concerning

appropriate sampling plans.

Copies of all test reports from participating laboratories

should be submitted to the compliance program administrators,

whether the items under test pass or fail. This procedure will

preclude reporting only successful test results, and will

prevent a manufacturer from repeatedly having an unacceptable

product tested until, by the laws of probability, it is

eventually passed. To guard against incomplete information

being sent, the complete laboratory job file should be open to

inspection by the program administrators.

The possibility of authorizing product manufacturers to

affix a NILECJ label or logo to products listed on a QPL has

been considered. Use of such a logo would have value in

bringing the NILECJ Equipment Program to the attention of law

enforcement officials. However, the use of a logo does have

the following disadvantages;

(a) A product may be tested and found acceptable for one

application, but not another. For example, a particular police

helmet may give satisfactory protection as a motorcycle crash

helmet, but not as a ballistic helmet. Specific product

certification cannot be communicated with a logo as it can be

with a QPL. A user who sees the logo but does not check the

QPL can develop an unjustified confidence in the utility of his

equipment. Careful labeling could minimize this problem.

(b) Logos are usually intended for equipment that has

passed qualification tests. A logo may, however, lead a user

to assume that an item has passed both qualification and

5
acceptance tests. That is, he may improperly assume that the

logo guarantees the quality of the shipment he receives.

(c) Comparatively speaking, potential enforcement problems

for QPL's should be smaller than for logos. That is, entries

on a QPL would clearly be under the control of NILECJ, but

there may be difficulties in forcing a manufacturer to stop

affixing the logo to products which no longer comply with the

standard, or in neutralizing the logo on those already in

distribution. There is, of course, the similar problem

involved in assuring that a procurement decision is based on a

current, not an obsolete, QPL.

Laboratory Lvaluation and Accreditation

Testing laboratories other than NBS will have to be

involved with compliance testing. Accreditation of these

testing laboratories will be an important part of the

Compliance Testing System. Only accredited laboratories should

be eligible for selection to perform qualification testing, and

only accredited laboratories should be recommended to police

officials who are selecting a laboratory for acceptance

testing . . . Laboratory accreditation should be distinguished from

laboratory certification. Certification implies a warranty or

guarantee of a laboratory's performance. There is no valid

approach to examining a laboratory so that guarantees of future

performance can be issued. Accreditation resulting from

laboratory evaluation does not ensure the quality of future

output. Rather, it indicates that there is no apparent reason

why the laboratory cannot perform a function adequately. The

only realistic goal of laboratory evaluation is accreditation,

not certification. However, laboratory performance can be

monitored after the initial accreditation (a) by comparing data

sent to the Compliance Information System from different

laboratories testing the same product, (b) via occasional

unannounced examinations of accredited laboratories and (c) by

requiring that accreditation be renewed every few years.

Laboratory accreditation can be revoked in cases of

misrepresentation of information or unsatisfactory performance.

Accreditation would be granted for those testing functions

of laboratories that meet pre-stated criteria. These criteria

will, for the most part, relate to the methods of test in
specific x.ILLCJ standarus, although sone criteria such as

raanacjerial independence froia luanuf acturers and vendors and

records keeping will have general applicability. The Law

Lnforceraent Standards Laboratory at the "National Bureau of

Standards is currently developing laboratory evaluation

criteria for the first few performance standards promulgated by

iilLLCJ . Laboratory accreditation criteria and evaluations can be

classified as either per f ornance-based or non-per f ornancebased. i.on-perf ormance-based criteria can relate, for example,

to the cleanliness of laboratory equipment or facilities, or to

the academic backgrounds of personnel. Performance-based

criteria, on the other hand, relate to the actual results of

tests on controlled samples by the laboratory under

examination

.

The objective of non-performance-based evaluation is to

predict performance. Since performance-based evaluations are

intierently more direct, they will be used to the greatest

extent practical.

Laboratory accreditation criteria can also be classified as

either technical or nontechnical. Technical evaluations of

testing laboratories are made to determine if tliey have the

capacity to carry out properly the tests specified in the

product performance standards. ijontechnical evaluations which

are also appropriate and have been mentioned earlier are

managerial independence from manufacturers and vendors, and

recoras keeping. /another nontechnical criterion which may be

relevant concerns equal employment opportunity (LLC) at the

testing laboratories. Tlie relevance of this criterion is a policy question best answered by i:iLLCJ.

The checking of laboratories to determine whether the prestated criteria are satisfied can be considered to be a threestep process--examination and evaluation by LLSL followed, if

merited, by ImILLCJ accreditation. Laboratory e:caminations

provide the information upon which laboratory evaluations are

made, v;h.ich in turn serve as the basis for LLSL ' s reccmraendation to LILLCJ for laboratory accreditation. On-site

examinations at the laboratories are expensive in term.s of

required travel and mcanpower . To the extent practical,

examinations will involve the laboratories in question sending

informLation to LLSL for evaluation. Lcwever , a few on-site
exaiainations vvill have Lo be made, some on an unannounced

basis, to check the information furnished by the laboratories.

Laboratories will be evaluated and accredited on a standard-by-standard basis. Isolation of the specific causes

of problems being experienced by laboratories failing to meet

certain performance criteria will be outside the scope of this

program unless a problem is widespread among testing

laboratories involved with the Compliance Testing System, or

the basis for the problem can be established with very minor

effort.

Compliance Information System

A "Compliance Information System" is needed as part of the

Compliance Testing System for the purpose of eliciting

information concerning the performance of devices listed on the

l.ILLCJ qualified products lists. Reports will be tabulated

from laboratories performing qualification and/or acceptance

testing. Cooperation in supplying this information will be a condition for being an accredited laboratory. In addition,

field performance data will be solicited from law enforcement

agencies using the equipment covered by KILECJ standards.

The Comipliance Information System v/ill help to assure that

(a) equipment listed on a IjILLCJ qualified products list

continues to perform in accordance with the requirem.ents of the

IJILECJ standard, (b) experience in field use does not indicate

an excessive breakdown rate or the existence of m.aintenance

problems, (c) any need for revision of the standard is

recognized, and (d) problems in the performance of any of the

testing laboratories are recognized. V7hen tabulated

information indicates the need, a special investigation will be

made. If, for example, it is determined that a manufacturer

has a quality control problem, continued listing of his product

on the QPL will be reviewed. If an excessive field breakdown

or maintenance problem is found, either the product will be

removed from the QPL or the applicable standard will be

revised, as appropriate. If a problem in testing laboratory

performance is found, the continued accreditation of that

laboratory will be reviewed.

Ihe National Highway Traffic Safety Administration (NIITSA)

,

Department of Transportation, announced in the Ilovember 5,

1973, issue of the Federal Register, page 30159, the

f
establishment of a program having a Standards Compliance

Information System and many other features similar to those

recommended for the NILECJ Compliance Testing System. The

NHTSA Compliance Testing System was planned with NBS/LESL

consultation; the breath alcohol testing equipment performance

standards for which it was established are being developed for

NHTSA by LESL. It should be noted that only equipment on the

UHTSA qualified products list may be purchased with NHTSA

funds — a policy which NILECJ may also wish to adopt.

Role and Qualifications of NBS

The large amount of both product qualification and

acceptance testing necessitates the involvement of testing

laboratories other than NBS. NBS/LESL 's role is in the

development of the standards, it^ the arbitration—if needed—of

any disputes which may arise concerning compliance test

results, and in the management of the compliance testing

system, but not in compliance testing per se. All of these

activities will of course be under the direction of NILECJ.

Each QPL will be a NILECJ document. Its content will be

based largely on the information obtained from the Compliance

Testing System. Likewise, the official list of accredited

laboratories will be a NILECJ document, although the

accreditations will be based on evaluations performed by

NBS/LESL.

NBS has had and continues to have considerable involvement

with laboratory evaluations. Current NBS projects involving

laboratory evaluation relate to the testing of cement and

concrete, paper and paper boxes, rubber, color and appearance,

mobile homes, mass standards, state metrology laboratories,

clinical chemistry and toxicology. A special Laboratory

Performance Technology Section has been established at NBS.

NBS management, together with other officials in the U.S.

Department of Commerce, are planning with industry and

professional associations the establishment of a national

system for laboratory evaluation; several public hearings have

been held on the subject at NBS.
Although i>iijS has almost no regulatory authority, it has

established a reputation of being a fair and careful arbitrator

iii disputes involving measurement. Members of the UBS staff

liave been among the vanguard in the development of improved

techniques for laboratory evaluation. An Association of

Official Analytical Chemists publication, "Statistical

Techniques for Collaborative Tests," v/as written by an L'BS

statistician, Dr. Vv. J. Youden. Chapter three of UBS Special

Publication 300 titleu " Interlaboratory Tests" is a collection

of papers on the subject by UBS authors. The editor of the

"ASTM bulletin" included the following quotation in his

introductory remarks to a paper about interlaboratory testing

by two liBS scientists: "This paper gives a very complete

treatment of the problem which almost every ASTM committee is

constantly trying to solve . . . (it is) a more comprehensive

approach to the problem of designing and interpreting

interlaboratory studies than has appeared in the literature up

to now." (ASTM Bulletin, July 1959, page 53).

During the development of performance standards, LESL must

do a certain amount of product testing to determine current

performance levels. This work is rarely sufficient to negate

the need for qualification tests after promulgation of the

standard, but pertinent test results obtained during the

development of the standard should certainly be utilized by the

Compliance Testing System.. Conversely, the compliance test

results vvill serve as early indicators that particular IJILECJ

standards should be considered for revision. Thus, close

liaison between the standards developm.ent program and the

Compliance Testing Program will be essential. This two-way

flow of information will be enhancea by both programs being

within L£SL.

Who Pays?

Uho should pay the costs of a national compliance testing

system for law enforcement equipment? Let us consider

separately each of the component parts of a complete,

coiuprehensive system.

Laboratory Lvaluation and Accreditation

In and of itself, a laboratory evaluation and accreditation

program is of no value. Such a program is, however, central to

any coiupliaiice testing systen, no matter hov; sim.ple or complex,

10.
large or snail. I.one of the otlicr possible components of gucI^ a systen-, can function without the availability of testinq

laboratories v;hose test results are accepted by both the buyer

and seller. These considerations under ly the question of v.-ho should pay for laboratory evaluation and accreditation.

In a mature, functionincj corr.pliance testincj system, we

recouuuend that most, if not all, of the full pro-rata cost of

evaluating and accrediting a testing laboratory be borne by

tiiat laboratory itself. In a competitive environment, this

would assure that accreditation would be sought by only those

testing laboratories to wliom it was financially attractive.

Clearly, this would be the most cost-effective and equitable

arrangement

.

In starting up such a program, however, when it is not

clear how much testing business will be generated, and who the

customers will be, it would be unreasonable and unrealistic to

expect the testing laboratories to bear the costs of their

evaluation and accreditation, plus the program start-up costs.

It is recomraenaed tliat the sponsor of the program, iJILLCJ, pay

the major portion or all of these costs, in order to get the

prograr.i under way and to attract as many testing laboratories

as possible into the program. It is further recommended that

the proportion of the costs borne by the testing laboratories

themselves be increased gradually, but as rapidly as the

circumstances warrant, until the program is fully selfsupporting, or nearly so. It must be realized that, for each

equipment item in the program, at least one testing laboratory

must obtairi enough business to justify its cost of obtaining

and maintaining accreditation. 'io assure this, some continuing

HILLCJ subsidy may be required, in particular cases.

Manufacturer Certification Based on Accredited

Laboratory Testing

Under this system option, once one or more testincj

laboratories are accredited to test an equipment item in

accordance with the appropriate rJILL;CJ standard, any and all

manufacturers of that item would be free to contract

individually with the accredited testing laboratory of their

choice, at a fee agreed to by them, to have their products

tested for compliance. The manufacturers could use these test

results in their advertising and/or marketing, could use them

as a basis for certifying Uiat their products complied with the

11.
appropriate standard, anu could therefore bid on procurement

contracts that required compliance with the IIILLCJ standard.

Vvhile sorae rationale exists for LILECJ financial support of

this program option, no simple means is envisioned for doing

so. It is therefore recommended that payment for services

under this program be fully borne by the individual

manufacturer or distributor. Ihe operation of this systen

would involve little or no Government participation; it would

probably be little used if Qualified Products Lists were

establishea (see below)

.

Routine Acceptance Testing

One of the options available to a purchaser, after he has

received a shipment of material claimed to comply v/ith the

requirements of a i.ILLCJ standard, is to verify its compliance

by having tiie shipment tested by a qualified testing

laboratory. uhere such acceptance testing is desired, it

should be performed on a f ee-for-service basis, entirely paid

for by the purchaser and/or by the supplier, in accordance v;ith

the requiremerit of their purchase agreement.

It is recoruucrided that the States and local jurisdictions

oe authorized and encouraged, but not required, to [jay for such

acceptance tests v/ith LL7iA block-grant or other LFAA funds

allocated to then. This authorisation should govern whether or

not the equipment itself is paia for with Lr:AA funds.

, Qualified Products lists

The existence of an official MLI'CJ Qualified Products List

for a particular equipm.ent itoni would piakc it very easy for a purchaser to acquire I.iILi;CJ-Gtai!card equipment. In sone cases,

such as v;hen the cost of testing was large compared to the cost

of the shipment, reference to a QPL v/ould be perhaps the only

way in which an objective, cost-effective purchase could be

made

.

The establishment and maintenance of a QPL for each

appropriate equipment item is thus a major tool whereby I.ITiECJ

could assure the quality of equipment procured by State and

local agencies. Similarly, the listing of its product on a iJILLCJ QPL could be of substantial economic benefit to an

equipment manufacturer. Joint funding of a QPL by L'lLLCJ and

the interested manufactuorer s is therefore warranted, and is

recomraended

.

A OPL program sJiould be run Ijv LEAA/!JILECJ itself, or on

its I)ehalf by another public agency.
r GLOSSARY

This Glossary is intended to assist the reader understand

the terminology by comparing and contrasting related terms.

The entries are grouped into three sections relating to the

subject areas of product standards, product evaluation and

laboratory accreditation. Related entries within each section

are also grouped rather than listed alphabetically. However,

an alphabetical listing of entries is more convenient for

locating the definition of a specific term. Hence, a separate

Index of Terms is included at the back of the Glossary.

Many of the definitions in this Glossary are based on

definitions in an unpublished document drafted earlier at NBS.

Some terms in that document are included in this glossary even

though they are not used in this report concerning compliance

testing. Terms such as, "mandatory standard", "quasi-mandatory

standard", and "consensus standard" are of general interest to

people working with standards and thus were included in this

Glossary.

13.
1, PRODUCT STANDARDS

1.1 STANDARD

A prescribed set of conditions or requirements—the

physical, functional, performance, or conformance

characteristic thereof—to be satisfied by a product.

Standards included under this definition are called product

standards or engineering standards.

1.1.1 VOLUNTARY STANDARD

A standard with which there is no obligation to comply. A

voluntary standard may become a quasi-mandatory standard,

mandatory standard, code, regulation, or rule as a result of

utilization or adoption by a regulatory authority.

1.1.2 QUASI -MANDATORY STANDARD

A Standard with which there is no legal obligation to

comply, but which is required in practice or under certain

conditions, such as a requirement of a marketplace or

compatibility with other products.

1.1.3 MANDATORY STANDARD

A Standard with which there is an obligation to comply by

virtue of an action by government or by an authority endowed

with the necessary legal power; called a code, regulation, or

rule

.

1.1.4 CONSENSUS STANDARD

A standard for which there is general agreement among those

affected by the standard that the prescribed set of conditions

and requirements are technically sound and meet the needs

prevailing at that time.

1.1.5 PERFORMANCE STANDARD

A standard which prescribes the acceptable functional or

operational characteristics of a material, product or system in

accordance with the use to which the performance would apply;

includes or references the test methods by which these

characteristics are measured.

14.
1.1.6 DESIGN STANDARD

A Standard which describes the required physical or

dimensional characteristics of a product or system and, in some

cases, its manufacture, construction or fabrication.

1.1.7 NILECJ STANDARD

A voluntary standard developed by the Law Enforcement

Standards Laboratory at the National Bureau of Standards and

promulgated by the National Institute of Law Enforcement and

Criminal Justice (NILECJ), U.S. Department of Justice, for

equipment used in lav/ enforcement, crime deterrence, or

criminal justice activities. NILECJ Standards are technical

documents consisting of performance and other requirements

together with a description of test methods.

1.2 TEST METHOD

A description of the test procedures, equipment and

methodology for testing a material or product in determining

its conformance to a standard or other set of conditions and

requirements

.

1.3 REFERENCE I^TERIAL

A material, substance, or device whose intrinsic properties

are used for physical comparison (e.g., an NBS Standard

Reference Material (SRM) which is a well characterized and

certified material produced in quantity and used to develop

reference methods of analysis and tests and to calibrate

measurement systems), sometimes referred to as a standard.

2. PRODUCT EVALUATION

2.1 PRODUCT COMPLIANCE TEST

A test to determinate the design or performance

characteristics of products for the purpose of establishing

their conformance with the requirements of applicable

standards, codes or other requirements.
2.1.1 PRODUCT ACCEPTANCE TEST

A compliance test to determine the acceptability of

delivered items which have been purchased under a contract

requiring compliance with the appropriate standard, code, or

other requirement.

2.1.2 PRODUCT QUALIFICATION TEST

A compliance test performed in advance of and independent

of any specific procurement action, for the purpose of

establishing a qualified products list or authorizing the use

of a logo.

2.2 QUALIFIED PRODUCTS LIST (QPL)

A list of products, and their manufacturer, which have been

tested and found to comply with the requirements of applicable

standards, codes or other requirements.

2

.

3 LOGO

A symbol, label, hallmark or statement authorized by a certification agency for affixing to a product indicating that

the product is in conformance v/ith specified provisions of the

applicable standard, code or other requirement.

, ; 3. LABORATORY ACCREDITATION

3 . 1 ACCREDITATION

The act of giving official recognition of specific

qualifications. Accreditation of a laboratory indicates that

there is no apparent reason why the laboratory cannot perform a

function adequately. It does not guarantee the quality of

future performance.

3.2 CERTIFICATION

The act of giving an official warranty or guarantee of

performance. There is no valid approach to examining a

laboratory for laboratory certification because guarantees of

future performance cannot be made.

3.3 TESTING LABORATORY

A place equipped and staffed to conduct product tests

including product compliance tests.

16.
3.3.1 INDEPENDENT TESTING LABORATORY

A testing laboratory which has no organizational tie or

financial interest in a manufacturer, vendor, or the promotion

of a specific product on which tests are performed. It has

sufficient breath of activity so that the loss or award of a specific contract for test services would not be a substantive

factor in the financial well-being of the laboratory. It may

offer test services under contract or on a fee basis, and may

be a profit or non-profit organization.

3.3.2 CORPORATE OR CAPTIVE TESTING LABORATORY

A testing laboratory organizationally affiliated with a product manufacturer or vendor.

3.4 LABORATORY EXAMINATION

The process of obtaining information in order to judge the

laboratory's capability for performing specified product tests,

e.g., product compliance tests. Laboratory examinations

provide information upon which laboratory evaluations are made.

Laboratory examinations can involve on-site and remote

activities; the remote activities involve the laboratories

sending information to the evaluation agency.

3.5 LABORATORY EVALUATION

The decision making process, using information from

laboratory examinations, upon which laboratory accreditation

or a recommendation for laboratory accreditation—is based.

3.5.1 LABORATORY EVALUATION CRITERIA

Statements prescribing the organizational and technical

resources, the equipment and facilities, the operational

procedures, and the minimum technical performance levels

required of a testing laboratory for accreditation.

3.6 COLLABORATIVE REFERENCE TESTING

A program or system in which a uniform material or product,

properly randomized or of controlled characteristics, is

distributed to participating laboratories. The laboratories

test the material or product and submit the test results to the

agency administering the collaborative test for evaluation.

Collaborative reference testing may be used voluntarily by

testing laboratories for self -evaluating or improving test

criteria and methodology. In conjunction with other procedures, collaborative reference testing may be used in laboratory

examination for accreditation or continued accreditation.
INDEX OF TERMS

Section

Accreditation 3.1

Certification 3.2

Collaborative Reference Testing 3.6

Consensus Standard 1.1.4

Corporate or Captive Testing Laboratory . . .3.3.2

Design Standard 1.1.6

Independent Testing Laboratory 3.3.1

Laboratory Accreditation 3

Laboratory Evaluation 3.5

Laboratory Evaluation Criteria 3.5.1

Laboratory Examination 3.4

Laboratory Examination Methodology 3.4.1

Logo 2.3

Mandatory Standard 1.1.3

NILECJ Standard 1.1.7

Performance Standard 1.1.5

Product Acceptance Test 2.1.1

Product Compliance Test 2.1

Product Evaluation 2

Product Qualification Test 2.1.2

Product Standards 1

Qualified Products List (QPL) 2.2

Quasi-Mandatory Standard 1.1.2

Reference Material 1.3

Standard 1.1

Testing Laboratory 3.3

Test Method 1.2

Voluntary Standard 1.1.1

"1
NBS-1 14A <REV. 7-73)

U.S. DEPT. OF COMM. BIBLIOGRAPHIC DATA

SHEET

1. PUBLICATION OR REPORT NO. NBSIR 75-719

2. Gov't Accession

No.

3. Recipient's Accession No.

4. TITLE AND SUBTITLE

A Compliance Testing System for NILECJ

5. Publication Date February 1976

6. Performing Organization Code

7. AUTHOR(S)

Robert Mills

8. Performing Organ. Report No.

9. PERFORMING ORGANIZATION NAME AND ADDRESS

NATIONAL BUREAU OF STANDARDS

DEPARTMENT OF COMMERCE

WASHINGTON, D.C. 20234

10. Project/Task/Work Unit No.

11. Contract/Grant No.

12. Sponsoring Organization Name and Complete Address (Street, City, State, ZIP)

National Institute of Law Enforcement and Criminal

Justice, LEAA

Department of Justice

Washington, D.C. 20531

13, Type of Report & Period Covered

Final

14. Sponsoring Agency Code

15. SUPPLEMENTARY NOTES

16. .ABSTRACT (A 200-word or less [actual summary ol most sigpificant information. If document includes a significant

bibliography or literature survey, mention it here.)

In order to assure that NILECJ equipment standards have the impact

intended, a Compliance Testing and Laboratory Accreditation program

IS needed to establish which items available on the market do, in

fact, meet the requirements of the standards. This report contains recommendations for such a NILECJ program. In brief, the proposed

program would (a) result in a body of qualification and acceptance

test data, (b) establish a list of testing laboratories competent

to perform these tests, and (c) set up a "compliance information

system" for the dissemination of this information to officials in

the criminal justice system.

17. KEY WORDS (six to twelve entries; alphabetical order; capitalize only the first letter of the first key word unless a proper name; separated by semicolons) Acceptance testlng; compllance testing; laboratory

evaluation; performance testing; qualified products lists; testing.

18. AVAILABILITY [X Unlimited

I

' For Official Di stribution. Do Not Release to NTIS

1

' Order From Sup. of Doc, U.S. Government Printing Office Washington, D.C. 20402, SD Cat. No. C13

IX ' Order From National Technical Information Service (NTIS)

Springfield, Virginia 22151

19. SECURITY CLASS

(THIS REPORT)

UNCL ASSIFIED

20. SECURITY CLASS

(THIS PAGE)

UNCLASSIFIED

21. NO. OF PAGES

24

22. Price

$3.50

USCOMM-DC 29042-P74