A111D3 073133

NATL INST OF STANDARDS & TECH R. .C

A1 11 030731 33

National Conference /Testing laboratory QC100 .U57 N0.591, 1980 C.1 NBS-PUB-C 19

-I NBS SPECIAL PUBLICATION 591

U.S. DEPARTMENT OF COMMERCE / National Bureau of Standards

National Bureau of Standards

Library, _ E-01 Admin. Bldg.

OCT 1 1981

131031

(>

Testing Laboratory Performance:

Evaluation and Accreditation
NATIONAL BUREAU OF STANDARDS

The National Bureau of Standards' was established by an act of Congress on March 3, 1901. The Bureau's overall goal is to strengthen and advance the Nation's science and technology

and facilitate their effective application for public benefit. To this end, the Bureau conducts

research and provides: (1) a basis for the Nation's physical measurement system, (2) scientific and technological services for industry and government, (3) a technical basis for equity in trade, and (4) technical services to promote public safety. The Bureau's technical work is per- formed by the National Measurement Laboratory, the National Engineering Laboratory, and

the Institute for Computer Sciences and Technology.

THE NATIONAL MEASUREMENT LABORATORY provides the national system of

physical and chemical and materials measurement; coordinates the system with measurement

systems of other nations and furnishes essential services leading to accurate and uniform

physical and chemical measurement throughout the Nation's scientific community, industry,

and commerce; conducts materials research leading to improved methods of measurement,

standards, and data on the properties of materials needed by industry, commerce, educational

institutions, and Government; provides advisory and research services to other Government

agencies; develops, produces, and distributes Standard Reference Materials; and provides

calibration services. The Laboratory consists of the following centers:

Absolute Physical Quantities 2 — Radiation Research — Thermodynamics and

Molecular Science — Analytical Chemistry — Materials Science. THE NATIONAL ENGINEERING LABORATORY provides technology and technical ser- vices to the public and private sectors to address national needs and to solve national problems; conducts research in engineering and applied science in support of these efforts; builds and maintains competence in the necessary disciplines required to carry out this research and technical service; develops engineering data and measurement capabilities;

provides engineering measurement traceability services; develops test methods and proposes

engineering standards and code changes; develops and proposes new engineering practices;

and develops and improves mechanisms to transfer results of its research to the ultimate user. The Laboratory consists of the following centers:

Applied Mathematics — Electronics and Electrical Engineering 2 — Mechanical

Engineering and Process Technology2 — Building Technology — Fire Research —

Consumer Product Technology — Field Methods.

THE INSTITUTE FOR COMPUTER SCIENCES AND TECHNOLOGY conducts

research and provides scientific and technical services to aid Federal agencies in the selection,

acquisition, application, and use of computer technology to improve effectiveness and

economy in Government operations in accordance with Public Law 89-306 (40 U.S.C. 759), relevant Executive Orders, and other directives; carries out this mission by managing the Federal Information Processing Standards Program, developing Federal ADP standards

guidelines, and managing Federal participation in ADP voluntary standardization activities; provides scientific and technological advisory services and assistance to Federal agencies; and

provides the technical foundation for computer-related policies of the Federal Government.

The Institute consists of the following centers:

Programming Science and Technology — Computer Systems Engineering.

'Headquarters and Laboratories at Gaithersburg, MD, unless otherwise noted; mailing address Washington, DC 20234. 'Some divisions within the center are located at Boulder, CO 80303.
Testing Laboratory Performance:

Evaluation and Accreditation

Nations! Bureau of StatMftlftt Uhrary. P.-01 Admin. AUG 2 7 1980

/7p-d Actdado

.US 7

Proceedings of a National Conference

held at the National Bureau of Standards,

Gaithersburg, Maryland, September 25-26, 1979

Gerald A. Berman, Editor

Office of Engineering Standards

National Engineering Laboratory

National Bureau of Standards

Washington, DC 20234

/

^•CAU Of " a

U.S. DEPARTMENT OF COMMERCE, Philip M. Klutznick, Secretary

Luther H. Hodges, Jr., Deputy Secretary

Jordan J. Baruch, Assistant Secretary for Productivity, Technology and Innovation

NATIONAL BUREAU OF STANDARDS, Ernest Ambler, Director

Issued August 1980
Library of Congress Catalog Card Number: 80-6001 10

National Bureau of Standards Special Publications 591

Nat. Bur. Stand. (U.S.), Spec. Publ. 591, 179 pages (Aug. 1980) CODEN: XNBSAV

U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON: 1980

For sale by the Superintendent of Documents, U.S. Government Printing Office, Washington, D.C. 20402

Price $5.50

(Add 25 percent for other than U.S. mailing)
PREFACE

TESTING LABORATORIES ARE BEING SUBJECTED TO EVER INCREASING

NUMBERS OF EXAMINATIONS, AUDITS, AND INSPECTIONS TO ENSURE THEIR

TESTING CAPABILITY, TO DATE, OVER 60 PROGRAMS, BOTH PUBLIC AND

PRIVATE, HAVE BEEN IDENTIFIED WHICH EVALUATE TESTING LABORATORIES

AND BESTOW ACCREDITATION IN ONE FASHION OR ANOTHER FOR TESTS CONDUCTED ,

THE PAPERS IN THESE PROCEEDINGS WERE PRESENTED AT A NATIONAL CONFERENCE ON TESTING LABORATORY PERFORMANCE EVALUATION

AND ACCREDITATION HELD AT THE NATIONAL BUREAU OF STANDARDS ON

SEPTEMBER 25"26, 1979, THE PURPOSE OF THE CONFERENCE WAS TO

PROVIDE A FORUM FOR THE DISSEMINATION OF INFORMATION AND THE DISCUSSION OF TECHNIQUES AVAILABLE FOR EVALUATING THE PERFORMANCE

OF TESTING LABORATORIES, AND ALSO TO PROVIDE INFORMATION ON THE OPERATIONAL ASPECTS OF SOME EXISTING AND PROPOSED ACCREDITATION

PROGRAMS AND SYSTEMS,

THE CONFERENCE WAS ORGANIZED THROUGH THE OFFICE OF TESTING

LABORATORY EVALUATION TECHNOLOGY AT THE NATIONAL BUREAU OF STANDARDS

BY DR. GERALD A. BERMAN WHO SERVED AS PROGRAM COORDINATOR AND

GENERAL CHAIRMAN, THE PROGRAM CONSISTED OF 29 PAPERS WHICH ADDRESSED

VARIOUS TECHNIQUES FOR EVALUATING THE PERFORMANCE OF TESTING LABORATORIES, QUALITY CONTROL ASPECTS OF THE TESTING FUNCTION, EXISTING

AND PROPOSED ACCREDITATION PROGRAMS AND SYSTEMS, AND INTERNATIONAL

COORDINATION,

EXCEPT FOR SOME EDITORIAL CHANGES, THE PAPERS IN THESE PROCEEDINGS ARE PRESENTED AS SUBMITTED BY THE AUTHORS AS CAMERA-READY

COPY,

iii
ACKNOWLEDGEMENTS

APPRECIATION IS EXTENDED TO THE FOLLOWING INDIVIDUALS FOR

THEIR OUTSTANDING EFFORTS IN CONTRIBUTING TO THE SUCCESS OF THE

CONFERENCE:

SESSION CHAIRMEN:

GERALD A. BERMAN, NATIONAL BUREAU OF STANDARDS

JOHN R. DISE, NATIONAL BUREAU OF STANDARDS

HOWARD I. FORMAN, DEPARTMENT OF COMMERCE

EDMUND G. FRANZAK, SAND I A LABORATORIES

DENNIS H. GALLAGHER, LEEDS AND NORTHRUP COMPANY

JOHN A, GRANT, AMOCO OIL COMPANY

ROBERT D. MEDZ, U. S, ENVIRONMENTAL PROTECTION AGENCY

BANQUET SPEAKER:

WILLIAM T. CAVANAUGH, MANAGING DIRECTOR

AMERICAN SOCIETY FOR TESTING AND MATERIALS

MEETING ARRANGEMENTS AND COORDINATION:

JOANN LORDEN, NATIONAL BUREAU OF STANDARDS

MARGARET HARPER, NATIONAL BUREAU OF STANDARDS

REGISTRATION, FINANCIAL MATTERS AND GENERAL OVERALL ASSISTANCE:

PAULA M. GARRETT, NATIONAL BUREAU OF STANDARDS

TYPING:

JACKIE ELLINGTON, NATIONAL BUREAU OF STANDARDS

•

iv
TABLE OF CONTENTS

PAGE

PREFACE 111

ACKNOWLEDGEMENTS IV

SESSION I

OVERVIEW OF EVALUATION AND ACCREDITATION

USER EXPERIENCES WITH LABORATORY ACCREDITATION 3 JAMES E, FRENCH

LABORATORY ACCREDITATION - - STATE-OF-THE-ART IN 1979 6 JOHN W. LOCKE

GENERIC STANDARD BASIS FOR LABORATORY ACCREDITATION 11 HARVEY E. SCHOCK, JR.

LABORATORY PERFORMANCE EVALUATION SERVICES OF THE U, S, NATIONAL BUREAU OF STANDARDS 15 JAMES 0. BRYSON, BRIAN C. BELANGER, R, KEITH KIRBY

SESSION II: EVALUATING TECHNOLOGY

THE MEASURING PROCESS AND LABORATORY EVALUATION 2 5 THEODORE W. LASHOF

INTER-LABORATORY ROUND ROBINS FOR DETERMINATION OF ROUTINE

PRECISION OF METHODS 31 CHARLES A, BICKING

MONSANTO 'S ANALYTICAL TESTING PROGRAM 3 5 DAVID M. HAILE AND WILLIAM M. HAYNES

LABORATORY EVALUATION TECHNIQUES - U. S. ARMY CALIBRATION

PROGRAM 41

CHARLES B. COULTER

THE VALUE OF SPLIT SAMPLE PROGRAMS FOR CONTRACTUAL

ACCEPTANCE OF IN-HOUSE LABORATORY PROCEDURES 45

RICHARD A, MULL INS

v
TABLE OF CONTENTS (CONT.)

PAGE

SESSION III: HEALTH SERVICES ACCREDITATION PROGRAMS

SUMMARY OF EVALUATIONS OF CLINICAL LABORATORIES PARTICIPATING

IN THE CENTER FOR DISEASE CONTROL EVALUATION PROGRAM 53 LOUIS C. LAMOTTE, JR.

EVALUATION AND APPROVAL OF LABORATORIES IN CONNECTICUT

IMPORTANCE OF VOLUNTARY STANDARDS 63

JESSE S. TUCKER

THE LABORATORY ACCREDITATION PROGRAM OF THE AMERICAN

INDUSTRIAL HYGIENE ASSOCIATION 67 FREDERICK I. GRUNDER

COLLEGE OF AMERICAN PATHOLOGISTS INSPECTION AND ACCREDITATION

PROGRAM 71 FRANCES RYAN

LABORATORY ACCREDITATION FOR TOXICOLOGY FACILITIES 75 HARRY W. HAYS

SESSION IV; ACCREDITATION SYSTEMS AND CONCEPTS

LABORATORY PERFORMANCE EVALUATION AND ACCREDITATION - IT'S ALL

IN THE IMPLEMENTATION 7

WILLIAM J, SMITH AND G. THOMAS CASTINO

LABORATORY ACCREDITATION BY DISCIPLINES 85

ROGER J, AMOROSI

ACCREDITATION PROGRAM FOR CANADIAN TESTING ORGANIZATIONS 88

ROB IE E. MacNINTCH

ECONOMIC EFFECT OF LABORATORY ACCREDITATION 92 DAVID KRASHES

VI
TABLE OF CONTENTS (CONT.)

PAGE

SESSION V! QUALITY CONTROL

THE ROLE OF THE QUALITY CONTROL MANUAL IN THE INSPECTION

AND TESTING LABORATORY g

ROBERT J. WENING

LABORATORY QUALITY PROGRAM REQUIREMENTS 104

THOMAS A, RATLIFF, JR.

QUALITY CONTROL PROCEDURES IN A LABORATORY TESTING CEMENT

FOR COMPRESSIVE STRENGTH 109

RICHARD D. GAYNOR AND RICHARD C. MEININGER

STANDARD THERMAL PERFORMANCE TESTING PROCEDURES 124

JACK D. VERSCHOOR

LABORATORY PERFORMANCE EVALUATION/ A NEW LOOK AT QUALITY

ASSURANCE IN THE TESTING LABORATORY 131

DONALD J. MC CLAIN

SESSION VI: EVALUATION PROGRAMS AND SYSTEMS

A MEASUREMENT ASSURANCE PROGRAM - THERMOMETER CALIBRATION 137

GEORGE T, FURUKAWA AND WILLIAM R. BIGGE

THE CERTIFICATION OF BUILDING PRODUCTS IN THE UNITED STATES 146

LESLIE H. BR EDEN AND LYNFORD K, SNELL

LICENSING PROGRAMS FOR FIELD TECHNICIANS AND CONCRETE

LABORATORIES IN MASSACHUSETTS 156

GEORGE H, BRATTIN AND CHARLES G. HANAFIN

THE CRIME LABORATORY ACCREDITATION PROGRAM OF THE AMERICAN

SOCIETY OF CRIME LABORATORY DIRECTORS (ASCLD) 164

ANTHONY LONGHETTI

CERTIFIED LABORATORY PERFORMANCE EVALUATION 169

H, JAMES BARTH

vii
TABLE OF CONTENTS (CONT.)

PAG

SESSION VII: INTERNATIONAL COORDINATION

PROPOSED U, S. POSITION PAPER FOR THE THIRD INTERNATIONAL

CONFERENCE ON RECOGNITION OF NATIONAL PROGRAMS FOR

ACCREDITING TESTING LABORATORIES (lLAC) SYDNEY,

AUSTRALIA, OCTOBER 22-26, 1979 173

HOWARD I, FORMAN

viii
SESSION I

OVERVIEW

OF

EVALUATION

AND

ACCREDITATION

CHAIRMAN: DR, GERALD A. BERMAN

NATIONAL BUREAU OF STANDARDS

National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

USER EXPERIENCES WITH LABORATORY ACCREDITATION

James E. French

Technical Consultant

Cortland, NY

Laboratory accreditation serves a two-fold purpose in the

marketplace. This relatively new technical function can provide

increased confidence in the capability and reliability of testing

laboratories by customers of their services. The function can also serve as a marketing tool for laboratory managers. In this sense there is an analogy with product certification. Although laboratory

accreditation may eventually become a leveling force, much as accreditation of hospitals and academic departments within colleges

has become, the enterprising laboratory manager should not regard

accreditation as a least common denominator activity at this time.

Experience of testing laboratories in several disciplines where

accreditation has been available or required are reviewed. Long

range roles for the accrediting function are discussed.

Key words: Accreditation; certification; colleges; confidence;

criteria; evaluation; hospitals; inspectors; marketing; qualification;

re-examination; technicians.

1. Introduction

Accreditation of testing laboratories

by Third-party organizations is a relatively

new function in the marketplace for technical

services. For purposes of this presentation,

Laboratory accreditation shall be characterized as including technical and administrative evaluations with periodic reexaminations. Although some accreditation

activities have considerable historical

continuity, most of those activities stimulating interest in laboratory managers are

of recent origin. The characteristics of

laboratory accreditation activities mentioned

here are not exhaustive, but need not be in

order to review the perception of this new

function among its users.

2. User perceptions

of laboratory accreditation

The direct users of laboratory accreditation are the testing laboratories themselves. The beneficiaries include several

levels of users of laboratory services.

These will be discussed under trends.

Informal surveys and discussions with

laboratory managers reveal that those who

have no prior experience are wary of the cost/

benefit factor. Industry executives presently

seem suspicious that accreditation will have

a least common denominator or leveling effect.

This reaction has been noted with regard to other forms of standardization especially so among scientist/businessmen. Laboratory

managers who are looking to the future see accreditation as a long range marketing tool.

They know that it will be difficult to implement on a broad basis, difficult to achieve,

and difficult to market of itself. In a complementary sense, these same industry

leaders realize that the ability to compare

competing service organizations, the presence

of external monitors for laboratory quality

control, and published statements about the

capabilities of testing laboratories based

on common factors will provide a new discipline as the economy relies increasingly on scientific testing.
3. Analogy with product certification

The modern marketplace demands "both products and services . Each category is

becoming more complex as it responds to marketplace needs. Consumers, he they

residential, industrial, commercial, or institutional, increasingly need evidence to support their confidence in the products and

services which they purchase.

The response to this need for product

confidence has heen product certification.

This activity is conducted hy independent

laboratories , trade associations, model code

bodies and certification councils operating

under either commercial or governmental mandate. Many of these programs provide marketing tools within certain constraints. As

such activities become established and wellknown, there seems to be an increasing

reliance on the information which they

provide to consumers

The response to the need for supporting

evidence with respect to testing laboratory

services is laboratory accreditation. When

this activity is based on consensus criteria,

evaluation by qualified inspectors, and

periodic re-examination the analogy with

product certification holds. The voids in

completing the comparison of the two systems

are accreditation of certifiers, on one hand,

and qualification of inspectors on the other.

With respect to product certification there

tends to be some confusion between accrediting laboratories and accrediting certifiers.

Evaluation of an independent laboratories

technical capabilities may not include

evaluation of the administrative capabilities

necessary for certification. The general lack

of this latter service particularly with

respect to product safety certifiers has

impeded the market acceptance of alternate

sources of certification. Since some laboratory accreditation activities have been

informal to date, only casual attention has

been paid to formalizing the qualification

of inspectors for particular kinds of

laboratories. On the record the budding

national laboratory accreditation programs

appear to be making effort to document these

qualifications and harmonize them.

k . Analogy with

college and hospital accreditation

Institutional accreditation is not a new activity in the marketplace for services.

Although the need for accreditation of

testing laboratories is being established

in present time, other service organizations

have been subjected to such evaluation for

many years . The efficacy of accreditation

for colleges and hospitals is taken for

granted to such an extent that many consumers

of their services are almost unaware of the

confidence which they obtain through accreditation. Other forces in the marketplace for

health care and educational services are, however , well aware of the importance of

accreditation. The many departments in a hospital, including its laboratories, must

be evaluated according to criteria wellestablished in the medical community and be

inspected by peers, on a regular basis. The

insurance industry seeks this supporting

evidence for its confidence in hospitals. On

a periodic basis the academic departments of

a college or university are evaluated by their

peers. Confidence in their diplomas by major

employers of their graduates is sustained

through accreditation. The fact that there

may be some elements of professional discipline in these fields is known. Laboratory

accreditors should be aware of this risk also.

The fact that accreditation of hospitals

and colleges is only barely perceived by the

consuming public, except when a withdrawal

makes local news, and that nearly all institutions maintain accreditation once it is

obtained makes some managers of commercial

testing laboratories believe that the

accreditation activity has a levelling effect

thus detracting from their innovation. This

belief should not become a value assumption

because the effect is still manageable at this point in time while policies and criteria

are still being formulated.

5. Typical markets where laboratory

accreditation is necessary or useful

Laboratory accreditation has been established in certain disciplines for a number

of years. Several of these programs will be

discussed in detail during the conference,

but I wish to establish the rather high

degree of user satisfaction with these programs in the markets which they now serve. The shortcomings of any particular program as viewed by laboratory managers will not be

reviewed in the presentation.

Accreditation of a construction

materials testing (CMT) laboratory by the

Cement and Concrete Reference Laboratory

(CCRL) Program is now often specified in

construction contracts. This recognition

has come about in part through the recommendation by those laboratories which have

passed this muster. CCRL accreditation

represents a kind of industry-wide quality assurance which establishes confidence among

both laboratories and their customers.
The College of American Patholigists

(CAP) Accreditation Program is basicallyoriented toward clinical service laboratories.

Some time after its establishment, the

j program was expanded to admit the development

laboratories of medical device manufacturers

.

In the experience of one particular firm,

I this accreditation enabled it to participate

in collaborative reference studies to conI firm performance of a new product. The other

CAP-accredited laboratories would have been

j less interested in participating in that

exercise had the manufacturer not also been

J CAP-accredited

.

In the discipline of non-destructive

testing (NDT), full-scale laboratory accredii tation is not yet feasible; however, certification of NDT technicians has been the basis

for reliability on the technology for several

j years. Laboratories offering NDT services

are expected to maintain staff certification,

! and testing contracts so specify. This

particular approach has provided mobility

j among technicians at the working level during

j a period of rapid growth in the discipline.

Some form of technician certification ought

to be coupled with laboratory accreditation

so that personnel changes can occur with

minimum effect on the organization's

accreditation.

While most accreditation programs are national in coverage, some activities are

' conducted on a regional or local basis

generally because of marketing needs

.

Examples include state and municipal level

recognition of laboratories to inspect and

test specialized electrical installations or pressure vessels not previously codecertified. These activities respond to the

needs of laboratories as well as the regulated

parties although not as formal as they should

be, regional accreditation programs through

their existence and user satisfaction demonstrate the high potential value of this vital

new service in the contemporary economy.

6. Future trends

The most significant trend yet to come

in the field of laboratory accreditations

is the mechanism for identifying and responding to the critical mass of interest for the

development of criteria for accreditation

of a given class of laboratories. One hopes

that with maturation of this activity, the

process of accreditation will eventually

become straight forward dependent on the

specific criteria applicable to particular

laboratories. This condition applies whether

the basis of accreditation is by product or by discipline or a combination.

The process of criteria development must

anticipate customer expectations for the

conduct of laboratory services, much the

same as this sort of reasoning is being

applied to the development of product

standards . Involvement of regulatory

authorities who are likely to recognize

accreditation in their jurisdictions will

also become increasingly important as more

aspects of product performance and safety

become subject to demonstrated scientific

testing.

Accrediting organizations must not be

aloof from the impact of their activities,

I believe they have a responsibility to

assist in promoting their activities through

public awareness of the criteria and wide

dissemination of their directories. They

must also grow out of the fear of advertising

by accreditees. It is unnecessary, in my

opinion, to believe that a person contracting

for laboratory services is going to be duped

into thinking that an accreditation will

cover up otherwise incompetent conduct. In this regard laboratory accreditors can learn

a great deal from the experience of college

and hospital accreditors. From the user's

standpoint, international cooperation is a vital counterpart to a domestic program.

Without it, much of the progress in international trade accomplished through the multilateral trade negotiations will be set back.

In conclusion, laboratory accreditation

can serve as a stimulus to the market for

testing services for a long time before it

dampens the field through over-regulation.

5
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

LABORATORY ACCREDITATION — STATE-OF-THE-ART IN 1979

JOHN W. LOCKE

OFFICE OF PRODUCT STANDARDS

U.S. DEPARTMENT OF COMMERCE

WASHINGTON, D.C. 20230

Laboratory accreditation systems which formally determine and recognize

that a laboratory has the capability to carry out specific tests or types of

tests are increasing both in number and in the number of laboratories examined

and accredited. The need for such systems can be traced to the growing need

for laboratory testing in general. These systems are being developed normally

to facilitate both national and international trade. Fifty-six laboratory

accreditation systems were recently examined in a Department of Commerce

study. Only 2 of the systems existed in 1947. By 1970 the number had grown

to 33, and by 1978 the number was 56 with a significant portion of this

increase occuring in 1977 and 1978. Over 5,500 laboratories are formally

recognized by these systems and, since many of the systems are new, this num- ber should increase substantially in the 80's. There is also a growing interest in the international recognition of national accreditation systems.

Public and private sector coordination to promote acceptance of accreditation

criteria and consolidation of accreditation systems is a growing need.

Key words: Laboratory accreditation; state-of-the-art in 1979.

1. Introduction

In a way, this whole conference is on the state-of-the-art in laboratory

accreditation. General descriptions of

laboratory accreditation models will be

presented. Descriptions of evaluation

technology which form part of laboratory

accreditation models will be presented.

Specific accreditation systems will be

described. Future concepts will be presented. And finally, the program will

conclude with a briefing on an upcoming

conference focusing on international co- ordination of other accreditation systems.

The state-of-the-art I intend to present will deal with the laboratory

accreditation systems in existence in the

United States at this time.

2. Definitions

Before describing the extent of laboratory accreditation in the United States,

I would like to start with a definition.

Laboratory accreditation is the formal

determination and recognition that a testing laboratory has the capability to carry out specific tests or types of tests. Under this definition laboratory

accreditation does not include the

development or promulgation of test methods or standards. Laboratory accreditation under this definition also does not include the act of certifying that a product meets a standard since such a cer- tification requires attention to many more

aspects in the production process than

simply to the testing function.

6
3. Need for Laboratory Accreditation

Laboratory accreditation systems have

been developed to improve the ability of

testing organizations to meet a wide

variety of needs. First, there is the

need of government to assure compliance to health and safety standards. Then there

is the need of procurement authorities to determine that a product received meets

the requirements specified in the purchase

request. Producers and manufacturers

often must establish the specific characteristics and performance of their

products as measured by some test method.

Health service agencies need accurate

testing information to diagnose potential

problems and take corrective actions.

Consumers are more and more interested in

specific performance characteristics and

ways to measure differences among products. General contracting groups such as

in the building trades must meet certain

specifications which require definitive

test information. Finally, many certifying agencies have been established to attest to the accurate performance of

products and testing is an integral part

of that determination.

The main objective of laboratory

accreditation systems is to facilitate

trade both nationally and internationally. Over and above this objective, however, laboratory accreditation is used to meet other objectives, including ensuring

validity of test data, promoting wide

acceptance of test data, providing more

efficient use of testing facilities,

giving credibility to more use of testing

laboratories, giving additional status to competent laboratories, promoting good

testing practices, improving test methods,

and providing information about accredited

laboratories. Such programs inherently

will upgrade the quality of measurement in

testing laboratories. They will emphasize

the need for calibration services, provide

models for good laboratory management performance, and aid in the improvement of

quality control in manufacturing. In addition, laboratory accreditation will be

an aid to the reduction of defective

products and a reduction in personal

injury, property damage, and product

liability suits.

4. Approaches to Laboratory

Accreditation Systems

There are two basic approaches to accreditation of laboratories. First is

7

the approach where laboratories are accredited to test specific products in

conformance to standards using test

methods pertinent to that product. Such

product-focused systems are designed for

the users of laboratories, particularly

for those desiring a certification that

products meet certain standards. These

systems typically identify the testing

methods to be included in the program

before accepting applications for accreditation. The programs provide accreditation for only a limited number of test

methods although they effectively serve

the needs of laboratories for recognition

to a particular set of clients.

Second is the approach where laboratories are accredited to conduct tests in

broad technical areas or groups. This

discipline-focused approach accepts a laboratory's statement of tests it can perform and then develops procedures to verify the laboratory's capability to perform the tests. The general area of

accreditation is announced, but the specific tests to be verified will depend upon

the laboratory's testing programs and the

accreditor's ability to examine the skills

of the laboratory to perform those testing

programs. Such programs are advantageous

to the laboratories since accreditation

can be granted for a large spectrum of test methods but are often difficult for users of laboratories to use since the

accredited tests may often not be in the

language of the product performance characteristics.

5. Operation of Laboratory

Accreditation Systems

Most laboratory accreditation systems

contain the following features:

A. They obtain objective information

about the testing laboratory's

capabilities either through a questionnaire or an application

form.

B. Each laboratory is visited and its

operations reviewed by an evaluation team.

C. Proficiency tests are established

for certain tests and the laboratory required to participate so that performance of the various

laboratories can be compared.
D. After evaluation of the results of

the first three features, a formal

recognition of the laboratory's

capability is granted.

E. The laboratory's performance is reassessed periodically to assure continued compliance with the

criteri a.

F. The accreditation system makes

known to potential users the capabilities of potential laboratories in its system.

6. Laboratory Accreditation Programs

in the U.S.

Mr. Charles Hyer, in a study performed

for the Department of Commerce in 1978,

examined 56 public laboratory accreditation programs in existence in the U.S. These programs were equally divided among

Federal agencies, state and local

agencies, and trade associations. The

rapid increase in the establishment of

these programs is illustrated in

Exhibit 1. Apparently laboratory accreditation programs fill a need because the

programs have been established recently at

an increasing rate. The number of laboratories formally recognized by these

systems is growing at an even faster rate

as shown in Exhibit 2. In this Exhibit

the number of laboratories granted

accreditation in 1978 was plotted versus

the year the accreditation program began.

Typically, these programs require several

years before significant numbers of laboratories become accredited. With the large

number of programs initiated in the

mid-70' s, it would appear that the number

of laboratories accredited will be increasing very rapidly in the 80' s.

With the increasing number of accreditation programs, we are beginning to observe a situation where laboratories are accredited by more than one system. This

overlap and duplication in the examination

of laboratories is just beginning to be

felt. For example, one laboratory provides operating space for numerous different accreditation agents at its facility,

and this operating space is almost con- tinually utilized. In other cases, we know of laboratories being accredited by

30 or 40 systems. It would appear that

this overlap and duplication is costly and

burdensome and will grow substantially in

the coming years, and it is one reason why

some sort of a national coordination of

laboratory accreditation systems appears

to be desirable.

7. Toward "Nationally Recognized"

Laboratories

Another factor leading to the promulgation of laboratory accreditation systems

is the enactment of laws and statutes

which specify that testing must be done by

"nationally recognized" laboratories.

Private groups may very well develop

systems which can be recognized nationally

by Federal agencies, or Federal agencies

and private groups may develop systems

which warrant national recognition. The

demand for national recognition, however,

will tend to increase the pressure for the

development of broad-based laboratory

accreditation systems capable of multiple

product testing and evaluation, or at a minimum, effective coordination of such

systems in the interest of economy.

Each laboratory accreditation system

has its own specific criteria which it will use to evaluate the testing laboratories. We have examined many of these

criteria; and although it can be said that

in general the criteria specify similar

requirements, the laboratories find that

fulfilling the requirements of these

criteria calls for individual effort for

each specific criterion. With the in- crease in laboratory accreditation systems

there will be a similar increase in the

work that laboratories must perform in

order to be accredited by these various

systems. Accordingly, some attempt needs

to be made to arrive at universally acceptable criteria for application in potentially duplicative, and perhaps conflicting, programs. We believe that this is an appropriate subject for the next national

conference on laboratory accreditation.

8. International Implications

International laboratory accreditation

conferences have been held in 1977 and

1978. A third one is scheduled for

October of this year and a fourth one is planned for the fall of 1980. The purpose

of these International Laboratory Accreditation Conferences (ILAC) is to examine

ways in which laboratory accreditation

systems in the various participating coun- tries can be multilaterally recognized.

If such recognition can be established,

then products tested in one country may be

8
accepted in another country without re- testing. The formal international recognition of accreditation programs will, of course, depend on treaties or other

arrangements made among the participating

countries. In the interim, ILAC is attempting to prepare a Directory of

Laboratory Accreditation Systems throughout the world. In an attempt to assure

some sort of minimum acceptable characteristics of these accreditation systems,

ILAC will be specifying minimum criteria

which are to be met by a laboratory

accreditation system in order for that

system to be listed in the Directory.

Some programs already in existence in the

United States may not be accepted for

listing in this Directory because they do

not meet the criteria being suggested.

More on this matter will be presented in

the last session of this conference.

Suffice it to say that international

coordination of laboratory accreditation

is of growing interest.

9. Summary

U.S. laboratory accreditation systems

are growing rapidly in number as are the

number of laboratories actually recognized. There are a number of forces in

existence both nationally and internationally which are beginning to focus on the need to coordinate these systems. The

key to such coordination will be the

development of a universally acceptable

! criteria for evaluating the testing

laboratories, a public and private sector

coordinating body to promote acceptance of

the criteria, and the consolidation of

accreditation systems.

9
EXHIBIT 1

NUMBER OF ACCREDITATION PROGRAMS IN THE U.S.

Ref.: C. W. Hyer, "Principal Aspects of

U.S. Laboratory Accreditation Programs," January 24, 1979

60

1945 50 55 60 65 70 75 80

Year Accreditation Program Began

10
National Bureau of Standards Special Publication 591, Proceedings of the on Testing Laboratory Performance: Evaluation and Accreditation, held at

MD, September 25-26, 1979. Issued August 1980

National Conference

NBS, Gaithersburg,

GENERIC STANDARD BASIS FOR LABORATORY ACCREDITATION

HARVEY E. SCHOCK, JR.

PRODUCT ASSURANCES CONSULTANT

P.O. DRAWER 130

HADDONFIELD, N.J. 08033

The paper provides an overview of generic criteria for laboratory

capability evaluation. The new American National Standard, ANSI/ASTM

E 5^8-T9 5 Standard Practice for Use in the Evaluation of Testing and

Inspection Agencies, is reviewed. It provides generic criteria for

evaluation of an agency for intended purpose, agency organization,

human resources, material resources, and quality systems. New work

is also reviewed relating to generic criteria development for accreditation systems and accreditors. Examples are provided of voluntary

standards for specific disciplines and product areas. Discussion is

provided of work in similar national and international areas, including

mandatory use. Additional opportunities are given of appropriate survey

and audit techniques for use in evaluation and accreditation.

Key words: Accreditation systems; certification; evaluation of inspection

organizations; evaluation of testing organizations; inspection standards;

laboratory accreditation; laboratory evaluation; regulatory liability;

test standards; voluntary consensus standards.

1. Introduction

Generic criteria are required to permit

development of a uniform base in preparing

specific working requirements to evaluate

testing and inspection agencies. These

generic criteria are provided in ANSI/ASTM

E 5

1+8-T9, Standard Practice for Use in the

Evaluation of Testing and Inspection Agencies

(Ref l). This standard serves as a starting

point to achieve effective and efficient

review of test and inspection organizations

and relates to criteria for the evaluators

and the evaluation system. In addition the

generic criteria are useful in developing a base for laboratory accreditation activity.

2. Needs

Business, legal, and standards practice

usually require final product or service to

be evaluated for performance against established requirements and compliance criteria.

11

However, in many areas it may be desirable

for additional information to be known about

an actual test or inspection process, through

evaluation of agencies or activities involved in earlier decision making processes.

In normal commercial practice this flow

of information and evaluation is considered

in the regular contract base, the need to

know, and with full regard for associated

proprietary information. However, with ex- panding national and international trade,

and involvement of third parties in commerce,

it is desirable for ground rules to be defined to aid in evaluations and possible ac- creditation, especially with indirect and

complex procurement arrangements. Also, it

may well be important for regulatory re- quirements to relate their specific criteria

to such defined normal commercial practice.

Such action may result in the least adverse

effect on commerce and productivity with

reduction in possible regulatory liability

(Ref 2).
Thus the need for the formation of ASTM

Technical Committee E-36, on Criteria for

the Evaluation of Testing and Inspection

Agencies, and with the development of ASTM

E 5^+8-79 Standard Practice Generic Criteria.

3. Considerations

With the disengagement in the 1960's of

the Department of Defense from on-site test

and inspection, there was a formal trend to improving visibility of quality and reliability information for products and services

from suppliers. An example of action taken

to improve this flow of data and accountability was the Electronic Industries Association

development of Standard EIA-1000, EIA Standard Procedure and Questionnaires for Electronic and Electrical Parts Suppliers

Quality and Reliability Assurance (Ref 3).

This provided a base for buyers to obtain

quick answers on questions before purchasing

electronic parts. The system was voluntary

and also included provision for sharing of

system reports.

Recognizing a need for furthering this

evaluation in many test and inspection areas, ASTM Technical Committee E-36 was formed to

prepare standard consensus generic criteria.

These criteria were to be used by others in

preparing specific requirements for qualification and accreditation of testing and inspection agencies. The Committee Scope included preparing generic standards on its own and preparing specific standards when

not in area of interest of other technical

committees and when coordination is needed

with them. Also the Committee Scope included

standard nomenclature and definitions, aiding and advising ASTM technical committees

in the preparation of standards, correlation

and consolidation of similar standards prepared by separate committees, and instituting cooperation between these technical

committees in areas of common interest.

Further, the Committee was to encourage,

secure, and promote cooperation between ASTM

and other organizations including those that

may accredit testing and inspection agencies.

And, the Committee was to advance, improve,

standardize, and unify such standards, including liaison with government and other

organizations

This presentation is not intended as a formal report on the work of ASTM E-36.

Rather it is intended for those not familiar

with the voluntary consensus system to aid

in the understanding of the responsive

nature of this work and its subsequent value

through application of specific criteria for

12

products and services in the voluntary syste

and also as appropriate in the mandatory reg

ulatory system. Therefore this is not a ful

report and those parties interested in more factual details should refer to the writer

and the Committee.

k. Test and Inspection Agencies

The generic criteria in Standard E 5^+8- 79 replaces the earlier edition of Standard

E 5^+8-77. For actual use this standard

should be interpreted by a technical standard for each specific product, service, or test method. This is obvious since no single standard could be directly applied with

full value in all possible applications and

conditions. For example certain industries

have special practices and requirements

which must be placed into perspective. Even

with such additional work there is much to

be gained through the consideration of outline format, applicable terms, definitions,

and possibly wording generated in the generic standard. Final decision for application

must rest with the technical committee re- sponsible for the final document.

Likewise care must be used if the stan

dard is applied in normal direct negotiations

and in dealings between buyers and sellers

where there may be special economic and proprietary concerns.

5. E 5^8-79 Criteria

Although only the actual Standard may

be used for full and proper understanding,

this paper calls attention to certain areas which are covered in detail in E 5^+8-79 •

First, the purpose for evaluation

should be identified and considered in developing any specific document from this

generic base. Obviously this purpose must

identify the system in which the specific

document will be used and criteria relating

to evaluator or accreditor. Without this

understanding it would be impossible to properly apply specific requirements in the

test and inspection environment.

Likewise the specific criteria must be

clear in relating to possible implications

in past, present, and future tense of the

application.

Second, the agency organization should

be considered for necessary identification,

affiliation, history, services, and past

recognition achievements. Here the intent

is to provide only that information which

must be considered necessary in specific

applications
Third, the human resources of an agency

should he presented to relate personnel, re- sponsibility, and authority with appropriate

job descriptions and identification of means

for maintaining personnel qualifications,

work experience, training history, and facts

on ensuring continued competence of personnel.

Fourth, the material resources of an agency should he identified relating to

I actual equipment and facilities, calibration

standards, and other information necessary

to ensure update and service.

Fifth, the quality system of an agency

should identify the procedural systems affecting the quality of services with details

on rationale of the metrology system and

support activities of calibration, traceabilj ity, data analysis, feedback, and necessary

survey and audit activities with periodic

i on-site reviews.

In summary, the standard provides a

I comprehensive format and approach, useful

internally as well as by prospective outI siders, where there is proper specific stan-

' dard consideration.

6. Evaluation System

Obviously the proper application of

> E 5^-8-79 requires a full understanding for

1 the associated evaluation system and if

applicable the full details on any additional accreditation system.

Although not covered in detail in the

I standard it is assumed there would be pro1 per consideration given in the system to

both prevent the generation of incorrect

I reports and to provide effective and efficient action should there be any questions

or need for control.

7. Evaluator Criteria

Likewise, the standard does not identify requirements for the evaluator and any

associated accreditors. Here again it is

expected such activities will be properly

qualified and under supervision to assure

effective and efficient operation, both for

the evaluator and the agency being evaluated.

8. Specific Criteria

The continuing effort of Technical

Committee E-36 has indeed been rewarding.

Through this voluntary consensus base there

have been developed various specific criteria

standards. For example, in the Manufactured

Housing area there are ASTM Standards E 5^2

and E 651. In the government voluntary area there is the Department of Commerce National

Voluntary Laboratory Accreditation Program

(NVLAP). And in addition there are several

possible mandatory government applications

in development, all based on E 5^+8-79. However, the subject of this paper is generic

criteria and further details must be the

subject of other presentations.

9- International Applications

The work of E-36 has received aid from

abroad, both in the preparation of the E 5^8-79 and in its application in other na- tional criteria standards and evaluation

systems. It is expected this cooperation

will continue. There is also some similar

effort in international standards bodies and

in ad hoc activities. Again, these are beyond the scope of this paper except to note

that this cooperation is expected to aid development of further generic and specific

criteria in areas of products, services, and

evaluation or accreditation systems.

10. Evaluation Effectiveness and

Efficiency

The development of E 5^8-79 has been

accomplished with full appreciation of demands for possible reviews, paperwork, and

delays in the effectiveness and efficiency

of normal productive operations. Recognition of the need for criteria to be cost

effective requires full consideration by

users of the standard and in development of

any associated specific criteria.

Special attention has been given to

setting up a structure that will be flexible

with time and changing conditions, especially with the possible mandatory adoption

of the criteria in regulatory processes.

It is strongly recommended that similar

consideration be given by others in generating specific criteria since such systems

may not provide for detailed understanding

and periodic reviews required under the

ASTM voluntary consensus system.

Likewise it is expected this document

will find good application in normal survey

and audit activities for quality assurance

programs and in other technical management

areas

.

Similar applications may also be found

in the growing requirements for current good

laboratory practice, current good manufacturing practice, and in reasonable testing

programs

.
The intent of this standard has been to provide another viable tool in the process

of applying standards in improving the productivity of the world. To this end, Technical Committee E-36 extends a cordial invitation for all interested to participate

in the ongoing work of the Committee and

in developing new projects which might fall

within the scope of the Committee.

References

(1) American Society for Testing and

Materials, 19l6 Race Street,

Philadelphia, PA 19103.

(2) Schock, H.E., Jr., Regulatory Cost

Management and Product Assurance,

Proceedings 1979 Annual Reliability

and Maintainability Symposium , 1582/79RM003, IEEE, New York, 1979-

(3) Electronic Industries Association,

2001 Eye Street NW, Washington, D.C.

20003.
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

LABORATORY PERFORMANCE EVALUATION

SERVICES OF THE U.S. NATIONAL BUREAU OF STANDARDS

J. 0. BRYSON , B. C. BELANGER, AND R. K. KIRBY

NATIONAL BUREAU OF STANDARDS

GAITHERSBURG, MARYLAND 20760

This paper describes the principal services provided by the U.S.

National Bureau of Standards that allow laboratories to assess and

maintain at a high level the quality of their measurements. The NBS

programs described in some detail in this paper include: Standard

Reference Materials, Calibration Services, Measurement Assurance

Program Services, and Collaborative Reference Programs. The paper

deals with the basic aims and objectives of these services. A brief

description of each service is provided, along with indications of how

these services have improved the performance of laboratories. Also

discussed are general concepts regarding the assurance of measurement

accuracy and compatibility among laboratories on a national and/or

international scale.

Keywords: Calibrations; Collaborative Reference Programs; laboratory

performance; measurement assurance; measurement evaluation technology;

measurement services, physical standards; proficiency testing; Standard

Reference Materials; testing; traceability

.

1. Introduction

Our society today relies on accurate

measurements for realizing equity in the

marketplace, for industrial quality control,

for equitable enforcement of and compliance

with government regulations related to the

environment, health and safety, and for

scientific research. In order that the un- certainty of measurements be sufficiently

small so that correct decisions can be made

based on measured data, measurement quality

assurance must be built into the measurement

system.

There is an increasing demand for the

accreditation of laboratories that are capable of performing reliable measurement ser- vices in all areas of the measurement field.

Accredited laboratories and the accrediting

authority must be particularly sensitive to

the need for measurement quality assurance. Calibration and standardization of laboratory equipment and procedures are essential

to obtain the reliable measurements necessary for modern technology and to assure

that, for fair trade and commerce, credible

testing laboratories can be identified and

called on to determine if the quality of the

product adequately reflects the specification requirements. Table 1 lists the principal factors that a laboratory should con- sider incorporating into an overall system

of quality assurance.

A key responsibility of the National

Bureau of Standards (NBS), as stated in the

act of Congress that establishes NBS and

subsequent amendments to that act, is to

establish and maintain national standards

of measurement, coordinate these standards

with those of other nations, and provide means and methods for making measurements

consistent with those standards. In other

words, provide a complete and consistent

national system of physical measurements.
"Agreed upon" standards are necessary

for making accurate measurements. As a familiar example, consider the prototype mass

standards Kh and K20 kept by NBS which are

the U.S. standards for the kilogram. By

mutual agreement of the measurement community, mass measurements in the U.S. are ultimately referred to these national standards

of mass.

One often hears of requirements to the

effect that "measurements or a reference

standard must be traceable to the National

Bureau of Standards". Presumably the intent

of such traceability citations is to insure

that measurements are sufficiently accurate

that proper decisions can be made. In a recent article, Belanger has described the con- fusion that can result when differing interpretations of traceability are adopted by

various parties^!]. If it is required that

the measurements made by an accredited laboratory be "traceable" to NBS, then it is

clear that the laboratory, the accrediting

party, and the customer must agree on the

definition of traceability and on the criteria as to what constitutes adequate traceability.

NBS prefers a definition of traceability

that focuses on the uncertainty of the measurements being performed.! It is clear that

NBS must provide convenient mechanisms by

which parties outside NBS can verify that

their measurements are accurate at some desired level of uncertainty relative to na- tional standards. Such mechanisms are needed whether or not one has accredited laboratories.

Laboratories that must make quality

measurements consistent with national standards have available a wide variety of NBS

measurement services from which to choose.

Table 2 shows the measurement transfer

1 In referenced! traceability is defined

as follows: "Traceability to designated

standards (national, international, or wellcharacterized reference standards based upon

fundamental constants of nature) is an attribute of some measurements. Measurements

have traceability to the designated standards

if an only if scientifically rigorous evidence is produced on a continuing basis to show that the measurement process is producing measurement results (data) for which the

total measurement uncertainty relative to national or other designated standards is

quantified.

16

mechanisms employed by NBS. This paper fo- cuses on those services of NBS that direct- " ly facilitate the making of accurate measurements by laboratories throughout the U.Ei

2. Calibration Services

ll

J e The calibration of working standards is] a service that has been offered to the public by NBS ever since it was formed. Today,

in a typical year, NBS provides a total of

several thousand calibrations for approximately 1000 different organizations - including industrial standards laboratories,

Department of Defense and other Federal

agency calibration activities, state and

local weights and measures activities, universities, hospitals, utility companies,

etc. NBS currently offers calibration sers vices for mass, volume, density, temperature

pressure, dc and low frequency electrical

measurements, rf and microwave measurements, i

radiation, and a variety of other measurements. The array of services is constantly

changing since each year NBS generally ini

tiates calibration services to reflect new

needs associated with emerging technologies.

Also, to make way for new services, NBS regularly phases out services that have become

less important to the measurement community

because of changing technology, services offered by the private sector, or declining

requirements

It is clear that NBS does not have the

personnel to calibrate all standards in the

U.S. of a given type that require calibra- ; tion nor would it be possible for us to of- 1 fer high level calibration services for all 1 conceivable measurement quantities for which;

accurate measurements are needed. Accordingly, NBS tries to provide a mix of ser- E vices that is responsive to the most important needs of the measurement community and

where possible to provide those services to high level laboratories that can in turn

provide calibration services and thereby

transfer accuracy to lower level field measurements. It is the policy of the Federal

government not to compete with private sec-

' tor services. Accordingly, NBS' services r are limited to those special high level cali

brations which are not readily available

elsewhere and high accuracy calibrations re-; quired for direct traceability to national

standards

NBS publishes a catalog of calibration

services, NBS Special Publication 250,

"Calibration and Related Measurement Services of the National Bureau of Standards".

An appendix to SP 250 is issued each June

and December giving up-to-date prices and
information on new services that are about

to be offered or services that are slated

for phase-out. Both of these documents are available from the Office of Measurement

Services at NBS.

Laboratories, whether formally accredited or not, wishing to maintain a high level of quality control over their measurements

can effectively utilize NBS calibration services, and many do so. In recent years, NBS

and its calibration customers have come to

recognize that the use of standards calibrated by NBS is by itself no guarantee that

the laboratory is performing adequate measurements. Even if a particular lab has a

\ standard accurately calibrated by NBS, it

probably will not be able to make accurate

measurements unless the operators are

skilled, the environment is adequately con- trolled and the measurement procedures are technically sound. Recognizing this fact,

laboratory accreditation procedures generally address all of these factors.

In the section which follows, special

NBS services that allow our customers to

< control their total measurement process are described [2].

3. Measurement Assurance Programs

NBS has developed special services

called Measurement Assurance Program (MAP)

Services to aid laboratories in establishing

their measurement uncertainty and maintaining it at an acceptable level. NBS currently provides these services for measurements

of mass, temperature (platinum resistance

thermometers), dc voltage, dc voltage ratio,

capacitance, resistance, electric energy

(watt-hour meters), laser power and energy,

and, on a trial basis, gage blocks.

The techniques used in MAP's are familiar techniques to those experienced in the

j quality assurance field. Unlike a calibraj tion which only checks the standard or instrument, the MAP is designed to assess the

performance of the entire measurement proi cess. A key element of MAP is the availability of a suitable transport standard.

The MAP transport standard is a device, ar- tifact, or material^ that is stable, rugged,

and well-characterized, and whose value is

accurately known relative to national standards at NBS. This standard is sent to the

Standard Reference Materials can be used

as the basis for a MAP when coupled with

appropriate methods and data analysis.

participating laboratories where it is measured as an unknown or "blind" sample. Since

NBS measures the item before it leaves the

Bureau and after it returns, its value while

in the participant's laboratory should be

accurately known. Any bias or offset (systematic error) associated with the measurements made by the MAP participant can be as- certained from the data returned to NBS.

Since an objective of the MAP is to establish the total uncertainty of the measurement process in the participating laboratory, it is also necessary for the laboratory to have suitable working standards

upon which measurements can be made at appropriate intervals to establish the process precision and insure that it remains

within acceptable limits. NBS provides advice on applicable quality control techniques such as the keeping of control charts

to monitor the random error (precision) of

the process and make sure that the process

remains in a state of statistical control.

Knowing both the systematic error or bias

by means of data taken on the NBS transport

standard and the random error established

by the measurements made in the laboratory

over a suitable period of time, it is possible for NBS to issue a test report that

quantifies the total measurement uncertainty

of the participant. NBS provides consulting

help to new MAP participants in identifying

problems and generally insuring that their

accuracy objectives are met.

NBS disseminates measurement assurance

services in two ways. Most users deal with

NBS on a one-on-one basis, but NBS and our customers are increasingly interested in

using a group or regional approach. This

second approach^] has been particularly suc- cessful in the case of dc voltage, and so NBS is looking for opportunities to utilize

the same approach for other quantities. The

regional or group MAP involves a group of

laboratories in one part of the country

(e.g., Southern California) interacting with

NBS as a group through one laboratory designated as the "pivot" laboratory. There appear to be a number of advantages both to NBS and to our customers from the use of the more efficient group or regional approach,

including lower costs to the participants,

faster turnaround time, and in the long run, less frequent need for interacting with NBS

to verify measurement accuracy.

The net result of a MAP is that a rigorous uncertainty statement can be developed for the performance of the measurement

system. Additionally, it provides a mechanism for noting changes with time or out17
of-control conditions. Should some organization, either private or governmental, wish

to institute a laboratory accrediation program for calibration laboratories, data from

a MAP (assuming an NBS service is available

for the measurement quantity of interest)

can provide an excellent tool for assessing

laboratory performance. Note that the MAP

approach is a "systems" type approach and

that it has much in common with the philosophy of the Standard Reference Materials

Program as described in the section which

follows

h. Standard Reference Materials

Standard Reference Materials (SRM's)

have been produced, certified and issued by

NBS since 1950 W . SRM's are well-characterized, homogeneous, stable materials (or

simple artifacts) with specific properties

measured and certified by NBS. Over 1,000

different SRM's are now available from NBS

[5]. The SRM inventory can be categorized

into three basic types of certified reference materials. These categories are: l)

materials certified for chemical composition

or purity, 2) materials certified for one

or more fundamental physical or chemical

properties such as pH or melting point and

3) materials or artifacts certified for other

properties usually associated with special

engineering type applications, e.g., sieve

sizing or color fading. Many of the SRM's

in this latter category are certified on a method-dependent rather than absolute basis.

SRM's are widely used throughout the world

in a variety of measurement applications including development and evaluation of measurement methods, assurance of long-term

measurement compatibility among different

laboratories and establishment of measurement traceability to NBS.

During fiscal year 1978, NBS distributed 37,000 SRM units to over 10,000 users throughout the world. Foreign laboratories

represent over 20 percent of SRM users. The

four major SRM user categories are: l) industrial research and quality control laboratories [61, 2) environmental analysis and

monitoring laboratories C 7] , 3) laboratories

performing clinical or health related measurementsL 8,9,10] an(} 1^) laboratories en- gaged in basic scientific and/or metrological research (e.g., establishment of an ac- curate temperature scale) [H 5 12] ,

One of the most significant outputs of

the NBS SRM program during the last 10 years,

has been the development of a greater understanding of the role of SRM's in total measurement systems. The NBS Office of Standard

Reference Materials (0SRM) has placed an ii creased emphasis on utilizing a "systems

approach" to accurate measurement, whereby

NBS SRM's together with standardized wellcharacterized test methods (reference meth

ods ) are used to transfer accuracy and es- tablish measurement traceability throughout

large multi-laboratory measurement networks

As indicated earlier, the systems approach

in the SRM Program shares many common fea

tures with MAP activities. A large number

of articles and monographs have been published by J. P. Call [13-15] and other members of the OSRM staff [l6-l8,6,7,8] con- cerning the systems approach to accurate

and compatible measurements.

The systems approach to measurement

compatibility recognizes the fact that SRM': provide a highly useful and important, but

by no means sufficient mechanism for achie^

ing measurement compatibility on a nationa]

or international scale. Measurement compal

ibility means that all measurement stations

within a given measurement network obtain

values within agreed-upon measurement un- certainties, when measuring a specific pro]

erty of the same material. The systems ap- proach to measurement compatibility is

illustrated in Figure 1 and described ex- tensively in the review paper by Uriano anc Gravatt[l6]

Figure 1 represents a hierarchical sys

tern of analytical methods and reference ma< terials each coupled to the other in the manner shown. The function of each component ( I to VI) is to transfer accuracy to

the level immediately below it and to help jj provide traceability to the level immediately above it, thus helping to assure mea

surement compatibility in the overall systtf

As we proceed from bottom to the top

the measurement hierarchy, accuracy require

ments increase at the expense of decreased

measurement efficiency. At the top of the

hierarchy are the so-called definitive metl

ods of analysis or test, which give the moE accurate values obtainable. 3 Unfortunately

most definitive methods (e.g., gravimetric

techniques for preparing analyzed gas SRM'e

[l6]) are usually time consuming, sophisticated methods and thus are not economically

acceptable for widespread and routine field

use. Definitive methods, however, are usee?

3 For definitions and more detailed discus>

sion of definitive, reference and field

methods, see ref

1. ren

[l6].

18
whenever possible to certify NBS SRM's.

SRM's then can he used to transfer accuracy

further down the measurement hierarchy, e.g.,

to verify the accuracy of or calibrate ref- erence methods.

i Reference methods, which may be automated comparative methods (e.g., nondisper-

.jsive infrared spectrometry for measuring

carbon monoxide in air) can then be used directly in the field or alternatively as a

•basis for developing or evaluating other

methods. Reference methods are also com- monly used for producing secondary reference

materials, which in turn are directly used

in routine field measurement applications.

Examples of reference methods are the Environmental Protection Agency reference methods for monitoring the ambient air criteria

•pollutants [20]. The Annual Book of ASTM

Standards lists hundreds of standard test

•.methods, which can be designated as refer- ence (or in some cases definitive) methods.

For example, the ASTM E-350 series of Standard Methods for chemical analysis of a

: variety of iron and steel products [21 ] con-

. stitutes an excellent example of reference

methods [l6]. In fact, the systems approach

to compatible measurement historically

:;ioriginated in the metals industries [ ^5 13]

,

where NBS SRM's and ASTM standard methods

both play a critical role in assuring mea- surement compatibility.

The accurance of numerous field methods can in principle be traced back to a definitive method in such a hierarchical accuracybased measurement system. In such a system,

SRM's and other reference materials play a

•.necessary, role in the transfer of accuracy.

However, SRM's, by themselves, cannot assure

[ high quality measurements. As in the case

, of MAP's, good methods, good laboratory

-practices, well-qualified personnel as well

as proper intra-laboratory (internal) and

inter-laboratory (external) quality assur- ance procedures are just as important as the

reference materials.

Variations of the idealized accuracybased measurement system are now in place in

• many industrial areas and are being implemented in the areas of clinical and environmental analysis [ 7 5 10 ] . Many other areas are

tjalso beginning to utilize this approach. An excellent illustrative example of the systems

approach to measurement compatibility, which

^involves both SRM's and ASTM standard meth-

. ods, is represented by the recent Department

of Commerce National Voluntary Laboratory

Accreditation Program (NVLAP) for testing

the properties of thermal insulation materials^]. This is the first NVLAP program

being implemented by the Secretary of Commerce. A number of NBS SRM's and ASTM

Standard Methods are cited in the criteria

[22] for accrediting laboratories, covering

such properties as particle size and thermal

resistance

.

Figure 2 schematically illustrates the

thermal resistance measurement system. NBS

is in the process of developing a series of

high and low density fiberglass SRM's^ cer- tified for thermal resistance (R-values) as a function of temperature and density. The

NBS SRM's are certified by a definitive method (The NBS Guarded Hot Plate Method). Having been certified, the SRM's can then be

used to control the accuracy of reference

methods such as the ASTM Guarded Hot Plate

Method (ASTM C1TT) or for direct calibration

of comparative reference methods such as the

Heat Flow Meter Method (ASTM C5l8). Thus

the NBS SRM's and ASTM methods are intended

for direct, continuous use in intra-laboratory quality assurance programs.

NBS is also developing a set of thermal

insulation test specimens for use in proficiency testing, i.e., an external or interlaboratoty quality assurance program. This

will provide a mechanism for directly monitoring (auditing) the performance of participating laboratories on a periodic basis.

The audit program provides the measurement

system with an appropriate feedback and

traceability mechanism.

Table 3 contains a summary of the most

common uses of SRM's in measurement applications. The reader is referred to various

papers cited previously [H ,13-17] for other

specific examples of the use of SRM's. The

review paper [l6] cited previously also dis- cusses the relative roles of various organizational components (e.g., voluntary standards organizations, national standards

laboratories, regulatory agencies) in assuring measurement compatibility.

5. Collaborative Reference Programs

NBS has been involved for many years

in programs for proficiency assessment of

laboratories in cooperation with private

industry and standards organizations. These

programs were initiated to assist laboratories in determining the accuracy level and

k The first of these SRM's, (SRM 1^50 - High

Density Fiberglass) is now available from

NBS 1 5 ]

.

19
precision of their testing and in improving

their testing performance. The programs are sponsored by a variety of organizations and

are conducted under a variety of procedures.

Each field of testing has its special problems and each sponsoring organization its own goals. Hence, no two of the existing

programs are identical, but each has been

designed to meet the specific needs.

In 1936, the first collaborative reference sample program was initiated for cement

as part of the NBS Research Associate Program

of the Cement Reference Laboratory of the

American Society for Testing and Materials

(ASTM) Committee C-l. In 1966, this program

was revised to essentially its present form

wherein two pairs of samples for physical

tests and two pairs of samples for chemical

analyses are distributed each year. Similar

programs for bituminous, soils, aggregates,

and portland cement concrete and bituminous

concrete were then established in rapid suc- cession, the last in 197^ • These programs

are under the sponsorships of ASTM and the

American Association of State Highway and

Transportation Officials, respectively.

In 1969 a bimonthly program for paper

and board testing was initiated under the

sponsorship of the Technical Association of

the Pulp and Paper Industry and a program

for control of the quality of shipping con- tainer components was established for the

Fourdrinier Kraftboard Institute. This later

program involves weekly testing and monthly

reports by the participants. A quarterly

program for the rubber industry was started

in 1970 with the help of ASTM Committee

D-ll, and a color and appearance program,

of interest to many industries and users, was established for the Manufacturers Council on Color and Appearance. This quarterly

program presently includes tests for gloss,

color and color difference, and retroreflectivity

The Collaborative Reference Program

(CRP) provides a participating laboratory

with a means for checking periodically the

level and uniformity of its testing perfor- mance in comparison with that of other laboratories. A major feature of all CRPs is

the periodic distribution to the participating laboratories of suitably selected and

prepared samples. The participants are instructed to test the samples in accordance

with standard test methods and supplemental

instruction provided by NBS. They are also

requested to report any deviations from the

standard procedure, as for example, when the

conditions in the test room are not strictly

in compliance with the requirements of the

standard. Special data forms are provided

to the laboratories for reporting results 1 to NBS.

The test results are analyzed at NBS

accordance with the Youden two-sample procedureL'23] using special computer programs

The results of the analysis are presented

using tables and Youden' s graphical technique, as shown in Figure 3. In this diagram, each point represents a laboratory

and the ellipse represents the bounds with

which 95 percent of the points for similar

laboratories are located.

j 6. Technical Support for the Nationa:

Voluntary Laboratory Accreditation

Program

'

j In February 1976, the Department of

Commerce (DoC) established the National

Voluntary Laboratory Accreditation Program

(NVLAP). The purpose of this program is t give recognition, through accreditation, t laboratories that have the capabilities to correctly perform specific tests according

to identified standard methods. There are three (3) different procedures for institu

ing laboratory accreditation programs unde

NCLAP. Procedure (A) is for the general

public and it requires that four distinct

phases be accomplished, i.e., finding of

need, establishment of a criteria committe

development of criteria, and the examinati

for accreditation. Procedures (B) and (C) are for the use of Federal agencies and pr

vate consensus organizations (consensus by

DoC definition5) respectively. Both (B) a

(C) permit by-passing the requirements for

"finding of need" and the establishment of

a criteria committee. NBS provides techni

cal support to this program by assisting ij the development of both general and specif!

criteria and by evaluating the capabilitie

of laboratories that request accreditation

[2U,25]. The examination methodology cove three phases: l) Questionnaire, 2) On- sit

inspection; and 3) Proficiency testing.

For proficiency testing the "true" or:- target test result for any particular test

is obtained by one of the following ways : I

(l) Manufacturer . For some propertie

of a sample, it is possible to determ

1

5 Section 7c. 3(b) of Federal Register,

Volume hk, Number 8l, April 25, 1979, Page1 ' 21+27^-2^282 (15 CFR Part 7c) lists six err

teria that "private sector organizations" > must meet in order to be in this category.

20
what the test result should be from information on how the sample was made.

However, each case has to be thoroughly

examined before manufacturing informa-

. tion can be used as the basis for determining the target or "true" values.

This approach is often useful in proficiency testing programs requiring qual-

. itative responses or identifications

only (e.g., is starch present or not).

f (2) Reference Laboratory . Sometimes a single laboratory, such as the National

Bureau of Standards, has sufficiently

: high competence and national recogni-

: tion that it can be used to provide the

target or "true" test result. This is

particularly useful when the laboratory

has the capability of and has agreed to carefully verify the correctness of

i every important dimension of its appa-

! ratus and every step in its application

j of the standard test method.

j(

i (3) Group of Reference Laboratories .

= When no single laboratory can be given

a national recognition as having sufficiJ ently high competence to set the national standard, it sometimes is possible

for a proficiency test coordinator to

use the results from a number of repI utable laboratories. This would be aci complished by pooling their results

H (after a suitable statistical check on

« the agreement among the results) in

[ order to establish the target or "true"

! test result.

: (1+) Reference Method . Under NVLAP pro-

| cedures, the standard test method will

a usually also be the reference method.

r

:i However, in some cases the standard test

a method may be so broadly written as to

y permit a wide variety of test equipment

9 and testing protocols. If in such a

[ case a particular protocol and equipment

J combination is recognized as a reference

' method (or can be shown through error

analysis to yield results well within

i the required precision and accuracy),

then the results obtained with that

method by one or more "reference" labi oratories are used to establish the

target test result.

(5) Participants . If there is a sufficient number of participating testing laboratories and an insufficient

'f, number of reference laboratories, then

r the test results of the participating

laboratories are sometimes pooled by a

I proficiency test coordinator to establish the target result for the individual participants. It is important j

that the pooled test results include

only test data from laboratories known

(on the basis of all available information) to be following the standard

test method. This is determined not

from the test data, but from an inspection report and from information submitted originally and with the test data.

( 6 ) Previous Proficiency Test and

Inter-laboratory Data . Sometimes the

same samples are used as were used in a previous proficiency test. If so, the new target test result is based on a weighted pooling of current and previous test results.

7 . Summary

The recognition of the need for a con- sistent measurement system led to the formation of the National Bureau of Standards and

in particular NBS 1 responsibility for "The

custody, maintenance, and development of the

national standards of measurement, and the

provision of means and methods for making

measurements consistent with those standards." The laboratory performance evaluation services of NBS benefit all segments of

our society through the promotion of quality

measurements relative to national standards

whether the concern is for fair trade in the

marketplace or determining the pull of gravity in a scientific investigation.

[l] B.C. Belanger, "Traceability - An

Evolving Concept," to be published in

ASTM Standardization News .

[2] J.M. Cameron, "Measurement Assurance,"

NBS Internal Report Number 77-12^+0,

April 1977.

[3] J.M. Belanger and L.J. Kieffer,

"Regional Measurement Assurance Programs for Physical Measurements,"

Proceedings of the 8th IMEKO Congress,

Moscow, U.S.S.R., May 1979.

[k] Cali, J. P., "The NBS Standard Reference Materials Program: An Update,

" Analytical Chemistry , U8 , 802A,

1976.

[5] National Bureau of Standards, "Catalog

of NBS Standard Reference Materials,"

NBS Spec. Publ. 260 , 1979 Edition,

National Bureau of Standards,

Wasnington, D.C. (In Press).
[6] Uriano, G.A. , "Standard Referenct Materials and Quality Control," Transactions of 25th Western Regional

Conference, American Society for

Quality Control , 1978, available from

American Society for Quality Control,

Milwaukee, Wisconsin, 53203.

[7] Uriano, G.A. , "The Use of SRM' s for

Quality Assurance of Environmental

Measurements," Proceedings of the

National Conference on Quality Assurance of Environmental Measurements , available from Information Transfer,

Inc., Il60 Rockville Pike, Rockville,

MD 20852.

[8] Mears, T.W. and Young, D.S., "Measurement and Standard Reference Materials

in Clinical Chemistry," The American

Journal of Clinical Pathology , 50 , 1+11, 1968.

[9] Cali, J. P., "Reference Materials in

Clinical Chemistry," Proceedings of

The International Confederation for

Clinical Chemistry , 3*+ , 2123, 1975.

[10] Boutwell, J.H., Ed., " A National Understanding for the Development of Reference Materials and Methods for Clinical

Chemistry , " published by the American

Association for Clinical Chemistry,

Washington, D.C., 1978.

[11] Kirby, R.K., "Standard Reference Materials for Thermophysical Properties,"

in Proceedings of the Seventh Symposium on Thermophysical Properties , American Society of Mechanical Engineers, New York, 1977-

[12] Schooley, J.F., Soulen Jr., R.J., and

Evans Jr. , G.A. , "Standard Reference

Materials: Preparation and Use of

Superconductive Fixed Point Devices,

SRM 767," NBS Spec. Publ. 260-1+1+ , National Bureau of Standards, Washington, D.C. , 1975.

[13] Cali, J. P., et al., "The Role of Standard Reference Materials in Measurement Systems," NBS Monograph lU8, National Bureau of Standards, Washington, D.C. , 1975.

[ll+] Cali, J. P. and Reed, W.P. , "The Role

of NBS SRM's in Accurate Trace Analysis," NBS Spec. Publ. 1+22, Proc . Symp. on Accuracy in Trace Analysis , U.S.

Govt. Printing Office, Washington,

D.C, 1976.

[15] Cali, J. P. and Stanley, C.A., "Measurement Compatibility and Standard

Reference Materials," Annual Review

of Materials Science, 5 , 329, 1975.

[16] Uriano, G.A. and Gravatt, C.C., "The

Role of Reference Materials and Reference Methods in Chemical Analysis,"

CRC Critical Reviews in Analytical

Chemistry, 6, 362, 1977.

[17] Uriano, G.A. , and Cali, J.P. , "Role

of Reference Materials and Reference

Methods in the Measurement Process,"

Chapter 1+, ACS Symposium Series 63,

Validation of the Measurement Process , J.R. DeVoe, Ed., Am. Chem. Soc .

,

Washington, D.C, 1977-

[18] Cali, J.P., LaFleur, P.D. , Reeder,

D.J., Schaffer, R., Uriano, G.A. and

Velapoldi, R.A. , "National Bureau of

Standards Position Paper," (See reference [7]).

[19] Hughes, E.E., "Development of Standard

Reference Materials for Air Quality

Measurement," ISA Reprint ih-JOk, In- strument Society of America, Pittsburgh, PA, 197 h.

[20] Clements, J.B., "Qualification of Ambient Methods as Reference Methods."

Spec. Tech. Publ . 598 , pp. 69-79,

American Society for Testing and Materials, Philadelphia, PA, 197*+.

[21] 1978 Annual Book of ASTM Standards,

Part 12, American Society for Testing

and Materials, Philadelphia, PA 1978.

[22] "Proposed Criteria for Accrediting

Testing Laboratories that Test Thermal

Insulating Materials," Federal Register , 1+3, 1+5290 1978.

[23] Mandel, J., and Lashof, T.W. , "Interpretation and Generalization of

Youden's Two-Sample Diagram," J. Quality Technology 6(l), 22-36,

January 1971+

[2l+] National Voluntary Laboratory Accreditation Program, First Annual Report

(for the period February 25, 1976, to

December 31, 1977), U.S. Department of

Commerce.

[25] National Voluntary Laboratory Accreditation Program, Second Annual Report

(for the period January 1, 1978, to December 31, 1978). U.S. Department

of Commerce.

22
SESSION II

EVALUATION

TECHNOLOGY

CHAIRMAN: DENNIS H. GALLAGHER

LEEDS AND NORTHROP

National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

THE MEASURING PROCESS AND LABORATORY EVALUATION

THEODORE W. LASHOF

RESEARCH ASSOCIATE

NATIONAL BUREAU OF STANDARDS

WASHINGTON, D. C. 20234

Graphical review of the theory of the measuring process. In view

of the theory, what should be examined in the evaluation of laboratories? What might questionnaires and inspection reveal? How good

are various types of proficiency tests?

Key words: Collaborative reference program, test method; interlaboratory

testing; laboratory evaluation; linear model; measuring process; proficiency testing; Youden two-sample analysis.

1. What is a Measuring Process?

A measuring process involves a relationship between the quantity of some property of a material, product or system

and an indicating device sensitive to that

property. A simple familiar example is the

thermometer which indicates the temperature

of the surrounding material by the expansion

of mercury in glass. Another example is the

voltmeter whereby the voltage is indicated

by the angular deflection of the needle.

Ideally there should be a stable

linear relationship between the quantity

being measured and the response of the

indicating device. However, even in such a simple example as the mercury-in-glass

thermometer the relationship is not exactly

linear nor is it stable especially before

1 the glass has been adequately "aged." Nonlinearity, however, is not a serious problem

if the device can be calibrated simply or a simple theory relates measurement and quantity. Examples of non-linear relationships

are ohmeter needle deflection vs electrical

resistance and transmittance as a measure of

concentration.

Generally, the relationship between the

measurement and the quantity being measured

may be represented graphically by a monotonically increasing or decreasing curve

(Figure 1).

Q Q

Figure 1. Relationship Between Measurement

(M) and Quantity (Q) Being Measured

25
One essential requirement is that the

relationship between measurement and quantity

be monotonic. The situation shown in

Figure 2 is unacceptable because of its

ambiguity: the measurement M could mean

that the quantity is either Q or Q^.

ignoring sample variability and other sample

problems, measurement of the samples will

result in a curve relating the measurements

to the quantity being measured (Figure 3)

Figure 2. Ambiguous Relationship: Does

Measurement (M) Indicate Quantity Q

1 or Q^i

A measuring process usually is defined

by a protocol or standard test method such

as an ASTM standard. In addition to sampling

error, a realization of a measuring process

involves four elements:

(1) The clarity and specificity of

the test protocol;

(2) An observer or operator and his

or her interpretation of the

protocol;

(3) The type, condition, and

calibration of the instrument

and equipment used;

(4) The environment (temperature,

humidity, air polution, air

movement, atmosphere pressure,

vibration, gravity, intensity

and wave-length distribution of

illumination, etc.)-

Consider a single laboratory and for the

moment one operator and one set of test equipment. Over a short period of time the

environment will usually remain reasonably

constant. Then for a series of samples

covering the test range of interest and

26

Figure 3. Monotonic Relationship Between

Measurement (M) and Quantity (Q)

If a second and a third series of

samples identical to the first series are measured in the same laboratory but by

different operators, or on different days,

or using different sets of equipment, then

a family of closely related curves will be

obtained (Figure 4).

Figure 4. Varying Conditions Within a Laboratory
If similar series of tests are made in a second laboratory, a second family of curves relating measurements to quantity will

be obtained (Figure 5).

M

Figure Varying Conditions Within and

Between Laboratories

An independent measure of Q may be

available in some cases, from the manufacturer of the samples (for example, concentration) or from results obtained by a superior reference method. An alternative

approach - and in most cases the only

available approach - is to represent the

quantity being measured by the average value

obtained by a number of laboratories all

applying the specified measuring process.

B$y so doing the curves representing each

laboratory usually are linearized (Figure 6)

.

M

(one lab)

M (all)

(averages all labs)

Figure 6. Linearization of Curves When

Plotting One Laboratory Against Averages

of All Laboratories

If a laboratory's results agree exactly

with averages over all laboratories, the

laboratory would be represented by a 45° line

through the origin (Figure 7A) . Generally a laboratory can have a zero error (e.g., for

a chemical method, failure to make a blank

correction) (Figure 7B) , an amplification

error (Figure 7C), or both (Figure 7D)

.

M

M (all)

Figure 7. Possible Situations: a) Laboratory Agrees with Averages Over All Laboratories; b) Laboratory Has Zero or Constant

Error; c) Laboratory Has Amplification or Proportional Error; and d) Laboratory has

Both Errors

This representation of the results of an inter laboratory testing program (or

"round robin" as it sometimes is called) is

known as the linear model [1, 2, 3] 1. The

deviation of the line from 45° through the

origin is a systematic error consisting of two components: the zero error and the

amplification error.

When the procedure followed by a laboratory differs appreciably from the standard procedure, some non-linearity may be

expected. Also, even if approximately

linear, serious laboratory-material interaction may occur. What is meant by laboratory-material interaction in the linear

model?

Figures in square brackets indicate

the literature references at the end of the

paper

.

27
When the actual values obtained by a laboratory for several materials are plotted

against the average values obtained by many

laboratories, the plotted points for the

laboratory usually do not fall exactly on the

best fitting line for the laboratory

(Figure 8)

M

M (all)

Figure 8. Deviations of Individual Test

Results from Laboratory's Line

There are three reasons why the points

deviate from the line:

(1) Sample variability;

(2) Moment-to-moment instrument/

environment/operator variability;

(3) Laboratory-material interaction.

The first two of these may be considered

random effects and theoretically can be

reduced as small as desired by replication

(i.e., by replicating the measurements on a series of test pieces and averaging the test observations). However, when this is done,

the points often still do not fall on the

line because of laboratory-material

interaction.

It is easier to give an example of this

interaction than to define it precisely.

Consider two materials that would give

exactly the same test value if a certain

interpretation of the test protocol is

followed exactly. Also, suppose that the

first of these materials is quite sensitive

to relative humidity but the second is not.

If a laboratory fails to control relative

humidity sufficiently close to a required

level (perhaps because of its interpretation

of the protocol) , it could obtain the correc

test value (or at least a value close to its

line) for the second material but not for

the first. It can be said then, that there

is an interaction between the sensitivity of

the material to relative humidity and the

failure of the laboratory to control relativ

humidity close enough. The distance of the

plotted point from the line for this labora- 1 tory and material will remain large no matte- how much replication is involved.

No matter how well the test protocol is

standardized and how well the laboratories

follow the protocol, some laboratory-mater ia

interaction will be present, just as there

will almost always be some systematic laboratory error and some random replication

error. Standardization does not result in

perfection, hopefully it does result in the

reduction of these errors to the level where • they have no practical significance.

Evaluation of Laboratories

1

les

tin

: pi

in view of the theory of the measuring

process, what is it that should be examined

in the evaluation of laboratories? Ideally

the examination of a laboratory with regard

to its capability to perform a specific test f

5

' method should provide information on its

systematic (zero and amplification) errors, 3 its ability to replicate its own measurement

and how that varies with test level, its

residual or interaction errors for each

class of materials, and the likelihood that 1 the laboratory's quality of performance will

be maintained. The last is a question of

being in statistical control.

ith

itti

equ

For the purpose of laboratory accredita

tion, the capability of a laboratory to per- form may be examined by three methods:

i (1) Information gathering, such as a questionnaire completed by the

laboratory and evaluated by

experts

(2) On-site inspection of the

laboratory;

(3) Proficiency testing.

j, As used here a proficiency test is any means by which the actual testing perfor- , mance of a laboratory is determined.

OSS

lillt

ici

tH

!®1

5ffl

ive

He

1«i

2 8
The capability to perform might be

examined exclusively by proficiency testing

in the form of a periodic interlaboratory

l(or round robin) testing program in which a number of materials are distributed simultaneously to each of the laboratories

(desiring accreditation or continued accreditation. If a sufficient number of materials

and replicate test specimens of each material

,are distributed to each laboratory and this

Is repeated, say, every few weeks, a very

complete picture of the laboratory's performance would be obtained, including

information on systematic, random and interaction errors and the stability of the

laboratory's performance over a period of

time. Unfortunately, such a proficiency

testing program is likely to be very

expensive both to administer and in testing

.time for each laboratory.

At the other extreme, in order to assure

pontrol over the four elements that enter

into each realization of the measuring

process, all laboratories that wish to be

Accredited could be required to have exactly

'the same equipment, calibrated frequently

py the same calibration service. Also, all

personnel (testing technicians and supervisors) could be required to have the same training, perhaps in a special course

established by the accrediting authority,

iven then, it might be necessary to be concerned with environmental factors such as gravity and barometic pressure. Can a laboratory located in Denver obtain the same

test results as one located close to sea

ILevel? Obviously if all laboratories are

ising identical equipment and identically

trained personnel, it is more likely that

l:hey will obtain equivalent test results,

Ifith relatively small systematic and interaction errors. Again, unfortunately, such

requirements are likely to be expensive and

aoreover tend to freeze the state-of-the-art.

Between these extremes are various

possible combinations and types of requirements, questionnaires, inspections and proficiency testing, the best combination varying with the nature of the test method, the

lumber of laboratories and so forth. For

example, with heavy dependence on an extensive questionnaire to determine details

concerning a laboratory's personnel,

equipment, quality control procedures,

idequacy of financing, and so forth, and on

a thorough inspection by trained inspectors,

a single proficiency test sample might be

ised. From the information obtained about

test equipment and procedures by the

juestionnair e and inspection, experts should

)e able to judge the likely performance of

the laboratory. The test result obtained by

the laboratory in testing the proficiency

sample will either verify this judgment or

it will not. If not, the result cannot distinguish between a systematic deviation, a random error, an interaction, or a poor

sample. A disqualified laboratory will

certainly claim the last.

One combination that should work well

in many situations includes for each test method

(1) A small number of pertinent

questions about the equipment and

its calibration, with the response

of the laboratory evaluated by

experts. The experts may generate

additional questions either for

mail response or for the inspection.

(2) A quick inspection to verify the

existence of the claimed equipment and calibration records, the

general condition of the equipment, and one or two critical

dimensions, the last being

especially important if equipment

design requirements have been

changed over the years.

(3) A periodic proficiency test, the

size and frequency of which depending on the cost of testing

and samples, and the likely

stability of the equipment. The

proficiency test should include

at least two test samples, one each of two very similar but not identical materials. Such a minimal proficiency test provides

for each laboratory just one degree of freedom for calculation

of systematic plus interaction

error and, if the materials are sufficiently similar, approximately

one degree of freedom for replication error. This minimum program

cannot provide information on how

the systematic error varies with

test level nor how much interaction error is present.

The above two-sample proficiency test was recommended by Dr. Jack Youden [4, 5] and is used in some of our Collaborative

Reference Programs which were established to help a participating laboratory evaluate its

performance in comparison with that of other

laboratories. The Youden approach has been

very successful because of its easily understood graphical presentation (Figure 9) in

29
which each laboratory is represented by a point on the graph, the point being at the

intersection of the coordinates of the laboratory's results for the two samples. As

the points for 95% of similar laboratories

should fall within the ellipse, the perfor- mance of a laboratory with a point appreciably

outside the ellipse is very questionable.

[3] Lashof, T. W. , Precision of Methods foi : Measuring Tensile Strength, Stretch,

and Tensile Energy Absorption of Paper

Tappi , Vol. 46, No. 1, January 1963,

pp. 52-59.

[4] Youden, W. J., Graphical Diagnosis of

Inter laboratory Test Results, Industri;

Quality Control , Vol. 15, No. 11, May

1959, 1-5.

[5] Mandel, J. and Lashof, T. W. , Interpretation and Generalization of Youden

Two Sample Diagram, Journal of Quality

Technology , Vol. 6, No. 1, January 197

pp. 22-36.

SAMPL

Figure 9. Youden Two-Sample Diagram

What has been presented here is quite simplified compared with the real world of

laboratory evaluation, but an understanding

of the theory of the measuring process as presented here, or a suitable extension of

it for a specific testing situation, is very

helpful in the design of appropriate

examination procedures for the evaluation of

laboratories

References

[1] Mandel, J., The Measuring Process,

Technometrics , Vol. 1, No. 3.,

August 1959, pp. 251-267.

[2] Mandel, J. and Lashof, T. W. , The

Interlaboratory Evaluation of Testing

Methods, ASTM Bulletin , No. 239,

July 1959, pp. 53-61 (TP 133-140).

J

30
[ational Bureau of Standards Special Publication 591, Proceedings of the National Conference

>n Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

Q), September 25-26, 1979. Issued August 1980

INTER- LABORATORY ROUND ROBINS FOR DETERMINATION OF

ROUTINE PRECISION OF METHODS

CHARLES A. BICKING

TRACOR-JITCO , INC. 1776 E. JEFFERSON ST. ROCKVTLLE, MARYLAND 20852

Attention is directed to the importance of determining precision

of test methods under routing conditions of use. Comparisons of precision values obtained under very limited conditions have a very nar- row range of usefulness. Such "ideal" precision values lead to incorrect use of "checking limits" in attempts to control performance of

operators and of laboratories and of statistically invalid rules for

retesting, rejection of results, and initiation of third-party referee

testing. Measures of precision under routine conditions, both withinlaboratory and between-laboratories , are necessary to identify the

contribution of the test method to the total variability of measurement

in research, manufacturing, and in exchange of materials and goods in commerce

.

Key words: Accuracy; between-laboratory precision; components of

variance; "duplicity"; ideal conditions; measurement process; "omnifariousness" ; precision; round robin; routine conditions; variance

model; within-laboratory precision.

1. Introduction

An essential preliminary to the planning of an inter-laboratory round robin is

agreement on the uses to be made of the resulting precision information. This is so jbecause there are many suitable models on which design of the round robin may be based.

Proper use of the information depends on the

components that have been included in the

model. If, for example, within-laboratory

precision is determined by one operator,

using one instrument, making repeated tests

on a standard sample within a short period

of time, ideal conditions may have been represented, but the result will be of little

use in describing agreement of test results

when the test method is used in research,

manufacturing, or in the exchange of materials in commerce. Precision obtained

under ideal conditions has limited, tightly

circumscribed uses in the development of

31

test methods, comparison of competing methods, and in evaluating modifications to methods. However, the best possible agreement of results under ideal conditions has

little relevance to agreement among measurements in routine use of test methods.

2. Measurement as a Process

The measurement of a property of a material should be looked upon as a process.

The major elements in the process are materials, methods, instruments (or reagents),

operators, time, environmental conditions,

and laboratories.

A variance model is applied to the process. The total variance is made up of com- ponents due to the elements which make up

the process. The process may be truncated

at any point. The model may be as simple

as the one representing the ideal conditions
mentioned in the Introduction; a standard

sample, tested two or more times by the same operator, using the same instrument over a very short period of time. The resulting

precision information is useful only for

describing this particular, truncated measurement process. It is not applicable for

any comparison with measurements made by

other operators using other instruments at different times.

On the other hand, the model may involve all the recognized elements or variables; two or more samples of a material,

each tested more than once by each of two or more operators using different instruments

(of the same or different design), at different times, under different environmental

conditions and in different laboratories.

The model may call for omitting any component, or can combine components in any mix.

It is extremely important to know exactly

what model is used in a round robin program.

The way components are combined determines how precision information may be used

to test the reality of observed differences.

In practical applications of a test method,

we are interested in the total contribution

of the measurement process to the apparent

variations in observations made on the product. We know that in use of the method

operators vary, instruments vary, that materials are not perfectly homogeneous and,

perhaps most obviously of all, that it is

difficult to repeat results time after time.

The extent to which each of these elements

does vary in the measurement process must

be taken into account.

3. Control of the Elements of the

Process

Control is partly in the hands of the

planners of the round robin, partly within

each of the participating laboratories, and

partly in the purview of the laboratories as

a group.

In the first instance, the planning

group needs assurance that the material

distributed for test is as homogeneous as possible. Evidence of this homogeniety may

be provided by assurance that the production

process was in statistical control. Even

with that assurance, samples for test should

be distributed at random.

3.1 Operator Control

Operator control is maintained through

plotting a control chart for each operator

in the laboratory. It is sometimes suggested

32

that this control chart be based on duplicate tests run periodically on routine samples. A control chart for the range of

duplicates is kept. The operator is con- sidered to be in control when all ranges ar within the control limits. This gives one measure of operator performance, that is,

how well results can be duplicated, on the

average, when measurements are made within

a very short period of time. One danger is

that the results may not be independent. A more serious deficiency in this approach is.: that, as experience invariably demonstrates

the real problems that operators have are with changes that occur in performance of

the test over time. These changes are due 1 to differences in set-up and calibration of

instruments, changes in reagents, temperature or humidity changes , fluctuations in

laboratory services (water, electricity,

etc.) or other uncontrolled or unrecognized

variables

Therefore, it is recommended that oper

ator control be based on control charts for

averages, as well as ranges, of periodic

replicate tests on portion of a standard

sample. Incidentally, a standard or a spiked sample is also useful in measuring

the accuracy of a method. Whereas precisio

refers to mutual agreement among repeated

measurements, accuracy refers to the agreement of the average of observations with a standard or accepted value. The standard

deviation provides an inverse measure on precision; the constant error, or bias, is an approximation of the accuracy.

3.2 Instrument Control

Instrument control is achieved by regular recalibration of instruments and throu

development and use of standard curves. Al

so, it is possible to follow the success of

these activities by maintaining a control

chart on the instrument, handled in a way

similar to the operator control chart.

3.3 Time Control

The effects of time may be evident in uneven performance by the operator or by an instrument but must also be watched as a phenomenon affecting everything that goes

on in the laboratory. Points out of contro: on the operator control chart for averages

may be due to an operator fault or to some j environmental change. The grouping of test replicates and their spacing in time should

be based on a rationalization of possible

occurrences in the laboratory environment.

The control plan must be developed with ful

attention to the theory of statistical

control.
Experience has shown that in a well-run

laboratory, good operator and instrument con-

- trol are achievable. Time control may be more difficult. It is not possible to tell

| exactly how much variability is due to operX ator and instrument unless these components

| are included in the round robin model. It

I is often assumed that they are controlled

i and that their contribution to the total

J variance is small. The assumption is made

§ that this small variance is due to system

| causes that are built into the test method

I at the time it was developed. Smaller oper3 ator or instrument variability (after all

I points out of control have been assigned a cause and the causes removed) would require

further developmental work on the method.

I Round robin models intended to produce in-

|j formation useful in routine application of

the method should usually include performance

[ of the method on more than one day, i.e.

,

: time should always be one component in the

model

.

i 3.^ Inter-laboratory Control

I Inter-laboratory control is attained by

ij use of a standard sample testing program or the exchange of samples in a proficiency

! testing program. Such programs are widely

a used among laboratories belonging to one ori ganization or an association of organiza-

- tions. Unfortunately, they do not exist

widely among independent laboratories joining together in a round robin with the ex- plicit purpose of measuring precision of

'! test methods

.

To an extent, use of a pre-test before

running the main round robin will serve the

purpose of control. However, control is an

• ongoing activity and short-term attempts to

j locate and correct causes of large inter-

- laboratory differences are only partially,

or even only marginally effective. There

are no short-cuts to good control.

h. Avoiding the Pitfalls

The shortcoming of the standard materj ial, one operator/instrument, short period

of time model is that it results in what

i W. J. (Jack) Youden dubbed "chemist's duplicity". Such a model usually results in

a more optimistic level of precision than

' when the model calls for independent ob-

. servations performed on different days. The

within-laboratory part of the "duplicity"

; model is usually accompanied by a betweenI laboratory part. The between-laboratory

part then confounds all differences between

operators, instruments, and time with real

- laboratory differences. This is an

33

"omnifarious" model which leads to an impossible situation when it comes to analyzing

the reasons for differences between laboratories.

The "duplicity model" may be useful in

comparing one method with another or measuring the effect of a modification of a method. Care must be taken that exactly the

same model is applied to both trials. Also,

this model may be useful in maintaining the

precision of a method (determining whether

it has changed), but, again, the model must

be strictly followed each time it is checked

The problem with publishing the ideal

precision in a method is that there is an implication that any single pair of results

run in any lab at any time should have as good a result. It becomes particularly reprehensible when precision information of thi

sort, whether expressed as a standard deviation or as a "checking interval" is used to try to control an operator or a laboratory.

See Section 3 on control of the elements of

a measurement process. At best, these numbers can be used only as a very rough guide

as to whether a new operator or a new laboratory is in the general region of precision

achieved in good laboratories. Even as a rough guide, the limitations of this model

must be recognized.

Unfortunately, when "checking intervals" are presented in a method, users of

the method are encouraged, in a completely

indefensible way, to use the intervals to call for retesting, for accepting the re- test with or without regard for the original

result, or for justifying comparative testing programs or the employment of a referee

laboratory. None of these things are indicated on the basis of such extenuated evidence .

With so little to recommend the practice of using minimum models, it is hard to

explain why more attention has not been

given to precision in the routine use of

methods

.

5. A Useful Routine Precision Model

Prerequisites for a precision model for use in round robin testing include existence

of a test method which has gone through a shakedown period in one laboratory, ruggedness testing to demonstrate that it is not

sensitive to minor perturbations in test

conditions, and which exhibits control within laboratories. It is also desirable that

it have been incorporated in an inter-laboratory proficiency testing program.
Because statistical control of the method has not always been recognized as a re- quirement in all laboratories, it is desirable that a preliminary round or rounds be

run to familiarize all participants with the

method and to work out any bugs which still

exist

Assuming that attention has been given

to such preliminaries, a number of laboratories, not less than six and preferably at least 10 should have agreed to praticipate

in the round robin. Homogeneous samples of

material must have been obtained and distributed at random among the participating laboratories .

If the method is to be used to test more

than one material, a number of different materials should be sampled so that the whole

range of use of the test will be represented.

Structural differences in materials, interferences of possible contaminants, and other

things that could account for a variable response should be considered in selecting the

materials. If the test applies to only one material, samples should be supplied at different levels of content of the property to be measured, for example, at different concentrations. Precision may be constant at all levels, may be proportional to the level,

or may respond non-linearly (logarithmicly,

exponentially, etc.) At least three levels,

and preferably more, are necessary to measure curvature.

Assuming that operators and instruments

are controlled and their contribution to the

total variation in the measurement process

is small, a useful model is to have one operator in each laboratory make duplicate

determinations on each sample on each of two

days. The within-laboratory precision will

be made up of components due to the duplicates and due to days. The between-laboratory precision will be made up of the withinlaboratory precision plus the component due

to laboratories.

At least the within-laboratory and between-laboratory standard deviations should

be presented. If for either of the categories of precision, the values are the same

(not significantly different) for all materials or for all levels of a material, only

a single value need be reported for withinand one for between-laboratory precision.

If the standard deviations are proportional

to the level, the precision may be reported

as the coefficient of variation; the standard

deviation divided by the average times 100.

If the precision varies with material or level, a table may be prepared grouping

materials or levels with the same (not significantly different) precision. If the

precision is related to the level curvilinearly, the results may be presented as a curve

6. Conclusion

The degree of skill employed in planning a round robin is indicated by how well

the planner has succeeded in avoiding "duplicity" and "omnifar-ousness" . The model

should be directly aimed at the purpose to which the precision information is to be

put, how and in what environment the test

method is to be used. Since standard methods of test are intended for use in all kirn of practical situations, it is imperative

that inter-laboratory round robins be planm

for determination of precision of methods ui der routing, not ideal conditions.
"ational Bureau of Standards Special Publication 591, Proceedings of the National Conference

<n Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

JD, September 25-26, 1979. Issued August 1980

MONSANTO' S ANALYTICAL TESTING PROGRAM

DAVID H. HAILE AND WILLIAM M. HAYNES

ENVIRONMENTAL ANALYTICAL SCIENCES CENTER

MONSANTO RESEARCH CORPORATION

DAYTON, OHIO 45407

Recognizing the need to ensure the validity of environmental data,

Monsanto has charged its Environmental Analytical Sciences Center (EASC)

with ensuring that all analyses for monitoring the environmental effects

of its operations are credible. Thus far, the effort has focused on the

credibility of the industrial hygiene analyses performed throughout the

company. Since round-robin testing programs are an effective tool to demonstrate the validity of analytical measurements, the EASC established

a Monsarto Analytical Testing Program. Under this program, modeled after

NIOSH's PAT Program, the EASC periodically supplies standard samples of

compounds of interest to various company laboratories. Standard samples

of materials, unavailable elsewhere, are prepared on sampling substrates

commonly employed by Monsanto laboratories.

Two compounds of major interest to Monsanto are acrylonitrile (AN)

and styrene monomer (SM) . Two methods are employed for workplace monitoring of these compounds. Some sites employ a modification of the standard

NIOSH method using charcoal tubes as the sampling medium. In this method

the sample is desorbed with a solvent prior to gas chromatographic determination. The remaining sites employ sample collection tubes packed with

either Porapak N or Chromosorb 101. In this method the sample is thermally

desorbed from the porous polymer tubes and injected directly into a gas

chromatograph. Generating standard samples of AN and SM on these two substrates requires different approaches. The charcoal tube standards are generated on a high-flow manifold at collection rates of between 100-1000

cc/minute using a high-rate permeation source. Porous polymer tube samples

are generated on a low-flow collection manifold (10-20 cc/min.) using lowrate permeation tube devices.

This paper describes the Monsanto Analytical Testing Program, the

standard generation system employed, and the problems encountered in generating known standards. The steps involved in documenting the composition

of the standard samples and the results of several round-robin evaluations

will also be presented.

Key words: Acrylonitrile; industrial hygiene; proficiency testing; roundrobin standards generation.

35
Good laboratory practices regulations

promulgated by the FDA [l] 1 and proposed by

the EPA [2] will affect the way health

effects testing laboratories conduct analyses. The EPA and OSHA have also ex- pressed concern about the quality of data

being submitted by environmental and

industrial hygiene testing laboratories.

GLP regulation, laboratory certification

and accreditation schemes have come into

being as a result of concerns of these

governmental agencies.

Because most of Monsanto' s plant

laboratories submit industrial hygiene and

environmental data to state and Federal

agencies, it is important that these data

comply with GLP concepts and be based on sound scientific procedures. To assure

that its plant laboratories produce data

of documented validity, Monsanto began an aggressive good laboratory practice program by establishing a good laboratory

practices group in the Environmental

Analytical Sciences Center (EASC) at Dayton in January 1978.

The programs established by this

group includes:

1. Preparing a manual spelling out

the essential elements of Good

Laboratory Practices.

2. Encouraging each plant laboratory

to prepare a GLP manual which

describes their site program.

3. Encouraging a voluntary GLP

laboratory audit program.

4. Field validation and audit of automatic continuous area monitors used

to analyze plant environments.

5. Helping plant laboratories work

towards American Industrial Hygiene

Association accreditation. The

ultimate goal is to have one plant

lab accredited in each of the five

Monsanto operating companies (two

have AIHA accreditation at this

time)

'Figures in brackets indicate the

literature references at the end of the

paper.

6. Setting up a standards preparation

laboratory which tailors standards to specific needs of Monsanto plants.

7. Interlaboratory proficiency testing of

plant labs by holding round-robin test? of compounds of interest to Monsanto. I This program has been dubbed the

Monsanto Analytical Testing (MAT) pro-j

gram, and is modeled after the National

Institute of Occupational Safety and

Health's Proficiency in Analytical

Testing (PAT) program.

1. The MAT Program

One of the minor elements of a com- prehensive GLP program is laboratory participation in round-robin studies. However,

the majority of round-robin studies available are not applicable to Monsanto Company

laboratories. That is, the types of contam

inants, samples matrices and sampling substrates available are not relevant to the

majority of analyses encountered by Monsant

environmental laboratories. To fill this

special need, the EASC established a standards generation facility. The mission of

the facility is to prepare environmental

standards that are pertinent to the analyse?

commonly performed within the company.

Participation in the program is com- pletely voluntary. Results are reported in

a coded format to ensure anonymity. All

supporting data relating to the homogeneity

and stability of the standard samples is

reported to participating laboratories with

the results of the study. The philosophy of result reporting is one of supplying the

laboratories with sufficient data to enable

them to make a judgement about their labora

tory's performance rather than grading or ranking laboratories.

To date the effort has focused on the

credibility of the industrial hygiene

analyses performed throughout the company.

Two compounds of major interest to Monsanto

are acrylonitrile (AN) and styrene (SM)

Both of these compounds are monitored

intensively at various plants throughout th< company. Some laboratories may analyze up

to 100 samples per month. It is thus impor- tant to demonstrate the validity of these

analyses, not only from the viewpoint of

worker safety, but also to provide a measurt of data validity since decisions concerning

plant operating procedures are sometimes

based upon industrial hygiene measurements.

36
Within Monsanto, two distinct methods

are employed for workplace monitoring of AN. Some sites employ a modification of the

standard NIOSH method using charcoal tubes as the sampling medium. Other sites employ

sample collection tubes packed with either

Porapak N or Chromosorb 101. The charcoal

tube method samples a larger volume (=20 L)

of air and employs solvent desorption

(usually 2% acetone in CS2 ) . The porous

polymer tube method samples a much smaller

volume of air (^3 L) and employs thermal

desorption of the sample directly into a chromatograph. Sampling an atmosphere of 2 ppm AN, which is the threshold limit value

(TLV) , the charcoal and the porous polymer

tube methods collect about 86 and 13 yg of

AN, respectively.

2. Generation of Standard Samples

Containing Acrylonitrile

Standard samples are prepared by drawing

a known volume of a dynamically generated

atmosphere of AN through the sampling tubes.

The AN atmosphere is generated with a Kin-Tek

Precision Calibration System equipped with a permeation source. Pure air is supplied with

an Aadco Model 737 pure air generator. The

AN contaminated air stream moves from the

permeation device and is diluted with additional air before it passes through a glass

mixing chamber to a multiport glass manifold.

(See Figure 1). The diluted air stream is

drawn through sampling tubes attached to the

glass manifold by a sampling plenum. The

sampling plenums were constructed in a similar manner to those used by the National

Bureau of Standards [3].

Figure 1. Schematic of Standards

Generation System

In the case of charcoal tube samples,

the sampling plenum consists of three sets of

| flow limiting orifices (hypodermic needles)

.

Four matched orifices comprise each set. The

orifices within each set were selected such

that the individual flowrates do not vary by more than ±2% from the set averages.

The average flowrates of the three sets of

orifices are 0.933, 0.440 and 0.110 L/min.

Using this apparatus (Figure 2) , three

different loadings of AN on the charcoal

tubes can be obtained simultaneously. For this study, the desired amounts of AN loaded onto the sample tubes are approximately 11, 43, and 90 yg of AN; simulating

the amounts that would be collected from 20

L of air containing approximately 0.2, 1.0

and 2 ppm of AN respectively.

Figure 2. Charcoal Tube Sampling

Plentum (one orifice assembly shown)

For porous polymer tube samples, the

sampling plenum (Figure 3) consists of

twenty flow- limiting orifices (33 gage

hypodermic needles) carefully selected to provide flowrates of within ±1.5% of one

another. Figure 4 shows the construction

of an individual orifice assembly. The

average flowrate of these matched orifices

is 12.28 mL/min.

37
Figure 3. Porous Polymer Tube Sampling

Plentum (four orifices shown)

A lower flowrate is required for

polymer tube samples since the pressure

drop across the packed tube increases

dramatically with increasing flowrate. At

a flowrate of less than approximately 35 mL/min the pressure drop across a tube

becomes negligible in comparison to the

pressure drop across the orifice. A constant flowrate is obtained regardless of

the flow characteristics of the sample

tube. Varying amounts of AN are deposited

on the tubes by varying the collection

time. Usually, the collection time is

adjusted to obtain either 6 or 13 yg of

AN, simulating the amounts that would be

collected from 3 L of air containing

approximately 1.0 or 2.0 ppm of AN, re- spectively.

Figure 4. Individual Orifice Assembly

Early in the program, the sampling

plenum was constructed with fine metering

valves in place of hypodermic needle or- ifices. This configuration was abandoned

because the metering valves were found to require periodic calibration and readjustment in order to maintain a set flowrate

variability of not greater than ±2%.

However, this type of apparatus will hold

its calibration for several days and was used to produce the first set of porous

polymer tube samples used in MAT Round 1.

3. Standard Generation Protocol

Before standard samples can be em- ployed in a round-robin study, an assessment of sample integrity must be undertaken. EASC uses the following general

protocol to ensure the production of

acceptable standards:

3 8
A. Determine the precision

and accuracy of the analytical technique employed for sample quality

control. This determination is made

at each contaminant level.

B. Determine the reproducibility of the

generated standards by producing and

analyzing several sets of standards at the desired loading levels. If

possible, verify the accuracy of the

generated standards by using an independent method(s) or standard(s)

to determine the contaminant loadings.

Ideally, the independent method (s) or standard (s) should have clear traceability to the National Bureau of

Standards. If no independent method

or standard is available, have a set of standards analyzed by a different

laboratory to verify loadings.

C. Determine the stability of the generated standard over at least a two week

period. Ideally, the standard should

be stable at room temperature for at least two weeks. However, in some

cases, sample preservation steps

(e.g., refrigeration) may be necessary.

D. Based upon the observed contaminant

loadings obtained in step B and/or C,

calculate the average observed loading

at the 99% confidence level.

4. Results of the Acrylonitrile

Round-Robin Studies

Two AN round-robin studies have been

completed. Six laboratories participated

in the AN on Porapak N study. Ten laboratories were involved in the AN on charcoal

round-robin. While the majority of participants were Monsanto Company Laboratories,

four laboratories outside of the company

participated in the charcoal tube study.

In both studies, each laboratory received

three samples and a blank. Two of the

samples were identical, simulating samples

contaminated to the action level (one-half

the TLV level). The third sample simulated

a sample contaminated to the TLV level.

The duplicate loadings were submitted to serve as a rough check on intralaboratory

precision. The results of the two studies

are given in Tables 1 and 2. For both

studies, the data indicate individual

laboratories have a good degree of intralaboratory precision. The average relative

deviation from the average of the two identical samples was 1.5% for the Porapak

N round-robin participants and 2.2% for the

charcoal round-robin participants.

3

The interlaboratory agreement was not as good. Laboratories in the Porapak N study

had relative standard deviation of 19.3% and

17% for samples similating the action and TLV

levels, respectively. Laboratories participating in the charcoal study displayed a slightly larger standard deviation of 24.5%

and 25.4% for samples representing the Action

and TLV levels, respectively.

The theoretical and average observed

loadings of the sample tubes for each study

can also be found in Tables 1 and 2. Since

EASC's standard generation facility has just

completed modifications to permit the independent verification of the contaminant

loadings, these values were supplied to MAT

1 and 2 participants as an estimate of the

sample loadings. Future round-robin samples

will have a more definitive number attached

to them. This will be accomplished by analyzing the contaminant air stream as the

samples are being prepared.

References

[l] Nonclinical Laboratory Studies, Good

Laboratory Practices Regulations,

Federal Register , Vol. 43, No. 247,

December 22, 1978

.

[2] Good Laboratory Practice Standards

for Health Effects, Federal Register , Vol. 44, No. 91, May 9, 1979~

[3] Cadoff, B. C., Hughes, E. E., Alverez,

R. , and Taylor, J. K. , Preparation of

Charcoal Sampling Tubes Containing

Known Quantities of Adsorbed Solvents,

National Bureau of Standards, Department of Commerce, Report Number NBS1R,

74-530, July 1974.
Table 1. Participant's Results, MAT Round 1 Acrylonitrile on Porapak N Tubes

AN Reported, yg

Sample Sample Sample

Laboratory X Y Z Blank

1 6 69 6 51 13 19 0 01

2 5 75 6 0 12 25 0 08

3 4 4 4 2 8 9 0

4 1 675* 4 854 10 46 0 027

5 4 33 4 45 9 37 0 01

6 6 68 6 61 13 36 0 00

X = 5.50 X

f

. = 11.2

(xSy) UJ

a=1.06 a=1.9

rel. a = 19.3% rel. a = 17%

Summary of supporting information for samples:

X5Y Z

Theoretical loading (yg) 6.94 13.74

EASC average observed

loading (yg) 5.92±0.34+ 11.26±1.34t

t99% confidence interval based on the analysis of 9 tubes

*data point omitted from calculation of mean; rejection

based on Q test at 90% confidence level

Table 2. Participant's Results, MAT Round 2 Acrylonitrile on Charcoal Tubes

AN Reported, yg

Sample Sample Sample Blank

Laboratory X Y Z

1 26 62 26 39 62 15 00.00

2 35 1 35 8 82 4 0.4

3 40 40 95 ND

4 29 27 63

5 35 5 42 9 79 3 ND

6 31 6 31 5 77 5 0.0

7 39 0 39 8 89 6 000

8 50 3 48 5 110 0 0.0

9 33 5 33 6 78 0 ND

10 18 22 20 04 38 71 00.00

X, . = 34 22 X 77 57

(z)

=

a = 8 38 o = 19 68

rel a = 24 5% rel . o = 25 4%

Summary of supporting information for samples:

X&Y Z Theoretical loading (yg) 43.36 99.91

EASC average observed

loading (yg) 40.71±0.88* 98.14±1.18*

ND - not detected

*99% confidence interval based on analysis of 24 tubes

40
tional Bureau of Standards Special Publication 591, Proceedings of the National Conference

Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaither sburg

,

, September 25-26, 1979. Issued August 1980

LABORATORY EVALUATION TECHNIQUES - US ARMY

CALIBRATION PROGRAM

CHARLES B. COULTER

US ARMY METROLOGY AND CALIBRATION CENTER

US ARMY MISSILE COMMAND

REDSTONE ARSENAL, AL 25809

The US Army utilizes three principal techniques in the evaluation

of its calibration laboratories: Technical Measurement Audits, Technical Inspections and Certification of Technicians. Each major calibration laboratory is subject to each of the above techniques or a combination thereof contingent upon its mission and location within the

Department of Army logistical complex. Each of the techniques is administered or managed by the US Army Metrology and Calibration Center

(USAMCC) at Redstone Arsenal, Alabama.

Technical Measurement Audits are objective in nature while Technical

Inspections and Certification of Technicians rely heavily on subjective

considerations. The combination establishes a comprehensive quality

assurance program for calibration.

Key words: Audit; calibration; calibration laboratory; certification;

evaluation; inspection.

1. Introduction

US Army calibration laboratories are operated in accordance with regulatory re- quirements and technical guidance provided

from higher echelon activities. The regulatory requirements and technical guidance assure, in theory, that the calibration laboratories perform at a level sufficient to

meet mission requirements. In practice, adherence to regulatory requirements and/or

higher echelon technical guidance does not

in itself assure quality performance. Additionally, laboratory personnel are always

subject to ignore the requirements and guidance. In order to assure satisfactory performance, the US Army periodically evaluates

its laboratories, utilizing principally

three different techniques: Technical Measurement Audits, Technical Inspections, and

Certification of Technicians. The techniques employed are used to confirm that the

laboratories are capable of and perform at a level sufficient to fulfill mission requirements .

There is no attempt to use any of the

aforementioned techniques to rank or rate

the laboratories against one another. The

primary intent is to confirm performance at

a predetermined level. It constitutes a pass-fail evaluation.

The primary responsibility for the

establishment, performance, and/or monitoring of the evaluation processes rests with

the US Army Metrology and Calibration Center

(USAMCC ) at Redstone Arsenal, Alabama. The

USAMCC is tasked with program management and

technical guidance for the total US Army

Calibration System. The evaluations serve

as the means to determine the effectiveness

of the program management and the validity

of the technical guidance.

2. Technical Measurement Audits

In essence, a Technical Measurement

Audit is a "hands on" performance test of a calibration laboratory's ability to provide
precision measurements. Unless problems are encountered, audits are conducted entirely

via mail. The USAMCC provides a primary or reference lab standard, with the certified

value unknown to the audited laboratory, to

the audited laboratory for calibration. The

value measured by the laboratory is then com- pared to the primary certified value of the

standard. The emphasis is on the accuracy

and precision of measurement. Management

and administrative practices are of peripheral concern only.

Parameters currently maintained in the

audit program are: DC Voltage, AC Voltage,

Resistance, Capacitance, Attenuation, VSWR,

Microwave Power, Absolute Pressure, Gage

Pressure, Length, and Mass. The standards

for the various parameters are supported by

the US Army Standards Laboratory (ASL), the

Army's primary level calibration laboratory,

located at the USAMCC. There are no rigid

intervals of calibration. The audit standards are calibrated as deemed necessary.

Each audit package is composed of no more than one DC and low frequency, one microwave and one physical parameter. (For

the remainder of this paper, audit package

refers to the hardware-audit primary or sec- ondary reference standards; and accompanying

forms and instructions shipped to the audited

facilities.) The audit packages are tailored

to the capability of the audited facility.

Upon receipt of the package, each facility

is allowed ten working days to perform the

measurements at the points specified and

place the audit package and report of measurement in the mail to the USAMCC. The

report of measurements must include all raw

data, manner of reduction (computations),

calibration procedure (to include traceability of laboratory standards) and name of the

technician peforming the audit.

Upon receipt of the report of measurement at the USAMCC, an analysis of the re- sults is performed. The analysis addresses

the absolute difference between the mean

value of the standard as measured by the

audited laboratory and the certified value

as provided by the ASL, and statistically

measures the dispersion (standard deviation)

about the mean value provided by the audited

laboratory. If both the mean value and standard deviation are judged acceptable and the

administrative requirements of the audit

have been met, the audited laboratory is

notified by letter and no further action is

taken. Because US Army laboratories utilize

US Army Technical Bulletins describing step

by step calibration procedures, little effort is expended toward analysis of the

procedure unless unsatisfactory results ar> obtained.

In the event that an audited laboratory's results are unsatisfactory (either j| mean value or standard deviation), that

laboratory will be required to cease suppo:|

in the affected parameter or downgrade the:; capability until the source of error is lo- cated and the conditions surrounding that .] error rectified. Initial effort, (joint bt.j tween the facility and the USAMCC ) as a re-,j sultant of unsatisfactory performance, is j directed toward cross checking of the audi"'] facility's standards within house, reeompu-j

tation of data, recertification of audit

standards and confirmation that the proper i procedure was followed in total. In gener;- the initial effort locates the source of

problem. If the problem cannot be isolatec

to drift in a standard (audit or laboratorystandard), computational error, or error o: omission in the procedure, then the effort

to locate the problem is expanded to incluc

examination of the technical correctness ol the procedure, actual capability of the lal

oratory's standards, training of the techn^

cian, competence of the technician or satis,

factory support of the laboratory's standai;

by the supporting activity. Concurrent wi1

the attempt to isolate the problem, the

audited activity must analyze potential im- pact upon supported items caused by such

unsatisfactory measurements. Corrective

action is taken as required to assure va- lidity of calibration of supported equipment .

Subsequent to the location of cause of the problem, a reaudit utilizing different t, serial numbered audit standards is conducts

to confirm alleviation of the problem. Ree sons for past failures in the audit "run th

full gamut". Although rare, the reason for

failure in an audit is sometimes never as- certained. Extensive investigation leads t no conclusions. If a reaudit is then performed satisfactorily, the laboratory resumes full operations.

Upon conclusion of an audit, the com- ; mander of the installation at which the latj

oratory is located is provided a report by

letter from the USAMCC. The report provide

quantitative results (difference, standard

deviation), a finding of AGREEMENT/NO AGREE

MENT (satisfactory or unsatisfactory) for

each parameter audited, complete discussion

(to include corrective action) or any problems encountered and comments concerning a facility's ability to react to and rectify

problems in the event of a NO AGREEMENT

finding.

42
Every effort is made to convince lab-

's oratory personnel that audits are for their

benefit as well as a test of their ability.

Excluding computational errors as a source

of failure, there have probably been as many

failures due to conditions external to the

audited laboratory's control as there have

c been directly attributable to the audited

! laboratory. When personnel view the audit

( in this manner and perform it as routinely

as a normal calibration then the results

t truly reflect the ability of that laboratory

: to meet its mission requirements. In reality, because of potential consequences of a

"I failure, a significant number of laboratories

i over-emphasize the audit by utilizing only

I certain technicians, increasing the number

1 of measurements, utilizing multiple methods

"j and increasing time spent. While an audit

: in this instance can check standards, pro-

: cedure, certain personnel, etc., it does not

I verify the everyday quality of the laboraj tory.

i Technical measurement audits are con-

:| ducted quarterly at eleven Army Area Cali-

' bration Laboratories (responsible for geo-

- graphical area support), semiannually at

| twenty-three Army Internal Calibration Lab-

'i oratories (responsible for single installai tion support ) and annually at the WATS Sup1 ply Center, Luxembourg (as a courtesy).

I 3. Technical Inspections

Technical inspections are on-site inspections of installation calibration programs. They differ from audits in their

scope and the fact that they encompass labi oratory customers as well as the laboratory.

| While audits are based almost exclusively on quantitative or objective judgments, inspec-

; tions require a significant degree of sub-

' jective evaluation.

The USAMCC conducts technical inspections annually at the ASL, six Continental

US (CONUS) Area Calibration Laboratories,

twenty-five CONUS Internal Calibration Facilities, approximately one-third of the fiftytwo Transfer Teams in CONUS, and contractor

facilities as the need arises. In addition,

j

1 the USAMCC has been tasked to expand the

Technical Inspections to include calibration

activities in Europe and the Pacific Theater.

Each inspection team is composed of one civilian team chief and two to four civilian

| and/or military team members . Technical

(hardware) expertise is usually provided by

Senior NCO's. Inspections require from

three to five days each, dependent upon size

of the installation to be inspected.

Transfer teams, as small mobile operations,

are generally inspected in one to two days

by two Senior NCO's.

Technical inspections address two principal areas: (l) The capability of the installation or activity to accomplish its as- signed calibration mission to include the

availability of technical skills, technical

information, measurement methods and accuracy

of measurements, and (2) Compliance with prescribed policies and procedures. Specifically the inspection team reviews: (l) Administrative and management practices, (2)

Availability, maintenance and use of current,

approved calibration procedures, (3) Control

of forms, records, reports and applicable

procedures, (k) The internal quality assurance surveillance program, (5) Control,

maintenance and operation of the calibration

recall system, (6) Environmental and safety

controls, (T) Knowledge of and compliances

with appropriate references, (8) Training

program, (9) Maintenance practices, and (10)

Technical capability of calibration technicians in the maintenance and use of measurement standards, use of hand tools, use of

calibration procedures, and performance of

calibration

.

The most time consuming facet of the

technical inspections (other than time re- quired visiting customer/owner areas) is the

review of the technical capability of the

calibration technicians. This is accom- plished through "end-item checks." Calibration technicians perform a "hands on" calibration for validity of calibration of

an item recently calibrated while an inspection team member observes the performance

"over the shoulder." Such a check encom- passes many of the ten specific areas of re- view listed previously, i.e., use of approved

calibration procedures, use of proper forms,

observance of proper safety practices, etc.

as well as the technical capability of ex- pertise of the technician.

Inspection findings are divided into

two categories: Deficiencies and Observations. Deficiencies are shortcomings that

affect the technical capability of the

laboratory, safety or health hazard or con- ditions that if allowed to continue would

lead to technical problems or health or safety hazards. Observations indicate con- ditions that are contrary to prescribed

regulations and policy or good management

practice but would not be of sufficient impact as to be classified a deficiency. As

the findings are made during the inspection,

every effort is made by the inspection team

to assist in alleviating the shortfall or initiating requisite action.
Selected activities receive an overall

rating of satisfactory or unsatisfactory at the conclusion of the inspection based upon

severity of observations and deficiencies.

An unsatisfactory rating may be the basis

for discontinuing calibration operations until the adverse conditions are corrected.

Satisfactory ratings may be accomplished by

directed corrective action, while unsatisfactory ratings necessitate a reinspection.

An unsatisfactory rating may be given

when any of the following adverse conditions

are noted: (l) An invalid calibration of a measurement system, (2) A safety or health

hazard, (3) Damage to measurement standards,

TMDE or property, (U) An adverse impact on the combat effectiveness of a weapon system

or tactical organization due to calibration

support, (5) Invalid calibration of a critical nuclear weapons TMDE, (6) Calibration of

TMDE or measurement standards when not in the

prescribed environment, and (T) Loss of

traceability to national standards.

At the conclusion of the inspection,

the commander of the installation or activity is informed of the findings. Upon return

to the USAMCC, the inspection team prepares

and forwards a comprehensive report, including required corrective action when appropriate, to the installation commander. The

commander in turn outlines corrective action

taken or to be taken. The next inspection

will confirm whether or not the corrective

action was accomplished.

h. Certification of Technicians

Certification of technicians is a relatively new program and is currently restricted in its application. Calibration technicians are encouraged to voluntarily enroll

in a skill certification program consisting

of a combination written and "hands on" performance examination to verify their skills.

Each participating installation is re- sponsible for developing and maintaining a plan for certifying its calibration technicians. The installation in a joint effort

with the USAMCC establishes a set of standards that are definitive and contain the

essential elements in sufficient detail to adequately evaluate the individual technician's performance.

The standards stress three basic aspects: (l) Core knowledge, (2) Speciality

knowledge, and (3) Demonstrated ability.

Core knowledge addresses the basic skills

and knowledge necessary to perform in a functional area, i.e., interpretation of

drawings and schmatics, use of basic test

equipment, etc. Specialty knowledge refei

to the specialized skills and knowledge

necessary to perform duties as stated in t job description. Demonstrated ability is

the "hands on" application of core and spe cialized knowledge.

The actual evaluation for certificati

consists of a written examination (prepare

by the USAMCC) encompassing both core and

specialized knowledge, a "hands on" profic

ency examination administered by a compete

individual or individuals and an experienc

profile that clearly identified and compar

a technician's actual experience and train

ing with the program requirements. The re suits of the evaluation are reviewed by th

installation's Certification Review Commit

tee who in turn recommend: (l) Approval 0 certification of the technician, (2) Furth

development and training of the technician

or (3) Withdrawal/reinstatement of the cer fication of the technician.

Current policy dictates a three year

interval for recertification of a technici

who routinely performs within the area for

which certification was awarded. If the

technician is reassigned to another area 03 does not routinely perform in that area, then a two year interval is invoked.

5 . Summary

The laboratory evaluation techniques

described above provide a comprehensive ex amination of the ability of US Army Calibration Laboratories to fulfill their mission requirements. Implementation of the

techniques requires considerable managerial

or program, and technical expertise and a degree of common sense. The techniques, i employed conscientiously will confirm the

ability/inability of an army laboratory to perform satisfactorily. To this author's

knowledge, utilization of a combination of

the above techniques has not failed to vali

date the status (satisfactory or unsatisfac

tory) of a calibration program correctly.

The techniques are sound. The effectivene

of the program, as with any quality assurance program, rests with management's abil

ty or desire to correct detected shortcomings .

4 4
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg

,

MD, September 25-26, 1979. Issued August 1980

THE VALUE OF SPLIT SAMPLE PROGRAMS FOR CONTRACTUAL

ACCEPTANCE OF IN-HOUSE LABORATORY PROCEDURES

RICHARD A. MULLINS, P.E.

OLD BEN COAL COMPANY

INDIANA REGIONAL OFFICE

OAKLAND CITY, INDIANA 47660

Because coal is heterogeneous, special procedures are necessary

for the determination of the quality. Commercially, the amount of

coal to be sampled in a year for a contract may be a million tons. Both the seller and the buyer need a proven system for contractual

acceptance of the determined quality. The necessity for including

both the automatic sampler and the laboratory in such a system is

discussed and the resulting split sample program is shown to be a natural development. The program serves the primary purpose of

determining the quality but it also serves as a continuous evaluating system of the in-house laboratory procedures.

Key words: Heterogeneous; in-house; laboratory; preparation;

riffle; sample; standards.

1 . Introduction

Almost every news report refers to coal

as /America's "Energy Ace in the hole." This

major energy source is used primarily as a

fuel for the generation of electricity. In

recent years, the advancements made in increasing efficiencies in utilizing this fuel

have been, in part, due to the construction

of larger steam generating stations. To

supply such users the increased tonnage necessary, more efficient coal handling methods,

also, have been developed. Unit trains with

dedicated equipment hauling as much as ten

thousand tons are common. The mines have installed facilities for rapid loading of these

trains, often loading at the rate of five

thousand tons per hour. The generating stations have installed facilities for rapid

unloading of these trains and some installations can unload such a ten thousand ton

train in less than an hour.

But coal is a most heterogeneous fuel

and special attention and special equipment

is necessary for the correct sampling. Contracts requiring the delivery of a million

tons a year are common and these contracts

usually include a premium/penalty quality

clause guarantee. For such a clause to be

meaningful, both sampling equipment operation

and the laboratory procedures must have con- tractual acceptance by both the supplier and

user. The split sample program with in-house

laboratory analyzing is one satisfactory and

successful solution. It is also a most ex- acting and demanding program.

2. Sampling System

It is preferable that coal be weighed

and sampled at the same jpoint in time. If

there is a lapse in time between these two

events, consideration should be given by

both the purchaser and seller to changes in

moisture during this interval and the con- sequent shift in relationship of moisture to

the true quality of the coal at the instant

when ownership of the coal trans fered from

one to the other C ID- The standard operating

condition is for the train to be weighed at

the loading site, therefore it is preferable

for the sampling to be done at the loading

site.

45
The specifications for such a sampling

system are normally those that meet the

A.S.T.M. Standards D-2013, "Preparing coal

samples for analysis" C2J and D-2234, "Collection of a gross sample of coal" [3 J. It is not unusual for the specifications to be

such that even more increments are taken than

are required to meet these standards. The

design may require as many as three or four

sample cutters in series. This is necessitated by the large primary increment. At a

5000 tons per hour loading rate for 3" x 0 coal, the primary increment can be as much

as thirty-three (33) tons per hour. A typical three stage sampler is shown in Figure 1. Such a sampling system with associated conveyors, feeders, crushers, etc. is a large

material handling installation and is expensive.

2.1 Sampling System Example

If a coal handling installation sampling

3" x 0 washed coal loading at the rate of

5000 tons per hour is used as an example,

what are some of the basic components? Such

an installation requires a primary cutter

with an opening of 9-inches to travel at the

rate of 18-inches per second at 75-second intervals. Each increment is 1400 pounds.

This is thirty-three tons per hour.

A large feeder discharges this coal increment to a secondary cutter, also with an opening of 9-inches. This will divide the initial increment to about 5000 pounds per hour.

This secondary sampler product is then

crushed to a top size of number 8 standard

sieve size. The crusher product is fed to a tertiary sampler with an opening of V%-inches.

Even with three sample cutters operating in

series, the tertiary product can be as much

as 30 kilograms per hour.

3. Sample Preparation

The final amount of product from the

automatic sampler is too much for the laboratory and the amount is divided, with en- closed rifflers, until the desired sample of

about 1000 g is available P4] . There are usu- ally at least three such samples retained: one is for the seller, one is for the purchaser, and one is retained as a referee

sample. Each of these is immediately sealed

in a heavy vapor-impervious bag and properly

identified.

4. Sample Handling

Prompt movement of the split samples of

coal to the laboratories is important. Although the coal sample is sealed in a heavy

vapor-impervious bag, it is essential that

the sample be analyzed as soon as possible

because deterioration is a problem. At one

time, A.S.T.M. Standards referred to deteri

oration of coal samples and warned of time

elapse not to exceed 30 days [53 . This has

since been removed from the standards becau

such a definite time implied and was often L assumed that the deterioration was not seri ous for these 30 days.

5. Sampling System Acceptance

The acceptance of an automatic samplin

system is equally important to both the sel

and the buyer. Both recognize the apprecia

economic importance of its use. Both are d

pendent on it for a representative sample.

It must function as designed and installed.

It must be able to sample a whole consignme

at approved intervals. It must function in' manner that the increments and the resultin

samples are free of contamination. It must

be dependable as an operating piece of equi

ment.

It is important to the seller because

the quality of the product must be known be

fore it can be offered in the market place.

It is important to the purchaser because th' quality of the product will be based on the

split sample. Improper operation can resul

in a bias sample and only extensive manual jj sampling may determine the effect on the sa

pie. Contracts include specific statements

allowing for observation and inspection.

Such a method for procuring a split

sample then, becomes a contractual accepted

procedure

6. Analytical Procedures

The analytical procedures followed by

the laboratories determining the quality fbj

contractual acceptance are those included i: the contract. The contract may state simpl

"All sampling and testing will be in accor- dance with current applicable A.S.T.M. Stan

dards [6] or it may detail specific conditions that must be considered as critical i\ analyzing for full compliance. Other recog

nized standards, such as Bureau of Mines an

International Organization for Standardization, are also designated as applicable.

The parameters considered important foj

the particular coal's use are included in tfj contract and these are checked by the labor

tories. The results are empirical and are reproducible by carefully adhering to definite prescribed methods and standards [7J

46
Laboratories testing the same coal sam-

, pie but using different standards may report

'i results that do not agree. Such differences

are not to be interpreted as a fact that one

laboratory is reporting incorrect results.

* Each can be correct for the method and standard used. Such differences, however, are expected to be constant. Such possible dif1 ferences are the reasons the standard to be

used must be included in the contract.

7. Laboratory Acceptance

The criteria for acceptance of any lab-

. oratory as reputable is difficult to define

! free of ambiguities. The personnel must be

" trained and experienced. The equipment for

the tests must be available and calibrated,

i Precision and accuracy, both, must be ex-

^ pected. These are obvious and no laboratory

] would be expected to be accepted without such.

, Then what criteria are recognized as estab-

; lishing contractual acceptance of in-house

laboratory procedures when the split sample

program is used?

J 7.1 Representative Sample

The split sample is the sample representing the whole consignment. It is the

| sample taken by the automatic sampler. It

is the sample that is divided without human

choice in its division. It is the proper

J size and amount for the laboratory.

7.2 Sample Preparation

The split sample will be prepared in an

identical manner for each consignment.

7 . 3 Routine Testing

The number of such samples supplied to

i each of the two in-house laboratories each

j year is large. If the average train weight

^ is 10,000 tons, then one hundred samples can

_ be expected for a million ton a year contract.

. Each sample, then, is handled and tested ac- cording to the standards in a routine manner.

7.4 Continuous Checking

As the number of samples in the program

increases, the statistical variations permitted will be predictable. Each laboratory's

J results will serve as a constant check on the

u performance of the other laboratory.

3 7.5 Continuous Cross-check

Few sellers supply coal to but one buyer

and few buyers receive coal from but one

seller. It follows then that there is a constant cross-checking of in-house laboratory

performance and procedures with the split

sample program.

7.6 Criteria for Accreditation

Such is the criteria for contractual

acceptance of in-house laboratory procedures.

It can be considered as volunteer laboratory

accreditation by contractual compliance.

8. Precautions for a Successful

Split Sample Program

8.1 Sampler Agreement

The automatic sampling system's specifications and design should be agreed upon as acceptable by both the coal buyer and the

coal seller. Its operation will meet such

specifications

.

8.2 Handling Agreement

The sample preparation and handling of

the product from the automatic sampler will

be under the supervision of trained personnel

and will be performed according to standards

agreed upon by both the coal supplier and

coal buyer.

8.3 Sample Delivery

The sample will be dispatched to both

laboratories without delay in the most efficient manner.

8.4 Statistical Analysis

A statistical analysis of the comparative results as well as interpretation of

variants must be continuous.

9. Round Robin Testing

The fuel laboratories in the coal industry do not rely on just split samples for

checking. Round robin samples are also used.

These samples are specially prepared from a

single "unknown" sample. Care is taken to

divide the original sample into as many equal

parts as needed by continually mixing and

separating the particals. Because of the

heterogeneous characteristics of coal, the

sample is normally reduced to at least 2.50

millimeter and usually to an even smaller

250 micron size. Such crushing, mixing and

separating is expected to loose some of the

original total moisture, an important parameter in all commercial coal quality evaluations. This loss does not negate the sample

as useful for checking but instead it becomes

exactly that, a sample useful for verifying

the results obtained by routine laboratory

procedures between comparing laboratories.
The value of such checking of routine laboratory procedures decrease, though, if special

handling, not routine handling, is initiated

by the personnel.

Special handling is expected to be used

if a proposed procedural change is being considered or new equipment is being standardized.

A standardized sample is a sample with

known values for such round robin testing.

10. Summary and Conclusions

10.1 Sampling is more than just a basic

adjunct to coal testing.

10.2 A split sample representing the

coal at the moment when the ownership transfers from the supplier to the buyer at the

time of weighing is the only sample that re- presents the true quality purchased.

10.3 A program of continuous analysis

of split samples will serve as a program of

continuous monitoring of procedures between

laboratories. Such a program promotes routine treatment of all samples.

10.4 Round robin samples are essential

for establishing technics and standardizing

equipment but the split sample program is

essential for establishing the quality.

10.5 As the number of such split sample

agreements increases for the same supplier or

for the same buyer, each in-house laboratory

is more recognized for its professional com- petency because of the increase in opportunities for multiple acceptance.

10.6 The value of split sample programs

for contractual acceptance of in-house laboratory procedures is that it shows that such a

program must be recognized as a logical criteria for continuous satisfactory laboratory

accreditation for purpose of commerce.

References

[l] 1978 Book of A.S.T.M. Standards, Ameri can Society for Testing and Materials , Part 26, pp. 314-315.

[2] Ibid., pp. 279-293.

[3] Ibid., pp. 310-326.

[4] Ibid. , Table 1, p. 285

[5] 1972 Book of A.S.T.M. Standards, Ameri can Society for Testing and Materials , Part 19, p. 28.

[6] Mullins, R. A., The Determined Value c Unit Train Coal , presented at American

Chemical Society Coal Seminar,

Charleston, W. Va. October 14, 1977.

[7] Op cit., Ref. 1., p. 189.

4 8
OLD BEN COAL COMPANY

TYPICAL THREE STAGE SAMPLER

-© Loading Belt Conveyor

.0 Primary Sampler

Primary Feeder

Reject

Primary Sample Crusher

Secondary Sampler

©Secondary Feeder

Secondary Sample Crusher

©Tertiary Sampler

Final Sample

Designed to Conform With

Requirements of A.S.T.M.

D-2013 & D-2234

W Return to System

49

SESSION III

HEALTH

SERVICES

ACCREDITATION

PROGRAMS

CHAIRMAN: ROBERT D. MEDZ

U. S, ENVIRONMENTAL PROTECTION AGENCY

*
at

p

B lilt

' *

part

if!l

US
r ational Bureau of Standards Special Publicati on 591 , Proceedings of the National Conference

i'n Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

D, September 25-26, 1979. Issued August 1980

SUMMARY OF EVALUATIONS OF CLINICAL LABORATORIES PARTICIPATING IN THE CENTER FOR DISEASE CONTROL EVALUATION PROGRAM

LOUIS C. LAMOTTE, JR., SC.D.

LICENSURE AND PROFICIENCY TESTING DIVISION

BUREAU OF LABORATORIES

CENTER FOR DISEASE CONTROL

ATLANTA, GEORGIA 30333

During the period from 1969 to 1979, the Center for Disease

Control has provided for the evaluation of about 1,000 interstate

clinical laboratories for licensure purposes, and has provided a performance evaluation program to an additional 1,700 clinical

laboratories that were not required to participate by the CLIA '67 law. Licensed laboratories are required to use personnel who meet

personnel standards, to meet applicable standards for internal

quality control of tests, and to perform satisfactorily in various

proficiency testing programs. In most cases the personnel and

internal quality control were not evaluated for nonlicensed

participants. A review of the data suggests that some laboratories meet or exceed most of the standards, but that a disturbing

number of laboratories do not. The demonstration of specific poor

performance is often a sufficient stimulus to improvement; in some cases regulatory authority seems necessary to bring about improvement. This paper will provide data based on the CDC program.

Key words: Clinical Laboratories Improvement Act of 1967; CLIA '67; Clinical laboratory evaluation; Evaluation, qualifications of laboratory personnel; Evaluation, internal quality control; Evaluation,

performance; Checklist; Proficiency testing; Improvement.

1 . Introduction

From 1969 through 1978, the Center for

isease Control (CDC) conducted a clinical

aboratory evaluation and improvement proram as a part of its responsibility for

icensing laboratories operating in intertate commerce. The 950 interstate

aboratories were expected to meet

tandards for personnel, for internal

uality control systems, and for performnce. Approximately 1,800 additional

aboratories participated as nonlicensed

olunteers in the performance evaluation

art of the CDC program. As of 1978, the

ealth Care Financing Administration (HCFA)

fas given regulatory responsibility for

licensing interstate laboratories as well

as for certifying for payment those laboratories that serve Medicare patients; CDC

continues to evaluate performance of nonlicensed participants. In addition, CDC now is responsible for conducting an independent assessment of the HCFA evaluation system and surrogate systems, offering

advice and constructive criticism where

needed

.

The purpose of this paper is to describe the methods, selected observations,

and impact of the evaluation and improvement program operated by CDC from 1969

through 1978.

53
2. Materials and Methods

There are standards currently applicable for clinical laboratory personnel

which specify minimal qualifications for the

laboratory director, technical supervisor,

general supervisor, and technologists.

These standards indicate education,

experience and training requirements for

each position category. 1 Personnel submit

information along with an annual application for licensure which is verified by an in-house study before a license is issued to the laboratory.

There are also standards currently

applicable for a quality control system for

clinical laboratories. The quality control

system includes procedures covering a range

of items from equipment calibration and

maintenance to validation of methods,

surveillance of results, remedial action for

defects, written procedures manuals,

records, and specimen handling and tracking. 2 A detailed checklist is used by CDC

Examiners to assure that each laboratory's

quality control system is uniformly

examined. The importance of this checklist

in assuring an objective and complete onsite evaluation cannot be over-emphasized.

A copy of one page from a CDC checklist is

shown in Attachment 1

Proficiency test samples for assessing

a laboratory's performance are prepared and

distributed by CDC in the categories of

microbiology, dianostic immunology,

clinical chemistry, pharmacology, and toxicology. ^ Figure 1 shows some of the samples

which are mailed.

Performance of participants is assessed

for summary purposes in relation to the performance of reference laboratories.

Generally, participants in the qualitative

tests are assessed in terms of the percentage that reported the correct identification

of each of the test organisms; in quantitative tests, the parameters provided by the

reference laboratories are used as the

targets for participants. The number of

clinical laboratories that received samples

in 1978 is shown in Table 1.

3. Results and Program Impact

As an example of the impact of the

program Figure 2 shows the progressive

degree of adoption of a particular standard

by 172 laboratories in the CDC program from

1969 through 1976. Note that at the outset

only about half of the laboratories had an acceptable preventive maintenance program;

by 1976 95% had a routine program. Figure

shows the percent of bacteriology labora- L tories that were checking their media in

1969, and the progressive improvement over •(.-: time. Figure 4 shows that the more recent! .- enrolled laboratories as a group adopted J this bacteriology quality control standard . promptly with less necessity for reiteration by CDC examiners.

I ill Another analysis of program impact is', afforded by the data presented in Table 2. V

The percentages of laboratories licensed

in 1971 (the class of 1971) that met

several selected quality control standards

j for each year from 1971 through 1977 are

shown. For comparison, the data for the more recently enrolled class of 1977 are also presented. Obviously, there was some];;

improvement in the degree of adoption of

these quality control standards.

In Figure 5, the performance of all

participants enrolled in bacteriology is

"shown, regardless of when particinating

laboratories came into the program.' The

newest enrollees will have little or no experience with the program, whereas

others will have had the benefit of program stimulation for years. Because

samples in each shipment are not identical[

the challange varies from shipment to shipment. However, the standard test

organisms are recycled regularly so in eve; year participants will have at least sever 1 opportunities to isolate and identify the

most important pathogens. The results indi cate that in most years from 10 to 20% or more of the participants make grades of lei,

than 75 on the shipment consisting of five

or more samples and as a consequence their^ ] efforts often reflect their best performan,

this measurement suggests that efforts to stimulate improvement should be continued. '

An abstract of the qualification

requirements for clinical laboratory

personnel (CLIA '67) can be obtained from ' the author.

2 . )i -t- A summary of the standards for internal quality control (CLIA '67) can be

1

' : obtained from the author.

3 A description of the proficiency tee i* program and shipping schedule can be

obtained from the author.

' Hi;

54
Evidence of improvement in bacteriology

presented in Table 3. Eleven laboratories

hat entered the program in 1972 in bacterilogy were followed through 1978. Only 7 ffered services in gonorrhea diagnosis in

972, and of these, 3 performed satisactorily — a pattern that has persisted

hrough 1978.

The degree of improvement is obviously

elated to how poor performance was when

he improvement stimulus was initially

bplied. Figure 6 shows that laboratories

testing relatively few specimens for

jionorrhea performed the poorest but

'enefited most from the program stimulus.

IP contrast, the high-volume and larger

aboratories performed so well initially that

Tiprovement is much more difficult to emonstrate

.

Laboratory performance in detecting

epatitis B surface antigen in sera is shown

in Table 4. The progressive abandonment of

he less sensitive second-generation test is

indicated; this was stimulated by the proram's demonstration that the test was less

nsitive than the third generation test.

The difference in variances between

ests for rheumatiod factor before and after

ne program provided standardization is shown

n Figure 7. Note that this program effort

suited in a considerable improvement from

very wide variance between laboratories to variance of only 0.93.

t

Program stimuli have a multiplier effect

n most disciplines. For example, demontration of poor performance can be expected

b stimulate individuals to improve their

erformance and to help the staff justify

jbstituting more accurate and precise

rocedures for archaic procedures. A case

n point is the erythrocyte count (Figure 8);

ip laboratorians have improved their individual performance, they have also replaced

ne less reliable procedures with more

bcurate and precise procedures. We

p commend as a guiding principle "If it's

brth doing, it's worth doing well."

The final example of program impact

oncerns the stimulating effect of a notice

o a laboratory that its license is to be

evoked because of poor performance. A

tudy of grades laboratories made in

acteriology for five shipments before and

jive shipments after receiving such notices

nows that their average grades increased

o 8.9 points. Concurrently, the average

rades of a randomly selected sample of

aboratories that did not receive a notice

declined 1.4 points. This suggests that

laboratories that are not motivated by

the simple demonstration of poor performance are motivated by the threat of

license revocation.

55
MATERIALS CFR 74.20(c) 405.1317 (a)(3)

14. The following are labeled (identity, titer Antimicrobials

or concentration, recommended Antisera

storage, preparation or expiration date) Control Organisms Media (tubes, plates kits) Disc

NO (Code) COMMENT

01080

01081 01082

01083

01084

Strips Reagent Solutions U OlOob

01086

15. Materials in use are "in-date", reactive, Antimicrobials and not deteriorated Antisera Control Organisms Media (tubes, plates, kits) Disc

01087 01088

01089 01090

01091

Strips Reagent Solutions 01092

01093 TESTING OF REAGENTS CFR 74.20(a) 405.1317(a)(1) 405.1317(b)(1)

16. The following reagents (in use as individual biochemical tests or as a part of a commercial

system) are tested each day of use to Catalase demonstrate a positive and negative Coagulase

biochemical reaction Ferric Chloride

Indole

Positive Reaction NO (Code)

Negative Reaction NO (Code)

Written Q.C. NO (Code) COMMENT

•01094

•01097

•01100

•01103

•01095

•01098

•01101

•01104

* 01096 •01099

•01102 * 01105

Methyl red

Nitrate Oxidase Voges-Proskauer

•01106

•01109

•01112

•01115

•01107

•01110

•01113

•01116

•01108

•01111

•01114

•01117

17. The following disc and/or strips are tested Bacitracin when each new vial is opened and each week Optochin

of use to demonstrate positive and ONPG

negative biochemical reactions. XV

XV

•01118

•01121

•01124

•01127

•01130

•01133

•01119

•01122

•01125

•01128

•01131

•01134

•01120 • 01123 • 01126 •01129

•01132

•01135

18. The following antisera are tested wnen each E. Coli new vial is opened and each nionih of use to Hemophilus

demonstrate positive and negative agglutina- Herella

tion reactions Salmonella

Shigella Streptococcal

•01136

•01139

•01142

•01145

•01148

•01151

•01137

•01140

•01143

•01146

01149

•01152

•01138

•01141

•01144

•01147

01150

01153

STAINS CFR 74 21(a) 405. 1313(b)( 1 )(,)

19. The following stains are tested each day of c?

PSl

!! e

use to demonstrate expected staining Hagella

characteristics. Methylene Blue Spore

•01154

•01157

•01160

s

0 1 1 63

01156

01159

01162

01165

20. Gram stain is tested when prepared and each week of use to demonstrate

expected staining characteristics (positive and negative reactions) •01166 •01167 01168

21. Each batch of the following media, if used by the lab, is tested for the following characteristics

GROWTH MEDIUM

Support Growth NO (Code)

Sterility No (Code)

Written Q.C. NO (Code) COMMENT

Blood Agar

Brain Heart Infusion Agar/Broth

Chocolate Agar

Heart Infusion Agar

Motility

01169

01172

01175

01178 • 01181

01170

01173

01176 * 01179 01182

01171 01174

01177

01180

01183

Mueller Hinton

Nutrient Agar/Broth

Thioglycollate Broth

Tryptic Soy Agar/Broth Todd Hewitt Broth

• 01184 • 01187 01190

01193 * 01244

01185

01188

01191 01194

01246

01186 • 01189

01192

01195

01247

CDC 3.1005 8-77 BACTERIOLOGY — Page 2 of 4 Pages

ATTACHMENT 1. EXAMPLE OF ONE PAGE OF A CHECKLIST USED BY

CDC EXAMINER DURING ON-SITE LABORATORY EVALUATION

re
FIGURE 1. PROFICIENCY TESTS SAMPLES PREPARED AND DISTRIBUTED BY CDC

ll

NUMBER OF LABORATORIES IN CDC PROFICIENCY

TESTING PROGRAM, BY CATEGORY, 1978

CATEGORY NUMBER OF LABORATORIES MICROBIOLOGY 1,229

Bacteriology 872

Special Gonorrhoeae Testing 353

Mycobacteriology 410

Mycology 438

Parasitology 676

Virology 107

Antimicrobial Susceptibility Test 872

DIAGNOSTIC IMMUNOLOGY 986

Immunology 771

Hepatitis B Antigen 371

Syphilis Serology 552

CHEMISTRY 1,555

Chemistry Profile 1,079

Endocrinology 708

Pharmacology- Digox in 523

Toxicology-Drugs of Abuse 455

Toxicology-Blood Lead 238

HEMATOLOGY 1,187

IMMUNOHEMATOLOGY 700

CYTOPATHOLOGY 475

i TABLE 1 57
DEGREE OF IMPLEMENTATION OF PREVENTIVE MAINTENANCE PROGRAMS

IN 172 CLIA LICENSED LABORATORIES SINCE PROGRAM INCEPTION

100

o

-oco oOCO

o> 70-

c

(/> D

60-

<u n 50-

o

T3 /lA- H-V-' C3

£

30-

c

a> 20-

u 10-

o52.00

67.00

85.00 87.00

81 .00

88.00 92.00

95.00

COMPOSITE OF STANDARDS: Preventive maintenance program

(69)

O

(70)

1234567

(71) (72) (73) (74) (75) (76)

Years after Initial Examination

FIGURE 2

DEGREE OF ADOPTION OF CLIA STANDARD IN BACTERIOLOGY

AMONG 37 LABORATORIES IN PROGRAM FROM ITS INCEPTION

lOOi

43.25 43.25

64.87

54.06

67.57 72.98

83.79 83.79

STANDARD: Each batch of media checked for sterility, ability to support growth, and proper reaction

(69)

O

(70)

1234567

(71) (72) (73) (74) (75) (76)

Years after Initial Examination

FIGURE 3
DEGREE OF ADOPTION OF CLIA STANDARD IN BACTERIOLOGY

AMONG 19 LABORATORIES ENTERING PROGRAM IN 1974

ioo

o

T3C

o

to

u>c

'{/> D

<D

~oc

D£

c

0)

cu Cl

90-

80-

70-

60

50

40

30

20-

10-

78.95

8947

STANDARD: Each batch of media checked for sterility, ability to support growth, and proper reaction

36.84

(74)

0

(75)

12

(76)

Years after Initial Examination

FIGURE 4

DEGREE OF ADOPTION OF QUALITY CONTROL STANDARDS BY CLASS

OF 1971 LABORATORIES AND BY CLASS OF 1977 LABORATORIES

Category

Percent of Class of 1971 labs met standards on date shown

Percent of

Class of 1977

and standard '71 '72 '73 '74 '75 '76 '77 met standards

Bacteriology

Reagents checked 14 18 27 45 64 70 82 43

Autoclave checked 43 64 91 82 100 90 100 100

indate reagents used 29 55 64 91 82 100 100 100

Serology

Tests controlled 33 60 64 64 70 90 91 57

Chemistry

Controls, pregnancy test 38 83 100 92 91 91 92 100

Immunohematology

Serum, cells checked 50 45 82 73 100 100 100 83

TABLE 2

59
Percent of Laboratories Making Grades of Less Than 75,

BACTERIOLOGY

NO. OF LABS: (149-736)

IV I II III IV IV I IV I IV I II III IV I II IV I II III IV I II III IV 1970 1971 1972 1973 1974 1975 1976 1977 1978 Means(x) of Grades of Upper 20% of Laboratories

i i i i i i i i i i i i i i i i

i i i i i i i i i i i i i i i

i i i i i 10099 98 98 10010099 100 100 100 100 100 10099 10097 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100

FIGURE 5

SUCCESS OF CLASS OF 1971 LABORATORIES COMPARED TO CLASS OF

1977 LABORATORIES IN ISOLATING AND IDENTIFYING

NEISSERIA GONORRHOEAE FROM MIXED CULTURES

Performance of Laboratory Performance of 1971 labs on date shown Lab 1977 labs in Number '72 '73 '75 '76 '76 '77 '78 Number '78 04-60 NE* S S S S S S 01-26 S 12-31 NE U S NP s S S 04-40 NE

13-13 NE S s S s s S 04-70 U 14-05 NE u s S s s u 10-30 S 20-36 S s s S s s s 14-25 S 22-14 u u s s s s s 15-24 S 29-07 u u s s s s s 19-68 S 32-12 NP NP s S s s s 26-16 S 40-04 s s s s s s s 29-66 NE

41-17 s u u s s s s 35-07 U 45-16 u u s s s s s 35-21 S 36-40 S 37-94 S

Percent S 43 36 91 91 100 100 ~9T 82

*NE, not enrolled for gonorrhea; S, satisfactory; U, unsatisfactory; NP, did not

return results

TABLE 3

60
PERFORMANCE OF LABORATORIES RELATED TO

DAILY TESTING LOAD- NEISSERIA GONORRHOEAE

Survey 1976 1 19771 1977 H 19781

No. Test Samples 2 1 3 2

Percent of Laboratories Correctly Reporting

N. gonorrhoeae, By Shipment

FIGURE 6

SENSITIVITY OF SECOND AND THIRD GENERATION

TESTS FOR DETECTING HEPATITIS B SURFACE

ANTIGEN POSITIVE SERA

Year Results on Proficiency Testing Samples

2nd generation tests 3rd generation tests # Labs % Correct # Labs % Correct

1975 114 79 206 99

1976 90 60 238 99

53 62 251 95

1977 48 66 274 98

48 80 273 99

1978 27 97 281 99

33 79 276 96

1979 5 60 322 99

Average Percent Correct 72 98

TABLE 4

61
DISTRIBUTION OF RHEUMATOID FACTOR

TEST RESULTS

COO

CL cr

140

120

100 80

60

40

20

0

a>E

ZJ

280

260

240

220

200

180

160

140 120

100 80

60

40

20

0

Nonstandardized

Variance =4.21

t128^64 f32 -16 *8 -4 t2 x2 x4 x8 xl6

Actual Standardized Values

Variance = 0.93

-4 ^2 x2 x4 x8

Dilutions Expressed as Multiples or Fractions

of The Median Value

figure 7

Percent of Laboratories with Results Outside

Limits of ± .5 x 10l2/l around Target Value

Red Blood Cell Count

NO. OF LABS : ( 129- 640)

ii iii iv, ,i ii lii iv,, I ii lii iv.,

i 1 ii 1 ii 1 ii— II III IV.. I II' 1970 1971 1972 1973 1974 1975 1976 1977 1978

FIGURE 8

62
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

pn Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

[fl), September 25-26, 1979. Issued August 1980

EVALUATION AND APPROVAL OF LABORATORIES IN CONNECTICUT

IMPORTANCE OF VOLUNTARY STANDARDS

JESSE S. TUCKER, PH.D.

LABORATORY DIVISION

CONNECTICUT DEPARTMENT OF HEALTH SERVICES

HARTFORD, CONNECTICUT 06101

The Laboratory Division of the Connecticut Department of Health

Services is responsible for licensure, registration, performance evaluation and approval of hospital, indpendent clinical, water, waste

water, food, dairy, air and a variety of other laboratories which perform tests of public health significance. Of the almost 400 facilities,

more than three-quarters of these participate in proficiency testing

and are inspected yearly.

The philosophy of performance evaluation and approval in Connecticut has evolved from one of complete authority and control by the

state to one which is based increasingly upon cost effectiveness, consistency, enforceability and participant input. Dependence on voluntary

standards now extends from granting equivalence for inspections conducted

by qualified professional groups to accepting proficiency testing results from well conceived programs. Most success in acceptance of

equivalent programs has been attained with clinical laboratories and

least progress has been made with environmental laboratories. This

difference probably is attributable to the newness of environmental as opposed to clinical laboratory legislation and the availability of acceptable programs.

Using the evaluation and approval of recombinant DNA laboratories

in Connecticut as an example, the advantages of using existing regulations and voluntary standards are discussed as a model of efficiency.

Key words: Clinical laboratories; cost effectiveness; environmental

laboratories; licensure; performance evaluation; physician's office

laboratories; proficiency testing; public health; recombinant DNA; registration and approval; voluntary standards.

1. Introduction

Responsibilities of State Agencies for

regulation of laboratories are increasing

more rapidly than the funds available to fulfill these responsibilities. An innovative

approach is needed to assure quality of laboratory results without excess expenditure

of taxpayers funds.

After almost ten years of a traditional

6

regulatory program in which state resources were required to maintain complete responsibility for inspection and proficiency testing

programs, the Connecticut Health Laboratory

has initiated a system involving voluntary

standards and participation by non-governmental organization in the inspection

and proficiency testing process. Specifics

of the Connecticut program including advantages, limitations and the need for

continued review and change are covered in

this report.
2. Scope

In Connecticut, the Laboratory Division

of the Department of Health Services is res- ponsible for licensure, registration, inspection, proficiency testing, and approval

of all laboratories performing tests of

public health significance on or for Connecticut citizens. Included are hospital and

independent laboratories, water, waste

water, diary, food and air testing laboratories, blood banks and plasmapheresis

centers. Recombinant DNA laboratories,

academic laboratories and laboratories using

dogs for teaching and research are also included. The Department of Health Services

has primary enforcement responsibility for

approval of laboratories included under Medicare, the Clinical Laboratories Improvement

Act (CLIA), the Safe Drinking Water Act

(SDWA) and National Polution Discharge Elimination Systems (NPDES) legislation. About

400 separate facilities, as well as the

proficiency testing specimens required to determine quality of the work done by them, are the responsibility of ten professional

and four non-professional employees. These

regulatory responsibilities cannot be

discharged effectively without at least

tacit acceptance of the basic principles of

voluntary standards.

3. Clinical Laboratory Regulation

At the request of the Connecticut Society of Pathologists and members of the

State Advisory Committee on Clinical Laboratory Regulation the Laboratory Division

began looking at alternatives to state

sponsored inspection and proficiency testing

programs. Hospital and independent clinical

laboratories had long been subjected to a multiplicity of programs required by federal,

state and peer review groups. After carefully reviewing the inspection and accreditation program of the American Association of

Blood Banks (AABB) an agreement for alternate

year inspections by the state and AABB was

concluded in November of 1977. The most

significant reasons for this decision were:

1. The AABB program was more thorough

than ours.

2. Funding was not available to provide

an equivalent program.

A somewhat different agreement was reached with the College of American Pathologists (CAP) in December of 1978. In essence,

the State of Connecticut accepted the biennial inspection of member hospital laboratories in lieu of the yearly state inspection. However, CAP agreed to a 10%

64

reinspection by the state. About one half

of the 50 hospitals are accredited by CAP

leaving the state with about twenty five

annual surveys and reinspections . It is

expected that some agreement soon will be

worked out with the Joint Commission on th(

Accreditation of Hospitals (JCAH) because i the recent "deemed status" granted to this

organization by the federal government to approve hospital laboratories for Medicare

purposes. It was not altruism which prompt

this arrangement . Rather , it was the increasing realization that federal funding

would not always be available, that regula-I

tion is not one of the more popular items

competing for state funds, that these inst:

tutions are providing quality laboratory

services as evidenced by eight years of

acceptable proficiency testing results and

that redundancy in inspection was creating more problems than it was solving.

Much of the impetus for statutory chai

(which in January 1,1980 will enable accepl ance of proficiency testing programs other

than those provided by the state) originate

from the sub-committee for Microbiology whi

functions as part of our mandated Advisory

Committee for Clinical Laboratory regulatic

This advisory committee includes peer revic

and has provided a perspective unavailable

to many regulatory agencies. Input from hos

tal and independent laboratory microbiologists has resulted in an appreciation of tl differences in problems encountered in dif -j- ferent settings as well as common problems

resulting from inspection and proficiency

testing. The disadvantages usually expects

from this "fox in the chicken coop" approac

have not materialized.

Another problem area which was improve"

by a change in statute was licensure of

physician's office laboratories. Previousl

a combined practice of two or more practitioners required licensure. Only a single

physician who performed tests in the course

of his practice was exempted. This law was unreasonable and unenforceable. The new

law establishes a licensure requirement for

practitioners which is based on volume and 1 complexity of testing as defined in regulations. We feel that this is a more logical

and economical approach than requiring

licensure for all practitioners performing

tests or basing an exemption on the number

of practitioners involved such as the

arbitrary five limit suggested by a recent 1 version of the proposed CLIA.

Peer review is not without problems.

Most peer reviewers are not thorough enough

in covering details. This thoroughness is
criticized frequently as nit-picking.

Any professional has available to

hiai textbooks, journals, seminars and

continuing education courses to satisfy his

i( intellectual requirements. However, he does

not always have and needs periodic scrutiny

to make certain that he and his employees

are properly taking and labeling specimens,

5 updating specifications, recording information necessary to achieve adequate quality

control, protecting themselves from biological and chemical danger and accurately rej porting results. Some of the most astute

laboratory scientists are remiss at keeping

records of their own work and helpless at requiring others to do so. There is no

i better way to be assured that records will' be

kept properly than to convince the laboratory

director or supervisor that his errors and

omissions will be discovered.

Additional problems in peer review are created by differences in the frequency of

inspections. An unfair burden is placed on independent laboratories which are required

to be inspected yearly under the same Medicare regulations which allow on-site inspection by JCAH and CAP, biennially. Inequities

also exist in the criteria used during inspection. A classic example with which our

group has been wrestling for some time is the

Medicare requirement for use of quality

control organisms on a daily basis for antibiotic susceptibility testing CI] which is

less stringent for CAP approved laboratories 121 .

Application of the term "voluntary" to proficiency testing and laboratory inspection

has resulted in some differences in interpretation. The pathologist who participates

in the CAP proficiency testing program may

make the interpretation that he participates

in the program of his own volition, that he

can withdraw at any time and that there are

no penalties if his performance is not within the prescribed limits. However, when a governmental agency accepts such a program

to meet requirements for regulatory purposes

the only voluntary thing about it is that

he has chosen this route rather than be

subjected to the alternative state program.

It is incumbent upon the agency granting

equivalent status to differentiate between

the good and poor performer and to use this

information to require improvement or to prevent any damage from poor results. Input

and compliance are the keys to voluntary

standards. Particpants must have the

opportunity not only to criticize the standards currently in use and to make suggestions for change but also to make compromises

and to adhere to the standards once adopted.

A cost effective program for determining

quality of laboratory performance will be

forthcoming when proficiency tests can be

conducted without prior warning and when the

correct response has been established well

in advance of the actual test. Several years

ago when this suggestion was made to Center

for Disease Control by our group it was con- sidered to be impractical. However, in the

recent Public Health Service publication

"Proposed Interim Proficiency Testing Program for Medicare Laboratories" [3] methods

for reference laboratories were recommended

for the chemistry program. It is expected

that additional definitive and reference

methods [4] will be established by groups

such as The National Committee for Clinical

Laboratory Standards (NCCLS)

.

At the Connecticut State Health Laboratory it is our opinion that a better

appraisal of laboratory performance in

clinical chemistry can be obtained by two unannounced proficiency tests than by the

six announced tests currently recommended in

the above reference by the Public Health

Service. Three specimens, two unknowns

(normal and abnormal) and one specimen with

known values would be used. All specimens

would be assayed using reference methods.

Statistically significant target values and

acceptable ranges would be established in

advance. After a suitable period to allow

all laboratories to adjust their procedures

to obtain values comparable to those from

the reference methods, the specimen with

assigned values would be discontinued. A

similar approach could be used for categories

other than chemistry.

After all the time and effort that has

been directed to performance evaluation

through proficiency testing, a serious problem still remains. Little progress has

been made in development of a system to determine integrity of the specimens. Until

such a system is developed, no assessment

can be made of the significance of specimen

handling (including methods for obtaining,

processing and transporting the specimen) on quality of laboratory performance.

4. Environmental and Public Health

Laboratories

The most complex regulatory problems now exist in the environmental area. Some

of the precise methodological requirements

of the Safe Drinking Water Act and the Water

Pollution Control Act are difficult to comprehend. Good proficiency test specimens

for volatiles and chlorinated hydrocarbons

are hard to find and we now prepare them in65
house. We have not yet had the opportunity to

to consider the problems involved in determining quality of air analysis. Even the

variety of analyses performed in different

laboratories is surprising. For example,

there are 19 laboratories in Connecticut

approved to perform tests required under the

Safe Drinking Water Act, ten approved under

NPDES and 48 approved for both. Of the 48

having combined approval, 8 of these are approved to do food and one of these also

performs tests for pollutants in air. Four

laboratories are approved to perform tests

only on domestic waste water. Eight laboratories in Connecticut are approved to perform

tests on milk as required by State and Interstate Milk Shippers regulations. Most of

the requirements for inspection and proficiency testing of laboratories in these

specialties currently are prescribed with

very little leeway for application of voluntary standards.

5. Recombinant DNA Laboratories

Laboratories which lend themselves best

to voluntary standards are the academic,

quality control and Recombinant DNA Laboratories. They require registration because of

their use of organisms of public health significance. Recombinant DNA laboratories fit

into this category because E. coli is included in the public health code as one such

organism. They are registered under existing regulations because those involved

finally agreed that this was a lesser evil

than a whole set of new state statutes and

regulations which could differ substantially

from existing federal requirements. The

only state requirement is that these facilities adhere strictly to NIH guidelines,

whether or not their work is federally

supported, and that they keep the state informed of changes in projects and advisory

committee members. They have been inspected

yearly and found not only to obey the guidelines but also to be an excellent source of

information on laboratory safety. This model

of simplicity in laboratory regulation is also

a model for the application of voluntary

standards in an area which the public considers to be critical for protection of their

health and safety.

1 am grateful to John Redys, Director of the

Laboratory, and members of the regulatory

staff for critically reading the manuscript

and to Phyllis Botticello for assisting in its

preparation.

References

[13 Clinical Laboratory Survey Report - Fo HCFA - 1557 (1-78), U.S. Department of

Health, Education and Welfare, Health

Care Financing Administration, p. 8.

[21 College of American Pathologists, Commi

sion on Inspection and Accreditation,

Inspection Checklist , Section IV , Micro

biology , Revision 1976, p. 10-

C3] Proposed Interim Proficiency Testing

Program for Medical Laboratories , U.S.

Department of Health Education and Welfare, Public Health Service, Center for

Disease Control, Atlanta, Georgia, Apri

1978, p. 38.

14] Reference Procedure for the Quantitativ

Determination of Hemoglobin in Blood , NCCLS Proposed Standard: PSH-15, The

National Committee for Clinical Laboratory Standards. Villanova, Pennsylviana

1979.

66
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

THE LABORATORY ACCREDITATION PROGRAM OF THE AMERICAN INDUSTRIAL HYGIENE ASSOCIATION

FREDERICK I. GRUNDER

SENIOR ENVIRONMENTAL HEALTH CHEMIST, BETHLEHEM STEEL & CHAIRMAN

AMERICAN INDUSTRIAL HYGIENE ASSOCIATION LABORATORY ACCREDITATION COMMITTEE

The accreditation program for laboratories will be described along

with the program to assist accredited laboratories to maintain their

accreditation through continued demonstration of competence and performance. An Industrial Hygiene Laboratory is a laboratory that analyzes

samples taken to assist in the recognition, evaluation and control of

the workplace environment. The factors which are used to determine

whether a laboratory qualifies for accreditation as an industrial

hygiene laboratory will be described in detail. Such factors include

personnel qualifications, quality control and proficiency testing,

facilities, recordkeeping, analyses performed, safety and analytical

procedures. A description of the accreditation process will be provided along with steps that are taken to assist laboratories not maintaining acceptable standards of performance. The role of the coordinator, the site visitor, the accreditation committee and the AIHA

Board of Directors will be outlined and possible changes in the program wi] 1 be discussed. The effectiveness of the program in improving

industrial hygiene laboratories will be reviewed.

Key words: Accreditation; Accreditation Committee; American Industrial

Hygiene Association; coordinator; industrial hygiene laboratories;

laboratory; PAT; performance; personnel; proficient; quality control;

reaccreditation; site visit

The first step in discussing the laboratory accreditation program of the

American Industrial Hygiene Association is to define the term industrial hygiene and

briefly describe an industrial hygiene laboratory. Industrial hygiene is that science

and art devoted to the recognition, evaluation and control of those environmental

factors or stresses, arising in or from the

workplace, which may cause sickness, impaired health and well being, or significant discomfort and inefficiency among workplace employees.

The industrial hygienist depends on the

industrial hygiene laboratory, a laboratory

that analyzes samples taken to assist in the

recognition, evaluation and control of the

workplace environment for data. The

laboratory is critical for determining

existing potential health hazards and

measuring the effectiveness of control measures implemented in previously identified problem areas. The laboratory data is interpreted by the industrial hygienist who

must then relate the findings to standards

set by the Occupational Safety and Health

Administration (OSHA) - with advice from the

National Institute for Occupational Safety

and Health (NIOSH) . It is important, therefore, that the laboratory provide data that can be trusted.

An examination of these definitions

shows a definite need for some method of

insuring the validity of the data such as accreditation of the laboratories.

Accreditation means professionalism to those

67
laboratories involved in the program. It is a mark of competence demonstrating that the

laboratory measures up to the strictest

possible standards. The sole aim of the

American Industrial Hygiene Association

accreditation program is to assist industrial

hygiene laboratories in achieving and

maintaining the highest possible level of

professional performance.

If a laboratory is not accredited,

there is no assurance that the data or information that it generates is reliable.

The accreditation program has helped laboratories find areas where improvement was needed and has assisted laboratories in

development of quality control programs

that insure valid data. All laboratories

participating in the accreditation program

will probably agree that the program has

been beneficial. Accreditation is a statement to both the public and to workplace

employees who are affected by the results

that a laboratory has met certain standards

set by recognized professionals in the field.

A laboratory accreditation program is and

will continue to be an effective tool in

the continued improvement in the perfor- mance of the industrial hygiene laboratory.

How does the accreditation process

work?

Application

Requested

.Interactions

Enroll

InPAT

, Coordinator

Review

Subcommitte

Review

, Site

Visit

, Committee

Review

• Additional

Information

Requested

Additional

Information

Requested

Recommendations

Additional

Information

Reques ted- Ac t ion

Figure 1

Board

Approval

Flow Chart for Laboratory

Accreditation Process

The flow chart in Figure 1 shows the variot

steps in the process leading to accreditation along with those places where assistance can be provided. First is the reques

for the application which is filled out am

sent to the laboratory coordinator at the ;£ American Industrial Hygiene headquarters ii Akron, Ohio. The coordinator who is employed by the Association reviews the

application, mainly for completeness and

then sends the application to two members

of the Laboratory Accreditation Committee

for a thorough review. The Laboratory

Accreditation Committee is a committee of | qualified volunteers who are appointed

annually by the AIHA Board of Directors

The laboratory is simultaneously enrolled

in the NIOSH Proficiency Analytical Testing program, better known as PAT. This

program requires bi-monthly evaluation of

supplied contaminants. Satisfactory performance judged against the results of all

participating laboratories is required to (ftk become accredited and to remain accredited

The first aspect of the application

which is considered important are the

qualifications of personnel. The director

of the laboratory must be a Diplomate of

the American Board of Industrial Hygiene ot have six years experience in industrial

hygiene chemistry after completing a medics

degree or an academic degree in one of the

basic sciences. The committee must judge

whether or not the applicant meets the six

years experience standard. The entire work

experience of the individual is considered jj to make certain that the laboratory is unde: the control of trained, competent, practic-j

ing industrial hygiene chemists.

This requirement is noteworthy for the

following reasons. Historically there has

been good corelation between the ease with

which a new laboratory has met this ex- perience criteria and measurements of performance subsequently achieved and maintained. The experience requirement has

often been a big hurdle for many laboratories - particularly those where the

industrial hygiene function has been a non-: specific entity of a larger general

chemistry laboratory.

The laboratory supervisor should have a; doctorate degree in a related science or a medical degree with an additional two years

experience in related procedures. Otherwise, he should have a master's degree plus

four years experience or a baccalaureate

degree and six years experience in related

procedures. There are also requirements

spelled out for the technologists and

68
j: technicians . The titles often do not match

•those used internally within the various

•organizations seeking accreditation, howi: ever, the program is more concerned with the

function of the individuals rather than with

the title.

The area of personnel qualifications

most often raises questions such that the

laboratory may have to provide additional

detailed information prior to review. The

subcommittee review may also suggest that someone else may be more appropriate as director until specific requirements are attained by the named individual. At the

'time of the site visit, suggestions again

fmay be made with regard to improving

qualifications and helping the individuals

'to become more familiar with industrial

health. Suggestions which are made are not

binding on management but are offered so that personnel structures will compare with

those of other accredited laboratories.

J Closely related to the importance of

having trained industrial hygiene chemists

jis the decision to accredit only laboratories performing a significant amount of

industrial hygiene work. This sounds

straightforward but is difficult to administer in practice. Many laboratories

are involved with water chemistry, air

pollution, solid waste or toxicology.

There are also laboratories that want to be

accredited under the program that do not

i have any identifiable industrial hygiene

I program. To measure the involvement of a

i laboratory in industrial hygiene the

application requires a statement of work

load in terms of origin and purpose

encompassing the past year's testing.

A second area of importance is in re- gard to the laboratory quality control program. In committee review, the quality

control program is given equal weight with

I the qualification of the personnel. If a laboratory does not have a good quality

control program, it will be very difficult

for any laboratory to obtain accreditation.

The requirements include a statement of

policy with regard to quality control and a manual which contains all quality control

work. Preferably, someone should be

designated as the laboratory quality control

coordinator, although for smaller laboratories this individual may serve in a multiple capacity. Control charts should be

i in use for those analyses most commonly

encountered and spiked samples should be

used for infrequently performed analyses.

The quality control policy should state

what action is taken when a result is found

outside the expected range. Participation

in other proficiency testing programs and

exchange of samples with other accredited

industrial hygiene laboratories is encouraged.

The application also requires a detailed

description of other factors:

1. Facilities

2. Record keeping system

3. Type and amount of analysis performed

4. Safety program

5. Analytical procedures

The act of answering the queries is,

itself, beneficial. Many interested laboratories, in an effort to respond, have recognized certain benefical changes which subsequently have been instituted. In a few

instances, applications have been delayed

until changes have been made. As stated

before, the purpose of the program is to assist laboratories in achieving and

maintaining a high level of professional

performance

.

After an application has been reviewed

and accepted by the subcommittee, and a laboratory has completed two PAT rounds

successfully, an on-site visit is scheduled

to verify the material presented and to examine the operation of the laboratory.

The site visitor is an expert in the field

of industrial hygiene and his objective is simply to offer constructive advice.

The site visitation is an important

aspect of the accreditation process. Site

visitors are selected based on their expertise in laboratory operations. Site visits,

which are scheduled at mutually convenient

dates, are usually conducted and completed

in one day. If deficiencies are noted,

recommendations are made to the laboratory

regarding changes which should be made.

Information from the application review,

site visit report and performance in the PAT program are submitted for final committee approval. The committee may request

additional information or make further

recommendations to the applicant. In some instances, a second site visit may be

required to insure that major recommendations have been carried out. After approval

by the committee, summary sheets are submitted for approval by the Board of

Directors of the American Industrial Hygiene

Association

.

Accreditation is granted for a period of

69
three years. During this period, a laboratory is required to participate in the PAT program and advise the committee on an annual basis of any major changes that occur.

A current list of accredited laboratories

appears in the Journal of the American

Industrial Hygiene Association.

The reaccreditation process operates in

a similar fashion except for a certain

simplification of the procedure. The re- accreditation application basically

requests information on the changes and improvements that have occurred over the

three year period. The subcommittee review

is not needed and the site visit is con- ducted with all the accumulated information

over the three year period, including

previous site visitor's and committee

recommendations, performance in the PAT program, changes in personnel and any other

changes of significance that may affect the

performance of the laboratory. The committee

then reviews the entire package and passes

its recommendation on to the AIHA Board of

Directors

As has been stated the purpose of the

program is to assist laboratories in upgrading performance. The first place for

providing assistance is during the co- ordinator review of the application. The

coordinator's job is to insure that the

application is filled out completely and

correctly. He may recommend greater emphasis in some areas and less in others not

only to assist the laboratory but to help

the committee better understand the nature

of a particular laboratory operation. He serves as the communicator, relaying re- commendations of both the committee and

site visitors and notifying the laboratory

when a final decision is made.

The subcommittee review points out areas where more information may be needed

and suggests potential trouble spots that

the site visitor may need to discuss with

the laboratory.

The site visitor prepares a written

report with recommendations for improvement which is part of the total package for

committee review. These recommendations

must be answered with specific statements

of action that will be taken by the laboratory.

The results of the PAT program are used to determine a laboratory's proficiency for those analyses performed.

Extra sets of samples are available for

laboratories having difficulties. Also,

exchanging samples with other laboratories

is suggested as a help in resolving difficulties .

A meeting of laboratory directors is

held annually at which time problems and

progress are discussed. Also, a quarterly

newsletter is published for information

exchange among laboratories. The newslette

carries items of interest submitted by

laboratories, new products available, and

summary information on PAT data.

Although the program has been in

operation for over six years changes are under investigation to insure continued

improvement. Most of the laboratories that

would be classified as industrial hygiene

laboratories are accredited. The committee

must now deal with what it considers border

line laboratories to insure that these are not excluded simply because they are not

strictly industrial hygiene laboratories.

Other changes or items under committee

review are:

1. The definition of proficient and non- proficient to conform with the definition used by NIOSH.

2. The requirements for laboratory director and supervisor which would leave

no questions regarding who would or would not qualify.

3. Additional procedures for assisting

laboratories that are having difficulties with the PAT program to insure

proficient analyses not only for those

analytes in the PAT program but for al~. analytes

4. The possibility of an interim accredits

tion for those laboratories that may nc exactly meet the requirements but whic?

obviously have analytical capability.

Response to the accreditation program

has been excellent. The list of about one hundred thirty (130) accredited laboratories

is still growing with no sign of decrease if

the rate of growth.

In summary, most laboratories agree the accreditation program has been beneficial tc

them. It is a statement to both the public ( and employees that a laboratory has met

standards as set by recognized professionals

in the field. The laboratory accreditation

program is and will continue to be an effective tool for the improvement in performance of the occupational health laboratory.

Kati

on T

7 0
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

COLLEGE OF AMERICAN PATHOLOGISTS

INSPECTION AND ACCREDITATION PROGRAM

FRANCES RYAN

COLLEGE OF AMERICAN PATHOLOGISTS

The College of American Pathologists Inspection and Accreditation

Program is a professional, voluntary, peer-review program of laboratory

improvement. The standards are developed by a Commission of volunteer

pathologists and the program is operated by this Commission, with support from the College's central office and computer center. Biennial

inspections are conducted by volunteer pathologists. Each pathologist

is assisted by a team of his choosing, representative of the size and

complexity of the laboratory to be inspected. The standards include

requirements on quality control, safety, environment, instrument maintenance and personnel. An inspection checklist is used to assure that

all areas are examined. Proficiency testing is a prerequisite and

results are an integral part of the accreditation decision. Emphasis

is placed on the educational aspects of the program. CAP accreditation

is accepted by the Joint Commission on Accreditation of Hospitals, the

Center for Disease Control, and some states. Accreditation by CAP

assures the physician and patient that test results are reliable.

Key words: Accreditation; College of American Pathologists; criteria;

inspection; inspector's manual; pathology; proficiency testing;

standards

.

1. Description

The College of American Pathologists

Inspection and Accreditation Program was developed almost 20 years ago with the primary objective of improving the quality of

clinical laboratory services and assuring the

accuracy and reliability of test results.

Although the program has grown in complexity

and effectiveness, the goal still remains one

of laboratory improvement through peer-review

and education. The College of American

Pathologists Inspection and Accreditation

Program is a professional, voluntary, peerreview program with about 1800 laboratories

currently participating. This represents

more than 10 percent of the labs in the

country.

2. Development and Operation

The program is under the direction of a Commission on Inspection and Accreditation;

a group comprised of a chairman and ten

regional commissioners. Each commissioner

is a highly qualified pathologist who has

been appointed by the President of the

College. The commission meets at least

quarterly to review its current standards

and to examine new or problem areas that

may exist. Each regional commissioner is

assisted by state commissioners in carrying

out the operations within individual states. The state commissioner's duties may vary

according to the responsibilities delegated

by the regional commissioner, but in most

cases, his major responsibility is to assist

in the recruiting, training and assigning of

inspectors

.

71
Each inspector is a board-certified

pathologist who has attended a recent inspector's workshop or received on-the-job training during an inspection with an experienced

inspector. Each inspector is assisted by a team of specialists, chosen by the pathologist-inspector, based on the size and complexity of the lab which they are assigned to inspect. For example, in a two or three

hundred bed community hospital, the inspector

might select a chemist, a chief technologist

and possibly a supervisory technologist. In

a university setting the team may include

several other pathologists and a number of

doctoral level assistants. The team is usually chosen from within the inspector's own

institution, but this is only because of the

logistics involved in recruiting inspectors.

We are currently developing a computer program to provide a data base on inspectors,

and hopefully, this will make it easier for

an inspector to recruit assistants from other

laboratories in his or her area.

The commissioners and inspectors, through

daily contact with the many laboratories in

the program, are continuously aware of the

expanding technology and changing needs within

the practice of laboratory medicine. Inquiries are addressed to the College from laboratories all over the country, questioning

the scientific relevance of certain requirements, or seeking advice on acceptable methods for meeting the requirements. All of

these inquiries are reviewed by the commission and are often referred to one of the

College's resource committees. The resource

committees are comprised of pathologists with

expertise in special areas such as toxicology,

parasitology and so on. The resource committees review the questions referred by the

commission and offer their expert knowledge

in resolving these issues. In many cases,

the questions raised by the participants or inspectors point up outdated aspects of the

program, which are then promptly changed.

Such input also adds to the further development of the program by calling attention to

details that might ordinarily be overlooked.

3. Standards and Criteria for

Accreditation

The standards required to achieve

accreditation are very comprehensive,

addressing all aspects of quality control,

such as methodology, reagents, control

media, equipment, proficiency testing, speci- men handling, and reports. They also include

requirements on safety, instrument maintenance, personnel, laboratory environment and

facilities. In order to assure that all of

these requirements are met uniformly in each

laboratory, the commission has developed

Inspection Checklists containing more than

1700 questions on all areas of the lab.

Today's checklists are computerized to alio

for constant updating, as the state-of-the-|:

art continues to expand. These checklists

serve as an aide to the inspector, since

there are so many requirements, and it is

very possible to overlook something without

such a guide. However, in a peer-review pr

gram, the real emphasis is on the professional judgement of the inspector. Through

his knowledge of the I&A Standards and his

experience as a pathologist, the inspector

evaluates, not only the routine systems of

quality control within the laboratory, but Jk

also the effectiveness of various approache,

the underlying attitudes and the management

principles which determine how well the lab

oratory is serving the patient.

k. Proficiency Testing

One of the major requirements for

achieving and maintaining accreditation, is

the successful participation in an interlab

oratory comparison program. Laboratories i the College's ISA. Program are required to participate in the proficiency testing programs offered by the College. The College

refers to its proficiency testing programs

as Survey Programs (not to be confused with

inspection). The College Survey Program is

the most widely accepted program of its kin

with more than 9000 laboratories enrolled,

providing the largest data base for interlaboratory comparison in the world. At the

time of the on-site inspection, the proficiency test results are reviewed by the

inspector. The regional commissioner again

reviews the results as part of the overall

review of the inspection report. All unacceptable results (those outside the limits

established by the comparison) must show

evidence of review and corrective action.

Once the laboratory is accredited, a semiannual summary of survey results is generated by the computer and sent to the lab.

Copies of this report are also sent to the

regional commissioner for review. This assures that an accredited laboratory is

continuously being monitored for proficienc;

in specimen identification.

5. Education

Continuing education through peerreview and professional consultation is an integral part of a program such as the

College's I&A Program. The inspector and

members of the team bring with them the

knowledge and experience they have gained

through past inspections, as well as their

! the

72
ywn professional experiences. This provides

-he setting for an exchange of knowledge that

„s beneficial for both the laboratory being

>->eviewed and those performing the review.

•"In recognition of this fact, the American

fedical Association has authorized the issuance of continuing medical education credit

to pathologists practicing in CAP accredited

Labs and to each pathologist who inspects

?Tor the College. One of the best examples

jf the educational process involved is seen

| La the summation conference. During this

conference the inspection team meets with

.she laboratory director and the staff to go over the findings of the inspection. The

Inspector's Manual [l] that is provided to

8.; he inspector each time he agrees to perform

an inspection, states that "The summation

Conference is an appropriate time to present

she objectives of the inspection and accreditation program and discuss improvement of

'she laboratory in educational terms."

6. Government and Professionel

Relations

There are many programs , both governmental and professional, that conduct inspections in the lab. For many years, the College has worked to eliminate duplicate

inspections through close liaison with other

ijgroups. The College has maintained an equivalency with the Clinical Laboratory Improvement Act of 1967 since its implementation.

This equivalency enables a laboratory accredited by the College to apply for an exemption

sfrom licensure and to substitute its College

inspection in lieu of federal inspection.

|;The College also had formal agreement with

'three states, Georgia, Connecticut, and

(Tennessee, to substitute CAP inspection for

jstate inspection and negotiations are cur- rently underway in several other states. In addition, the College has recently entered

into an agreement with the Joint Commission

on Accreditation of Hospitals, whereby hos-

[pital based laboratories accredited by CAP

are not inspected by JCAH. The College also

maintains liaison with many of the specialty

societies within the laboratory field, such

as the American Association of Blood Banks

and the American Association of Clinical

Chemists

.

7. Inspection Process

A laboratory achieves accreditation

from the College through the following steps:

First they phone or write our office to

request an application. We send the lab a packet of material consisting of a Long

[Form Questionnaire, the Inspection Checklists, which I showed you earlier, a booklet

of Standards and a covering letter instructing the lab on how to complete the information required. The Long Form Questionnaire

contains questions on test volume, instrumentation, methodology, and quality control.

We also ask that the lab provide a personnel

form on all professional and supervisory

personnel, a sketch of the laboratory and

copies of proficiency test results for the

past year. This data provides information

to the inspector before his visit, which

indicates what will be involved in the

inspection. In this way he can also select

an appropriate team. It also supplies information to our office which is kept in a permanent file

.

The checklists give the lab a chance to review the requirements in each section before submitting the forms for inspection.

Since the standards are stringent, it often

takes several months for the laboratory preparing for its first inspection to institute

all of the quality control measures that are required for accreditation. Procedure manuals need to be updated, documentation re- viewed, and checks made for health records,

safety manuals and maintenance records.

Often the lab preparing for its initial

inspection will send a representative to

one of the Laboratory Improvement Seminars

that are presented throughout the year by

the commission.

Once the completed forms are received

in our office, a notice is sent to the com- missioner informing him that the lab is

ready for inspection. The commissioner

then appoints an inspector to visit the

laboratory and notifies us so that we can

send the data on the lab to the inspector.

The inspector reviews the data and contacts

the lab to arrange for an inspection date.

All inspection visits are announced. The

inspection is normally scheduled about five

weeks from the time the lab submits the

completed forms.

The inspector and the team then visit

the laboratory. They hold a preliminary

orientation meeting with the lab director

and department heads. In a hospital lab,

the inspector also meets with the hospital

administrator and chief of staff. The

inspection team then goes through the lab

and completes the inspection checklists.

After the inspection, the inspection team

and laboratory staff assemble for the sum- mation conference, mentioned earlier.

After the on-site visit, the inspector

sends his report to the College computer

center where a computerized list of

73
deficiencies is generated. This computer

report, along with the inspector's written

comments is sent to the regional commissioner

with the rest of the file. The commissioner

then reviews the report and sends a copy to

the lab director with appropriate comments,

requesting that the deficiencies he corrected

and that documentation be provided for each

corrective action. The lab is given 30 days

to respond to the report.

Once the commissioner receives the

response from the lab, he carefully reviews

the entire report, and once he is satisfied

that each deficiency has been adequately cor- rected and documented, he sends his recom- mendation to accredit the lab to our office,

and an accreditation letter is sent to the

laboratory under the chairman's signature.

A certificate is also ordered at this time.

Formal approval is made by the entire com- mission at its next regularly scheduled

meeting. If the lab fails to meet the standards required for accreditation, it is advised on ways to correct the deficiencies

and invited to re-apply after one year.

Each year there are approximately twelve to

sixteen labs that are denied accreditation.

Each step of this process is monitored

on our computer and a weekly report is sent to our office so that we can keep track of

all the labs being inspected. Once the laboratory is accredited, this information is

entered on the computer and the time is

automatically set so that the lab receives

reinspection forms 90 days before its accreditation is due to expire. The accreditation,

by the way, is valid for a two year period.

During the interim year, between on-site

visits, the lab is sent a set of the Inspection Checklists and asked to perform a self

inspection. This enables the lab to evaluate its current status and correct any deficiencies that may have cropped up since the

last inspection. A copy of the self inspection report is sent to the inspector who is

assigned to perform the next on-site visit,

so that he can check to see that the deficiencies noted in the interim year have been

corrected.

References

I [l] Commission on Inspection and Accreditation, Inspector's Manual for

Accreditation of Medical Laboratorie

College of American Pathologists, 19

p. 8.

Our data shows that each subsequent

inspection uncovers fewer and fewer deficiencies. This is a clear indication that the

program is meeting its primary goal of laboratory improvement.

7^
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

LABORATORY ACCREDITATION FOR

TOXICOLOGY FACILITIES

HARRY W. HAYS

U.S. DEPARTMENT OF AGRICULTURE

SCIENCE, EDUCATION AND ADMINISTRATION/

AGRICULTURAL RESEARCH

BELTSVILLE, MARYLAND 20705

Certification of toxicologists and accreditation of toxicology

laboratories have been the subject of concern to the Society of

Toxicology since 1973. Numerous committees have studied the problem

and all came to the conclusion that such programs would greatly

enhance the science of toxicology. This paper describes certification

and accreditation programs for toxicologists and toxicology laboratories .

Key words: Accreditation; certification criteria; evaluation;

toxicology laboratories.

1. Introduction

For some years, there has been concern

on the part of the regulatory agencies in the

Federal government about the standards of

performance of laboratories submitting data

on chemicals that must receive approval prior

to shipment in interstate commerce. Congressional committees and consumer organizations

have been quick to blame toxicologists for

having inadequate performance standards.

! When certain laboratories were charged with

failure to meet good laboratory practices,

the credibility of toxicologists suddenly

came under fire. Officials of the Food and

Drug Adminstration and the Environmental

Protection Agency testified before Congressional committees that data from toxicology

laboratories could not be relied upon and

something would have to be done to ensure

the quality of performance by laboratories

submitting data to regulatory agencies. The

Society of Toxicology became deeply concerned

about these charges and responded by supporting and sponsoring a certification and ac- creditation program.

2. Certification

In June, 1979, the American Board of

Toxicology was incorporated in the District

of Columbia. It is now in the process of

preparing a "self-evaluation examination"

program which will be available to anyone

who wishes to test his or her skills on com- petence in toxicology. The examination will

be graded and persons found to be deficient

in any area(s) will then have an opportunity

to take refresher course(s) in preparation

for a full-fledged examination for certification. The response to participate in this

program has been most encouraging and indicates that toxicologists regard toxicology

as requiring a high degree of professionalism. The Board is totally independent of

the Society of Toxicology and elects its own officers and Directors. It did however re- ceive substantial financial support from the

Society to help defray expenses in getting

the program underway.

75
3. Accreditation

The development of reliable toxicological information depends upon the caliber of

the laboratories from which basic data are obtained.

In July, 1979, the Toxicology Laboratory Accreditation Board was incorporated

in the District of Columbia to provide a program for laboratory accreditation. The

Board consists of twelve members, with approximately equal representation from industry, government, academia, and consulting

laboratories. The Board is independent of

the Society of Toxicology, but received a grant in support of the organizational work.

The purpose of the Toxicology Laboratory Accreditation Board is to encourage,

promote, and maintain high standards of performance in toxicology through a system of

accreditation by (l) recognizing those laboratories demonstrating competence in various aspects of toxicology; and (2) promoting good laboratory practices in toxicology.

The Board will accredit a toxicology laboratory in specific functional areas for which

the laboratory requests accreditation and

demonstrates that it meets the standards

and requirements for accreditation. Any

laboratory applying for accreditation must

designate on the application those functional areas for which accreditation is desired. For each area, the toxicology laboratory may receive full accreditation,

provisional accreditation, or accreditation

withheld. To receive full accreditation , an applicant must have completed and prepared

reports or publications of studies in the

functional area for which accreditation is

sought. Failure to obtain full accreditation in one functional area will not nor- mally influence the action of the Board in

considering other areas within the same laboratory. A list will be published an- nually, specifying functional areas in which

full accreditation has been granted. Provisional accreditation will be granted to a laboratory which meets all requirements in

staff, facilities and procedures but has not demonstrated proficiency. Within a limited

period of time (to be determined by the

Board), the laboratory must meet the deficiency provision to become fully accredited

or provisional accreditation will be re- voked.

A laboratory that has been fully ac- credited and subsequently found to have

serious deficiencies will be notified of

those deficiencies and given the status of

probationary accreditation . Within a

76

specified period of time (to be determined

by the Board) the deficiencies must be cor rected or accreditation will be revoked.

Laboratories from which accreditation is

withheld will be given a report listing th

deficiencies and the laboratory may reappl

(no earlier than six months) before consid

eration will be given for accreditation.

Laboratories applying for accreditation wi

be required to pay a nonrefundable applies

tion fee, the amount being determined at t time of application, the size of the labor

tory, number of professional and technical

staff, number of animals housed, and an es mate of the time required for the site vis

To help defray expenses of the Board, accr ited laboratories will be required to pay

an annual fee

It is anticipated that toxicology lab

oratories will seek accreditation for the

following reasons: (l) to serve as eviden

that an accrediting organization of profes

sional peers considers the laboratory capable of performing toxicological investigations, (2) to provide outside consultati

with the assurance that the procedures to followed are consistent with the "state of

the art" and, (3) to serve as a means of

determining whether facilities and equipme

or procedures could be improved upon

The Board is currently in the process

of preparing a manual on procedures for

accreditation which will include instructions for filing applications, payment of

fees, and preparations for site visits.

The site visit team will consist of two or more individuals, including experts in the area for which accreditation is being song

The review will be based on guidelines for

good laboratory practices, as established

by the Board. Particular attention will h

given to such things as qualifications of

personnel, adequacy of facilities and equi;

ment, scientific soundness of protocols,

conduct of study, appraisal of the results

and the program for quality assurance.

It is hoped that the establishment of

the Toxicology Laboratory Accreditation

Board will lessen the need for government

intervention. It is essential that person:

working in laboratories be given freedom t< pursue new ideas while at the same time wo: under self-imposed high standards. The

Accreditation Board fulfills this need because it is a voluntary program.
SESSION IV

ACCREDITATION

SYSTEMS

A N D

CONCEPTS

CHAIRMAN: JOHN A. GRANT

AMOCO OIL COMPANY
: ti

at

tor;

*
tional Bureau of Standards Special Publication 591, Proceedings of the National Conference

. Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaither sburg

,

i, September 25-26, 1979. Issued August 1980

LABORATORY PERFORMANCE EVALUATION AND ACCREDITATION - IT'S ALL IN THE IMPLEMENTATION

WILLIAM J. SMITH AND G. THOMAS CASUNO

UNDERWRITERS LABORATORIES INC.

NORTHBROOK, ILLINOIS 60062

Based on many years of experience in the development of product evaluation information under Federal,

State and local Codes and Standards and related in-house

studies of associated accreditation programs, the critical features of these programs have been identified in

terms of the degree to which they are likely to achieve

their objectives. Most accreditation programs place

heavy emphasis on written documentation of conformance

to established criteria and in-house quality assurance

programs, stressing test equipment calibration and

maintenance, with limited attention to the critical control features and measures of the extent of in-house

implementation. A review of the critical features of

laboratory accreditation programs indicates that these

programs must include: (1) "hands-on" proficiency of

personnel, (2) feedback mechanisms for program improvement, (3) procedures-in-case-of non-compliance and/or

errors, (4) random unannounced inspections of test

facilities and (5) independent countercheck reference

testing. Accreditation programs lacking in one or more

of these critical features can drift to a "least common

denominator" modus operandi and fall short of original

objectives

.

Key words: Countercheck reference tests; critical

control features; feedback; implementation;

inspections; laboratory accreditation; laboratory performance evaluation; noncompliance; proficiency.

Introduction

Virtually all laboratory accredi- stressing test equipment calibration

ration programs currently sponsored and maintenance. In most cases

:>y Federal, State, and local regula- limited attention is given to

:ors place heavy emphasis on written critical control features of

Jocumentation of conformance to laboratory quality assurance pro-

?stablished criteria and in-house grams which, if given higher

quality assurance programs, generally priority in evaluating the

79
performance of individual programs

would provide for an overall higher

level of accuracy and uniformity of

test results and/or product certifications .

We believe that a review of key

elements of laboratory evaluation

and accreditation programs is necessary to fully appreciate the

significance of the phrase: "It's

all in the implementation." These

include: "hands-on" proficiency of

personnel, feedback mechanisms for

program improvement, procedures-incase-of non-compliance and/or errors,

the importance of random unannounced

inspections of test facilities and

the need for independent countercheck reference testing.

Consideration must be given to

current laboratory accreditation

programs that appear to be drifting

into the "least common denominator"

modus operandi with accredited

laboratories marginally satisfying

established criteria. Accredited

laboratories unwilling to lower

their performance standards to this

"least common denominator" can, by

choice, restrict their involvement

in particular areas of product testing and certification or withdraw

from the entire accreditation program. Emphasis upon implementation

of the essential criteria identified

as a result of these considerations

must be stressed in order to minimize

inequity and improve the overall

effectiveness of any laboratory

accreditation program.

Studies of Accreditation Programs

Underwriters Laboratories is

currently involved in a number of

laboratory accreditation programs

covering a wide spectrum of products

and services. This involvement has

provided opportunities to study the

key features of each program and

evaluate strengths and weaknesses.

These programs are all related to

UL's activities concerned with the

evaluation and testing services of

materials, products, and systems

for public safety. These programs

are sponsored or managed by all

segments of government — Federal, ; State, and municipal — and, in soznf ? cases, by private organizations.

\

The High Velocity-Short Range Progra

i The nature of accreditation

criteria and their implementation

vary greatly among programs. Some

are limited to the most basic

elements necessary to accredit a facility: place of business, chief]

person to contact at the facility,

financial status, staff size, and

the nature of services provided,

accompanied with a prepaid fee. In

these cases, the accreditation

process is simple, and limited in

scope. Problems, if any, usually i: "ri relate to some minuscule detail

overlooked in the application

process. We refer to these progran

as "High Velocity-Short Range"

programs

The Paper Based-Middle Range Progra' ~

Other accreditation programs, ' however, are supported by a set of" - : "rules and regulations" or accredi- 1 tation criteria, varying in detail,5 :i and covering a broad range of info]' !' mation about the operation and

qualifications of the laboratory

seeking accreditation. Ownership

and management structure, resumes

of key personnel and their report- I ing relationships, details of the 1 :; physical plant and test equipment, '

: record keeping, report format and

in-house review procedures, certi- I fication mark or label control,

feedback and corrective action pro'

cedures, test equipment standardi- ^ zation and calibration, laboratory

quality control procedure and free-' ; : dom from conflict of interest are among the details usually included

80
3j I

.n these criteria. Applications for

accreditation usually must be accomjanied by voluminous paper detail,

locumenting or responding to, in one

form or another, the established

criteria. Accreditation under these

:ypes of programs, what we term "Paper

iased-Middle Range" programs, is

>ased primarily upon a review of the

^aper submitted by the applicant and

:he prepayment of fees. Again, any

iroblems usually relate to overlooked

>aperwork detail.

These paper based programs are

lumerous; an example which typifies

hem occurred several years ago. UL

tade application for accreditation

nder the designated Rules and

egulations. On the surface, the

•rogram appeared to be detailed,

l|[|road scoped and comprehensive. We

kroceeded with some apprehension as

o whether or not we could readily

bet all the criteria that had been

stablished while at the same time

jeing pleased that the authors of

rjjcjhe program showed such initiative in

Ijhe development of comprehensive

uality assurance criteria for

ccreditation for all applicants. We

ade considerable effort to gather

q^jll the voluminous paper required to

espond to the criteria and furnished

:he several copies required along

:ith the accreditation fee. Subsequently, after one or two trips to

[he accrediting organization office

fJo discuss certain details, we were

ccredited and began to implement our

ervices under that accreditation,

ased upon the number of hurdles we

|ad overcome to gain accreditation

nitially, we fully expected a conjinuing and fruitful dialogue with

(he accreditation program administraors concerning the implementation

If the program. Instead, our total

nvolvement with the administrators

or the past 2-3 years has been an

nnual letter reminding us that coninued accreditation was contingent

m the payment of the renewal fee.

\

tP

>5

Ctj

ie

at,

id

tier:

::

A

re

Multi-Faceted-Broad Range Programs

Finally, there are accreditation programs that involve the full

range of elements including: a com- prehensive application to establish

qualifications, initial and followup on-site inspections, a required

in-house quality assurance program,

proficiency testing and at least

an announced intent at the equitable implementation of the

originally established accreditation criteria, including deaccreditation procedures.

These are the types of programs

that must serve as the foundation

upon which the laboratory accreditation programs are built, if indeed

the overall concept is to succeed.

In fact, it is only through the use of the multi-faceted-broad range

approach that the overall need for

the establishment of individual

laboratory accreditation programs

can be supported. If high velocityshort range or paper based-middle

range programs continue to be the

trend, the entire concept of laboratory accreditation will no longer

be supportable.

The Accreditation Panacea

Many accreditation programs

depend upon the occurrence of non- conformances in the field to

trigger actions by the accreditor

to seek correction on the part of one or more participating accredited

laboratories, or to begin deaccreditation proceedings. In many

respects this "de facto accreditation" system has merit. With one essential difference it is this

type of system which the work of

third party laboratories has been

judged, accepted for many years and

continues to be judged and accepted

in numerous product testing areas,

long before formal accreditation

programs came into vogue. It is

the system under which UL has grown

and gained National and Inter81
national acceptance. The essential

difference in the emerging system of

accreditation is that participating

laboratories initially receive an aura of high competence and performance capability under a cloak of

accreditation. Whether or not the aura is deserved or later grows

depends upon the nature of the

accreditation process. A superficial and ineffective accreditation

program can lead to acceptance of

the "least common denominator"

modus operandi, wherein the accredited laboratory marginally satisfying

the criteria sets the level of performance toward which laboratories

accredited under the program migrate

or feel pressure to follow the downward trend. The sinking ship

analogy may fit in this situation;

one either elects to say on board

and ride it down with the others,

or take to the lifeboats, salvaging

what you can in the process.

Critical Control Features

Based upon many years of

experience and related in-house

studies, it is recommended that five

"critical control features" be

included in laboratory evaluation

and accreditation programs and

implemented in conjunction with such

programs. These features are:

(1) "hands-on" proficiency of personnel, (2) feedback mechanisms for

program improvement, (3) proceduresin-case-of non-compliance and/or

errors, (4) random unannounced inspections of test facilities and (5)

independent countercheck reference

testing. While each of these

elements is individually treated in

greater detail herein, it is vital

that they be implemented collectively; only in this way will the

overall benefit to a given program

be realized through the synergistic

effect produced.

(1) Hands-on Proficiency of

Personnel - The present emphasis

82

on evaluation and accreditation of

laboratories based upon review of

paper documentation of qualificatioi

should be reversed. On-site evaluation should include the witnessing

of the conduct of tests or the per- : formance of service by those person

nel that are actually engaged in th< work. Objective evaluations,

including oral interviews designed

to elicit the level of familiarity

with the test method and equipment

should be held with involved person

nel at both supervisory and technic

levels

(2) Feedback Mechanisms for

Program Improvement - Few existing

accreditation programs include

mechanisms for feedback from other

laboratories or users of the

product or service. Such mechanism

should be developed and their use encouraged. Feedback mechanisms

could include a system for categori

ing and analyzing complaints, perio

seminars with accredited laboratory

personnel and users, periodic infor

mational letters or bulletins to

participants that call attention to

any emerging sub-standard features

of the program with recommendations

or directives for corrective actior

and frequent in-depth analysis by

the administrators, with input

solicited from the participants, tc determine if the program is meetinc

its objectives. To be successful,

however, the utilization of feedbac

mechanisms must be for the purpose

of improving the program, rather

that solely being the source of

implementing punitive actions.

(3 ) Procedure-In-Case-Of Non

Compliance or Errors - This control^-

feature is perhaps of the greatest

overall importance to the effective ness of an accreditation program.

If the actions taken to correct a non-compliance, either under the

accreditation program or the inhouse quality assurance programs o: the participating laboratories,

represent long-term procedures
i/herein the non-compliance (i.e.,

aboratory program defect, incorrect

est data generation, etc.) is

1 lowed to continue until a resolu- ion is found or on a so-called

emporary provisional basis , wh ich

n reality becomes a permanent conlition, then meaningful procedures

n-case-of non-compliance exist in

lame only.

Effective procedures-in-case-

>f non-compliance are based upon

:he premise that actions associated

'ith a proven non-compliance must

ease. This means, for example,

hat an accredited laboratory ceases

o conduct certain tests under the

lanner of the accreditation until

he non-compliance is resolved; this

leans that manufacturers of the

roducts so tested cease using the

on-complying product performance

est data in conjunction with the

arketing of the product; this means

hat if the non-compliance continues,

eaccreditation follows expediciously

.

Indeed the strength and substance

f the answer to the question "what

appens in case of non-compliance?"

epresents the strength and subtance of the entire accreditation

rogram. This question must be

sked of all participants in any

ccreditation program and the nswer carefully analyzed by the

ccrediting organization. Organizaions utilizing multi-faceted broad

ange accreditation programs will

ecognize the weak, non-substantive

nswers as being characteristic of

neffective in-house laboratory

ality assurance programs.

(4) Unannounced Inspection of

ist Facilities - It is recognized

hat the initial on-site evaluation

f personnel and test equipment

hould be announced. Laboratories

ay need a reasonable amount of time

o determine that the facilities are,

n fact, ready for accreditation

nder particular criteria of a given

rogram. Once accredited and

providing services under the program,

laboratories should be subject to

random, unannounced on-site inspections. They serve as a means for

early detection of substandard performance, disclosure of major

changes in key personnel and significant variations in major test

equipment and serve as a countercheck that important recommendations

for improvement have been implemented. Most importantly, this

unannounced visit permits the

examiner to make a more accurate

evaluation of the laboratories 1 day-to-day operations and avoid onetime demonstrations capability that can be associated with announced

facility inspections.

(5 ) Independent Countercheck

Reference Testing - Another very

effective mechanism for monitoring

the work of laboratories is to

continually subject the results of

a wide spectrum of their work to

review and countercheck. Anonymity

among participating laboratories is

a necessary requirement, as is the

need for reporting the results to

participants. Given the opportunity to compare the results of

their work with other laboratories,

program managers are much more

likely to correct non-complying

performance. Similarly, they are

in a much better position to demand

that accreditors take action to

correct deficiencies indicated

through interlab reference specimen

testing.

Summation

UL's experiences with accreditation programs have not been entirely

on the negative side. We are cur- rently participating in some accreditation programs that have been care- fully and knowledgibly conceived and

uniformly and equitably implemented.

83
The success or failure of any laboratory accreditation program ultimately depends upon the persistence

and judgment of the administrators to

implement the program in a uniform,

equitable manner with insistence upon

compliance with all features of the

program's criteria.

It is perhaps appropriate to

close with a quote by Elbert Hubbard,

who wrote the famous "Message to

Garcia." In his published notebook

he said: "We shall never get the

right idea of work until we see at

the bottom of it is public service."

81*
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

LABORATORY ACCREDITATION BY DISCIPLINES

ROGER J. AMOROSI

CHAIRMAN, BOARD OF DIRECTORS

AMERICAN ASSOCIATION FOR LABORATORY ACCREDITATION (AALA)

359 MILWAUKEE AVENUE, HALFDAY, ILLINOIS 60069

Abstract

In Laboratory Accreditation Programs by Discipline - the laboratories are accredited for performance of groups of tests or specific tests

within a discipline of testing. Individual laboratories may be accredited in more than one discipline and for one or more classes within each

discipline. The evaluation process for discipline accreditation covers

the technical and ethical competence of the personnel, and the equipment

and quality control procedures for the discipline/groups of tests that are enumerated in the laboratory's application for accreditation.

Key words: Classes of tests; directory of accredited laboratories;

discipline; laboratory accreditation.

1. Introduction

The principles of "accreditation by disciplines" are not generally understood, although they are utilized successfully by the

largest national system, Australia's National Association of Testing Authorities

(NATA) . They are also utilized by the

<\merican Association for Laboratory Accreditation (AALA) , and are acceptable to the

National Voluntary Laboratory Accreditation

Program (NVLAP) under their optional procedures (15 CFR Part 7c) for private sector

Drganizations.

2. Nine Technical Disciplines

The NATA and AALA systems have nine

technical disciplines as follows:

1. Acoustical and Vibration

Measurement

2. Biological Testing

3. Chemical Testing

4. Electrical Testing

5. Thermal Testing

6. Mechanical Testing

7. Metrology

8. Non-destructive Testing

9. Optics and Photometry

Each discipline has sub-disciplines

referred to by NATA as sections and/or

classes of tests, and by AALA as groups of tests. A group of tests within the electrical discipline may cover a type of test such as high voltage testing or be by broad

product classification such as batteries.

NATA divides the Non-destructive Testing discipline into seven sections:

Radiographic examination of metals

Radiographic examination of non-metals

Radiographic examination of components

and assemblies

Ultrasonic examination of metals

Ultrasonic examination of non-metals

Ultrasonic examination of components

and assemblies

Other non-destructive tests

Each section is divided into classes of tests. Applications for accreditation may

be made for one or more classes of tests or for one or more items within a class of

85
tests. Examples of the breakdown of two NATA

sections within the Non-destructive Testing

discipline are as follows:

Radiographic Examination of Components

and Assemblies

Aircraft structures

Components and assemblies

(applicant to specify)

Ultrasonic Examination of Non-metals

Ceramics and refractories

Plastic laminates

Plywood

Other specified non-metals

The above examples help illustrate that

a laboratory is evaluated for accreditation

by sub and sub-sub disciplines, not for all

the testing within the discipline. The laboratory specifies the extent of its capabilities in its application and requests accreditation accordingly.

3. Evaluation Process for Discipline

Accreditation

All valid laboratory accreditation programs involve an on-site evaluation of the

capabilities at the laboratory. After the

laboratory's application has been reviewed

and deemed to provide adequate information,

trained, professionally qualified inspectors

are assigned to visit the laboratory to evaluate its technical competence to perform the

types of tests indicated in the application.

The evaluation covers the personnel,

equipment, organization, laboratory and

quality control procedures for the discipline

and all involved types of tests. Since discipline accreditation is more broad-range

than accreditation "by product-by standard"

the review of "personnel and adequacy of procedures must be very extensive. Demonstrations of typical tests during initial evaluations and follow-ups are required.

The initial inspection of the laboratory

requires a minimum of one day and up to several days dependent upon the size of the

laboratory and the number of classes of tests

in each discipline. A similar amount of time

is required to evaluate the same laboratory

for a single product being tested per one or

a limited number of standards.

4. Discipline Standards and

Accreditation Criteria

The NATA program has a separate booklet

that provides information on requirements for

registration in each discipline. Each booklet describes the coverage of the disciplii

and the classes of test, the application a; evaluation procedure and other pertinent i' formation.

The AALA program is similar and each

discipline accreditation program is based

upon voluntary consensus standards that

have been and are being developed in ASTM

and other standards writing organizations.

These separate discipline standards follow

the generic criteria that are provided in

ASTM Standard E-548 "Standard Practice

Generic Criteria for Use in the Evaluation

of Testing and/or Inspection Agencies" and

ISO Guide 25 "Guidelines for Assessing the

Technical Competence of Testing Laboratori

5. Directory of Accredited Laboratories

An important function of all laborato

accreditation programs is to make availabl

to all interested parties on a current bas

a directory of accredited laboratories, wi

clear designations of the discipline and

types of tests covered by the accreditatio

The user of the accreditation program must

be made aware of the basis of the evaluations, the extent and limitations of the

listings, and the reevaluation procedures.

This full disclosure is important since th user of the accreditation and testing serv ices must determine if the accreditation

process is adequate for his use or whether

he wants to add additional requirements tc satisfy his specific needs.

In addition to the above information,

the NATA Register of Laboratories details

the terms of the accreditation and lists

the signatories. The terms of accreditatj

are those classes of test applicable to ti work of the laboratory, modified by rangef

of measurements where they are applicable,

and by statements of uncertainty of measu: ment where they are applicable. The approved signatories are the laboratory off; cers to whom the Council has given specif:

approval to sign NATA endorsed test documents. Endorsed test documents may be

signed by these signatories. If any of Ell leaves the laboratory, he is automaticall;

terminated. The NATA Directory is in the

form of loose-leaf sheets, with one or mo: sheets for each laboratory. Updating she<

are distributed frequently.

6. Conclusion

There are hundreds of laboratory ac- creditation programs covering specific,

limited areas of test; they are operated

j numerous organizations, such as corporati s trade associations and government agencie
ori i!

nd are generally limited to their own use

uch as for a product certification program,

s a consequence of these splintered and un- oordinated accreditation programs, most

aboratories in the U.S., especially the

ulti-discipline laboratories doing testing

or fees, are inspected many times every year

his becomes expensive and represents unecessary duplication of efforts.

The only national systems in the U.S.,

hat have as their objective providing ac- reditation in all or most test areas, are

: VLAP (government operated) and AALA (private

Rector program).

It is important to recognize that ac- reditation by discipline is a broad-band

pproach and as such permits a greater

ortion of the total accreditation assignLent to be done in a shorter period and in more efficient manner.

.on, ':

Lis

:s

:ati: it!

iges:

;

jle,:

!SH j r I m

Cif!

CU-]

f tl

all".,

the | uki

sheii

:atif
National Bureau of Standards Special Publication 591, Proceedings of the National Confer

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersbui

MD, September 25-26, 1979. Issued August 1980

ACCREDITATION PROGRAM FOR CANADIAN TESTING ORGANIZATIONS

ROB IE E. MacNINTCH

CERTIFICATION AND TESTING DIVISION

STANDARDS COUNCIL OF CANADA

OTTAWA, ONTARIO, CANADA

KIR 7S8

The paper briefly describes the constitution, role and responsibilities of the Standards Council of Canada, which is the accrediting

authority for the national voluntary accreditation program for testing

organizations in Canada. It also touches on the composition of the

National Standards System (NSS) , a federation of independent autonomous

organizations working towards the further development and improvement

of voluntary standardization in the national interest. This system,

which is under the aegis of the Standards Council, will include

accredited testing organizations as one component.

The document, "Criteria and Procedures for Accreditation of Testing

Organizations" (CAN-P-4) was approved and published by Council in 1978. A limited (pilot) accreditation program is being conducted this year

(1979) in order to evaluate implementation procedures and estimate costs

for a national program. It is hoped that this limited program will

provide sufficient information for Council to make decisions in 1980

concerning a national program.

Key words: Accreditation; Canada;

Council of Canada; Testing.

1 . Introduction

This presentation describes the

Standards Council's accreditation program

for Canadian testing organizations but

first it may be of interest to provide some information on the constitution of the

Council. The Standards Council of Canada

was created by an Act of Parliament in 1970

(The Standards Council of Canada Act). Its membership consists of 16 Federal and

Provincial officials and 41 representatives

from private organizations which have

interests in the field of standardization.

However, it is important to emphasize that

the Council is not a government agency nor

are its employees public servants. In effect, it is independent of government in

its policies and operation although it is

financed by a grant from the Federal

Government

Laboratories; Standards; Standards

The prime objective of the Council

(as stated in the Standards Council Act)

is to foster and promote voluntary

standardization.

The Act also states that the Counci

in carrying out its objectives, may

"accredit, in accordance with criteria a procedures adopted by the Council, organ

zations in Canada engaged in standards

formulation, testing and certification

The Council was not created as a sujt organization duplicating existing expertje

and facilities. It does not itself prepfe

write or publish standards. These funct ins are performed by the Standards -Writing

Organizations (SWOs) who are accredited

the Council in accordance with approved
criteria. Similarly, the Council does not plan to certify products or operate testing

laboratories but is implementing programs

to accredit both Certification and Testing

Organizations

.

2 . National Standards System (NSS)

The National Standards System is

formally defined as "A federation whose

components are accredited standards-writing,

certification and testing organizations,

the Canadian committees concerned with

international standardization and the

Standards Council of Canada. The System

provides a coordinated approach to the

development and advancement of voluntary

standardization in the national interest".

The programs for accreditation of

certification and testing organizations are now in the implementation phase and the current status of the testing program will

be described later.

3. SCC - Organization

And now before introducing the main

topic of my presentation, I feel it might

be of some value to briefly describe the

SCC organization. The Council itself meets

only twice a year and the responsibility

for its affairs between these meetings is

delegated to a nine member Executive

Committee. This Committee meets six times

a year.

The day-to-day work of the Council is

planned and co-ordinated by a permanent

staff of approximately sixty. Four branches

headed by Directors are responsible to the

Executive Director. Three of these branches

are located at the Council Headquarters in

Ottawa with one (the International Standardization Branch) situated in Mississauga,

Ontario (near Toronto)

.

IID1C1

:ia ail

The specialized professional expertise

and experience is provided to Council by

various Advisory Committees. Thus the

Advisory Committee on Certification and

Testing (ACCT) is the body responsible for

supplying advice and guidance to Council on these activities. The ACCT is a 24 member

body which was formed in 1974 and since that

time has operated under the chairmanship of

Mr. J.E. Elliot, Director of Engineering,

a ^Vehicle Quality and Safety, Chrysler Canada,

-:*'- ; Ltd.

fiinctfl

;ing

litei

roved

4. Accreditation of Testing Organizations

The document "Criteria and Procedures

for Accreditation of Testing Organizations"

(CAN-P-4) was approved at the Twenty-First

Council Meeting on June 5, 1978. Before

discussing this document a resume of the

background of the program including a brief

outline of its scope might be in order.

One point which I would like to stress

is that the Council program for the accre- ditation of testing organizations envisages

testing as a function in itself and not

solely in its relationship to certification

programs. The goal is to develop a voluntary accreditation program that will

identify those organizations that are com- petent in their fields of testing and also

promote a general improvement of testing

services in Canada. Such a program would

permit competent organizations in all fields

of testing to qualify for national accre- ditation. This would include calibration

services, testing for product development,

research and contract monitoring as well

as testing in support of certification

programs

.

It is planned to provide an accreditation program in support of product and

technical testing in all fields. However,

it should be noted that the National

Research Council of Canada is responsible

for maintaining Canada's national metrological standards while the Department of

Consumer and Corporate Affairs is responsible for regulating and inspecting

measuring devices used in trade.

Several other Federal Government

departments conduct programs to determine

if manufacturing and service organizations

are qualified in accordance with legislated

requirements or, in other cases, to specific contract specifications.

In addition to these government

activities there are also some non- government agencies operating qualification

programs. One example is the standard

"Qualification Code for Concrete Testing

Laboratories" (CSA A283-1974) . This

standard was developed by the Canadian

Standards Association which is an accre- dited standards-writing organization of

the National Standards System. The program

is operated by the Certification branch of

that organization.
4.1 CAN-P-4

Reference was made above to the approval by Council of a criteria and procedure

policy document for the accreditation of

testing organizations. This document was published in September, 1978, with the

title "CAN-P-4 - Criteria and Procedures

for Accreditation of Testing Organizations".

In addition to approving the draft

CAN-P-4, Council also authorized the

Advisory Committee on Certification and

Testing (ACCT) to proceed with the development of implementation procedures, the

investigation of legal aspects and the

implementation of a limited program to estimate costs. The first two activities

have been completed and the implementation

of a limited program has been initiated.

This limited program is essentially a pilot

project and on its completion (the evaluation of a small number of representative

testing organizations) recommendations will

be made to Council concerning the implementation of a national accreditation

program.

In format CAN-P-4 consists of a Foreword, Preface, nine Sections and three

Annexes

There are seven criteria which are summarized below:

Criterion 1 : Ability to operate and

maintain an adequate testing capability.

Criterion 2 : Staff-administrative

competence.

Criterion 3: Staff-technical com- petence .

Criterion 4: Adequate facilities,

Criterion 5 : Documented and acceptable procedures for maintenance of records.

Criterion 6 : Willingness to allow

examination of records, etc.

Criterion 7 : Independence of

operation.

CAN-P-4 is, in essence, a "generic"

type document in that its criteria and

requirements must be met by any applicant

for accreditation regardless of the field,

or fields, of testing in which it is

engaged. "Field of Testing" is defined as "...a range of related testing activities

as defined by the Standards Council of

Canada" and in Annex "C" to CAN-P-4 they are listed as - Chemical, Electrical, Mechanical

Non-Destructive and Physical.

4.2 Implementation of the Accreditation

Program

Accreditation of testing organizations

is a two-level procedure. The first level

consists of compliance with CAN-P-4 and the

second level is defined by supplementary

documents related to that portion of the

specific field for which the applicant has

requested accreditation. These supplementary documents may be National Standards

of Canada, other established standards or a client's specifications. It will be the

responsibility of the applicant to supply

this documentation and identify those

portions of the field (or fields) of testing

for which it claims competence.

It is realised that while a small

testing organization would probably be

interested in being accredited for a small

portion of a specific field, a large organi

zation might well request accreditation in

several fields.

The evaluation of a testing organization is conducted by a Testing Accreditatior

Sub-Committee (TASC) of the Advisory

Committee on Certification and Testing

(ACCT). This sub-committee consists of a chairman and two members and it is

authorized to request assistance from

specialists when necessary. After the

evaluation is completed the TASC makes its

recommendation through ACCT to Council.

The Standards Council is the approving

authority for accreditation.

5 . Other National Programs

There has been a considerable amount

of documentation generated in the past few

years by other countries regarding laboratory accreditation.

Some national accreditation programs

are already in operation and the Standards

Council has been able to take advantage of

the previous effort which has been expended

by others in this area.

Specifically, I would like to refer to the work done by the U.S. Department of

Commerce and the National Bureau of Stan- dards in the development of their NVLAP

90
sndei

program. Other sources from the U.S. are

ASTM, which produced its Standard Recommended Practice E-548-76, and the American

Council of Independent Laboratories (ACIL)

which has also been active in this area. The program of the American Association

for Laboratory Accreditation is also being

noted with interest.

From the U.K. we have also benefited

by the work done by the British Standards

Institution (BSI) and the British Calibration Service (BCS) . Their procedures

and documentation have been of considerable

assistance to us.

We have drawn on the Australian

experience where the National Association

of Testing Authorities (NATA) has had a laboratory registration program in existence for over thirty years - indeed it is

probably the pioneer in this respect. A

more recent national scheme is that of New

Zealand where the Testing Laboratory

Registration Council (TELARC) has been in

operation for the past few years.

On the international scene the

Council has participated in the 1977 and

1978 International Conferences on Recognition of National Programs for Testing

Laboratories (ILAC) and will be represented at ILAC 1979.

6. Conclusion

It is anticipated that the program

for accreditation of testing organizations

will assist in identifying those organizations which have demonstrated competence.

Objectivity, impartiality and accuracy are

the major elements to be considered in

establishing the credibility of an organization which provides such testing

services

.

A Directory of organizations which

have demonstrated their ability to comply

on a continuing basis with common agreed

criteria and procedures will, it is

believed, considerably enhance their

'national status and recognition. It also

should be of considerable value to

potential users of their services.

The program for accreditation of

testing organizations is a part of the

National Standards System which is by

definition a voluntary federation of

organizations. The success of this program will depend to a large extent on

the response it receives from both the

participating organizations and the users

of their services. It is still too early

to determine this response or to assess the impact on national testing activities.

91
National Bureau of Standards Special Publication 591, Proceedings of the National Conferenc

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

ECONOMIC EFFECT OF LABORATORY ACCREDITATION

DAVID KRASHES

PRESIDENT, MASSACHUSETTS MATERIALS RESEARCH, INC.

WEST BOYLSTON, MASS. 01583

A fear of some commercial laboratory owners about

national accreditation is that laboratories remaining

unaccredited will be able to charge lower prices and,

thus, garner the majority of the testing business. This

study determined average prices for tensile tests of

metals, relative amounts of these tensile tests performed

in "well-accredited" and "poorly-accredited" laboratories;

and determined the effect of being accredited, under

today's systems of accreditation, on the quantities of

tensile testing business obtained by commercial laboratories. In geographic regions where being "accredited"

is recognized as important by a major segment of manufacturing industry, most of the tensile testing, regardless

of price, is performed in "well-accredited" laboratories.

In geographic regions where no strong segment of consuming

industry requires "accreditation", most of the tensile

testing is done in laboratories "poorly accredited" by

today's systems. Thus, if the consuming industry recognizes the value of accreditation, there will be an economic

advantage for accredited laboratories: they will be able

to charge more and also may gain more testing business.

In regions where no strong segment of industry recognizes

the value of accreditation, accredited laboratories will

suffer economically. The economic effect of any proposed

laboratory accreditation program should be studied prior

to initiating the program.

Key words: Accreditation; costs; economic; prices; testing.

1. Introduction tory owners seem to say they spend

great quantities of time preparing

The owner or manager of a commer- documentation and undergoing inspeccial laboratory is always prey to the tions. Presumably, there would be ai suspicion that competitors charging economic effect of national accredi-:

lower prices will garner most of the tation.

available business. Many types of

testing are looked upon by consuming The Federal Register of April 2 industries and governmental agencies 1979 published a comment by the Depa

as commodities: quality is either ment of Commerce, page 24276, that

taken for granted or not cared about. said quite strongly that there is noj

need to study the economic impact of

Everybody says laboratory ac- accreditation programs, because thos

creditation costs money. All labora- requesting the accreditation program

92 I
il *

Depfj

it

s

t

of

,grai I

will already have considered the economic impact. My interest was piqued.

I had neither heard nor read of factual economic studies.

I proposed to Dr. Berman that the

economic impact of accreditation on tensile testing ought to be studied,

and I proposed to do so on a large

number of laboratories. I suggested

that I would come here today to report

on data and results of which I had,

at that time, no knowledge. I see,

as a matter of fact, that I am the

only speaker in the entire program

discussing economics.

2. The Study

The tensile test of a metal is

a pull in a testing machine of a sample that has already been prepared

to a specific shape and size. Preparation charges for test samples

vary and are often quoted as a fixed

price. However, the price of the

test alone is almost always quoted

on a fixed price basis.

Within certain bounds, the test

must be performed approximately the

same way by almost all laboratories.

Properly done, about 5 to 7 tensile

tests per hour are possible. Shortcuts, which sometimes lessen quality,

enable as many as 10 to 12 tests per

hour to be done. This lowers cost

and can lower price. The price any

one commercial laboratory charges for

a tensile test can be quoted; is

likely to reflect the way the laboratory does the test; and also may reflect competitive conditions the

laboratory faces.

I sent questionnaires to over

100 commercial laboratories in the

eastern United States and asked the

prices they charged for tensile tests.

I also asked questions which would

indicate whether or not each laboratory was in a condition that would

probably enable it to pass an ac- creditation inspection. A copy of

the questionnaire is attached to the

written copy of this talk. As you

might expect of a novice, I didn't

ask exactly the right questions; and

I had to telephone a great many laboratories both to get answers and to

form opinions as to their accreditation status.

I divided "accreditation status"

into 2 categories: "good" and "fair".

Laboratories that had quality-control

manuals and had already been approved

by accreditation systems that I believed to be thorough (such as the

aircraft engine industry) , I considered to be in the "good" category.

Laboratories without quality-control

manuals or which had been inspected

only by customers such as welding

fabricators in the nuclear industry,

who often perform "less-than-rigorous"

inspections, I considered to be in

the "fair" category. I have personally visited a number of the laboratories in this study and I talked by

telephone to many more. While there

may be disagreement with my definition

of accreditation categories, I am quite certain that I placed most of

the laboratories in the proper category .

3 . The Data

In the eastern United States,

with perhaps not enough laboratories

reporting, Table 1 shows the average

price overall for tensile tests, and

the average prices charged by "fair"

Table 1. Average Price Per Test

Eastern U.S.

(Incomplete Data-41 Laboratories]

Overall $12.12

22 Accredited "Good" 11.86

16 Accredited "Fair" 13.53

and "good" categories of laboratories

The "good" and "fair" are within 12%

and 2%, respectively, of the overall

average. All prices over $25. and

all laboratories doing under 5 tests

per week are excluded.

Now, here is a rule of thumb

regarding pricing: an industrial

customer, who has been regularly

using a certain vendor, will not

shift to another vendor on a price

basis alone if the reduction of

price is less than 10%. In the

laboratory industry, users seem to

require even greater reductions of

price to induce them to change from one commercial laboratory to another.

The data of Table 1 doesn't show

enough variation of price to cause

shifts of customers. Oddly, the

93
"good" category seems to charge a lower average price.

Table 2, in which the volume of

tests performed in various laboratories is considered, throws a different light on the subject. Here

we see that most of the tensile tests

in the eastern United States are performed by the "good" category at

Table 2. Price and Quantity

Eastern U.S.

charged by the "good" laboratories

is than the overall average. The

"fair" laboratories are almost at

the average. This is a situation in

which price differentials might induce movements of customers from

higher priced to lower priced laboratories .

So, what do we find? Table 4 shows that most of the tensile testing in New England is performed at

a price above the New England average

Tests

Per Day

20 or more

10-20

5-10

Under 5

Average

Price

$11.58

8.78

12.45

13.70

Where

Done

100% in Good

60% in Good

60% in Good

30% in Good

Table 4 . Price and Quantity

New England

prices lower than the eastern average.

Although not shown in the table, 2/3

of all tests done in the eastern

United States are priced below average

and half of all the tests done are performed in the "good" category laboratories . These broad-scale geographic data must be considered with caution.

They cover too much territory geographically, whereas most tensile

testing of metals is done on a regional basis. Probably most commercial

laboratories are regional: serving

a small geographic region. In the

eastern United States, probably very

few tensile test samples travel more

than 100-150 miles. The foregoing

data overlooks regional competition,

and the data to follow provides substantiation .

In New England, Table 3, there

are 5 "good" and 6 "fair" laboratories

Table 3. Average Price Per Test

New England-11 Laboratories

Overall Eastern

Overall N.E.

5 "Good"

6 "Fair"

$12.12

7.75

11.50 (+48%)

7.92 (-2%)

performing tensile tests. See how

much higher, 48%, the average price

Tests

Per Day

20 or more

10-20

5-10

under 5

Average

Price

$ 9.85

10.40

5.50

Where

Done

100% in Good

60% in Good

0% in Good

of $7.7 5 and most of it is performed

in "good" category laboratories. In

fact, 60% of all the tensile testing

is performed in the "good" laboratories at above the overall average

price

In New England, most of the

purchasers of tensile testing are willing to pay higher prices in order to have their tests performed

by "accredited" laboratories.

New England, incidentally, is the

headquarters of two of the nation's ' largest aircraft engine manufacturer

Pratt & Whitney and General Electric'

Both companies have very strong laboratory inspection and accreditation

systems. Subcontract vendors to

these two companies abound, and they are forced to use accredited labora

tories for their testing. Many othe

industries in New England recognize

that accreditation by Pratt & WhitneJ

and General Electric means a laboratory does good work.

For the Metropolitan New York

City area, which includes New York

City, Long Island, and northeastern

New Jersey, Table 5 shows the averag

price charged by "good" category

laboratories to be higher than the

overall average; while the average

price charged by "fair" laboratories

is significantly lower than the

9h
overall average. Here, again, the

price discrepancy seems great enough

to warrant purchasers moving from the

higher priced to the lower priced

laboratories. Yet, Table 6, we find

that all laboratories doing more than

10 tests per day are in the "good"

category. In fact, 70% or more of

all the tensile tests done in this

region are at above the average price

for the region and are performed in

"good" category laboratories.

Table 5. Average Price Per Test

Greater Metropolitan New York

7 Laboratories

Overall National

Overall

5 "Good"

2 "Fair"

$12.12

13.86

15.50 (+12%)

9.75 (-30%)

In another small geographic are near New York, of five laboratories

performing tensile tests, two are

"good". Complete data on volume of

tests was not obtained, but one

"good" laboratory, charging a price

60% of that charged by the others,

was doing the heaviest volume of

tensile testing found in the eastern

United States.

This seems to indicate that a well-accredited laboratory that is

able to charge a lower price can attract a very significant volume of

the tensile testing market.

The Philadelphia area, Tables 7 and 8 , shows a complete reversal of

anything discussed so far. Here,

Table 6. Price and Ouantity

New York Area

Tests

Per Day

20 or more

10-20

5-10

Average

Price

$15.00

15.00

12.33

Where

Done

100% in Good

100% in Good

33% in Good

The Metropolitan New York region

is complex and less well understood.

There is a significant aircraft industry accreditation program, and

there may be a "rub-off" factor.

There may also be a transportation

problem: users hesitating to entrust

samples to a slow and difficult

shipping situation, and, hence, using

the nearest laboratory.

Within the region, there is one

small geographic area in which two

laboratories are "good" and one laboratory is "fair". One "good" laboratory and the "fair" laboratory compete "head-on" at the same low price.

The "good" laboratory does about 50% more tests than the other. Surprisingly, the third laboratory, which is

also "good", charges 70% more than

the other two and still obtains 30%-

50% of the tensile testing business

in the area. This seems to indicate

that purchasers of tensile testing

in this sub-area value accreditation

more than price.

Table 7. Average Price Per Test

Greater Philadelphia Area

Overall National

Overall

3 "good"

3 "fair"

$12.12

10.33

15.00 (+45%)

5.70 (-45%)

Table 8 . Price and Quantity

Philadelphia Area

Tests

Per Day

Average Where

Price Done

20 or more

10-20

5-10

less than 5 12.25 75% in good

6.50 0% in good

there is a severe price discrepancy

between the "good" and "fair" categories. The price discrepancy

appears great enough to make users

of tensile testing move to the lower

priced laboratories. The "good"

category laboratories have only 25%

of the tensile testing market in the

region that extends from Baltimore

to central New Jersey and westward

halfway across Pennsylvania.

This region contains a heavy

foundry industry, diverse mundane

manufacturing industries, and no strong aircraft or nuclear industry

that relies heavily on independent

testing laboratories. This is a region in which a tensile test is a

95
commodity. Price, therefore, is a very strong consideration. Published

advertisements and visits to laboratories indicate that sub-standard

techniques may be in use and, thus,

may hold down cost.

4 . Conclusion

There is an economic effect of

laboratory accreditation. In geographic regions where most consumers

of tensile testing believe accreditation means quality, the laboratories

that seem to be best qualified are getting the major share of the business. This is so regardless of

whether they charge higher or lower

prices. In geographic regions where

consumers of tensile testing are not

aware, on a widespread basis, of

accreditations and possible variations

of quality, the laboratories charging

the lowest prices are getting most

of the business.

Thus, if industry is aware of

the value of laboratory accreditation,

the economic effect of national ac- creditation will be good for accredited commercial laboratories: they

will obtain more business and at

higher prices. If industry does not

care about laboratory accreditations

and quality, the lion's share of the

business will go to lower-priced

laboratories, whether they are ac- credited or not.

Organizations desiring a national

laboratory accreditation procedure

for a given test or category of tests

must study the potential economic

effect. This should be done by

gathering data on a regional basis in

order to avoid disruptions, "splitting the market" for the test, in some regions.

A "split" market" would mean

that some commercial laboratories in

a region would forego either the

accredited or the unaccredited portion of their business. The effect

would be to lessen the capacity for

doing that type of test in the region.

Since commercial laboratories are an integral part of much manufacturing

industry, a decrease of testing

capacity could be disruptive to

industry

The economic effect of a laboratory accreditation program cannot be

taken for granted. The potential

economic effect must be studied

regionally to be sure that requester;

of a LAP from one region do not disrupt the market for testing in anoth<

region

5. Appendix - Questionnaire

Sent to Laboratories

What do you charge for performing a tensile test (not including

machining) of a standard metal sampL

Do you perform your tests or keep your records in accord with re- quirements of any industry, government agency, company or anyone (do

not include ASTM) . (If you follow

only ASTM, your answer should be

"no" .

)

IF YES

Are you regularly (at least

once each year) inspected by the

accrediting agency?

If yes, do they give you a certificate or place you on an "approved list"?

If yes, how many different

accreditations like the above do

you undergo for tensile testing?

Do you have a typed or written

manual that describes the following:

how you perform a tensile test, how

you prepare and maintain test re- cords, how you check the quality of

your tensile tests, who in your laboratory is qualified to perform

tensile tests?

96
SESSION V

QUALITY

CONTROL

chairman: edmond g. franzak

sand i a laboratory
! 5
National Bureau of Standards Special Publication 591, Proceedings of the on Testing Laboratory Performance: Evaluation and Accreditation, held at

MD, September 25-26, 1979. Issued August 1980

National Conference

NBS, Gaithersburg,

THE ROLE OF THE QUALITY CONTROL MANUAL

IN THE INSPECTION AND TESTING LABORATORY

ROBERT J. WENING

B0WSER-M0RNER TESTING LABORATORIES , INC.

P. 0. BOX 51

DAYTON, OHIO 45401

An indispensable tool for every testing laboratory is an adequate quality control manual. Such a document is a cornerstone of

the laboratory's credibility and an important ingredient in its as- sessment for accreditation.

Several years ago, the American Council of Independent Laboratories (ACIL) , an organization with a long history of interest in

laboratory accreditation, made a significant contribution in this area. ACIL commissioned its Laboratory Accreditation Committee to develop a guidebook on laboratory quality control. The committee's

product was published by ACIL in 19 76; its title is "Manual of Practice: Quality Control System: Requirements for a Testing and In- spection Laboratory."

In this paper, ACIL will review the development of the manual

and its contents. Included in it are sections on Organization;

Operational Procedures; Personnel; Equipment and Calibration; Reference Samples; Recommended Personnel Basic Requirements; Inventory of

Standards and Equipment Requiring Calibration; and Sources of Reference Samples.

The paper will also assess the impact of this manual on the laboratory community. Samples of adaptations of the manual's guidelines

by individual laboratories in various disciplines will be presented.

Also, reference to the manual by other bodies, such as Government

agencies, will be discussed.

Finally, the paper will estimate the importance of quality control manuals and programs in the day-to-day operation of a laboratory

and in relation to the broader issues of evaluating and accrediting

testing laboratories.

Key words: Calibration; equipment; evaluation; laboratory; manual;

personnel; quality assurance; quality control; systems.

99
Dating back to the sixties and early

seventies the independent laboratories,

especially those involved with the aerospace industry, were in quite a dilemma.

(Although the term "independent" is used

in this paper when referring to laboratories, the principles of Quality Control

apply to all types of laboratories.) The

laboratories began to feel the pinch when

multitudinous clients and governmental

agencies began their own methods and interpretations of Quality Control Programs and

audits of testing laboratories. The laboratories also began to get requests for

"Who certifies you?" or "Who accredits

you?" or "What national agency qualifies

you?". Some laboratories were hiring

people strictly for the purpose of working

with auditors. It was at this point nearly a decade ago that a group of us in the

American Council of Independent Laboratories decided that something must be done to develop a uniformity in the system. When

the concept of a laboratory qualification

or accreditation program was presented,

naturally a committee was formed to develop

the idea. Many people in the laboratory

business said that it was impossible to accredit laboratories or to develop criteria due to the great diversity of laboratories and types of testing and inspection. However, in developing the concept,

it became so simple that it seemed ridiculous that there are, in essence, just

three main ingredients in an accreditation

program for a laboratory, or really any

type of service. They are: People, Equipment and a Quality Control Program. It is

not necessary to accredit to every specification, or to every material, or even to every product. This is the muddle in which

we have been involved. As we progressed,

it became more clear that the backbone of

any accreditation program was to be the

Quality Control Aspect.

It soon became quite apparent that in

order to have an accreditation program, it was necessary to have an "accreditor" and

"accreditees". The accreditees we had,

they were the members of ACIL but the ac- creditor was lacking. If we, the ACIL, were to be the accreditation agency, then

in essence, it would be a self-certification program and lack the meaning that was wanted and needed.

The emphasis was then switched, by the

committee and council, to attempt to develop a Quality Assurance System that would be

uniform for testing laboratories regardless

of field of operation and be readily adaptable for an accreditation system.

The Quality Assurance System

The Quality Assurance System was then

examined and the keys to the concept were

established.

These keys were:

Laboratory Organization

Operational Procedures

Personnel

Equipment and Calibration

Reference Samples

Let us examine each of these keys in some detail:

Laboratory Organization

A look at the Laboratory Organization

provides an excellent perspective of the

Quality Assurance Program. It gives: the

type of work that can be performed, the

geography covered, the function of the main

laboratory and branches, historical data,

structure or organization for capabilities,

a look at the operational and support departments, and indication of services

available

Operational Procedures

Operational Procedures are a vital par

of the Quality Assurance System. Written

procedures describing such items as workfloi

routes; sources utilized for inspections;

listings of certifications, qualifications

and laboratory approvals by other agencies;

detailed procedures for conducting tests

analyses, inspections, etc.; and detailed

procedures for sampling or selecting speci- mens for testing, are all a necessary part

of operations.

Personnel

Personnel is a major key to establishment of a Quality Assurance System in a laboratory. It would really be great if we could all have a cadre of perfect personnel

but I don't think that anyone has achieved

this perfect work force. Personnel management systems therefore needed, are Job

Responsibility Mandates (job descriptions),

Job Training Programs, Personnel Classifications and Personnel Biographies.

100
Equipment and Calibration

In order to establish credibility

in the field of testing, all equipment

used must be accurate, reliable and dependable. This constitutes the physical

facilities of the laboratory. A com- plete list or inventory of all equipment requiring calibration or standardization must be maintained; sources for

calibration and required frequency must

be kept updated constantly and those en- vironmental conditions required must be

met

.

Now this is where, as they say,

the rubber meets the road. The keystone to all systems, Quality Assurance,

Quality Control, Accreditation, etc.,

etc., is DOCUMENTATION. Unless it's

recorded, it has no value. Records

must be maintained and corrective actions verified. Suitable records shall

be maintained for calibratable laboratory standards and test equipment,

whether used internally or externally.

Each inventory listing should contain

the following:

1. The name of the manufacturer.

2. The equipment model and serial

number.

3. The properties subject to standardization

.

4. The range of operation and the

range of calibration.

5. A reference to a recognized

calibration procedure.

6. The frequency of calibration.

7. Allowable tolerances or maxi- mum sensitivity.

8. The source of verification.

suitability - shall be established by

each laboratory. Subsequent to the

cross-referencing of test equipment and

calibration standards, the necessary

sources and required frequency for ver- ification shall be established. The success of the calibration system is primarily dependent upon the quality control supervisor who is responsible for

scheduling calibrations or standardizations. An individual selected for this

position should be personally responsible and able to delegate authority, but

not responsibility , to subordinates. Determination of realiztic calibration intervals is dependent upon the basis of

stability of the instrument or standard,

its purpose and degree of usage.

Certain calibrations and test methods require controlled environmental

conditions to insure the accuracy of re- sults. Dependent upon the physical property and degree of sensitivity of the

calibration or test, certain environmental factors such as temperature, humidity, cleanliness, vibration, voltage,

radio frequency interference, pressure

and atmosphere must be controlled. The

extent of the control required however

should be specified.

Written procedures and documented

evidence of calibrated standards and test

equipment provide the basis for a sound

Quality Control Program. Procedural detail may be individually derived by each

laboratory or may be altered to conform

to established recognized standards.

Primarily the scope of the operation

must satisfy the referee or inspector,

as well as conform to the Client's re- quirements. Sufficient detail should

be incorporated into an in-house procedure so as to make the document instructional to internal personnel as well as meaningful to external referees. The

following information is pertinent to such a document:

9. A chronological history of re- pairs, modifications or substitutions .

10. Traceability of reference

standards to the National Bureau of Standards or accepted

values of natural physical

constants

.

The proof of adequacy of standards,

and test equipment in particular - capability for accuracy - stability and

Type of calibration or standdardization being performed.

Specifics such as calibration

points or intervals, accuracy

required, governing specifications, etc.

Necessary standards and equipment .

Record of test data.

101
5. Equipment condition.

6. Final disposition.

Measuring and test equipment shall

be calibrated by the laboratory or an external commercial facility, either of

which must utilize reference standards

whose calibration has been certified as being traceable to the National Bureau

of Standards or has been derived from

accepted values of natural physical constants. Certificates or reports shall

attest to the fact that the standards

used in obtaining the results have been

compared at planned intervals with a standard traceable to the national

standard or with independently reproduced standards.

To assure continuity of satisfactory operations, provisions must be

made for the prevention of equipment

and personnel inaccuracies. A documented system of existing practices including periodic inspections or operational checks to insure stability between calibrations shall be established

to assure uniformity of performance and

to serve as a basis for alteration of

standardization intervals.

Records provide objective evidence

that calibration schedules are complied

with and that the accuracy of the equipment or standards is being maintained.

Proof of performance should include

reference standard certificates, data

sheets containing actual values of both

present and prior calibrations, a summary of routine maintenance and repairs

made and the final calibration form

which suitably cross-references the

equipment to the reference or transfer

standard.

Reference samples

After all the personnel, equipment

and calibration systems are in order and

are being documented, then the job of

Quality Assurance is still not completed.

It is a well known fact in statistical

analyses that variables do occur, that

makes results fall outside of accepted

limits. We must therefore establish con- fidence limits. The true indication of

a good Quality Control Program is in the

results obtained; accurate, reliable, re- peatable results. This is in effect

"the proof is in the pudding", and is

determined by objective and competent

personnel using standard methods and accurate equipment, instruments, and materials to ascertain the repeatability and

accuracy of the laboratories' test results. This proof is obtained by the use of reference samples an/or inspection

services. There is a need in this area

for the establishment of "round-robin"

testing and comparative results. There

are many organizations capable of originating and performing this service, but at this time more is needed.

Impact of the Manual on the Laboratory Community

To the uninitiated in the ways of

Quality Control the first impact upon

starting is ghastly horror. It is "horrendous" job to develop a Quality Assurance Program starting from point zero.

It is a lot of work and takes considerable time. It is costly, and there is no use kidding you about it. Those who have

done it know only too well what a project

it is. Furthermore, it is a continuing

job, an ongoing project that needs con- stant attention. Once established, however, the Quality Assurance Program can be and is very beneficial. Its use can be beneficial in sales of services, it

is irreplaceable in accreditation programs and benefits are boundless in loss

prevention programs.

The Quality Control Manual is an instruction kit that you have written for

yourself to guide your laboratory operation in the production of quality work.

I have seen some of the manuals produced

by laboratories and they are excellent.

When completed, a Manual may well be con- sidered to be a proprietary document, due

essentially to the cost that it has taken

to produce it. Many are also considered

to be proprietary due to the content that

a laboratory has included for its own use but not intended for public dissemination. I do not feel that I am at liberty

to even suggest the names of laboratories

that I know, that have completed their

manuals, but if you inquire you might

find some laboratory that is willing to share its manual with you.

Use of the Manual of Practice

I personally feel extremely pleased

that the Manual of Practice, Quality Control System has been accepted by many

laboratories and also by many agencies now writing accreditation criteria. It has become the backbone of many systems now in operation. I come from a genera102
tion of laboratory personnel who needed

no quality control because they were the

experts - their word was final - if they

said so, that was it. Well, times have

changed - thank God. And now when we say

it, we have the documentation to prove

it, we have the system to back us up.

The use of the Quality Assurance

System in day-to-day operations of the

laboratory is of the utmost importance.

Operational checks of equipment before

the day's work begins, the check of instruments and the constant use of standards have done more to upgrade testing

and inspection services than we can

realize. And this is only the beginning.

The broader impact of the Quality

Assurance System is in the accreditation

process. The certification or accreditation of laboratories or inspection

agencies is a failure without recourse

to the Quality Assurance System. Studies have shown that a good Quality Assurance System, with proper documentation,

, provides the basic need for the accreditation audit needed.
National Bureau of Standards Special Publication 591, Proceedings of the National Conferenc

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

LABORATORY QUALITY PROGRAM REQUIREMENTS

THOMAS A. RATLIFF, JR. , P.E.

R & R ASSOCIATES

P.O. BOX 46181

CINCINNATI, OHIO 45246

The need for quality control of laboratory results was identified

early by The National Institute for Occupational Safety and Health when

a great disparity of analytical results was noted among 14 industrial

hygiene laboratories. The result was a four pronged attempt to improve

performance: 1 - An educational program was developed; 2 - An interlaboratory proficiency program, PAT, was started; 3 - A laboratory

quality control manual and a standard for laboratory control were

developed; 4 - A laboratory accreditation program was developed by AIHA

under contract.

The Environmental Protection Agency, the Occupational Safety and

Health Administration, the Food and Drug Administration, the American

Society for Testing Materials and others have described similar programs

in various standards and regulations. Finally the American National

Standards Institute (ANSI) Z-l Committee on Quality Assurance and its

Systems and Procedures Sub-Committee have agreed that there is a need

for a generic Quality Control Standard for laboratories which will differ

from a manufacturing quality control standard. A laboratory quality

control standard is being drafted by a writing committee under the aegis

of the Biomedical Division of the American Society for Quality Control.

Keywords: Accuracy; control chart; control limits; corrective

action; data validation; precision; proficiency analytical testing;

quality control; statistical quality control.

1. Introduction

Under the Occupational Safety and Health

Act of 1970, the National Institute for Occupational Safety and Health was given the

responsibility for organizing the analyzing

and the reporting of samples collected by

OSHA Compliance officers in the detection of

contaminants in industrial atmospheres.

Under the provisions of Section 7(c)(1)

of the above Act, contracts were let by OSHA

with 12 state industrial hygiene laboratories

to assist in handling the work load.

Early in the program, to determine the

level of agreement among 14 laboratories, 12

state and 2 NIOSH, known samples of contami- nants were sent to the participating organi-l

zations. Not surprisingly, 14 different

results were reported.

Accordingly, a number of steps were J taken in an effort to improve laboratory

performance by obtaining better agreement

| among laboratory results. To put it another

way, a concerted effort was made to improve

the quality of laboratory product, the "pro-;

duct" being reports of analytical results. t It was sought, therefore, to improve precision and accuracy of method determination

reports, while not providing methods evaluation information.

10U
2. Quality Improvement Measures 2.3 Laboratory Accreditation

Over a period of time NIOSH has taken

the following steps to improve laboratory

performance quality:

2.1 Proficiency Analytical Testing

Program

NIOSH started the Proficiency Analytical

Testing Program in May, 1976 known as PAT.[1]

The Chemical Reference Laboratory, in Cincinnati, by contract prepares known samples

of contaminants -- asbestos, lead, silica

dust, cadmium and zinc, and one of eight

solvents on charcoal collection tubes. These

are mailed at two month intervals to participating laboratories. Results are returned to the CRL. There they are tabulated and forwarded to the participant laboratories.

Each round has four sets of samples

(subgroups) plus one blank. We are currently

preparing control charts similar to (Figure 1)

for each laboratory and each contaminant.

Plotted control limits help give an indication of drift tendencies, and a picture of

whether or not labs were running their determinations out of control. Current data are

used to determine statistically valid control

limits.

In addition, selected labs under contract

have been given "hands-on" assistance in setting up and maintaining a quality control

program. This assistance was given by a con- sulting firm under contract to NIOSH.

2.2 NIOSH Laboratory Quality

Control Manual

In order to improve and make consistent

the quality of results obtained by the laboratory the NIOSH Industrial Hygiene Service

Laboratory Quality Control Manual (Technical

Report #78)[2] was published and widely distributed. This draft manual, which identifies

and describes eleven elements of a laboratory

quality control program serves two purposes.

It assists small laboratories in setting up

a quality program, including preparation of

a quality manual, and it provides text material for courses given by the Training and

Manpower Development Division of NIOSH.

We have found that even in a very small

organization, four or five people can suc- cessfully conduct a viable quality program.

NIOSH has also contracted with the

American Industrial Hygiene Association to support preparation and validation of an accreditation program for industrial hygiene

analytical laboratories. This accreditation

includes among other things, the need to establish and maintain a quality control system within the laboratory.

2.4 Quality Training

The Training and Manpower Development

Division of NIOSH periodically offers a course to assist interested laboratories in

obtaining accreditation. A substantial portion of this three-day presentation is devoted to quality control matters, as can be seen from the following course subjects

covered:

History of Laboratory Accreditation

The Current Program

Criteria for Accreditation

Accreditation Applications

Proficiency Analytical Testing

Intralaboratory Quality Control

Frequency Distribution and Measures of

Control Tendency and Variability

Measuring Variability

The Normal Curve and Analysis of

Control

Control Charts

X - R Control Charts

Problem Session - Control Charts

Problem Session Review - Control Charts

Specific Applications of Quality Control

Charts

Special Situation Control Charts

Data Tests and Other Statistical Techniques

2.5 Laboratory Quality Control

Specification

NIOSH has developed a Laboratory Quality

Control Specification entitled, "Industrial

Hygiene Laboratory Quality Program Requirements". [3] It identifies 22 elements needed

in a laboratory quality control program as follows

Statement of Objectives

Policy Statements

Organization

105
Quality Planning

Standard Operating Procedures

Recordkeeping

Chain-of-Custody Procedures

Corrective Action

Qua! ity Training

Document Control

Cal ibration

Preventive Maintenance

Reagent and Reference Standards

Procurement and Control

Sample Identification and Control

Laboratory Analysis and Control

Interlaboratory and Intralaboratory

Testing Programs

Handling, Storage and Delivery

Statistical Quality Control

Data Validation

System Audits

It can readily be seen that though there

are some similarities to a manufacturing

quality control system, many of these elements

are peculiar to the laboratory environment.

2.6 Other Laboratory Quality

Control Activity

While NIOSH was engaging in these attempts

to improve laboratory quality, other organizations were concerned about similar problems

and undertook various steps to alleviate them.

The U.S. Environmental Protection Agency

Quality Assurance and Environmental Monitoring

Laboratory is publishing a monumental threevolume "Quality Assurance Handbook for Air

Pollution Measurement Systems"^] Volume I - "Principles" is in print and it identifies and

discusses in detail 23 elements of Quality

Assurance as follows: Volumes II and III are

in preparation.

ELEMENTS OF QUALITY ASSURANCE

Document Control and Revisions

Quality Assurance Policy and Objectives

Organi zation

Quality Planning

Training

Pre-Test Preparation

Preventive Maintenance

Sample Collection

Sample Analysis

Data Reporting

Procurement Quality Control

Cal ibration

Corrective Action

Quality Costs

Interlaboratory and Intralaboratory

Testing

Audit Procedures

Data Validation

Statistical Analysis of Data

Configuration Control

Reliability

Quality Reports to Management

Quality Assurance Manual

Quality Assurance Plans for Projects

and Programs

The Mine Safety and Health Administra

tion is carrying out an active interlaboratory test program to standardize the

accuracy of reference standard gas concentrations used in calibrating laborat

gas chromatographs

In September 1973, The Occupational

Safety and Health Administration published : Title 29 Code of Federal Regulations, Partj

1907 Accreditation of Testing Laboratorie!

This regulation, now temporarily revoked, j.s a double requirement for quality control.

First, testing laboratories, testing and

approving a wide range of safety and healt

related equipment from ladders to fire ex- tinguishers, must demonstrate existence of

a satisfactory quality control system in

order to gain accreditation. In turn, the

accredited testing laboratory must demand

that its clients, who are manufacturers of

products over which OSHA excercises sur- veillance, have an effective quality contr

system, ensuring this by periodic quality

surveys and audit of off-the-shelf purchas

of their products.

While these and other government agenc were developing and attempting to use thes

relatively new laboratory quality control

techniques, two other groups were beginnin

to exhibit interest in this area. The fir was what was formerly the Biomedical Contr

Technical Committee of ASQC. (Now the Bio-t medical Division). Early in 1973, NIOSH

established liaison with this organization

which now has a laboratory quality control

writing committee of which the author is a member. NIOSH has solicited and received

from this committee outstanding interest

and assistance in the preparation of the

Laboratory Quality Control Specification

noted above.

A primary interest of this ASQC commit

is in the preparation of a laboratory qua! assurance standard and work is proceeding

this end.

106
Secondly, the organization meeting of

the American National Standards Institute

Committee Z-l on Quality Assurance was held

on February 5, 1975. Within the Z-l Committee there is the systems and procedures

sub-committee. One of the functions of this

committee is to identify the need for one or more generic ANSI standards for the management of the quality system.

The American Society for Testing Material

has published in draft a "Recommended Practice for Laboratories Engaged in Sampling

and Analysis of Atmospheres". Its requirements for Laboratory Quality Control are similar to those of the NIOSH quality

specification.

I am sure that all present are familiar

with the NAVLAP provisions for laboratory

qual ity control

.

The FDA regulations 21 CFR 1000 "Diagnostic Radiology Facilities Quality Assurance

Programs" and 21 CFR 82 "Medical Devices"

provide typical examples of regulatory

requirements for laboratory quality control.

Since laboratories of all kinds fall

generally under the service industry category, NIOSH feels that its "Specification for

an Industrial Hygiene Laboratory Quality

Control System" can with some judicious

editing and pruning, be made applicable to the operations of any type of laboratory

and, by extension, to any type of service

industry.

The foregoing has been an explanation of

the efforts of various governmental industrial and standards publishing individuals

and groups to develop and agree upon an acceptable quality control system standard

for laboratories of every kind. Once one

has accepted the basic premise that all

laboratories receive samples of some kind

from some source; must identify and keep

track of them; analyze or test them; re- port on results; keep adequate records;

maintain the accuracy of test and analytical

equipment by systematic calibration and

carry out all or most of the quality control

elements listed as necessary above, then it seems "as follows night, the day" that a laboratory quality control system standard

is not only necessary but feasible and

finally, inevitable.

[3]

[4]

[1] Rati iff , T.A. "Quality Control in

Occupational Safety and Health," Annual

Technical Conference Transactions , American Society for Quality Control,

1974, pp. 129-131.

[2] Industrial Hygiene Service Laboratory

Quality Control Manual , Technical

Report #78, National Institute for

Occupational Safety and Health, Cincinnati , Ohio, 1974.

NIOSH Specification for Industrial

Hygiene Laboratory Quality Program" Requi rements , National Institute for

Occupational Safety and Health, Cincinnati , Ohio , 1976/

Quality Assurance Handbook for Air

Pollution Measurement Systems, Volume I- "Principles" , U.S. Environmental Pro- tection Agency, Quality Assurance and

Environmental Monitoring Laboratory,

Research Triangle Park, N.C., 1975.

107
Figure 1 - Sample Control Chart For Each Laboratory

For Each Contaminant

108
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

QUALITY CONTROL PROCEDURES IN A LABORATORY

TESTING CEMENT FOR COMPRESSIVE STRENGTH

RICHARD D. GAYNOR AND RICHARD C. MEININGER

NATIONAL READY MIXED CONCRETE ASSOCIATION

900 SPRING STREET, SILVER SPRING, MARYLAND 20910

This paper describes an in-house quality control system

used by the Joint Research Laboratory (JRL) of the National

Ready Mixed Concrete Association and the National Sand and

Gravel Association (NRMCA-NSGA) . The system is used to control

the quality of the results of compressive strength tests of

Portland or blended cements by ASTM Method C109 but can also

be used for other physical and chemical tests of cement and

concrete.

Key words: Cement and Concrete Reference Laboratory; cement

and concrete testing; compressive strength; control charts;

in-house quality control; quality control system.

Introduction

The NRMCA-NSGA Joint Research Laboratory

routinely participates in the Cement Reference Sample Program of the NBS Cement and

Concrete Reference Laboratory (CCRL) in which

pairs of cement samples are distributed twice

a year. In addition the JRL has obtained a large number of sealed samples of a local

cement which are tested as a "standard" or "reference" cement whenever an "unknown

cement" is tested and when the CCRL samples

are tested. The paper describes the system

used to control the quality of tests for cement compressive strength by ASTM C109.

The system can also be used for other physical and chemical tests of cement.

Control charts are maintained of the

deviation of JRL results from the average

of other laboratories participating in the

Cement Reference Sample Program and of the

deviation of the reference cement from es- tablished averages. The data generated

demonstrate that in the JRL the most troublesome variations are those which occur

over long periods of time. Standard deviations of tests made within a period of

several days are approximately half as large

as the standard deviations of tests made

over a period of several years. Tests of

standard cement made at the same time as the CCRL reference samples show excellent

correlation. That is, if the standard

cement results are high, results on the

CCRL reference samples will also be high

relative to the average of all laboratories

in the program.

109
CCRL Cement Reference Sample Program

Since 1965 the CCRL has distributed

pairs of cement samples every six months to

a group of 120 to 180 participating laboratories. Each laboratory is asked to perform

physical and chemical tests in accordance

with designated ASTM Standard methods. The

cement reference sample program is described

in papers by several authors . [1, 2 , 3 ,4] 1 Re- sults are reported to the CCRL and in a short time each laboratory receives a consolidated report giving the average values , multi-lab precision, single lab precision and

other statistical data along with scatter

diagrams for each test. Further, each laboratory receives a numerical rating from 5 to 1 depending upon the deviation of its re- sult from the average of all other laboratories.* The rating scale is in multiples

of the multi-lab standard deviation in ac- cordance with the schedule in Table 1.

Use of the Cement Reference Sample Data

Although the cement reference sample

data have been useful in the development of

precision statements in several ASTM methods

and in evaluating methods and procedural var- iations there has been little written describing how a laboratory can use the data to improve its own performance or control test

procedures, aside from the examination of

the scatter diagrams and the tabulation of

the numerical ratings from 5 to 1 described

earlier.

Table 1 summarizes the ratings of the

JRL (Laboratory 112) for 3 and 7 day compressive strength for C 109 mortars.

In the CCRL program two samples of ce- ment are distributed to participants twice a year. For compressive strength each laboratory is asked to prepare a single batch of

C 109 mortar with each cement and to mold

six 2 inch cubes for test in compression.

For all samples except numbers 9-12,

three cubes are broken at 3 days and three

at 7 days. Samples 9 to 12 were Type III cement and the test ages were 1 and 3 days.

Since the JRL does not perform all of the

physical and chemical tests the samples are

large enough for the main operator to mix

two batches from each sample. Generally one additional operator makes an additional

batch as a part of a continuing operator

training program. Only the results of the

first batch mixed are reported to the CCRL.

lFigures in brackets indicate the literature references at the end of the paper.

*Using CCRL multi-lab standard deviation.

In 1968 the JRL secured a supply of

carefully blended cement and sealed a large

number of samples in metal cans to provide e additional "standard cement," Lot 4538. Th£

cement is tested with each CCRL sample and

whenever an important unknown sample is re ceived for testing.

Table 2 is the detailed compressive

strength data for tests of samples 51, 52,

and the standard cement. Generally, Batch A

is mixed on one day and Batch B on the folic

ing day. Method C 109 defines a "test" as tl. average of tests of 3 cubes and although va

ues for individual cubes are reported to the CCRL only the average values are used in the CCRL reports and tabulations. Table A is a summary of the deviation of the JRL test

values from the average for all participati

laboratories. Table B is a similar tabulation for the standard cement, Lot 4538.

Tables A and B are appended to this paper

Table 3 is a summary of various aver

ages and standard deviations calculated fro

the strength deviations in Tables A and B Figure 1 shows a good correlation between t deviations from the CCRL sample averages an the deviations of tests of Lot 4538 standar

cement mixed at the same time. The Lot 453

deviations are calculated from the average

used for that lot. Plotted values are for

tests of individual batches. The line show

is drawn through an overall average CCRL

deviation of -10 psi and an average Lot 453

deviation of -75 psi and is parallel to the

line of equality. No regression analysis

has been made.

;

:

11

»

11

The overall variation is calculated as the standard deviation of the individual

batch deviations in the respective tables

taking into account the sign of each deviation. The average deviation, -26 and

+6 psi for the CCRL samples are negligible

and near zero.

For Lot 4538 the deviation averaged -' and -74 psi. These are barely of statistical significance. The corresponding averages were zero after sample 40. The overall

standard deviations range from 182 to 229 ps

The differences between standard deviation!

for CCRL and Lot 4538 samples are not significant. Further, although the 7-day

values appear larger they are not significantly higher than the 3-day values. A representative value for the overall longterm standard deviation would be about 212

psi.

110
The short term batch-to-batch standard

deviation from batches mixed on consecutive

days can be estimated from the range of

strength test results obtained on Batches

"A" and "B" (or the range of the deviations).

These ranges and their averages are given in

Tables A and B. Table 3 gives the standard

deviations and averages values for the CCRL

and Lot 4538 tests. Again the values for

3- and 7-day tests are not significantly

different, and a standard deviation of 98

psi is representative of the short term variation at either age.

The short term or between batch standard

deviation ranges from 2.5 to 3.8 percent of

the average for Lot 4538 and can be compared to

the single laboratory standard deviation con- tained in the precision statement for Method

C 109 of 3.8 percent. Note that the C 109

precision statement was derived from the ce- ment reference sample program data.

The overall standard deviations are roughly 7 and 5.5 percent of the average and

are similar in size to the C 109 multi-laboratory coefficient of variation of 7.3 percent.

Construction of Control Charts

As noted in the previous section the

overall or long term standard deviation is over twice the day-to-day standard deviation and even if the day-to-day component

could be eliminated the remaining long term

standard deviation would exceed 170 psi. In marked contrast with typical cement company

laboratories which test cement every day the

JRL often tests only a few cements in the 6 month interval between reference samples.

Presumably this relatively small amount of

practice makes it difficult to avoid the

long term procedural variations..

Until additional analysis was undertaken for this paper the JRL maintained control charts on the average deviation of two

batches with 3s and 2s control limits based

on the overall standard deviation. Limits

were symmetrical about the average deviation.

At the present time a decision has been

made to maintain:

a. Separate 3-and 7-day control charts for

the reference samples. (Deviation from

CCRL average.

)

b. A control chart for 3-and 7-day tests of

Lot 4538. (Deviation from Lot 4538 average. )

c. A difference 2 sigma limit for acceptable

agreement between duplicate batches applicable to all usual cements.

d. A difference 2 sigma limit for acceptable

agreement between the three cubes that are averaged to constitute a test. Control Charts for Reference Samples

Figures 2 and 3 are the control charts

for tests of cement reference samples.

Strength is expressed as a deviation from the

CCRL average for all laboratories. Each

point represents the average for two batches.

The chart line connects the average deviation

obtained on each pair of samples distributed

in the program. Three sigma control limits

are given for the average deviation for the

two batches mixed for each sample and the

average of the pair of samples distributed at

a given time (4 batches). The standard deviation used is that for batches mixed over a short period. The limits are computed re- spectively as

0 ± 3s/ (2) % for 2 batches

0 ± 3s/ (4)^ for 4 batches

Note that laboratories which do not have

enough cement to test duplicate batches can likely estimate the short term standard deviation from the range of deviations of the

pair of samples mixed in the program. Suitable values could also be derived from duplicate batches of a reference cement or even from the C 109 precision statement.

Figure 4 is the control chart for 3 and

7 day tests of the standard cement Lot 4538.

Again construction is similar and the short

term standard deviation, averaged for 3-and

7-day tests, is used.

Examination of Control Charts

,

Figures 2, 3 and 4

1. Low results were obtained on reference

samples 5 and 6 and this is the primary

reason why the standard Lot 4538 cement

was used with sample 7 and all later

samples

.

2. Samples 11 and 12 produced markedly divergent results , but the reference cement

tests were normal. It is clear now that

the sample numbers were interchanged since

the average for the sample pair is reasonably good.

Ill
3. No ready explanation is available for the

high results on samples 17 and 18 and only

moderately high results on Lot 4538.

4. High results were produced in all tests of

samples 21 and 22.

5. Unfortunately no reference cement was tested in the period from sample 29 to 34.

During this period the laboratory was in

temporary quarters before moving into new

facilities in 1974.

6. Samples 31 and 32 produced low 7-day

strength and high 28-day strengths. No

good explanation was discovered.

7. Starting with sample 39-40 a formal control chart procedure and on-going analyes were implemented. Agreement between

batches and samples greatly improved.

Strength levels in all tests decreased progressively through samples 51-52. Procedures were thoroughly reviewed. Training

started on a new backup operator. Tests

demonstrated that cube density had decreased

and that increased tamping pressure would

increase both density and strength. When

samples 53 and 54 were tested the Lot

4538 strengths and densities returned to normal levels. The report on the averages

for the CCRL samples will not be available

until early October and, therefore, these

deviations are not available in Figures

2 and 3.

Use of Standard Cement Lot 4538 Data to Correct Tests of Unknown Cements

Figure 5 is a plot of the Cement Reference Sample Deviations (Table A) minus the

Lot 4538 deviations (Table B) for batches

mixed at the same time. In each instance,

the deviations are the average for two batches or rounds.

Figure 1 showed that the strength levels

on the reference samples correlate well with

levels on the Lot 4538 cement tested at the same time. This suggests that when unknown

cements are tested the results can be cor- rected by the amount that Lot 4538 tests,

made at the same time, are high or low.

Figure 5 is a test of this hypothesis

and except for a couple of early results the

systematic variation of results shown in the

control charts is eliminated. Table 4 provides the averages and standard deviations

of these "corrected" deviations along with

the JRL deviations in tests of the CCRL ref- erence samples and Lot 4538. The period included is from sample 15 through 5 2. 112

The "corrected deviations" average 93

and 119 psi at 3 and 7 days, respectively.

These arise because during the period the

average strength of the Lot 4538 tests was

93 and 113 psi less than the values used in

computing the Lot 4538 deviations.

3 days: 3045 - 93 = 2952 psi

7 days: 4230 - 119 = 4111 psi

Therefore/ when an unknown cement is tested

the best estimate of its 3 day strength

would be

Xcorr = xv

- (Xr " 2952)

where

Xcorr is the estimated 3 day strength

Xv is the result of testing the unknow

cement

Xr is the strength of Lot 4538 mixed a the s ame time

(In each instance results are the average o: tests of two batches.)

The standard deviations of the correct< values in Table 4 are dramatically improved

The 95 percent confidence limits on tests

corrected in this manner should be about

± 250 psi.

Within Batch and Batch-to-Batch Surveillanc

The control charts described employ av erages of 2 and 4 batches of mortar and arej

designed to control the troublesome long

term variations. Control charts for ranges^

within and between batches could be maintained but the JRL has decided to use D2S

limits for the three cubes averaged for a test and for the two or three batches gen- . erally mixed.

The range of the three cubes tested fr a batch at a given age in the reference sair pie program were extracted and expressed as standard deviation percent. The average 3 and 7 day coefficient of variations were

2.1 and 2.8%, respecrively . The precision

statement for ranges between 3 cubes is

"The range between 3 cubes made frc a single batch of C 109 mortar tested

at 3 or 7 days should not exceed 7 or

9 percent of their average, respectively (D2S). The corresponding value

for the range of 3 cubes tested at 28

days is likely somewhat larger than

that for 7 day tests, but the 7-day

values will be used until additional

data is developed. [5]
No corresponding precision statement for

agreement of tests of 3 cubes is given in

ASTM Method C 109.

The precision statement for agreement

of batches made within a period of several

days is derived from the between batch standard deviations in Table 3. The standard

deviation chosen is 3.0 percent and is applied for all ages. Note that the C 109

precision statement uses a single- laboratory standard deviation of 3.8 percent. The

JRL precision statement is

"The range between batches (rounds)

of C 109 mortar made by a single operator should not exceed the following at 3 or 7 days : (Comparable ranges

for tests at 1 day are likely somewhat

larger and those at 28 days somewhat

smaller.

)

Range of 2 rounds

Range of 3 rounds

Range of 5 rounds

Range , % of

Average, D2S

8.4%

9.9%

11.7%

Note that the D2S values should be

exceeded about 1 time in 20 in the

long run .

Summary

The Joint Research Laboratory has participated in the cement reference sample

program for the past 13 years. For most of

this time a laboratory standard cement has

been tested with the reference samples and

when other cement samples are tested.

To control the results of ASTM C 109

compressive strength test results the laboratory has developed:

a. Difference 2 sigma limits for the

range of the three cubes tested at each age.

b. Difference 2 sigma limits for the

range between batches (rounds) mixed

within a short period.

c. Control charts for the deviation of

the reference sample results from

the grand average of all laboratories on each sample.

d. Control charts of the deviation of

the laboratory standard cement

from its established average.

The procedure of expressing the JRL ref- erence sample results as deviations from the

grand average for participating laboratories

permits the use of simple, familiar, easily

understood statistical procedures, calculations and control charts.

A study of the reference sample data

demonstrates that in the JRL multiple

batches can be tested to reduce short term

testing variations to the desired levels.

However, C 109 mortars tested at intervals

of 6 months or more can be expected to show

larger variations or bias which will be

characteristic of the particular day or week on which they are tested.

If the "standard cement" yields a high result on a given day, the CCRL reference sample mixed on that day will be similarly high. Using the standard cement deviations to correct the CCRL reference sam- ples mixed on the same day significantly

improves agreement with the CCRL averages.

It appears that if the laboratory standard

cement is mixed each day an unknown cement

sample is tested the JRL can use the standard cement results to estimate the strength

that would be obtained if the 100 to 200

CCRL laboratories had tested the cement. The 95 percent confidence limit on such an estimate would be approximately ± 250 psi.

References

[1] Crandall, J. R. and Blaine, R. L. , "Statistical Evaluation of Interlaboratory

Cement Tests," Proc. ASTM V 59, 1959,

pp 1129-54.

[2] Youden, W. J. , "Statistical Aspects of

the Cement Testing Program," Proc. ASTM

V 59, 1959, pp. 1120-28.

[3] Arni, H, T. , "Precision of Air-Permeability, Turbidimeter and No. 325 Sieve

Fineness Data," ASTM STP 473, Fineness

of Cement, 1968, pp 20-44.

[4] Mandel, John and Lashof, T. W. , "Interpretation and Generalization of

Youden 's Two Sample Diagram" Jour. Quality Tech., V. 6N1, Jan. 1974, pp 22-36.

15] ASTM C 670-77, "Recommended Practice for

Precision Statements for Test Methods

for Construction Materials."

113
Table 1. Summary of JRL Ratings on Compressive Strength

intiuacu aire luc ratxngs ror .5 ana / aay strength of 46 CCRL samples that

have been tested.

Deviation from JRL,

Rating Av, Standard Expected % of

No. Deviation (1) % of Labs Samples

5 0-1 68.3 73.9

4 1 - 1.5 18.3 14.1

3 1.5 - 2.0 8.8 7.6

2 2.0 - 2.5 3.3 3.3

1 + 2.5 1.2 1.1

(l)Using CCRL Multi-lab Standard Deviation.

Table 2. Typical JRL Compressive Strength Data

(Cement Reference Sample Program, CCRL)

Test

Sample

No. Operator Batch

Age

,

Days

C 109 Cube Compressive Strength, psi

1 2 3 Av. (2) 51 WDH A(l) 3 2990 2990 2900 2960

7 4415 4260 4560 4410

B 3 3060 2940 3040 3015

7 4370 4435 4315 4375

52 WDH A 3 3280 3365 3480 3375

7 4200 4200 4210 4205

B 3 3435 3320 3430 3395

7 4160 4370 4350 4295

Standard WDH A 3 2490 2540 2480 2505

(Lot 4538) 7 3670 3730 3770 3725

B 3 2565 2625 2700 2630

7 3635 3780 3735 3715

(1) Reported to CCRL.

(2) Only the average value for tests of 3 cubes is used in scatter diagrams

and computations by CCRL.

Ilk
Table 3. Summary of JRL Averages and

Standard Deviations (J-146)

Test Age

Item 3 Day 7 Day

Overall Variations

1.1 CCRL Deviations (From the CCRL Average)

(to No. 52) N 82 84

X, psi -26 + 6 s, psi 182 227

1.2 Lot 4538 Deviations (From the Average Used for Lot 4538)

(to No. 54) N 38 38

X", psi -78 -74

s,psi (%) 208 (7.0) 229 (5.5)

Average Strength psi 2967 4156

Between Batch Std. Dev.,

psi (%) (by range)

2.1 Samples 1-40

2.2 Samples 1-52

CCRL 95 131

Lot 4538 114 (3.8) 124 (3.0) Av. 104 128

CCRL 83 HI

Lot 4538 91_ (3.1) 105 (2.5) Av. 87 108

Table 4. Correction of JRL Deviations of CCRL Samples

Using Lot 4538 Data — Samples 15-52

Data for samples 7-12 are not included because of difficulties experienced

early in the program.

No. Average Std. Dev.,

Item Values Deviation psi

3 Day Tests

Deviation

(a) CCRL 30 -20 179

(b) 4538 15 -113 200

(c) Corrected 30 +93* 126

7 Day Tests

Deviation

(a) CCRL 30 21 207

(b) 4538 15 -98 233

(c) Corrected 30 + 119* 121

*(c) = (a) - (b) 115
600

3 400

> 200 <

uo

e

o

Oc

o% "400

*H>

-200

-600

1 i

j

i 1 r

! » /*

—*—: : / . . ::

| :.:

. : . !...;'

- t

: : x v

X x

„ * /

*x, X*.

* /• X

X

: «. . x •

• • 4 «?x

x • y •*

• •* :* AY 0 *

- *—*— X . ; L_ ._ f~*

X /y*

< •

'

x

• 3 Day

X 7 Day

Individual Batches

Samples 7-52

. .

.

j .

. j;.".: :

' j: . . : :

-600 -400 -200 200 400 600

Deviation of Lot 4538 from 3045 psi

at 3 Days and 4230 psi at 7 Days

Figure 1. Comparison of Compressive Strength

Deviations on Lot 4538 with Deviations in CCRL

Cement Reference Sample Program

(Series J-146)

CCRL Sample No. Year

Figure 2. Control Chart for 3 Day Strength,

Cement Reference Samples — JRL (Series J-146)

116
1 10 20 30 40 50 60

'66 '71 '76 '81

CCRL Sample No. Year

Figure 4. Control Chart for Tests of (Series J- 146)

Cement, Lot 4538

117
600

400

200

-200

•400

-600

1 • -X-

* —*

*>X

X

»

c.

:

X

v

i x7 *3

•* X

•

1 ' ' X X

•

•

• X •3 Day,

X7 Day, x3 =

5T? = 1

93 psi

L9 psi

10 20 30 40

CCRL Sample No,

50 60

Figure 5. Reference Sample Program Results (Series J-146) Corrected for Standard Cement Results

118
Appendix A

Table A. Deviation of JKL Results from Average Compressive

Strength in the Cement Reference Sample Program

ft test is the average of results from 3 cubes from a batch of ASTM C 109 mortar. The detion is the difference between this test result and the average for all participating labs.

pie

3.

Operator

Date

3 Day Strength Tests

Deviation, psi (JRL-CCRL) Batch

Batch Range

,

A B Av. psi

7 Day Strength Tests

Deviation, psi (JRL-CCRL ) Batch

A B Av.

Batch

Range

,

psi

ACE

(8/66)

ACE

(3/67)

WDH

(8/67)

WDH

(2/68)

20

150

144

69

121

-375

118

71

165

-66

-270

-337

13

166

154

2 78

-196

356

^276

66

118

92

24

139

149

38

105

95

157

12

34

141

363

711

-147 136

-3

155

137

89

365

321

23

81

-80

+84

+ 2

86 -26

30

-364

-516

-440

-62

109

23

154

143

103

230

2 390

170

55

WDH

(8/68)

WDH

(2/69)

170

166

749

559

-55

144

1034

459

-112

-11

-62

-916

509

-203

115

310

235

100

80

7

362

354

35

367

-752 424

56

187

122

-557

389

- 84

45

374

390

70

(8/69)

WDH

(2/70)

279

96

19

6 14951

100

260

90

286

33 51238

168

136

152

235

205

JK

(8/70)

JK

(2/71)

265

379

125

138

230

374

-16513

248

376

312

-145

76

-34

35

5

40

125

79

336

-198 247

229

331

-108 212

154

334

244

-152 230

39

150

5

90

35

WDH

(8/71)

109

387

174

197

142

292

217

65

190

354

493

249

478

302

486

394

105

15

(con't.

)

119
Table a vcon u» j

3 Day otrengtn lests 7 Day Strength Tests

Oper- T~\/-s uevidLion » » -! +- i fit-i ricr-i f TT3T — f^C^TiT \

t psi \o l-v_kxj; Da 4- /-iVi Deviation, psi (JKL-CCRL) B< Sample ator R.3J1CJS / Batch

No. Date A B Av. psi D Av. i 23 WDH -13 187 87 200 170 405 ZOO „ 24 (2/ /2) 1 *3Q 1 7Q 1 CO x jy x /y xsy 182 312 O /t 7 123 268

25

ZD

27 WDH -59 41 - 9 100 Z H j? 32

ZO \£/ 1 -5 ) Z Z 1 X*± -73 37 -18

-12 7

Z Z -49

30 (8/73) 84 131

**J 1JL 1J7 47? *X 1 Z 32 (2/74) -302 947 Z •* 1

0 J) 1 /I

35 WDH -359 -319 -339 40 -287 -392 -340

Jo .CO _1 _QQ DO lJO V O Z o zUo — xxo TIB

z xo -229

37 WDH -116 54 -31 170 Its / 37

J O lb/ / D J .3U*i ~xy*± zfiy Tin -280 -105 -192

- 7Q/O

WDH o4 J4 by bU 268 103 186

40 (2/76) 103 28 _66_ 75 101 31 DO

o z JLZu

41 WDH 57 37 47 20 -64 56 — A

a n 4z / -oJ ZO 7n 74 159 1 1 C XXO

xu ~56

43 WDH -7 28 10 35 -81 -31 _ 3D

\4/ II) JO JU J Z

C

z> iy Q A COZ)Z XX "2

45 WDH -136 -126 -131 10 -66 -96 -81

la inn

\

1°/ 1 1

— yz — yz y z u -139 -74 -106

94

/J 1 WDH -24 -234 -129 210 -65 -75 -70

48 (2/78) -172 -167 -170 5 -158 -203 -180

-149 -125

(con't.

120
Table A (con" t.

)

3 Day Strength Tests

Oper- Deviation, psi (JRL-CCRL)

Sample ator Batch

No. Date A B Av.

7 Day Strength Tests

Batch

Range

,

psi

Deviation, psi (JRL-CCRL)

Batch

A B Av.

Batch

Range

,

psi

49

50 WDH

(8/78)

179

258

-94

-48

-136

-153

-144

85

210

215

221

-65

-156

-140

-188

-164

150

65

51

52 WDH

(2/79)

-486

166

-431

-146

-458

-156

-307

55

20

•387 -210

-422

-120

-404

-165

-284

35

90

5354

WDH

(8/79)

365

195

65

200

Av. Range, R

1-40

1-54

108

94

148

125

*Sample numbers were probably interchanged. Excluded from calculations.

121
Appendix B

Table B. Deviation of Lot 4538 Tests from Established Average Values

Tests of C 109 mortars made with carefully preserved samples of cement, Lot 4538. Values

are expressed as the test value minus 3045 psi at 3 days and the test value minus 4230 psi

at 7 days.

3 Day Strength Tests 7 Day Strength Tests

Oper- Deviation, psi (X-3045) Batch Deviation, psi (X-4230) Bat

Sample ator Batch Range , Batch Ran

No. Date A B Av. psi A B Av. ps

1

3

5

7 WT1H (2/68)

240 220 230 20 135 275 205 1A

Q wun. (8/68)

95 145 120 50 -15 35 10 E

1WDH

(2/69)

160 -225 -32 385 45 -295 -125 3<

15 WDH

(2/70)

210 -20 95 230 250 -80 85 3

17 JK

(8/70)

65 195 130 260 15 250 132 2

19 JK

(2/71)

-105 -115 -110 10 -90 -25 -58

21 WDH

(8/71)

240 215 228 25 450 275 362 1

23 WDH

(2/72)

-60 0 -30 60 15 65 40

25

27 WDH

(2/73)

45 -110 -32 155 0 -85 -42

29 WDH

31 WDH

(con't.)

122

. IB
Table B (con't.)

3 Day Strength Tests

OperSample ator

No. Date

Deviation, psi (X-3045 ) Batch

Av.

7 Day Strength Tests

A B

Batch

Range

,

psi

Deviation, psi (X-4230) Batch

Batch Range

,

A 13 Ay. psi

33

35 WDH

(2/75)

430 -410 -420 20 -420 -405 -412 15

WDH

(8/75)

-25 -205 -115 180 -175 -250 -212 75

39 WDH

(2/76)

-40 -190 -115 150 40 -85 -22 125

41 WDH

(8/76)

-40 -90 -65 50 -40 -125 -82 85

43 WDH

(2/77)

-70 -30 -50 40 -40 35 - 2 75

45 WDH

(8/77)

-75 -80 -78 -45 -35 -40 10

47 WDH

(2/78)

-395 -295 345 100 -395 -225 -310 170

49 WDH

(8/78)

355 275 315 80 -400 -405 -402

51 WDH

(2/79)

540 -415 -478 125 -505 515 510 10

53 WDH

(8/79)

-95 -95 -95 -125 85 -20 210

Av . Range , R

:: 1-40

1-53

129

102

140

118

123
National Bureau of Standards Special Publication 591, Proceedings of the National Conferenc

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

STANDARD THERMAL PERFORMANCE

TESTING PROCEDURES

JACK D. VERSCH00R

RESEARCH AND DEVELOPMENT CENTER

JOHNS-MANVILLE SALES CORP.

DENVER, COLORADO 8021T

Formal standardized procedures for thermal performance tests were established at the Johns-Manville R&D Center as part of a program to assure reliability of test data. A number of requirements had to be met by the

procedures. While the tests are based on ASTM and other similar recognized

standard methods, the procedures had to be sufficiently detailed to insure

consistently reliable results by all operators, some of whom may not have

had long-term experience in thermal testing. Likewise, the procedures had

to cover a wide variety of types and forms of materials, test equipment

and temperature ranges. A complicating factor was the inter-relation

between general lab procedures applicable to all activities, procedures

for checking general purpose equipment and specific thermal test equipment, thermal test methods, and test specimen preparation procedures for

specific types of materials.

This paper describes the formats for a series of formal cross- referenced standardized procedures for thermal performance tests. The

paper also details some of our experiences in preparing and implementing

these test procedures.

Key words: Laboratory test procedures; quality assurance lab test; standard test methods; thermal performance test.

1. Introduction

The basic objective of a quality assurance program for a testing laboratory is to

insure the consistent reliability of the test

results obtained. This was also the objective of the program at the Thermal Conductivity Laboratory, or K-LAB, at the JohnsManville Research and Development Center in

Denver, Colorado.

under development, and competitive product

In addition, a limited amount of outside o contract testing is performed. Thermal te procedures involve establishing steady-sta

heat flow under prescribed conditions, and

then measuring the heat flux and the surfa

temperatures. Related properties, such as physical dimensions and density, are also

determined because of their influence on thermal performance.

The K-LAB performs a wide variety of

thermal performance tests. The scope includes various types and forms of materials,

which may be tested individually or in com- bination with other materials and at various

temperatures. Samples submitted for test include materials from regular plant production

for routine quality control, new products

121+

The K-LAB has long operated under in

formal standard thermal test procedures.

Many of these served as the basis for star dard test methods, subsequently published

ASTM, ANSI, or ISO standards. Formerly, 1 testing quality assurance relied on the si and long-term experience of the K-LAB ope]

tors with their testing equipment. While
: ' : test procedures generally followed the re-

'>i quirements of the published standard test

methods, there were many minor deviations

where experience had demonstrated published

test criteria were either too lax or overly

restrictive, or special needs had to be met.

The incentive for the J-M K-LAB to es- tablish formal test procedures came from

several directions . There was need for further improvements in precision and accuracy.

The volume of testing had increased. Laboratory personnel have been added, promoted,

| and transferred, resulting in a transient

! personnel situation. The requirements of

j the national and state thermal test laboratory certification and accreditation programs

were another factor. An internal requirement

in establishing the formal K-LAB standard

procedures was that they be compatible with

j other J-M and R&D systems already in existence. For example: The R&D Information

Center had previously established procedures

to be followed in recording data in permanent lab notebooks, and also had general

| formats for reports. Thus, the K-LAB procedures had to complement the other systems.

2. Standard Procedures

Thermal performance testing was analyzed

carefully to identify areas where standard

procedures were needed to meet testing quality assurance requirements. The task was to

separate common activities of the various

steps in testing, and to rearrange them in

logical combinations that would lead to the

preparation of cohesive standard procedures.

For example, many of the details of laboratory operation apply to all activities irrespective of the complexity of the test procedure involved or the type of material being tested. Certain pieces of laboratory

equipment, such as temperature indicators

j and recorders, are in general use rather

• : than being associated with a particular

piece of thermal test equipment. Further,

•: except for physical size consideration, the

.

: steps involved in the preparation of speci-

. mens for testing is more a function of the

character of the material being tested than

i the particular piece of thermal test equipment that will be utilized.

What evolved was a plan for six classes

of standard procedures, with the recognition

that extensive use of cross referencing

would be utilized. These are enumerated in

Table 0.

Table 0 - Classification of Thermal

Test Procedures

1. General lab operation

2. Check adjunct apparatus

3. Check thermal test apparatus

it. Thermal test methods

5. Test specimen preparation

6. Standard reference check specimens

The first area covers the general operation of the K-LAB, including as standard

procedures, authorization of tests, handling

of samples received, retention of data and

information, and reporting of test results.

The nature of thermal testing requires the

use of both common or general test equipment

that might be used in several test procedures, and specific test equipment for only

one type of test. Checking procedures for

general or adjunct test equipment are cov- ered in the second area, while specific test

apparatus are in the third area. The actual

testing procedure to be followed in conducting a specific test is covered in the fourth

area. Procedures for the preparation of

specimens for the test are covered in the

fifth area. This is divided by types of

materials, i.e., batts and blankets, rigid

board and block, pipe insulation, and re- fractory materials. In the sixth area, the

preparation and handling of standard reference check specimens is covered.

The decimal system was utilized to

classify types of procedures within a class,

and sections within a particular procedure.

Thus, as shown in Table 0, the number

h identifies a procedure as a thermal test

method. The number it. 3 is a thermal test

method for a specific piece of test apparatus, in this case, the high temperature

calorimeter complying with ASTM-C182; the

number it. 3.7 is a section within that test

method.

Details of the formats used in each of

the areas follow.

3 . Format s

The first area of standard procedures

deals with laboratory operation. This has

125
"been divided into two procedures; 1.1 covers

the general mechanics of the K-LAB operation,

while 1.2 details reports. Because of the

importance of the proper written communication of technical data by testing laboratory,

reporting procedures were put into a separate

procedure. In this procedure, general re- quirements to be met by all reports, in addition to specific formats for the five

types of reports issued by the K-LAB, are

covered. Table 1 details the format for 1.1

and 1.2 procedures.

Table 1 - Format - Laboratory

Operation Procedures

1.1 General K-LAB procedures

1.1.1 Scope - applicability, categories of

procedure

1.1.2 Hazards - general caution

1.1.3 Index - procedures

1.1.1+ Test Authorization

1.1.5 Assignment - assignor, responsibility

1.1.6 Schedule - priority of tests

1.1. T Test Data - reference to permanent

notebook records

1.1.8 Report - reference to 1.2

1.1.9 Sample disposition - retention

1.1.10 K-LAB information - types of records,

retention

1.2 K-LAB reports

1.2.1 Scope - all written communication of

thermal performance data

1.2.2 Types - description, distribution

limits

1.2.3 Style - no set writing style, units

(US vs. metric)

1.2.1+ General requirements - applicable

all reports

1.2.5 Format - test report

1.2.6 Format - development report

1.2.7 Format - external report

1.2.8 Format - memo report

1.2.9 Format - letter report

An essential part of the quality assurl ance program of any testing laboratory is

the periodic checking and calibration of

testing equipment used. Calibration record!

of all testing equipment in the Material

Performance Section or physical testing

group at the J-M R&D Center, which includes 1 the K-LAB, have been put on computer. A

typical print-out, (Figure l), describes th'l equipment including its range, usual loca- j tion and person responsible. Calibration

information includes who usually performs

the work, recommended frequency, dates of

past calibrations and the next due date. Jj periodic review of the computer files, attn

tion can be focused on equipment past its | normal check or calibration date.

Standard pieces of laboratory equipment, such as scales and balances, are ser-l viced routinely by an outside contractor,

frequently the manufacturer's local representative. No attempt is made to dictate

checking procedures for this equipment.

Primary electrical and temperature standar

and measuring equipment are sent out peri

odically to a qualified local metrology laj

oratory with standards traceable to NBS.

Again, no attempt is made to dictate procedures followed. Only for that test equi:

ment which is checked and calibrated by

K-LAB staff have standard procedures been

written. Those pertaining to adjunct test]

apparatus are covered by the 2.x series of

procedures (see Table 2 for format details

Similarly the procedures for checking therj

mal test apparatus are in the 3.x series

(see Table 3)

Table 2 - Format - Adjunct apparatus

check

2.x. 1 Scope - type(s) apparatus covered

2.x. 2 Hazards - general caution, and spe- cific hazards (if any)

2.x. 3 Apparatus - description of apparati

checked

2.x. h Frequency - frequency of checks

2.x. 5 Test equipment - list of test equi

ment

2.x. 6 Procedure - detailed check/calibra

tion procedure, precautions, recorj

explanatory drawings

126
Table 3 - Format - Thermal test

apparatus check

3.x. 1 Scope - type(s) of thermal test

apparatus covered

3.x. 2 Hazards - general caution, and

specific hazards (if any)

3.x. 3 Standard Test - reference to standard

test method, such as ASTM, and list

of exceptions (if any)

3.x. h Apparatus - description of apparatus

checked

3.x. 5 Frequency - frequency of checks (may

he various levels of detail)

3.x. 6 Log - permanent records to he kept

3. x. 7 Procedure - test equipment and standards required, detailed check/calibration procedure, precautions,

explanatory drawings (daily checks

of thermal test equipment are cov- ered as part of area h on thermal

test methods )

.

The procedures to he followed in actually conducting a thermal performance test

are described in the h.x series. When the

program is completed a separate procedure

will have been prepared for each type of

thermal test apparatus. Details of the format for 4.x procedures are shown in Table h.

Table h - Format - Thermal test

methods

4. x. 1 Scope - type(s) apparatus covered,

materials typically tested, temperature range and size and other limits.

h.x. 2 Hazards - general caution, and spefic hazards (if any)

H.x. 3 Standard test - reference to standard test method such as ASTM, and

list of exceptions (if any)

4.x. 4 Specimen - test specimen description,

and reference to standard procedures

5.x on preparation

k.x. 5 Apparatus and equipment - list of

test equipment required

h.x. 6 Procedure - Initial startup, daily

checks of apparatus, detailed test

procedure, precautions, data requirements (including examples) and daily

shut down

h.x. 7 Calculations - equations, procedure,

hand calculator program (if used)

and example

U.x. 8 Figures - explanatory drawings

Procedures describing the preparation

of thermal test specimens are divided by

type of material to be tested, with separate

procedures for batt /blanket , loose- fill, and

rigid board and blocks. Because of special

requirements, separate procedures were also

prepared for pipe insulation and for refractory materials. Details of the format 5.x

procedures for test specimen preparation are included in Table 5-

Table 5 - Format - Test specimen

preparation

5.X.1 Scope - type of material covered

5.x. 2 Hazards - general caution, and specific hazards (if any)

5.x. 3 Standard test - reference to standard

test methods, such as ASTM, and list

of exceptions (if any)

5.x. k Apparatus and equipment - list of all

equipment required

5.x. 5 Procedure - selection, I.D., cutting

to size, drying and other pre-test

conditioning, physical measurements,

data requirements, and precautions

5.x. 6 Calculations - equations, procedure,

hand calculator program (if used),

and example

5.x. 7 Figures - explanatory drawings

Preparation of procedures for the last

area, the preparation and checking of standard reference test specimens, has been

held in abeyance. An increasingly important

portion of thermal performance testing re- lies on heat flow transducers to determine

thermal flux. In the past, NBS has made

available a moderately heavy density glass

fiber board in 1 inch thickness only, as a standard thermal reference material (SRM).

This standard was adequate for calibration

of heat flow transducers when thermal testing accuracy requirements were far less

stringent than today. Current technologyhas shown that many discrepancies can occur when an apparatus calibration based on the

current NBS thermal SRM is extended to materials of different characteristics than

the standard, and especially when extended

127
to the greater thicknesses common in today's

insulation standards.

NBS now has a program under way to develop equipment which will permit them to

offer a wider variety of thermal SRM's. Out- need for 6.x procedures will he inversely

related to the success and timing of the NBS

program.

k. Observations

The preparation and implementation of

standard thermal performance test procedures

has required a large investment in time on the part of the personnel involved. This

was not unexpected, as the need for accuracy

and completeness of presentation was recognized early. Limited availability of time

necessitated establishing priorities, so that the more important procedures were attacked first. K-LAB personnel had to con- tinue regular testing activities at the same time

The question of how much detail to include in a standard procedure requires

judgement. A fine balance is required between providing sufficient detail to cover

all requirements, and overloading a procedure with the "obvious." One device which

worked well in the preparation of an initial

draft, was for an experienced operator to use a pocket dictating machine to record

every step he followed as he went through

a test. This was edited to insure essential

details were covered and non-essential ones eliminated.

all operators having standard procedures

available. A perhaps not so obvious benefit

was the critical analysis of each step during the preparation of a standard procedure.

The following questions were asked many

times: Is this step really necessary? Is there a better way in terms of improved

accuracy or reduced operator time?

We plan continued efforts to complete

the balance of the standard procedures. In the future, we also plan periodic review,

to insure that the standard procedures represent the best methods for achieving reliable test results and that the operators are following the methods established.

The author is deeply appreciative of the

K-LAB staff for their helpful comments on the general plan, and for their assistance

in preparing most of the actual procedures

Without their extra efforts the program

could not have reached its present stage.

You will note reference in each procedure to the hazards involved. While not

particularly dangerous as such, thermal

testing does involve possible personal contact with elevated temperature surfaces,

high voltages, dusts, and cutting devices

such as a band saw. Over the years, the

R&D Center has maintained an enviable safety

record within Johns-Manville . This has been

achieved only by constant attention to the

hazards involved and careful work habits on the part of the personnel.

The availability of computer-controlled

drafting facilities expedited the preparation of explanatory diagrams. These were prepared quickly, were of excellent clarity,

and in contrast with photographs showed only

those details considered pertinent. Figure

2 is an example of a computer drafted diagram.

The obvious benefit of improved testing

precision and accuracy has been realized by

128
LU

a:cj

COLU

Z

LUz

:2S3

LU

—

Li.

COo

o

o

CiLu

CO\

<;

r-i pvM

C3O

21

Ci

Ui

Li

LU

CO

LU

_!

sr. <I

i Ci<l>

r-4 o

CJZ

<ILU

cnLuZ

Li

lii O

S

LU

_J3

-co Z

o

Cl. COLU

Li

LU

LU

>-

CJ

_i

LL.

r-4 LU

_!

=5<r

=—

i

Z

<:o

LU

_!S&f- <TZ —!:LLJ Lir-i <LLU O C3

<rLi

LUZZ

C3C0 LL.O DO

r-i LU—

:CC •Z

o

co

LU

Ci

CO fJ3 •— LlO

-Jo

! O

z

lUt-

:>:

Lif-Oi

rn

COCD

<Z

LU

Li

LU

CJ

•C

LiLU

Lio

LiCilus:

CD

LUSi s <L\ 3^OO

UJo

2:5:

>-z

a

Ni

Ciz

<z

<£. }-

Ni CO

CiCJ

CO

CO

z

CJ

<cNi

CiLU

Ci<L

Ll. Z

COy

<r

CO s

CO

CJ

LUCi

CJ

CJz

CJ

CO

LU

Ci

z o

<c •-*

•Ci <r

<r

rCO

_5

-i O

Ci

ii

CO

ii

CJZ

<E

_iz

23

<r

CJ

crNi—

Ci

-

'Si

-U

LJ

LUCi

CO

CJ

caz

3

LU

CJ

r-i O

z

LU

CO

COz

LUZ

a

LUZ

Figure 1 - Example of Computer Calibration Record

129
Siffi Johns-Manville

TEST METHOD

JM - 1Q74

PRO Du C T Insulating Firebrick, Castables

Refractory Fiber

TEST METHOD NO.

493-4.3

TESIThermal Conductivity using

J-M Calorimeter

PAGE NUMBER

(HIS ME T H OO IS THE SAME AS Til AT SPECIFIED IN THE FOLLOWING STANDARD SPECIFICATIONS:

STORAGE

TANK

130
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

LABORATORY PERFORMANCE EVALUATION

A NEW LOOK AT QUALITY ASSURANCE IN THE TESTING LABORATORY

DONALD J. MC CLAIM

QUALITY ASSURANCE STAFF, BARTON INSTRUMENTS

A UNIT OF INTERNATIONAL TELEPHONE AND TELEGRAPH CORP.

CITY OF INDUSTRY, CALIFORNIA 91016

Safety and liability lavs, consumer awareness and Federal laws

governing test requirements necessary to qualify a component for use in

Nuclear reactors has caused a new look at laboratory performance. The

independent laboratory is preferred by legal departments since it avoids

the connotation of conflict of interest in any litigation which might

result if a component fails. Quality Assurance organizations have re- vised their methods of audits and approach to the control requirements

needed by independent test laboratories. This reexamination has shown

that in many cases laboratory performance has not been what the laboratory customer expected. This paper presents not only the requirements

for nuclear qualification but assesses those performance standards cur- rently found in test laboratories. Further and most important, it presents several solutions to the problems. These are solutions that have

been used successfully. They require, as most human endeavors, understanding and cooperation as well as acceptance of certain responsibilities

on both the laboratory and the laboratory customer.

Key words: Advent of safety, liability and Federal Nuclear laws requiring

tests; consumer awareness and legality; customer and laboratory responsibilities; inadequacies of test laboratories; Institute of Electrical and

Electronic Engineers Specifications; laboratory capabilities; laboratory

performance evaluation; manufacturers response to laws and consumers; one way to solve the problem; pretest quality planning; tests required for

nuclear or safety related products; what the laboratory customer expects

in a test.

1. Introduction

1.1

The advent of the safety and liability

laws and Federal laws governing safety related products for Nuclear applications has

caused Quality Assurance organizations to revise quality programs in the area of qualification testing of products.

1.1.1

Included in these laws is Title 10,

Code of Federal Regulations, Part 50,

131

appendix B, commonly referred to as 10CFR50,

which is applicable to all activities affecting the safety related functions of

nuclear power plants and describes the quality program requirements for design, con- struction, and operation of these structures,

systems and components. In addition, ANSI

Ni+5.2 was prepared for general industry use

as a guide, for implementation of 10CFR50,

appendix B. The design control requirements

of these documents includes methods of design verification, one of which is qualification testing.
1.2

Cursory discussions with legal experts

leads to the conclusion that qualification

tests by an independent test laboratory is

preferred, since it precludes allegations in

litigation regarding conflict of interest in

test results. Therefore, the independent

laboratory performance becomes extremely

important

1.3

Qualification testing of safety related

products is specified by the Institute of

Electrical and Electronic Engineers specifications, IEEE- 323 and 3hk.

1.3.1

In addition, revised laws for product

safety and liability have resulted from consumer awareness and have necessitated changes

in manufacturers ' thinking regarding product

qualification.

1.3.1.2

As a result, qualification testing of new products, review and possible requalification of existing products has become mandatory in many company operating policies.

1.3.1.3

Such has been the case at ITT Barton

whose written policy is to perform qualification testing on every product we sell.

1.3.1. it

In addition, the testing imposed by the

Institute of Electrical and Electronic Engineers specifications IEEE-323 for safety re- lated products such as instruments used in

Nuclear power plants is stringent. I will

briefly discuss these tests in order that

you will be more able to appreciate the se- rious nature of what I have to say relative

to the subject of laboratory performance.

2. Test Classification

2.1

The IEEE-323 specification defines a Class IE safety related product as, "The

safety classification of the electric equipment and systems that are essential to emer- gency reactor shutdown, containment isolation reactor core cooling and containment

and reactor heat removal or otherwise are

essential in preventing release of radioactive material to the environment."

2.1.1

Each type of Class IE power equipment

must be qualified by analysis, successful

use under similar conditions, or by actual

test to demonstrate its ability to perform

its function under normal and Design Basis

Events (DBE's).

2.1.2

Design Basis Events are earthquakes,

high radiation levels and loss of coolant

accidents

2.1.3

Most manufacturers of Class IE safety

related products qualify by a combination c analysis and testing.

3. Test Requirements

3.1

Specification IEEE-323 requires ageing or simulated end of life test typically 10

to kO years, as the first of a series of

tests that must be performed sequentially.

ITT has found that very few laboratories

have this capability or specialized knowledge .

3.1.2 Ageing

3.1.2.1

The ageing requirement exists so that

by test it can be proven that an instrumen

to be used in nuclear applications can ope

ate within design parameters and operation

accuracies for 39 years, 36k days, 23 hour

and 59 minutes and then suffer a Design

Basis Event such as an earthquake or loss

of coolant accident and the instrument wil

still operate during and after the Design

Basis Event.

3.1.3 Radiation

It follows, therefore, after completi

of ageing tests the instrument is subjecte

to the next IEEE-323 test of Gamma Radia-

| tion at the level of 200 x 10° Rads. For

comparison, the human body limit of Gamma

radiation if hOO Rem. The radiation tests insure that materials such as wiring and

132
electronic components will not fail due to accumulative Gamma radiation absorbed over

the life of the instrument and therefore,

will still perform the intended function at end of life during a Design Basis Event.

3.1.*+ Seismic

3.1.U.1

The next test in sequence is the Seismic simulation test specified in IEEE- 3^

and is conducted on a biaxial seismic simulator. Testing consists of 5 operational

basic earthquake or OBE tests of approximately 35 seconds during which the instrument operation and outputs are remotely recorded. These OBE tests are followed by one

SSE or Safe Shutdown Earthquake test of

approximately 35 seconds but at higher re- sponse acceleration in peak G's. Frequencies are enveloped from 1 Hz to 100 Hz at nominal peak response acceleration of 2 to

15 G's and are conducted in both horizontal

and vertical axes.

3.1-5 Loss of Coolant Accident

3.1.5-1

The last test is again contained in IEEE specification 323 and is called the

LOCA or Loss of Coolant Accident test. Here

the specimen is subjected to saturated steam

at 320°F and left to operate while recording

outputs and operational functions hopefully

within design limits for as much as 100 days

The specimen under test is sprayed with a caustic solution during the 1st 2k hours of

LOCA test.

3.1.6 Test Laboratory Performance

3.1.6.1

Although we have had many successful

tests at such laboratories, we have also

experienced such problems as inadequate

test planning, test set-ups that fail during

test, inadequate audit of test functions by

laboratory personnel, missing or erroneous

log keeping and data recording and even the

destruction of our test specimen. ITT Barton and other quality assurance organizations have reexamined test laboratory re- lationships with prospective customers by

improving communications through mutual

understanding of what is expected during

the conduct of a test. Responsibilities

are incumbent on both parties. Although

there are several solutions, an approach

which has been successful is pre-test quality planning.

3.1.6.2 Pre-test Quality Planning

3.1.6.3

A joint meeting, prior to entering into

any contract, which must include technical,

purchasing/contracts and quality assurance personnel during which the following subjects, at a minimum must be discussed:

- Referenced quality program

requirements

.

- Errors and omissions in proposed p.o. and specifications.

- The technical approach to test methods.

- Instrumentation requirements.

- Test set-up adequacy.

- Test set-up check-out prior

to test start.

- Test process checklists.

- Test deviations.

- Test data recording and re- tention . - Test reports.

3. 1.6. it

Contractor responsibility must embrace

the philosophy to assist the laboratory in

achieving timely and specified performance

results

.

3.1. T Laboratory Performance

Requirements

3.1.7.1

The laboratory customer is purchasing

expertise, not just facilities and test personnel assistance. The experience of the

laboratory can do much to prevent aborting

tests, assuring required test conformance,

reducing costs, and improving laboratory

competitiveness

.

3.1.7.2

Instrumentation and test set-ups must

provide the intended results. Assurance of

this results from set-up check-out, prior to

notification to contractors that the laboratory is ready. Such action insures mini- mum costs to both laboratory and contractors .

3.1.7.3

In complex tests such as those referenced in IEEE-323, which include simulated

life ageing, gamma radiation, seismic and

loss of coolant accident (LOCA), simulation

requires quality control improvement . The

133
use of process control checklists has been

successful. These lists are orderly checks

of test set-up, instrumentation operation,

test sequences and ancillary and monitoring

equipment to observe and record data and

specimen performance.

3.1. 7. It

Contractor responsibilities include

pre-test quality planning for the same reasons mentioned. Quality personnel must be

involved at the earliest possible time.

3.1.7-5

Test deviations in laboratory instrumentation and test specimen failures re- quire immediate verbal notification and

laboratory quality procedures should require written, timely submission of details,

including cause and corrective action, with

provision for contractor engineering and

quality personnel approval signatures as applicable .

3.1.7.6

In testing to verify design of safety

related devices under 10CFR50, reporting of

defects and non-conformances as required by

10CFR21 will be imposed and provides severe penalties for failure to report. Laboratory

customer quality personnel must monitor to assure performance to requirements by laboratory personnel, instrumentation and test

specimens

3.1.7-7

Test data are the end product purchased

by the contractor. Data must be complete,

accurate and legible. A chronological test

log must be maintained by laboratory personnel with entries of time, date and signatures .

3.1.7.8

There must be a pre-stated mutual agreement for data retention. Some data are re- quired to be held for years and contractual

relief to the laboratory should be given in

writing. Original data being turned over to

the contractor one year after approval of

final test report has worked well.

3.1.7.9

Retained data in the laboratory must

be protected and considered proprietary.

Pass or fail information and results are of

value to contractor competition.

3.1.7-10

Review of test data must be made at pre-determined intervals consistent with

the agreed upon test plan

h. Conclusion

h.l

Therefore, inclusion of those items

presented and in particular, the pre-test

quality planning efforts will assure that

products tested will conform to advertised

performance specifications and/or con- tractual obligations.

The author wishes to acknowledge the encour ment and technical assistance of:

Mr. G. Welt, Director of Quality

Assurance

Mr. R. DeLon^ Project Engineer

Mr. E. Romo

,

Engineering Manager

Ms. P. Davenport, Document Specialist

Ms. C. Ruiz

,

Quality Assurance Clerk

ITT BARTON INSTRUMENTS

13k
SESSION VI

EVALUATION

PROGRAMS

A N D

SYSTEMS

CHAIRMAN: JOHN R. DISE

NATIONAL BUREAU OF STANDARDS

National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

A MEASUREMENT ASSURANCE PROGRAM - THERMOMETER CALIBRATION

GEORGE T. FURUKAWA AND WILLIAM R. BIGGE

TEMPERATURE MEASUREMENTS AND STANDARDS DIVISION

CENTER FOR ABSOLUTE PHYSICAL QUANTITIES

NATIONAL BUREAU OF STANDARDS

WASHINGTON, D.C. 20234

The platinum resistance thermometer (SPRT) , calibrated in accor- dance with the specifications of the International Practical Temperature

Scale of 1968 (IPTS-68), is the international standard for temperature

measurement in the range 13.81 K (-259.34°C) to 903.89 K (630.74°C) to which the United States subscribes. Such SPRT's are used to calibrate

other SPRT's, "industrial resistance thermometers", or other types of

thermometers, or used directly where highly accurate temperature mea- surement is desired. There are many laboratories in the United States

that calibrate SPRT's and other resistance thermometers at various

levels of accuracy. A measurement assurance program (MAP) on the

calibration of SPRT's has been developed at the National Bureau of

Standards. In the program, three pre-cal i brated SPRT's are shipped to

a participating laboratory for calibration. Upon return, the SPRT's

are recalibrated at the NBS and shipped to the next laboratory. The

two NBS calibrations (before and after) are compared with those of the

participant. This paper describes the results of the MAP measurements

on the SPRT's.

Key Words: Calibration; check thermometers; fixed points; International

Practical Temperature Scale of 1968; measurement assurance program;

platinum resistance thermometer; reference thermometer; thermometer.

1. Introduction

The National Bureau of Standards (NBS)

has the responsibility to establish, maintain, and develop the standards for temperature measurements for the nation's industrial

and scientific communities and, in cooperation with other national laboratories, to establish international uniformity of

temperature scale. The dissemination of

these standards and the methodology of

temperature measurements is achieved through

appropriate publications, consultations,

calibration services, Measurement Assurance

Programs (MAP), and temperature measurement

seminars 1. The International Practical

Temperature Scale of 1968 (IPTS-68) is the

present basis for internationally uniform

values of temperature [1,2]. 2 The IPTS-68

is based on the assigned values of the

temperatures of specified, reproducible

equilibrium states of pure substances

(defining fixed points) and on standard

instruments calibrated at these temperatures.

Between the fixed-point temperatures,

interpolation equations relate the

because of the increased interest in

thermometry the last few years, these

precision thermometry seminars are now being held twice a year at the NBS, in March and in September.

2 Figures in brackets indicate the

literature references at the end of this

paper.
indications of the standard instruments to temperature values of the IPTS-68. The

platinum resistance thermometer is the

specified standard instrument from 13.81 K (-259.34°C) to 903.89 K (630.74°C), the

platinum-10% rhodium alloy versus platinum

thermocouple from 903.89 K to 1337.58 K (1064. 43°C), and the optical pyrometer above

1337.58 K. The assigned values of temperatures of thirteen equilibrium states that

define the IPTS-68 are given in table 1 (the

states and values with asterisks). For the

specified interpolation equations of the

various temperature ranges, see reference

[2]. As part of the study of properties of

substances or of the direct investigation of

other suitable thermometric reference points,

temperatures of various equilibrium states

other than those of the defining fixed

points are constantly being published.

(Research is conducted at the NBS to improve

the realization of the defining fixed points,

as well as other suitable fixed points.)

Most of the temperatures of the equilibrium

states are obtained with thermometers that

have been calibrated in terms of the IPTS68. Recently the Consultative Committee for

Thermometry (a subcommittee of the International Committee of Weights and Measures)

published a compilation of such measurements

for secondary thermometric reference points

between 13 and 3700 K [3], Values up to the

gold point (1337.58 K) from the compilation

are included in table 1. The vapor pressure

versus temperature relation for some substances given in the compilation are excluded from table 1. These secondary reference

points may be employed to check the calibration of thermometers calibrated in terms of

the IPTS-68 or used to calibrate such

thermometers or other thermometers for use over a limited temperature range. [In the

calibration of thermometers other than those

specified by the IPTS-68, the temperature

intervals of calibration must be small

enough so that the temperature unit (i.e.,

the size of the degree) of the thermometer

scale is the same as that of the thermometer

specified by the IPTS-68.]

This paper is a brief report of the MAP involving the calibration of platinum

resistance thermometers in accordance with

the specification of the IPTS-68. (Henceforth, a platinum resistance thermometer

that meets the IPTS-68 specifications for a standard will be referred to as SPRT). The

range of calibration of the SPRT that will

be discussed is from the oxygen point [i.e.,

90.188 K (-182.962°C)j to 903.89 K. This is the range where most of the temperature

measurements are made and where the longstem SPRT is normally used. Also, for

convenience, this MAP concerning the calibra

tion of SPRT's will be referred to as SPRTMAP. Since the calibration of the SPRT can

be accidentally changed during use, procedures for continued testing of the integrity

of the measurement process (internal MAP) is

described as part of the SPRT-MAP.

SPRT's are delicate instruments. They

must be handled in a manner that avoids

tapping or bumping against table tops or other objects. They should not be vibrated.

When used properly, they will give accuracie

of ± 1 mK over most of the specified tempera

ture range. Such SPRT's are used to calibra

other SPRT's, industrial type platinum

resistance thermometers (PRT), or other

types of thermometers (e.g., thermistors,

mercury thermometers, quartz thermometers,

thermocouples, etc.) or used directly where

the highest accuracy of temperature measurement is required (e.g., standard cell

baths, length standards, electric power,

calorimetry, etc.). Hence, the SPRT is the

standard to which temperature measurements

over a broad range is referred. A MAP is needed at every level of thermometry accurac whether the accuracy that is claimed is in

the calibration laboratory or in the user laboratory or plant. Although this paper

deals with SPRT-MAP, the concept, the

repeated check measurements, and the documen

tation of the MAP are applicable to other

levels of thermometric accuracy.

To disseminate the IPTS-68 to the

thermometry community, SPRT's of various

laboratories are calibrated at the NBS in

accordance with the IPTS-68 at the specified

fixed points (oxygen point, triple point of

water, tin point, and zinc point). There

are a number of standards laboratories in

the United States that calibrate SPRT's at accuracy levels of 0.01 to 0.001 K for inhouse use or for other organizations. Their

SPRT calibrations are traceable to the NBS

through their reference SPRT's that have

been calibrated at the NBS. These laboratories have one or more of the specified

fixed points, or have baths that are con- trolled near the fixed-point temperatures,

for calibration of the SPRT's. The operatic

of the fixed points is checked or standardiz

using the NBS calibrated reference SPRT.

Where baths are used, the reference SPRT is used to calibrate the test SPRT's directly.

These reference SPRT's are handled extremely

carefully, e.g., some are "hand carried"

between the NBS and the standards laboratories. However, in spite of careful

documentation of traceability of the refer- ence SPRT to the NBS, the calibration may

change or there may exist unknown weak links

in the measurement process of the laboratory

itself. The SPRT-MAP has been designed to

138
TABLE I

Some of the Reference Points of Thermometry'

ui i i uri um State

Temperature

T(K), IPTS-68

U f~i 1 1 *i ~] -1 h\ y* n 1 im ^ 5 d 5 e cyu 1 1 1 Dr 1 urn State

Temperatu re T(K), IPTS-68

o M ( TD \* e-Hp \\r

)

1

J

RI *

0 1 Hnny (TP) \ \ r

)

97A "?nR

n Hh ( TD ^ 2 *

' QR7 z) D 1 Hnny (FP\ [r Y j 234 L Jt .014 314

D Yd 1 / 0.49* T /~ 0 D n "i n +

1 Lc rU 1 1 1 L 27"? 1

c, CIO. ID C-H2 i Dr

J

. 2ft* CO hn^Uo ^td^ \ 1"

)

97"3 1 £* CIO. ID n_H f RD ^

ri — 2 ' 20 3Q7 PhDnnvwhon7Qno 1 TD

1

rllCIIUAjr UciIacMc \ 1 r ^ 3nn mn

0U ( tr rv -ft ^ 2

V ^ ' )>* P

J

23 867 Ga f MP 302 JUL . Q21

1

Np (TP^ 24 562 JUL ^ t" p ^ m P n i n 1" * J LCOIII IV 1 lit 373 "IR*

lie \ d r y 27 11 02* U 1— Rpn7nir ^ ri H f T P ^ 30^ R2

n ( tr a--v^ 8D1 Tn (FP

1

) 42Q 7R4

n (tp^* ^2 \

1

r

' 54 3fil *

0 u 1 Oil (FPl* ^ r r J ^DR 1 1 ftlJU J .1101

1^2 l

1 r J 63 1 4fi 1 HO Ri (FP^ R44 RQ2

N CrpI 1^2 \ Di i 77 OHH Cr\ !fP) \'YJ RQ4 ?Rft R3 7QR* Ph ^ CD ^ r \Y Y ) DUU . D D£ MrAr f\uYRPl

j ft7 0 / 9Q/1 Hnny ^brj (&P\ D£3 . O

1

n f r d ^ * l>2 ) Qn 1 RR* £n ^ r K

;

U2L . /

0

CU (TP) 90 686 ^ ( RP) 71 7 R9/1 Kr (TPl 115 770 LU-HI CUucClIC [r\Y j Oil 1 . f 1 Kr (BP) 119 800 Sb (FP) 903 905

Xe (TP) 119 388 Al (FP) 933.607

Xe (BP) 165 090 Cu-Ag eutectic (FP) 1052.

7

5 co

2 (SP) 194 ,674 Ag (FP)* 1235.08*

C0

2 (TP) 216 .580 Au (FP)* 1337.58*

Reference points with asterisks (*) are the defining fixed points of the IPTS-68.

^Reference points listed without the asterisks (*) were taken from reference [3]. Where more than one value was given, the average was taken. Values only up to the gold point

1337.58 K are given here; for values above this temperature, see references [2,3].

c Except for triple points and where pressures are indicated, the values of temperature

are for equilibrium states at a pressure of 1 standard atmosphere (101,325 Pa).

TP = triple point, BP = boiling point (vanishingly small vapor fraction),

CP = condensation point (vanishingly small liquid fraction), FP = freezing point,

SP = sublimation point, e-h^ = equilibrium hydrogen, n-h^ = normal hydrogen,

tr = solid phase transition, and MP = melting point.

e The Ar (TP) may be used as an alternative to the 0

2 (CP) and the Sn (FP) may be

used as an alternative to the steam point to obtain the IPTS-68 in terms of the SPRT.

139
compare the output, i.e., a standards laboratory's SPRT calibrations, with those of the

NBS.

2. IPTS-68 in the SPRT Range

The SPRT temperature scale of the IPTS68 is based on the resistance ratio W(T)

defined by

W(T) = R(T)/R(273.15K), (1)

where R(T) is the SPRT resistance at IPTS-68

temperature T (kelvins) and R (273 . 1 5 K) is the resistance at 273.15 K (0°C). [The

Celsius temperature t(°C) is defined by

t = T - 273. 15.

]

3 The SPRT resistor must be

"annealed pure platinum", supported in a "strain-free" manner, and have a value of

W(100°C) not less than 1.39250. The resistor must be hermetically sealed inside a protective sheath filled with dry gas. The

interpolation equations are expressed in terms of W. Hence, to determine the value

of temperature on the SPRT scale, the

resistance of the SPRT must be measured at the temperature of interest and at 0°C.

2.1 Interpolation Equations

From 0°C to 630.74°C (903.89 K) , the

values of IPTS-68 temperature are defined by

t = V + 0.045( t7100^C ^)(Xtt M WT

/ t' -.w t"

1 419.58 6 C

1)

1)( 630.74°C D°C (2)

where t' is defined by

W(t') = R(t')/R(0°C) = 1 + At" +

2 Bt' .

Equation (3) is equivalent to

V - l[W(t')-l] + 6(^)

hoo°c

where

3 Henceforth, temperatures will be

expressed interchangeably in °C and K, based on the definition.

(3)

(4)

A + B X 100°C (5)

and

B(100°C)

A + B X 100°C

Equation (2) is independent of the SPRT; it

is intended to adjust the SPRT temperature

scale, given by equation (3), to be closer

to the thermodynamic temperature scale. The

thermometer constants R(0°C), A, and B are determined from calibration at the triple

point of water, the steam point or the tin

point, and the zinc point. (Henceforth, the

triple point of water will be abbreviated

TP). The constants a and 6 are derived from

the constants A and B according to equations

(5) and (6), respectively. When the steam

point (100°C) is used in the calibration,

the constant a can be obtained directly from

equation (4) , i.e.,

a = [W(100°C)-1]/100°C = [R(100°C) -

R(0°C)]/R(0°C) X 100°C (7)

After the thermometer constants A and B are obtained, tables of W(T) at integral values

of t are calculated for the SPRT from 0°C to

631 °C using equations (2) and (3).

Below 0°C, the W(T) relation of the

SPRT is given by

W(T) = W*(T) + AW(T)

where W*(T) is the reference function

defined by

20

(8)

T =

I a.[UnW*(T) + j=0 J

3.28)/3.28] J K (9)

and AW(T) is a deviation function which is a polynomial in T. The coefficients of

equation (9) are given in reference [2].

The reference function was developed from

the results of comparison of SPRT's against

gas thermometers. The range from 13.81 to 273.15 K (0°C) is divided into four subranges, each with its specified deviation

function. The deviation function for the

subrange 90.188 K (-182.962°C) to 273.15 K,

iko
the range included in the SPRT-MAP, is given

by

AW(T) = b(T-273.15 K) +

e(T-273.15 K)

3 (T-373. 1 5 K) (10)

where the constants b and e are determined

from calibration at the TP and the measured

deviations AW at the steam point (100°C) and

at the oxygen point or the triple point of

argon. If the tin point instead of the

steam point is used, W(100°C) for the SPRT

should be calculated from equations (2) and

(3). (Note that for the tin point, t = 231. 9681 °C but t

1 = 231.9292°C). After the

thermometer constants b and e are obtained,

tables of W(t) are calculated for the SPRT

from -183°C to 0°C using equations (10),

(9), and (8).

2.2 Fixed Points

Of the four fixed points of the IPTS-68

that are involved in the SPRT-MAP, the TP is

the most important. Although R(0°C) for the

resistance ratio W may be obtained from

measurements at the ice point, in routine

measurements it is most accurately obtained

from the resistance measurements at the TP,

R(TP). In an intercomparison of fifteen TP cells at the NBS, the averages of readings

on each cell were all within a range of ±0.1 mK [4]. Also, in another test, the range of

readings obtained over three days on each of

eight TP cells was 0.01 mK [5]. In terms of

the resistance value at 0°C, 0.01 mK corresponds to 4 X 10" 8 R(0°C).

The freezing points of tin and zinc are realized at the NBS by using samples of

these metals that are nominally 99.9999

percent pure. These samples are issued as NBS Standard Reference Materials SRM-740 and

SRM-741 , respectively. The freezing points

of cells that have been assembled using

these metals have been found to agree within

±0.1 mK [6,7]. During the freezing process

the furnaces are controlled about 1 K below

the freezing point to give a complete freeze

duration of 14 to 16 hours. Normally, the

SPRT's are calibrated during the first 50

percent of the freeze, during which time the

temperature change is not more than 0.2 mK. At the NBS, the SPRT's are calibrated

at the oxygen point by a comparison method

in terms of reference standard SPRT's, using

a copper block apparatus that is maintained

during the measurements as nearly isothermal

as possible close to the oxygen point. The

results of repeated measurements over a two- year period on "check SPRT's" show the

standard deviation of the oxygen-point

calibration to correspond to ±0.16 mK. (See

the section on check SPRT's.) This oxygenpoint calibration can be compared to the

calibration methods used by many standards

laboratories, where the SPRT's are calibrated

in baths that are controlled close to the

fixed points in terms of reference SPRT's.

3. Internal MAP

In addition to international comparison

of temperature scales by interchanging

calibrated SPRT's, the NBS maintains an internal SPRT-MAP. This consists of a bank

of fixed-point cells (TP, tin point, and

zinc point), which have been tested and

found to yield temperatures that agree

within ±0.1 mK (see section 2.2), and a bank

of reference standard SPRT's (for the oxygen

point and for temperatures between 13 and 90

K). The results of the measurements with

these cells demonstrate that the fixed

points used at the NBS are highly reproducible and, since materials of exceptionally high purity are used, the temperatures

obtained are close to those of pure sub- stances. As part of this internal SPRT-MAP,

measurements are obtained with check SPRT's

whenever the fixed-point devices are used in SPRT calibration. The following is a brief

description of the internal SPRT-MAP at the

NBS (for details, see references [8,9,10]).

3.1 Check SPRT's

In a calibration laboratory it is extremely important that the laboratory's

own measurement errors be small enough so that the results of the users of the calibrated instruments would not be affected by

those errors. This means that the measurement process of the calibration laboratory

must always be under control and must produce results that always lie within these

allowable limits of measurement error. (At

the NBS, procedures are constantly being

sought to improve its measurement accuracy

and to eliminate possible accidental calibration errors.) In order to establish the

validity of a single calibration on a new SPRT, i.e., that the variability of the

measurements is within the allowable limits

of measurement error and that the calibration process is under control, there must

be redundant measurements of a control or check SPRT. (A check SPRT is defined herein

as an SPRT used to monitor the measurements

at a temperature.) Any abrupt shift in the

calibration of the check SPRT would indicate

that there may be problems with the fixedpoint devices or with the measurement

instrument, possible change in the

ikl
calibration of the reference SPRT or the

bath temperature control is not operating

properly, or that the check SPRT was accidentally "bumped." The long record of

measurements on the check SPRT gives information on the limits of calibration error of

any new SPRT.

The procedure that is employed at the

NBS to monitor the calibration of every

batch of SPRT's is as follows. Different

long-stem type check SPRT's are assigned to the zinc-point, tin-point, and oxygen-point

calibration measurements. To obtain the

corresponding values of W at these fixed

points, the resistances of the check SPRT's

are also measured in the TP cell with every

measurement obtained at the zinc, tin, or oxygen point. In the cases of the zinc and

tin point measurements, the freeze is initiated using the check SPRT and the first

equilibrium readings are obtained on the

check SPRT. This is followed by calibration

of the test SPRT's (usually six). After all

of the test SPRT's are calibrated, the check

SPRT is read again in the freezing-point

cell. The second reading with the check

SPRT for the given freeze must not differ

from the first by more than 0.5 mK. Usually

the difference is not more than 0.1 or 0.2

mK because the calibrations are made during

the first 50 percent of the freeze (see

section 2.2). Also, the readings should be

consistent with those obtained in earlier

freezes. (For those laboratories that

employ NBS calibrated SPRT's to standardize

the freezing points of tin and zinc before

calibrating the test SPRT's, the reference

SPRT's can serve as their check SPRT's.)

With the oxygen-point calibration apparatus,

a second reference standard SPRT is calibrated in terms of the working reference

standard SPRT in the same manner as the test

SPRT's. The standard deviations of the

values of W of the check SPRT's obtained

over a two-year period were ±0.28 mK, ±0.30

mK, and ±0.16 mK for the zinc, tin, and

oxygen point calibrations, respectively.

3.2 Secondary Reference Points

Secondary reference points have many

applications. If the calibration of a reference SPRT is uncertain, rather than to be recalibrated, it can be tested at one of

the secondary reference points in the range

where temperature measurements are to be

made. To detect any possible accidental

calibration errors at the defining fixed

points (e.g., transcription error) in

routine calibrations, measurements at one or more of the secondary reference points could

be included as part of the calibration

process. Although the calibration would not

be in accordance with the IPTS-68 specif

cations, SPRT's may be calibrated at the

secondary reference points for limited

interpolation use. Differences between sue secondary calibrations and the IPTS-68

calibration should be expected, depending

upon the error in the temperature value of

the fixed point; upon variations in the

realization of the fixed point, e.g., sampl

purity or technique; and upon differences i| the SPRT's. To minimize the error in

realization of the secondary fixed points,

sealed secondary reference cells are being

developed at the NBS [11,12].

The selection of a secondary referena

point, instead of a defining fixed point fi testing the calibration of a SPRT, is

expected to be based on its greater conve- nience and its temperature being in the

region where measurements are to be made.

However, the TP is a highly reproducible a easily realized defining fixed point;

measurements at the TP should detect any

large changes in the calibration of the

SPRT. (In the calibration of SPRT's at th|

NBS the readings are obtained in the following order: R(TP), R(Zn), R(TP), R(Sn]

R(TP), R(0

2

), and R(TP). When the SPRT's

are to be used only above -50°C, the last ;J two readings are omitted. Over a two-year

period the average of the standard deviations of three or four R(TP) readings of

SPRT's that were calibrated was about ±0.1

mK, which indicates the stability that coi

be expected in SPRT's over a short time

interval [10].)

SPRT-MAP 1

It is essential that a laboratory

engaged in the calibration of SPRT's have

least two reference SPRT's and several che i>; SPRT's to monitor the calibration process

Before one of the reference SPRT's is seni

to the NBS for calibration, it should be

calibrated in terms of the second referenn

SPRT (or else the calibrations of the SPR' should be compared.) Upon return of the

first reference SPRT from the NBS, it should be calibrated again in terms of th^ second reference SPRT. If the SPRT's werf

handled with sufficient care during use,

two calibrations, before and after, shoul

agree within the measurement uncertaintie

(In a series of tests over six years, whe

SPRT's were exposed intermittently to -18

0, 100, 232, 420, 450, and 480°C, the

calibrations at the fixed points did not change by more than 1 or 2 mK.) Also, if

the second reference SPRT had previously

been calibrated in accordance with the IPjl 68 (or at the NBS), the two calibrations

should agree with the new NBS calibration

lk2
ic:

i li

ncfi:

f:

:

Accidental agreement of the two user-laboratory calibrations can result if the same error is made in both calibrations, e.g.,

the fixed point may be 0.01 K colder than

the assigned value. However, the comparison

with the NBS calibration should reveal this error, i.e., the measured value of W and the

value of W given in the NBS calibration will

be different for the "temperatures." When

the second reference SPRT is sent to the NBS

for calibration, similar comparison calibrations should follow. Hence, by such procedures a "do-it-yourself" internal MAP can

be established. Obviously, more calibrated

reference SPRT's can be used to improve the

comparison statistics of the reference

;SPRT*s. In the calibration of test SPRT's,

check SPRT's must be employed to monitor the

calibration process each time. The results

jon the check SPRT's can also support the

stability of calibration of the reference

SPRT, particularly if the two SPRT's are of

Ithe same quality. (Hence, as previously

tated, the measurement results on the check

PRT's serve to monitor the complete calibration process.

)

The SPRT-MAP service provided by the

NBS requires calibration of three NBS owned

5PRT's. The participating laboratory is requested to calibrate the SPRT's by procures that are normally employed in the

laboratory for the calibration of SPRT's or

,i n measurements using SPRT's. (Some laboratories use SPRT's principally to calibrate

thermometers other than SPRT's and only

-arely calibrate SPRT's.) Work sheets that

accompany the SPRT's request information on temperature standards that are employed

[i.e. , fixed points and reference SPRT's

With their calibration constants), geometry

fhjjind other data on the fixed point devices

;;

:

; ind baths that are important in temperature

:r measurements (e.g., immersion depth, use of

: iluminum or graphite bushing, temperature

.^equalization blocks in baths, etc.), the

;: pertinent measurement data, and the final

alibration tables that are normally furn' shed to "customers." Immediately upon

f Receipt of the SPRT's, and just prior to shipment of the SPRT's back to the NBS, the

./« aboratory must determine the R(0°C) of the

PRT's at two currents (1 and 2 mA). These

ot Joeasurements in conjunction with the calibration data will show (1) whether the SPRT's

.;piere harmed during shipment, (2) whether the

PRT's were carefully handled by the labora-

'notrfory» (3) whether the measurement procedure

-f the laboratory is standard, and (4)

hether the resistance unit is close to our national unit of resistance. When the

ions Vibration is. completed, the SPRT's are

eturned to the NBS for recal ibration and

phipment to the next participating laboraave

tory. The SPRT-MAP comparison is usually

reported back to the participating laboratory within three to six weeks after receipt

of the calibration results, depending upon

the calibration situation at the NBS and

other factors.

The laboratories that have participated

thus far in the SPRT-MAP are AVCO (Systems

Division), Boeing Aerospace Company, Duke

Power Company, Leeds and Northrup Company,

Pratt and Whitney Aircraft, Rockwell International (Autonetics ) , Rosemount, Inc.,

Sandia Laboratories, U.S. Air Force Aerospace Guidance and Metrology Center, U.S. Army White Sands Missile Range, U.S. Army

Metrology and Calibration Center, U.S. Navy

Eastern Standards Laboratory, and U.S. Navy

Western Standards Laboratory. Two of the

laboratories have completed their second

SPRT-MAP. The range of accuracies that was desired by the laboratories was about ±0.01

to ±0.001 K; a few laboratories felt that

accuracies within ±0.1 K were satisfactory

in some of their applications.

As part of the SPRT-MAP, calibration

tables of W(T) at integral values of temperatures were received from the participating

laboratories. The values at -183, 100, 232,

and 420°C were compared with those obtained

at the NBS. The values at these temperatures

are sufficiently close to the fixed points

(oxygen, steam, tin, and zinc points,

respectively) to consider them equivalent to those obtained at the assigned temperatures

of the fixed points. Figure 1 shows the

300

! too

50

10 -

5-

o -

S 1 3

i 0.5 I

3

I

<0.1

SPRT-MAP

o

o ©

z 9 6

©

o :

v o

©

6 «

-183 100 232 CALIBRATION TEMPERATURES. °C (IPTS420

68)

ih3

Figure 1. Standard deviations of

calibrations of various laboratories from the National Bureau of

Standards calibration of three

SPRT's near the fixed points.
standard deviations of each of the various

laboratories from the NBS calibrations for

the three SPRT's at the above temperatures.

Most of the standard deviations are shown to be within ± 5 mK, but there are some deviations that are substantially larger. Very

few laboratories made measurements at the

steam point (the measurements with the tin

point cell being easier); however, the

calibration values at the steam point are compared for all of the laboratories.

Many of the laboratories have been

shown to be performing excellent measurements; the SPRT-MAP results have added

confidence to their work. Where problems

occurred, such as adjustments for differences in barometric pressure on the freezing

point or for radiation losses, they were

readily detected and corrected. The SPRTMAP work sheets helped to document better

the measurement process. Where the SPRT-MAP

showed that the measurements are not up to expectation, it has helped to indicate

possible improvements in the measurement

process.

5. Conclusion

To maintain high quality measurement

assurance in SPRT calibration, it is essential to have

1. ) temperature standards (fixed

points or reference SPRT's) that

are compared regularly in terms of

the national standards,

2. ) redundant sets of measurements to test continuously whether the

calibration process is under

control (e.g., measurements on check SPRT's that are always

included in the group of SPRT's

being calibrated),

3. ) comparison of the calibration

(i.e., the output) with the NBS.

All three support each other to attain

measurement assurance.

For users of calibrated SPRT's (i.e.,

those who are not necessarily involved in

the calibration of SPRT's) the calibration

of the SPRT may be checked by employing one or more of the many fixed points listed in

table 1 in the temperature range of interest.

Such measurements also provide more detailed

measurement assurance.

6. Future Thermometry MAP

A number of improvements will be made

in the current SPRT-MAP service provided by

the NBS, particularly in the area of providing a documentation of the SPRT-MAP

concept and measurement procedure to serve

as a guide for maintaining a good SPRT-MAP.

(For details of the measurement process

reference should still be made to [6,7,8,

9,10].) The document will show samples of

good work sheets to help formulate a suitable measurement process. The document will

include samples of internal MAP for continuous monitoring of the process (see section

3.). Most of the laboratories that have

participated thus far in the SPRT-MAP follow

internal MAP procedures. To reach a broader

thermometry community, an industrial platinur

resistance thermometer MAP is being developei

to serve those that use industrial type

PRT's instead of SPRT's. The MAP procedures

are expected to be very similar to those of

the SPRT-MAP.

REFERENCES

[1] The International Practical Temperature

Scale of 1968. Adopted by the Comite

International des Poids et Measures,

Metrologia 5_, 35-44 (1969).

[2] The International Practical Temperature

Scale of 1968. Amended Edition of

1975. Metrologia 1_2, 7-17 (1976).

[3] Crovini, L., Bedford, R.E., and Moser,

A., Extended list of secondary reference points, Metrologia 1_3, 197-206

(1977).

[4] Furukawa, G.T., and Bigge, W.R.,

National Bureau of Standards (1975),

Unpublished data.

[5] Furukawa, G.T., and Pfeiffer, E.R.,

National Bureau of Standards (1979),

Unpubl ished data.

[6] Furukawa, 6.T., Riddle, J.L., and

Bigge, W.R., Investigation of freezing

temperatures of National Bureau of

Standards tin standards, in Temperature, Its Measurement and Control in Science and Industry, Vol. 4, ed. by

H.H. Plumb, Instrument Society of

America, Pittsburgh, 1972, pp. 247-263.

[7] Furukawa, G.T., and Riddle, J.L.,

Comparison of freezing temperatures of

the National Bureau of Standards SRM740 zinc standards, Document No. 78-16

Comite Consul tatif de Thermometrie,

twelfth session, May 1978.
! [8] Riddle, J.L., Furukawa, G.T., and

Plumb, H.H., Platinum resistance

thermometry, Nat. Bur. Stand. (U.S.),

Monograph 126, 129 pages (April 1973).

[9] Furukawa, G.T. Riddle, J.L., and Bigge,

W.R., The International Practical

Temperature Scale of 1968 in the region

13.81 K to 90.188 K as maintained at the National Bureau of Standards, J. Res. Nat. Bur. Stand. 77A, 309-332

(1973).

r

. [10] Furukawa, G.T., Riddle, J.L., and

Bigge, W.R., The International Practical Temperature Scale of 1968 in the

region 90.188 K to 903.89 K as maintained at the National Bureau of Standards, J. Res. Nat. Bur. Stand. (U.S.)

80A , 477-504 (1976).

[11] Mangum, B.W. and Thornton, D.D., The

gallium melting-point standard, Nat. Bur. Stand. (U.S.) Special Publication

481 , 35 pages (June 1977)

.

,:6 [12] Furukawa, G.T., Realization of triple

point of argon in a transportable

sample cell, Document No. 78-14, Comite

Consul tatif de Thermometrie, twelfth

session, May 1978.

I

!
National Bureau of Standards Special Publication 591, Proceedings of the National Conference on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

THE CERTIFICATION OF BUILDING PRODUCTS

IN THE UNITED STATES

LESLIE H. BREDEN AND LYNFORD K. SNELL

OFFICE OF ARCHITECTURE AND ENGINEERING STANDARDS

DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT

WASHINGTON. D. C. 20410

Based on a survey of certification for building products United

States, HUD has developed qualifications and procedures for administrators in carrying out certification programs. In addition, a glossary of terms, used in the HUD Certification of Building Products

Program, has been developed. In the United States, manufacturers

provide certification of a product to a particular standard while

adminstrators provide a validation of the manufacturer's certification. Several examples of certifications will be discussed. As an adjunct to certification, HUD and specific program administrators,

have developed basic criteria to be used by administrators in approving laboratories. An example of the use of these criteria in the

carpet program will be extensively discussed. It is anticipated that

a NVLAP for the testing of carpet will be developed. Finally, HUD's

view on administrators, building products, future certification and

laboratory accreditation programs in the United States and international certification will be presented.

Key words: Building products; certification; laboratory accreditation.

Presently, there is an intense interest

in the certification of building products by

code officials, architects, and government

agencies associated with the building industry. The purpose of this paper is to provide

an understanding of the Department of Housing

and Urban Development's certification of

building products programs and to provide

some basis for the establishment of a national policy in the United States for certification of building products and for other

products when the system can be determined

applicable.

Certification, of a product to a standard, refers to the authentication, or attesting by a manufacturer or vendor, that

the product being sold complies with a particular standard. It is important to

note that the manufacturer or vendor is the

only one that can make this declaration or certification.

i

it

ire, M.

!!ti

•:C!

'5

; t. Figure I illustrates three types of

statements of compliance found in certifi-'

cation programs. The first type is the

self-certification statement where the

manufacturer declares that the product is

in compliance with a standard. For exampl<

the manufacturer makes the applicable test: on the product and exercises the quality

control deemed necessary to assure that

the production continues to comply with th - standard.

HUD BUILDING PRODUCTS CERTIFICATION PROGRAM

MANUFACTURER

Product or Sorvico lllil

NOARD 7fr

CERT STATEMENTFICATION

OF COMPLIANCE

s

-

2 StU • ANSI ZMj\ | i THIRD PARTY |

'is

'"it

1

11+6
The second type is similar to the

first, except more formal, and extensive

records are kept in accordance with established pratices. such as the American

National Standards Institute Standard

jZ 3^ .2 (which requires recording of the

frequency of testing, description of the

manufacturing process and/or quality control

programs.

)

The third type of statement of compliance is the administrative type where neutral

third parties actually collect , review and

file test reports; check the manufacturer's

quality control program and validate the

manufacturer's certification.

Third party organizations or administrators are sometimes used to conduct certification programs to validate the manufacturer's

declaration of certification. Such programs

; involve required test frequencies and plant

I visits to assure that quality control proced- ures are being followed. The administrator/

/alidator does not certify products, but

-ather gives a testimonial based on test

"ecords and inspections that the manufacturer

_s complying with the provisions of the

standard. The administrator/validator allows

|;he use of his registered mark or label for as

! .ong as the product complies with the standard

ind withdraws the use of the mark or label

lihen noncompliance is established. This clear

listinction between producer and the administrator's involvement is important. The

reducer is ultimately responsible for the

' ertification of the product and the adminitrator is responsible for the conduct or

Operation of the specified certification program. Minimum requirements for testing

frequencies, plant visits, etc. are set forth

n the certification program and can be

ncreased in intensity at the discretion of

•ne administrator relative to the producers

ability to comply with the standard. The

ublic is informed of complying products

hrough the periodic issuance of a directory,

Isting or marking of products. Noncomplying

roducts are deleted from such issuances

.

pus, it is only the producer or vendor that

ssumes the risk, the liability and the

rofit from the sale of the product. It lould be pointed out, however, that in some

oreign countries, the government itself acts

3 the administrator or may grant a waiver

d administrators relieving them of any

liability responsibilties. In these cases the administrator can also certify the product along with the producer.

Appendix I contains a list of some of

the third party organizations validating

building products.

Generally, there are three classes of

administrator type certification programs.

The first class only serves as the manager

or administrator of the program but does not

do any testing. In this case, the administrator approves other laboratories to do

the testing and makes audits to verify the

quality control level in the plant. The

second class of administrators do not

approve other test laboratories but does all

of the testing itself. Finally, class three

administrators are characterized by having

an open system where a nonprofit organization is chartered to sponsor a program and

the administration is put out for bid. An example of this type of program is the

Safety Glazing Certification Council (SGCC)

program and the Insulating Glass Certification. Council (IGCC) program for insulating

glass. In some instances, there are com- binations of types within a sample certification program.

Certification programs, involving

administrators or third party type,

generally perform the following ten duties:

1. Administrates a program relating to a specific standard;

2. Obtains a license agreement between the

manufacturer and the administrator;

3. Approves laboratories for testing;

4. Reviews initial test data and quality

control data of manufacturers to determine acceptance;

5. Issues notices of validation;

6. Allows the use of administrators' labels

on manufacturers products;

7. Issues a directory or listing or prod

ucts in the program when applicable;
8. Samples and periodically visits the

plant to determine continuing compli- ance with the standard;

9. Decertifies products that do not com- ply with the standard after intiial

acceptance

10. Maintains an appeal process or challenge procedures in cases of dispute.

In order to clarify many of the other

terms used in certification, a list of terms

used in the HUD Building Products Certification Programs appears in Appendix II.

The use of certification programs is

growing in acceptance because they are meeting the needs of the vaious segments of the

building industry. One of the main purposes

of any certification program is to provide

some assurance to the user that the product

actually delivered complies with the designated standard and continues to do so on an ongoing basis.

For manufacturers, certification removes

the likelihood of unfair competition based on

spurious claims and provides for a fair and

equitable basis for marketing a product.

Certification programs also assist the ac- ceptance and establishment of new products.

Many times, new products, without a long

history of performance, are not accepted by

users. Certification can provide some information and assurance that the product at least complies with a standard and that a quality control program exists for continued

production. Often this assurance can be

used in place of a long history of usage.

Obviously, however, certification programs

can never guarantee 100 percent assurance or performance but can provide users with some degree of confidence that the product com- plies with the standard and will perform

satisfactorily provided that the reference

standard is meaningful.

Certification programs also provide

manufacturers with a creditable assurance

for their products that they alone could not

provide. These programs can, in addition,

provide some assistance to manufacturers'

quality control programs and prevent con- fusion and misrepresentation to consumers

with regard to claims made for the product.

In addition, certification programs used in

conjunction with viable standards reduce the

chances of premature failure thereby enhancing cost effectiveness and conserving energy

when a product fails and has to be replaced.

Not only is the cost of the produce and labor involved but also the loss of energy

used to manufacture the product. Consumers

also benefit directly because it aids in

making value judgment choices and reducts

the need for duplicate evaluations by different users, by providing impartial and ac- curate information on the product. Many

times these judgment choices involve concerns

for health and safety.

A classic example of the use of certification program in achieving many of the

benefits described above, is the HUD Carpet

Certification Program. Originally, 60 percent of the carpet provided to HUD- insured

mortgage homes did not comply with the

existing standard even though manufacturer

declared their carpets complied with the

standard. After one year, many of these

carpets deteriorated and necessitated costly

replacement. Many honest and reputable car- pet manufacturers lowered their quality

levels because they could not compete on an equitable basis with manufacturers that did

not actually comply with the standard. The

purchase of carpet was particularly difficult for users, because at the time of delivery it was impossible to visually deter- ; mine if the carpet was in compliance with

the required standard. With the advent of

a carpet certification program, the percentage of carpet in the HUD program not

complying was reduced to less than seven

percent. Most of the seven percent noncom- plying carpets, however, involved only a few factors which were of a marginal fail- ure nature, consequently, the amount of car- pet that would constitute any major failure

would be negligible. It appears this level

j of compliance cannot be lowered without a more intensive testing and monitoring pro- , gram which would be extremely expensive for

manufacturers and consumers. We do not believe this is necessary.

f

11+8
Basically, the cost of the carpet

certification program to the manufacturer

has been less than a cent per square yard of

carpet. The program is generally accepted

by the carpet industry with about 80% of the

mills participating. Many skeptics thought

the program would stifle innovation by

imposing new demands for rigid quality controls but each year the number of qualities

in the program continues to grow. Architects,

designers and others are finding the benefits

of this program are many are using the administrator directories in choosing a carpet.

In balance, carpet manufacturers have

been pleased with the results of the program

because they can compete on a fair and

equitable basis. Concomitantly, the growth of

carpet sales has continued. Consumers are also content with the program because they can determine characteristics of any carpet in the

program by checking the marked number on the

back with the listing in the administrator's

directory. In addition, a challenge procedure and the testing of field samples has

added creditability and prestige to the

program.

One of the key elements, and the most

expensive part for an administrator to carry

out in a certification program, is approval or accreditation of testing laboratories.

Generally, the following criteria are used for

approving or accrediting laboratories in con- junction with a proficiency or collaborative

reference program and an initial audit visit.

1. Description of the laboratory including

name and address along with an organizational chart of the laboratory;

2. Declaration by the laboratory of its

ability and services to be offered;

3. Possession of a laboratory operations

control manual;

4. Submission of the name and qualification

of responsible personnel including a record of training and an examination of

competency;

5. Description and photograph of equipment

including calibration, verification and

maintenance records;

6. Examination of the laboratory for

environmental and safety concerns;

7. Possession of sample test reports

including procedures for testing and

copies of all relevant standards;

8. Possession of a record of an appeals

process available to users in cases of dispute.

Our programs require that the laboratories be approved only for the particular

tests they are able to conduct. This

follows the NVLAP concept which the Department of HUD has encouraged and supported

by submitting a carpet laboratory approval

program for the Department of Commerce

action.

Internationally, there are several

groups engaged in certification activities.

The International Standards Organization

(ISO) has established a technical committee

(Certico TC 149) to address the concerns

of certifying products on a world wide

basis. This Committee has issued several

documents including guides for the

principles and practices of certification

along with guidelines for assessing the

technical competency of testing laboratories.

One of the goals of Certico is to develop a universal system for the acceptance of

products complying with ISO standards. This

system would remove trade barriers and reduce

the need for redundant testing by several

countries. Presently the committee is workin four areas: (1) Self and third party

certification; (2) Quality assurance; (3)

Marking; (4) Basic documents.

It is anticipated that a product for

international certification will be suggested

in the coming year. Along with Certico, an

ad hoc committe on International Laboratory

accreditation (ILAC) is developing procedures

for accrediting laboratories. Their efforts

currently are confined to (1) legal problems

of accrediting laboratories; (2) organizing

a directory of organizations accrediting

laboratories; and (3) obtaining a list of

terms and needs of laboratory accreditation.

Finally, there is an effort by the United

Nations (UN) to develop harmonization and

149
control rules for buildings and building

products. While efforts of these organizations are long term, it appears they

are progressing at a rapid rate. It is

anticipated, however, that the immediate form

for certifying products internationally will

be bilateral memoranda of agreement letters.

Looking at the future of certification

in the United States, it appears there is a need for a national policy for the certification of products. One method to initiate

this effort, is to obtain an intergovernmental agency agreement on a standard

criteria for the certification of products.

It is very frustrating for manufacturers and consumers to determine the necessary

requirements for a particular product. For

example, insulating materials must comply

with the requirements of HUD, DOC, CPSC,

DOD, DOI, etc. What is needed is a single

document that contains all of the requirements necessary for certification. It is

HUD's belief that the responsibility for

building products rests with HUD. This does

not mean that the functions of the other

agencies are usurped, but rather there should

be some focal point on certification for

manufacturers of building products. Equally,

in the private sector, there should exist the

development of a national certification com- mittee to provide for a unified approach on

the certification of products.

We believe a viable single system for

certification and laboratory accreditation

can be established through a quasi private

sector-government endeavor that would be

recognized and used nationally. Further, it

would provide a base for international

recognition of these programs.

150
APPENDIX I

PRIVATE BODIES WITH CERTIFICATION PROGRAMS

AAMA * Architectural Aluminum Manufacturers Association

ADL Associated Dallas Laboratories

AGA American Gas Association

AHAM Association of Home Appliances

APA American Plywood Association

ARDL * Akron Rubber Development Laboratory

ARI Air-conditioning and Refrigeration Institute

ARL Applied Research Laboratory

ASSE American Society of Sanitary Engineers

AWIA # American Wood Inspection Agency

AWW * A.W. Williams Inspection

AWWI # American Wood Window Institute

BHMA Builders Hardware Manufacturers Association

CLFMI Chain Link Fence Manufacturers Institute

EPTL # El Paso Testing Laboratory

ETL * ETL Testing Laboratories

HPMA * Houston Chemical Service

HVI Home Ventilating Institute

IAPMA International Association of Plumbing & Mechnical

IGCC Insulating Glass Certification Council

ITL * Industrial Testing Laboratory

JPMA Juvenile Products Manufacturers Association

MEA * Metallurgical Engineers of Atlanta

NAHB * National Association of Home Builders

NPA * National Particleboard Association

NSF National Sanitation Foundation

NWMA * National Woodwork Manufactures Association

PFS Plywood Fabricator Service

PTL Pittsburgh Testing Laboratory

RIS Redwood Inspection Service

SL * Skeist Laboratories

SPIB Southern Pine Inspection Bureau

SW Southwestern Laboratory

TCA Tile Council of American

TECO * Timber Engineering Company

TPIT * Timber Products Inspection & Testing Services

UL Underwriters Laboratories

UST * U.S. Testing

WCLI * West Coast Lumber Inspection Bureau

WGS * Williams Grademarking Services

WHL * Warnock Hersey

WQA Water Quality Association

WSC Water Systems Council

WWPA * Western Wood Products Association

* Registered in the HUD Building Products Certification Program

151
APPENDIX II

HUD BUILDING PRODUCTS CERTIFICATION PROGRAM

DEFINITION OF TERMS

Acceptance Sampling Plan - A specific plan that states the sample size (number) and the

associated acceptance and rejection criteria.

Accreditation Agency - An organization that conducts and administers an accreditation systeir

and recognized to have the competency to evaluate that testing facilities are adequate to

conduct tests.

Accredited Test Facility - A test facility which has been approved by an accreditation

agency.

Accreditation - A public declaration by a national agency that a testing laboratory has

the qualifications necessary to perform a specific testing activity.

Administ rator - Designated by the sponsor of a certification program to perform the execute

dut-tes required to manage the affairs of that program.

Agency - An organization designated by some authority to engage in testing, inspection,

and/or administering certification programs.

Applicant - The person or (organization) requesting accreditation of its certification

program

Approval - The acceptance of a testing laboratory or administrator by an approving body.

Association Test Facility - A test facility organized and operated by a trade or profession;

organization and providing test services for products.

Audit - A formal verification.

Authority - The responsible person in charge of the work or the authorized representative. [

( Calibration - The comparison of two objects, one of which is of a known or accepted values,

to determine any deviation from the one being compared.

Certification of Approval - A seal, statement of conformance, label, classifiction, directo

listing, endorsement or other affirmation that a product complies with applicable

standards.

Certification - The procedures by which products or services become certified.

Certification Mark - The sponsor's or validator's sign or symbol that identifies a product I or service as being certified. The certification of compliance is by the producer or vendq

Certification Program - An organized system where products or services may be certified to specific standards on a uniform equitable basis.

Certified - Attested by the producer or vendor under the procedures of a certification

program as satisfying the requirements of the referenced standard.

152
Certifier - The producer or vendor who certifies that the products or services meet the

requirements of a referenced standard.

Collaborative Reference Testing - A program where a uniform material or a products of

controlled characteristics, is distributed to participating laboratories. The laboratories

test the materials or products and submit the test results for evaluation.

Commercial Test Facility - Provides a test service under contract or on a fee basis.

Competence - Requisite abilities including requirements such as training, experience,

initiative, supervision needed and communication skills.

Compliance Assurance - The process of appraising the manufacturers quality control

program in conjunction with monitoring, auditing and inspecting the product or services

being offered.

Conformity Certification - The action of certifying by means of a certificate or mark that

a product or service conforms to a standard.

Consensus - Substantial agreement of those concerned with the scope and provisions of a standard as judged by a recongized authority. Consensus implies more than the concept of a sample majority, but not necessarily unanimity.

Criteria - Detailed requirements for examining and evaluating a laboratory or an administrator to assure competency.

Evaluation - A formal judgment of the ability of a testing laboratory or administrator

to perform satisfactorily based upon analytical data or other judgment factors.

Evaulation Team - A group of individuals recognized as competent and equipped to evaluate

testing facilities or act as a referee on technical questions.

Field Inspector - A trained person capable of obtaining samples representative of production

and knowledgable in the evaluation of quality control records, test methods and instrument

calibration.

Independent Test Laboratory - A testing laboratory which has no orgnization tie or financial

interest in the manufacture, vending or promotion of the specific product or services

being tested.

Inspection - The process of measuring, examining, gaging, or comparing the product

or service with the specified requirements.

Inspection Level - The relative sample size or number for a given amount of production.

Label - A sticker, nameplate, or similar printed statement appearing on a product or packaging material which provides instructive or required information.

Laboratory Accreditation - An authoritive action for evaluating test facilities and

approving those judged competent to perform specific tests.

Laboratory Environment - The aggregate of all conditions, such as temperature, humidity,

and lighting, that may influence the testing or performance of the product.

153
Laboratory Evaluation - A process of inspecting, examinining or assessing the organization

to determine its capability and competence to carry out specific tests.

Laboratory Evaulation Criteria - Statements prescribing the organizational resources, the

equipment and facilities, the operational procedures, and the minimum techical performance

levels required of a testing laboratory for accreditation.

Laboratory Examination - The process of inspecting and obtaining information in order to

judge the laboratory's capability for performing specific tests.

Licensee - The manufacturer or vendor authorized by contract to use the sponsor's or administrator's certification mark for validation purposes.

Logo - A symbol, label, hallmark or statement, authorized by sponsor or administrator

indicating certification or compliance with the specified standard by the producer or vendc

Mandatory Standard - One which is established by law and the observance of which is

compulsory.

Manufacturer - See Producer.

Manufacturer's Test Facility - A test facility organizationally affilated with a manufactur

and providing test services on the product and services of that manufacturer.

Monitoring - The act of observing and reviewing certification programs.

Person - Any individual, consumer, organization, educational intitution corporation, sociel

Federal, state or local government agency.

Prerequisite Testing - Preliminary testing carried out before a formal collaborative or

other testing program is carried out.

Procedural Guide - A description of the certification program containing the administratis

testing program, standard reference, and other information necessary to carry out a laboratory or administrator function.

Producer - A person or orgnization that produces, manufactures, assemblies, or packages a product or offers a service that is capable of being tested for conformance with the

applicable standard.

I Product - Any manufactured item, construction, material or service described by a specific

standard

Product Certification - Attesting product conformance to a referenced standard using an organized program.

Proficiency Testing - A systematic testing program where a sample is measured by a number : of laboratories or a random series of samples tested to determine the accuracy of measure

ment.

[ Qualified Personnel - Competent persons adequately trained in the applicable test procedur

equipment operations, and calibration methods.

Quality Assurance - A systematic approach for all actions necessary for the manufacture of a product or performance of a service to comply with a designated standard.

15»t
Quality Control A means by which a manufacturer achieves conformity to a given standard

to prevent the production of defective items.

Reassessment - Repetition of the complete examination and evaluation initially required of a laboratory or an administrator seeking accreditation.

Redactory or Technical Library - A library of standards, codes and other technical information necessary for carrying out a testing procedures, and verifying that the product

complies with standard.

Reference Facility - Recognized as professionally competent properly equipped and staffed

to evaluate the technical content and adequacy of a test method and acts as a referee

on technical questions.

Reference Material - A material, substance or device whose intrinsic properties are used

for physical comparison.

Self-Certification - The certifiction of a product by a producer or vendor of the product

being certified.

Sponsor - An organization under whose authority a certification program is developed,

promulgated, and financed and with whose name the certification program is identified.

It may also own or be in process or registering a certification mark.

Standard - A rule for course of action established by an authority, custom, or general

consent. It would include a physical embodiment of a unit of measurement.

Technician - An employee assigned to perform the actual operations of a testing or isnpection program.

Testing - An evluation or a determination of a physical or performance characteristic

for the purpose of establishing conformance to a standard.

Testing Agency - An organization staffed and equipped for measuring, testing, or inspecting

products

.

Test Data - The quantitative and qualitative information derived from testing.

Test Method - A description of the procedures, equipment, and methodology for determining

if a product is in conformance wholly or partially to a standard or parts of a standard.

Testing Facility or Laboratory - A person or organization whose functions include testing,

analyzing, or inspecting products or services. This also includes the physical plant

required to house the test equipment and records.

Third Party Certification - The certification of a product by an organization which has no organizational tie or financial interest in the manufacture, vending, supplying or promotion

of the product being certified.

Traceable to a Primary Standard A documented chain of comparisons connecting a working

standard to a primary standard being maintained by some responsible organization.

User - An organization or activity which makes use of a test laboratory or an accreditation

agency

.

155
National Bureau of Standards Special Publication 591, Proceedings of the National Conference on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

LICENSING PROGRAMS FOR FIELD TECHNICIANS AND CONCRETE LABORATORIES

IN MASSACHUSETTS

GEORGE H. BRATTIN, CHIEF STAFF ENGINEER

CHARLES G. HANAFIN, STAFF ENGINEER

THE COMMONWEALTH OF MASSACHUSETTS

STATE BUILDING CODE COMMISSION

BOSTON, MASSACHUSETTS 02108

The state-wide licensing of field concrete technicians and

concrete testing laboratories has been in effect in Massachusetts

since January, 1975. As of this date there are twenty licensed concrete testing laboratories and approximately 1250 licensed field

concrete technicians. Prequalifying agencies were designated by the

State Building Code Commission to insure a proper mechanism to license

technicians and laboratories. Proven test methods, procedures, and

standards were adopted, wherever possible, to give the programs credence

and to ease their implementation. The need, development, operation,

rules and regulations, and evaluation of the programs are set forth.

Key words: Accreditation; concrete testing laboratories; field concrete

licensing; prequalifying agency; testing agency.

1. Introduction

The first known state-wide licensing

programs for field concrete technicians and

concrete testing laboratories were implemented in January, 1975, by the promulgation

of the Massachusetts State Building Code

under the jurisdiction of the State Building

Code Commission as authorized by the

Massachusetts legislation (Chapter 802 of

the Acts of 1972).

Prior to this time there were no re- quirements for licensing or for accreditation of field concrete technicians or con- crete testing laboratories in the Commonwealth. Also there were no uniform building

code requirements throughout Massachusetts,

which proved a hardship to the design and

construction industry.

Since the implementation of the programs, almost five years ago, twenty con- crete testing laboratories and over 1250

field concrete technicians have been

licensed by the state. 156

2. Need for Concrete Accreditation

Programs

The recommended practices of the

American Concrete Institute and other

nationally recognized agencies state that

inspection agencies should be hired primarily on their capabilities and qualifications, and not on a cost basis. Unfortunately, this credo often gets lost in the

shuffle and inspection contracts often go

to the lowest bidder, who may or may not be

qualified, unless there is an effective

accreditation or licensing program.

The final acceptance or rejection of

concrete in a structure is normally based

on the strength of the concrete meeting or exceeding a specified criteria in the project specifications which is based on the

compressive strength of cylindrical concret

specimens, presumably fabricated, handled,

cured, tested, etc. in accordance with

American Society for Testing and Materials I (ASTM) standards, some twenty-eight days

after the concrete is incorporated into the
structure. If the procedures of the ASTM

standards are not followed (i.e., specimens

are improperly fabricated, handled, cured,

tested, etc.), the cylinder will normally

indicate a lower strength or strength potential than concrete properly incorporated

into a structure

.

Low or improper strength results, in some cases caused by improper inspection

or testing procedures, often causes delays

in construction, load tests, lawsuits and

other fallout which adds to the cost of

construction and imposes economic hardships

on all concerned with the project.

Other concerns over the reliability of

the concrete test results as they relate

to public safety and structural integrity

are simultaneously brought into focus.

Based on economics and safety it is,

therefore, important and essential that all

field concrete technicians and concrete

testing laboratories be accredited or licensed to ensure that they are qualified

to perform the required tests in accordance

with the applicable standards and specifications .

3- Implementation Mechanism

There is some law which recognized the

impetuous not to change an unsatisfactory

condition until motivated by disaster. In Massachusetts this motivating disaster was

the January, 1971 collapse of a sixteen

story reinforced concrete apartment structure under construction at 2000 Commonwealth

Avenue in Brighton which resulted in four

fatalities. The resulting press coverage

and public concern lead the Massachusetts

Legislative to propose legislation which

would prosecute field inspectors for not performing all the tests required to prevent such an occurrence in the future.

The construction industry in general,

including design professional, suppliers,

and testing agencies felt that the proposed

legislation was too vague, took a negative

approach to the situation, and would have

established an unneeded commission, which

is the apparent legislative procedure for

dealing with a crisis. The Massachusetts

Construction Industry Board (MCIB, incorporated as The Massacusetts Concrete

Industry Board before broadening its scope)

was formed to speak for the industry. The

State Building Code Commission, MCIB, and

other concerned agencies and individuals

proved successful in preventing the passage

of these bills based on their commitment

to provide positive action in state-wide

programs

.

The field concrete technician and con- crete testing laboratory licensing programs

were adopted or developed by the Commission

as a result, based on the developmental work

of MCIB and assistance of the National

Bureau of Standards (Contract No. 5-35-776)

[l] Code.

h . Development

The provisions of the licensing programs were based on nationally recognized

standards, practices, and procedures,

wherever possible, to provide efficient and

reliable programs, free of specialized

encumberances and related specialized costs.

Key to the programs was the establishment of a prequalifying agency, recognition

of established testing agencies to examine

and evaluate applicants, and the adoption

of the testing agencies proven testing

programs

.

k.l Concrete Testing Laboratories

The Commission designated the Cement

and Concrete Reference Laboratory (CCRL) as the prime testing agency to examine and

evaluate laboratories desiring to be

licensed.

The designation of the nationally

recognized CCRL lent credence to the program. No new agency or new testing procedures had to be established. Periodic

reevaluation, contained in the program,

was made to conform to the normal CCRL

inspection tour cycle.

The New England Division, Corps of

Engineers, also has been designated as a testing agency to perform timely examination

and evaluation as required between CCRL's

normal inspection tours. Due to the nature

of their work and reputation, there is no conflict of interest. Their performance

and fees are compatible with CCRL.

The Construction Materials Safety

Board (CMSB) was established by the Commission to act as the prequalifying agency

to examine, or cause to be examined, the

testing agency examination and evaluation,

and to make its recommendation to the

Commission for appropriate action.

CMSB also reviews application for

materials, devices, products and methods of

construction not included in the State
Building Code. It is made up of nine members consisting of representatives of the

Commission, engineering, architecture,

testing laboratories, construction, and

university faculty. The CMSB performs its

services voluntarily. It generally meets

one-half day per month. Administrative and

secretarial duties are performed by Commission staff.

The Laboratory Accreditation Sub-Committee was formed by CMSB to act on its

behalf to perform the detailed evaluation

of the testing agency's report and subsequent examination and evaluation of the

laboratory's affidavits, where applicable,

certifying correction of deficiencies cited

in the testing agency's report. In some instances the Sub-Committee requires additional proof of deficiency correction

prior to making its recommendation to CMBS.

The Laboratory Accreditation Sub-Committee is made up of five members consisting

of representatives of a federal laboratory,

state laboratory, private laboratory, material supplier and Commission Staff. The

makeup of this Sub-Committee provides ex- pertise required for evaluation. The SubCommittee performs its services voluntarily.

It generally meets one-half day per month.

Administrative and secretarial duties are performed by the Commission staff.

k.2 Field Concrete Technicians

The Commission designated MCIB as the

testing agency to examine and evaluate all

persons desiring to be licensed as concrete

field technicians. MCIB submits all examination results and evaluations to the prequalifying agency for their action.

MCIB was incorporated in 1973 to foster the development of quality construction and to promote and support high standards of quality. Its membership includes

individuals, firms and organizations con- cerned with construction, design, teaching,

testing, and production.

MCIB developed an accreditation program

[2] for field concrete technicians as part

of its commitment to public safety and the

legislature. Previous work of the National

Ready Mixed Concrete Association was beneficial. MCIB began its examination and

accreditation of field concrete technicians

in the summer of 1973.

MCIB established a certification com- mittee to be responsible for the examination

and evaluation and its reliability. No new

158

agency had to be established and the Commission adopted a proven working program.

CMSB was established as the prequalifying agency to examine the evaluation of

the testing agency. The Commission staff

normally acts in its behalf in the field of

concrete technicians programs.

5. Operation of the programs

3 The mechanism for the operation of the

licensing programs is relatively efficient.

5.1 Concrete Testing Laboratory

i 1. The laboratory makes application \ to the Commission staff.

2a. The laboratory requests testing

agency (CCRL or New England Division, Corps j of Engineers) examination through Commission

staff which sometimes directs the laboratories to the testing agency directly.

2b. The results of the testing

agencies examination are forwarded to the

prequalifying agency (the Laboratory Accreditation Sub-Committee of CMSB) through the

Commission staff within ten days of receipt

by the laboratory.

2c. Any laboratory deficiencies cited

in the testing agency's report are to be

corrected within two months and the prequalifying agency shall be so notified by

affidavit submitted through the Commission I staff.

2d. The prequalifying agency normally:

meets within the month to approve results

and/or corrections to any deficiencies so cited, requests for more information, or recommends denial based on the results and

lack of sufficient information received.

3a. The laboratory sends the prequaliH;

fying agency a notarized statement that key

positions are filled by qualified personnel

along with their resume, through the Commission staff. The key positions are the

director of testing services, supervising

laboratory technician, and supervisory field

technician. One person may fill more than

one position if qualified.

3b. The prequalifying agency approves;

the personnel criteria, requests more infor-;

mation, or recommends denial based on the

information submitted.

h. The laboratory sends in the licensing fee (of one hundred dollars per
year) to the Commission staff.

5. Upon completion of the prequalifying agency's examination, and receipt of

the licensing fee, action is taken by the

Commission on the issuance of the concrete

testing laboratory license.

6. The licenses require renewal annually. The laboratory shall send the prequalifying agency (a) an affidavit that the

compression testing machines have been calibrated and verified with equipment traceable

to the National Bureau of Standards. (b) A

notarized renewal form indicating key personnel along with updated resumes. (c) The

licensing fee.

7. The prequalifying agency and Commission take appropriate action on the license renewal.

8. The laboratory shall notify the

Commission within seven days of any vacancy

of any key position and take appropriate

action to obtain approval of a person to

fill that position.

5.2 Field Concrete Technician

1. The person requests an application

form from the Commission staff.

2. The person makes application for

examination by the testing agency (MCIB),

and includes the examination fee (currently

thirty- five dollars).

3. The MCIB Certification Committee

notifies the applicant of the next field

concrete technician examination. (Special

mini-examinations can be held in case of

determined hardship).

h. The Certification Committee arranges for the concrete, place of tests,

equipment, and jurors.

5. The applicant performs physical

tests on items b through f and demonstrates

knowledge of items a and g.

a. ASTM C1T2: Sampling Fresh Concrete.

b. ASTM Cli+3: Test for Slump.

c. ASTM C31: Making and Curing Test

Specimens in the Field.

d. ASTM C231: Test for Air Content - Pressure Method.

e. ASTM C1T3: Test for Air Content - Volumetric Method.

f. ASTM C138: Test for Weight per

Cubic Foot (Density).

g. ASTM C192: Storage and Transportation of Test Cylinders.

6a. The applicant fails the examination if he has failed two of the perfor- mance tests, b through f above.

6b. The applicant is given a "partial"

if he fails one of the performance tests and

must take a retest on that performance test,

in order to pass.

7. Three jurors independently grade

the applicant as to pass or fail on each

test. The jurors are accredited field

concrete technicians and normally represent

three different disciplines to provide

knowledgeable and unbiased results. The

jury system is the backbone for the reliability of the program.

8. At least two members of the

Certification Committee are present at the

examination to independently evaluate the

applicant as to passing or failing. If

the Committee is not in concurrence with

the pass or failure of the candidate, the

jurors are consulted which normally results

in resolution of the situation. If the

situation cannot be resolved it is forwarded to the Board of Trustees of MCIB

for their action.

9. At least two members of the Board

of Trustees of MCIB, being elected officials

of that Board, review the examination results for final evaluation and forward the

results to the prequalifying agency. MCIB

also notified the applicant of MCIB accreditation or failure.

10. The accredited applicant sends in a picture with his application form and

licensing fee (currently twenty dollars for

two years) to the Commission staff,

11. The Commission staff normally acts

on behalf of the prequalifying agency (CMSB)

and forwards the results to the Executive

Director of the Commission.

12. The Executive Director issues the

licenses for those who qualify upon approval

by the Commission.

13. Renewals are issued bi-annually by

the Executive Director upon receipt of the

159
renewal licensing fee (currently twenty

dollars )

lit. Concrete Field Technicians working for government agencies are exempted

from license fees.

6. Rules and Regulations

The Commission publishes "Rules and

Regulations for the Licensing of Concrete

Testing Laboratories" [3] and "Rules and

Regulations for Concrete Testing Personnel"

[It] which should be referenced for details.

Many of the requirements contained therein

have already been addressed. Other highlights of these regulations are presented.

6.1 Concrete Testing Laboratories

1. All laboratories including branch

and project laboratories engaged in the

testing of concrete for use in structures

subject to construction control by the

State Building Code shall be licensed.

Structures not subject to control by the

Third (Latest) edition of the code are:

1. Any building containing less than

thirty-five thousand (35 5 000) cubic feet of

space

2. Any single or two family house

or any accessory building thereto;

3. Any building used for farm purposes; and

It. Retaining walls less than ten (10)

feet in height.

2. Except as modified by the rules

and regulations, all concrete testing laboratories shall conform to designated provisions of "Inspection on Testing Agencies

for Concrete, Steel and Bituminous Materials as Used in Construction," ASTM E329-T2.

The personnel qualification requirements are broken down into the three categories established by ASTM E329-T2, that

of director of testing services, supervising laboratory technician, and supervising

field technician. One person who is qualified may fill more than one position. The

ASTM E329-T2 criteria have been broadened

for the director to recognize practical

expertise as follows:

The testing services of each laboratory

(main, branch or project) shall be under

the direction of a Director of Testing

Services who shall be a full-time resident

employee of that laboratory and shall be

qualified in accordance with any one (l) of

the following three (3) sets of requirement s

a) He shall be a Professional Engineer, registered in the Commonwealth of

Massachusetts with at least five (5) years

of experience in responsible charges of worls related to Structural Engineering, Construction Engineering or Construction Materials

Testing. He shall be subject to demonstrate

his ability to interpret the results of

tests of concrete and concrete aggregates

as stated in ASTM E329-T2; or,

b) He shall have a Bachelor's Degree - in Engineering from an accredited institu- - tion and an additional total of three (3)

years experience performing tests on con- crete and concrete materials which shall

include two (2) years as a laboratory technician or supervisor. He shall be subject

to demonstrate his ability to interpret the;

results of tests of concrete and concrete

aggregates as stated in ASTM E329-T2; or,

c) He shall have at least eight (8)

years experience including five (5) years

experience as a laboratory technician or supervisor and shall be subject to demonstrate his ability to interpret the results

of tests of concrete and concrete aggregates as stated in ASTM E329-T2.

It. On structures subject to construction control, affidavits must be submitted

with the building permit application to show that the laboratory is licensed,

5. Provisions are provided for penalties and revocation of licenses for false

report or cause.

6. Any aggrieved laboratory has the

right to appeal to the State Building Code

Appeals Board in lieu or prior to any court action

6.2 Field Concrete Technician

1. All field concrete technicians

engaged in the testing and/or inspection

of concrete in structures subject to con- struction control (See 6.1 item l) shall

be licensed.

2. Provisions are made for revocation

of license for non-compliance with the rule

and regulations, Code, or standards of good

practice

160
3. Any aggrieved person has the right

to appeal to the State Building Code Appeals

Board.

7. Evaluation of the Field Concrete

Technician and Concrete Testing LaboratoryPrograms .

This evaluation is conducted on the

eve of five years of operation of the field

concrete technician and concrete testing

laboratory programs.

7.1 Positive Aspects

1. Parties of firms involved with or licensed by the program believe in its

merits and it has received the overall support of the industry.

2. The operational mechanism and rules

and regulations have had only minor modifications during the five-year period. This

indicates good development and implementation of the programs

.

3. No complaints have been received on the caliber and performance of agencies

(CCRL: New England Division, Corps of Engineers; or MCIB) or the prequalifying agency

(CMSB). Their results are evaluated as re- liable and above reproach.

h. Several complaints have been re- ceived by MCIB on the evaluation of the

field concrete examination by applicants.

All except one were withdrawn upon showing

the applicants the grading on the jurors

data sheet. In the one exception, the Board

of Trustees of MCIB voted to overrule the

findings and the jurors committee's recommendations. In this case it appears that

the applicant was arrogant and may have

invited harsh critique in some judgmental

areas by the jurors. It was also alleged

that one juror was not present during part

of one test and relied on another juror for

his grading. The aggrieved applicant was

able to convince the Board, a juror, and a Certification Committee member of his thorough knowledge and ability to perform the

tests during this hearing, which is what it

is all about

.

5. The use of the appeals provision

to appear before the State Building Code

Board of Appeals has not been used and no litigation has been entered against any

portion of the programs. This indicates no major grievances.

6. The modifications of the ASTM E329-

72 of the personnel qualifications for

director of testing services has allowed

several experienced and capable persons to continue to act in or move into this capacity. This would not have been possible

under that standard as written.

7. A better appreciation of the performance criteria and test limitations has

been realized. Many take the examination,

pass the technician examination and are accredited by MCIB for educational or other

appreciative reasons. Many accredited technicians live outside of and do not practice

in Massachusetts. Many accredited technicians are in related industry and not testing.

8. Experienced and practicing technicians had a failure rate of one-third on the

examinations held during the implementation

of the program. This leads to the supposition that at least one-third of the field

tests were performed improperly prior to the

program and indicates its need and benefit.

7.2 Areas of Concern

1. The major area of concern has always been that of monitoring and enforcing

the programs

.

Project affidavits are required to be

submitted with the building permit application to indicate that technicians and laboratories are licensed on structures subject

to construction control. This places the

responsibility of providing required ser- vices on the person who signs the affidavit

subject to persecution if the terms of the

affidavit are not carried out. The responsibility of ensuring that the project affidavit and its intent are carried out primarily rests with the local municipal building

official and/or the state inspector who re- views the application and issues a building

permit. Building officials and their staff

are not always cognizant of all details of

the Code let alone ancillary rules and regulations. In many instances they do not

have the time or staff to effectively monitor all phases of all the construction projects in their jurisdiction. One municipal

building official did shut down a project

on which an unlicensed technician was performing as a licensed field concrete technician. This project was shut down for

approximately twenty four hours until a licensed technician was present at the site.

The program further relies on the voluntary self-policing action of the industry

as a whole. Suppliers and contractors

should want to see qualified technicians
and laboratories on the project so as not to have the tests indicate lower potential of

concrete strength than that incorporated into the structure. Other licensed laboratories and technicians should want to see only qualified personnel and firms active to

enhance their chances at obtaining work and

also enhance the professionalism of their

field. Sometimes it proves difficult to volunteer.

Rumors have been heard of violations:

a. Licensed laboratories have sent out unlicensed technicians to projects when licensed technicians are not available, in

order to cope with the variable work loads

typical to the testing industry.

b . Field equipment has been used that

is not in satisfactory condition to pass

CCRL inspection, and which may not provide

reliable results within designated accuracies, during heavier than normal work

activity.

c. Licensed technicians do not always

conform to the recommended testing standard

provisions or to project specifications.

The going technician pay scale (currently

averaging in the range of four to five dollars per hour) may not always invite dedicated personnel.

d. The morale and professionalism of

the licensed technicians is not as high as

it was during the early stages of accreditation and licensing.

e. One laboratory does not have a fulltime director of testing services.

Rumors are difficult to investigate.

Formal written complaints have been the designated mechanism to initiate investigation.

Only a few formal written complaints have

been received and these were rated invalid

based on investigation by state inspector of

the Department of Public Safety.

More attention to monitoring and en- forcement of the programs by local and state

inspection agencies is required to alleviate

concern over rumors and to take action on any proven violations.

2. The staff of the Commission has

been honestly criticized in not taking timely action on applications and other matters,

this is especially true during times of code

rewrite and specialized projects. This is not the fault of the program but one of work

loads and/or administration. Sufficient

time and staffing should be allotted to the

program in order to readily process the administration and secretarial work on a weekly basis

3. No training program is part of the

field concrete technician licensing program.

Some applicants cannot find a satisfactory

way to prepare for the performance examination. Several firms provide training ser- vices (for a fee on the order of fifty dollars). Some do a good job but others do not allow enough time to sufficiently train the

applicant, which can cause an economic hardship (around eighty five dollars) on the

applicant without the desired results. The

MCIB Certification Committee is taking this

matter under advisement.

h. Not all of the Massachusetts State

Agency Testing laboratories participate in

the program. The proficient agencies, the

Metropolitan District Commission, the Massachusetts Bay Transportation Authority, and

the Mass Port Authority do participate.

Notable in its absence is the Department of

Public Works.

5- High competition for work, amongst

the laboratories, reportedly leads to econ- omic magic, by some, in order to break even

on some projects or to survive. The programs to date do not appear to have fully

alleviated the cost shopping practice.

7.3 Cost of the Program on the Technician and Laboratory

1. The field concrete technician must

pay thirty five dollars to be examined and

twenty dollars bi-annually for licensing.

This is considered reasonable.

[

2. The only cost incurred by the con- crete testing laboratories for the program

is the one-hundred dollar annual licensing

fee. The one-thousand dollars for the biannual to tri-annual CCRL inspection and the

approximate two-hundred dollar fee for annual compression testing machine calibration

are considered as normal operating overhead

for proficient laboratories.

7.h The Cost to Administre the Program

The cost to the Commonwealth to administer the program is minimal. The economical administration of this program rests

with the self-sufficiency of the testing

agencies and the voluntary services of the

prequalifying agency.

162
l.k.l Concrete Testing Laboratory

1. CCRL, the designated testing agency,

derives its own funds to operate its function.

2. The Concrete Sub-Committee of CMSB,

the prequalifying agency, volunteers an average of two man-days per month for its half

day monthly meetings.

3. The Commission staff averages three

to four man-days per month to administer the

program. The time spent is higher during

CCRL inspections and annual renewals hut

tends to be less during other periods.

f.k.2 Field Concrete Technician

1. MCIB, the designated testing agency,

derives its own funds to operate its function by the thirty five dollar examination

fee, which was established on a break-even

basis. The normal cost of these examinations is in the order of two-thousand dollars to process around thirty candidates.

The concrete and examination facilities are normally donated. The biggest expenditure

is for a fifty dollar honorarium to each

juror. Other expenditures include coffee,

lunch, and administration costs.

2. The Commission staff acting for

CMSB, the prequalifying agency, averages

three to four man-days per month to administer the program.

7 . 5 Summary

The programs have provided sense of

pride and professionalism amont the technicians, better appreciation of the tests, and

more reliable results. They have been universally accepted within Massachusetts, and

other states have developed programs based

on them. They have proven efficient and

economical to administer.

to ensure qualified personnel are on the

project to oversee proper incorporation of

the concrete and related materials into the

structure to realize construction and specified results. Other programs under consideration in Massachusetts are:

Plant Concrete Technician

Field Concrete Inspection

Laboratory Concrete Technician

Construction Supervisors

The programs are currently being deferred in favor of other projects and budgetary considerations. It is hoped that

these programs can be developed and implemented without the need for a disaster

triggering mechanism.

9-0 Acknowledgements

To those who have forwarded these programs, with no personal recognition or economic reward, for the privilege of sharing in

the effort to improve quality standards in

construction

.

[l] Vincent, L. W. , Laboratory Evaluation

and Accreditation Program , Massachusetts State Building Code

Commission, NBS Contract No. 5-35-766,

October, 197k.

[2] Brattin, G. H. , Accrediting Field

Concrete Technicians , Concrete

Construction, November, 1976.

[3] State Building Code Commission, Rules

and Regulations for the Licensing

of Concrete Testing Laboratories .

[k] State Building Code Commission, Rules

and Regulations for Concrete Testing

Personnel

.

Uniform criteria has been established

throughout the State. The programs are worthwhile and have been proven successful,

but require more vigilant monitoring and en- forcement to sustain or improve upon their

rated merits. Other planned programs are

just as essential.

8. Other Related Programs

The implementation of the field concrete technician and concrete testing laboratory programs are a couple of positive

steps in ensuring quality concrete construction. Other programs should be considered

163
National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

THE CRIME LABORATORY ACCREDITATION PROGRAM OF

THE AMERICAN SOCIETY OF CRIME LABORATORY DIRECTORS (ASCLD)

ANTHONY LONGHETTI

DIRECTOR, CRIMINALISTICS LABORATORY

SAN BERNARDINO COUNTY SHERIFF'S DEPARTMENT

SAN BERNARDINO, CA 92415

and

CHAIRMAN, ASCLD COMMITTEE ON

LABORATORY STANDARDS AND EVALUATION

The American Society of Crime Laboratory Directors (ASCLD) has

formulated a crime laboratory accreditation program with the following

objectives: 1) to improve the quality of laboratory services provided

to the criminal justice system, 2) to offer to the general public and to users of laboratory services a means of identifying throughout the

nation those laboratory facilities which satisfy accreditation criteria,

3) to develop and maintain criteria which can be used by a laboratory

to assess its level of performance and strengthen its operation, and

4) to provide an independent, impartial and objective system by which

laboratory facilities can benefit from a total organizational review.

A significant part of the program is the Standards Manual, which

identifies the principles and standards, discusses them and lists

evaluation criteria for each of the following areas: 1) laboratory

management and operation, 2) personnel qualifications, 3) procedures

and instruments /equipment, and 4) physical plant and security.

A pilot program is currently under way to test the on-site

evaluation aspect of the accreditation program. Four crime

laboratories of varying size, geographic location, and organizational

placement have volunteered to serve as "test sites" for this phase of

the program

Key words: Accreditation; American Society of Crime Laboratory

Directors (ASCLD); certification; management; on-site visit;

personnel qualifications; physical evidence; proficiency testing;

standards.

I6h
1. Introduction 1.1 Definition

The National Advisory Commission on Criminal Justice Standards and Goals made

the following recommendation in 1973 in the

volume titled POLICE [l] 1:

"(Recommendation 12.1, Certification of

Crime Laboratories)

"It is recommended that a national

program be established to insure that all

tests and analyses performed by state,

regional, or local laboratory facilities are procedurally sound and scientifically valid.

The program should provide for the certification (sic) of those facilities whose

testing procedures and scientific analyses

meet the minimum standards set by the

agency administering the program.

"1. An existing national agency or organization should be designated to administer the program. This body should

develop minimum standards by which it can measure every crime laboratory's level of

proficiency.

"2. The national agency or organization should conduct periodic evaluations

of every state, regional, and local

laboratory to determine its level of

proficiency in performing laboratory tests.

In conducting the evaluation, it should

rate the laboratory only on the basis of

those tests which it actually performs in

rendering services.

"3. The national agency or organization should, on the basis of the

evaluation, certify every laboratory that

meets or exceeds the designated minimum

standards in all the tests which it performs. "

The results of a LEAA-funded

proficiency testing research program added

objective substantiation to the intuitive

feeling of many practitioners that some forensic science laboratories, for varying

reasons, were not performing all physical

evidence examinations at a level consistent

with the state-of-the-art in the criminalistics profession [2], One general finding

of the report was that "a wide range of

proficiency levels among the participating

laboratories exists, and in general, there

are several evidence types with which the

laboratories are having serious difficulties."

^Figures in brackets indicate the

literature references at the end of the

paper.

The National Advisory Commission on Criminal Justice Standards and Goals, as previously noted, used the term "certification" in reference to the process by

which a facility , in this instance a crime

laboratory, is evaluated to determine

whether or not it meets minimum standards

of performance. Clearly, the proper term

is "accreditation", defined as: "....a

procedure by which a non-government agency

determines that a program of study or an institution meets prescribed standards. The

institution may, for example, be a college,

a university, or a hospital." £3]

2. The ASCLD Crime Laboratory

Accreditation Program

The American Society of Crime Laboratory Directors (ASCLD) was organized in

1973. One of its principal objectives is to promote, encourage and maintain the highest

standards of practice in the field of crime

laboratory services. The Committee on Laboratory Standards and Evaluation has

progressed toward that goal by developing a Standards Manual and by working to establish

an accreditation program that will be

financially self-supporting and, consistent

with the definition of accreditation, be

peer administered.

The ASCLD accreditation program is

totally neither product oriented nor discipline oriented. The crime laboratory's

"products" that are tested (specifically,

they are identified, sometimes quantitated

and often compared) include an almost

infinite variety of items that potentially

may become physical evidence in a criminal, or civil, investigation and trial. Most

commonly, the items tested include firearms,

documents, bloodstains, seminal stains,

drugs, poisons, hairs, fibers, paint, glass,

flammables, explosives, and soils. Less

common, but by no means unusual, are metals,

vegetation, paper, wood, plastics and saliva.

The term "discipline", as applied to a crime laboratory, is not clearly defined.

Within the criminalistics profession, it is

consistently applied to sub-specialties

that are actually groupings of physical

evidence types. For example, personnel

certification efforts now in progress have

defined the four sub-specialties, or "disciplines", of firearms and toolmark

examination, serology, drug (controlled

substance) identification, and "trace

evidence" (hair, fiber, glass, paint,

soil, etc.)
ASCLD is using neither the relatively narrow product -focused approach that determines only a crime laboratory's capability

to test a specific item of physical evidence

(e.g., a bloodstain) to some specified

standard using a definitive test method, nor

the equally restrictive discipline-focused

approach used by the National Association of

Testing Laboratories in Australia (such as chemical testing or instrumental analysis).

Rather, it has selected a broad accreditation

program that identifies principles and

standards for each of the following four areas of a crime laboratory's operation:

1) laboratory management and operations,

2) personnel qualifications, 3) procedures

and instruments /equipment , and 4) physical

plant and security.

2.1 Laboratory Management and Operations

The Laboratory Management and Operations

section of the Standards Manual covers the

managerial functions of planning, organizing,

directing and controlling. The planning

function, for example, is defined as the

analysis of relevant information from the

present and the past and an assessment of

probable future developments so that a course

of action (plan) may be determined that

enables the organization to meet its stated

objectives. The objectives are then outlined

in terms of principles and standards, followed by a brief discussion and by the listing

of evaluation criteria. The latter are intended for use by the laboratory for selfassessment and/or by the site evaluation

team during its visit. The criteria are

1) does the laboratory have a stated list of

objectives? 2) have the objectives been

communicated to all employees? and 3) do the

objectives appear relevant to the needs of

the community serviced by the laboratory?

The functions of organizing, directing

and controlling are handled in the same manner; i.e., with a statement of the

principle(s) involved, an enumeration of the

standards as they apply to the particular

function or sub-function, a brief discussion

of each standard, and a listing of the

evaluation criteria for each function or

sub -function.

2.2 Personnel Qualifications

The qualifications of both the professional, and, to a limited extent, the

support personnel within a crime laboratory,

are considered and judged "in context" with

the situation in which the individual works.

Most important, therefore, is the existence

in the laboratory of a system such that

"reliability is assured with respect to procedures, equipment, processing and

interpretation of results and that the

examinations are as complete as adequate

laboratory equipment permits and the case situation warrants." If, for example, in a particular type of examination there is

close supervision by a qualified forensic

scientist, the need for a highly qualified

person doing the work is reduced, assuming

that court requirements are satisfied.

Qualifications are considered in the

ASCLD program for personnel in the following

categories: 1) management, 2) controlled

substances identification, 3) toxicology,

4) trace evidence examinations, 5) serology,

6) firearms/toolmark examination, 7) document examination, 8) latent fingerprint

examination, and 9) non-testifying support

personnel. Using serology to illustrate th(

principles and standards as stated, the

former require that the forensic serologist

should have knowledge of basic biological

sciences sufficient to understand the basis

of forensic serological tests and sufficienl

knowledge of chemistry to prepare test

specimens and solutions and to understand

the mechanisms involved in the testing

procedures. In addition it is recommended

that the forensic serologist have adequate , knowledge of the statistics (population

frequencies of blood groups) in the

discipline and have sufficient mastery of

the techniques and procedures so as to be

able to produce reliable results. The

standards are stated as follows: "1) The

forensic serologist should have completed

the science requirements for a baccalaureati

degree in a natural science or in

criminalistics, 2) The forensic serologisi

should have education and experience/trainii

commensurate with the analyses and testimony

provided, 3) Those persons conducting

serological examinations should have

successfully completed an extensive series . of proficiency tests, and 4) It is

166
recommended that those persons conducting

serological investigations meet the qualifications of an appropriate peer group

certification board, if and when such a board is formulated. (Crime Laboratory

Accreditation Program - rev. 5-15-79.)

Standards No. 3 and No. 4, in regard to proficiency testing and certification, are restated for every one of the personnel

categories. Because certification is a voluntary effort, however, there is a statement in the introductory part of the

personnel qualifications section to the

effect that absence of certification of an individual does not necessarily imply that

the person is unqualified.

2.3 Procedures and Instruments /Equipment

The thrust of this aspect of the ASCLD

program is stated simply. Are the

procedures and instruments /equipment used by

the laboratory either generally accepted in

the field, or supported by adequate scientific data, and are materials necessary to conduct these analyses and/or examinations

available? Thus, the criteria for

evaluation require that a crime laboratory

use scientifically acceptable procedures and

prescribed controls and standards, together

with instruments and equipment that are adequate in specifications, checked for

calibration and maintained in good working

order.

Unlike testing laboratories, crime

laboratories do not generally have, nor are required to use, "standard", "accepted" or "recommended" methods. Criminalistics is a relatively new discipline and is comprised

of many identifiable forensic science subspecialties such as serology, firearms

identification and questioned documents

identification. Also, methods and procedures are to some degree dependent on the

requirements of the laws and regulations of

the jurisdiction(s) served by the crime

laboratory, and criminalists are concerned

that courts may discredit a scientific

examination, however valid, if it were not

carried out by a "standard" method.

Finally, many criminalistic examinations,

whether an identification of an unknown or

the comparison of two or more items to determine common origin, are so dependent on varying quantities of the physical evidence

available, or so affected by the possibility

of contamination, that standardization of

the procedure, method or test is difficult.

The Forensic Sciences Foundation is

currently carrying out a LEAA-supported

project entitled "Criminalistics Methods of

Analysis Feasibility Study". Its purpose is

to develop and test a mechanism for determining the acceptability of alternative

scientific methods. Should a workable

mechanism be developed, perhaps one that

would include a compendium of acceptable

scientific methods, the procedures section

of the ASCLD accreditation program could

benefit from it. It should be emphasized

again, however, that the procedures or methods used by the crime laboratory are but

one aspect of the accreditation program;

management practices, personnel qualifications, physical plant, and evidence

security are equally important.

2.4 Physical Plant and Security

Space considerations, design considerations, security and personnel safety are the elements of this fourth section of the

accreditation program. Adequate and proper

space, including bench areas, storage areas,

and clerical and other support space is an obvious necessity for efficient operation

of any laboratory. It lowers the risk of

mishandling or contaminating physical

evidence and keeps personnel morale at a level conducive to high productivity.

Proper design of the crime laboratory also

facilitates its operation. For example,

inadequacy or misplacement of utilities can hinder or even prevent certain instruments

from functioning properly.

The need for physical evidence security

and integrity sets a forensic science

laboratory apart from other types of

laboratories. It is, in fact, a serious

enough element within the accreditation

process that failure to meet the standards

for "chain of evidence" or integrity of

evidence would undoubtedly cause a crime

laboratory to fail.

To ensure that laboratory personnel

work in a safe environment, standards for

safety equipment, for the handling of

carcinogenic/toxic/dangerous materials, and

safety procedures and rules must be

established and adhered to.

167
3. Pilot Program for On-Site Evaluation

of Crime Laboratories

After final review of the Crime

Laboratory Accreditation Program by the

ASCLD Board of Directors, a pilot on-site

evaluation test was designed and completed.

Four laboratories of varying size (from

four to more than forty professional

personnel), differing geographic location,

and organization placement ranging from

local to regional to state, volunteered to serve as "test sites". The site evaluation

teams consisted of four (at one site, three)

crime laboratory directors. The make-up of

the teams varied only slightly; many of the

team members participated in more than one site evaluation as either part of the

inspection team or as a "host" laboratory.

The report of the on-site evaluation

sub-committee has not been completed.

Tentative conclusions suggest that many of

the evaluation criteria must be re-written

in a form that is more objective and more readily capable of measurement. The on-site

visit protocol must be standardized in terms

of procedure, the time spent on each section

of the evaluation criteria, and the training

of the on-site visit team members. A

comprehensive and well-planned on-site visit

is an absolute necessity in a meaningful

laboratory accreditation program.

4. Implementation of the ASCLD

Accreditation Program

Acceptance of the Standards Manual, as distributed, by the membership of the

American Society of Crime Laboratory

Directors at its October, 1979 meeting will

signal the Laboratory Evaluation and

Standards Committee to request approval to prepare a proposal for implementation of an accreditation program. To be considered and

developed, then, would be such factors as

1) the composition and bylaws of the

Laboratory Accreditation Board, 2) the

length of accreditation period(s), 3) the

method of application for accreditation,

including the types of supporting documents

that should be submitted prior to the onsite visit, 4) the procedure for on-site

visits, 5) re-accreditation, 6) a "scoring"

system, pass/fail or otherwise, and 7)

financial support of the program.

5. Summary and Objectives of Accreditation

Accreditation of a forensic science

laboratory involves the assessment of a number of factors, including the management

of the laboratory, the general overall

competency of the staff, suitability of the

working environment and the evaluation of

the procedures used in the course of its

operation. A fundamentally sound and

objective accreditation program will benefit

the criminal justice system, specifically

the users of crime laboratory services, by

1) improving the quality of laboratory

services provided by the criminal justice

system, 2) offering to the general public

and to users of laboratory services a means

of identifying those laboratory facilities

which satisfy accreditation criteria, 3)

developing and maintaining criteria which

can be used by a laboratory to assess its

level of performance and strengthen its

operation, and 4) providing an independent,

impartial and objective system by which

laboratory facilities can benefit from a total organizational review.

References

Cl] National Advisory Commission on Criminal Justice Standards and Goals,

Police , January 1973, pp. 316-317.

[2] Peterson, J. L., Fabricant, E. L., and

Field, Kenneth S., Crime Laboratory

Proficiency Testing Research Program , Government Printing Office, October,

1978.

[3] Sanders, H. J., Do Chemists Need

Added Credentials?, Chem. Eng. News , March 31, 1975, pp. 18-27.

I

i

168
01 National Bureau of Standards Special Publication 591, Proceedings of the National Conference

on Testing Laboratory Performance: Evaluation and Accreditation, held at NBS , Gaithersburg,

MD, September 25-26, 1979. Issued August 1980

CERTIFIED LABORATORY PERFORMANCE EVALUATION

H. JAMES BARTH

USDA-FSQS-SCIENCE

WASHINGTON, DC 20250

The USDA-FSQS-Science Certified Laboratory Program commenced in

late 1962 in response to the meat food industry for more rapid analytical results on official samples. Initially, non-government Chemistry

laboratories that requested certification and that met our requirements were certified for moisture, protein, and salt determinations.

In 1969, with the advent of the cooked sausage regulations, the Certified

Laboratory Program was extended to cover fat determinations in addition

to the previously mentioned determinations.

Presently, the Certified Laboratory performance evaluation is an ongoing program. Analytical results are compared between the Certified

Laboratories and the USDA Laboratories. These analyses are monitored

using computer generated reports. Each USDA Certified Laboratory Reviewer

is responsible for verifying these reports for accuracy and following up

on unacceptable analytical variations. In addition, the Certified

Laboratory Reviewer makes an unannounced on-site review of each Certified

Laboratory per year. Review reports contain information regarding all

aspects of the Certified Laboratory's analytical operation and include

recommendations for necessary corrective action including decertification if necessary.

Key words: Accuracy and precision of the checking authority; laboratory

procedures; reliability of the sample collection; tools for laboratory

evaluation.

1. Introduction

Certified Laboratory Performance evaluation is a multiphase operation. In order to provide a fair evaluation of a laboratory's

ongoing performance a procedure had to be

devised to consider three main points:

1. Accuracy and precision of the checking authority

2. The reliability of the sample

collector

3. The actual laboratory procedures

both administrative and analytical

To ignore any one of these points would

result in a loss of integrity of the overall

laboratory evaluation program.

2. Accuracy And Precision Of The

Checking Authority

The continued accuracy of the USDA-FSQSScience Laboratories is monitored with known

check samples and blind check samples. In addition, the Laboratory Quality Control

Officer maintains laboratory control charts

and requires that each analyst maintain his

own quality control charts. These charts

are regularly reviewed by the laboratory

Quality Control Officer to determine that

work is maintained to established performance

standards

.

169
Certified Laboratory evaluation begins

with the sample collector who selects the

split samples used to check the certified

laboratory's continued analytical capability.

Proper training of the sample collector

includes sample selection, preparation,

handling, labeling and mailing procedures.

Improperly prepared samples may introduce

sample variability, exceeding analytical

variability, yielding analytical results

whose variability could not be blamed on the

laboratory. We cannot realistically measure

analytical variability under these conditions.

3. Laboratory Procedures

Laboratory Quality Control is stressed

at all times, and includes a careful critical

points evaluation of sample receipt, record

keeping, sample preparation, in-laboratory

sample tracking, analutical procedures, calculations, and reporting of results. Standard solutions and laboratory equipment are

to be regularly checked and records of such

checks are to be recorded in detail in the

Laboratory Standards Book. We require the

laboratory to determine sample acceptability

upon sample receipt. Sample acceptability

includes sample integrity, proper identification, and sufficient amount of sample for

the requested analysis. Complete sample

information must be kept in a sample record

book.

4. Tools For Laboratory Evaluation

The primary tool for ongoing laboratory

performance evaluation is the companion sample computer program. Companion samples are

split samples analyzed by both the Certified

and the assigned USDA-FSQS Laboratories.

Companion sample results are stored by computer operation for each laboratory. This

processed data provides a profile of the con- tinuing laboratory performance.

To keep data errors to a minimum, each

Certified Laboratory Reviewer is provided

with a copy of the weekly summary of newly

matched data (between the Certified Laboratory and the USDA-FSQS-Science Laboratory)

The computer highlighted data that exceeds

the accepted limits of analytical variation

is rechecked to determine the accuracy of

the USDA-FSQS-Science Laboratory or Certified

Laboratory data as well as correctness of

data entered into the data base. In addition,

the Certified Laboratory Reviewer contacts

the Certified Laboratory to discuss the

possibility of analytical problems occurring

when the computer report shows a bias

developing.

The Certified Laboratory Reviewer makes

an annual unannounced on-site review of each

Certified Laboratory and of the Inspectors

who utilize the Certified Laboratory. Review

reports contain information regarding all

aspects of the Certified Laboratory's analytical and administrative operation and provide

recommendations for necessary corrective

actions including possible decertification.

The Certified Laboratory coordinator

maintains an overall laboratory performance

file showing original certification results,

ongoing laboratory performance, review

reports, computer generated data reports,

and any correspondence regarding recommendations for corrective action.

5. Summary

By constantly pursuing any area where

errors may occur, by continuously monitoring

the check samples from the Certified Laboratory and finally by maintaining a good Certified Laboratory Review Program, a good measure of laboratory performance can be

achieved

170
SESSION VII

INTERNATIONAL

COORDINATION

CHAIRMAN: HOWARD I. FORMAN

U.S. DEPARTMENT OF COMMERCE

lational Bureau of Standards Special Publication 591, Proceedings of the

j>n Testing Laboratory Performance: Evaluation and Accreditation, held at

D, September 25-26, 1979. Issued August 1980

National Conference

NBS , Gaithersburg,

PROPOSED U.S. POSITION PAPER

FOR THE THIRD INTERNATIONAL CONFERENCE

ON RECOGNITION OF NATIONAL PROGRAMS FOR

ACCREDITING TESTING LABORATORIES (ILAC)

SYDNEY, AUSTRALIA, OCTOBER 22-26, 1979

OFFICE OF PRODUCT STANDARDS

U.S. DEPARTMENT OF COMMERCE

WASHINGTON, D.C. 20230

The Third International Conference on Recognition of National Programs for

Accrediting Testing Laboratories (ILAC/79) will be held in Sydney, Australia,

October 22-26, 1979. ILAC is an informal assemblage of nations and international organizations whose objective is to examine ways in which accredited

laboratories in each country could be internationally recognized. The primary

work of ILAC/79 was performed by three Task Forces which have -- (1) analyzed

legal problems raised by the recognition of nationally recognized laboratory

accreditation systems; (2) drafted an international directory of organizations

which operate accreditation systems; and (3) developed a description of the

needs, objectives, and effects and consequences of laboratory accreditation

and prepared a list of basic terms and definitions. The Task Force reports

are summarized in this position paper; proposed U.S. positions with respect to these Task Force reports are also included. The preconf erence briefing is being held as part of this national conference to inform all interested parties about the content of ILAC/79 deliberations and to receive comments and

suggestions from the participants with respect to the position the U.S. should

take at ILAC/79.

Keywords: Definitions; directory of accreditation systems; international;

laboratory accreditation; legal constraints.

1. Introduction Task Force A was to make an analysis

of the legal problems raised by the

recognition of national laboratory

accreditation systems, based, among

other things, upon the answers to a questionnaire which was reviewed and

approved at ILAC/78.

The first International Laboratory

Accreditation Conference (ILAC/77), held

in Copenhagen in October 1977, designated

task forces to explore relations with ISO,

to study legal problems associated with

Accreditation, and to consider the preparation of an international registry of

accrediting organizations. The second

Conference (ILAC/78), held in Washington

Hn October 1978, established Task Forces

jft, B, and C to continue and extend the

vork of the original task forces (i.e.,

Task Forces 1, 2, and 3), and to report on

:hat work at the third Conference

ILAC/79 ) to be held in Sydney, Australia,

in October 1979.

Task Force B was to decide on information to be sought from accrediting

organizations which would, in turn, be

assembled by the Australian delegate

into a draft directory of organizations or bodies which operate accreditation systems. The Task Force would

analyze the problems encountered in

collecting and presenting the information, including costs and size of the

173
task, and make proposals for the maintenance and dissemination of the directory.

Task Force C was to prepare, in

cooperation with ISO and other con- cerned international organizations, a paper on needs, objectives, and

effects and consequences of laboratory

accreditation, and to prepare a list

of basic terms and their definitions

relative to laboratory accreditation.

These task force reports constitute

the main items on the agenda of the third

Conference (ILAC/79) to be held in Sydney

in October 1979.

This position paper is based on the

reports of Task Forces A, B, and C. The

proposed draft directory of organizations

and the subsequent analysis of costs,

size, maintenance, and dissemination of

the directory have not as yet been made

available to ILAC delegates.

Comments on this position paper and on the Task Force reports should be made at the preconf erence briefing session to be

held at 2 p.m. on September 26, 1979, at the National Bureau of Standards, or should be submitted in writing on or before October 5, 1979, to Dr. Howard I. Forman, Deputy Assistant Secretary for

Product Standards, U.S. Department of

Commerce, Washington, D.C. 20230; (202)

377-3221.

2. Overall U.S. Position

The United States supports the informal assemblage of nations and international organizations represented by ILAC,

and the objectives and work of ILAC as reflected in the activities, and reports of

ILAC's various task forces.

3. Report of Task Force A

3.1 Summary

In response to the questionnaire

distributed by Task Force A, over 600

pages of information from 18 countries

were received and analyzed. Task Force A considered not only specific legal con- straints to the formal, national recognition of accreditation systems but also

relevant factors bearing on the activity

of testing laboratories.

The main points made by Task Force A

in its report are:

A. In many instances, mandatory

requirements for testing products

prescribe that only specific

laboratories named in the law or regulation may carry out tests

relevant to these requirements.

Accordingly, it may be necessary

for such laws or regulations to be

amended to allow other testing

laboratories which have been

accredited under a recognized

accreditation program to carry out

the appropriate tests on the

products covered by those laws or regul at ions.

B. Some accreditation bodies apparently are able to accredit foreign

testing laboratories; but either

the extent of such accreditation

is not clear, or their specific

authority to provide such accreditation is not clear.

C. As a general rule, countries do not impose on laboratories in

foreign countries requirements or obligations which materially

differ from those imposed on their

domestic laboratories.

D. There is relatively little evidence of specific laws or regulations which immunize an accrediting body from liability or acts

of negligence. No court decisions

whereby accrediting organizations

or testing laboratories were found

liable for any act of negligence

in accrediting a testing laboratory were reported.

E. Full reciprocity for acceptance of

accredited laboratories among

countries should be based on requirements such as: the need

for comparable standards and

procedures; the need for procedures to insure continuing product,

compliance; the need for the

accrediting body to be independent

of the laboratories it accredits;

the need for test reports to be in the language of the country

extending acceptance of the

accreditation program; and the

need for reciprocity to be open

only to countries extending mutual

reciprocity.
3.2 Recommendations of Task Force A

That Task Force A be continued and be

assigned the following tasks (to be

reported at I LAC/80 in Paris):

A. Look further into the existence of

legal reasons preventing the

formal accreditation of foreign

testing laboratories. To assist

in this task, request the delegations responding to the Task Force

A questionnaire to provide further

information on the following items:

i. For those accreditation programs which do not permit

accreditation of foreign

testing laboratories, list

major reasons for refusing to

do so and indicate what needs

to be done to eliminate such

restrictions.

ii. For those accreditation programs which accredit foreign

testing laboratories, identify

any major difficulties which

may have arisen from such

accreditation, and list

accredited laboratories

together with the extent to which such laboratories have

been used.

B. Identify the possible extent of

unstated but real problems of

those accreditation systems which

do not provide accreditation to foreign testing laboratories.

Include, particularly, problems

associated with the difficulty in

instituting liability proceedings

against a foreign testing laboratory, and problems in use of

public funds to pay for such

accreditation such as cases where

the agency is not willing to use such funds to accredit foreign

testing laboratories.

C. For those laws and regulations

silent on foreign laboratory

accreditation programs, ask the

reporting country to specify how

the laws may be amended, revised,

or otherwise interpreted to provide for recognition of such programs. Task Force A recognizes

that appropriate criteria for

evaluating laboratories are essential to the operation of a

recognized accreditation program

and that ILAC may wish to compile

suggested criteria.

D. If ILAC should undertake to com- pile appropriate criteria, Task

Force A should be invited to examine those criteria to identify

any legal problems or recommend

what laws need to be enacted or amended to allow use of such

criteria.

E. Task Force A was presented with

the possibility that mandatory

testing systems may be an important source of technical barriers

to trade. Task Force A should

determine the likelihood of this

possibility and propose ways and means for eliminating or minimizing that possibility.

3.3 U.S. Position

Concurrence with recommendations A, B, and C of Task Force A, since these are logical extensions of the work already

begun and appear to be necessary to more completely utilize the data already collected. Concur also with recommendation

D, suggesting that the chairman of the

next scheduled ILAC Conference be authorized to request assistance from Task Force

A whenever, in his judgment, there is the

need suggested in recommendation D.

The U.S. strenuously objects to recom- mendation E of Task Force A. ILAC is an informal assemblage of nations and international organizations primarily interested in developing arrangements whereby

testing laboratories and their data in one country will be recognized and accepted as valid in another country. Some participants in ILAC may be government officials

having responsibility for, or for other reasons are concerned with, international

trade negotiations. Some of those officials have indicated a desire to use ILAC

as a forum to discuss issues which are not basic to ILAC's principal purposes and

objectives, particularly issues concerning

matters which have for many years been the

subject of discussions in the multilateral

trade negotiations (GATT Agreement), and

which will continue to be discussed in the

context of the implementation of those

trade negotiations.

The U.S. sees no logical reason why

ILAC's forum, which is intended to be a

175
temporary one with plenary sessions held (iv)

only once annually, should be used to discuss any of the same issues that are, by formal agreement of the nations

involved, subject to disposition by the (v) terms of that Agreement. The lack of

logic in having ILAC spend any of its

limited time on such matters is even more

apparent in view of the fact that ILAC has

no means by which it can resolve any such

matters.

(vi) For a more detailed explanation of

this position, reference is made to a letter of June 19, 1979, which Dr. Forman,

Head of the U.S. Delegation to ILAC/79,

sent to the ILAC/78 Heads of Delegations

from the European Common Market Countries.

4. Report of Task Force B

4.1 Summary

Task Force B developed an outline for

the provisional directory shown in Annex 1 of its report. That outline proposes a "foreword" describing the background of

ILAC, the purpose and layout of the directory, methods for collecting information,

and methods for collecting updated information in the directory. The outline proposes "Part I" of the directory to include

one-page summaries of the general testing

arrangements in each of the countries

represented in ILAC. The outline proposes

"Part II" of the directory which would

contain one-page summaries of each of the

accreditation programs existing in each of

the countries represented at ILAC.

To be eligible for inclusion in

Part II of this directory an accrediting

organization must operate its accreditation system to meet the following conditions:

(i) It must be actively engaged in the

accreditation of laboratories.

(ii) It must specify accreditation of

the testing laboratories in terms

of well-defined fields of testing,

scientific disciplines or technologies, or specific products or tests.

(iii) Its technical criteria for

accreditation must be formulated

by persons possessing the necessary technical competence in the

relevant field of testing. (vii)

Its criteria for accreditation

must be published and be generally

avail able.

It must only accredit laboratories

in respect of tests performed in

accordance with test specifications agreed between the laboratory and the accrediting organization.

Its evaluation of laboratories

must be in terms of criteria which

provide that:

(a) Laboratory structure and

management are clearly defined

and are organized in such a way that the integrity of its

staff and operation is assured;

(b) Laboratory staff are suitably

qualified and experienced and

technically competent for the j j work in which they are engaged; • n

jj 6 (c) Laboratory equipment is appropriate for the tests undertaken and that it is properly

installed, maintained, and

calibrated at intervals prescribed by the accrediting \ tj authority. Adequate records ; 8 of calibration and servicing t j must be maintained;

' in (d) The testing environment and j j laboratory accommodation are r tj appropriate for the tests c

fa undertaken;

'

ill (e) Laboratory practices such as-- i k sampling (where appropriate)

sampling identification

test methods and procedures

supervision of staff

keeping of records

checking of results and cal- , culations are satisfactory; r \\

i (j (f) The laboratory records system

is secure and contains full r,

;j t( details of all tests under- : ^ taken;

k

(g) Test documents present accur- I tj), ately, clearly, and unambigu- ; jrj ously the test results and all relevant information.

(No item vii was included in the

Report.

)

176
(viii) It must assess laboratories using

teams of impartial experts who

have expertise in the area of

testing in which accreditation is sought.

(ix) Its assessment must be completed

by a formal report by the assessors.

(x) It must reassess its accredited

laboratories at regular and

reasonable intervals.

(xi) It must publish a list of laboratories which it has accredited.

Entries in Part II are to be made

in terms of nine fields of testing

(chemical, electrical, etc.).

4.2 Recommendations of Task Force B

Task Force B agreed to meet in Sydney

immediately prior to ILAC/79 to update its

report, and to meet again after ILAC/79 to expedite the work and take into account

the comments made at ILAC/79.

4.3 U.S. Position

Task Force B should be encouraged to continue the development of the directory. A plan for developing and distributing the directory in final form

should be prepared by the Task Force for

presentation at ILAC/80. The draft directory should be distributed to all ILAC

representatives as soon as possible for

comments. The task force should resolve

all the issues raised by the comments and,

alternatives for presentation at ILAC/80

for resolution.

The "Conditions for Entry in Part II of Directory" presented in Annex III of

the Task Force report and reprinted above

represent criteria for evaluating accreditation organizations. Although these con- ditions appear to be reasonable, there are many which will be subject to interpretation (e.g., "(vi )(e) ... laboratory practices such as sampl i ng. . . are satisfactory."). The U.S. believes that ILAC/79

or ILAC/80 should vote on each condition

listed in Annex III before these conditions are put into effect in the final

directory.

5. Report of Task Force C

5.1 Summary

Task Force C presented a report on the

needs, objectives, and effects and con- sequences of laboratory accreditation

(Appendix 1 of the Task Force C report).

The intent of the report was to discuss

why laboratory accreditation is desirable,

what it can hope to achieve, and the

impact such accreditation would have on the community which utilizes such a system.

Appendix 2 to the report includes a set of definitions of basic terms used in laboratory accreditation. Definitions

compatible with ISO definitions were used

wherever possible.

5.2 Recommendations of Task Force C

The Task Force submitted the following

recommendations:

A. ILAC/79 should encourage all na- tions to develop national systems

for accreditation of their testing

laboratories along lines which

general ly wi 1 1 be recognized and

utilized by other nations.

B. Work begun in providing the definitions shown in Appendix II of the

Task Force report should be con- tinued in an attempt to arrive at

a comprehensive set of definitions. ISO should be invited to present such a set of definitions

to ILAC/80, but if ISO does not accept this invitation by January 1, 1980, ILAC should request

that Task Force C continue the

work.

C. A thorough comparison of the criteria used by the major accreditation systems should be undertaken

by Task Force C for ILAC/80 leading to minimum criteria for multilateral recognition.

D. Each country should establish a central inquiry point to provide

information on accreditation

systems and accredited laboratories.

177
5.3 U.S. Position

Suggest that the report and Appendix 1 be incorporated as the introduction to the

proposed directory and that the definitions, Appendix II, be included in the

appropriate section of the directory.

Further suggest that Task Force B be given

the responsibility for receiving suggested

revisions to the definitions and for preparing a set of final definitions for use

in the directory.

178

* U. S. GOVERNMENT PRINTING OFFICE : 1980 311-046/199
114A (REV. 9-78)

U.S. DEPT. OF COMM.

i BLIOGRAPHIC DATA

SHEET

1. PUBLICATION OR REPORT NO. NBS SP 591

2. Gov't Accession No^ 3. Recipient's Accession No.

TTLE AND SUBTITLE

esting Laboratory Performance: Evaluation and Accreditation

5. Publication Date August 1980

$. Performing Organization Code

UTHOR(S)

erald A. Berman, Editor

8. Performing Drgan. Report No.

ERFORMING ORGANIZATION NAME AND ADDRESS

NATIONAL BUREAU OF STANDARDS

DEPARTMENT OF COMMERCE

WASHINGTON, DC 20234

10, Project/Task/Wo* Unit No.

11. Contract/Grant No.

JPONSORING ORGANIZATION NAME AND COMPLETE ADDRESS (Street, city, state, ZIP)

ame as item 9.

13. Type of Report & Period Covered Final

14. Sponsoring Agency Code

* SUPPLEMENTARY NOTES

Library of Congress Catalog Card Number: 80-600110

Document describes a computer program; SF-185, FIPS Software Summary, is attached. iBSTRACT (A 200-word or less [actual summary of most significant information. If document includes a significant bibliography or

'tereture survey, mention it here.)

j Proceedings of a National Conference on Testing Laboratory Performance: Evaluation

hd Accreditation held at the National Bureau of Standards on September 25-26, 1979.

^enty-nine papers address various techniques for evaluating the performance of

esting laboratories, quality control aspects of the testing function, existing

id proposed accreditation programs and systems, and international coordination.

!EY WORDS (six to twelve entries; alphabetical order; capitalize only the first letter of the first key word unless a proper name;

operated by semicolons)

:creditation; audit certification; laboratory accreditation; laboratory performance

Valuation; quality control; testing laboratories.

VAI LABILITY %x] Unlimited 19. SECURITY CLASS

(THIS REPORT)

21. NO. OF

PRINTED PAGES

For Official Distribution. Do Not Release to NTIS UNCLASSIFIED 179

J Order From Sup. of Doc, U.S. Government Printing Office, Washington, DC

20402.

20. SECURITY CLASS

(THIS PAGE)

22. Price

D Order From National Technical Information Service (NTIS), Springfield, VA. 22161 UNCLASSIFIED

$5.50
TIRED

OF HUNTING

FOR THE

INFORMATION

YOU NEED?

Simplify your search with

COMMERCE PUBLICATIONS

UPDATE!

Tracking down needed information can be

frustrating, expensive, and time consuming. If you're a user of Commerce Department

publications, Commerce Publications Update

can help.

For only $17 a year, you'll get a handy biweekly

newsletter announcing 100-200 new U.S. Department of Commerce publications, reports,

pamphlets, charts, periodicals—even news

releases. You'll get titles, prices, complete

ordering instructions, and annotations where

appropriate.

Every two weeks, you'll learn what we've issued or exporting . . . current economic indicators

. . . housing starts . . . marketing . . . the weather

. . . foreign economic trends . . . fisheries

. . . telecommunications . . . population . . . and

much more.

Get your information the easy way. Use the orde

card to subscribe to Commerce Publications

Update today.

U.S. Department of Commerce

ORDER FORM To:

Enclosed is $

Superintendent of Documents, U.S. Government Printing Office, Washington, D C. Credit Card Orders Only

check,

money order, or charge to my

Deposit Account No. ]-

VISA*

Order No._

Total charges $_

Credit

Card No.

Fill in the boxes below.

Expiration Date Month/Year

Please enter my subscription to COMMERCE PUBLICATIONS UPDATE (CPU) (File code 2- E) at $17 per year (Add $4 25 for other than US mailing)

Company Name

Individual s Name — First Last

Street address

City

(or Country)

State ZIP Code

PLEASE PRINT OR TYPE

For Office Use Only

Quantity C

Enclosed To be mailed

Subscriptions Postage

Foreign handling MMOFJ OPNR

UPNS

Discount Refund
NBS TECHNICAL PUBLICATIONS

PERIODICALS

JOURNAL OF RESEARCH—The Journal of Research of the National Bureau of Standards reports NBS research and develop- ment in those disciplines of the physical and engineering sciences in which the Bureau is active. These include physics, chemistry,

engineering, mathematics, and computer sciences. Papers cover a broad range of subjects, with major emphasis on measurement

methodology and the basic technology underlying standardization. Also included from time to time are survey articles on topics

closely related to the Bureau's technical and scientific programs. As a special service to subscribers each issue contains complete

citations to all recent Bureau publications in both NBS and non- NBS media. Issued six times a year. Annual subscription: domestic

$13; foreign $16.25. Single copy, $3 domestic; $3.75 foreign. NOTE: The Journal was formerly published in two sections: Section A "Physics and Chemistry" and Section B "Mathematical

Sciences." DIMENSIONS/NBS—This monthly magazine is published to in- form scientists, engineers, business and industry leaders, teachers,

students, and consumers of the latest advances in science and

technology, with primary emphasis on work at NBS. The magazine

highlights and reviews such issues as energy research, fire protection, building technology, metric conversion, pollution abatement,

health and safety, and consumer product performance. In addition, it reports the results of Bureau programs in measurement

standards and techniques, properties of matter and materials, engineering standards and services, instrumentation, and

automatic data processing. Annual subscription: domestic $11;

foreign $13.75. NONPERIODICALS

Monographs—Major contributions to the technical literature on

various subjects related to the Bureau's scientific and technical activities. Handbooks—Recommended codes of engineering and industrial

practice (including safety codes) developed in cooperation with inI terested industries, professional organizations, and regulatory

bodies. Special Publications—Include proceedings of conferences sponsored by NBS, NBS annual reports, and other special publications

appropriate to this grouping such as wall charts, pocket cards, and

bibliographies. Applied Mathematics Series—Mathematical tables, manuals, and

studies of special interest to physicists, engineers, chemists,

biologists, mathematicians, computer programmers, and others engaged in scientific and technical work.

National Standard Reference Data Series—Provides quantitative

data on the physical and chemical properties of materials, compiled from the world's literature and critically evaluated. Developed under a worldwide program coordinated by NBS under

the authority of the National Standard Data Act (Public Law

90-396).

NOTE: The principal publication outlet for the foregoing data is the Journal of Physical and Chemical Reference Data (JPCRD)

published quarterly for NBS by the American Chemical Society

(ACS) and the American Institute of Physics (AIP). Subscriptions,

reprints, and supplements available from ACS, 1 155 Sixteenth St., NW, Washington, DC 20056.

Building Science Series—Disseminates technical information developed at the Bureau on building materials, components,

systems, and whole structures. The series presents research results,

test methods, and performance criteria related to the structural and

environmental functions and the durability and safety characteristics of building elements and systems.

Technical Notes—Studies or reports which are complete in themselves but restrictive in their treatment of a subject. Analogous to monographs but not so comprehensive in scope or definitive in treatment of the subject area. Often serve as a vehicle for final reports of work performed at NBS under the sponsorship of other government agencies.

Voluntary Product Standards—Developed under procedures

published by the Department of Commerce in Part 10, Title 15, of

the Code of Federal Regulations. The standards establish nationally recognized requirements for products, and provide all concerned interests with a basis for common understanding of the

characteristics of the products. NBS administers this program as a supplement to the activities of the private sector standardizing

organizations. Consumer Information Series—Practical information, based on NBS research and experience, covering areas of interest to the con- sumer. Easily understandable language and illustrations provide

useful background knowledge for shopping in today's tech- nological marketplace. Order the above NBS publications from: Superintendent of Documents, Government Printing Office, Washington, DC 20402. Order the following NBS publications—FIPS and NBSIR's—from

the National Technical Information Services, Springfield, VA 22161

.

Federal Information Processing Standards Publications (FIPS

PUB)—Publications in this series collectively constitute the Federal Information Processing Standards Register. The Register

serves as the official source of information in the Federal Govern- ment regarding standards issued by NBS pursuant to the Federal Property and Administrative Services Act of 1949 as amended.

Public Law 89-306 (79 Stat. 1127), and as implemented by Executive Order 1 1 7 1 7 (38 FR 1 23 1 5, dated May 1 1

, 1 973) and Part 6 of Title 15 CFR (Code of Federal Regulations).

NBS Interagency Reports (NBSIR)—A special series of interim or

final reports on work performed by NBS for outside sponsors

(both government and non-government). In general, initial dis- tribution is handled by the sponsor; public distribution is by the National Technical Information Services, Springfield, VA 22161,

in paper copy or microfiche form.

BIBLIOGRAPHIC SUBSCRIPTION SERVICES

The following current-awareness and literature-survey bibliographies

are issued periodically by the Bureau:

Cryogenic Data Center Current Awareness Service. A literature sur- vey issued biweekly. Annual subscription: domestic $25; foreign

$30. Liquefied Natural Gas. A literature survey issued quarterly. Annual

subscription: $20.

Superconducting Devices and Materials. A literature survey issued

quarterly. Annual subscription: $30. Please send subscription or- ders and remittances for the preceding bibliographic services to the National Bureau of Standards, Cryogenic Data Center (736) Boulder, CO 80303.
J.S. DEPARTMENT OF COMMERCE

liational Bureau of Standards

Vashington. D C. 20234

IFFICIAL BUSINESS

'enalty for Private Use. $300

POSTAGE AND FEES PAID

U.S. DEPARTMENT OF COMMERCE COM—219

SPECIAL FOURTH-CLASS RATE

BOOK