U.S.

TMENT

OF

MERCE

National

Bureau

of

Standards

ffatftina! Bureau of Standard Reference bOOk DOl to bo

Library, pi Admin. Bug, taken from the libra i'V APR 8 1971

NBS SPECIAL PUBLICATION 335

Innovative Metrology

Key to Progress

Proceedings of the 1970

Standards Laboratory Conference
NBS TECHNICAL PUBLICATIONS

PERIODICALS

JOURNAL OF RESEARCH reports National

Bureau of Standards research and development in physics, mathematics, chemistry, and engineering.

Comprehensive scientific papers give complete details of the work, including laboratory data, experimental

procedures, and theoretical and mathematical analyses. Illustrated with photographs, drawings, and

charts.

Published in three sections, available separately: • Physics and Chemistry

Papers of interest primarily to scientists working in these fields. This section covers a broad range of

physical and chemical research, with major emphasis on standards of physical measurement, fundamental

constants, and properties of matter. Issued six times

a year. Annual subscription: Domestic, $9.50; for- eign, .$1 1.75*. • Mathematical Sciences

Studies and compilations designed mainly for the mathematician and theoretical physicist. Topics in mathematical statistics, theory of experiment design,

numerical analysis, theoretical physics and chemistry, logical design and programming of computers

and computer systems. Short numerical tables. Issued quarterly. Annual subscription: Domestic,

.$5.00; foreign, $6.25*. • Engineering and Instrumentation

Reporting results of interest chiefly to the engineer

and the applied scientist. This section includes many

of the new developments in instrumentation resulting

from the Bureau's work in physical measurement,

data processing, and development of test methods.

It will also cover some of the work in acoustics,

applied mechanics, building research, and cryogenic

engineering. Issued quarterly. Annual subscription:

Domestic, $5.00; foreign, $6.25*.

TECHNICAL NEWS BULLETIN

The best single source of information concerning the Bureau's research, developmental, cooperative and

publication activities, this monthly publication is designed for the industry-oriented individual whose

daily work involves intimate contact with science and

technology

for engineers, chemists, physicists, re- search managers, product-development managers, and

company executives. Annual subscription: Domestic,

$3.00; foreign, $4.00*.

• Difference in price is due to extra cost of foreign mailing. Order NBS publications from:

NONPERIODICALS

Applied Mathematics Series. Mathematical tables, manuals, and studies. Building Science Series. Research results, test methods, and performance criteria of building materials, components, systems, and structures. Handbooks. Recommended codes of engineering

and industrial practice (including safety codes) de- veloped in cooperation with interested industries,

professional organizations, and regulatory bodies. Special Publications. Proceedings of NBS conferences, bibliographies, annual reports, wall charts,

pamphlets, etc. Monographs. Major contributions to the technical

literature on various subjects related to the Bureau's

scientific and technical activities. National Standard Reference Data Series.

NSRDS provides quantitive data on the physical

and chemical properties of materials, compiled from

the world's literature and critically evaluated. Product Standards. Provide requirements for sizes, types, quality and methods for testing various industrial products. These standards are developed cooperatively with interested Government and industry groups and provide the basis for common understanding of product characteristics for both buyers and sellers. Their use is voluntary.

Technical Notes. This series consists of communications and reports (covering both other agency and

NBS-sponsored work) of limited or transitory interest. Federal Information Processing Standards Publications. This series is the official publication within

the Federal Government for information on standards

adopted and promulgated under the Public Law

89-306, and Bureau of the Budget Circular A-86

entitled, Standardization of Data Elements and Codes

in Data Systems.

Superintendent of Documents

Government Printing Office

Washington, D.C. 20402
UNITED STATES DEPARTMENT OF COMMERCE • Maurice H. Stans, Secretary

NATIONAL BUREAU OF STANDARDS • Lewis M. Branscomb, Director

Innovative Metrology—Key to Progress

Proceedings of the 1970

Standards Laboratory Conference

H. L. Mason, Editor

Presented by the National Conference of Standards Laboratories

June 15-17, 1970

National Bureau of Standards

Washington, D.C. 20234

f,

(J, S National Bureau of Standards. Special Publication 335

Nat. Bur. Stand. (U.S.), Spec. Publ. 335, 132 pages (Mar. 1971) CODEN: XNBSA

Issued March 1971

For sale by the Superintendent of Documents, U.S. Government Printing Office, Washington, D.C. 20402 (Order by SD Catalog No. C13.10:335), Price $1.50
NATIONAL BUREAU OF STABS*®

fid fOO

.U57 , Wo. ^ 31

Abstract

The biennial Standards Laboratory Conference of the National Conference of Standards Laboratories convened at the Gaithersburg facilities of the National Bureau of Standards June 15-17, 1970. The theme of the meeting, Innovative Metrology—Key to Progress, was amplified by 23 papers

presented at technical sessions devoted to new technologies and applications, laboratory management

and operations, new methods of optimizing calibration intervals, new ways of managing, and new

international developments.

Key words: Metrology management; National Conference of Standards Laboratories; physical measurement; Proceedings NCSL.

Library of Congress Catalog Card No. 70-608384
NCSL 70

CONTENTS

Page NCSL Board of Directors—1970 iv 1970 Standards Laboratory Conference Committee iv Foreword, Lewis M. Branscomb v On the Conference theme, J. L. Hayes 1 Measurement in a changing world—keynote address, Lewis M. Branscomb 3 Innovation in measurement, Myron Tribus 9 Session 1. New Technologies and Applications

Radiological mensuration technique, H. Bardach and S. F. Wisnieff 11 An automated time domain instrument test console, L. S. Kreyer 23 Standards laboratory applications of a computer-automated system, Frederick B. Seeley 29

Cost reduction in calibrating bolometer mounts. William F. Dentinger and Louis dePain 35 Session 2. Laboratory Management and Operations Reports

Measurement comparison programs, Herbert S. Ingraham, Jr. 39

Report on the 1968-69 mass measurement agreement comparison round robin, Lloyd B. Macurdy 41 NCSL specifications committee, Frank J. Dyce 47

Cost visibility exchange program—a new approach to cooperative savings, R. J. Barra 51 Session 3. New Methods of Optimizing Intervals Keys to optimum calibration intervals, Rolf B. F. Schumacher 57

Intervals by exception, Jerry Glassman 65

Optimizing calibration intervals, Donald J. Greb 73 Optimum cycles determined with confidence, P. A. Seamans 81 Session 4. New Ways of Managing

Breakthrough techniques for metrology work, Lloyd B. Wilson 85 The new State standards program, Robert Mills 89 The importance of visibility and control in laboratory management systems, John L. King __ 95

Data system for improving instrument reliability, Frank J. Dyce 99 New U.S. Air Force audit program, Lloyd W. Root 105 Session 5. New International Developments

NPL work on the determination of 2e/h by the ac Josephson effect, B. W. Petley and K. Morris 111 The impact of advanced electronic technology on measurements, Richard F. Clark 117 Design and development through metrology, Prem Prakash 121 Telecommunication measurements for automatic production and maintenance. H. Fleischer 125

Present status of electrical standards and their traceability in Japan, M. Kanno 129

iii
NCSL BOARD OF DIRECTORS—1970

Jerry L. Hayes, Chairman

U.S. Navy Metrology Engineering Center Pomona, Calif. 91766

Herbert S. Ingraham, First Vice Chairman

Radio Corporation of America, Defense

Electronic Products Division

Moorestown, N.J. 08057

Frank J. Dyce, Vice Chairman

Martin Marietta Corporation

Orlando, Fla. 32805

James F. Hadley, Vice Chairman

Bendix Corporation, Kansas City Division

Kansas City, Mo. 64141

William L. Vandal, Jr., Vice Chairman

McDonnel Aircraft Company

St. Louis, Mo. 63166

Paul H. Hunter, Secretary Western Electric Company, Inc. Winston-Salem, N.C. 27102

Don I. Hervig, Treasurer

U.S. Army SAFEGUARD System

Huntsville, Ala. 35807

Joseph M. Cameron, Sponsor's Delegate

National Bureau of Standards

Washington, D.C. 20234

Edward J. Arsenault, Delegate Member

General Electric Company, DVICS

Cherry Hill, N.J. 08034

William E. Boyes, Delegate Member

Sandia Corporation

Albuquerque, N. Mex. 87115

Paul D. Long, Delegate Member

Western Electric Company, Inc. Reading, Pa. 19603

Thomas N. Marshall, Jr., Delegate Member

National Aeronautics and Space Administration

George C. Marshall Space Flight Center

Huntsville, Ala. 35812

Wesley H. McPhee, Delegate Member

Massachusetts Institute of Technology

Instrumentation Laboratory Division

Cambridge, Mass. 02139

Harvey W. Lance, Past Chairman

National Bureau of Standards

Boulder, Colo. 80302

1970 STANDARDS LABORATORY CONFERENCE COMMITTEE

CONFERENCE CHAIRMAN, W. L. Vandal, Jr. TECHNICAL PROGRAM, E. J. Arsenault, R. J. Barra, C. Boyer, C. E. White

EDITOR, PROCEEDINGS, Dr. H. L. Mason

FINANCE CHAIRMAN, R. T. Cook

ARRANGEMENTS CHAIRMAN, W. R. Tilley

iv
FOREWORD

The growth, effective application, and wise regulation of technology depend upon the ability to make valid measurements. Such measurements in turn depend upon the existence of sound and evolv- ing physical standards and the effective functioning of a complex national measurement system, important elements of which are the standards laboratories that comprise the National Conference of Standards Laboratories.

The dramatic increase in the importance of measurement in areas of great public concern points

to future NBS-NCSL interactions on a broader scale in keeping with the theme of the 1970 Conference —innovative metrology. In addition to the traditional ties to NBS through involvement with the basic

measurement standards, NCSL's strengthened contacts with the measurement methodology in NBS

areas of materials research and applied technology will contribute materially to technological progress

that will benefit NBS, NCSL, and the technological community they serve. Because of this mutual concern for measurement, NBS is pleased to sponsor NCSL, to host its 1970 Conference, and to publish these Proceedings. Primary responsibility for the technical content of

the papers rests, of course, with the individual authors and their organizations.

Lewis M. Branscomb, Director

v

NCSL 70

ON THE CONFERENCE THEME

J. L. Hayes, NCSL Chairman

U.S. Navy Metrology Engineering Center, Pomona, Calif. 91766

As noted in your program, the theme of the 1970

Standards Laboratory Conference is "Innovative Metrology—Key to Progress." For those of you who are

in a financial squeeze this year (and who isn't?), per- haps the theme would be better stated as Innovative

Metrology—Key to Survival! Either way you put it, the application of innovations and fresh new ideas into the management and operation of the calibration and

metrology business is essential if we are to survive in today's economic climate. We can't survive unless we

progress into more effective ways of managing the calibration resources that are at our disposal.

Innovation is a catchword that many of us accept

as being full of hope and excitement; however, it is also full of emotional impact and heavily burdened

with responsibility. To innovate is to do more than come up with a bright idea. Innovations are impotent

actions unless they precipitate valid change. Progress

inevitably means change. Change begets fear of change;

resistance to change is often the result. In addition, natural inertia must be overcome. If a sufficient will does not exist to execute the innovation, little or no

gain results. As managers of metrology operations, we all need

the will to win, to persevere and doggedly to pursue

innovations to fruitful conclusions. We also need the courage to try, and in trying, the courage to fail, if need be, to make significant contributions. I feel that our present responsibilities and the theme of the Conference bring us face to face with a dual problem of considerable magnitude. We first must have the energy

and willingness to precipitate innovation. We can ac- complish this by brainstorming and establishing a working environment that is conducive to the generation of new ideas which are directed towards improving our operations both in quality as well as in quantity. Secondly, we then must have even greater amounts

of determination to test that environment by actually

bringing about the changes that must be made in spite of the natural bureaucracy of our surroundings and

the need for security that both our personnel and ourselves feel. This is bound to take all the wits we can

muster, the sensitivity to the effect this has on the people involved in the change, and the absolute deter- mination to succeed. You've all seen it, so have I

the situation where someone has a great idea but it never gets hatched into useful reality because no one was willing to suffer the real agonies of implementation. Perhaps there's a human simile that fits here: the

joys and heady delights of innovation or conception

must be followed by the pain and agony of project

completion or childbirth, the only difference being that on the one hand we must force it to happen; on the

other, it occurs naturally.

There is no question in my mind that we can innovate, nor is there any question that we must innovate.

No matter what the fiscal climate of our nation is, it

is our responsibility to do our jobs in the most cost- effective manner. That means that we should first chal- lenge the very jobs we're doing and confirm that they

are meaningful. Once having established that, we

then must seek the most cost-effective means of execut- ing those jobs and of maintaining appropriate quality

levels in doing so. Specifically, operating cost reductions must be achieved without a loss in product quality. In our case, we must calibrate instruments faster and better and at the same time provide our customers

with test equipment which is even more reliable during

its period of use than it is now. Is this a wild idea?

Maybe. An impossible dream? No, sir! I contend

it can be done and is being done. It can be done by

each of you if you have the desire. It must be done

by each of you if you wish to survive in your present

job environment and if you wish to make a meaningful contribution to your company or organization.

I firmly believe we can together develop ways and means to achieve significant improvements and to save our organizations significant financial resources if we

put our minds and our hearts into it. Individually, we may have certain ideas that have not only been

thought of but have been tried and proven to work.

If we can but share these together, the mutual benefit

to all of us within NCSL could be incredible. NCSL

intends to initiate regional meetings in the fall of this year that will give each of you a chance to meet with

your counterparts from the surrounding area to share

ideas and innovations. Hopefully, from each of these

sessions a composite listing of ideas, both tried and

untried, can be collected and disseminated for possible

application within your own organization. I've done some of this in my own organization and the results have been exciting and full of cost-reduction potential. But, during the next three days you'll hear of numerous innovations that you can employ. To achieve progress through innovation, you've got to want to change, you've got to instill in your people the desire

to change, and you've got to make the change. It's up to you

1

NCSL 70

MEASUREMENT IN A CHANGING WORLD—KEYNOTE ADDRESS

Lewis M. Branscomb

Director, National Bureau of Standards, Washington, D.C. 20234

It is a pleasure for me to open this Conference, a meeting that I believe will be both fruitful and effective.

I think it significant that the group here today is not quite so large as it sometimes has been. The field of metrology, along with every other field of technical endeavor in this country, seems to find itself on the defensive now. But I think it would be a mistake

for us to misconstrue that situation: I do not believe

that there is any significant diminution in the basic

public faith in science or in the importance of the

fruits of technology to the national life. It is true that in the metrology field some people

feel that the challenge of reliable quantitative meas- urements may not be quite as glamorous as space

travel or some other dramatic new application of sci- ence. And sometimes we complain that measurement

standards work cannot get management attention or

that it is too low on the priority totem pole. Many

people end up feeling that running a standards lab is rather like running a stock room, and you'll have to admit that there are many standards labs that do operate rather like a stock room—providing instruments

that are calibrated with tender loving care to people

who use them in some manner that may not be well known to the man who calibrated the instrument. And

the standards lab that runs like a stock room is not

usually in a good position to know whether or not the user made effective use of the effort that went into the original calibration. But like everything else, the measurement standards

field is feeling the winds of change, change that will dramatically elevate the importance and the profes- sional standing of standards work. Public awareness

of measurement is also, I believe, going to increase.

I will speak in a few moments of the importance of measurement in environmental and technological management. But let me parenthetically call your attention

to the fact that after 100 years of debate and 11 years

of consideration of a specific measure, the Congress

did, a year and a half ago, pass a Bill authorizing a study by the Department of Commerce of the conse- quences to the United States of increasing use of metric measures. We have one year left of the three given to make the study, and in August 1971 the Sec- retary of Commerce must report to Congress the results of our considerations.

I believe that this study, if done properly, will shed

light on the significance of measurement in the national life. The role of measurement is badly misunderstood in most quarters, and only if we understand

it properly can we assess the consequences to the United

States of changing our measurement language. That,

of course, is what going to the metric system really means—substitution of the international system of measurement for the present mixed system of customary units and various versions of the metric system

used in different disciplines.

As we examine the consequences of metrication we will be able to sort out those parts of the measure- ment system that concern the language we use from

those parts that actually influence the technology and

will require changes in hardware in order to adapt to a change in the measurement system.

We are carrying out this study through different methods for gathering information. One of these methods is a set of national conferences opening in August 1970 with a meeting in Deerfield, Mass., at which we will have heard the views of the engineering

professional societies and several large segments of manufacturing industry. Other conferences will deal with such sectors of the economy as education, con- sumer problems, labor, and others. It is my personal

hope that the metrological community will make an ef- fort to provide us with input on the advantages and

disadvantages of increasing use of metric measure. But whatever the conclusions of the study, I believe

that as of a year from this summer we will see major

attention being given to the significance of our meas- urement language in the national life. We are also now seeing a dramatic increase in the importance of effective standardization, for quantitative, compatible measurement is essential to the wise

regulation of technology for the public benefit. The

last 15 years have seen the United States science policy focus on the exploitation of our scientific capability

through technological application. We have done some

fantastic things in that period to enhance our military

security, to increase our sources of power production,

and perhaps most dramatically, to make possible man's

first voyage to the moon.

But the next 10 years will be increasingly characterized by the Government's effort to see that science is used for the benefit of man and that technology is

3
mastered by mankind rather than becoming his master. As a Nation we must learn how to live in harmony

not only with nature but also with man's own works,

the processes and the products of his technology.

It is obvious that regulation of technology for en- vironmental improvement requires measurements to find out what is happening to the environment, to es- tablish technological alternatives for its management,

to determine the levels of pollution which will be ac- ceptable under various circumstances, to trace pollu- tion to its sources, and finally to be the tool for regulating environmental pollution. For it is clear that we

cannot eliminate pollution of the environment. The

only alternative we have, if we are to expend energy

and not violate the second law of thermodynamics, is to determine the level and the character of that pollu- tion in reurn for the benefits that the polluting tech- nology will provide to us. The challenge is to find those kinds of pollution,

whether they be thermal or otherwise, that are rela- tively acceptable in relation to the benefits to be gained.

This will require an elaborate process of setting a standard and of regulating the performance of every

polluting organization or individual in the society in terms of that standard. If the standard is set on a steep part of the cost curve, then the financial incentive to come close to the edge of that standard without

dropping below it will be very great. And that will put great stress on measurements of not only the en- vironment, but of the technological process that pro- duces the pollution. Finally, when conflicts over the management of the environment reach the courts, the

validity, the credibility of the measurement system will come under judicial review. And I have said to many

of my colleagues who have been debating in recent days about whether or not there really is an important

future for measurement science, if you want to make

yourself a bundle, quit your job and set up a small

consulting company to provide expert testifiers to chal- lenge in court the quantitative creditability of our measurement system. I believe that in the environmental area the measurement capability leaves very, very much to be desired, and that under a very severe court challenge many of the measurements we make

could be shown to a jury to lack the credibility that is required.

Concern over the regulation of technology clearly goes far beyond environmental concerns. Public at- titudes are fast changing concerning safety and, be- yond safety, the consumer's pocketbook. Variations

in quality control that were once accepted as the nat- ural consequence of the vagaries of mechanical systems

are no longer acceptable. Society is now asking that we extract on a routine production basis the quality of technology that we know is in principle possible at a reasonable price.

Each new area of concern (environmental pollution, consumer safety and protection, etc.) will reflect itself back through the chain of engineering events. That

is, it will not only concern the measurement of the properties of the product or the amount of pollution in- jected into the environment, but will go right back

through production, production engineering, design,

research, to conception of the basic ideas. Thus the measurement system will be called upon to establish not only credibility, but assurance of accuracy and control. Regulatory teeth and major issues will stand behind the requirement for assurance of accu- racy and control. This will force us to face up to a

credibility gap in measurement standards work—in metrology, to use that shorthand label for all of the quantitative measurement assurance services in which we are engaged. This gap is created by a widespread

impression that the processes for assuring measurement

coherence too often obscure the purposes, and as a re- sult fail to achieve those purposes. So in the interest of conserving our resources, those of us who are in the measurement business need to ask ourselves several questions: What is the purpose of a given measure- ment? What is the intended user's need? What is his measurement capability? To answer these questions, we need a more effective information feedback system.

We must get out into the field and find out just what

is going on so that we can adapt what we are supplying to what is actually needed and used. We at NBS must give attention to several different phases of our responsibilities. First, we are responsible for establishing the central basis for physical meas- urements in the United States—establishing, improving, and maintaining these basic measurement stand- ards in our laboratories. These measurements realize the SI quantities—the quantities defined in the inter- national system of measurement. Second, we must

work down the measurement chain from these fundamental quantities—mass, length, time, temperature,

luminosity—to the hundreds or perhaps even thou- sands of practical measurement processes that are needed in today's science and industry, such as cryo- genic flow, microwave impedance, plasma thermometry,

and so on. Third, we must work on the problem of transferring into the secondary laboratories throughout the United States the basic measurement capability

that we generate here. Fourth, we must assure ade- quate verification that the transfer is sufficiently effec- tive for the user's requirement. This requires an active, outward-looking role on our part. There is no reason for NBS not being able to transfer its accuracy under all the conditions required into the laboratories of users in such a way that the user knows that he is tied back to the Bureau in an operationally effective way. Every other standards labora- tory has a precisely parallel responsibility with re- spect to its clients, and to that extent all of you share

with us the same nature of responsibility. Let me give

you a couple of recent examples to indicate some of

the things we are doing in two of those four responsibilities I named—those having to do with the estab- lishment of standards and their more effective dissemination. In length measurement, technology, through development of the laser, has again come to revolutionize

measurement science as measurement science in turn

pulls and leads technology. The laser obviously has

not only given the promise but the actuality of revolutionizing precision distance measurement because of

4
its particularly attractive properties of coherent light of very high brightness. We have recently in our re- search program found a way to stabilize a laser to an

accuracy of about one part in 10". We have in fact

built two completely separate lasers each oscillating in the near infrared at a frequency of about 1014 Hz, in- dependently stabilized them with completely separate

apparatuses, beat them together, and found the beat frequency to be only about 100 Hz. This development

can be carried a good bit further. We have the prospect of a wavelength standard whose reproducibility

is nearly competitive with the reproducibility of the frequency standard itself. Close on the heels of that development is the real likelihood that in the next year

or two our scientists will succeed in multiplying the frequency standard all the way up to the optical range, permitting direct comparison of the standard of wavelength with the standard of time interval. At that point we will have the opportunity, if we wish, to assign an arbitrary value to the speed of propagation of elec- tromagnetic radiation, and discard either the standard

length or the standard of time at our convenience.

This is an example of how rapidly the scientific basis

for measurement is proceeding.

Let me now mention a new contribution that we are making in time standards dissemination. Many people

are under the assumption that the two most essential uses of time are the measurement of time interval or elapsed time, and the determination of absolute time

in relation to the motions of the earth in its astronomical context. The former obviously is essential for any

physical measurement in which time enters as a parameter because, except in the field of cosmology, time

enters physical processes as a time interval. Absolute

time, or epoch, is vital to navigation, since we navigate on the surface of the globe with respect to geographic

coordinates. It is obviously convenient to do so in relation to the position of the surface of the earth in

its external environment of the moon, planets, and

stars. But it is increasingly important in technology

to be able to achieve simultaneity, which means the absence of a time interval between two remote locations. Simultaneity is now of very great importance

in many applications. Included among them are problems of space navigation, of collision avoidance for

aircraft, and in telecommunications. Pulse code modulation communications are made possible if there is an independent time base at all points in the system

so that the train of pulses can be decoded at the re- ceiving end. Two of the ways of achieving simultaneity around

the country are to proliferate atomic clocks or to es- tablish a satellite time signal system. You can proliferate atomic clocks, since every man's cesium is the same as the next man's, but the capital investment in clocks is rather prohibitive. The clocks must be car- ried from place to place to keep them synchronized.

Carrying clocks from one place to another is of course used to establish a coordinated time base for independent laboratory clocks, but this process is also expensive. The satellite system is a perfectly practical means

of establishing simultaneity technologically, but it is even more expensive.

Our laboratories in Boulder are now experimentally

investigating the feasibility of another way of estab- lishing simultaneity through color television. The idea

is basically a very simple one. The three major net- works in New York, in transmitting their color pictures, generate a very accurate ZV2 -megahertz subcarrier. It is possible to put a modulation on the subcarrier during the interval of time between pictures.

There are several lines on the TV screen on which one

can impress a code that will transmit a time signal

throughout the television network. An encoder at- tached to the TV set displays numeric time information on the screen. It is then only necessary for a given

laboratory to determine once what the phase delay is between its location and the source of this coded sig- nal, in order for the television signal to establish the time difference between the absolute time reference maintained by NBS and the laboratory's local clock. This provides a running correction for the local clock

as well as a basic time base in hours, minutes, sec- onds. This system is being evaluated experimentally

over TV stations in Denver and Washington. It is also being used to provide the frequency comparison be- tween the Naval Observatory clocks in Washington and

those of our laboratories in Boulder. We have per- mission from the FCC, AT&T, and the major TV net- works to proceed with the examination of this system

with a view to implementation on a national basis. We

hope to evolve a dissemination system which can pro- vide synchronization accuracy of a microsecond or

better in any laboratory which otherwise would have

to accept the millisecond precision inherent in real- time use of the ionospherically propagated signal from

wwv.

Beyond improving the basic standards and means

for transferring those standards effectively into the laboratory of the user, we must study the whole process of measurement in order to ensure that it serves

its intended purpose. Some people think of a meas- urement in terms of a measuring instrument. I think

the word "calibration" tends to convey that image. But a measurement can be understood only in terms

of a total man-device system. Heisenberg, in developing the uncertainty principle, taught us that there is really no meaningful measurement that does not affect the system being measured. We must also recognize

that the human beings involved in the measurement

system are an important influence on the measurement

as well as being influenced by it. We must also in- corporate the people making and interpreting a meas- urement in our understanding of a measurement. A measurement results when we have a theory for

the measurement, and know the system response in re- lation to a range of system circumstances—all those

that might show up in the measurement, including a

subset of special circumstances called calibration. This

special subset of circumstances gives the restrictions needed when the system is configured to permit a

theoretical tie back to the basic measurement standards. In the determination of physical properties car- ried out in many laboratories all over the country, this philosophy is too often ignored and as a result there

is entirely inadequate attention given to systematic er5
rors involved in absolute measurements of all kinds;

errors that derive not only from the intrinsic properties of the device being used but also from the manner and skill with which it is used. It is important to determine when absolute accuracy is really needed.

Then we must ensure that we really get what we re- quire. In the past we have too often accepted a cali- bration system that failed to assure this accuracy in actual practice. Perhaps such failures have occurred

in part because of realization that requested accuracies were not always required.

In the area of materials management we have an at- tractive method of transferring accuracy from a central standards laboratory into the field. This is the de- termination of certain properties of matter in absolute

units with known precision and accuracy. We are es- tablishing here at the Bureau a program on such

"bench mark data" devoted to a properly documented

determination of selected properties. And we will not

use the traditional standards of excellence of the scientific literature for judging the quality of our work in this area. Instead we treat a bench mark data problem just as any systems engineering problem, in which

you set up the necessary procedures for documenting

that such accuracy has been achieved well enough to satisfy any group of skeptics. I mention this program

because the scientific community accepts the idea that work may be done and published as correct and yet

still not be believed by the peers of the scientist per- forming the work. In the laboratories of those con- cerned with the measurement system we cannot ac- cept a system in which measurements are not accepted

as credible. In the establishment of absolute data, it is essential that we learn how to document an experiment so that it can stand on its own. A consistent national measurement system properly

coordinated with the system of other nations depends on several things: the existence of a theory of measure- ment based on the relationships among physical quantities; the establishment of a set of units and reference standards; the development of a set of technical pro- cedures for comparing a quantity to be measured with

the measurement unit; and finally, the development of measurement equipment for carrying out such comparisons and for determining multiples and submultiples of basic units. Too often in the past the thinking

of metrologists has been dominated by such matters

as traceability, calibration certificates, accuracy charts,

fee schedules, echelons, calibration intervals, accuracy

ratios, and class tolerances. These are material re- quirements or, if you like, design requirements, as op- posed to the performance requirements that we would

like to apply to the measurement system.

To illustrate the inadequacy of merely requiring

traceability, consider the following situation that could occur and indeed does occur in the area of mass measurements. Suppose a laboratory has its reference standards regularly calibrated by NBS, by shipping

them here and having them measured here. And having the best conditions for storing its weights, so they

are properly cared for, the laboratory finds that there

is essentially no change in the values as measured by

the NBS over an extended period of time. Surely this

is traceability at its best. Yet the measurements made

by such a laboratory may be subject to great error if environmental controls in the user laboratory are not

adequate, or if procedures are not properly followed, or the measurement equipment is in some way faulty.

Stated another way, we may know all about the lab- oratory's standards, but still know little about its ability to measure.

I was even told a few days ago of a standards lab- oratory which was making repeated traceability certifi- cation of the standards of a subordinate laboratory.

This subordinate laboratory seemed to have an extra- ordinarily good record, as its standards showed little deviation from year to year. Then it was discovered

that the subordinate laboratory had not even unpacked

the standards it submitted to the central laboratory!

Its staff just left them in the box and shipped them

back for recalibration the next year. Well, that is perfection in traceability and zero in measurement assur- ance! Concern over calibration interval, accuracy ratio,

and traceability is quite popular in the maintenance

of standards, and standards are a vital element in a

consistent system of measurement. But standards alone

are not enough to meet the demands of modern tech- nology.

The interchangeability requirements of industrial processes, the compatibility and authenticity require- ments of guidance and control systems, and the compatibility of measurement controls in commerce are

all examples of a need for a coordinated measurement

system. However it is the kind of measurements made

by the system and not just the quality of the standards

that is at issue. Measurements in a physically consistent system for which the requirements of compatibility and authenticity apply are incompletely and

sometimes even incorrectly characterized by the properties of the standards used. Our focus then should

be on the measurements that count and the process

that generates those measurements. Although NBS establishes measurement standards

for some 50 basic physical quantities, the transfer from

these standards to practical measurements is a responsibility resting primarily with users. However, the Bureau must know the user's measurement needs, so

that it can be responsive to providing standards and

methodology that are suited to those needs. The

transfer of measurement capability can be achieved

with the usual calibration approach, but for the highest accuracy work the laboratory may need to set up

an independent realization of the basic units from

their fundamental definitions. Or more economically,

a laboratory may tie to the Bureau by comparative

procedures so that that independent laboratory can be

as good as if the Bureau had set up an alternative laboratory of its own on their site. It is the answer to the following question that we

seek as a new approach for the dissemination of meas- urement capability: What procedures, how much work,

what kind of control, what performance criteria would

be needed to constitute evidence as to the quality of measurements of a laboratory? Practical measure- ment can be easily described in terms of its elements,

6
the concept of the quantity to be measured, the perti- nent physical laws, the various instruments used, the physical standards, the operators and the appropriate

procedures, the environment in which the measure- ment is to be carried out, the computations to be made,

and finally the means of establishing parameters of performance.

The measurement process is essentially a production

process, the product being numbers—that is, measured

values. A well-known characteristic of a measurement

process is that repeated measurements of the same

thing result in non-identical numbers. To specify a measurement process involves ascertaining the limit- ing mean of the process, its variability or imprecision

due to imperfections in the system, the possible extent

of systematic errors or bias from identifiable sources, and the overall limits to the uncertainty of independent measurements. When we have tolerances of ± 5 microinches on

ball-bearing diameters, the uncertainty in our measurements may be a large part of the tolerance. We

have to cope with the problem of stating in numbers

the degree to which we have achieved the desired goal. In all cases, the metrologist must understand the real goal of measurement effort. For example, in the case

of the ball bearing, the important element may be the need for a matched set of balls of identical diameter,

and it may be possible to produce sets of balls that match to much better tolerances than one would assign

to the value of an individual ball in an interlaboratory

comparison study.

Another example is in the measurement of light out- put from fluorescent tubes—the product of a 1 billion dollar industry. Here it appears that the marketplace

hinges on 1 percent differences in levels of performance

claimed by manufacturers—that's 30 lumens out of

3,000. On the other hand, we do not have sufficient confidence in our basic standards here at the Bureau

to assure us that it is possible to measure the light

emitted by a fluorescent tube to sufficient accuracy to ensure a one percent transfer of accuracy into the industry.

And it may be that in this industry not only the accuracy but even the precision with which fluorescent tube output can be measured is not as good as 1 percent. But the question that we must ask is "Is it nec- essary to try to have every lamp manufacturer in a

position to measure fluorescent tube output in absolute

units—that is, in lumens to better than 1 percent? Or

is it sufficient to ensure harmony in the industry with

respect to luminosity precision, allowing normal market forces to determine the adequacy of the level of

light output?" These questions are not easy to an- swer because they hinge not only on direct cost studies,

but on questions of confidence in the market process

to which measurement has an important contribution.

The key element in evaluating cases of this kind is that of obtaining a sequence of measurements of the same quantity. In standards laboratories this is fairly easy to achieve simply by remeasuring one of the lab- oratory standards as if it were an unknown, or using one or more items typical of the test items for the same

purpose. The routine calibration of the same object

used as a check standard tells us what the measurement

process can do. It is not just a simulation of the cali- bration process—it is the real thing, without the need

for any assumptions. Continued measurement of the check standard provides the basis for the precision statement and correlating its behavior with environmental procedures or other factors enables us to measure or set bounds for possible systematic error. For

control of the measurement process, this check stand- ard need not have the stability one would require of members of the standard group. Predictability of its performance is the basic requirement.

Having established control in one's own laboratory one then asks—what is the most effective way of co- ordinating my measurement process with others in the national measurement system? The problem is that

of transfer of measurement capability and again one

has to study this transfer mechanism as a process and

determine its operating characteristics just as was done

for measurements in his own laboratory.

The Bureau is considering new approaches for the

transfer of measurement capability in selected areas to avoid the wasteful duplication implied by having each

user develop his own procedures for assuring the ade- quacy of the transfer. For example, in voltage stand- ards the Bureau is engaged in a program of developing procedures using shippable standards for trans- ferring the value from the national reference group to other primary laboratories. The old practice of moving the customer's reference group to NBS for cali- bration left unanswered the question of whether the group survived the return trip unchanged, and placed on the user the burden for the study of the transfer mechanism.

The first or pilot program of this type was in the calibration of mass where the question of transportability of the standard does not arise. We first estab- lished that a state of statistical control existed in our own laboratory. Participating laboratories were re- quired to evaluate their own performance parameters,

and with the aid of a pair of kilogram standards bor- rowed from the Bureau could know to what degree

their work would be compatible with others in the system. A fundamental difference from their usual procedures is the emphasis on the continued verifica- tion of control within the laboratory and between it and other parts of the measurement system. This tells the laboratory how well it performs—and is more

meaningful than a certificate that says how well some

other standards laboratories performed. The amount

of effort is dictated by the measurement needs of the user and not by tradition. Similar efforts, which we

call Measurement Analysis Programs, are under way

for voltage, resistance, capacitance, temperature, length,

and several other quantities.

To optimize our efforts in serving the national meas- urement system, we must look at such objectives of our

calibrations as achieving compatibility among a number of laboratories. Even if an individual in a lab- oratory sends a device to the Bureau, and we make

the assignment of its value with the smallest possible

error, his job is not done. For example, if he sets up

his balance in a breezy hallway, his measurements are

7
not necessarily improved by having a super-accurate

NBS mass calibration. He still has to evaluate, in his own laboratory, the component of error due to his breezy hallway. In cases where the device may be al- tered by the trip to the standards laboratory his knowledge will be no better than the evidence in his laboratory that it has survived the trip. If there are problems in comparative measurements

or in transporting the standards it may be better that we develop standards that we can ship to the other

laboratories. Then they can calibrate their reference standards on their own site by using their own comparator and procedures, and all the associated sys- tematic errors will affect this transfer to the same

extent as it will their future measurements. This

eliminates a source of bias that would be present if the reference standards were calibrated at the standards

laboratory. The user then works only as hard as he

needs to in order that the error in the local inter- comparison be as small as required, while NBS concentrates on the maintenance of the absolute units and

the problem of the transport of those units to his laboratory.

These examples are illustrative of our desire to make

sure that our measurements are not only accurate but

also optimally relevant to the actual end use of the measurements. The measurement process is a highly

complicated one and needs to be evaluated constantly

as to efficiency and effectiveness, so that the services

that you and we provide will meet neither more nor

less than is required to satisfy the needs of users. My remarks have ranged from generalizations to technical details of the measurement process. I wanted

to emphasize the fact that any standards laboratory

has, I believe, the same responsibility that we now

are increasingly feeling—namely, the assurance of the

effective transfer of measurement into the hands of

users. And I believe that this requires a relationship

between the standards laboratory and the user in which

the people and the system are coupled and the stand- ards laboratory has the opportunity and the obligation

to evaluate the effectiveness of the user's measurement. Once a standards laboratory is in a position to do that, then that laboratory becomes an effective and

indeed an indispensable tool of management. We in the Bureau are finding in various areas of technology

that the integrity of the measurement system is increas- ingly impacting the policy-making levels of govern- ment concerned with the regulation of technology. If a standards laboratory is in a position not only to calibrate instruments, but to assess the quality of the job being done by those who are making measure- ments of importance to the laboratory's sponsor or

clients, then that standards laboratory becomes an

important element of overall management and tech- nological control in the client institution.

8
NCSL 70

INNOVATION IN MEASUREMENT

Myron Tribus

Assistant Secretary of Commerce for Science and Technology

U.S. Department of Commerce, Washington, D.C. 20230

This appearance marks a new experience for me. When I talk to engineering or educational groups I can

always say, "I feel right at home here because I have

been in this business." Tonight for the first time I can say to a group, "I feel right at home here because

I run a standards laboratory too." The word "run"

is a bit strong for in a real sense no one as far from

the bench as I am can "run" a precision laboratory. But my pride in this association encourages

exaggeration.

I am tempted to say, "My standards laboratory is better than your standards laboratory," but that would

not be fair. It is, but by definition, not necessarily by

any particular effort of mine.

I have often said that what you purchase from a

standards laboratory is negentropy—the reduction of uncertainty. We tend to think of that quantity in relation to the standards laboratory in a rather re- stricted sense. But it has an extended sense, and that sense of negentropy has a good deal to do with the reasons for which we are gathered here this week.

It is accepted, certainly by this group at any rate, that civilization is built on measurement. Civilization

is also built on negentropy. Our ability to plan and to build depends on our ability to know the limitations

of things, of devices, of institutions, and of processes. So in this sense the standards laboratory does indeed,

within its scope, provide the very stuff of civilization. As a main prop of civilization, the standards labora- tory is subject to the forces and currents of that

civilization. One of the main currents of our civiliza- tion, and one of its tribal gods, is innovation. And

so we are gathered here to perform a ritual soul- searching in quest of that elusive, magical power

which resides in all of us and merely needs to be

bidden to spring forth and grant its bounties. Unfortunately, my experience as engineer, educator,

and administrator has shown me that innovation, like most gods, is a capricious servant. Once summoned,

it has a way of overpowering sense and reason, of demanding an allegiance beyond that which benefits the believer. It becomes an end in itself. This aspect of the innovation problem is balanced

by the fact that few people, even those who talk and

promote innovation, really want it. Innovation dis- rupts our routines and our security. It challenges

our competence, and there are few of us who will put

up with that sort of thing.

So there are two conflicting currents here which the management of a standards laboratory, like any other modern enterprise, must keep in balance—the impulse

to change, and the resistance to change. The key to that balance is being able to recognize what constitutes progress—not change or innovation, but progress. Change for change's sake must be avoided; but no

change which promises improved operations and

services should be lost. These considerations are common to all management,

not just to the standards business. But in the stand- ards and calibration activity we have an added dimension which makes the decision-making process more

difficult and makes desirable changes perhaps more

difficult to introduce. That added dimension is the great degree of routine and established procedure

which a standards laboratory must have to insure the accuracy and repeatability of its results. Obviously

these routines must not be disturbed for light and

transient causes. Neither should the people in a standards operation

be disturbed unnecessarily. They are a special breed

with an orientation toward and a commitment to excellence. They will, rightly, resist any incursion into the quality of their work.

How does the manager of a standards and calibra- tion operation keep it moving forward with all deliberate speed in full recognition of these realities? It seems to me the first necessity is a firm understanding

of his operation and what it can and should do. What do we expect of a first-rate standards labora- tory? Two things stand out in my mind. First is the

ability to give the customer an unambiguous statement

of what the laboratory has done for him. Second is the provision of a range of services to meet the varying

needs of various customers. Some customers want

the utmost in accuracy in a calibration; some are

willing and able to sacrifice a decimal place or two for a quicker turnaround time; many are not so interested

in the extremes of accuracy as they are in extreme

certainty of the limits of inaccuracy. A first-rate standards laboratory must be able to meet the needs of

this range of customers and to be able to tell them

exactly what they are getting for their money.

9
It is up to the manager to make sure that his labo- ratory can meet the demands made on it in a sound

and economical framework—that is, in terms of the usual management decisions which must balance

services against investment in floor space, hardware,

personnel, and so on. Even as the manager develops for himself a good

idea of what his organization is about, he must communicate that idea to the people who work for him.

If he has an obligation to provide his customers with an unambiguous statement of what the laboratory is about, he owes the same to his people. The more

clearly the aims of the laboratory are stated and understood, the more surely the individual employee can

relate himself to the mission and feel a part of it. What I am advocating here is that a laboratory

should have a clear set of performance criteria for

itself and its employees. This is a necessity for knowing where you are and where you intend to go. More

to the point, these performance criteria give you a

basis for innovating. Like all performance criteria,

they make it possible to evaluate new technology in objective terms to see if it advances the state of the art or is merely change for the sake of change. Per- formance criteria enable the laboratory to make desirable innovations without undue disruption in operations and service.

I think you can understand my insistence that you

"know thyself." It is only to the extent that you understand your own operation that you can manage for innovation. But, how many of you really know the

trade-off between cost and accuracy in your operation?

Do you know the trade-off between cost and turn- around time? Do you know your customers' trade-offs on these items? An even better question: does your

customer know his trade-offs on such items? If not, does your service include helping bim to determine

them? I can think of no greater waste of resources than giving your customers services they really do not need and should not have to pay for. So the message is: measurement man does not live by precision alone, but by all the dimensions and

interactions in the measurement and calibration process. If we are to talk of innovation, we must talk about innovation in all phases of that process. We

must not fall into the trap of thinking that innovation

is always aimed at one more decimal place. One can

easily see how a standards laboratory could improve

itself right out of business by pushing accuracy, and

thereby costs, in an unreasonable way. Innovation

may be aimed at one more decimal place, but it is equally possible and equally valuable, under the proper

circumstances, for it to be aimed at cutting cost for a

given accuracy, at increasing the knowledge of the uncertainty in a calibration, or at any one of a number

of other factors. To manage this process, you must know in a quantitative sense what the parameters of your operations

are, and most especially the cost-benefit relationships

in your operation, your customers' operations, and in the interactions between you and them. It is only with

this sort of knowledge that you can tell progress from

change and distinguish improvement from innovation.

In this connection I think that the recent development by the National Bureau of Standards of the Measurement Analysis Program is a significant step

in the right direction. This MAP—the idea must be

a good one since it already has an acronym—belongs

to the next generation of calibration techniques. NBS

is now offering MAP for mass calibrations and is investigating the possibilities for calibrations of voltage

standards and of gage blocks. MAP recognizes that the laboratory in possession of

a nicely calibrated set of weights has indeed nothing

else for certain. It is entirely possible to misuse

dreadfully a well-calibrated set of weights and provide

very poor calibration services. MAP is designed to

calibrate, in effect, the laboratory's entire operation,

to spot errors in handling, random or systematic

environmental effects, and so on. Even more important, over a period of time the MAP data give the standards laboratory a concise and

quantitative history of its operation so that it can spot

instantly any anomalies which might crop up. Now, you see, we are beginning to develop a tool which will give the manager of a standards laboratory

the kind of broad and continuing overview of his operations that I have been advocating here tonight.

The very facts that MAP is brand new, that it is just

being developed in a few areas, that it is being well received by the laboratories which have begun to use

it, are all evidence of how little most laboratories

really know about their own programs.

I am glad to say that NBS—one of my departments —is taking the lead in such an important effort. I know they will continue to develop and to expand MAP

to the benefit of the entire measurement system. I think all of you here should be thinking in these terms and using such tools as this to develop the most complete picture possible of the operations in your own

laboratory, your relationship to your customers, and

your place in the measurement system. It is only with

this kind of outlook that you can reliably tell culturally

induced innovation from true progress in the measure- ment and calibration arts.

10
NCSL 70

SESSION 1 : NEW TECHNOLOGIES AND APPLICATIONS

Chairman: O. L. Linebrink

Battelle Memorial Institute, Columbus, Ohio 43201

THE EVOLUTION OF A RADIOLOGICAL MENSURATION TECHNIQUE

Herbert Bardach and Stanley Wisnieff

Aveo-Lycoming Division, Stratford, Conn. 06497

A radiological mensuration technique was devised to measure the minimum wall thickness anchoring from interior corner points of small castings invested with internal cavities. The factors involved in the development of the mensuration technique which culminated in the manufacture of a semi-automatic machine tooled with a strontium 90 gage were examined in detail. These factors included included the phenomenology of beta transmission through plural media, the subtleties of counting geometry, and the characteristics of the beta spectrum, all of which contributed in some

measure to the efficacy and uniqueness of the beta emitter as a mensuration tool. The technique involved the positioning of an encapsulated strontium 90 point source contiguous

with the interior points at which the minimum wall thicknesses were to be measured and deter- mining these thicknesses by referencing the observed beta transmission counting rates to a

calibration curve.

Key words: Attenuation of betas; beta-transmission phenomenology; counting geometry; mass absorption coefficient derivation; strontium 90 wall-thickness measurement gage; thin-wall mensuration.

1. Introduction—The Mensuration

Problem

The emphasis in gas turbine design for greater

efficiencies and increased power/weight ratios has led

to the use of higher operational pressure and rotational

speeds. This has resulted in an increase in the stresses imposed on the power-stage turbine blades. Accordingly, in order to abate these stresses, it has been re- quired to invest these turbine blades with internal passageways to facilitate air-cooling and improve their metal temperature profile. This has been accomplished

by an investment casting process which can be used to cope with the complex curvilinear dimensional re- quirements of the blade's contoured surfaces. A typi- cal air-cooled turbine blade casting is shown in

figure 1. As a consequence of the effort to reduce blade metal

temperatures by the incorporation of air-cooling passageways, quality control inspection techniques had

to be bolstered to insure the retention of the blade's

structural soundness and dimensional accuracy. Such

inspection mainstays as x ray and Zyglo which were

well established for determining flaws had to be supplemented—for another structural characteristic had

soared into prominence: wall thickness. What previously had been spot-checked by a destructive section- ing technique had now to be scrupulously determined,

for if during the investment casting process an occasional miscast occurred in the form of an internal core

shift, so that blade walls buttressing one side of the core were made too thin, blades with such unduly thin walls

BIAOE CASTING VANE CASTING

Figure 1. Typical castings measured for wall thickness

by radiological mensuration technique.

U
might rupture under the imposed centrifugal forces.

If one such blade were mounted in a power turbine

wheel of 62 blades and should fail, the resultant weight

imbalance might cause the entire wheel to fail with

severe damage to the engine.

Several well-established mensuration techniques, in- cluding eddy currents and ultrasonics, were adapted

to tackle the wall-thickness measurement problem, but

all were confounded by an obvious difficulty: None of them could satisfactorily measure the wall thickness at the points of highest stress concentration—the linear distances (shown dashed in fig. 2) from the narrow

ends of the covered space approaching tangency with

from zero to a fixed maximum energy which uniquely

characterizes the particular species of beta-emitting

radioisotope. The effect is the generation of a con- tinuous beta energy distribution having two termina: j zero energy and a maximum energy. The beta spec1 trum thus reflects a complex beta-emission probability

distribution, to wit: Each radioactive atom has a probability that a beta particle will be emitted; and additionally, each radioactive atom has a probability dis- tribution associated with the range of possible beta

particle kinetic energies that can be emitted. (See

fig. 10 in appendix for depiction of a typical beta | spectrum.

WALL THICKNESS MEASUREMENTS SOUGHT

ARE SHOWN AS DOTTED LINES

BLADE SECTION NO CORE SHIFT BLADE SECTION WITH CORE SHIFT

Figure 2. Properly cast blade compared to core-shifted blade.

the external blade surfaces. It became apparent that current mensuration techniques reliant upon macrodispersive or echo-sensing phenomena could not focus

sufficiently on these wall thicknesses to provide accurate

results. Subsequently, an innovative dial micrometer

gage was tried, but the narrow (5/64 in.) access holes

to the internal passageways, together with the steep

thickness gradient in the region of the tangency points,

especially when core shifting was pronounced, caused

difficulties which could not be surmounted. The conditions for resolving the problem were now beginning

to clarify. What was needed was a mensuration tech- nique which had very high resolution capability—

technique which could virtually focus in on the thin- walled corner pockets caused by core shifting.

2. Genesis of the Radiological

Mensuration Technique

After some rumination, the idea of using beta transmission for wall-thickness measurement was lit upon. The phenomenology of beta transmission

through thick foils was well established—electrons or positrons when given kinetic energy will issue in random orientations from the nuclei of radioactive

atoms undergoing nuclear-energy transformations at random time intervals. The kinetic energies imparted

to these charged particles are multivarious, ranging

Thus, beta particles of multivarious energies and

manifold orientations, many travelling close to the speed of light, stream pointillistically from the nuclei

of origin into the ambient matter, sustaining energy

absorption loss and scattering interactions with the atomic electrons and atomic nuclei. The interactions are profuse and the path of the beta particle is very

tortuous—a path that is the compounding of four

interaction phenomena:

( 1 ) Scattering by atomic electrons without beta energy degradation (scattering) ; (2) Scattering by atomic electrons with beta energy degradation (ionization) ; (3) Backscattering by atomic nuclei without

beta energy degradation (backscattering) ; (4) Scattering by atomic nuclei with energy

degradation resulting in transformation of beta energy

to electromagnetic wave energy (Bremsstrahlung)

Indeed, a moderately energetic beta particle sus- tains such a multiplicity of energy absorption loss and

scattering collisions while passing through a 0.020-in. thickness of stainless steel that it resembles a wavering,

weakening drunk, who in trying to toe the long white

line to its terminus reaches the terminus by virtue of

the multiplicity of his random deviations. During his reeling walk, the drunk is not apt to sober up, nor is the beta particle likely to convert its kinetic energy to Bremsstrahlung despite the multiplicity of collisions

12
t experiences. Actually a beta particle of 1 MeV

Energy, which sustains 103 to 104 collisions while

iassing through a 0.020-in. thickness of stainless steel, uffers a Bremsstrahlung generation of only a few

bercent of its total energy loss. This fortuity proves

o be of cardinal importance relative to the efficacy of

eta transmission as a wall thickness measurement

technique, as will be seen later. As a result of the multitudinous energy absorptions

and diverse scatterings it experiences, the effective path

ilength of the beta particle of kinetic energy E is proIportional to the actual path length and is a simple

linear function of E. Thus if 2-MeV monoenergetic

1 electrons were used for thickness measurement, the attenuation characteristic would be described by noting

the diminution, with increasing material thickness, of

!the conical solid angle formed by the pencil of rays

representing the range of possible effective electron path lengths. Then the extent of the conical solid angle emanating from the point of electron entry would

be constrained by the maximum effective path length

of the 2-MeV electrons. If this were 0.050-inch in

steel, then the loci of possible electron emission sites for an absorber thickness X would be circular areas

irR2 where R2 = (0.050) 2 - X2 which would plot

approximately as a straight line on Cartesian coordinates for a middle range of X thickness values. A formalization of the range of the electron path

lengths is Feather's rule, an empirical formulation

relating maximum electron energy to maximum thick- ness penetration for material of a given density. It states that for E > 0.6 MeV, the range of the electrons

in the material is given by

/?(g/cm2

) = 0.542 E - 0.133

where the range R is the maximum thickness penetration for the given material. Another empirical formulation is #(mg/cm2

) = 412 E

1 - 265 ~ °- n954 ln£,

which is applicable to the range of 0.01 ^ E = 2.5 MeV.

It is remarkable fact that both the identity and

uniqueness of the beta spectrum are preserved even as beta particles issuing from the nuclei of origin are gradually attenuated by successive thicknesses of

material. Thus, a beta spectrum having a maximum

energy of Emax retains its characteristic form regardless of the fraction of its total energy that has been

absorbed by matter. In other words, if an absorber

is used to reduce the beta particle intensity to a frac- tion of its original intensity, the beta particles streaming out of the surface of the absorber have the same

mass absorption coefficient 1 (cm2 /g) as the betas streaming into the absorber. This implies that the average energy loss per incremental path distance —AEt /&X is independent of X and is solely a linear function of the total energy Et ; also, that the probability distribution of orientations induced by scattering

and backscattering is independent of X. Thus,

dEt /dX = —kEt and Et — Eu e~~kX ; but as Et is equal

to the product of the average beta kinetic energy and

number of beta emissions, or Et = N • E&v%, and £avg

is invariant since the beta spectrum does not change

its proportionality, we then have Nx — N0 e~vx where

N0 is the initial number of beta emissions, Ns is the number of beta emissions transmitted through absorber

thickness X, and jx is the linear attenuation coefficient. A graphical display of mass absorption coefficients for a few pure beta emitters and their corresponding

thickness penetration values for Inconel having a density of 7.9 g/cm3 are given in figure 3. Feather's

s>

S i.i

MASS ABSORPTION COEFFICIENT.!*1/,))

3 4 5 10 20 1 1 I I I I

|

30 40 50 60 CIVT/G 1 1 1 I I I I

30 40 50 60

1 The mass absorption coefficient for a given absorber is the linear attenuation coefficient or transmission constant of the absorber divided by its density.

MAXIMUM THICKNESS PENETRATION IN INCONEL

(AS GIVEN BY FEATHER'S RULE)

Figure 3. The relationship of beta energy to mass ab- sorption coefficient and Inconel plate thickness.

rule is also delineated. The corresponding equivalent mass absorption coefficients for electromagnetic wave- form radiation (gamma or x ray) are indicated adjunctly to the depicted beta emitter. However, since

the authors, after a fairly thorough search, were unable

to find a commercially available radioisotope of halflife greater than one year which was either a pure x ray emitter (electron capture made), a pure gamma

emitter, an x ray and beta, or gamma-ray and beta emitter

with a composite mass absorption coefficient comparable

to that of any pure beta emitter, no identification of such

alternates is included. In fact, the use of a beta- gamma emitter of disproportionate mass absorption

coefficient would result in the gamma emission contributing a fairly constant background noise to the

beta's attenuation slope. To illustrate: The only beta

emitter of any consequential halflife which is more

energetic than yttrium 90 is the daughter of ruthenium

106. Ruthenium 106, a pure beta emitter of 0.039

fma!1 with a halflife of one year, decays to rhodium

106, a 30-s halflife daughter which has a beta of 3.53 Emax with an emission probability of 0.68 per atomic disintegration and gamma emissions of 0.513

13
MeV and 0.625 MeV with emission probabilities of

1.00 and 0.53, respectively. Since the mass absorption

coefficients of the 0.513 MeV and 0.624 MeV gammas

are less than 1/40 that of the 3.53 beta, the gammas

would behave as a significant noise factor, and thus contaminate the beta attenuation curve. It appeared,

therefore, that yttrium 90 loomed as the optimal choice

for measuring wall thickness in the range of 0.015 to 0.040 in., which was the anticipated dimensional range

of forthcoming mensuration problems.

Strontium 90, a pure beta emitter of 0.54 Emax with

a halflife of 28 yrs, decays to yttrium 90, a 64-hr

halflife virtually pure beta emitter of 2.27 jB max (1.734

gamma emission less than 0.0002 probability), which

in turn decays to stable zirconium. Whereas the strontium betas are stopped by a 0.009-in. stainless

steel foil thickness, the more energetic yttrium betas

are merely attenuated by 0.015-in. to 0.040-in. thicknesses. Since the attenuation gradient for Inconel

thickness in this range is steep and very nearly ex- ponential, wall thickness differentiability was readily

obtainable. The attenuation gradient may be ex- pressed as the change in counting rate (customarily

counts/minute) per unit change of material thickness,

where counting rate is the number of discrete radiations entering a radiation detector and counted during a time interval. The attenuation of a pure beta

emitter such as yttrium 90 with increasing material

thickness over a significant fraction of the yttrium 90

beta penetration range is closely approximated by a

straight line on semilogarithmic paper, where counting

rate is the logarithmic scale and material thickness

the linear scale. From a macroscopic point of view, the principle of using beta transmission for measuring blade wall

thickness at a particular site appears simple and

straightforward. A radioactive point source is juxtaposed beneath the blade wall site, and a radiation

detector positioned in proximity above. The number

of beta particles issuing from the radioactive source and transmitted through the blade wall is then de- termined as a counting rate by the detector, and the

wall thickness determined by referral to a calibration curve relating counting rate and wall thickness for the given counting geometry. See figure 4. However,

the reader can appreciate the pitfalls which can befall the user who is unaware of the intricate phenomenological interactions which may be contributory to the calibration curve. The previous discussion concerned

only the phenomenology of beta transmission through

a single medium. If now the problem is complicated

by the condition of plural media, as is the customary

condition in counting configurations, whatever contributory factors may be present in the single medium

will be magnified.

There are customarily seven media involved in the usual counting configuration. Beta particles issuing

from the radioactive source transmit sequentially

through the radioactive source itself, through the source containment, through air, through the wall

thickness, through air, through the detector window,

and finally into the detector sensitive volume.

If the radioactive source, source containment, detector window, and detector sensitive volume are maintained without material or spatial alteration throughout the series of counting periods, these media

will merely represent collectively an absorber which

reduces the effective range of wall thickness measurement. Clearly, if these media have collectively a high mass absorption coefficient, thickness measurement may

be precluded over a significant fraction of the thick- ness range.

Figure 4. Calibration curve, yttrium 90 transmission versus Inconel

plate thickness.

14
3. Counting Geometry Considerations

In the macro sense, the key to obtaining consistently accurate results is the maintenance of constant counting geometry during the measurement sequence.

Several subleties are implicit in this statement, since

the maintenance of constant counting geometry de- pends not only upon the spatial configuration of the three elements—radioactive probe (radioactive source and source containment), wall thickness to be measured, and radiation detector—but also upon the spatial configuration of the wall thickness itself. If the wall thicknesses to be measured have variable thick- ness gradient profiles or do not have the same ambient

matter distribution, measurement errors may ensue. A brief example will elucidate the factor of ambient

matter distribution. If the reader refers to figure 1 and notes the numerous air-cooling cuts along the edge of the turbine blade, and imagines a point source

of radioactivity moving along the interior corner of

the core space in which these holes terminate, he may

appreciate the periodically ascending and descending

counting rate that would be observed as the point source traverses the path length, and further appreciate that blades with such cuts might give thickness

variations from 0.002 to 0.015 in. in excess of the thickness readings for blades without such cuts. In- deed, this adventitious finding was used as a calibra- tion check to ascertain the proper spacing and positioning of these cuts! The turbine blade wall thickness measurement problem involves measurements of variable wall thickness

profiles, since it is the exceptional case of core shift that produces excessively thin walls. If the reader

refers to figure 2, he can visualize the possible core

shifts that would create this condition.

The dimensional properties of the radioactive probe were of paramount importance to the wall thickness

measurements to be undertaken. It was desirable to assure that the radioactive probe closely approximated

a point source, for excessive spatial extension of the source or of the source containment would broaden

the surface area impinged by the beta particles and

thus lessen focusing ability. The two constraints on

the diminution of spatial extension were construction

capabilities and rigidity considerations.

4. Description of Radioactive Probe

The radioactive probe (see fig. 5) was a sealed source comprising essentially a stainless hypodermic

tube within another stainless hypodermic tube, the inner tube encapsulating two 0.002-in. ceramic microspheres 2 perfused with a total of three microcuries of

ionically bound strontium 90. The strontium 90

microspheres were situated at the sealed tip end of

the inner tube, in a pocket formed by the insertion and securement of a 0.0094-in. piano-wire plunger in the inner hypodermic tube's 0.010-in. i.d. in a manner

so as to snugly buttress the microspheres against the inner surface of the tube's sealed end. The outer hypodermic tube, with an i.d. of 0.020 in. and an o.d.

2 The microspheres were fabricated by 3-M Company, St. Paul, Minnesota.

Figure 5. Strontium 90 probe.

of 0.032 in. tapering down to 0.025 in. at its tip, served

as an overshoe to prevent wear of the 0.018-in. o.d. inner tube encapsulating the strontium 90 source. If and when 0.0005-in. wear of the overshoe occurred, a diurnal calibration check, involving the comparison of a one-minute test count with the calibration standard

count of 3500 cpm, would be sufficient to discriminate

0.0005-in. wear from the wall thickness of 0.0065-in. thickness at the tapered radioactive end. After construction, the probe's beta emission profile

along its length was determined. With the strontium

90 probe in a fixed position and a 1.4 rag/cm2 end- window Geiger-Mueller detector viewing it from above,

a 0.015-in. stainless steel foil was interposed between

detector and used as a length interceptor coverlet. The foil was indexed in 0.01-in. increments to cover

successively greater lengths of the probe, and the counting rate of beta emissions was noted for each

position of the foil. For probe length interceptions of

zero, and of 0.01 through 0.09 in. the counting rate was steady at 10,800 ±500 cpm; however, when the

foil was indexed to 0.01 in., the counting rate dropped

dramatically to 6450 cpm, indicating at least partial coverage of the strontium 90 source by the foil. Subsequent indexing to probe length interceptions of

0.11, 0.12, 0.13 in., etc., failed to alter significantly the counting rate, which was sustained at the level of 6500 ±300 cpm. Thus it could be concluded that the radioactive source was effectually contained with

the 0.09 to 0.10-in. interval of the probe length. Itera- tion of the foregoing procedure for other rotational orientations of the probe established the near-radial

isotropy of the probe's beta emission profile at the 0.09 to 0.10-in. interval, there being a maximum deviation of 7 percent in beta emission intensity attributable

to the minute variation in hypodermic tube wall thick- ness girding the radioactive microspheres.

5. Phenomenology of Beta Transmission

Through Plural Media

There are two principal factors involved in the mechanics of beta transmission through plural media:

15
geometric dispersion and attenuation. While the former concerns dispersion in space, or in-vacuo dis- persion, the latter refers to the combined energy

absorption loss and scattering interaction phenomena

of betas with the atomic electrons and nuclei.

If the medium is air and the beta emitter is suffi- ciently energetic, attenuation will be negligible for

brief transmission paths. Under such conditions,

transmission through air may be deemed geometric

dispersion. Consequently, if an yttrium 90 point source opposite a flat surface is separated by an air gap, the interception of the yttrium 90 betas by the surface will be represented by a probability density

distribution which is a function of geometrical considDETECTOR

METAL

Figure 6. Chart guide for "Phenomenology of beta transmission in plural media" discussion.

erations alone. Referring to figure 6, wherein a planar

representation of a symmetric counting geometry configuration is depicted, we note that if surface 1-5 were assumed as being flat, the probability distribution of betas along surfaces 1-3 and 3-5 would be precisely the distribution of cos 6 where 6 is the angular dis- placement of the geometric ray from axis 0-3. In extending this analysis to the actual three-dimensional

counting geometry configuration, the planar representation is revolved about the axis of symmetry generating circumferences which have probability values corresponding to a cos 0 {A tan 6) distribution.

If the constant A is disregarded, this distribution reduces to sin 9. Since there is symmetry, the posi- tion of the beta in the circumference is immaterial

and the sin 6 distribution can be properly depicted

as being distributed along lines 1- 3 and 3-5. Thus,

the probability of a beta entering circumference 5

would be greater than that of beta entering circumference 4, and the probability of a beta entering a circumference more remote from the origin that cir- cumference 5 would be greater than that of a beta entering circumference 5, and so forth. This consideration instructively emphasizes the importance of

minimizing the source size, source containment, and

source distance from the thickness to be measured, if

it is anticipated that the wall thickness profile may

vary. Ideally, the radioactive source should be a

point source contiguous to the point at which the wall thickness is to be measured. If this is nearly accomplished, then the procession of beta transmission

through the wall thickness will resemble the conical

dispersion of rays shown issuing from point 2. Beta transmission through matter combines the

factors of geometric dispersion and attenuation. Beta

particles entering at point 2 undergo so many diffuse deflections and collisions in a 0.020-in. metal thickness

that the probability distribution relating to attenuation

is merely a function of the effective path lengths

travelled by the betas. The factor of beta attenuation, however, is complicated by an ancillary factor: Bremsstrahlung.

Bremsstrahlung is electromagnetic wave radiation

which is generated by the interaction of the beta's wavefront with the coulomb field of an atomic

nucleus. During this interaction any possible fraction

of the beta's energy can be converted to electro- magnetic wave radiation. Thus, when the beta in- tensity is sufficiently high, the multiplicity of interactions results in the generation of a Bremsstrahlung spectrum characterized by the diminution of photon intensity with increasing photon energy; i.e. photon intensity is maximized as the photon energy

approaches zero, and is minimized as the photon

energy approaches Emax. Since the probability of pro- ducing Bremsstrahlung is a function of the number

and intensity of the nuclear coulomb fields confront- ing the beta particles, the intensity of Bremstrahlung

generated will increase to a maximum value for a material thickness corresponding to the beta range.

Thereafter, for greater material thicknesses, Bremsstrahlung attenuation will occur, albeit at a much

lesser rate than that of beta attenuation since the medial Bremsstrahlung photon energy of 0.2 MeV

has a mass absorption coefficient about 1/30 that of

its parent beta. However, the Bremsstrahlung photon

contribution to the beta attenuation curve will be in- consequential except for thicknesses approaching the range since the percentage of total beta energy converted to Bremsstrahlung is Z£max /3000 where Z is the absorber's atomic number, or about 2.5 percent

for strontium 90-yttrium 90. Since the Bremsstrahlung contribution to the yttrium 90 beta attenuation curve is insignificant for Inconel thickness loss than 0.040 in, we shall now

resume the analysis with the imposed constraint of

this thickness limitation. Referring to figure 6, one notes the typical effective path lengths depicted as originating from point 2. As

before the geometric dispersion probability distribu- tion is sin 0, but with the complication that 6 now

has a limiting value fixed by the range of the £max betas through the medium. If we calculate the relative probabilities of beta transmission along these path

lengths, as effectuated by the attenuation factor using

e

-i36.5(o.o2o/cos $) wnere 136.5 is the approximate linear

16
Table 1. Calculated thickness penetration probability values versus experimentally derived probability values

Relative probability values

Wall thickness

Combined geometric Calibration

dispersion and curve

attenuation factors (cpm)

(inch) 67° V o-136 5 ( 020/cOS 8

5 = 0°

sin 8 — 0.08795 5400

0.030

53

g-136.5 (.030/cOS 8)

8 = 0"

sin 8 — 0.01564 980

0.040

37° 2 e

-136.5 (.040/cOS 8 ) 8 = 0"

sin 0 = 0.00312 175

attenuation coefficient 3 per inch of yttrium 90 in a medium of 7.9 g/cm3 density, the proportion of these attenuation factor probability values will correspond

fairly well with the counts per minute (cpm) values given by the calibration curve for the thinner wall

sections, but suffer progressively poorer correspondence as the thickness increases, by dint of the fact that

the geometric dispersion conical solid angle is diminishing with increasing thickness.

It should be noted that these attenuation factor probability values would correspond to the calibration curve cpm values if a pinhole collimation window

had been interposed between the measured wall thick- nesses and the detector during the generation of the

calibration curve. Recalculating the relative probability values for yttrium 90 transmission through Inconel thicknesses

of 0.020, 0.030, 0.040 in., but now considering both

the geometric and the dispersion factors, and using an

effective maximum penetration path of 0.0505 in. (the

0.057 in. range of yttrium 90 betas in Inconel diminished by the source containment thickness of 0.0065

in.), we obtain the summations given in table 1. The

calibration curve cpm values abstracted from figure 4

are juxtaposed to facilitate comparison.

A collation of complete results is given graphically

in figure 7. The results are normalized. As antici- pated, the beta particle attenuation values calculated from the derived analytical expression of the combined

geometric dispersion and attenuation factors of an

uncollimated counting geometry corresponded much

more closely to the calibration curve values which

were determined from uncollimated counting geometry

measurements than do those values calculated from

the derived analytical expression for a collimated

counting geometry.

If these calculations are performed for other se- lected points on lines 1-3 and 3-5 and repeated at these points for varying gradients, it becomes clear that varying wall thickness gradients can produce seri3 Linear attenuation coefficients are experimentally derived; however, it is Dossible from a consideration of first principles to derive not only the analytical expression of the beta spectrum but also to use that beta spectrum in coordination with interaction-phenomena

factors and count-geometry factors to derive the calibration curve

analytically. An attempt to do this in a simplified manner is presented

in the Appendix.

2--

.020 .030 .040

INCONEL WALL THICKNESS. INCH

Figure 7. Comparison of calibration curve of figure 4

with analytically derived calibration curves.

ous measurement errors when the radioactive source

is not contiguous to the point at which the wall thick- ness is to be measured, but this implies precise positioning, for if the point source had been placed at point 5 rather than at point 3, the wall thickness measured would be somewhat greater than thickness 3-6.

Once the probability distribution of betas issuing

from surface 7—8 has been determined, the geometric

dispersion considerations are once more undertaken

for the air path to the detector window. Here again

important parameters loom, namely: the size of the

17
detector window, its distance from the surface of the

wall thickness facing the detector, and its orientation

with respect to the wall thickness. The window should be oriented to favor the line wall

thickness measurement sought. Collimation of the window or reduction in window size may be used to minimize "thickness summing" in the region of the

line wall thickness; however, this is contraindicated

if sample positioning is not precise since serious positioning errors may ensue. Some mention should be made at this point of backscattering since it is often a significant source of error. As the probability of backscatter (deflection greater than 90 degrees, of a beta by an atomic nucleus) is steeply ascending with diminishing energy of nonrelativistic betas, and inconsequential for relativistic betas, the following discussion will concern only the former.

The probability of backscatter relative to the possi- ble modes of interaction can be given as 0.25 Z

2 /j84 where is the ratio of beta velocity to the speed of

light in the medium considered, while the probability

of ionizing collision, the principal attenuation mode,

is given as (2 Z//3

4) In ( EV2 // ) where / is the average

ionization energy lost by the beta per collision and E

is in eV units. For Inconel the value of / is about 330

eV 1/ s 12 Z). Hence, the ratio of ionizing colli- sion probability to backscatter probability for beta energies less than 100,000 eV is 8/Z In (£\/2/330) or about 1.7:1 for 0.1 MeV betas. Thus, if there are

surfaces in proximity of the radioactive probe which

function as beta backscatter reflectors, and the count- ing geometry configuration has the detector viewing

these surfaces, a significant alteration of the calibration curve values may occur.

6. Test Results

A rigorous performance test was used to determine

the feasibility and precision of the pullulating radiological mensuration technique: Each of 24 vanes was

to be measured at 24 sites—these sites comprising

four interior corners at three cross-sections at both

sides of the vane—with the time allowance for meas- urement limited to 15 seconds. The measurements were made by manual insertion

of a strontium 90 radioactive probe, similar to the one

described in the text, which was flagged and graduated

to cue the operator as to the proper probe orientation

and probe insertion depth. An unshielded low-level beta background counter (background less than 5 counts/minute I was used to determine the beta trans- mission rates. A typical vane is depicted in figure 1. It was necessary, of course, to establish calibration curves. All 24 vanes were first counted at each of

their 24 sites. A 6-s counting rate was deemed suffi- cient to ensure good counting statistics, since the av- erage counting rate was about 600 counts/6 s and the anticipated count differential per 0.002-in. thickness

differential was 180 counts/6 s at a thickness of 0.026

in., the anticipated wall thickness median. Thus, if the precision criterion was a 0.002-in error at a two-sigma

deviation ( where for the appertaining Poisson distribution, sigma equals the equare root of the total number of amassed counts, the background being negligible) a two-sigma deviation would be ±2\/600 or ±48,

a value comfortably within the 180 count differential per 0.002 in. Eight vanes, those which to a first ap- ! proximation collectively provided an optimal range of counting rates for establishing calibration curves over an anticipated span of 0.015 to 0.040-in. thicknesses, were recounted for 1 min, and then sacrificed for the ] calibration curves. It was possible by virtue of the

flatness of the vanes, and through careful positioning

of the detector, to resolve the 24 formative calibration

curves, one for each wall measurement site, to a single master calibration curve. The eight vanes culled for sacrifice were sectioned

in four parts, and optical measurements taken of the minimum line wall thicknesses at the tangency points

at the confluence of the interior corners and internal

surfaces. The precision of these optical measurements

had a probable error of 0.0005 in. The sets of data

'

relating counts per minute to wall thickness for each

of the formative 24 calibration curves were then fed

to a computer which was programmed to determine

the value of the slope which best fitted each set of

data. The criterion used was the method of least ! squares. The slopes of each of these curves deviated

from the slope of the Inconel attenuation curve by less than 10 percent, the average being 5 percent.

Subsequently, a second computer program was run

using as data input the slope value of the Inconel at- tenuation curve and again the method of least squares was used, this time to determine the best-fitting slopeintercept value at a 0.015-in. thickness. Having es- tablished the constants of the equation

log counts at thickness X = —(slope • thickness X I + counts at thickness 0.015 in.),

a master calibration curve had now been resolved from

the 24 formative calibration curves, and this curve was used for determining the wall thickness of the remaining 16 blades from the 6-s counts which had

already been taken.

Table 2. Error distribution

Strontium 90 Gage Thic kness Measurements

Error Frequency

Thin Wall Medium Wall Thick Wall

Error* (X 10- i in.) 0.017-0.025 0.026-0.030 0.031-0.040

in. in. in.

0-1 97 93 23 m-2 35 52 20 2V2-3 8 25 12 3V2-4 3 7 6

41/2** 0 0 3

5 0 0 0 51/2** 0 0 1

*Deviation of strontium 90 gage measurement from 10X

reticuled optical measurement. **Measurements were taken to nearest 0.0005 in.

18
All 16 blades were then sectioned in four parts and

optical measurements taken of each of the 24 sites. A comparison was made between optical measurements

and radiological measurements,

in table 2. The results are given

7. The Machine

On the basis of the celerity and efficiency with which

the test measurements were dispatched, interest was

stimulated for the design of a prototype semi-automatic machine version of the strontium 90 radiowith the preset counts dialed into the limit switch for each of the eight counting positions.

If the displayed counts exceed the preset counts, in- dicating an overly thin wall, the reject lamp lights, the probe is withdrawn, the turntable automatically

indexed, the part stamped reject, the turntable re-in- dexed, and the part discharged into the reject bin. The

turntable then indexes the next blade into measure- ment position and the cycle is repeated. Accepted

blades which finish the cycle are simply discharged

into the accepted bin. The machine has proven to be fast, highly accurate,

Figure 8. Closeup of turntable, probe positioning mechanism, and detector mounts of radiological mensuration machine.

logical mensuration technique. Such a machine was

built for Avco Lycoming Division by the Accurate Machine Tool Company, Cleveland, Ohio. An overall view of its moving elements is given in figure 8. The

machine can be described briefly as follows: A turntable outrigged with four mounting fixtures

is indexed sequentially to four stations: sample loading, measurement, accept-reject stamping, and acceptreject discharge. The machine has both manual and

automatic functions. The manual function merely in- volves loading the blades and indexing the turntable

to the measurement station. Then the machine takes

over. In rapid sequence the probe, guided by servo motors

regulating orientation in five coordinates {X, Y, Z,

orbital, and azimuthal), is advanced to and retracted from four preset positions in the blade's interior pas- sageways, each time advancing into and pulling out

of a 5/64-in. diam port at the blade tip. At each

position a 6-s count is taken by the end-window GeigerMueller detectors viewing the convex and concave

walls of the blade. These counts are displayed on two

Baird-Atomic 530 readouts and electronically compared

and reliable. In a 1-hr period, the machine can di- mensionally check out a hundred blades at eight meas- urement sites while maintaining an accuracy within

0.002 in. over 95 percent of the time, for the thin- walled blades it is seeking to discriminate.

8. Appendix. A Non-Rigorous Analytical

Method for Determining the Beta

Transmission Calibration Curve

The beta transmission calibration curve which re- lates beta count-rate intensity to absorber thickness is derivable from the beta spectrum. If the beta spec- trum has been experimentally determined as by a magnetic focusing /3-ray spectrometer, or analytically

derived as from Fermi's theory of beta decay (Ref. 1), an empirical formulation such as Feather's rule can be

used in conjunction with the previously discussed characteristics of beta transmission through plural media

to perform successive integrations over the portion of

the beta spectrum which can transmit through the ab- sorber thickness span of interest. In other words, if

it were desired to determine the beta transmission cali19
bration curve of yttrium 90 for Inconel absorber thick- nesses of 0.015 through 0.040 in., one would first de- termine the beta energy having a 0.015-in. range. This

would then constitute the minimal beta energy required

for penetration through the 0.015-in. thickness. There- upon, using sin 6 to generate the circumferentially

symmetric probability density distribution of betas emitted from the surface, one calculates the sin 6 dis- tribution for each beta energy of the beta spectrum

capable of transmitting through 0.015-in. Inconel, and

appends thereto the corresponding beta emission probabilities as weight factors. Having obtained a column

of numerics which are summed up to give a probability

intensity value, one repeats this process for other thick- nesses in the span 0.015 to 0.040 in. In this manner

one generates the probability intensity values of the beta transmission curve. The justification of this method lies in the fact that the beta spectrum proportionation is virtually invariant even as the beta energies are being diminished by successive absorptions.

Thus one can reconstruct the beta spectrum with monoenergetic electrons and apply Feather's rule and the

sin 6 distribution to the continuum.

Table 3. Calculated beta transmission

probability intensity values

Beta

emis- Beta sion Energy fre- —

(MeV)quency Range 0.020 0.028 0.036 0.044 0.052 0.060 0.0675

(inch)

Generated sin 6 beta emission

probability values* per

absorber thickness (inch)

0.5 4.5 0.020 0 0 0 0 0 0

0.6 3.3 .028 0.792

0.7 2.4 .036 .833 0.702

0.8 1.5 .044 .879 .772 0.574

o.y 0.7 .052 .923 .842 .722 0.533

1.0 0.25 .060 .940 .885 .802 .685 0.500

i.i 0.05 .0675 .953 .910 .845 .758 .838 0.456

1.17 0 .0722 0 0 0 0 0 0 Beta transmission

probability

intensity value

6.86 3.70 1.61 0.582 0.166 0.023

Feather's absorption data 240 112 18 19 6.60 1.75

(relative intensity values)

* Numerics are weighted sin 8 values representing approximate

relative probability intensity values for variously energied RaE

betas being transmitted through and emitted from the surface of different thicknesses of aluminum absorber. Each sin 6 value is the complement of the cos 9 value representing the ratio of absorber thickness to beta penetration range.

In the absorption experiments of RaE betas through

aluminum foils, Feather used a thin mica-windowed

Geiger-Mueller detector. The RaE source was thinly

spread over a 3-cm area closely matched to the area

of the detector window. The source was mounted in a slight cavity in a wooden block and positioned at a

distance of 2 cm from the window.

In order to give credence to the this proposed

method, the authors used the original data of the RaE

(bismuth 210) beta transmission curve through alumi- num absorbers as experimentally determined by Feather

(ref. 2), in conjunction with the RaE beta spectrum

experimentally derived by Neary (ref. 3). See figure

9. For the sake of simplicity and elucidation, a his- togramic area summing technique was used instead of integration to determine the sin 6 distributions. These

approximations were sufficiently accurate as will be

borne out later. Since the maximum beta energy of RaE is 1.7 MeV,

the maximum penetration depth of RaE betas in alumi- num was 0.0722 in., and this thickness was adopted as the constraining effective path-length side of the conical solid angles formed by the beta transmission paths

in the considered span of thicknesses from 0.020 to 0.060 in. A summary tabulation of calculations is given in table 3. To facilitate comparison, some datum

point values from Feather's RaE beta transmission curve are presented in the same table.

04 0.6 0.8

KINETIC ENERGY. MEV

[1.2 E MAX

Figure 9. The beta spectrum of radium E [bismuth

210) from Neary (ref. 3).

Normalizations of the beta transmission curve de- rived in table 3 and Feather's beta transmission curve

are compared in figure 10. The reader will note the

close correspondence for the thinner aluminum foils and the lesser correspondence for the thicker foils. This is attributable to the absence of a Rremsstrahlung correction factor in the analytical method exemplified by table 3. Since the fraction of total beta energy which is converted to Rremstrahlung is ap- proximately ZE/3000 or 0.005 for aluminum foils shielding RaE betas, and the beta intensity for an unshielded RaE source is about four times the intensity

for a source shielded by 0.020-in. aluminum foil, then

20
we have as the maximum possible Bremsstrahlung

count contribution 0.005 X 6.86 X 4 = 0.14 count, as- suming that only one Bremsstrahlung photon is pro- duced per beta particle. Since the source was about

2 cm from the 1-cm diameter window, and the pre- ponderance of the Bremsstrahlung photon spectrum

may be assumed as penetrating the aluminum foil thicknesses, the fraction of Bremstrahlung entering

the detector had effectually two origins: Either the Bremstrahlung photon was generated at a point in the aluminum foil medium which was sufficiently close

to the point of beta entry into the medium to constitute a possible intermediate path position of a beta which could pass through the aluminum foil, or the photon was generated at a point in the aluminum foil medium or ambient matter too far removed from the point of beta entry to constitute such an intermediate

.030 .040 .050 .060

ALUMINUM FOIL THICKNESS, INCH

path position. Clearly the conical solid angle of the beta particles intercepted by the detector was approximately the same as that for the Bremsstrahlung be- cause of the limited 0.0722-in maximum penetration

distance of the beta. Thus since the fraction of the

total conical solid angle intercepted by the detector was

detector window area ^ \/\2

spherical surface area

then the Bremsstrahlung contribution derived from

beta particles not at possible intermediate path posi- tions was approximately 1/12 of 0.14 or 0.012 count. This value was relatively constant for all absorber

thicknesses. In addition the Bremstrahlung contri- bution derived from beta particles at possible inter- mediate path positions had a variable value of absorber thickness

0.012 X

0.0722

Figute 10. Comparison of Feather's curve depicting

radium E beta transmission through aluminum foils with analytically derived curve.

Thus, although there would be produced for a 0.020-in. aluminum absorber a total Bremsstrahlung contri- bution amounting to 0.012 + ( 0.020/0.0722 X 0.012) = 0.015 count, which is negligible compared to 6.86

counts, the Bremstrahlung correction for 0.052 and

0.060 thicknesses was significantly 0.02 and 0.022

counts, thus raising the calculated count values of 0.166

and 0.023 to 0.186 and 0.045, respectively. Accordingly, the adjusted normalized values would be 2.71 and 0.66. These Bremsstrahlung corrected values are shown in figure 10 as a dotted line.

If the reader proceeds to determine the beta trans- mission calibration curve by this a priori method he

will observe that the transmission curve is only ap- proximately a straight line on semi-logarithmic paper.

Clearly this preliminary determination will provide

him with a good guide for performing accurate curve

fitting of the experimentally derived data in establish- ing the calibration curve. Of course the user must

take into consideration the effective cumulative ab- sorber thickness due to the source containment and

the detector window, and adjust the abscissal values

accordingly. Furthermore, he must devise his count- ing geometry configuration so as to minimize backscatter.

9. References

[1] Evans, Robley D., The Atomic Nucleus, (McGraw-Hill Book

Company, Inc., New York, 1933) . An excellent simplified treatment of Fermi's Theory of Beta Decay is given in Chapter 17.

[2] Feather, N., Proc. Cambridge Phil. Soc. 34, 599 (1938)

[3] Neary, G. J., Proc. Roy. Soc. (London) A175, 71 (1940).

21

NCSL 70

AN AUTOMATED TIME DOMAIN INSTRUMENT TEST CONSOLE

L. S. Kreyer

EG & G, Inc., P.O. Box 98, Goleta, Calif. 93017

This paper describes a computer-controlled system which automatically tests the performance of oscilloscopes and other time domain instruments. The system utilizes advances in pulse technology

to speed up the calibration; process by replacing frequency domain test methods with time domain

test methods. Cost effectiveness of this system in terms of reduced man hours to calibrate an

instrument, reduction of systematic errors due to operator bias, and consistent reporting of test results are described.

Key words: Automatic measuring devices; automatic performance test; computer control systems; instrument calibration procedures; instrument performance testing; pulse technology; time domain

testing.

1 . Introduction

The growth of sophistication of electronic systems

has been proceeding at a rapid rate, due largely to inventions in electronic components and in the science

of cybernetics. These systems require a high degree

of expertise on the part of the technical community to maintain and to calibrate at a level which will assure measurement reliability.

Both the quantity and complexity of the testing and

measuring equipment used by a calibration facility to maintain these systems are growing in direct proportion to the systems themselves. The metrology and calibration program of every or- ganization is directly affected by this increase in the measurement requirement within its jurisdiction. Exact

quantities from which to judge are not available, but one may reasonably estimate that the burden placed

upon a calibration facility is growing at an exponential rate. The calibration laboratories are not only faced with

moving through the shop the apparatus requiring cali- bration or repair, but they must also acquire and

evaluate variables data on each piece of equipment

handled. The calibration process, beginning with writ- ing instrument calibration procedures, then measuring

parameters, recording the measurements, making ad- justments if necessary, etc., must be streamlined and

made more cost effective. In addition, the enormous bookkeeping problems as- sociated with the proper recording and processing of calibration data and their statistics have never been

adequately solved. Metrologists have long been aware

of the necessity of analyzing variables data on all indi- vidual equipments requiring routine calibration. The

potential gains resulting from such an analysis have

not been realized because of the difficulty and cost of acquiring and storing reliable data on all the equipment involved. Solutions must be found for the problem of the growing calibration backlog and the as- sociated bookkeeping.

2. Computer-Controlled Test Systems

Introduction of the minicomputer has made the use

of dedicated computers in areas of test and measure- ment economically feasible. Combinations of these machines with programmable test equipments have re- sulted in the introduction of automatic test systems

which have applications in calibration laboratories. Implementation of these computer-controlled test sys- tems in the calibration laboratories would automate many of the steps in the calibration process. For ex- ample, instrument calibration procedures could be

written in the form of program test tapes, which are more easily revised and distributed than the present

manuals; instrument performance parameters could be

tested more easily using automatic test machinery;

variables data could be recorded more reliably and

cheaply using automatic data logging. Any well de- signed automatic test system will provide sufficient input/output equipment to automatically load instru- ment test procedures and record variables data; and

will contain a computer which is easily interfaced with

programmable equipments and has sufficient memory

capacity to allow programming in one of the higher

level languages.

A factor which has not generally been considered,

however, is the reduction of complexity in automatic

test systems made possible by the use of time domain

test methods.

23
3. Performance Testing Using

Pulse Techniques

Time domain test methods are much more readily

incorporated into a computer-controlled system than

frequency domain methods because the elements needed

for stimulus generation and response measurement in time domain are pulse generators and digital-to-analog converters I stimulus ) ; sample and hold amplifiers and

analog-to-digital converters (response). These ele- ments are natural computer peripherals and are quite

easily interfaced to a computer and packaged in computer architecture. Further, they naturally fit into re- flexive control modes (self test). Frequency domain

test methods require oscillators (or sweep generators)

which are programmable to many discrete frequencies,

power amplifiers programmable to many output levels (stimulus), AC/DC converters, digital voltmeters, fre- quency counters, power meters, VSWR meters, etc. (response). These elements represent highly complex

instruments which are costly and difficult to interface

into an automatic test system. Their complexity, in turn, requires additional system downtime to assure

that the system is in calibration. The choice between time domain and frequency do- main test methods is dictated by the instrument being

tested. There is a natural tendency to reduce instru- ment bandwidth as higher accuracies are achieved,

resulting in two classes of test instruments: high-accu- racy narrow bandwidth, and low-accuracy wide bandwidth. The time domain testing approach is divided into two test categories:

1. Transient response test—determination of instru- ment frequency bandpass.

2. Steady state test—determination of instrument

sensitivity, balance and gain.

The former technique utilizes a fast pulse generator

to generate a stimulus and a sample and hold amplifier/

ADC to measure a response. The latter method uses

a DAC to generate a stimulus and an ADC to measure

a response. Dividing the test into these two categories

provides two benefits in system design:

1. It is not necessary to know the absolute amplitude

of the fast pulse generator in order to perform the

transient response test. This is true because the computer normalizes all measurements (rise time, overshoot, topline aberrations), with respect to measured

pulse amplitude.

2. Steady-state tests can be performed at DC using

the more accurate DAC/ADC.

4. Comparison of Pulse and Frequency

Response Measurements

The performance of electron voltmeters (EVM's) is usually checked by applying a known sinusoidal volt- age at a number of discrete frequencies and measuring

the output with a calibrated meter. An alternate to

this technique is to apply a fast pulse to the instru- ment under test and measure its response. Since this

fasl pulse i- composed oi a Hal frequenc) spectrum

ranging from zero cycles ( DC I to beyond cutoff of

the instrument, the shape of the output pulse describes

the instrument's frequency response. This section de- scribes tests in which frequency and pulse responses

of an HP-400H EVM were measured and compared at several settings of the frequency compensating net- u i irks. Figure 1 shows a block diagram of the HP-400H

EVM. This instrument is composed of an attenuator, wide-band video amplifier, full-wave rectifier, and mil- liammeter. A large amount of negative feedback is used around the rectifier-amplifier loop to correct for nonlinearities introduced by the amplifier and rectifier. The output milliammeter indicates the average value

of the waveform, and its scale is calibrated in terms

of the rms value of a sine wave.

CATHODE

FOLLOWER

FEEQBAf

NETWORK

I

I

I 1'

I I

' FREQUENCY RESPONSE 1 DIVIDER CALIBRATION ' FREQUENCY RESPONSE

FIGURE 1. Amplifier-rectifier voltmeter block diagram.

The input attenuator is divided into two sections. The first is a 1000:1 capacitive-resistive divider with

a conventional frequency adjust (C4). This adjustment is one of the most critical of all in the instru- ment, and any maintenance or frequency "tweaking"

should not be made unless its performance is checked.

The first test, then, was to check C4.

Frequency response of the instrument in its properly

calibrated condition is shown in curve IF of figure 2. We can see that the response is flat from 10 Hz to 4

MHz, per specification. The time response, shown in curve IT of figure 2, shows a standard step function. By purposely misadjusting C4, the frequency response showed a break point at 1 kHz, with the output sta- bilizing at 80 percent (at 6 kHz) of the low-frequency

value out to 4 MHz. The time domain response cor- responding to this condition is shown in curve 2T of figure 2. Examination of curve 2T shows that the pulse has

a rising characteristic with time. Proper setting of C4 would produce a square pulse. The departure of the curve from a step function is found to be 20 percent, the same as that produced by frequency response methods. It is interesting to note that, experimentally, proper setting could be made in the time domain much

more easily and accurately than in the frequency do21
1.5

IF / I"

_ FREOUENCY RESPONSE TEST POINTS _^l±Am)L-u>±imL-u> \ nub—u-uilib i AiAiiifc A

i nun

FREOUENCY , Hz

/"

_ PROPER

CURVE 1. CURVE 2.

1 4 C4 PROPERLY CAPACITY ADJUSTED

TOO LOW -v

ADJUSTMENT

2T

1 1 1 1 1 1

1 1 1 1

1

TIME, psec

Figure 2. Effect of misadjusting C4 on HP-400H.

nain. Note that this effect occurs at long pulse lengths,

ind has a time constant of approximately 100 /xs. From this we would expect the "break point" in the Frequency domain to be —- 3kHz.

The measured break point was very close to this value. The second test was to misadjust C21, the main

high-frequency compensator in the feedback loop. As

can be seen in figure 3, too little capacity resulted in the frequency domain curve 3F and its time transform

curve 3T. Note that 30 percent peaking occurred in frequency response. This is vividly shown by the ex10" FREQUENCY , Hz

Figure 3. Effect of misadjusting of C24 on HP-400H.

25
ponential ringing in the time domain (about 2 cycles

to 1/e). When C21 was adjusted too high, a value

rolloff occurred beginning at 1 MHz (curve 4F of fig. 3). Feedback amplifiers of this type are usually compensated for maximum frequency response and, as

such, will produce a small amount of peaking near the unity gain crossover frequency. The peaking is re- lated to pulse response, phase margin, and damping

ratio. In general, for maximum flat-amplitude re- sponse the quadratic damping factor will be about 0.7. This produces no peaking in the frequency domain,

but will produce about 5 percent overshoot when the EVM is subjected to the step input. Thus, optimum

frequency response will be achieved if the time do- main check is for 5 percent overshoot. Several other frequency compensation adjustments

can be made for individual voltage ranges of the test instrument. Tests using frequency and time methods

produced curves 5 and 6 of figure 4 for C14. These

curves are similar to those produced by C21 (fig. 3).

Discrete frequencies normally used to test this in- strument are shown by the circles on figure 2. The

main amplifier has not been tested because weak tubes

will cause a minor change at the very low frequencies

(10 Hz) and at the very high frequencies (>1 MHz)

due to a change in gain-phase margin of the feedback

system. These parameters are usually checked by ex- tensive frequency testing at the extremes of the frequency range, the very place where pulse testing is most sensitive. Therefore, we must conclude that pulse testing is as sensitive as frequency testing in detecting misadjustment, and that pulse response measurements are much

faster and better adapted to automated systems than

discrete frequency measurements.

5. Applications

Tlic K(i\(r model 3003 time domain test system is an interesting application of pulse test methods in a computer controlled test system.

Referring to figure 5, the system consists of ten major assemblies. A description of the function of each

follows

1. Operator Input/Output Console—Accepts program

tapes and/or test console operator keyboard entries and converts these data to serial format for servicing

by the stored program controller. It also converts program status messages and operator instruction to English Language statements which are typed on the operator console. Further, it logs instrument per- formance test results on punched paper tape for machine processing.

2. Stored Program Controller—Stores instrument

test program, transmits program status messages to the

test console operator, interprets and executes test proTIME, nsec

Figure 4. Effect of misadjusting of C14 on HP-400H.

26
HP 2114B STORED

PROGRAM CONTROLLER

ASR 33 OPERATOR

INPUT/OUTPUT

: ^>

ASTRODATA

P 27023 PROGRAMMED

DC SOURCE

ASTRODATA

27015/

27016

DC VOLTMETER

ASTRODATA

27032 DIGITAL CONTROL

UNIT

SAMPLE AND

HOLD DELAY

SIGNAL INPUT

CALIBRATION

TEST OUTPUT

V(t) OUT DELAY OUT

GR 1591 PROGRAMMED

AC SOURCE

105-125 VAC

FROM DIGITAL CONTROL UNIT

EG&G 3003 PROGRAMMED

CALIBRATION

FIXTURE

VERTICAL SIGNAL

LV POWER SUPPLIES

EH 153 SAMPLE

AND HOLD UNIT

EG&G 3003

SIGNAL CONDITIONER

UNIT

CLOCK

(TRIGGER)

DELAYED TRIGGER OUTPUT CALIBRATOR OUTP UT HORIZONTAL OUTPUT VERTICAL OUTPUT

TO PROGRAMMED

AC SOURCE J

HP 5330 PRESET COUNTER

TEKTRONIX

TU-5

PULSE GENERATOR

„ sample and hold [calibration "'fixture

Figure 5. EG&G Model 3003 time domain instrument test system; functional

block diagram.

cedure instructions. Transmits programming com- mands to the digital control unit, and compares test results with test procedure tolerance limits.

3. Digital Control Unit—Decodes programming com- mands from the stored program controller and routes them to the proper interface unit. Accepts digital data from measurement units and routes them to the stored program controller.

4. Analog-to-Digital Converter—Converts analog volt- ages, from the programmed calibration fixture and the strobing voltmeter to digital numbers which are then

fed to the digital control unit for processing by the stored program controller.

5. Digital- to -Analog Converter—Converts digital control unit to six analog output voltages. These volt- ages perform the role of stimulus to the instrument

function generators and amplifiers, and of control in- puts to analog interface units.

6. Strobing Voltmeter—Converts wide bandwidth in- put signals to narrow band output signals for digitization by the analog to digital converter. Under pro- gram control, it strobes signals from the programmed

calibration fixture at precise times for measurements

of voltage as a function of time.

7. Programmed Calibration Fixture—Under pro- gram control, it (1) conditions power supply voltages

to proper voltage and impedance levels for measure- ment by the analog-to-converter unit, (2) delivers pre- cision pulse waveforms to the instrument amplifiers

for transient response tests, and (3) conditions analog

voltages from the D/D converters for amplifier gain

tests. 8. Programmed Line Voltage Source—Supplies ac power to the instrument under test, and varies the

line voltage from 105 to 125 V under program control, for instrument power supply line regulation tests. 9. Preset Counter—Synchronizes the instrument un- der test with the strobing voltmeter, and services those program interrupts which signal functional completion. The counter unit provides a time base for real time sampling for the 100 /xs/cm and slower oscillo- scope sweep rates, and serves as an electronic frequen- cy counter for measurement of time mark interval,

pulse generator, repetition rate, etc. These equipments, in conjunction with programmed

test routines (software) developed by EG&G for in- strument calibration, constitute a test system which

automatically sequences the test necessary to validate performance of an instrument and which indicates accept/reject for each parameter in the performance

test sequence.

6. Operational Procedures

Introduction of automated test systems into workload areas requiring skilled people can lead to job dis- satisfaction, particularly if the system denies the tech- nician the feeling of accomplishment which he derives

from making a significant contribution toward some

goal. The engineer who reduces the worker to the

role of feeding a machine errs in his systems design;

he should recognize that:

1. Computer-controlled systems are capable of highspeed data logging, automatic equipment set up, and

computation (including comparison to tolerances),

but are not capable of making qualitative judgements.

2. Human beings have slow responses and cannot

rapidly make computations, but are capable of qualitative judgements.

27
Systems designed for use in calibration laboratories

should delegate to the computer the functions of measurement, computation and control, but should

make the operator an integral part of the system. The

operator is relieved of the rather mundane tasks of equipment setup, data recording, repetitive measure- ment comparisons and computations, but is the sole judge of instrument overall quality, and is responsible

for basic parameter determinations.

For example, in oscilloscope testing, the operator

must judge CRT display quality, trigger amplifier

hysteresis characteristics, amplifier unbalances, etc. The computer queries the operator on these measure- ments and proceeds as directed. Having stored measurement tolerances from the operator acceptances,

the computer can go on to repetitive tests. An important distinction is made herein. The

oscilloscope cathode ray tube is a useful indicator, but was never intended to be a precision measurement

device. Having established that the CRT is working

properly, the system can extract all other parameters

directly from amplifier outputs (in fact, more ac- curately, since the CRT reading accuracy is proportional to beam writing rate, focus, pattern distortion,

flicker, etc. )

Perhaps the most unique feature of the time domain

system is the interface unit, which among other things,

transforms the electrical signals extracted at the CRT

plates to levels suitable for processing by the strobing

voltmeter without sacrificing measurement system fre- quency bandwidth.

7. Conclusion

Criteria which make automatic test systems suitable

in calibration laboratories have been reviewed. These

are:

1. Input/output compatibility with the data reporting system;

2. Computer architecture and programmability

3. Interfaceability of programmable test equip- ments;

4. System complexity required to provide total calibration workload capability.

The first three criteria are met by any well-designed

automatic test system. The fourth criterion can be

effectively met by systems which utilize time domain

test methods.

The author wishes to thank Mr. Jerry L. Hayes for valuable assistance in estimating the performance re- quirements for oscilloscope testing; Mr. Arthur

Garafolos for assistance in defining the need for auto- mation in calibration laboratories, and Phil Painchaud

for valuable insights in metrology practices.

This work was supported by the U.S. Navy Metrology Engineering Center, Pomona, Calif.

28
NCSL 70

STANDARDS LABORATORY APPLICATIONS OF

A COMPUTER-AUTOMATED SYSTEM

Frederick B. Seeley

U.S. Army Metrology and Calibration Center, Redstone Arsenal, Huntsville, Ala. 35481

A computer-automated calibration system whose accuracy and precision make it suitable for standards laboratory applications is described. The system has been applied to electrical, radio- frequency, electro-optical, and electro-mechanical measurements. Results are summarized that include

saturated standard cell, standard resistor, resistance thermometer, micropotentiometer, and

attenuator calibrations. Additional topics include the experience in computer programming,

reliability, and systems operation.

Key words: Computer automation; resistance thermometers; resistor intercomparison ; standard cell surveillance.

1. Introduction

The purpose of this paper is to describe how a computer-automated system is being used to perform many

precision measurements typically required of a stand- ards laboratory. The approach taken using this auto- matic system often has differed from classical techniques and apparatus normally used by standards

laboratories. Several typical applications that will he

discussed illustrate the use of an on-line instrumentation computer to perform real-time control of experiments, computer-corrected measurement and stimuli,

unattended measurement processes, and the use of

alternative techniques that would be impractical with- out the computational speed of the computer. Some

applications to be discussed will also illustrate how

rapidly precision measurements can be made and how

statistical techniques can be used to increase further

the confidence in the measurement results. It is not

the intent of this paper to present the full details of each application, but rather to present an outline of the methods and apparatus used and the results that have

Figure 1. Computer-automated calibration system.

29
been obtained. The automatic system has been applied

to intercomparison of saturated standard cells, to within ±0.2 /xV, standard resistors to within 1 ppm,

resistance thermometer calibration within 0.01 °C, and

to the calibration of micropotentiometers and rf at- tenuators. The general methods used to obtain these

results will be presented, as well as a brief system

description, comments on the system software, operation, and general applications.

2. System Description

The computer-automated system shown in figure 1

is being evaluated for general metrology and calibration applications in the Army's calibration system.

However, the system is available commercially and is representative of many typical computer controlled instrumentation systems. The extent of parameter

coverage in a single system does make this one some- what unusual, however. Measurement and stimuli instrumentation, all computer controlled, operate over

the frequency range from dc to 12.4 GHz. The

computer is a 16K-word memory, 16-bit-word machine

with an input/ output extender that houses the inter- face between the computer and the instrumentation. A

magnetic tape recorder, photoelectric tape reader,

teleprinter, and graphic terminal provide the primary

peripheral equipment for handling programs and data. The instrumentation is extensive and includes two

digital voltmeters, a dc voltage/current source, an ac

voltage source, a ratio transformer, a wave analyzer, a frequency counter, and an X-Y recorder. An auto- matic network analyzer is included as a subsystem for making s parameter measurements of rf devices from

100 MHz to 12.4 GHz. Two switching units, a 200- channel crossbar scanner and a 4-channel 16-output

distribution switch, are also included as a programmable portion of the system. The software supplied

by the manufacturer of the system includes two highlevel languages, BASIC and FORTRAN. The BASIC

software package can be used to program the entire system, both microwave and nonmicrowave instru- ments. Assembly language capability also exists and

has been used for some special applications.

Before discussing the measurement applications, it is appropriate to mention some of the reasons for the

size (if this particular automated system. First, the concept of using a single, automatic facility to make

broad spectrum of measurements is being developed.

In this sense, the system is multipurpose and is being

used in a "job share" mode. Measurements and ex- periments are brought to the proximity of the machine,

rather than using several different automatic systems

at different locations. This operational concept relies upon the speed with which measurements may be made,

upon its reliability, and also on certain automatic

features which make possible 24-hr unattended operation, A second reason for developing a single multi- purpose system is that Army doctrine requires that calibration service be brought to the user's site. This

mobile calibration service is at lower echelon than what

would normally be considered "standards laboratory"

work. A development project is investigating the use

of computer-automated equipment for mobile calibra- tion operations in the Army. A mobile automatic

calibration system is presently being envisaged as a multipurpose measurement and calibration system.

3. Applications

3.1. Saturated Standard Cell Intercomparisons

A group of four saturated standard cells in an air- bath has been used in our evaluations of the stability

and precision of the d-c voltage measurement and

source instrumentation. It was originally intended to have these four cells periodically remeasured at our

standards laboratory and use in all subsequent calculations the individual cell emfs that had been assigned.

However, it was noted in a simple cell intercomparison

design that when the emf differences between these four cells were measured using the system's digital

voltmeter and crossbar scanner, the measured results were in agreement with the differences calculated using

the emfs determined in our standards laboratory to a

fraction of a microvolt! These surprising results prompted a redesign of the cell intercomparison experiment based on NBS Technical Note 430. Twelve channels of the crossbar scanner were con- nected to the four cells so that the 12 possible emf

differences between the four cells could be measured

by the system's digital voltmeter. A BASIC language

program was prepared which measures each of these 12 differences and calculates the differences of each

cell from the mean emf of the group, the new emf of each cell, the deviation of each observation, and the standard deviation of a single observation. Each

observation is actually an average of 10 voltage meas- urements made by the DVM. In the cell intercomparison, the difference emfs, which are on the order of 1 to 20 fiY, are measured by the DVM on its 100-mV

range, whose resolution in the last window is one

microvolt. However, the average of 10 or more single

observations has been found by direct test to be correct

to 0.2 fxV plus any zero offset. The design of the cell intercomparisons does use a left-right balance tech- nique and corrects for the zero offset in the system.

The entire cell intercomparison process is very rapidly

executed, taking about 12 s to perform. This time

includes the 120 voltmeter readings, crossbar switching, and the calculations mentioned above.

The process has been statistically studied over two

time intervals, one 16 hours long, the other 6 months

long. The primary intent of the shorter term experiment was to study the precision of the instrumentation

in the 16-hr period. The cell surveillance was per- formed, along with other DVM tests, every 30 min

beginning at the close of the normal workday. The

system's software clock was used to time and initiate the total experiment which lasted approximately 5 min.

The process was automatically repeated every 30 min

until interrupted by personnel the next morning. The

results of one such overnight cell intercomparison are shown in figure 2. As can be seen from the control

charts, the standard deviations for the standard cell intercomparisons are approximately 0.3 aiV. Similarly, the upper and lower control limits are 0.3 iiV.

30
Figure 2a. Typical 16-hour standard cell emf.

Figure 2b. 16—hour standard deviation of standard cell group.

The study of the same measurement process over a 6-months period was conducted by running the stand- ard cell surveillance program once a day. Each day

on which the program was run, the data were stored on magnetic tape. A separate analysis program wa9

written for accessing the data from tape and calculat- ing the average standard deviation, the average emf of each cell in the period, upper and lower control limits,

etc. The program also plots the above information on

control charts. The results of the longer term process

are consistent with those of the short-term experiment,

i.e., within 0.3 microvolts.

3.2. Standard Resistor Intercomparisons

The results obtained by using the digital voltmeter

to measure emf differences of 1 to 20 pN suggested

that the same measuring system could be used in lieu of a galvanometer at the output of a Wheatstone bridge

for measuring two terminal resistors. In this method, no attempt is made to balance the bridge. Rather,

the input and output voltages of an unbalanced bridge

are measured by the DVM and crossbar scanner sys- tem. For precision resistor intercomparison on the order of 1 ppm, a substitution technique is used. Four

resistors are placed in mercury-wetted stands wired in a bridge arrangement. The resistance value of only one of these resistors must be known precisely and

enters the program as the value of the "standard."

The precise values of the remaining three resistors are not critical in the calculations. The preliminary steps of the program include asking the operator "how many

unknown resistors" does he intend to measure? The

operator is asked to enter (via the keyboard) the

fractional or ppm value of the resistor being used as

a standard. After these preliminary manual steps, the measure- ment sequence is initiated by the computer after a programmed wait period. A programmable supply

applies voltage to the bridge network, the digital volt- meter is switched via the crossbar across the bridge

input, which is measured ten times and averaged.

Then the crossbar switches the digital voltmeter to the bridge output and an average of ten measurements is obtained. Next, the polarity of the power supply is reversed and the process is repeated. These two sets of measurements are subtracted to correct for small thermal emfs and zero offset in the system. The

process halts and a message is displayed that instructs the operator to "remove the standard and place the

first unknown resistor in the stand." An empirically

determined wait of 10 s was found to be necessary to allow for decay of transient emfs that arise when

resistors are placed on the stand. The computer then

initiates the same measurement process described

above, to measure the bridge input and output voltages

with the unknown resistor in the bridge. In less than

1 s, the ppm fractional deviation from nominal is printed out. The program continues instructing the operator to place unknown resistors in the mercury

stand until all are measured and the corresponding answers are displayed.

Two mathematical models have been used to calculate the value of the unknown. The first of these can

be used when the standard and unknown are nominally

equal. The input impedance of the digital voltmeter

is approximated as being infinite and the simple

expression,

(Vx - Vs) Cx = Cs + 4> Vin

is used to calculate the fractional or ppm deviation,

Cx, of the unkonwn from the nominal value. Cs is the ppm deviation from nominal of the standard. The

31
measured bridge output voltages for the unknown and

standard are Vx and Vs, respectively. Vin is the bridge

input voltage. A general solution has also been used.

Essentially, it consists of solving a system of five simultaneous equations that describe the bridge net- work. The five unknown quantities are the four cur- rents that flow through the four resistors in the net- work and the product of the current times the standard

lor unknown) resistor. By solving the system of equations, no assumptions are made about impedance of

the digital voltmeter and the measurements can be

made over a wider range of validity. Using either method, there is no delay that the operator can perceive, other than the 10-s delay mentioned above.

Some typical results of standard resistor comparisons are illustrated in figure 3. Tests have been con- ducted to determine the precision and resolution of

the method using resistors in air at 23 °C ±0.1 °C. The precision or repeatability of the unbalanced bridge

technique has not been studied statistically over a

long time period. However, the range of results is generally less than 0.5 ppm when measuring 100 to 10000 ft resistors. The accuracy of the difference measurement has been established by shunting a standard resistor with 1 Mft (±0.01% tolerance). In summary, it has been found that the combined effect of systematic and random uncertainties is less than 1 ppm for resistor intercomparisons.

3.3. Resistance Thermometer Calibrations

The redetermination of the constants for a platinum

resistance thermometer is a tedious time-consuming

task in most laboratories. An approach for automating this task was selected which was intended to yield

results adequate for most applications. The goal was

to automatically redetermine the thermometer constants and calculate a new table of temperature and resistance

ratios so that the thermometer could be used to make

measurements of temperatures with uncertainties not

THE STD. CORRECTION IS:?SE-•06 HOW MANY RUNS-'S PLACE UNKNOWN RESISTOR IN STAND

THE UNKNOWN RESISTOR CORR. = 6.93227E-06

PLACE UNKNOWN RESISTOR IN STAND. THE UNKNOWN RESISTOR CORR. = 6.59911E-06

PLACE UNKNOWN RESISTOR IN STAND

THE UNKNOWN RESISTOR CORR. = 6.SG571E-06

PLACE UNKNOWN RESISTOR IN STAND. THE UNKNOWN RESISTOR CORR. = S.0S602E-06 PLACE UNKNOWN RESISTOR IN STAND. THE UNKNOWN RESISTOR CORR. = 5.86614E-06

READY

REMARK: STANDARD '30234 = 1000.005 OHMS

UNKNOWN '30235 1000.005 OHMS +27^70

Figure 3. Resistor intercomparison output.

exceeding 0.01 degrees Celsius. The approach which was selected is to use a standard and test thermometer

in thermal equilibrium with a liquid bath or other

gradient-free environment. The resistance of these two thermometers is measured at three or more temperatures, and each temperature calculated. From

these data, it is a comparatively simple matter to calculate the least squares coefficients of a second

degree polynomial. In turn, these coefficients can be

used to generate a new table of resistance or resistance

ratio as a function of temperature.

The system's digital voltmeter and crossbar scanner

are used to measure the voltages across the standard

and unknown or test thermometer, and also the voltage across a standard resistor. The resistor and two ther- mometers are connected in series and 2 mA thermometer current is provided by a programmable constant current source which is stable to 10 ppm per day. The

resolution and linearity of the digital voltmeter are adequate for determining the resistance ratios of the two thermometers with respect to the standard resistor. For example, the linearity of the digital voltmeter has

been found by direct test to be better than 10 ppm.

This amounts to an equivalent temperature error of

less than 0.0005 °C. The resolution of the digital

voltmeter on its 100 mV range is limited to 1 fjN for

a single observation which corresponds to a temperature resolution of 0.005 °C. By averaging ten or more digital voltmeter observations, a resolution ap- proaching 0.001 °C has been achieved.

A BASIC language program has been prepared

which accomplishes the calibration of one platinum

resistance thermometer with reference to a standard

thermometer. Figure 4 illustrates the instructions,

data, and results as displayed on the system's graphic

display terminal. In this example, calibration is per- formed at the ice point, room temperature, and 70

degrees. The software clock is used to program a 5-min wait period to allow each thermometer to reach

thermal equilibrium with the bath. After this 5-min

wait, the resistor and two thermometer voltages are automatically measured and stored in memory. The

operator is instructed to place thermometers in the next bath. The process is repeated until the third set of data is obtained. The temperature is then calculated from the measurement data for the standard re- sistance thermometer and its constants which have

been previously included in the program. The meas- urement data, temperature, and resistance of the un- known thermometer are listed. A solution for three thermometer constants, the zero resistance, the coefficient for a linear temperature term, and a coeffi- cient for a quadratic temperature term, is found.

These constants are used to generate a table of

resistance ratio and resistance as a function of temperature for the unknown thermometer. In the ex- ample shown in figure 4, the table has been generated

in 5-deg increments, which of course is an arbitrary

choice. Any temperature interval may be chosen.

The accuracy of the results has been verified by

intercomparing thermometers each of which have manufacturer's or NBS calibration reports. In general,

the goal of ±0.01 degrees uncertainty appears to have

32
BUN

PRT CALIBRATION PROGRAM. NUMBER OF TEMPERATURE SATHS^S

INSERT PRT'S IN BATH NO. 1 INSERT PRT'S IN BATH NO. 2 INSERT PRT'S IN BATH NO. 3 -1.03+9+E-02 25.491 ++.2928 29.962+ 72.8955 32.8131

A3=-6.7923BE-07

0 1 25. +92

5 1.01993 26.0001

10 1.03982 26.5072

15 1.05969 27.0135

20 1.07951 27.519

25 ' 1.09931 28.0236

30 1.11907 28.5273

35 1.13879 29.0301

+0 1.158+B 29.5321

+5 1.1791+ 30.0332

50 1.19778 30.533+

55 1.21735 31.0329

60 ' 1.23691 31.5313

65 1.256+3 32.029

70 1.27592 32.5258 75 1.29537 33.0217 80 1.31+79 33.5167

85 1.33+19 3+.0109

90 1.35353 S+.50+2

95 1.37285 3+.9967

100 1.39213 35. +883

C/E'JIhTIONS FROM FITTED CURUE -1.O3+9+E-02 -1.56+27E-05 +4.2328 8.18++6E-05

-^.a'iSS 9.33260E-05

Figure 4. Resistance thermometer output.

been attained in the range of 0 to 100 degrees Celsius,

though the process is still under study to determine

its actual limit. There is no reason that the program

and apparatus cannot be expanded to calibrate more

than one test or unknown thermometer at a time. Core

storage, analog switching, and the constant current source are adequate to handle 10 or more thermometers. The practical limit of the number of thermometers that could be tested in one 15-min period is felt to be the size of the gradient-free temperature baths

that would be suitable.

3.4. Rf Applications

Other applications of this computer-automated sys- tem have included rf measurements from 100 MHz

to 12.4 GHz using the Network Analyzer subsystem.

The Network Analyzer has been described in the litera- ture [1, 2] 1 and will not be repeated here. The Net- work Analyzer has been experimentally evaluated by

measuring rf devices that had been previously measured by NBS or our own standards laboratory. Results of such measurements have more than substantiated the accuracy claims of the manufacturer. A

combination of the rf and dc system instrumentation

is being used to determine the ac—dc differences of rf micropotentiometers to better than 1 percent relative

to a standard micropotentiometer.

4. System Software

The BASIC language compiler supplied with this system is similar to the BASIC language used in com1 Figures in brackets indicate the literature references at the end of

this paper.

mercial time-sharing computer services. It provides

conversational-style, on-line program modification in the same way. This is commonly referred to as an

interpretive compiler. The BASIC compiler for this system allows external subroutines to be accessed by

a CALL statement. Such subroutines or instrument

drivers were provided by the manufacturer to supply

data to, control, and read data from the measurement

and control instruments in the system.

This interpretive or on-line mode, while expensive

in terms of computer core (requiring approximately

6K of core), has been found to be convenient and

satisfactory. The measurement application programs

discussed in this paper all were prepared in the BASIC

language by personnel with no previous programming

experience. No BASIC language application programs

were provided by the system manufacturer. The time

required to prepare and debug a measurement pro- gram varies, of course, with its complexity. The pro- gram using the simple model for the unbalanced

Wheatstone bridge was prepared in approximately

5 hr. The standard cell surveillance measurements

program was prepared in less than one day. However,

the refinements and experimentation with a working

program before it can be regarded as finished have

taken several weeks to a month.

5. Conclusion

The measurement of emf, resistance, and temperature are common problems of most standards laboratories. The automated apparatus and methods de- scribed above do not necessarily yield results that represent the ultimate with respect to precision and

total measurement uncertainty. However, the measure- ment results are satisfactory for the majority of cali- brations performed by industrial and Department of Defense standards laboratories. The applications dis- cussed in this paper were chosen to illustrate how

little or no measurement uncertainty need be sacrificed when automating a measurement process to increase

productivity and reduce skill levels. The author wishes to acknowledge the special efforts of Mr. William J. Barron for his direct assistance in both the supporting experimental work and in the preparation of this paper. Many other staff members

of the U.S. Army Metrology and Calibration Center have also contributed to this work. Among those whose efforts should be acknowledged are M. Shelton,

George Marshall, and R. Free.

6. References

[1] Hackborn, R. A., An automatic network analyzer

system, Microwave J. 11, No- 5, May 1968.

[2] Hand, B. P., Developing accuracy specifications for automatic network analyzer systems, Hewlett- Packard J. (Feb. 1970).

33

NCSL 70

COST REDUCTION IN CALIBRATING BOLOMETER MOUNTS

William F. Dentinger and Louis dePian

Weinschel Engineering Co., Inc., Gaithersburg, Md. 20760

Ordinarily a great deal of time is spent in calibrating bolometer mounts. Manual bridges require considerable care for proper balances, bias adjustments, and calibration procedures. New instruments have been designed to automate many of the manual operations. These instruments are self-balancing and maintain precise power levels with minimum adjustments. Due to the fact that in previous setups

a series of measurements had to be made in order to average out errors caused by temperature

changes, the new automated system described in this paper can save about 90 percent in measurement

time. In addition, a less skilled operator can perform the measurement with the same accuracy.

Key words: Automated dc substitution; bolometer calibration; radiofrequency power measurement;

self-balancing Wheatstone bridge.

1. Introduction

This paper presents a new technique in making rf (radiofrequency) power measurements which results in an overall cost reduction in the calibration of bolometer mounts. Employing this technique permits

a calibration accuracy of better than 99 percent. The

range of power involved is from 0.5 to 10 mW over

the frequency range 100 MHz to 12.4 GHz for coaxial

systems and 2.6 to 12.4 GHz for waveguide systems.

Power measurements of this type are currently

needed in (1) primary standards laboratories, (2)

secondary standards laboratories, (3) inspection sta- tions, and (4) production testing facilities. The measurement approach is known as dc substitution. It uses a sensitive element (a bolometer) whose

electrical resistance varies with temperature. This

element is placed in a Wheatstone bridge circuit as shown in figure 1. When the proper amount of dc

power is supplied to the bridge, the bolometer element

heats to the proper temperature to make its resistance equal to that of the other three resistors, and the indiFigure 1. Dc substitution method, assuming unity

calibration factor for bolometer.

cator will show a null. If we now supply an additional

rf power to the bolometer element, it will heat up to a

higher temperature, thus changing its resistance and

unbalancing the bridge. However, if we also decrease

the dc power to the bridge by an amount equal to the applied rf power, the temperature of the bolometer

element will remain the same. This keeps the bridge

in balance, and the indicator nulled. The amount of applied rf power is assumed equal to the difference

of the dc power before and after applying the rf power

rf = dci — dc2 = dc substituted.

This method of measurement by using dc substitu- tion has proven to be one of the most accurate and is the basis of most of the existing measurement setups.

However, there is one important consideration: not all of the rf power incident on the bolometer mount

reaches the bolometer element. Therefore the actually

applied rf power is not exactly equal to the dc substi- tuted power. A small correction factor, called the

"calibration factor," is needed: _ . T

. dc substituted power _ P&Q Calibration factor K — —=—:— — 75— . rt incident power rlf

In order to determine the calibration factor of an unknown test bolometer mount (i.e., to calibrate a test bolometer mount) , we must compare it with a standard

bolometer mount. The procedure is as follows (see

fig. 2). Since we can accurately measure the dc

substituted power (P",ic ) of a test bolometer mount,

its calibration factor (KT ) can be determined if we

can also accurately determine the rf power incident

upon the bolometer mount. This rf power (P' r r) is calculated by measuring the dc substituted power

(P'dc) in a standard bolometer mount and knowing

35
its calibration factor (Ks ) with an accuracy traceable

to the National Bureau of Standards: P' rt = P'dc /Ks . The essence of the calibration is therefore to calculate accurately and conveniently the rf power incident

upon the test bolometer mount.

2. Previous Method

The method previously used to calculate this rf power required a laborious procedure to determine

first the dc substituted power (P',i C ) to the standard

bolometer mount. The essential steps employed in

this procedure were as follows:

1. Supply d-c voltage to the standard and manually

adjust the voltage until the bridge is nulled. This is accomplished by adjusting a variable potentiometer.

2. Compare the dc voltage applied to the bridge

with an accurate dc voltage calibrated with a precision

potentiometer.

3. Read the dc voltage volue E0

.

4. Apply rf power to the standard and repeat step 1. 5. With an external potentiometer, measure the change in dc voltage to the bridge by comparison with

the voltage EQ

.

6. Read the change in d-c voltage /\E.

DC

SUBSTITUTION

P' (MEASURED)

DC

STANDARD

BOLO. MOUNT

S (KHOWW) (UNKNOWN)

Figure 2. Determination of calibration factor KT : Calculate

P'i f — P'dc/Ks, Kt = P",i,./P ,-f •

7. Calculate the dc power using the formula

P'dc = 0.025/?o (2E0 - AE) AE raW

(where R0 is the bridge resistance)

8. Calculate the unknown rf power by using the formula

P'rt

*P'dc calibration factor of standard bolometer mount

The total time required for an experienced operator

to make the measurement by following these steps is about 10 min. However, during the relatively long

time required to perform these steps, the room temperature is unlikely to remain constant, and this will introduce an error in the measurement. Additional

errors will also be involved if during the measurement

time the rf power source varies in magnitude and

frequency. In order to reduce the effect of these

errors, the measurements must be repeated from three

to five times. Thus the total time required to make

an accurate (better than 99%) measurement is usually

about 30 min at best.

3. New Method1 A new instrument developed by Weinschel Engineering, called the precision rf power level controller,

greatly simplifies and automates the dc substituted power measurement we previously described. The

salient features of its design are as follows:

(a) provision for about 30 mW of dc power to the standard bolometer mount;

(b) decrease of dc substituted power by accurate

fixed amounts, 0.5, 1.0, 5.0, or 10.0 mW.

(c) through an automatic leveling loop, the rf power supplied to the bolometer mount is forced to remain equal to the preselected dc substituted power,

i.e., 0.5, 1.0, 5.0, or 10.0 mW.

1 Weinschel Engineering Application Note 12, Calibration of bolometer mounts with leveled power, Form 867-3/70.

Figure 3. Measurement setup using new method.

36
Because of these design features, the measurement is now simplified, and involves only the following steps:

1. Supply d-c voltage and manually adjust it until the bridge is nulled. This step of the procedure has

been greatly simplified.

2. Select desired decrease of power level (0.5, 1.0,

5.0, or 10.0 mW, to within ± 0.1% + 1 MW) by

setting a switch.

3. Apply rf power, which will automatically be

equal to the selected power of step 2.

4. Calculate the unknown rf power using the formula

pr _ selected dc substituted power

rf calibration factor of standard bolometer mount

The total time required for this new method is about

3 min, providing a saving of 90 percent of the time

previously needed. To obtain other desired power

levels for dc substitution, external resistors may be

added to the precision rf power level controller. Figure 3 shows a complete measurement setup using

the new method described in this paper.

4. Comparison of Methods

The previous method required about 30 min by an

experienced operator. The new method requires only

3 min for the same measurement for the same accuracy, and its simplicity allows a less experienced operator

to make the measurement. The savings in time and

costs for the measurement are obvious.

37

NCSL 70

SESSION 2: LABORATORY MANAGEMENT AND OPERATIONS REPORTS

Chairman: F. J. Dyce

Martin Marietta Company, Orlando, Fla. 32805

MEASUREMENT COMPARISON PROGRAMS

Herbert S. Ingraham, Jr. RCA Corporation, Defense Electronic Products, Moorestown, N.J. 08057

The measurement comparison program of 1965 circulated unknowns in pairs, allowing use of Youden diagrams to distinguish systematic from random error. In 1969, two pairs were sent out

several months apart, allowing check of the correlation effect; a round robin was started on acceleration; a 24-inch end standard was included in the physical package.

Key words: Accelerometer round robin; computer program for Youden diagrams; correlation effect; end standard of length.

Innovation in metrology as a key to progress is the theme of our Conference. Certainly the establishment

of a Measurement Comparison Program of national scope meets the criterion for innovation. The basic

program is now a historical innovation. The current

question, to paraphrase another, is, "What have you

innovated for me lately?"

To determine the changes which constitute innovation in today's Measurement Comparison Program, we

must compare the program as it stood at the 1968

Standards Laboratory Conference and what useful changes have taken place since that time. The concept of circulating measurement unknowns

in pairs was, in itself, a significant innovation (in

1965) in the Comparison Program. This allowed the use of the Youden diagram concept to separate, to a high degree, systematic error from random error in the calibration and measurement process. This was a

first in the measurement comparison process and was

an extremely valuable step.

It was felt, however, that measurement of a pair of unknowns at nearly the same point in time left something to be desired. Laboratories traditionally repeat

the same type of measurement several times a year, and it was felt that, if both items of a pair were measured at the same time, random errors would tend to be masked by an undesirable "correlation effect." To obviate this correlation effect, we are currently

splitting our "pairs" so that they are measured at different times, separated by at least a month. When

the results of this program are analyzed, we would

expect to have a truer picture of a laboratory's measurement capability than was possible before, particularly if the correlation effect was significant.

One such split has already been completed in the mass comparison and will be reported by Lloyd B. Macurdy. These data, and data from other programs

where the split pair technique can be applied, should

provide a base for evaluation of the significance of correlation effect. Other program changes since 1968 include the ex- pansion of the Program to other areas of metrology

than were previously included. These include the addition of a long (24-in) end standard in the physical

package, the separate circulation of the mass packages,

and, more recently, the start of the first round robin

comparison in acceleration, currently circulating to 20

participants. (The acceleration comparison is the first in which the pairs of unknowns were deliberately

separated to evaluate the correlation effect.) In the last major round robin program, all data

reduction and analysis were accomplished by the route

supervisors. This was a lengthy and burdensome

process, accomplished only through the dedication of the route supervisors. Now the task of data reduction

and analysis has been assumed by the National Bureau

of Standards, and they have added a certain amount

of innovation.

Primary to this effort has been the development of a plotting program for the CalComp plotter to generate the Youden diagram directly from the data and

to obviate the manual plotting done before. In addition to this, a handbook has been developed to assist in the analysis of the Youden plot, for use by partici39
pants in analysis of their results. This plotting

program can be made available to those who are conducting measurement comparison programs, either as an intercompany program or as an area program,

and who wish to use the Youden presentation for data

analysis and have a CalComp plotter available. The value of the Measurement Comparison Program

can be improved through innovation in the future.

Other comparison techniques, currently under consideration, will result in pilot comparisons of limited scope to aid in evaluation of the techniques. The most

useful of these will be incorporated into the Program

to provide the laboratories of the metrology com- munity with the most effective method of assessing the contribution of their own innovations to their measurement capability.

10
NCSL 70

REPORT ON THE 1968-69 MASS MEASUREMENT

AGREEMENT COMPARISON ROUND ROBIN

Lloyd B. Macurdy

Staff Mctrologist, Mettler Instrument Corporation, Princeton, N.J. 08540

Seventeen laboratories took part in the round robin measurements of the NCSL Mass Comparison

packages during 1968 and 1969. In order to speed up the measurements the laboratories were

divided into two groups. The weights were calibrated by the NBS Mass Laboratory at the beginning and again at the end, and Mettler Instrument Corporation measured all sets near the middle of the

series. Two sets of weights were sent to each laboratory at times separated by two months or more

in order to provide data on errors made on the same day (within-group variance) and on errors between measurements at longer intervals (between-group variance). Each set of weights consisted

of four pairs of one-piece stainless steel weights of 100 g, 25 g, 10 g, and 1 g. Data include weighing

errors of the various laboratories and Youden plots of the measurements. Some errors proved to be

larger than had been expected. The results illustrate the need for verifying the accuracy of measure- ment on some rational basis.

Key words: Between-group variance; mass comparisons of 1969; systematic and random weighing

errors; within-group variance.

This round robin test is of especial interest because

it includes 15 laboratories whicb do not participate in the NBS Measurement Analysis Program for Mass

Measurement. Two laboratories which do participate

in the NBS MAP program are also included. They are

the IBM Measurement Standards Laboratory in San

Jose, California, and Mettler Instrument Corporation

in Princeton, New Jersey. There is very little in- formation available about the capabilities for mass

measurement in laboratories outside the NBS MAP

program. These test comparisons provide a sample

of the errors which occur in various standard

laboratories. Eighteen laboratories expressed a desire to take part

in the mass comparison tests. One laboratory with- drew before the start, leaving 17. Measurements were made by the National Bureau of Standards Mass

Laboratory at the beginning and again at the end, and

by Mettler Instrument Corporation near the middle of each schedule, and by 16 other laboratories. The test packages consist of pairs of weights of 100 g, 25 g, 10 g, and 1 g. Each participant received two packages at times separated by several weeks or months. In order to reduce the time required, the participants were divided into two groups. One group

included the Midwest and far-West areas. The other included the East and South. Four packages were

prepared and circulated, two for each group. All four packages were mailed from the National Bureau

of Standards on December 2, 1968. Package A-l

(weights 1, 2) was completed in September 1969,

Package A-2 (weights 3, 4) in October 1969, and

Packages B-l (weights 5, 6) and B-2 (weights 7, 8) were completed in April 1969. The weights are of one-piece stainless steel, which provides freedom from

certain sources of variability which would be associ- ated with screw-knob type of construction, and with

plated or lacquered surface finish. We arranged to take test data on pairs of weights because of the ad- vantages of analyzing data in pairs. Youden plots can show deviations within pairs taken on the same day or within pairs taken on different days. The NBS values

also make it possible to plot the errors with reference

to a "zero error." Two questions are of especial interest: (1) What

are the errors of measurement? and (2) How well is the uncertainty estimated? Each participant was in- structed to weight the test weights as they would weigh

standard weights. The method to be used was left entirely to the participant. Each participant was also asked to estimate the systematic and random components of the uncertainty.

The results of the test are shown in figures 1 through

8 for each of the four loads in the two packages.

Weighing errors are plotted with reference to the NBS

values and are shown as circles in groups of four for nominally identical weights. Measurements by the National Bureau of Standards and by Mettler are

identified, while data for the other participants is designated by code number. The uncertainty as esti- mated by the participant is shown by a vertical line which represents the plus or minus sum of the esti41
mates for the systematic and random components.

The precision actually obtained is shown by the

scatter within the groups of four. In 64 percent of

the groups, the precision as indicated by the scatter is comparable to the scatter of the NBS measurements. By this I mean that the scatter is not greater than

about two or three times the range of the NBS values. Youden plots of the data in figures 9 to 13 show a pronounced tendency for the points to be scattered along a diagonal line, indicating the presence of sys- tematic error. In these figures the NBS values are shown as stars and the group mean is shown as an

asterisk. Comparison of figure 9, showing data taken

for a pair on the same day, with figure 13, showing

data taken for a pair on different days, indicates that the correlation effect is slight for 100 g

weights.

To summarize these results, systematic error was

larger than the estimates of uncertainty in eight of the groups of four, involving five different laboratories. Imprecision was larger than anticipated in nine

of the groups of four, involving six different laboratories. It appears that in 36 percent of the groups the precision was such that it is subject to improvement

if that is desired. In 14 groups of four, or 22 percent,

the estimate of uncertainty was too small. In 12

groups of four, or 19 percent, the estimate of uncertainty seems unduly large. The measurements are much better than their estimates. Thus there is diffi- culty with either overestimation or underestimation of the uncertainty 43 percent of the time. One way in which overestimation of the uncertainty can occur is by reliance on manufacturer's tolerances

as the basis for accuracy statements. Any tolerance statement by a manufacturer applies to the limits on

the error in a large number of items. Most of the individual items will have errors smaller than the

limits for the entire group. In the case of a tolerance on the error of combinations of built-in weights, only

certain combinations of dial settings can lead to the maximum error. The remaining combinations will have smaller errors. Tolerances have their proper

place in purchase specifications, but more detailed in- formation is needed for estimates of uncertainty.

Some simple, rational method is needed in order to provide a basis for estimating the uncertainty. The

use of corrections for the individual weights of a re- liable set of standard weights will provide more precise information than can be obtained by reference to the tolerances.

Figure 1. Errors in weighing 100-g

weights, group A. ERROR

mg —+0.5

Figure 2. Errors in weighing

100-g weights, group B.

ERROR

mg

f Q 5

78 59

-0.5

\2
ERROR

mg

0.1 4%— 00

Figure 4. Errors in weighing

25-g weights, group B.

_z0.1

ERROR

mg

+0.3

O NBS

0-0

Figure 3. Errors in weighing

25-g weights, group A.

-0.3

52 53

NBS 66

74

ERROR

mg

+0.1

0.0

-0.2

Figure 5. Errors in weighing 10-g weights, group A.

43
ERROR

mg

0.1

0.0

_=0.1

Figure 6. Errors in weighing 10-g weights,

group B.

_z0.20

Figure 8. Errors in weighing

1-g weights, group B.

-0.05

Ficure 7. Errors in weighing 1-g weights, group A.

44
p 0.1

x:

0.4-

' -0.0

<_) l±j g -0-4

CM 1Zd

cd

0.8

o -

.2 - o

-

.6 -

cD o. 08 -r 5Z

o. 04-

CD 0

00 - t—

UJ

oc o

04 - r>

CD

CJ

CD o

(X

CtC CD

- 0 12 -

CM

0 . 1 6

- -o.

CD o. 02 -i z:

o. 00-

CD

1—*~ o. 02-

CD

UJen

cn- o. 04-

CD

CD

CD o. 06-

cx

enCDo"0 08-

0 10-

o

•

*> o

o

o

6 -1.2 -0.8 -0.4 -0.0 0.4 0 100 GRAM , CORRECTION IN MG

Figure 9. Youden plot of 100-g Values, package A-l.

0 -06

t, 0.04

i L_

©

©

© ©

©

o

o

•0 .06

-0. 10-0. 08-0. 06-0. 04-0. 02 -0. 00 0. 02

1 GRAM, CORRECTION IN MG

Figure 12. Youden plot of 1-g values, package A-l.

Figure 10. Youden plot of 25-g values, package B-2.

CD -0.

32 -0 .28 -0 .24 -0 .20 -0 . 16 -0 . 12 -0 .08

25 GRAM, CORRECTION IN NG

-1.6 -1.2 -0.8 -0.4 -0.0 0.4 0.8

100 GRAM j CORRECTION IN MG

Figure 13. Youden plot of 100-g data taken on differ- ent days, packages A—l and A—2.

Figure 11. Youden plot of 10-

g

-0 .07 -0 .06 -0 .05 -0.04 -0 .03 -0 .02 -0 .Q\values> packagt B-l.

10 GRAM 5 CORRECTION IN MG

45

NCSL 70

NCSL SPECIFICATION PRESENTATION

Frank J. Dyce

Martin Marietta Company, Orlando, Fla. 32805

This paper describes in summary the requirements of the National Conference of Standards

Laboratories "Calibration System Specification." It indicates how the specification was written by

members of the NCSL Specifications Committee in answer to a DOD request. The specification contains the minimum essential requirements of a calibration system. The paper discusses the

steps required to approve the specification as a recommended practice and to obtain Government

approval and adoption.

Key words: Calibration records; calibration system specification; intervals; procedures; recall;

reliability ; traceability.

1. Introduction

In May of 1969, our committee published a compilation of ten government specifications which affect stand- ards and calibration laboratories. A paper describing

the compilation was presented at the 1968 Standards

Laboratories' Conference at Boulder, Colorado; at that time several representatives from DOD suggested

that NCSL should write its own specification. This

request was followed up by a questionnaire which in- dicated unanimous support for the idea. The ques- tionnaire results also revealed 19 specifications which

were omitted from the original compilation, all of which had calibration system controls in them. The

committee felt that our profession was not so complicated that we required 29 specifications to control

ourselves. In addition, the specifications were, in some cases, conflicting and in many cases, ambiguous;

therefore we have prepared a NCSL calibration system

specification delineating the minimum essential re- quirements of a calibration system for control of measuring instruments and standards. This paper will describe in some detail that specification.

2. The Index

The first step in preparation of the specification was

the decision as to what it should contain. The major

categories discussed in the compilation were selected and with some combination and rearrangement, yielded

the following index:

INDEX

1. Purpose

2. Government Audit

3. Subcontractor Controls

4. System Description

5. Records

6. Recall System

7. Calibration Intervals

8. Labeling

9. Calibration Procedures

10. Reliability

11. Traceability to National Standards

12. Environmental Controls

The task of preparing these sections was assigned to individual committee members. The results were compiled and circulated for comments among the entire committee. The following is a summary of the re- quirements specified. Reference is made to the authors

of the original draft of each section.

1. Purpose Author: Frank J. Dyce

This specification defines the minimum essential requirements of a calibration system for control

of measuring instruments and standards. It applies to those laboratories which specify it as

their governing document.

2. Government Audit Author: Frank J. Dyce

Calibration system audits should be restricted to those items specifically covered by the applicable

contract requirements.

3. Subcontractor Controls Author: Frank J. Dyce

The prime contractor shall be responsible for his subcontractors having a calibration system that meets the requirements of this specification as it applies to the products furnished.

(Since this specification covers the minimum es- sential requirements, it is the contractor's responsibility to insure that the subcontractor meets at

least these requirements.)

47
4. System Description Author: John Skinner

A written description of the calibration system

shall be maintained. It may be a single document

or several documents which define how the re- quirements of this specification are met. It shall include at least the following items:

Organization: A basic statement which defines the

responsibility and authority of the calibration laboratory.

Measurement Equipment Controls: Selection of

initial intervals, recall procedures, records, labels. Technical Control System: Outside sources, en- vironment, interval adjustment.

Calibration Capability List Terminology : Definitions of special terms having

significant usage in the calibration system.

(This section states specifically that the system

description will contain only those items necessary

for a procedural review of a calibration program.

It permits the use of several documents which

may be standard company procedures. I have

personally seen a 200-page book put together to meet previous system description requirements.)

5. Records Author : James Gilbert

Records shall be maintained for each item of measuring equipment upon which a calibration

has been performed.

Records shall be maintained for a minimum of one year plus one calibration interval. Equipment Identification Records: A nonrepetitive identification shall be assigned to each item

of measuring equipment in the calibration system.

Equipment Location Records: Records shall be

maintained to provide information necessary to locate equipment due for calibration or repair.

Calibration History Records: Records shall be

maintained which indicate the present calibration

status and previous calibration history of measuring equipment.

Calibration Control Records: Records shall be

maintained on all measurement equipment requiring calibration to provide the information neces- sary for the control and audit of the calibration system.

I This section limits records to calibrated equipment and limits the length of time required for keeping the records. It provides flexibility in identifying and locating equipment and lists the minimum specific requirements of the record sys- tem.)

6. Recall System Author : William McCallum

A documented system shall be established with the purpose that no standards or measuring equipment with an expired calibration interval shall be

used for a product measurement.

User Notification: The equipment user should be

notified on or before the calibration due date. Equipment Availability: Items must be made avail- able for calibration on or before their calibration due date or be removed from service, conspicuously tagged, or restricted from use.

(This section clearly states the purpose and re- quirements of a mandatory recall system. It leaves no room for misinterpretation.)

7. Calibration Intervals Author: Don Greb

Measuring equipment and standards shall be cali- brated at periodic intervals designed to assure that

specified accuracy shall be maintained.

Interval Application: Intervals may be applied to single items or groups of items which are rea- sonably homogeneous with respect to family and

reliability. Interval Assignment: Calibration intervals shall be assigned for the maximum probability that the item(s) will be within accuracy specification when

returned for calibration. This probability will be

consistent with calibration system organizational

requirements and resources. Reliability, failure, usage and cost should be factors considered in as- signing calibration intervals. Interval Adjustment: Calibration intervals shall be adjusted as necessary on the basis of calibra- tion data or other information which may support

a change.

Interval Shortening: Calibration intervals shall be

shortened on time or usage dependent instruments

to assure continued accuracy.

Interval Lengthening: Intervals may be lengthened

when indications are that it will not reduce re- liability below desired minimums.

Interval Extensions: Intervals may be extended by

laboratory supervision due to critical test or pro- duction schedules or other requirements provided.

Historical data support the extension and/or

product acceptance is withheld pending the re- sults of the eventual calibration. (This section discusses assignment, adjustment

and extension of intervals in a complete unambiguous format. It permits both fixed and variable

intervals, and extensions within limitations.)

8. Labeling Author : Frank J. Dyce

All measurement equipment and standards shall be labeled or coded to indicate their calibration

status. Labels shall include date calibrated, re- calibration due date, identification of calibration personnel, equipment identification, calibration agency. Label Impractical: When labels are impractical,

codes should be used or recall records should be

monitored.

Nonstandard Labels: "Calibration Deviation,"

"Not Required," "Do Not Use" and outside agen- cy labels should be used as applicable to cover

deviations from normal calibration technique.

9. Calibration Procedures Author: R. R. Willett A current document shall exist containing suffi- cient information for the calibration of measuring

instruments and standards to the required accu- racy for product measurements.

48
Contents: It may be a local document, another

company's procedure, a manufacturer's procedure

or a composite of the three. It should be refer- enced under one cover with local approval.

Format: The local procedures shall include the following information—application, specification,

preliminary operations, step-by-step procedure;

calibration table, worksheet and appendix.

Usage: Locally approved calibration procedures

shall be followed in the calibration of measuring

equipment and standards and shall be available

in the calibration area. (This section basically combines the requirements

from several specifications. It has sufficient flexi- bility to permit use of existing legitimate procedures, yet enough firmness to require complete

procedures.)

Reliability Author: M. H. Brenner The calibration program shall be conducted to maintain measurement equipment above a specified minimum acceptable reliability level such that instruments remain within their certified specification limits over their calibration interval. Minimum Reliability Level: It shall be the objective of the calibration program to achieve and

maintain a minimum documented reliability level consistent with organizational requirements and

resources. Family Reliability: An analysis on a manufacturer model number basis shall yield overall re- liability numbers consistent with this goal.

Corrective Action: The individual instrument or family intervals shall be adjusted or the instru- ment specifications deviated (where usage require- ments permit) as required to obtain the relia- bility goal. Certified Specification Limits: Upon receipt for

calibration, the instrument must be tested to de- termine if it is within tolerance prior to any ad- justment or repair.

Reliability Records: Shall be monitored periodically for number of instruments, number of cali- brations and reliability by instrument family.

Reliability Below Minimum: If the family relia- bility drops below the minimum level, one or more

of the following should occur: interval adjustment, specification deviation, failure analyses, calibration procedure change, or application in- vestigation.

Reliability Above Minimum: If the reliability re- mains consistently above the minimum level, the

calibration interval should be increased to the point where the minimum level can be maintained

with optimum equipment utilization and minimum

service costs. (This is a requirement which is only briefly covered in most specifications; however, it is an area we felt needed strong coverage.

Reliability emphasis separates the good from the bad programs. The actual minimum reliability level will vary between companies but it should

be documented.)

11. Traceability to National Standards

Author: Carl Boyer

The calibration program shall provide the means

to trace the accuracy of all calibrated measure- ment equipment through all echelons of standards

to a value in terms of U. S. National Standards.

Documented Traceability: Where applicable, cali- bration of measurement equipment shall have doc- umented traceability except for Ratio Standards,

Natural Physical Constants, and Derived Standards. Traceability Documentation : Documentation which

substantiates and identifies the means of achieving

traceability shall include as necessary: calibration agency, standards used, calibration technique,

data, uncertainty, environmental conditions, date

of calibration, and identification of the calibra- tion personnel.

12. Environmental Conditions Author: Frank J. Dyce

The calibration environment should be controlled

to the extent required by the most environmentally

sensitive measurement made in the area. Compensating Corrections : Where control is in- adequate, compensating corrections should be

added to the calibration results. Environmental Records: Records of environmental

conditions are only necessary when the require- ments are tighter than the laboratory norm or the measurement is based solely on the environment

it was calibrated within. Environmental Considerations : Particular atten- tion should be given to the following environmental conditions in developing calibration techniques

and evaluating measurement data: Temperature, voltage, humidity, dust, vibration,

air currents, RFI, E.M.I., line voltage, grounding,

and lighting.

(Environment has to be only as good as required

by the measurements. Recording is performed

only when necessary to interpret results and cor- rections are applied only when required.)

3. Conclusion

This specification delineates the minimum essential requirements of an equipment calibration system in an unambiguous format. We will have a committee meeting during this conference to make final changes to it. It will then be submitted to the NCSL Board of Directors for preliminary approval before presentation

to DOD for their comments. The minimum we hope

to achieve with this document is NCSL-recommended

practice which will be adopted by laboratories not

presently working to a Government specification. Several of our committee members indicated interest along

these lines. The maximum we hope to achieve with this document is to obtain DOD approval and have them specify

it in lieu of the other 29 specifications.

We feel that with this document we have established

a platform from which DOD can view what industry

feels are the requirements of a calibration system. We

49
have also given the new laboratories a general outline

of what they must do to develop a calibration control system. The various sections of this document can be expanded into a recommended practice just as has

been done with the recommended practices on cali- bration intervals and calibration procedures. We will then have a detailed document, corresponding to this general one, available to our membership.

The National Conference of Standards Laboratories

Specifications Committee

Frank J. Dyce, Chairman

Martin Marietta Corporation

Orlando, Florida

William McCallum Carl Boyer

General Electric Co. Honeywell, Inc. Daytona Beach, Florida Annapolis, Maryland

Robert B. Willett John M. Skinner

Collins Radio Co. General Dynamics

Cedar Rapids, Iowa Pomona, California

James Gilbert Donald J. Greb

Vitro Laboratories Lockheed Missiles & Space Co.

Silver Springs, Maryland Sunnyvale, California

Marshall H. Brenner F. L. Swann

Boeing Co. Aerospace Group Systems Group of TRW Inc.

Seattle, Washington Redondo, Beach, California

R. J. AUen J. A. Walz

Royco Instruments, Inc. General Electric Menlo Park, California Bay St. Louis, Mississippi

L. B. Dunkleberger D. J. Stemper

Kaiser Aerospace & Electronics Holt Instrument Laboratories Palo Alto, California Oconto, Wisconsin

Bernard W. Busch 3M Company

St. Paul, Minnesota

50
NCSL 70

COST VISIBILITY EXCHANGE PROGRAM—

A NEW APPROACH TO COOPERATIVE SAVINGS

R. J. Barra

Westinghouse Defense & Space Center, Baltimore, Md. 21203

A preliminary survey by the Calibration Systems Management Committee indicates 600,000 manhours per year spent on 9 types of instruments, including oscilloscopes. Member organizations

are asked to contribute cost data with the objective of saving one million dollars in calibration and

maintenance costs by 1972.

Key words: Calibration and maintenance manhours; digital counters; digital voltmeters; exchange program on costs; oscilloscopes.

a new cost visibility exchange program:

OBJECTIVE .

The 1969 NCSL Directory listings show about 5,000

persons working primarily in standards laboratories. Assuming that a similar number of people are working in lower echelon laboratories, the total population

becomes 10,000. At 2,000 manhours per man year,

the total expenditure is twenty million manhours per

year. As you can see, hundreds of millions of dollars are involved.

Why so much? What are we spending these manhours on? What instruments? If we knew what the

dollars were being spent on, we might be able to do

something about it. The Calibration Systems Management Committee of NCSL had to know the answers

to these questions. To be most effective we have to concentrate on the high-cost areas. What we wanted was overall cost visibility. But the committee members alone should not be the only ones to benefit from

this greater cost visibility. Each laboratory manager

would like to know how his costs compare with costs

in other laboratories. With the capability to make

cost comparisons, we could detect otherwise hidden

areas where cost reductions might be achieved. The

exchange of cost data would not only provide broader

visibility to all laboratory managers, but it would also provide first, a communication link between labs with common interests, and second, a foundation for cooperative action on improved methods and procedures

leading to substantial cost reductions for all participating laboratories. Recognizing ( 1 ) that hundreds of millions of dollars were involved, (2) that present cost visibility is quite limited, and (3) that cooperative action was al- most non-existent, the committee established the fol- lowing objectives:

• To develop the management control techniques re- quired for optimum performance of calibration sys- tems.

melt the ice

Figure 1. A new cost visibility exchange program:

objective . . . melt the ice.

• To provide cost visibility to managers.

• To provide means to achieve cost reductions, with- out degradation of quality, through improved methods and procedures.

51
CALIBRATION SYSTEMS MANAGEMENT COMMITTEE

R. J. Barra, Chairman

PLANNING AND CONTROL

E. Arsenault W. McDill

:. Boyer C. Stone

OSCILLOSCOPES & PLUG-INS

D. J. Greb

J . Mauldin

OSCILLATORS

R. Bailey

GENERAL LAB METERS

D. J. Heron

ELECTRONIC METERS

J. W. Rodgers

R. Willett

DIFF. VOLTMETERS

D. Mitchell

COUNTERS

N. D. Smith

PRESSURE GAGES

J. Riley

Figure 2. Committee organization chart.

DIGITAL VOLTMETERS

J. L. Rickert

Recognizing that the committee would be more ef- fective if its members had clear and measurable goals, the following specific goals were established:

1. To collect 2 million manhours of specific cost

data.

2. To disseminate these data to participating laboratories in the form of cost visibility reports.

3. To provide the means for NCSL members to save

a million dollars before the 1973 NCSL Conference.

The committee's plan of action is as follows: Phase 1 —Determine where the calibration dollars are

going. Conduct a survey to find those items

which have the greatest potential cost reduction to the greatest number of NCSL members. Phase 2—Collect, analyze and report the cost data on

these groups of instruments.

Phase 3—Restructure the committee into sub-committees for in-depth probings.

Phase 4—Collect and develop cost reduction ideas, and

provide a means to exchange them.

The committee's progress toward the attainment of our goals can best be measured by reviewing actions

to date. Less than a year ago Phase 1 began. On

July 30, 1969, a questionnaire on calibration interval versus cost was sent to all delegates and sponsors. About 25 percent of the membership responded. The

data collected exceeded one million calibration and

maintenance hours. Sixty percent of these hours were

associated with just nine groups of instruments. The

committee achieved one of its first objectives; that is, we found out where most of our calibration dollars were going. The committee's first cost visibility re- port, covering these nine groups of instruments, is given in the appendix. How does your lab compare

with the data presented? Are your intervals longer

or shorter? Are your annual calibration maintenance

hours lower or higher?

608,000 HOURS

40

16,000 GENERAL q>

LAB METERS 9t 600^95

PRESSURE /

0 GAUGES DVM'S 1,500

Figure 3. Results of calibration cost survey,

Fall 1969: Percent total hours.

As listed in ments are:

the report, the nine groups of instruOscilloscopes

Oscillators/Generators

Electronic Meters

Oscilloscope Plug-ins

Electronic Counters

General Lab Meters

Differential Voltmeters

Pressure Gages

Digital Voltmeters

52
How large a sample did these data represent? Not

too large. For example, a few comparative figures are given below:

A. Oscilloscopes—28,000 out of about 350,000

B. Counters (Digital)—6,700 out of 175,000

C. Digital Voltmeters—1,500 out of 75,000

Using the weighted average figures from the survey,

the total calibration and maintenance effort for these instruments is over 3,300,000 manhours per year. The

breakdown is as follows: A. Oscilloscopes—350,000 at 5.8 manhours/year/

scope equals 2 million manhours.

B. Counters—175,000 at 3.9 manhours/year/

counter equals 700,000 manhours.

C. Digital Voltmeters—75,000 at 7.8 manhours/

year/ DVM equals 600,000

manhours.

As you can readily see, the cost reduction potential is significant. Assuming that the above figures are rep- resentative of the total population, the one million

manhours of the survey is only about 5 percent of the

total effort, which figures to be about 20 million manhours. This first cost visibility report was just part of Phase 2. This month you will have a second chance

to be a contributor to this program. The committee

needs additional data to supplement the information

received from the 1969 survey. Cost visibility ex- change cards are being sent to all delegates. We are looking for specific cost information on these groups

of instruments. Send in as many cards as you wish.

If you are interested in getting data on all these groups

of instruments, then a card for each group must be

contributed. Each card is your ticket to greater cost

visibility and an opportunity to achieve substantial

cost reductions for your laboratory.

QUANTITY HRS/YR AVG HRS/YR/ITEM

Oscilloscopes 28,373 169.912 5.8

Oscillators, Signal Generators & Function Generators 30.813 146.082 5.4

Electronic Meters (V.T.V.M.'s &

Electronic Multimeters) 44,663 102.606 2.1

Scope Plug- Ins 17,073 47.870 2.2

Counters 6.741 36.642 3,9

General Lab Meters (Panel Meters, Portables & V.O.M'sl 15,989 35.089 1.9

Differential Voltmeters 7.209 29.405 3.3

Pressure Gages 9.580 26.265 1.9

Digital Voltmeters 1,540 14.513 7.8

Totals 161,981 608.384

Table 1. Instruments with highest calibration I maintenance hours

CAL/MAINT

HRS/INST/YR CALIBRATION INTERVALS (MONTHS) TOTALS

2 3 4 6 9 12 2.2-5

76

11.504 580

3641 47

125 67 15.259

781

6 8

159 759 6180

2428

25

523 65

64 6269

3934 9-12

653 312 4243 380 34

76 76

1464 MORE THAN 12 33 249

50

115 134 1 83 499 TOTAL INST 812 1429 20.942 4711 271 208 28.373 TOTAL HRS 8253 15,095 117.803 25.688 1695 1378 169.912 AVG HRS/YR/INST 10.2 10.6 5.6 5.5 6.3 6.6 5.8' GOVERNMENT

INDUSTRY •WEIGHTED AVERAGE

Table 2. Oscilloscopes

CAL/MAINT CALIBRATION INTERVALS (MONTHS)

HRS/INST/YR TOTALS

2 3 4 5 6 7 9 12

2 4

127 50

12.614 2302 1014 730

154 147 16,357

331 4-8

490 2416

90 46286 3972

918

193 11,209 1504 9-12

10 184 115 309 MORE THAN 12 488 52 505

57 1 505

598 TOTAL ITEMS 488 679 3302 16,798 7308 1014 884 340 30,813

TOTALS HRS 10.367 5227 26.428 64.066 31,967 3486 2833 1708 146,082

AVG HRS/INST/YR 21.2 7.7 8.0 3.8 4.4 3.4 3.2 5.0 5.4* GOVERNMENT

INDUSTRY

•WEIGHTED AVERAGE

Table 3. Oscillators, signal and function generators

53
CAL/MAINT

HRS/INST/YR

CALIBRATION INTERVALS (MONTHS)

TOTALS

1 2 3 4 5 6 9 12

1.5-2.5 — — — 13,398 5534 21,674 — 40,606 46 66 272 MA

3 6 — — — 55 47 66 — 349 547

150 613 33 72 806 127 90 1891

7 - 10 — — — 15 — — — — 15 90 6 27 416 33 72 9 653

11 100 12 12 227 76 46 6 — — — — 355 TOTAL ITEMS 317 232 732 603 13,650 6478 22.182 469 44,663

TOTAL HRS 4221 1786 4043 3759 30,922 15,850 40.368 1657 102,606

AVG HR/INST/YR 13.0 7.7 5.5 6.2 2.3 2.4 1.8 3.5 2.1* GOVERNMENT

INDUSTRY

WEIGHTED AVERAGE

Table 4. Electronic meters {pointer type, active circuit)

CAL/MAINT CALIBRATION INTERVALS (MONTHS) TOTALS HRS/INST/YR 2 3 4 5 6 12

1.3-2.5 11,364

114

11.364 114

2 6 5 1254 65 90

2420

604 210 2420 2223

6 10 339 184 36 1 559

11 40 229 2 18

5

113

25

131 262 TOTAL ITEMS 568 1440 124 11.454 3277 210 17.072 TOTAL HRS 7870 5098 974 20.773 12.578 604 47.870 AVG HRS/INST/YR 13.4 3.5 7.6 1.8 38 2.8 2.2* GOVERNMENT

INDUSTRY •WEIGHTED AVERAGE

Table 5. Scope plug-ins

CAL/MAINT

HRS/INST/YR CALIBRATION INTERVALS (MONTHS! TOTALS 30R LESS 4 5 6 9 12

0 8 0.9 1048 1048

1 2 2576 6456 1452 10.484 324 85 409

2.1 4 1794 562 2356 300 406 570 1 1277

5 - 35 180 180

232 2 1 235 TOTAL INST 532 2576 1794 8236 2202 649 15.989 TOTAL HRS 3201 5558 5918 12.194 6618 1600 35.089 AVG HRS/INST/YR 60 2.2 3.3 1.5 3.0 2.5 1.9* GOVERNMENT

INDUSTRY •WEIGHTED AVERAGE

Table 7. General lab meters (pointer type,

passive circuit)

CAL/MAINT CALIBRATION INTERVALS (MONTHS)

HRS/INST/YR TOTALS 30R LESS 4 5 6 12

1.5-2 1310 1310

3 5

6 2490

1 2490

7

6 10

117 850

1274 13 84

1 934

14C5

11 20

281 169 75 525

21 60 5

57 8 5

65 TOTAL INST 466 4791 1310 89 85 6741 TOTAL HRS 7068 25.719 2210 1182 463 36,642 AVG HRS/INST/YR 15.2 5.4 1.7 13.3 5.4 3.9* GOVERNMENT

INDUSTRY

•WEIGHTED AVERAGE

Table 6. Counters

54
CAL/MAINT

HRS/IIMST/YR CALIBRATION INTERVALS (MONTHS) TOTALS 1 2 3 4 6

2.4 - 3.0 3940 3940 - 1314 -

3.1 -4 —

—

——

1310 —

—

— —

1 1310

1

5 10 —

297 —

88 385

11 - 50 12 - 12 147 33 53 14 247 TOTAL INST 147 33 5600 26 89 7209 TOTAL HRS 5662 4120 18.735 356 532 29,405 AVG HRS/INST/YR 38.5 3.1 3.3 13.7 6.0 3.3* GOVERNMENT 'WEIGHTED AVERAGE

INDUSTRY Table 8. Differential voltmeters

CAL/MAINT

HRS/INST/YR

CALIBRATION INTERVAL (MONTHS) TOTALS

3 4 6 12

0.9-1.0 164 164 2364 2364 2-3 3238 3238

396 57 1000 1453 4-5 1615 1615

6 OR MORE

111 630 5 746 TOTAL INST 507 57 6483 2533 9580 TOTAL HRS 2232 179 21.323 2531 26,265 AVG HRS/INST/YR 4.4 3.0 3.3 0.9 1.9- GOVERNMENT 'WEIGHTED AVERAGE

INDUSTRY

Table 9. Pressure gages

Phase 3 has also been implemented. The committee

has been restructured into an organization with eight

sub-committees. The organization chart is shown in figure 2. Several delegates, who have contributed a

large portion of the data on these instruments, have

already been appointed to serve on subcommittees

that are of major interest to them. A planning and

control staff has also been established to assist the committee chairman in the establishment of common

guidelines for the subcommittee chairmen to follow. However, each subcommittee chairman will be given

maximum freedom to operate his subcommittee autonCAL/MAINT

HRS/INST/YR

CALIBRATION INTERVALS (MONTHS) TOTALS

1 2 3 4 6

4 - 5 557 131 22 710

6 - 10 190 190

141 17 158 11-15

44 60 129 233

16 20

81 7 5 4 93

21 -35 16 16 92 14 30 136 TOTAL INST 136 312 740 136 216 1540 TOTAL HRS 3119 3945 546S 531 1450 14,513 AVG HRS/INST/YR 229 12.6 7.4 3.9 6.7 7.8" GOVERNMENT 'WEIGHTED AVERAGE

INDUSTRY

Table 10. Digital voltmeters

omously. As mentioned previously, the key objective

of each subcommittee will be to provide the means

for members to achieve cost reductions.

Phase 4 is just about to start. Each subcommittee

chairman will be responsible for providing the means

for NCSL members to achieve cost reductions asso- ciated with his instrument group. For example, some

of the means are:

1. Collection, analysis, and reporting of cost data;

2. Operation of an information exchange pro- gram;

3. Operation of a trouble-shooting summary li- brary;

4. A telephone directory linking delegates with common interests. Each subcommittee will have its own goals, such as:

1. Collection of a specific amount of data;

2. Reporting of this data on schedule;

3. Helping members save a specific amount of

dollars in a specific amount of time. When all the subcommittee goals are put together

they will meet or exceed the total goals of the committee. Once again, these goals are:

1. To collect 2 million manhours of cost data.

2. To help NCSL members save one million dollars before the 1972 NCSL conference. Be a CONTRIBUTOR.

Help MELT THE ICE.

55
I

lii

mi

or

III

irn 4

inb

re

ti

ai

irb

li

If
NCSL 70

SESSION 3 : NEW METHODS OF OPTIMIZING INTERVALS

Chairman: Carl Boyer

Honeywell, Inc., Annapolis, M<1. 21404

KEYS TO OPTIMUM CALIBRATION INTERVALS

Rolf B. F. Schumacher

Autoneties Division, North American Rockwell, Anaheim, Calif. 92803

The search for optimum calibration intervals or for methods to obtain them can be frustrating. Our theories concerning the behavior of measuring instruments are deficient, and as a result, so are

all theoretical models attempting to establish optimum intervals. Nor can we hope to determine

empirically what opimum intervals should look like. The many variables in calibration control systems, affecting quality and behavior and performance evaluation of measuring instruments,

forbid us to compare the meaning and merit of calibration intervals achieved in one calibration control system with those achieved in another such system. Time does not permit us to experiment

in one system until we achieve calibration intervals which are even near optimum. We must be modest and practical. We must lower our sights. This paper attempts to show the main sources of our ignorance about instrument behavior and

about the influence of a calibration control system on instrument behavior and on our evaluation of

this behavior. It offers seven keys to the establishment of an efficient method for calibration interval adjustment. Emphasis is on efficiency, practicability, economy. The purpose of adjusting calibration

intervals is to maintain a given level of instrument quality level at the lowest cost. Having found a satisfactory calibration interval adjustment method, one should look to other components of the

entire calibration control system for opportunities to lower costs.

Key words: Calibration intervals; economy in adjusting quality level; measurement uncertainty;

reliability of working instruments.

1. Introduction

Calibration intervals and the problems of determining them have been with most of us in industry ever

since the latter half of the 1950's when the Government began requiring periodic recalibration of measuring tools and instruments used by its defense contractors. The principle was sound: measuring instru- ments deteriorate with time and use; so, once in a while, they have to be recalibrated to make sure the measurements made with them are correct. Thus, cali- bration intervals were introduced, set, and adjusted, to

recall measuring instruments periodically to insure that each is within its tolerances at all times when in use. Soon, however, we found that this was not possible,

and we had to be satisfied with keeping most measuring instruments in tolerance most of the time. But how many, and how much of the time? This question

brought the two key problems of interval determination into clear focus, to wit:

1. Finding the function for time of instrument de- terioration (when does a given instrument get out of tolerance?

)

2. Finding the optimum quality level (how many

instruments should we, and can we afford to, have in tolerance at any one time when in use?) [I] 1 Closely connected with the second problem, finding

the optimum quality level, is the problem of measuring this quality level. This, in turn, leads to questions

about the standards against which the quality level is measured, the tolerance specifications for the instru- ments, and the calibration procedures—are they controlled, stable, and adequate? —about policies, personnel, etc. How certain are we that an instrument stated

to be in tolerance was really performing the way we

wanted? Uncertainty afflicts the measurement of the quality level of our instrumentation and blurs our

vision of instrument deterioration. Thus, the two key

problems of interval adjustment, namely the determination of instrument quality level and the time-depend- ence of instrument deterioration, are connected by the same factors which introduce uncertainty in both. We conclude that all elements of a calibration control system are inseparably linked together. An ex- amination of calibration intervals leads to an examination of the entire system. In the following pages

1 Figures in brackets indicate the literature references at the end of

this paper.

57
we first examine the factors influencing calibration in- tervals and then see what we can say about methods

of interval determination in general and how to de- velop a critique for them. Unfortunately, at the end

of this critical examination, we shall not find a method

for determining calibration intervals that can be generally recommended. But we shall see what we can do to select a calibration interval adjustment method

that will permit optimizing our calibration control

system.

To do that, we must first debunk the notion that any

one mathematical approach can solve the problem of obtaining optimum calibration intervals. We must be

fully aware of all the factors influencing calibration

intervals in order to innovate methods which will yield

optimum intervals and more cost-effective calibration control systems.

This paper deals only with the problems of calibration intervals for large numbers of working instru- ments. Where calibration intervals are determined on

the basis of standards observed individually and analyzed in detail, other considerations enter and have

been discussed elsewhere [2, 3]

2. Factors Determining the Length of a Calibration Interval

2.1. Time Dependence

At the basis of the concept of calibration intervals

is the notion that a measuring tool or instrument deteriorates as the effects of chemical or physical action

accumulate with time. But how do they accumulate?

Does the instrument deteriorate faster when we use it frequently? Or does it deteriorate independently of usage ? In most cases we do not know. There will be some

types whose deterioration is a function of usage alone. And there will be others whose deterioration will be a function of their exposure to the atmosphere. And

there will be still others whose deterioration is a function of something else. What is the factor contributing

most to their deterioration, and does the deterioration progress linearly, exponentially, stepwise, or how?

For most measuring instruments it would be impossible

to find the answers to these questions. And even if we could, we might find almost as many patterns of behavior as we have instrument types. Could we possibly have as many different interval adjustment methods? Certainly not. Our ignorance concerning measuring instrument be- havior will force us to make assumptions when estab- lishing calibration intervals. We generally make those assumptions which appear plausible and which most

likely apply to at least some of our instruments. So,

let me propose that, in establishing calibration intervals, we make the assumptions that help us develop a method that is inexpensive to implement and use. In a study to determine whether shorter calibration

intervals produce instrument families with higher in- tolerance ratios, one large family of like instruments

by one manufacturer, and with the same model designation, was divided into four groups of equal size. Each group was assigned a calibration interval of 5, 10, 15, or 20 weeks. Figure la is a compilation of

the in-tolerance percentages of the groups measured at

recall. ( Obviously inoperative instruments were ex- cluded from the count and charted separately.) The

study on instrument type A was repeated with type B

(fig. lb). Were the calibration intervals a major

factor determining the in-tolerance percentages of the instruments charted in figure 1? There is no clear answer, but the tolerance specifications against which

the instruments are calibrated appear to be a more important factor than the calibration intervals in this limited situation. One additional thought: The assumption that the moment at which an instrument slips out of tolerance

is uniformly distributed over a period of time, rather than as some function of time, may lead us to some

radically new methods of calibration control. However, for the purpose of the following discussion, I shall assume that there is at least some vague rela- tionship between an instrument's calibration interval and its in-tolerance condition, and that a low in-tol- erance percentage requires short calibration intervals and attendant high calibration costs. But let us re- member that these statements of conventional wisdom

deserve our most critical questioning.

100

! N TOLERANCE

III BLE ORIGINAL TOI ERANCES

"^IN TOLERA1JCE

/ /

/

ORIGINAL TOLE WNCES / V / \/

/ X

/ N

/

i

1NOPER

4 /

/

/

/ In-Tol

Intervals ( erance Ratios va Cal fan Electrical Mca£

Ibration uring /

/

/

/

Instrumen for Two Different Spacifications.

5 10 15 20 CALIBRATION INTERVAL IN WEEKS

Figure la. Instrument Type A.

58
-

IN rOLERANCE

DL UULb UK1U1NAL lULLIWNULS

ORIGINAL TO LERANCES

/

/

/

/

f/

1

INOPERATIVE - /

/

* /

/

/

/

/

/

In- Interv

Instru

Tolerance Ratios vs Us of an Electrical nent for Two Diffcr(

Calibration Measuring

nt Specifications.

t

10 16 20 CALIBRATION INTERVAL IN WEEKS

Figure lb. Instrument Type B.

2.2. Quality Level

By instrument quality level I mean the number of measuring instruments in tolerance, as a percentage

of all instruments under consideration. It is often as- sumed that this level is linked to calibration interval. Soon after it was realized that we will never be able

to have all measuring instruments in tolerance all the

time, arguments arose about how many we must have

in tolerance. Some people maintain that 95 percent

of all instruments should be in tolerance at recall. Some say this is too high a percentage; others maintain it is too low. For ail practical purposes, we do

not know what the quality level should be. We should stop here for a moment and consider the predicament we are in. We do not know what is happening to measuring instruments (the question of time

dependence), and we do not know what should happen (quality level). Are our calibration interval ad- justment methods in sad shape? You bet they are! What could be done is to calculate the cost of a calibration system producing a given quality level, say 90 percent, and also the cost of an alternate system

producing, say 95 percent. We could also determine

the dollar value of the reduced rework and scrap, and

estimate the incommensurate effects, like increased customer satisfaction. If there are no incommensurates, the quality level we want is the one which minimizes the difference between the cost of obtaining it and the dollar value of the benefits. If there are incommensurates, the difference is the cost of the in- commensurate benefits. Somebody in control over the system can then decide whether the benefits are worth

the costs or whether he would like to buy additional

benefits at the indicated rate. Examples of how an optimum quality level may be

determined are given in references [1] and [6]. This

brings us to the question of how to measure this quality level.

2.3. Measuring the Quality Level

Like all measurements, the measurement of instru- ment quality level is associated with uncertainties. The

sources of these uncertainties are found throughout

the entire calibration control system. One large and

unnecessary error stems from the practice of measuring quality level at recall, i.e., at a time when a measuring instrument is back in the calibration laboratory

for recalibration. What we really want to know is whether the instrument was in tolerance when it was

being used. Therefore, we should measure this quality

level in use, i.e., at the time when the instruments are being used, when it does make a difference whether an instrument is in tolerance or not. The effects of measuring at recall or in use are discussed in detail

in Appendix A.l. Other main sources of uncertainties which prevent

us from obtaining the instrument quality level we need

are found in the tolerance specifications against which

instruments are calibrated, and in the choice of characteristics the instrument must meet (as contrasted with

those which are incidental ) . Also, the procedures we

use when calibrating an instrument influence the magnitude and direction of systematic errors and the possible magnitude of random errors. These uncertainties becloud our decision as to whether an instrument is declared in tolerance or not, and whether or not to change calibration intervals. Then there are personnel policies, instrument-handling policies, treatment of tolerances, etc., which affect that decision. For a more

detailed discussion of error sources, see Appendix A.2.

It stands to reason that all elements which are sources of uncertainty should be examined with a view

to reducing the chances of making interval changes in- advisedly. But more than that, all elements must be

compatible with the objectives of the calibration control system. For instance, instrument tolerances must

be only as tight as product tolerances call for—not more, not less. And finally, all elements must be compatible with each other. For instance, if the interval adjustment method requires a technician to know the condition of an instrument when it was recalibrated

previously, the system must make this information

available to him.

Thus, there are many things which have a decisive

influence on the calibration intervals which we finally

assign to measuring instruments. And I submit that study of these holds more promise for minimizing cali- bration system costs and optimizing calibration inter- vals than a concentration on the mechanics of adjusting intervals.

59
3. How to Determine Optimum

Calibration Intervals

As explained in the preceding paragraphs, we are confronted with a number of factors which we suspect

will affect the quality level of measuring instruments.

Since calibration intervals are a function of this quality level, they are also a function of all these factors and their unknown effects. From the preceding discussion, we can derive the following seven "Keys to Optimum Calibration Inter- vals": Key No. 1 —There is no optimum calibration interval per se, and there is no optimum interval adjustment

method per se. Since we do not know how all important factors of

a calibration control system interact, we cannot optimize calibration intervals or interval adjustment methods theoretically. And to do it empirically, by trial and error, we have neither the time nor the funds in general. Hence,

Key No. 2—We cannot hope to optimize, in the strict- est sense of the word, a calibration interval; we can

only hope to arrive at a reasonably satisfactory one. "Optimum," i.e. "fairly cost-effective, satisfactory"

calibration intervals are the result of an "optimization"

of all important factors of a calibration control system,

of which the interval adjustment method is only one. Key No. 3—An interval adjustment method can be

"optimized" only in relation to a specific calibration

control system; a method satisfactory for one system

may be unsatisfactory for another. Key No. 4—An interval adjustment method "optimizing" intervals for a given control system —is economical to establish and to operate, —causes intervals to respond promptly to changes

in the system, —zeros in quickly on calibration intervals which

optimize the quality-cost relationship of the system.

Key No. 4 states the main criteria for selecting a

calibration interval adjustment method.

The largely unknown factors influencing calibration

intervals cannot be described by a mathematical moded; assumptions will have to serve in lieu of missing

knowledge. Hence,

Key No. 5—A satisfactory interval adjustment method

is one which yields an acceptable quality level at low

cost, regardless of the underlying assumptions, as long

as such assumptions do not contradict our experience.

There may be a number of equally satisfactory methods for any one calibration control system.

From this follows Key No. 6—An established satisfactory interval ad- justment method requires re-examination and possible

changes whenever another major component of the calibration control system is changed.

The emphasis here is clearly on the economics of choosing and establishing an interval adjustment method. Let us not get carried away by elegant mathematics. Several plausible interval adjustment methods

have appeared in various trade publications. Yet a check with the authors of several of these a few years

after publication revealed that the methods were either never implemented or were abandoned after a short

trial. They had failed the crucial tests of practicability or economy or both. In Appendix B, I describe an

interval adjustment method which has met these tests, although its theory is assailable. But it works in the environment for which it was designed. It may not be suitable for adoption in other calibration control systems.

An interval adjustment method and the resulting calibration intervals must above all fulfill the require- ments of Key No. 4, and be compatible with other components of the calibration control system. Once

these conditions have been satisfied, our attention may

then be diverted from calibration intervals toward

other problem areas within the system. An important key towards Progress to Innovative Metrology be- comes clearly visible: Key No. 7—"Feed the opportunities" which the cali- bration control system offers for obtaining optimum

intervals, "and starve the problems" presented by sys- tem components.2

4. Conclusion

Show me a calibration control system which achieves

a given quality level for its measuring instruments at the lowest cost and I'll show you calibration intervals which are optimized in relation to all other parts of

this system, and which permits you to direct your

resources towards the solution of other problems. If your interval adjustment method reasonably meets the requirements of Key No.4, future effort will probably

yield higher returns when applied to other problem

areas. Credit is due to Messrs. E. M. Hicks and J. D.

Mitchell of Autonetics for their many valuable comments and suggestions.

Appendix A. Sources of Uncertainties of

Instrument Quality Levels

A.l. Methods to Measure Instrument

Quality Levels

(a) At Recall—This is the method that should not be used unless it has been shown that there is a known

relationship between the quality level measured at recall and that found when the instruments are in use. For the quality level measured at recall to bear a meaningful relationship to the quality level of the in- strument population in use, we must first remove the bias which the calibration interval adds to the raw

ratio of "instruments in tolerance" to the total of instruments recalled. Some instruments are recalled

for calibration much more frequently than others, and

ideally those instruments are recalled most often which

need it the most. The bias may be removed by calcu2 Quote by Peter Drucker, management consultant and lecturer at NYU Graduate School of Business.

60
lating the percentage in tolerance as shown in table 1, using the formula

Percent in tolerance =

Jiiix + n2 i2 + n3i3 + . . . +nkili +

X 100

Qxk + Q2 i2 + Qj, + ... +<M+

where is a recall interval in weeks and Q1( is the total number of instruments recalled at interval 4> Table 1. The effect of the bias of calibration intervals on the apparent fraction of instruments in tolerance

No. of No. of instr. No. of recalls Total recalled No. of Inter- Fraction instru- per No. of in toler- instru- vals in toler- ments in instr. recalls ance per ments in ance at tolerance & per per year (cols. (Total) weeks recall at recall year year 3x6)

1 2 3 4 5 6 7 100 5 0.70 70 10.4 1040 728

100 10 0.80 80 5.2 520 416

100 20 0.90 90 2.6 260 234

100 30 0.96 96 1.7 173 166 1993 1544

Actual (unbiased) percent in tolerance: 70X5 + 80X10 + 90X20 + 96X30 x ^ = ^

%

100 (5 + 10 + 20 + 30) Apparent (biased) percent in tolerance

1544 T99H 77.5%

Another shortcoming of measuring the instrument

quality levels at recall is that it does not tell us whether

the system is taking appropriate actions with regard

to individual instruments or instrument families. If

it allows instruments which are usually in tolerance to be recalled very frequently, the percentage of instru- ments in tolerance would be deceivingly high, conceal- ing serious problems. The main argument, however,

against measuring the instrument quality level at recall

is that the obtained figure per se is totally irrelevant. When measuring instruments are back in the calibra- tion laboratory for recalibration, they cannot affect production cost and product quality.

(b) In Use—This is the percentage that is really of

interest. We want to know what the quality of measuring of those instruments is at the time they are being

used, when they could have affected production cost and product quality. To know this, however, we have

to go out on the floor and measure it. A practical approach to measuring the quality level of measuring instruments in use would be to select a small sample at random and calibrate it. Using timehonored methods of quality control, small sampling

methods, and control charts [4] , we could in due time

determine whether our calibration control system is actually a process "under control," whether it is kept

at the level which we have determined as being optimal, and how the calibration control system responds

to changes, "corrective actions," etc. This could be done at moderate costs which may be

more than offset by savings, because we would not have to recall all instruments to measure the quality

level. We could use the recall strictly to operate on

the calibration control system, i.e., to do only that which is necessary to maintain the quality level of measuring instruments under control. These considerations may open the field to alternatives to the pres- ently applied recall methods. However, regardless of whether we continue applying the old concepts of

recall or use new ones, we should measure instrument

quality levels "in use" rather than "at recall" to obtain

meaningful figures.

A.2. Error Sources in the Measurement Process

To measure the actual quality level of measuring

tools and instruments, one needs, as for all measurements, a standard, a comparison process, and decision

criteria. These are discussed in the following

paragraphs.

(a) Decision Criteria: Attributes versus Variable

Data. The upper and lower limits of an acceptable

quality level should be determined at the time when

the optimum quality level is determined. In theory,

the instrument maintenance program should then be

designed so that the measurement process is in control within these limits. Practically, however, upper and

lower control limits will be a compromise between

what is economically desirable as far as the production

process is concerned, and what is practically achiev- able as far as the instrument maintenance program is concerned.

Quality levels in instrument control systems are

usually given in "percent in tolerance" figures. I per- sonally prefer the use of "percentage defectives,"

because it is more in line with quality control lingo

and suggests the use of systematic quality control methods. Either statistic is based on an attribute—it is either in tolerance or out of tolerance. And that is

all that's needed for the measurement and control of quality level, since the vast majority of measuring

instruments require the simultaneous control of a number of parameters or characteristics. What counts in the case of determining common calibration intervals are only the statistics which show how often the in- struments are out of tolerance (and perhaps which

parameters are out of tolerance) not by how much

they were out of tolerance. Variables data, although more costly to obtain and

record, may be needed to analyze the behavior of unreliable instruments or instrument families. But these should be taken only when such analyses appear desirable. ( It should be remembered that we are talking

here about the bulk control of large quantities of working instruments, not about instruments which

which must be controlled individually, like standards

[2,3].) An enormous amount of variable data serves no purpose other than the satisfaction of some policy

provisions. And that is no good reason to take such

data. Variables data are somewhat more prone to error than attributes data. Thus, recording of attributes data may enable us to measure the quality level of measuring instruments more accurately.

61
(b) The Standard: Specifications. The standards

against which measuring instruments are tested are

usually their specifications. Much is being done now

to make specifications more meaningful; much more

remains to be done. Problems commonly encountered

in deciding what specifications to use and what they mean have been discussed in detail elsewhere [5] . I will, therefore, touch on some of these problems only

peripherally where they have direct bearing on cali- bration intervals. The specification we need to work with in a cali- bration control system must be a realistic one. Some

tolerance specifications, as they are given by instru- ment manufacturers, are at best optimistic. Some

instruments hold their quoted tolerance, say 1 percent,

only in the laboratory: some seem to hold their quoted

tolerance only in a random fashion; some others hold

it for a short time and require frequent recalibration

to remain useful; however, some instruments hold

their manufacturers' quoted tolerance for very long

intervals. In the case of an instrument whose performance

gradually deteriorates with time, we could select either

tight tolerances and short intervals, or loose tolerances with long intervals, or something in between. Which

tolerance-interval pair should we select? Generally speaking, we should select tolerances and

intervals which offer us the greatest return. The

optimum tolerance specification depends on the quality,

cost, and behavior of the instrument as well as the

cost of its maintenance. Let us take, for instance,

instrument A which costs $2,000 and which is capable

of holding its maximum realistic tolerance of 0.1 per- cent of some characteristic only at the expense of considerable maintenance costs (perhaps frequent recali- bration), say $700 a year. The same instrument may

be capable of holding a lesser tolerance, say 0.15 percent, easily, requiring only one annual calibration of $100 a year. Now let's look at the alternatives.

If we need a 0.1 percent instrument and the next

better one, instrument B, costs $5,000 with annual

maintenance I including calibration ) costs of $200, we may be better off with instrument A and high mainte- nance expenses and frequent recalibrations. On the other hand, there may be a less costly alternative available for the 0.1 percent instrument, but we may

also need 0.15 percent instruments for which the

alternative would be instrument C, a $1,000 instru- ment with $350 maintenance costs, including fairly

frequent recalibrations. In this case, it may be better

to use instrument A, assigning to it a 0.15 percent

tolerance and calibration intervals of one year. The optimum tolerance specification, like the optimum calibration interval, can be determined only in the context of economic alternatives—the requirements

of the production process, the handling and care of the instruments in their use environment, the training

of operators, etc. Calibration intervals change when

specifications change, but specifications are functions

of variables outside the calibration control system.

Figure 1 illustrates the effect of doubling the specifi- cation tolerance applied to a considerable number of

a popular VTVM. At 15 weeks, for example, the percentage in-tolerance jumped from 46 to 86 for type

A; while for type B it jumped from 78 to 93. There is also the problem of what to include under

the characteristics which determine whether an instru- ment is in tolerance or out. It may or may not matter whether an instrument with rated input impedance of

infinity has actually an input impedance of 50 Mfl,

whether an instrument with 5 minimum sensitivity

has actually 6 /xV sensitivity, whether a comparator

with rated contact pressure of 10 g has only 7 g contact pressure. If it does not matter, such characteristics should be left out of the consideration of an

instrument's in-tolerance condition. Many, many in- struments have been labeled out of tolerance for failure of some inconsequential characteristic to be within

manufacturer's specifications. Management can do

much to lengthen calibration intervals by precisely

defining which characteristics must be considered and

which must be excluded when judging an instrument

out of tolerance. The question of how to treat an

instrument with a broken line cord or damaged line plug may be trivial to engineers and supervision, but

not to the technician who may decide to call such an

instrument out of tolerance because something was wrong with it. (c) The Comparison Process. Calibration proce- dures are widely considered as crutches for dimwitted

calibration technicians. Adequately staffed calibration laboratories frequently consider procedures unnecessary and the respective Government requirements a nuisance, but this is based on a misconception.

A calibration is a measurement process. A measure- ment deserves its name only if it is repeatable. The

procedure describes the process and, therefore, makes

it repeatable. Two different measurement processes, intended to measure the same quantity, mai by chance

yield the same numerical result. On the other hand, two measurements made by the same process may yield two different results. No result means much without

a procedure which tells us how we got that result. The measurement method which a procedure de- scribes will influence the magnitude of systematic

errors in the result. The number of relevant details

in a procedure influences the magnitude of random

errors. Therefore, adequate calibration procedures

are prerequisites for meaningful interval adjustment

methods. Calibration intervals can be expected to vary as calibration procedures change.

( d ) Policy. The policies governing a calibration control system have a marked effect on calibration

intervals and the methods for optimizing them, and

it is in this area of policy where management can

exert its greatest influence in reducing costs and maximizing output. We have already noted that the toler- ance specifications against which an instrument will be judged must be carefully defined for cost-effective interval adjustment methods. But there are other

policy elements which will affect results, e.g., the question of how to treat completely inoperable instru- ments, or instruments which are obviously malfunctioning, or how to treat the tolerances of calibration standards.

<>2
Some laboratories have a policy of subtracting the tolerance of the standard from the tolerance of the instrument under test if the tolerance ratio is larger

than 1 :4, for instance. One popular standard used to calibrate the VTVM's of figure 1 provides only a 1:2

ratio on the tolerances quoted by the manufacturer.

By subtracting the tolerance of the standard, one would

in effect cut the tolerance of the VTVM in half, or expect the instrument to perform within a tolerance

it was not intended to meet. On the other hand, ignoring the tolerance of the standard would be tantamount

to increasing the instrument's tolerance and increasing

its percentage in-tolerance. Other important factors by which laboratory policy

influences the quality level for measuring instruments

and tools concern the generation, maintenance, and

use of data sheets from which quality decisions are made; the motivation of laboratory personnel to show

high in-tolerance percentages; the motivation of lab- oratory personnel to charge time to uncontrollable

repair.

Personnel selection and training are still other

factors. In one instance, we have found that new- comers to the calibration laboratory were assigned to calibrate the common varieties of measuring instru- ments. As these technicians became more proficient,

they "graduated" to more sophisticated types of measuring instruments. The result was that the commontype instruments had a lower quality level than when

they were maintained by more experienced personnel.

This had a marked effect on the overall quality level of measuring instruments, because of the relatively

large number of common-type instruments.

Appendix B. An Empirical Interval

Adjustment Method

The following describes an empirically developed

method for the adjustment of calibration intervals of measuring tools and instruments. It is offered as an example of a method meeting all the requirements of the economics, practicability, and speed of response

for a given calibration control system. It is not rec- ommended for adaptation to other calibration control systems without careful analysis of compatibility with

other system components and system objectives. The

method achieves a given quality level in use within

reasonably narrow bounds of uncertainty, even though the decisions concerning interval changes are based on the in-tolerance conditions of individual

instruments at recall. Exhibit 1 shows the Decision and Assignment Tables

from which the calibration technician determines cali- bration intervals. With an instrument given him for calibration is a record of the most recent tolerance codes for that instrument. These codes—1, 3, or 5

are defined below. He checks the instrument and

records its incoming tolerance code. Using the Decision Table, he selects either an asterisk or one of

the letters p, m, d. The asterisk means "Do not change the instrument's current interval"; a letter refers him to the Assignment Table, from which he

selects a new interval.

Exhibit 1. Interval adjustment schedule

DECISION TABLE

Previous Tolerance Codes Incoming Tolerance Code

CRIS TREND

(Left to right) 1 3 5

111 V d

* 311 * d

* 511 nV d

* #13 * m m

#15 * m m

#31 * m * #33 * m m

#35 * m m

#51 * d

* #53 * m m

#55 * m m

ASSIGNMENT TABLE

New Interval Interval Current Interva 1 Current New Current New

Interval p m d Interval P m d Interval I' m d

5 7 5 21 25 14 1') 37 12 25 33

6 8 5 5 22 26 14 20 38 43 26 34

7 9 5 6 23 27 15 21 39 1

27 35

8 10 5 7 24 28 15 22 40 16 27 36 9 11 6 8 25 29 16 22 41 47 28 37 1 10 13 6 9 26 30 17 23 42 48 28 38

£s 11 14 7 10 27 31 18 24 43 49 29 39 a 12 15 7 11 28 32 19 25 44 50 29 40 £ 13 16 8 12 29 33 20 26 45 51 30 41 g 14 17 8 13 30 35 20 27 46 52 31 41 H 15 18 9 14 31 36 21 28 47t 52 32 42 ~ 16 19 10 14 32 37 21 29 48 52 33 43

17 20 11 15 33 38 22 30 49 52 34 44

18 21 12 16 34 39 22 31 50 52 35 45

19 22 13 17 35 40 23 32 51 52 36 46

20 24 13 18 36 11 24 32 52 52 37 47 *Do not change interval. #Any or no entry. **Do not release instrument; contact supervision for instructions, tlnstrument must be recalled at least once a year for service. Calibration schedule, however, may be determined as follows: p—Add 10 weeks if Interval 2:47. m or d—Assign 52 weeks if Interval >52. The tolerance codes are defined as follows:

1—In tolerance (All calibrated characteristics of the instrument were in tolerance as received, and Code

5 does not apply)

3—Out of tolerance (At least one calibrated

characteristic was out of tolerance, and Code 5 does

not apply)

5—Any of the following:

a. Out of tolerance condition indicated by the instrument's user on the accompanying calibration

requisition form and confirmed during calibration.

b. Malfunction; calibration impossible prior

to repair, provided the required repair could have

changed one or more calibrated characteristics of the instrument (excludes minor repairs not affecting cali- brated characteristics, like line cord repair, blown

fuse, etc.)

c. Instrument found out of tolerance when

calibrated to a tighter specification than it was previously calibrated to.

63
The interval adjustment method does not provide

for calibration intervals of less than five weeks, and

any instruments for which a shorter interval is indi- cated are pulled out and subjected to individual

analysis. The method also provides for a mandatory

analysis of instrument types whose average calibration

interval is seven weeks or less, to determine whether

changes in specifications, calibration procedures,

standards, or usage may improve the balance between

maintenance costs and instrument utility. The five and seven week limits were arbitrarily selected. We

feel that an instrument or type which needs recalibration so frequently is generally costing too much in periodic maintenance. We have annual data-processing reports listing the average calibration interval of each instrument type; a single instrument with a

calibration interval significantly shorter than the

typical average interval is also subject to examination.

References and Bibliography

[1] Schumacher, Rolf B. F., A unified approach to the mainte- nance and calibration of measuring instruments, paper

presented at the 2d Natl. ISA Instrumentation Maintenance Symp. (Houston, Tex., 1967).

[2] Schumacher, Rolf B. F., Keeping standards and measure- ments under control in a standards laboratory, Paper

#68-612, presented at the 23d Ann. ISA Conf. (New York,

N.Y., 1968)

[3] Schumacher, Rolf B. F., Statistical control in a standards

laboratory, Meas. and Data (May-June 1969).

[4] ASTM Manual on Quality Control of Materials, ASTM

Spec. Tech. Publ. 15-C (Jan. 1951).

[5] Couture, R. J., and Held, T. M., Calibration /service specifications, an improved way to document test equipment

specifications, Paper #68-607, presented at the 23d Ann. ISA Conf. (New York, N.Y., 1968)

[6] Hicks, E. M., On optimizing accuracy ratios and instrument

quality levels, Engineering Rept., Metrology and Calibration Dept., Autonetics, Anaheim, Calif. (1970)

64
NCSL 70

INTERVALS BY EXCEPTION

Jerry Glassman

U.S. Navy Metrology Engineering Center, Pomona, Calif. 91766

"Intervals by exception" is a new approach devised at the Navy Metrology Engineering Center

to control the evaluation and adjustment of calibration intervals for the Navy Calibration program.

It furnishes management with a powerful tool with which to monitor calibration intervals and assure

reliability of equipment.

What are some of the interesting features of this new technique?

1. It enables management to concentrate on problem equipment by pinpointing "dogs" and "gems,"

i.e., statistically poor and statistically good equipments.

2. It allows management to select "optimum" intervals based on automatically derived reliability tradeoffs.

3. It motivates laboratory personnel by providing an information feedback loop for problem

instruments.

4. It lowers operating costs by funneling analysis and engineering effort into the areas of greatest needs and most promising returns.

5. It lowers costs by lengthening intervals on family types and stabilizing reliability levels.

6. It provides automatic computer monitoring of all decisions (cause-effect feedback)

This system is presented with charts that illustrate its operation, outputs, and management-oriented controls. The presentation includes data analyses, adjusting of intervals, detection of "dogs" and "gems," resultant changes in system operating characteristics, underlying mathematical

assumptions, and mathematical models.

Key words: Calibration intervals; exponential distribution; reliability, systems effectiveness, tradeoffs.

1. Introduction

The manager of a calibration laboratory is faced

with a multitude of decisions, many of whose effects can influence overall operations for years to come. The decision to change a calibration interval can affect numerous system operating characteristics, such as:

a. equipment reliability/maintainability,

b. laboratory operating costs,

c. repair time,

d. operational schedules,

e. calibration time,

f. manpower distribution,

g. equipment availability,

h. personnel motivation.

The complexity of the interactions of these important system characteristics makes it incumbent on the manager to avail himself of simple decision-making

tools. Sophisticated system-effectiveness techniques

are necessary to generate the tradeoff alternatives required by management. However, the decision- making apparatus and management-oriented monitoring functions should provide complete visibility and

control, and immediate feedback of system effects. An analytical method has been devised to evaluate equipment calibration intervals and provide justifica- tion for recommending changes based on the cumulated results of past calibrations. The approach is rigorously developed and contains several unique

features not found in other systems. Additionally,

this system provides management with tools with

which to make difficult optimization decisions without

referring to the underlying mathematical processes. The tradeoffs are made available to him, and the predicted system effects are provided for several

alternatives. A decision-making system is effective only if it is used to make decisions. It is a valuable management

tool if it is easy to use and provides overall system

visibility and direct feedback of effects. "Intervals by exception" offers new management decision-making

approach to calibration interval determination and

interpretation without compromising mathematical

rigor.

2. Systems Based on Model Number

or Serial Number

Most calibration interval analysis systems fall into one of two categories:

1. Intervals by model number. All equipments of the same model number are treated as if homogeneous,

and a calibration interval is established for the model

number family.

65
2. Intervals by serial number. Each unit of serially

numbered equipment is assumed to function according

to its own unique operating characteristics, and a

calibration interval is established for each individual

equipment.

Both of these general approaches betray serious

shortcomings in application. If the deficiencies are

not eliminated, the resulting decisions are seldom optimum and are often erroneous. If the deficiencies are

eliminated, the systems become overcomplicated, and

lose the characteristics that initially made them

attractive. Examples of deficiencies related to calibration in- terval systems based on model number or serial number are as follows: Case 1. Intervals by Model Number

a. Different manufacturers might have produced

equipments having the same model number.

b. Different fabrication methods might be used for equipments of the same model number.

c. Equipments might have experienced design modifications without model number changes.

d. Cumulative operating time is different for equip- ments of the same model number installed at various

dates.

e. Operational/environmental differences may exist for equipments of the same model number.

These and other factors can cause significantly

varying operating effects that should be considered

or removed. The usual treatment entails subgrouping

the equipments by significant factor levels, and then

establishing separate intervals for each subgroup.

Figure 1 is an example of the multiplicity of subgroups that arise when as few as three significant factors are considered. As can be seen, the magnitude

of the problem rapidly becomes unmanageable as the number of significant factors increases. The serial number system evolves when it becomes advisable to consider all the factors contributing to the equipment

operation.

Case 2. Intervals by Serial Number

a. Initial establishment of intervals for new equip- ments is at best an educated guess; when the serial number philosophy is used, the historical data avail- able is of little use for establishing intervals.

b. The amount of calibration data accumulated for one serial number is generally insufficient for meaningful statistical analysis. Therefore, the assurance

of equipment reliability is essentially impossible for most equipments.

c. Management control systems based on serial number data are completely meaningless in a system

effectiveness context; i.e., the resultant decisions can- not be related to other system parameters, nor are meaningful tradeoffs available.

d. Administrative and routine controls are difficult. As a result of the above shortcomings, serial-based

interval evaluation systems tend to substitute ritualistic and arbitrary rules for rigorous statistical methods.

3. The New Approach

Recognition of these discrepancies precipitated an

investigation of new approaches to calibration data

analysis and interpretation in this vital area of metrology. "Intervals by exception" is the result of this

FACTOR 1

FACTOR 2

FACTOR 3

MODEL NUMBER GROUP

T

II I I

i i i i

!

i 1111 J L

SIMPLE

MANAGEABLE

DIFFICULT

IMPRACTICABLE

III I

I

i i i i

i ! _ J I L_l _l 1 SERIAL NUMBER SYSTEM

Ficure 1. System complexity induces conversion from a model number to a serial number system when the number of factors considered is increased.

66
investigation conducted at the Navy Metrology Engineering Center for application to the Navy calibration

program.

Imagine a group of instruments of a particular

model number designation. Ideally, these equipments

should be capable of achieving a specified reliability,

when used as required for a given time between cali- brations. However, because of the diverse influences

of factors affecting equipment reliability, each serial- ized equipment of this model may follow a different pattern of failure. The random interaction of factors causes most

equipment to be influenced by a combination of ele- mentary negative effects. Thus, compensatory factorlevels tend to balance and the residual effect varies

slightly around the average. Hence, the majority of equipments have similar operating characteristics. The only equipments that display operating characteristics significantly different from the average are

those having a preponderance of negative factor-levels or positive factor-levels. These "exceptions" in the model number group, and only these, need be considered for special interval treatment. To illustrate the effect of this random interaction, suppose that the ratings and weights shown in table 1 are arbitrarily assigned to items, A, B, C at factorlevel 1 in figure 1, to items 1, 2, 3 at factor-level 2, and to items Z, Y, X, V, U at factor-level 3. For ex- ample, subgroup A1Z consists of equipments manufactured by company A, using production method 1, and having an operating age of Z months; its rating

would be —3 as shown in figure 2. At the other extreme, subgroup C2V rates as +3. Each of these two equipment subgroups would probably have an

operating reliability deviating significantly from the average, one much poorer and one much better. We

designate such equipments as "dogs" and "gems,"

respectively.

Table 1. Assumed factors, levels, level ratings, and

weights for a hypothetical model number.

Factors Levels Ratings Weights

Manufacturer A -1

B • 0 C + +1

Production method 1 -1

2 + +1 3 • 0

Operating age Z -1

Y • 0 X • 0 V + +1 u • 0

Ratings legend:

+ better than average • average — poorer than average

As can be seen in figure 2, most of the subgroups

(10 of 14) have ratings of —1, 0, or +1; these would

probably have near-average operating reliability, and

may be considered to be "equivalent equipments."

Subgroups with ratings —2 or +2 are borderline

cases which may or may not show exceptional

reliability.

C3U

A3X B3X B3V

A2Z A3V B2X

A1Z A1 Y A1V A2Y B2Y C2U C2V

-3 -2 -1 0 +1 +2 +3 SUBGROUP RATING

Figure 2. Hypothetical histogram of equipment

subgroups by subgroup rating.

Figure 3 illustrates the simplicity in concept of

"intervals by exception." Compare this to the complexity and multiplicity of subgroups generated in figure 1.

MODEL NUMBER GROUP

DOGS EQUIVALENT EQUIPMENT GEMS

Figure 3. Simplified representation of classification based on intervals by exception.

"Intervals by exception" is a new approach devised

to evaluate and adjust calibration intervals based on

a compromise between the simplistic model number

and the ritualistic serial number systems. Not only

does it allow for statistical analysis of the majority

of instruments by model number, but it pinpoints

those equipments requiring special attention.

4. Norms, Dogs, and Gems

In general, equipment belonging to model number

families exhibit similar operating characteristics if manufactured and operated under similar conditions. However, within any model number family, minor

differences exist that cause particular serialized items

to perform significantly better or worse than the model number norm. Differences in manufacturers,

production method, operating time, or other factors can cause these variations. By isolating (excepting)

these significantly deviating items, realistic calibra- tion intervals can be determined for the remaining

conforming equipment comprising the majority of the model number group.

67
Equipments that perform significantly differently

than the norm are called "dogs" or "gems" according

to whether they are, respectively, poorer than or better than the group average. These are the exceptions in the model number group:

1. Dogs—individual serialized items that yield a

statistically poorer out-of-tolerance rate than the family average.

2. Gems—individual serialized items that yield a

statistically better out-of-tolerance rate than the family average. Analyses of these exceptional equipments provide a valuable potential for generating cost savings, improving equipment operation, and disclosing new

areas of engineering knowledge related to equipment

performance characteristics.

5. Mathematical Assumptions and Tests

The basic direction of "intervals by exception" is toward management-oriented control and monitoring

features. However, there is a mathematical basis for

the tradeoffs and decision criteria. The detailed mathematical models and statistical tests that are used are contained in the Appendix.

The following mathematical assumptions are made

in applying this technique at Navy Metrology Engineering Center:

1. The times-to-out-of-tolerance are exponentially

distributed according to the function ke~kt. 2. The probability of out-of-tolerance increases

monotonically as the time between calibrations is in- creased for a given equipment.

3. Failure characteristics for all equipments of a

given model number are described by the same timeto-out-of-tolerance function, except for a small per- centage of "dogs" and "gems."

The analyses of the records, for a given family of equipments and a given calibration interval, follow

the general guidelines described below:

1. Data are purged of irrelevant and erroneous

entries; i.e., obvious mistakes and human errors are removed from the data base.

2. The data are adjusted using criteria and re- quirements based on known or assumed system effects. For example, the tails of the time-to-failure distribution are truncated to avoid using nonrepresentative

or heterogeneous data; the times to out-of-tolerance are estimated, because only times-to-discovery of out- of-tolerance are available, rather than actual times-todriftout.

3. Statistical tests of significance are applied to detect changes in equipment operation. This requires

computer grouping of data by calendar times and computer testing of the consistency of the out-of-tolerance

rates. If a statistically significant change is detected,

only the most recent consistent data are utilized for analyses. Therefore, the data used for analyses are homogeneous. The effects of design modification,

procedural changes, and environmental/operational

differences are discovered, isolated, and analyzed

separately.

4. The cumulative Poisson probability distribution

is used to test the relationship of the parameter for an

individual serial number to the average for the model

number. This detects statistical outliers; i.e., serial- numbered "dogs" and "gems."

5. An estimate of the out-of-tolerance rate is ob- tained for the remaining group of equivalent

equipments.

6. Predictions of the probabilities that the equip- ments will remain in-tolerance are made for several

different calibration intervals. This provides tradeFlGURE 4. Sample of a calibration interval analysis, 01/07/70.

TIME

MODEL NUMBER 370 MFR: WEC BETWEEN

SUBMISSIONS N OOT

4.5 — 5.4 2 0

* 5.5 - 6.4 11 2

6.5 — 7.4 6 0 DATA USED *89.2 PERCENT* 7.5 — 8.4 8 1 YEARS = 1/67-12-68 8.5 — 9.4 25 2 N' -95 9.5 — 10.4 13 1 OOT' = 8 10.5 — 11.4 15 0

11.5 — 12.4 7 0 ANALYSIS RESULTS 12.5 — 13.4 5 1 K = 0.9055 PERCENT *13.5 — 14.4 5 1 T — 883.5 15.5 — 16.4 4 1 ESTIMATED RELIABILITY .90 .85 .80 .75 16.5 — 17.4 1 0 CALIBRATION INTERVAL 12 18 25 32 17.5 — 18.4 2 0

18.5 — 19.4 5 1 CURRENT INTERVAL = 12 DOG AND GEM SERIAL NO. NO. LOCATION

DETECTION—1/70 NUMBER REPORTED OOT CAL ACT CUST P

DOG 10432 6 3 TAX SHOP-3 .029 DOG 19313 5 BBT DB-14 .043 GEM 26813 10 0 AAA PT-109 .019

68
CALIBRATION LABORATORY COMPUTER

CALIBRATION

SYSTEM

OOO

OOO

OOO

TESTS PERFORMED

RAW

DATA

FEEDBACK

RESULTS:

Optimum Intervals Decreased Costs

Increased Performance

More Meaningful Calibration Motivated Personnel

GEMS

FEEDBACK

Figure 5. Flowchart of closed-loop feedback system for intervals by exception.

69
ofTs enabling management to make the decision for adjusting intervals with total visibility of the effect on reliability for the model number population.

7. New methods are being developed to derive

reliability/effectiveness and interval/cost trade offs for determining optimum calibration practices. These

trade-offs will allow management to consider the sys- tem effects of out-of-tolerance equipment and evolve

criteria for repairing versus replacing items.

8. An original approach is being developed for esti- mating and applying confidence limits to reliability/ interval decisions. Some equipments perform so well

that few failures develop in them. Current methods

do not provide a high degree of assurance for extend- ing intervals on these kinds of "gems." The new

confidence-limit approach allows extension of intervals with increased statistical assurance.

6. Results

It was necessary to design new computer output

formats for the calibration interval analysis so that the results could be efficiently presented to management. These also provide prompt feedback of de- cisions to cognizant user activities, and facilitate engineering analyses of "dogs" and "gems."

Figure 4 is a sample presenting the purged calibration data, the estimated reliability/interval trade-offs,

and the "dog" and "gem" serial numbers. Note that

the data for "equivalent equipment" includes only the number reported and the number out-of-tolerance (00T) ; reports of damaged and inoperative equip- ments do not appear. The major computer analyses subprograms are:

a. Data truncation.

b. Data adjustment.

c. Statistical tests for significant differences be- tween data groups and for the establishment of the data base.

d. Determination of "dogs" and "gems."

e. Calculation of reliability/calibration interval

trade-off alternates. Figure 5 is a flowchart of the closed-loop feedback

system implemented for "intervals by exception."

Table 2 is an example of the input to management

showing the interval/reliability trade-offs for selected values of out-of-tolerance rate.

OUT-OF-TOLERANCE

RATE, K CALIBRATIONINTERVAL RELIABILITY

.016 7 .894 10 .852

14 .799 18 .750

.017 6 .903 10 .844

13 .802 17 .749 .018 6 .898 9 .850

12 .806

16 .750

Table 2. Partial table of tradeoffs between estimated

calibration intervals and reliability values, by out-of- tolerance rate, k.

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 MONTHS-BETWE EN -CALIBRATIONS

FIGURE 6. Distributions of current and projected intervals for 60 families of equipments.

70
7. Conclusions

' A new philosophy has been presented for inter- preting calibration intervals. This philosophy, called "intervals by exception," has been developed into an

(

i analytical system for evaluating calibration data.

Statistical techniques, resourcefully applied, constitute

the framework of this system. The system enables

management to confidently make interval adjustment

t decisions with complete visibility of their probable

: effects on equipment reliability.

This system is being used at the Navy Metrology

Engineering Center to evaluate calibration data and

evolve decision criteria on which to change intervals.

i A test program was conducted on 60 equipment types

submitted for calibration at short intervals. Optimum

decisions were made for these 60 equipments; some

intervals were increased, some decreased, and some

not changed. The average interval for these 60 was

increased from 6.8 months to 8.2 months. Figure 6

shows the exponential probability distribution of the 60 projected intervals superimposed on the distribu- tion of the 60 intervals currently in use (equal numbers at 3, 4, 6, 9 and 12 months). As can be seen, there is a significant shift (increase) in the mean.

Also, there are many more instruments at longer

intervals on the projected curve, and the percentage

out-of-tolerance has been reduced from 30.1 to 15. This is clear evidence of the long range (recurring)

cost improvements generated as a result of using

"intervals by exception."

It has been demonstrated that meaningful results can be obtained by approaching a problem in an orig- inal manner. It is hoped that this effort acts as a

catalyst for imaginative inquiry into other problems

in metrology in addition to its being used in its own

right.

Acknowledgement is due Kim Bruce, mathematician,

for' reviewing and testing the mathematical models,

manually demonstrating system performance characteristics, and proofreading the final draft.

8. Appendix

8.1. Reliability

It is hypothesized that the probability that an in- strument remains in tolerance is a function of the elapsed time since it was last calibrated. This probability, called reliability R(t), is equal to the complement of the cumulative probability distribution P{t)

for times-to-out-of-tolerance for all instruments of that family; i.e., R(t) = l-P(t).

The underlying distribution of times-to-out-of-tol- erance is generally assumed to be an exponential of the form.

f( t ) =ke~kl

, 0^ t< oo, = 0 , otherwise

where k is the out-of-tolerance rate; k>0, and e is the base of natural logarithms. The cumulative distribu- tion function, which represents the probability that an equipment goes out-of-tolerance before time t is

P(t) = fj( t) dt= (ke-ktdt= l-e-kl

The probability that an equipment remains in-toler- ance for a time interval equal to t is

R(t) =J*f(t)dt= l-JJ{t)dt = e-ki

where R{t) is the reliability. It can be seen that R(t) + P(t) = 1. For a more detailed discussion,

see reference [l]. 1

8.2. Data Consistency

If equipment of a given model number undergoes

design, procedural, or operational revision, its out- of-tolerance rate may change. Only relevant calibra- tion data should be utilized when decisions are made

affecting the current configurations and uses of equip- ments. A statistical test is performed to detect signifi- cant changes in out-of-tolerance rates for data grouped

by calendar time. The probable occurrence of out-of-tolerance reports

in any specified period can be mathematically described

by a Poisson process with parameter equal to the ex- pected number of out-of-tolerance reports.

' ( ' ~ Xl

where f{X) is the probability of obtaining X out-of- tolerance reports, k is the family out-of-tolerance rate, kt is the expected number of out-of-tolerance reports,

e is the base of natural logarithms.

An exact F-test of equivalency for two sets of Poissongenerated data on out-of-tolerance reports per unit time is derived in reference [2] and shown below:

F1_,(2(X,+ 1),2J1 )

where Xx and X2 are the number of out-of-tolerance reports for any two groups of data, Fj and T2 are the corresponding accumulated operating times for these groups, and

7\ TV

8.3. Detecting Dogs and Gems

Tests of significance are performed to detect statistical "dogs" and "gems" by comparing data for each

individual instrument with the model number failure rate estimate. The following computations are per- formed for each instrument:

1. The expected number of out-of-tolerance reports

for a given serial number with operating time Tg is calculated: kTg = TgZri/ZTi

where k is the estimated out-of-tolerance rate for the model number family; r, is the cumulative number of

1 Figures in brackets indicate the literature references at the end of

this paper.

71
out-of-tolerance reports for the ith serial number, Ti

is the cumulative operating time for the ith serial number.

2. The kTg (expected number of out-of-tolerance re- ports for the given serial number) is compared to the

Tg (observed number of out-of-tolerance reports for

the given serial number).

3. If kTg < Tg,_ _^e-^g[kTgV

is calculated.

4. If P.i = 0.05, the serial number being tested is a

statistical "dog."

5. If kTg > rg ,

is calculated.

6. If PB = 0.05, the serial number is a statistical "gem."

9. References

[1] Zelen, Marvin, Ed., Statistical Theory of Reliability, p. 15

(Univ. Wisconsin Press, Madison, 1963)

[2] Brownlee, K. A., Statistical Theory and Methodology In Sci- ence and Engineering, 2d ed., p. 145 (John Wiley & Sons,

Inc., New York, 1965)

72
NCSL 70

OPTIMIZING CALIBRATION INTERVALS

Donald J. Greb

Manager, Metrology and Standard Tools, Lockheed Missiles and Space Company, Sunnyvale, Calif. 94088

This paper describes a two-method system for using simple attributes calibration data to adjust and

optimize calibration intervals. Each method is used only where its superior characteristics predominate; the combination of methods produces a total system which exploits the advantages of each

without having to suffer with the disadvantages. The "fixed interval through data" method is employed where there are at least 27 bits of data, the applicaion being unique in that the data is treated as a random sample from an infinite population, sampling variations are accounted for, and

all decisions to extend calibration intervals are characterized by statistical validity. When there are only from 5 to 26 bits of data, statistical validity is not possible and the) "fixed interval through

engineering intuition" method is utilized. However, intuition and judgment are applied only to a set of decision rules, and strong, specific data support is required. The system has not been extended to less than 5 bits of data, although there would seem to be no problem in so doing. Net results in the

first 18 months of use are 495 changes involving 34,200 units, with annual savings of 24,100 technician

hours, and with no noticable deterioration in overall instrument reliability.

Key words: Data support; fixed calibration interval; instrument reliability; statistical validity; two-method optimization.

1. Introduction

The combination system of adjusting and optimizing calibration intervals described herein was de- veloped over a period of about two years in the Lockheed Measurements and Standards Laboratory (MSL).

The "fixed interval through data" method was worked

out first, and after six months implementation it was

evident that the method is highly satisfactory as applied to instrument groups for which a substantial amount of data is at hand. However, it also became

obvious that the method was either woefully inadequate or totally unusable for families for which few

data are available, and that a satisfactory means of handling the latter was badly needed. Out of this need evolved the second method and the combination

of the two has quite satisfactorily solved a problem of very long standing.

2. Definitions

Operative/ Inoperative. An instrument is operative

when all performance parameters are functioning and

capable of yielding data or information. An instru- ment is inoperative when one or more performance

parameters are disabled and incapable of yielding data

or information.

Operational Reliability. (R0 ) for a particular group

of instruments, over a given period of time, is the

ratio of the number of operative units to the sum of operative and inoperative numbers of units. In-tolerance/Out-oj-tolerance. An instrument is intolerance when all performance parameters are func- tioning within specifications. An instrument is out- of-tolerance when one or more performance parameters are not functioning within specifications. ln-tolerance Reliability. (R,) for a sample of units submitted for calibration is the ratio of the number

in-tolerance to the total number operative.

Actual In-tolerance Reliability. (Ra ) is unknown

except where data are at hand for an entire population,

and no sampling is involved. Ri is an estimate of Ra , and statistical limits for Ra with any desired degree of confidence can be determined from the data used to calculate Ri. Although Ra is unknown, its limits and

the degree of confidence associated with the limits are highly useful.

3. Alpha-Code Data Base

The basic source of reliability information is the alpha-code marking, directly related to operational and

in-tolerance reliability. The alpha-code is a single

letter recorded by a technician on the Service Report

and it provides significant information on the as-re- ceived condition, reason for submission, and work per- formed. The alpha-codes, instrument categories to which they apply, and category definitions are: A—ACCEPTANCE — applicable to instruments

which are

• New or serviced for the first time

• Returned from vendor, storage, or held for parts

• Put into hold for manual status

7.\
• Submitted by any facility external to Bay

Area MSL

• Placed in calibration storage without calibra- tion

• Sent by In-place to an MSL laboratory

D—DAMAGED—applicable to instruments which

show evidence of physical damage or operational misuse.

F—INOPERATIVE - - applicable to instruments

showing one or more performance parameters

incapable of yielding information.

C—OPERATIVE IN-TOLERANCE - - applicable

to instruments with all performance parameters

functioning within specifications.

0—OPERATIVE OUT-OF-TOLERANCE — appli- cable to instruments with one or more performance parameters not functioning within specifi- cations.

4. Reliability Computations

Figure 1 shows the logic and procedure used by the technician in assigning an alpha-code to the instru- ments submitted to MSL.

P — average population of an instrument group over

a number of months, m. N = Pm = total number of units submitted to MSL

in m months.

reliability Rn and in-tolerance reliability Ri are computed as: A+D

Ri =

m

A+D

C+0

Y-0 C

Y Y

5. Fixed Interval Through Data

After literature search, discussion with our laboratory personnel and customer representatives, and ex- amination of considerable data, the following criteria for extension of calibration intervals were set:

• Ri must be 95 percent minimum and R0 must be

92 percent minimum.

• At the new interval, the probability that Ra is at

least 92 percent must be 80 percent or more.

Originally, 90/90 percent was chosen for Ra rather than 92/80 percent because 90 percent is the midpoint

of the 85 to 95 percent in-tolerance acceptability range. However, one of our customers uses an 80 percent

confidence factor almost exclusively in reliability cal- culations, so in order to conform to their normal practice the confidence factor was dropped to 80 percent

Acceptance Inoperative Out-of-tolerance

F

D Damaged In-tolerance

Figure 1. Logic for assignment of alpha-code.

These codes are mutually exclusive, so that there is one and only one correct alpha-code for each item. The only situation where the technician may have a

difficult judgment is in determining whether an instru- ment has suffered physical damage or has been misused in operation. It should be noted that the in- tolerance/out-of-tolerance determination is made by

comparing a quantitative measurement with specified

parameter limit (s) and that each code provides substantial information on what the instrument is and

what it is not. For example, the code "C" provides

information that the instrument was operative, in-tol- erance, had not been subjected to physical abuse or operational misuse, and was not in any one of the conditions classified as ACCEPTANCE.

The estimates of reliability are based on the numbers of instruments accumulated over a stated period

in these various alpha-code categories. Operational

and the reliability figure raised to 92 percent to compensate. Figure 2 is the chart of in-tolerance reliability versus sample size available, used for determination of permissible calibration interval extensions in accordance with the criteria above. Zone 0 permits no change, Zone 1 permits an increase of one-third,

Zone 2 permits an increase of 50 percent, and Zone 3

permits the interval to be doubled. (Extensions from

6 to 12 or 12 to 24 are not permitted because of other

restraints.

A minimum of 27 bits of data is required to satisfy the criteria and provide statistical validity. The 27-

bit minimum is a function of the Ra 92/80 percent

criteria and will be larger or smaller as these percent- ages are increased or decreased. An important point

with respect to the criteria is that inoperatives (malfunctions or failures) are not ignored as they are in many calibration interval schemes. R„ must be a

74
20 40 100 200 400 1000 2000 SAMPLE SIZE Figure 2. Chart for determination of permissible

extensions in calibration intervals.

minimum of 92 percent and thus acts as a permissive/

restraining factor in extension of calibration intervals but does not influence the magnitude of the extension

since it has no influence on Ra .

0.00 IO0 1.50 CALIBRATION I NTERVAL - t

Figure 3. Two-step process for determination of zone boundary lines in figure 2.

Calculation of zone boundary lines. The method used

to calculate the zone boundary lines of figure 2 is a straightforward two-step process involving elementary

reliability statistics theory. Figure 3 illustrates the

process. First, an R, at 1.00 t (the current calibra- tion interval) is assumed and noted as point A. Point

B is calculated on the assumption that the failure1 rate k is small and constant within bt (where b is 1.33,

1 In this discussion the term "failure" refers to an out-of-tolerance condition, i.e., failure to meet accuracy specifications, rather than an

operating failure where there is a complete malfunction. When Ri is high and failure rate is low (as is generally the case), it is valid to assume that the failure is of the "chance" or random variety and

that the exponential relation applies. This subject is treated in many

books on reliability. (For example, see pages 3-5 of Reliability Theory

and Practice, by Igor Bazovsky. Prentice-Hall, 1961.) In particular, it should be noted that the calibration process is essentially identical to the "overall" process referred to on page 5.

1.50, or 2.00, depending on which zone boundary line

is being calculated ) . In this situation the reliability

equation R = e~bkt applies where R — reliability, e = base of natural logarithms, k is the failure rate, and t is a variable calibration interval. In our calcula- tions the failure rate is assumed for points A and B.

If Ri = 97.00 percent at A, then the number 1.00 kt can be read as 0.0305 from tables of the descending

exponential (as in the NBS Applied Mathematics Series 14). Then kt is multiplied by the ratio of the new calibration interval to the old, and the new value

of the R, estimate is rated from the tables using expo- nent bkt. In figure 3, b = 1.50, whence bkt — 0.04575

and Ri = 95.53 percent; this is independent of sample size. Now let us examine the second step. It is desired to calculate a sample size such that, when Ri = 95.53 percent, the probability is 80 percent that Ra is 92 per- cent or greater. This involves confidence limits, and

for these calculations the booklet Confidence Limits

for Attributes Data, a Lockheed document, has been

used. (An alternate method is described on page

373 of Engineering Statistics by Bowker and Lieber- man, Prentice-Hall, 1959. The Lockheed booklet is used because the calculations are much simpler and

the range of available confidence limits is wider). A

value of N is assumed, the number of failures calculated

as N (1 — Ri). and the 80 percent lower limit of Ra found in the tables. If Ra is over 92 percent, the as- sumed value of /V was too high, and if Ra is less than

92 percent, it is too low. The /V and R t values thus found fix one point on a zone boundary line in figure 2 and the process is repeated for enough points to define each such line accurately. As might be deduced

from the above discussion, point C of figure 3 is a function of point B, sample size, and the desired confidence level. When sample size becomes high enough

so that TV (1 — Ri) is greater than 10, the Lockheed

tables are no longer usable and some other method

must be used. When sample size is large and percent

defective small, the Poisson distribution may be used

to calculate these confidence limits. For reference,

see paragraph 2.4.5, page 438, Quality Control and

Industrial Statistics, Acheson J. Duncan, Richard D.

Irwin, Inc., 1959, or many other books on mathematical statistics. Sample calculations of two boundary

line points are given in the Appendix.

The statistical techniques used to calculate boundary

line sample sizes when there are no defectives in the sample (/?,=: 100%) are necessarily different from

those above. When /?, = 100 percent, point A is al- ways 100 percent and point B is also always 100 percent. However, the problem can be solved by ap- proaching it through the combination of sample size and the magnitude of calibration interval change per- mitted in Zones 1, 2, and 3 respectively. Zone 3 will be used to demonstrate the logic involved in making

the calculations. Zone 3 permits the calibration in- terval to be doubled and when the calibration interval

is doubled, the rate of data input is approximately

halved, assuming a constant population. Therefore,

in the case of Zone 3 and Ri = 100 percent, we need to

75
answer the question, "What is the minimum sample

size which, when divided by 2, provides that Ra is at

least 92 percent with a probability of 80 percent?"

The Lockheed tables are useful in making this deter- mination. Using the 80 percent lower limit table, we

see that a sample size of 19 with 0 failures has an 80

percent lower limit of only 91.88 percent but a sample

size of 20 has an 80 percent lower limit of 92.27 percent. By doubling 20, we obtain 40 as the minimum

sample for Zone 3 when /?j = 100 percent. Following

the same logic, sample sizes for all three zones are determined as follows: Zone A B C D

1 20 0.75 1.33 27

2 20 .67 1.50 30

3 20 .50 2.00 40

where A = minimum sample at which R„ = 92 percent

with a probability of 80 percent when

Ri = 100 percent.

B = Amount of data obtained after extension

compared to that obtained prior to extension. C = 1/B = the factor by which A must be multiplied to obtain the sample size which per- mits extension.

D = A X C = Minimum sample sizes for Zones

1, 2, and 3 when /?, = 100 percent.

6. Fixed Interval Through

Engineering Intuition

The use of the "fixed interval through data" method

is limited to instrument families large enough so that

at least 26 operative instruments are serviced in some

arbitrary period of time and the zones of figure 2 are

calculated on the assumption that the data are a sample and that sampling variations have been taken into consideration. However, there are many instrument

families which take a very long period of time (2 to 5 years or more) before 27 operative instruments will be serviced, and it is therefore highly desirable to develop other criteria applicable to instrument families with fewer than 27 which will permit calibration in- terval extensions for them.

To proceed along these lines requires dependence

to some degree on considerations other than statistical sampling theory. The most important of these considerations are:

1. The quantity of data exceeds, sometimes by a substantial amount, the instrument population, so that

the uncertainties of sampling variation incorporated

into the other method are diminished by a great

amount, probably an order of magnitude.

2. The period of time over which the data are accu- mulated exceeds, sometimes by a substantial amount,

a full calibration interval so that the chance that an

active unit might not be included in the data is quite

minimal.

3. No increase in calibration interval will be made

if there are any out-of-tolerance units in the sample.

Rn is calculated in the normal manner and must be at least 92 percent.

4. The items being considered for interval extension

will almost always have some comparable items on which the interval was extended, using data for which

the sampling uncertainties were accounted for.

5. Engineering judgment and knowledge concerning instrument design, function, application, and usage

is a legitimate substitute for recorded data, even though it is quite difficult to qualify that judgment.

6. The operational risks are relatively small be- cause of the small instrument populations involved.

On the basis of these factors, table I identifies per- missible interval extensions in terms of both the absolute and relative amount of data on hand. Examples

of changes permissible under these rules: Active population 9, calibration interval 3 months, sample size 23 —

Category I — Change to 6 months

Active population 14, calibration interval 6 months, sample size 19 —

Category II — Change to 9 months

Active population 3, calibration interval 4 months, sample size 8 —

Category III — Change to 6 months

Active population 7, calibration interval 9 months, sample size 11 —

Category IV — Change to 12 months

The decision matrix of table 1 has some characteristics which distinguish it from other schemes employing intuitive judgment:

1. All the intuition is applied to generation of a

set of decision rules, thus allowing the application of judgment to specific cases to be precise and consistent.

2. Once the decision matrix is made, any clerk,

technician, or laboratory personnel can apply the rules with ease.

3. All decisions are supported by a strong, well- defined data base.

Table 1. Permissible extensions of calibration intervals with 5 to 26 bits of data, fixed interval through engineerCategory

Sample

Size Quantity of Data Relative to Active Population Changes Permissible, months

I 16-26 Minimum of double 1-2, 2-1. 3-6,

II 16-26 Equal or greater, but less than double 2-3. 3-4. 4-6, 6-9, 9-12, 12-18, 18-24

III 5-15 Minimum of double 2-3, 4-6, 6-9, 12-18

IV 5-15 Equal or greater, but less than double 3-4., 9_12, 18-24 Note: Changes permissible in categories II and IV are also permissible when data are in categories I and III.

76
7. Calibration Interval Decreases

Theoretically, the logic and statistical methods for decreasing calibration intervals and thereby bringing

Ri up to an acceptable level should be the same as

those used for increasing intervals. As a practical

matter, this does not work at all. Assume for the moment that the criteria for decreasing intervals are made comparable to those for increasing intervals:

• Ri must be less than 85 percent.

• At the new interval, the probability is 80 percent

that Ra is at least 92 percent.

l I I

i—i

r~rr

d 80

w 75

° 70

100 Sample size 400

Figure 4. Chart for determination of calibration

interval decreases.

Note that the second criterion is identical to that for increasing intervals. Using the same calculation methods as previously, zone boundary lines were calculated and are shown in figure 4. The differences between

figure 2 and figure 4 are very startling:

• The "no change" zone is very small.

• The zones are reversed in that the most severe change zone (1) is next to the no change zone (0)

and the least severe change zone (4) is farthest re- moved from the no change zone (0).

• Except for the no change zone, the smaller the sample, the more severe the change.

• There are no zones at all for reducing intervals by 25 percent (12 to 9 months, for example), 33 per- cent (9 to 6 months, for example), or 50 percent (12

to 6 months, for example).

The second and third points are explained by the

fact that: the smaller the sample, the larger the uncertainty; and the larger the uncertainty, the more se- vere the necessary action to be sure that desired re- sults are forthcoming. The last point shows that there

is no value of R, below 85 percent such that a 25, 33, or 50 percent decrease in interval will make Ra — 92

percent with any reasonable degree of certainty.

These conditions, therefore, show rather clearly the severe limitations of attempting to improve R, by de- creasing the interval and the prudence of exploring

other, more fruitful corrective actions.

8. Procedure and Documentation

When a calibration interval change is to be made,

a Reliability Bulletin is initiated by the Measurement

Standards Laboratory (MSL) Reliability Engineer,

who provides the required data, as shown in table 2. The bulletin is signed by the MSL Manager and ap- proved by the cognizant Navy and Air Force Cali- bration Specialists. It thus provides the vehicle for generation and approval of a change and also provides permanent documentation for the change.

9. Limitations on Calibration

Interval Extensions

There are several situations and conditions where

it is prudent, at least in the initial phases of application, to impose specific arbitrary limitations on inter- val extensions, even though the data might permit

greater or more frequent extensions:

1. The maximum individual extension is 3 months

for all instruments except those with 12 or 18 month

intervals, where an increase of 6 months is permitted.

2. The data used to support a change shall cover a

period equal to at least one-half the current interval,

regardless of how much might be available over a

shorter period.

3. After an interval extension has been made, no

further extension can be considered until a period of time equal to the sum of the old and new intervals has

elapsed, and none of the data used to consider a sec- ond extension shall be older than the new calibration

interval. If a change from 4 to 6 months is made, for example, no further extension can be considered for

Table 2. A typical reliability bulletin

NOTIFICATION OF RECALL PERIOD CHANGE

Manufacturer/Model — Electronic Measurement TO Series Power Supplies

Station Code — LEMK

Change to be Made — 09 to 12 months

Effectivity Date — 2 March 1970

Quantity Involved — 27 Data Supporting Change:

For the 13 months ending January 1970 —

Total calibrated — 68

Operative — 58, R0 = 97.1%

In-tolerance — 57, Rc = 98.2%

A = 7 D = 1 P = 27 Sample size 58, Rc = 98.2 percent falls into Zone 3 which permits extension of the recall period from 09 to 12 months.

77
10 months and the data for considering an extension

at that time can be no more than 6 months old. This

limitation accomplishes two things: (a) it allows the data bank to be purged of all data from the old in- terval period, and I b ) it allows for a full cycle of the population to be processed and therefore assures that any subsequent consideration will be based on data from the entire population.

4. The calibration interval of an instrument powered by a battery with a limited life, an instrument

needing regular mechanical attention, or one with

similar functional time-dependent limitations, will be

governed by those considerations and not by R JRU - 5. Calibration interval extensions shall be made

only four times per year on the first Monday of March. June, September, and December. These times

coincide with the new printing of preprinted Service

Reports, with calibration interval being part of the preprinted information.

10. Results

The family size for more than half the equipment

reaching MSL is between 1 and 9 instruments; other percentages are 15 percent for 10 to 19 items; 18 per- cent for 20 to 49. 8 percent for 50 to 99, 4 percent for 100 to 199, and two percent for 200 or more. The

first changes made by the two-method system de- scribed were effective 1 December 1968. Including

those which were effective on 2 March 1970, results have been as follows:

• Total interval changes—495.

• Total instruments involved—34,200.

• Maintenance of 88-92 percent in-tolerance reliability, as shown by figure 5.

• Steady reduction in total scheduled calibrations per unit per year from 2.35 to 2.06, or 12.3 percent.

• Steady reduction in total scheduled calibrations per year—16.6 percent from 119,600 to 99,600 or 16.6 percent. The difference between 16.6 percent

and 12.3 percent is accounted for by a 4.3 percent

drop in active instrument population.

• Exponential reduction in annual technician hours. The reduction is expected to reach 24,100 hours by

August 1971. Note that a full interval must elapse

before the reduction associated with a specific change

begins to be realized.

• Major changes in distribution of calibration in- tervals, as shown in figure 6, where each stepped

column covers seven quarters. The pattern of changes

is quite clear. Early in the program, emphasis was

on changes from 3 to 4, 3 to 6, and 4 to 6 months,

since 39 percent of the total population was on three- month intervals and the yield in hours is high. Later

the emphasis shifted to changes from 6 to 9 and 9 to 12 months, and finally a substantial quantity from 12

to 18 months.

With the passage of time, many more extensions

will be made and a considerable number of second

and possibly even third extensions. It is estimated

that by the end of 1972 the total saving in annual

technician time will be on the order of 45,000 to 50,000 hours.

The writer wishes to acknowledge the large contribution of Messrs. D. B. Schneider, Research Specialist, and C. W. Gebhardt. Reliability Engineer, the Lockheed Measurement Standards Laboratories, to the theory and development of the statistical aspects

of the system. Also instrumental in the overall for- mulation of these methods was Mr. M. L. Brink of

the Navy Special Projects Office.

In - Tolerance Reliability - %

85

J FMAMJJASONDJ FMAM

1969 1970

Figure 5. In-tolerance reliability, 1969 and 1970.

IV,
Distribution Of Calibration Intervals

40

Z

§ 30

o,O

a.

i <t—Ot—LUo

65

20

10

4 6 9 12 CALIBRATION INTERVAL IN MONTHS

18

Figure 6. Quarterly changes in distribution of calibration intervals.

11. Appendix: Sample Calculations of Boundary Line Points

The calculations below show determination of two

points on the 2.00 t boundary line (between Zone 2 and Zone 3) in figure 2.

Point 1 Ri = 0.9800 at 1.00 t Ri = 0.9604 at 2.00 t, p = 1 - 0.9604 = 0.0396

N = 60 Np = 2.376 Ra = 0.9225

(assumed N too large)

N = 50 Np = 1.980 Ra = 0.9169

(assumed N too small)

N = 55 /Vp = 2.178 Ra = 0.9199

(assumed N slightly small)

N = 56 /Vp = 2.218 Ra = 0.9205

Therefore, /V = 56, Ri — 98 percent are the coordinates.

Point 2 (using Molina's Tables of Poisson's Exponential Binomial Limit, D. Van Nostrand, 1947)

Ri = 0.9650 at 1.00 t Ri = 0.9312 at 2.00 t, p = 1 - 0.9312 = 0.0688

N = 726.7 2 Np = 50 Np (80%) = 56.41 Ra = 1 - 56.41/726.7 = 0.9224

N = 581.4 Np = 40 Np (80% ) = 45.78

Ra = 1 - 45.78/581.4 = 0.9213

yV = 436.0 /Vp = 30 /Vp ( 80% ) = 35.06

Ra = l- 35.06/436.0 = 0.9196

N = 465.1 Np = 32 Np (80% ) = 37.22

Ra = l~ 37.22/465.1 = 0.9200

Therefore, N = 465, R, = 0.965 = 96.5 percent are

the coordinates.

Zone 0—No change

Zone 2—1.50f 2 to 3 months

4 to 6 months

6 to 9 months

12 to 18 months

Zone 1l.S3t 3 to 4 months

9 to 12 months

18 to 24 months

Zone 3—2.00*

1 to 2 months

2 to 4 months

3 to 6 months

2 Non-integral sample sizes used so that Np can be integral to simplify interpolation in the tables.

Example of use: sample size = 75; in-tolerance = 73; Ri —

73/75 = 97.3 percent ; t = 6 months. Sample size = 75 and R i = 97.3 percent lies in Zone 2, per- mitting extension to 9 months.

Zone 0—No change; Zone 1 —Reduce interval by more than

75 percent; Zone 2—Reduce interval by 75 percent; Zone 3

reduce interval by 66 percent.

Point A = Rj at 1.00 t; point B = best estimate of Ri at 1.50 t; point C = Rn

is equal to or greater than 92 percent over 80 per- cent of the time

79

NCSL 70

OPTIMUM CYCLES DETERMINED WITH CONFIDENCE

P. A. Seamans

Electronics Laboratory, General Electric Company, Syracuse, N.Y. 13201

The calibration history of a family of instruments provides data on how often the instruments were

recertified and whether each was in calibration when recertified. These data permit the calculation

of the optimum calibration interval for a desired level of confidence in the instrument readings. Three case histories (oscilloscope, RX meter, VTVM) show this method is effective. The histories of

a family of oscilloscopes over a thirteen-year period show that calibration cost tends to be roughly

constant with time and thus does not seem to follow the "bathtub curve." All instruments inves- tigated seem to have equal likelihood of failure in all intervals, and thus the bathtub curve does not seem to hold in general.

Key words: Calibration histories; equal likelihood; oscilloscopes; RX meters; vacuum-tube voltmeters.

1. Introduction

For the past several years we have been using a technique for determining the optimum length of time

between successive calibrations of different kinds of instruments. The technique uses data from the cali- bration history of a given family of instruments and

allows a balance to be struck between the cost of errors due to possibly faulty equipment and the cost

of calibrations. The technique gives a numerical measure of the level of confidence the instrument user can have in the readings his instrument gives. The

technique has been proven effective for a wide variety

of scientific equipment used in the electronics industry, and it is probably suitable for most instruments.

2. Collecting Instrument Data

We have established in the Instrumentation and

Measurements Subsection at the General Electric Elec- tronics Laboratory a computer-based system of keeping records on the calibration and maintenance of many kinds of instruments. 1 This system keeps histories of over 10,000 instruments with about 30 manhours a week of effort, largely of a clerical nature. Data processing cards (DPC) that record pertinent

information about the calibration of an instrument

1 This computer-based system is fully described in a report, Instru- ment Calibration Records: Establishment of a High-Confidence Data

Bank. Interested readers should write Technical Information, General Electric Company, Electronics Laboratory, Syracuse, N.Y. 13201, and request TIS Report R69ELS-115.

TEST EQUIPMENT

WORK TAG |

TEST EQUIPMENT WORK TAG

| DPC ]

CUSTOMER

RECALL 8 DELINQUENT

REPORTS

WORK TAG

RECERTIFIED EQUIPMENT

RECORDS 8 CONTROL HISTORY MSP DEPT a UNIT CALIBRATING ELECTRONICS INVENTORY AREAS LABORATORY REPORTS

COMPUTER

FACILITY

RECERTIFIED EQUIPMENT

| DPC] | REPORTS | [ WORK TAG ] DPC 1

Figure 1. Flow-chart of computer-based system for keeping

instrument calibration records.
m n m id n in *-i h« o « o «r^« to m «-i *-i r> <o »^ vr> r> t\ »nr3 H r) n rin h n rif) ri ^ O HHrlHHridHHQ) H H*4a O D e I) «4 **4 wH H H H w<© H »H H »H H rl H^HHHrJH^ *-i rl H Hri H «-t | HHHHr(HHdH(VHdHHHriHHHH*lTff4HHHHHHHHHHHtlT4dHHr(HHHrlHHriHHHd

j ae rjr^o o«ofvitJOoaDeno{noooo«woBowoDot)K) o o t, c o o o a c o o «o an ) h

< -*fO««*»«* <-<«OI<>«« » « rlPO f » (O » .-l»«'»«OW'»«-w»''» »-t« ««<>« HI O (M

»- — — —. _. — — _ — — — — O mNON»Iln«n« HMO! (MM» O M»)MWMt<N«CX«« HMO IM IMH[y(MNMIMNrlNN«IOIC«Hr,)a>«

ow nnoiniM o«oi n ootv « n n m(M koi o ox x x o« o o«x o cm ox onn ox a x o« o« o» o««

0000 00 0<ZiOOOOOOOOOOOOOOOOOOCOOOOOC>OOC>OOOOOOOOC30C30000 Od)d)iD9}Xid4)a)])i)a)tiio3ia)9)])i)OQata>D4)iDioiDxtaoto4)d4)a)SoodioA(fl3)io(OWOO ao> co

3

I\l«N*N»an«Mg><IMNNNI)«niDNOINHKriM*N<>NO(MNrlNMNNIUnri(\INNIV(MT<*(l CO

to

CO©o&

o

~se

-* • hi CO *- z> x o j in x e 3 m tf> cn to z> v>=> «n 3 => in o w jo bijki kijio.ox in k- x j> x >c z> ^ x x ft x cn x cn m :j x tn o x xxx u>xnxu>x xui xma x -O

z>x 3«3r>o3x^-«zjoox3oaix r> u z> d x x ] x j i o _i :m :> j x j n o x z> :n z> * j x •« z> x ^O

boc

>.^^^V.^^V.V.V.V.V.V.V.^>»V.>v^V.^'S.'^V.-V'N.'N.'N. -V>v'V'N.'%.>w'S.'N.'V"S.-v-s.'V'V ,V.'N.>>.V.'V-VV.V.V. cn»p)OinN««fl^ <h« o o n « nno> DhnKO w o oo ro u ry ,( on f) ~* xi on MN«nh •-«<=» C« a NCd O O HOI <J HH 111 OOIHCXM aoHrfrt^CMrfoTOHnHOfllfl 001(0 HM>f«ll>0 (MOO

g

.h»x ri Oi T-t»r»- o rit^ tMJrtariNv nNan«» an«« rt«\ k a«s afltn a n« » MIA O K>K a O o DOD do o t1 ii [i n t > »H f j H o d o t"j j>*hooooooowoooooo v40owooc30ooouooow '

aoi-or ro h» to ^ no nr> o o oh NnMShsNnis«NoroN«K*«Nt<i w»r\ o at^ oo> nMnnm J;

•~»T\J Nflli>«'0 «•« « T-t<» 3 0»NII)»T« « O HCMI> .-I O X Ho il«\ «» » 1 K H<1 N « « « « 1\» t O D_ nnnr)n«»i> k> k> o od o « nsi««««»ii)«oinr)«(Oooo umm> nKarODoroaoi vnoi

(D to O fO ao AO "U XJ O CO U d> CO CD f— <D CO 3) «o 'O CD CO "D OO CD D CO O ao O CO O CO CO ,' > ao CO O CO 'O CD CO CO CO 00 ^0 CO ID 00 CO

o

»3 o> o ' hio t ir i ii H^rytM oo*4*4 C^IT% inii\«NOi4«»Ain^inkto»(>(iinroroLn'i^in<i<ooNrvivooc> ' ^ i\i«innnr)nnn«KM« ocsoocjr^t^r^aocoaocoio 7iK>cotnirtirt««« «ir>ir\ir\ir>ioir\i(\ir>i(Mr>>nirt<>«4> ~- HHHririHt4HH«ionnsNNNN*irf««««t 3H ^-fit\ n tn * x * ir*^-*-T*x^x*x^*«"«r* »l ,1 W ^ H »1 ,1 ^ H IM t\l CVi fU IV W !M i-) ^ n Kl ^"1 K) n f\ Jl 4' <_l O O l> ( f> t> > O l> O (> O '> t> l> O O*

82
form the input to the computer ( fig. 1 ) and "reports"

of various kinds form an output which permits esti- mation of optimum calibration intervals. One of the computer reports is a "history report,"

consisting of the calibration history of a given family

of instruments and useful in calculating the optimum

intervals between calibrations. Figure 2 shows a sample history report for a certain kind of oscillo- scope plug-in. It includes an instrument code that corresponds to the family of instruments; the name

of the manufacturer and his model designation; the

identification serial number assigned to each different

I instrument; and a work tag number, there being one work tag for each job that is performed in our cali- bration area. Other entries indicate the date the cali- bration was performed, the technician who performed

the calibration, the nature of the trouble, and the cost

of calibration or repair. But of major interest to us

in determining the optimum interval between calibra- tions are the date entries and the "status" entries,

which indicate (among other things) whether an in- strument was found to be in calibration when it was

checked. These two entries are used to derive the

actual number of calibrations per year and the actual number of calibrations per defect, which in turn are used to calculate the optimum number of calibrations per year. After an instrument has been judged in or out of calibration, it is either recalibrated or given "preventive maintenance," whichever is appropriate. (Preventive maintenance might involve bringing the instru- ment's performance closer to nominal values even though it was within manufacturer's specifications;

replacing a weak battery or tube to avoid failure before the next recertification date; or cleaning switch

contacts or wiper arm contacts to permit intermittent readings during the time until the next calibration.)

Closely associated with the in-calibration or out-of- calibration status of an instrument is something we

call "failure"—it occurs when the calibrating per- sonnel determine that erroneous readings could have

been made without the user's knowledge. An example

might be a pointer that is bent, though not enough

to be noticeable to the inattentive eye; or there might

be friction or jewel roll present. "Failure" would

have occurred if the instrument was out of balance

and was used in several planes. A more subtle example might be a defect in an oscilloscope power supply

that causes the rise-time trace to be incorrect, a

"failure" that may be more serious than if the oscillo- scope did not function at all. In fact, if the power

supply had failed (in the conventional sense), then

in accordance with our definition it would not be a

"failure."

3. Calculating the Optimum

Calibration Interval

The data found in the history report (fig. 2) en- able us to calculate the actual number of calibrations per year, (C/t), for a given family of instruments,

where C is the number of calibrations performed over

a period of t years; and the actual number of calibrations per defect, (C/D), where D is the number of

defects, or instances of an instrument's being out of

calibration. Then it can be shown that the optimum

number of calibrations per year can be expressed:

(CA)opt = (C/t)

'^°pt

where the term (C/D) opl , the optimum number of calibrations per defect, is found by entering a Poisson

distribution table for a given confidence level. An example will indicate the utility of this expression: Suppose that, for a particular family of instru- ments, calibration takes place four times a year and

that to date a total of 158 such calibrations have been

made, including 46 "failures." Then we have (C/t) = 4, and (C/D) = (158/46) = 3.43. If we wish 90

percent of the instruments returning for calibration

to be within manufacturer's specifications, then we

find from the short Poisson table (table 1) that (C/D) cpl = 3.9, and the above expression becomes:

(C/«) w =4(||)

= 4.55 calibrations per year

(calibration every IIV2 weeks)

Table 1. Poisson distribution for at least one failure occurring in interval

Confidence Level (C/D) opt (Calibrations Per Defect) %

99 6.7 95 4.7 90 3.9 85 3.4

4. Evaluation of the Technique

Our experience with this technique proves that it works and is applicable to test equipment. To demonstrate that we can predict optimum calibration in- tervals with confidence, I cite three examples:

Generators prior to 1968 had been calibrated every 26 weeks. When we first applied our method of find- ing optimum calibration intervals, we found that in order to maintain a 90 percent confidence level, the

cycle should be shortened to 15 weeks. When we

had collected a year's data at the new interval, we

computed the optimum value again and found it to be

18 weeks. (The reason for the three-week discrepancy is unclear, but the important thing to note is that our first attempt to adjust the optimum value was in the correct direction.)

Perhaps a better example for illustrating the accu- racy of this technique would be our experience with

RX meters. As in the case of the generators, the

calibration interval for these instruments had been 26

weeks. Our computations showed that the cycle should

be shortened to 15.8 weeks. We thus shortened the cycle to 15 weeks; at the end of a year we checked the

results and found the optimum to be 14.7 weeks—very

close to the originally calculated value.

83
A third example, involving a lengthening rather than a shortening of the calibration interval, involves

several families of VTVM's. One of these families was routinely calibrated every 13 weeks, but the others were calibrated every four weeks. After we studied

the failure rate of these instruments, we found we

could lengthen the calibration intervals 5, 7, or 10

weeks for those that had been calibrated every four

weeks; and the thirteen-week instruments were now

calibrated every 26 weeks. A subsequent study after a year's operation showed these revised calibration

intervals still to be correct.

5. Discussion

Many readers are perhaps acquainted with the "bathtub curve" that is said to reflect the life and

death of instruments (fig. 3). It is my opinion that

this curve is not entirely applicable to the broad spec- trum of test equipment, although perhaps some mechanical systems follow it. If test equipment obeyed

the bathtub law, the age of a piece of equipment would

be the determining factor of the calibration interval. Instruments would have to be segregated by age, and

in a large calibration area the economics of instru- ment maintenance would be difficult. It would then

be possible that instruments purchased this year would

have a different calibration cycle than the same type

purchased a year or two ago. But the technique I have described for determining the optimum calibration interval assumes a Poisson distribution, which is used when the same number of failures are expected

in each interval. To me this suggests that test equipAGE OF INSTRUMENT

Figure 3. The well-known "bathtub curve" that

is said to reflect the useful life of instruments.

ment failures might be a linear function of use, de- pending on environment and care, rather than on age

as long as the age is reasonable.

I have recently completed a study of a well-known

manufacturer's oscilloscope. I collected data on 29

instruments, including purchase date and the average

cost of calibration for four years (1966-69). A total of 252 calibrations was included in the study of these instruments, some of which were purchased as long

ago as 1957. A plot of these data is scattered about

a horizontal line (fig. 4) rather than the bathtub

mentioned above. The cost of calibrating instruments

12 years old was not much different from that of equipment purchased quite recently. The anomalous

value at 10 years (*) results from a modification which

involved expensive repairs.

c/5 tr

<-> a. I I I

I I I

I ' I I I I 0 2 4 6 8 10 12 AGE OF OSCILLOSCOPE IN YEARS Figure 4. The 1966-69 cost of calibrating 29 oscil- loscopes aged 1 to 12 years scatters about a hori- zontal line and does not seem to follow the bathtub

curve.

The procedure described may bring about a possi- ble savings by the mere fact that an analysis of the history of instruments may permit the elimination of "dogs." Furthermore, some cycles can perhaps be

lengthened, resulting in further savings. The method

is, of course, of little use if some sort of systematically

scheduled calibration system is not in force. In re- viewing any family of instruments, I include only

those instruments that have been recalibrated at least half as often as the optimum calibration interval would

require.

84
NCSL 70

SESSION 4: NEW WAYS OF MANAGING

Chairman: J. R. Van de Houten

Bell Aerosystems Corporation, Buffalo, N.Y. 14240

BREAKTHROUGH TECHNIQUES FOR METROLOGY WORK

Lloyd B. Wilson

Management Consultant, 145 Tanners Pond Road, Garden City, N.Y. 11530

Metrology work involves the conflicting and diametrically opposed problems of control versus breakthrough. Controls are vital to assure accuracy and reliability. However, breakthroughs are necessary to provide improved accuracies, ranges, and types of measurements. The dichotomy of this situation comes principally from the differences in attitudes involved. Ideally, control and break- through should be carried out by two different types of people because of these differences. Breakthrough techniques, coupled with proper attitudes and a systematic method for establishing

objectives and evaluating alternatives, provide a powerful set of new tools for improvement. Examples

are given showing the possibilities for applying these tools to reduce costs and provide needed measurement and calibration services.

Key words: Managerial breakthrough techniques; personnel selection for metrology; steady control versus sudden change.

1. Introduction

This is a paper about problem solving. The type

of problem which will be covered is breakthrough,

where breakthrough is defined as a process by which

improvements of a major nature are made rapidly.

Breakthrough obviously relies heavily on creativity

and innovation. However, it is more than this be- cause of the rapidity of the change involved; break- through is a dynamic word which denotes a sudden

or step-function type of change.

This paper stresses the achievement of breakthrough on a systematic basis. This is possible because break- through, like many other types of problem solving, has

become a science which can be performed on a

rational, step-by-step basis. Evidence of this is found

in the literature on the subject of breakthrough itself, as well as on closely related subjects such as creativity,

innovation, and various processes for analyzing situa- tions and making decisions. The information in this paper is based on these key points which have been

condensed into an integrated and relatively simple

approach to breakthrough, with particular attention

to its application in metrology work.

This paper has several purposes, as follows:

(a) To explain why breakthrough techniques are

desirable and feasible for more extensive use in metrology work.

(b) To give information about how breakthrough

can be achieved and to provide leads to sources of information about more detailed aspects of breakthrough.

(c) To suggest some metrology areas which need

breakthroughs and also some possible ways of attain- ing such breakthroughs.

2. Characteristics of Metrology Problems

Requiring Breakthrough

Breakthrough techniques, coupled with success- oriented attitudes and a systematic method of estab- lishing objectives and evaluating alternatives, provides a powerful set of new tools for solving improvement type problems. Today there are increasing

pressures to come up with faster and better solutions

to such problems. These pressures arise from the following characteristics which many metrology problems have:

• Metrology work involves the conflicting and di- ametrically opposed problems of control versus breakthrough. Controls are vital to assure accuracy and reliability of metrology work. However, breakthroughs are necessary to provide improved accuracies, ranges, and types of measurements. The dichot- omy of this situation comes from the differences in

attitudes of the people involved. Ideally, control and

breakthrough should be carried out by two different

sets of people because of these differences.

85
• Metrology problems are complex; they have

many variables.

• Metrology problems are generic in the sense that they apply in about the same way to many different organizations; i.e., they cross organizational lines and

therefore appear not to be the responsibility of any

one organization.

• Adequate attention and funding are difficult to justify because the problems are principally of a support nature. This means that they are far from

the basic problem sources—i.e., the exotic technological requirements of space work, military work, etc.

3. Breakthrough Versus Control

One of the better explanations of breakthrough is that given in Dr. J. M. Juran's Book, Managerial

Breakthrough [l]. 1 This explanation is particularly

good because breakthrough is described by comparing

it with its opposite function—control. For example,

Dr. Juran makes the statement that "All managerial

activity is directed at either breakthrough or control. Managers are busy doing both of these things and

nothing else." Dr. Juran also describes breakthrough

as the creation of good or necessary changes, while

control involves the prevention of undesirable changes.

Obviously, both types of activities are necessary in an

organization, although the type and amount of each

will vary from time to time and from one organization to another. The differences in the results expected from control and from breakthrough cause radical differences in the methods of obtaining each of them. This is particularly true in the use of people, for either break- through work or for control work. The selection

I -9 RESTLESS 9 |_ -f INNOVATOR

< 6 Q

9-9 ADAPTIVE

PLANNER _j_

1-5 FIREFIGHTER +

5-5 CONSERVATIVE

-f- PLANNER

I - I COMPLACENT

FATALIST

9 - I INHIBITOR +

I CONTROL

23456789 ORIENTED ATTITUDES

Figure 1. Coordinate analysis of attitudes toward

control versus improvement.

1 Figures in brackets indicate the literature references at the end

of this paper.

of people for either breakthrough or control work

should take into account their education, their work

experience, their motivation, and their attitudes. However, people are rarely 100 percent control- oriented. Instead, there are many possible combinations, as are shown in the coordinate analysis chart

of figure 1.

4. Breakthrough Concepts

People are basically goal-oriented; that is, they

continually set goals for themselves, work to reach

such goals, and, when each goal is reached, select a new goal to work towards. Goals for achieving control are usually given highest priority, particularly

for the solution of crisis or "firefighting" types of problems. However, when there is is additional time

available, it is usually spent on improvement-type

goals. Breakthrough, in its most general sense, is the technique used to attain improvement-type goals. In

a more restricted sense, breakthrough is the technique

used to attain improvement goals in a relatively short period of time—short, that is, with respect to the time

such improvement changes might normally be expected

to take on a strictly evolutionary basis. Some of the important characteristics of break- through are as follows:

• A systematic approach is usually the best for achieving success in breakthrough work. The syste- matic approach increases the probability of reaching

the breakthrough objectives and cuts the time require- ments compared with a nonsystematic approach.

• For breakthrough to get started, there must first be a promoter or advocate—i.e., someone who feels that improvement is not only necessary but that it also is feasible.

• Breakthrough requires the following types of leadership qualities: —Constructive discontent with the status quo —Willingness to take risks —Good sense of timing —Aggressiveness —Decisiveness —Confidence.

• The amount and kinds of breakthrough which

can be achieved are directly related to the favorability

of the climate in the organization. A poor organizational climate will mean that breakthrough is likely to be either impossible or at least stunted and mediocre. —The most important climate factor in an or- ganization is the attitudes towards breakthrough of top management and of the immediate boss of each

person involved in breakthrough.

—The second most important climate factor in an organization is the attitudes towards breakthrough

of a person's peers as well as the attitudes of the people working for him on any aspect of a break- through task.

5. Steps for Achieving Breakthrough

The nature of breakthrough-type problem solving

is shown in the block diagram of figure 2. Reading

horizontally across the top of the diagram, we find a

86
RECOGNIZE DEVISE IMPLEMENT FOLLOW-UP

PROBLEM —

SOLUTION —

SOLUTION

- > AND CONTROL

RESULTS

1

CLARIFY

OBJECTIVES —

GENERATE ALTERNATIVE POSSIBLE SOLUTIONS —

ANALYZE CONSTRAINTS —

ADVERSE CONSEQUENCES

RESOURCE LIMITATIONS PEOPLE- GENERATED STALLS, OBJECTIONS, ROADBLOCKS —

MAKE DECISION BASED ON COMPARISON

OF ALTERNATIVES WITH CONSTRAINTS —

ANALYZE

POTENTIAL PROBLEMS

Figure 2. Steps in breakthrough-type problem solving.

simplified process for solving problems in the four

steps—recognize problem, devise solution, implement

solution, and follow up and control to get the desired

results. However, under the "devise solution" step the diagram also shows an additional series of steps

i.e., clarify objectives; generate alternative possible

solutions; analyze constraints in terms of adverse con- sequences, resource limitations, and people-generated

stalls, objections, and roadblocks; make decision

based on comparison of alternatives with constraints; and analyze potential problems. Ideally, all that should

be necessary to do is go through the steps shown

here and you will automatically be able to achieve

breakthroughs in any problem area. However, unless care is taken the results will be far from ideal—usually

because of (a) inadequate attention to the peopleproblems involved in achieving breakthrough, or (b)

the difficulties imposed by state-of-the-art technology

limitations. The approaches discussed here should

be able to help avoid or overcome these limitations and obstacles to a considerable extent. The basic approach to obtain breakthrough should

involve the following key elements: —Using a systematic approach based on the steps

shown in the block diagram of figure 2. —Paying attention to more than just getting the basic ideas on how to achieve the desired type of breakthrough. A "total" or systems approach must

be used; this involves solving all of the subproblems

necessary to convert the basic idea into a realistic service, method, procedure, or a working piece of hardware or software. —Setting objectives or goals. To help these goals come true, they must be broken into sub-goals, with

plans for achieving each of the sub-goals. The goals

and sub-goals must be put in writing in order to clar- ify them and to make them understandable to people

who must work to achieve them. These same people

must identify with the goals by seeing the relationship

of the breakthrough goals and sub-goals with their own personal goals and by using visualization tech- niques to keep the goals and sub-goals clearly in mind

while working towards them. The concepts of selec- tive perception and positive expectation come into play

when goals are treated in this way. Selective perception helps people to become more sensitive to anything which will aid them in achieving goals which

they understand and have identified with [2]

—Generating breakthrough-oriented attitudes in the people assigned to work on the breakthrough project.

This includes inducing in each person a positive mental attitude and a self-image which is directed towards

successful completion of the breakthrough project. —Use of various creativity and innovation tech- niques to achieve a broad spectrum of alternative means of solving the main breakthrough problem and

its subproblems. —Clarifying the objectives through the use of tech- niques such as situation analysis, critical factor analysis, and function analysis. This helps avoid the effects of finding the right solution for the wrong objectives. —Including in the decision-making criteria for a particular breakthrough not only the objectives to be

achieved but also the difficulties likely to be encountered and ways of preventing or minimizing such

difficulties. This requires consideration of possible

undesirable side effects, limitations imposed by re- sources reasonably available, and people-generated

stalls, objections, and roadblocks. —Learning how to cope with breakthrough-type

tasks by starting with a relatively small task on a sort

of trial-run basis. Success in accomplishment of this task should be followed by progressively larger tasks, to increase the skill in handling breakthrough-type

work. This gradual approach also helps to build confidence—a vital factor for attempting any type of breakthrough work [3]

—Screening the alternatives by systematically checking them against the objectives, undesirable side ef- fects, resources and people-generated stalls, objections,

and roadblocks. In addition, objectives should be

categorized as either "musts" or "wants" [4]

—Recognizing the importance of motivation in get- ting people to work effectively on the breakthrough

project. Motivation can be developed and improved,

and in this way much of the dormant creative potential

and other problem-solving skills of people can be re- leased and effectively focused on the main problems

and subproblems required to achieve the desired

breakthrough [2]

87
6. Possibilities for Application of

Breakthrough Techniques to

Metrology Problems

Several areas which suggest themselves as good possibilities for application of the breakthrough tech- niques discussed in this paper are as follows:

• The calibration recall problem.

• The problem of traceability of calibration to NBS.

• Improved utilization of measuring equipment and

standards.

• How to obtain a compatible system of measure- ments and tests throughout the United States.

7. References

[1] Juran, J. M., Managerial Breakthrough (McGraw-Hill, New

York, N.Y., 1964)

[2] Williams, J. Clifton, Dynamics of Motivational Management

(Success Motivation Institute, Waco, Tex., 1969)

[3] Bobbe, Richard A., and Schafer, Robert H., Mastering

change: Breakthrough projects and beyond, Management

Bull. No. 120 (Am. Management Assn., New York, N.Y., 1968).

[4] Kepner, Charles H.. and Tregoe, Benjamin B., The Rational Manager (McGraw-Hill, New York, N.Y., 1965).
NCSL 70

THE NEW STATE STANDARDS PROGRAM

Robert Mills

Office of Weights and Measures, National Bureau of Standards, Washington, D.C. 20234

In 1965, Congress appropriated funds for new state standards and instruments in mass, length, and

volume. The laboratories receiving this package are, in most cases, under the administration of state

offices of weights and measures. These labs not only check the accuracy of field standards used by

weights and measures inspectors, but also serve as local measurement centers to perform industrial

tests. The equipment and training given to laboratory personnel are described in this paper, as well

as some of the administrative details and the utilization we foresee for the state laboratories.

Key words: Length measurements; mass measurements; metrology; standards; state laboratories; volume measurements.

1. Introduction

In 1965 the United States Congress approved a two

million dollar program to provide states with new

physical standards and measuring instruments in mass, length, and volume. The National Bureau of Standards was charged with the responsibility of

assisting the states in the development of state lab- oratories, design and procurement of standards and

precision balances, and final installation. Today 40

states are participating in the State Standards

Program.

These newly equipped metrology laboratories en- able the states to assume their important role in our

national measurement system during a time when sci- ence and technology are advancing at an accelerating

rate. It is important that the membership of the National Conference of Standards Laboratories be

informed about the New State Standards Program.

Thus, it is my pleasure to discuss with you the pro- gram's origin, some of the administrative details, and

the utilization we foresee for the state laboratories. Men have endeavored to establish effective stand- ards of measurements for many centuries. Edward II proclaimed in 1324 that the inch was equal to the length of three barley corns, taken from the center of an ear, placed end to end. In the 16th century, the rod was the length of the left feet of 16 men lined up

as they left church on Sunday morning. Perhaps

these standards sufficed in their time, but it is clear

that they would not fulfill today's need for standardization. In our country, there has long been a parallel be- tween the existence of accurate standards and the es- tablishment of effective weights and measures activities. The Constitution of the United States empowered

Congress in Article 1, Section 8 to "fix the standards

of weights and measures." In 1821, John Quincy

Adams recommended that Congress act to bring about

uniformity in weights and measures. The Senate

passed a resolution in 1830 directing the Department

of the Treasury to compare the standards of weights

and measures in use at the principal customhouses.

Large discrepancies were found to exist. As a result, Congress directed the Treasury to fabricate standards

of weights and measures for the customhouses and

States, thereby authorizing the distribution of the first State standards. This resolution also established the

Office of Weights and Measures. Two additional dis- tributions of standards to the States were authorized

by Congress in 1866 and 1881. By the beginning of the 20th century, the Industrial Revolution had produced a growing awareness of the need for uniform, nationwide measurement techniques

and measurement standards in the United States. The

Office of Weights and Measures was transferred to the National Bureau of Standards in 1901, the year

the Bureau was established. Traditionally, regulatory authority in weights and

measures rests with the States rather than with the Federal Government. There are 50 state weights and

measures jurisdictions, and in some cases local authorities within the States, separately exercising their authority to check supermarket scales, gasoline pumps,

packages in the marketplace, taximeters, fabric and

cordage measuring devices, farm milk tanks, truck

scales, fuel oil meters—virtually every weighing and

measuring device used commercially. Yet, despite

this diversification of responsibility, measurement

accuracy and uniformity must be maintained to protect both the buyer and seller, and to provide for orderly commerce on a national basis. Uniformity

in specifications, tolerances, and test procedures is achieved in most states by accepting the recommendations [1]

1 of the National Conference on Weights

1 Figures in brackets refer to the literature references at the end

of this paper.

89
and Meaures, another conference sponsored by NBS.

Accurate field test standards must also be used by

weights and measures inspectors. This requirement

implies the need for accurate laboratory standards. By 1965, the laboratory standards of many States were

no longer adequate.

2. The Program

Congress responded to the need for State reference standards by establishing the State Standards Pro- gram. The National Bureau of Standards was given

the responsibility of acquiring the new standards and

instruments, training State laboratory personnel, and

assisting the States to provide adequate laboratory

facilities. Primary responsibility for the project lies with the NBS Office of Weights and Measures, which

offers assistance and technical support to local and

State weights and measures agencies. The NMS Metrology Division performs the calibrations of the new

standards and provides technical counsel when

needed.

States initiate the process of qualifying for receipt

of the State Standards "package" by demonstrating

the need, and by making a commitment to provide

suitable laboratory space and personnel. NBS then

discusses details of requirements for laboratory facili- ties with State architects or engineers, and provides

additional technical assistance as construction proceeds. The construction may be modifications to existing facilities or the erection of a completely new

building. Construction costs to the states have ranged

from a few thousand dollars up to about .$135,000. A

minimum of 1,000 square feet is needed for a suitable laboratory facility.

The new standards have been furnished at the rate of approximately 10 states per year. To date, for- mal presentation ceremonies, usually attended by the NBS Director and by the Governor, have been held

in 23 states. Table 1 lists the states in each group

and the dates of the formal presentation ceremonies

which have been held. The grouping of the states was determined by their needs, and the date on which

they provided the required laboratory facilities and

personnel.

The Office of Weights and Measures of NBS provides training for laboratory personnel at three levels: basic, intermediate, and advanced. The two weeks

of basic training are completed before the official pres- entation of standards. The first week is conducted

at the NBS Office of Weights and Measures laboratory, or, to accommodate distant States, at established

State laboratories, usually just prior to the presentation ceremony and concurrently with the final installa- tion of the new laboratory equipment. Metrologists

who are replacements for previously trained personnel

receive their basic training at NBS.

Intermediate seminars are intended for State metrologists who have completed basic training. These

seminars are usually 40 hours long and can be held

either at NBS or in regional seminars. They begin

by reviewing basic subjects before continuing to more

advanced material. The advanced seminars are for

Table 1. Progress in presentation of new standards

to the 50 States

Presentation Presentation Group I Date Group II Date

- Crfciliiornia. 1 1 / 99 /A7 11/ ZZ/ 0 1 AArkansas a 1 1 o / /en 4/ io/oy

Connecticut 3/ A/fift 0/ <*/ Do Florida Delswfire 1 / zAjf Uu Georgia 1 A / Ol / AO iU' zi/ oy

Illinois f\o /Q/fn / y/ o i Hawaii Q/ 9 /AO o/ Z/ Oo Kentucky 7/19/68 Maine 5/21/69

New Mexico 12/ 8/67

Ohio 6/ 8/67 North Carolina 2/25/69

Oregon 10/27/67 Pennsylvania 3 /3/69

Tennessee o/ z/ uo ^^est Virginia c;/ iQ/7fl Utah in/ v>/fi7 \X/ ic/>nncin 0/3/ 7/6.Q i / \jy

Presentation Presentation Group III Date Group IV Date Alabama AAlaska

Idaho £ / A /7fi Colorado

Indiana Massachusetts Maryland A / A.ll(\ 41 / Hel /U Michigan

New Jersey Minnesota

North Dakota Nevada

Oklahoma New York

Texas 6/ 4/70 South Carolina Vermont South Dakota Wyoming 4/13/70 Virginia

Remaining States and Territories Arizona Iowa

Kansas

Louisiana

Mississippi Montana

Nebraska New Hampshire

Rhode Island Washington

District of Columbia

Puerto Rico

Virgin Islands

more experienced metrologists. They are held at NBS

and provide a broad exposure to the Bureau's professional staff. Some thought has been given to scheduling advanced seminars the week just before or just

after future NCSL dates so that State metrologists can attend this Conference.

Our intermediate and advanced training programs

are continuous in that metrologists are encouraged to attend a seminar once every year. Participants are

carefully selected for each seminar so that experience

and level of achievement are compatible. Seminars

at all three levels have had, on the average, four or

five participants. We expect to continue our present

schedule of from 10 to 12 seminars a year.

3. State Standards

Table 2 lists the standards and instruments in the

State Standards "package." Note that both metric

and avoirdupois, or U.S. customary standards, are

included. The mass standards in item (1) and (2)

90
comprise a 5, 3, 2, 1 series (e.g., 5 g, 3 g, 2 g, 1 g, 0.5 g, etc) with duplicate 1 kg, 1 mg, 1 lb, and 1 yJb

weights. The small flat weights (i.e., smaller than

1 g and 0.002 lb ) are made of Nichrome or alumi- num. Except for the 500 lb standards, which are 303

stainless steel, the larger weights are made of a stainless steel, developed especially for precision mass

standards, having a density of 8.0 g/cm3

. The syste- matic uncertainties in the reports of calibration range from 30 mg for the 30 kg weight to 2.9 [xg for the

1 mg weight. The five precision balances are the best present technology offers. The precision given in ta- ble 2 are manufacturers' tolerances and are included

to indicate the capabilities of these instruments. Except for the 5,000 lb equal-arm Russell balance, all of these instruments are single-pan, substitution balances. Prior to the beginning of this program, the standards

and balances were developed through the cooperative

efforts of NBS and the manufacturers. Reference

[2] discusses the development of the 30 kg balance.
Figure 3. The 30 kg and Russell balance.

Table 2. State Standards

1. METRIC MASS STANDARDS—30 kg to 1 mg, fig. 1

2. AVOIRDUPOIS MASS STANDARDS—50 lb to 1 n\b

3. 500 LB MASS STANDARDS (2)

4. PRECISION BALANCES, figs. 2 and 3: 160 g capacity—0.02 mg precision

1 kg capacity—0.2 mg precision 3 kg capacity—1 mg precision 30 kg capacity—2 mg precision 5000 lb capacity—0.01 mg precision

5. LENGTH BENCH—5 meter/ 16 feet

6. TENSION WEIGHTS—20 lb

7. LABORATORY MICROSCOPE—0.300 inch X 0.002 inch

8. PRECISION STEEL TAPE—7 meter/25 feet

9. STEEL TAPE—30 meter/ 100 feet

10. PRECISION STEEL RULE—18 inches X 0.01 inch

11. METRIC PIPET-BURET ASSEMBLY—5 liters to 10 milliliters

12. U.S. CUSTOMARY PIPET-BURET ASSEMBLY—1 gallon

to 120 minims

13. FIVE GALLON STANDARD—slicker plate type

Items (8) and (10) are the state primary length standards. The report of calibration for the 25 ft/7m

tape gives an uncertainty of 0.002 in. and 0.05 mm for any interval. Uncertainty of the 18 in. scale is re- ported as 0.001 in. For the volumetric standards, the reported uncertainty is 0.001 gal for the 5-gal slicker plate measure. 0.1 minim for the 120-minim buret, and

0.005 ml for the l()-ml buret.

4. Utilization of Laboratories

The primary role of a State measurement center is to serve as a reference laboratory for weights and

measures field standards. In almost every case, the

State standards equipment is received by the State weights and measures department. State laboratories

also check the standards used by other Stale agencies

and officials, such as the state chemist; veterinarian;

dairy, feed, and fertilizer inspector; public health in- spector; highway department; and petroleum laboratory. From the outset there was no intention to restrict activities of the State Standards Laboratories to weights and measures activities, or to other state reg- ulatory agencies. Rather, the policy has been to en- courage states to serve within their capability the measurement needs of industry and commerce as well as educational and research institutions. There was evidence of an industrial demand at State metrology laboratories even before the initiation of the State Standards Program. Massachusetts, for one, has

had an active program of providing calibrations for

their industries. They are just now beginning to re- ceive their new standards, and anticipate an increase

in both the quality and quantity of the measurement

services provided to industry. The demand for in- dustrial calibrations at the State laboratories has been

made apparent by the activities of some of the labs which have received their new standards. Arkansas is a good case in point. Table 3 indicates the scope of

the industrial tests conducted there in addition to tests of their own field standards and tests for other State agencies. Admittedly, the development of the Arkan92
sas laboratory is exceptional. Few industrial tests were

I performed before the new Arkansas standards were

| presented in April of 1969; the table covers the period

i up to March 1970. Of course, Arkansas is not one of our most industrialized States. Several other States having a more industrialized economy, such as Cali- fornia, Illinois, or Pennsylvania, perform even more

industrial tests. Most States charge fees for the industrial tests they

perform. Thus, costs are paid by industries which

have the need for tests with standards traceable to NBS. Of course, the States pay the costs for testing

their own field standards.

Table 3. Arkansas weights and measures laboratory

industrial customers

1. Electronic Weights and weight kits from 2 kg to 1 g Companies (Class S) and 50 lb to 1/32 oz (Class F)

2. Petroleum Metal volumetric test measures from 1 gal Equipment to 500 gal. All tests were certified to 1 Companies part in 2000.

3. Pipe Line Metal test provers; two 600 gal and one Companies 1,015 gal. Accuracy 1 part in 2000.

4. Bottling Testing of graduated neck flasks used to Companies spot check fill machines.

5. Scale Weights and weight kits from 1/32 oz to Companies 1.000 lb and 1 g to 25 kg. Class S, F, and T tolerances were required

6. Pharmaceutical Weights and weight kits from 1 mg to 20

Laboratories kg. Class S tolerances.

7. Electrical Various weights, Class T tolerance. Companies

8. Oil Refining Calibration of tank trucks: Accuracy 1 part Companies in 2000.

9. Manufacturing Weight kits from 1 mg to 5 kg: Class S Companies tolerance was requested.

10. Milling Weights from 4 oz to 5 lb; Class F toler- Companies ance were requested.

11. Transport Calibration of tank trucks: Accuracy 1 part Companies in 2000. Note: The Arkansas Laboratory has seven large-capacity provers having nominal capacities between 29 and 1,000 gallons.

In the training program, State metrologists are en- couraged to consult with NBS about tests that are other than routine. Our office has a policy of being as helpful as possible in such situations. Jobs which are be- yond the capabilities of particular State laboratories

are referred to NBS. Clearly it is important to know

the capabilities of each State laboratory. This sur- veillance is accomplished, at present, through the Laboratory Auditing Program (LAP), which provides for

tests to monitor the performance of the instruments,

personnel, and environment. Table 4 indicates the nature of LAP exercises by listing the titles of the first ten problems. Problems 4 and 9 involve the exchange

of test kits between our laboratory and the States. Data from all LAP exercises are sent to us for review

and comments. Certificates are awarded annually to

States which have met minimum requirements of participation.

Table 4. Laboratory auditing program problems

1 Sensitivity test 2 Tolerance test (direct-reading) 3 Precision test of 1-kg balance 4 Tolerance test and weight calibration 5 100-g balance precision 6 3-kg balance precision

7 30-kg palance precision 8 Calibration of length bench

9 Testing of a steel tape 10 Precision of Russell balance

5. The Future

The surveillance effort will have to be increased as the measurements performed by the State laboratories

become more sophisticated. States should record a continuous flow of surveillance data so that control

charts can be maintained. The more advanced State metrologists are presently doing this. They receive an analysis of their control chart data from the Office of Weights and Measures. In this way, the capability

of their laboratory can be determined realistically. Our

intention is to make this a more universal practice in the State laboratories.

Insufficient surveillance data will result in a large

uncertainty as to a State laboratory's performance ca- pability. The proper surveillance effort will be deter- mined by the calibration demands on that laboratory.

Time required to accumulate surveillance data will probably range from one to ten man-hours per week

(not necessarily time spent by the metrologists; data can be taken and compiled by an aide)

There has been an increasing number of inquiries

recently concerning the possible extension of State laboratories into fields of measurement other than mass,

length, and volume. The greatest interest is in temperature and gage-block measurements. The Massachusetts laboratory has been in the business of testing

thermometers for some time. California is testing

electric meters in addition to other measurements. We

at the National Bureau of Standards are preparing to advise the States, upon request, concerning test equipment, test procedures, and test uncertainties in fields where the need can be demonstrated.

Getting back to the State Standards Program, the future offers a triple challenge to the states, to the National Bureau of Standards, and to potential industrial users if full utiliztion of these State measurement centers is to be realized. The States must, of course pro- vide adequate financial and administrative support for

their laboratories. They must staff their laboratories

with high-caliber personnel. The National Bureau of Standards must maintain a vigorous support program

in training and consulting services. Potential industrial customers must be informed about their State laboratory and make available information on their

03
future measurement requirements. Industrial interest and support can be helpful in encouraging the states

to properly fulfill many of their measurement require- ments. The NBS Office of Weights and Measures in- vites inquiries from any interested parties concerning

the current status of the laboratory program in their particular state. Comments and suggestions are, of course, welcome.

6. References

[1] Nat. Bur. Stand. (U.S.), Handb. 44, 3d ed., Specifications, Tolerances, and Other Technical Requirements for Commercial Weighing and Measuring Devices.

[2] Aimer, H. E , Bowman, H. A., Jensen, M. W., Macurdy, L.

B., Peiser, H. S., and Wasko, B.. A direct-reading twoknife 50-pound balance of high precision suitable for State weights and measures laboratories, J. Res. Nat. Bur.

Stand. (U.S.), 68C (Eng. and Instr.), 141 (1964).

0)
NCSL 70

THE IMPORTANCE OF VISIBILITY AND CONTROL

IN LABORATORY MANAGEMENT SYSTEMS

John L. King

Standards and Calibration Laboratories, General Dynamics/Convair Division, San Diego, Calif. 92112

The Test Equipment Control Engineering Section has overall responsibility for 2719 different models of 40,000 items, from purchase to final disposition. Computerized system shows decision- makers the history of costs, locations, and interface with operations; yields savings of 1800,000 per

year.

Key word:: Data for decision; standards laboratory management; test equipment records; use, storage, or surplus.

In the present economic environment, the application of new and improved methods of visibility and

control in the standards and calibration laboratory be- comes a high-priority goal. The methods of visibility

and control we are going to explore have evolved over

a period of years and are presently operational in the Standards and Calibration Laboratories at the General

Dynamics Convair Division. They were developed

during the best possible environment for designing a

control system; namely, a continuing decrease in the departmental budget, an increase in responsibility, and

a demand for continued product quality. Such conditions certainly cause one to look around for new controls to maintain quality and optimize cost! Understandably, the methods developed are tailored

to Convair operations. Convair calibration laboratories may or may not be similar to those in other companies, but one thing is certain . . . many of the problems and the customer requirements will be very

familiar to everyone. Before we discuss these methods, it will be necessary

to acquaint you with Convair's basic philosophy of op- eration and organization of calibration activities. The

Standards and Calibration Laboratories are a department of Reliability Control. They are further divided

into three sections; Electronic Metrology, Mechanical

Metrology, and Test Equipment Control Engineering.

The Electronic and Mechanical Metrology sections in- clude the calibration laboratories. These laboratories are further segregated into 10 major laboratory areas and 21 subgroups identified by the category of equipment they service. Each laboratory is operated under

the technical direction of a graduate engineer, with

highly skilled technicians doing the actual calibration and maintenance. The Engineer-in-Charge of each

laboratory approves all work done. He certifies the work by means of a Reliability Control Department

metrology stamp assigned to him. There are no regular inspection department or other quality control personnel assigned to the standards and calibration

laboratories. The laboratories under the Electronic and Mechanical Metrology sections service approximately 40,000

items of test and measuring squipment, of which ap- proximately 30,000 items are in continuous use in Convair operations.

The Test Equipment Control Engineering section is the group most pertinent to this discussion, since it plays a dynamic role in the control system. Consequently, though each section is involved in control to some extent, we will examine the responsibilities and

operations of the Test Equipment Control Engineering

section in detail. The primary responsibilities of the Test Equipment Control Engineering section are to:

1. Develop and provide policy procedures for the

entire Standards and Calibration Laboratories Depart- ment.

2. Respond to Convair Division requirements for general-purpose measuring and test equipment, whether

for new applications or replacement purposes. This

response includes "cradle to the grave" activities, such

as acquisition, control, storage, and eventual final dis- position of the equipment.

3. Provide a central record control system for all work entering or leaving the Department.

4. Maintain all calibration history and traceability

records for each item of equipment and provide cali- bration recall schedules.

5. Establish all calibration intervals and initiate or approve any status change in measuring or test equip- ment.

6. Maintain general-purpose measuring and test equipment issuance stockrooms at practical and appropriate locations throughout the Division.

7. Provide engineering technical support for the so- lution of Division measurement problems.

05
To fulfill these assignments, the Test Equipment Control Engineering section is composed of:

1. A part-time staff technical and management policy

writer.

2. A records and data group.

3. A test equipment stockroom operations group.

4. A test equipment applications engineering group.

All applications engineers are graduate engineers with

laboratory experience.

Now. let's discuss visibility and control problems

and solutions. The question may be raised as to why

the "cradle to the grave" responsibility for general- purpose measuring and test equipment is vested in a calibration laboratory operation. Why? Because

this concept permits overall visibility and control lead- ing directly to continuous cost optimization opportunities for the company. Let's investigate the truth of

this statement.

If a company allows any individual user or designer

to specify his test equipment preferences by manufacturer and model, a high-cost chain reaction is set in motion. Start with the unit itself. It may not meet

the task requirements or even its own published specifi- cations. Get your money back? Hardly. The cost of

the unit is probably a fraction of the cost of maintaining the production schedules involved in our businesses. Next, consider spares. Spares must be pur- chased for all the different models that are specified

and subsequently purchased. Then calibration proce- dures must be obtained for all different models. Cali- bration equipment must be obtained for all different models. By now, our chain reaction is well under way.

Calibrating and operating technicians must be trained on the different models. There is also the possibility that once the unit has served its original purpose it may

be useless for other applications. These considerations

are seldom considered by the individual designer or

test equipment user, simply because he lacks an overall view. . What is necessary for a group to provide a rational,

standardized test equipment program for a large com- pany? It must have complete responsibility and control over the entire program. It must have expert tech- nical knowledge of all measuring and test equipment

used in the company and of the costs of maintaining

such equipment. It must have intimate knowledge of vendor reputations in the various fields, gained from

actual instrument comparisons. It must be able to communicate on the technical level of the engineer.

Where can you find a group with these capabilities?

If your calibration laboratories are manned with graduate engineers, each one a technical expert on a particular category of equipment, which he also maintains

for the company . . . then you certainly have the base

for such a group. As an example of what can be accomplished, the

test equipment applications group developed computer lab-run listings of all test equipment in the Convair

Division. Results showed 7.326 different models in the 40,000 items of equipment. By working closely

with users to determine application requirements, and

with calibration laboratory engineers regarding specifications and choice of equipment, it was determined

that only 2,179 different models were actually required. : And this was only the first look! Some of the 5,147

unwanted models were replaced immediately by pre- ferred models that were in storage status. The remaining unwanted models were marked for surplus on a systematic basis. This was accomplished by establish- ing a zero retention level for the unwanted units on

the storage control lists. Consequently, if a user does

not have an immediate need for a unit, it is declared

surplus rather than being stored for backup purposes. On the subject of test equipment disposition, how

many of us have pondered over the question of whether to dispose, or not to dispose, of test equipment?

More to the point is the pertinent question: Who makes

the decision? Certainly the decision to use, store, or dispose of a unit of test equipment will either save, or

cost, the company money. There is no middle ground.

It will be one or the other. Without a capable control group, the decision is usually made by those least qualified to make it, usually a test equipment stock- room operation. Furthermore, their decision is usually

to store the unit, since this is a pretty safe decision. In fact, the buck can get passed pretty far up the management ladder before someone will commit himself on this apparently simple decision. Why does this hold true? Because no company-wide visibility on the matter is available to the decision maker; therefore, no

decision rules have been formulated.

About three years ago the Test Equipment Control

Engineering section was confronted with this problem

by one of our customers. The customer pointed out

to management various contractual clauses which re- quired customer-owned equipment to be returned within a certain period of time after it had been declared

excess. Moreover, he claimed we were keeping his equipment in storage for years as potential backup

equipment. Unfortunately, the allegations proved true. A fast check showed that not only was the storage in- ventory huge, it was snowballing rapidly. The inventory included both customer and company equipments

which a cursory investigation showed were worthless

for future requirements. Economically, the inventory was almost a disaster. The lesson was driven deeper

when other equipment was located in storage that was

superior to equipment in use. Today, the decision to use, store, or declare equipment surplus is made within one to two days after the user decides the unit is excess to his needs. The de- cision is made as a matter of routine, with complete

confidence that it is correct. An engineer from the

test equipment applications group uses managementdesigned decision rules, along with engineering judgment regarding the condition of the instrument, to ar- rive at his decision within a few minutes after the question arises. Visibility makes this possible: visibility of all test equipment in the company and where it is, visibility of present and known future needs of all users in the company. Visibility of preferred test equipment. Visibility of the desired number of like units for backup

capability, the current headcount of like units in stor- age, and the precise condition of each one. Of course, we are talking about a computerized test

96
equipment control system with inputs of all parameters

of test equipment cost, control, and life-history on

every interface of the equipment with operations. OutI puts are obtained in the form of tab runs and listings, in the formats required for intelligent decision-rule formulation or equipment control. These outputs are then placed in the hands of all those requiring the in- formation for decision making or equipment control. This point may seem trite, but all too often information is available within a company but its existence

is unknown to the poor soul who really needs it. Some of the management tools provided by the computerized test equipment control system are:

1. Purchase cost of the unit.

2. Maintenance cost for the unit.

3. Chronological history of the unit, including numbers and dates of calibration, numbers and dates of

failures, types of failures, and whether or not the

failure was detected by the user.

4. User and location of equipment.

5. Equipment stored and whether operational or needing repair. Also the maximum number of each

type to be held in storage.

6. Calibration and maintenance time on all equipment serviced by the individual technician.

7. Preferred test equipment for standardization.

8. Backlogs of individual laboratories.

9. Future workload, both total and individual laboratories. Inputs regarding user requirements, immediate and

future, are also provided to the section by means of

test equipment requests.

Management use of these tools in their various possible combinations and forms will:

1. Provide the Standards and Calibration Laboratory management, higher management, and the user with actual cost information, including test equipment

calibration and maintenance costs for each user in the company.

2. Provide calibration recall listings for all users and

the various calibration laboratories.

3. Determine the types of equipment needed for pur- chase both as new requirements and replacements.

4. Determine department and individual laboratory

efficiencies.

5. Determine training requirements for department

engineers and technicians and possible user training

requirements.

6. Provide decision rules for equipment disposition.

7. Determine total and individual laboratory manpower requirements, both present and future.

8. Determine calibration intervals from statistical and user information.

A few comments on calibration intervals seem in order here, not from the viewpoint of the methods to use to determine correct intervals but for the application of interval control. Often failure rates are used

to determine calibration intervals which will then sat- isfy a specified quality level. Of course, this process

is reversible and the calibration interval can be used

to predict the failure rate. If a high failure rate on

certain test equipment does not affect the quality of the end product, and if such high failure rate would

optimize the total company cost (i.e., user cost due to the high failure rate versus calibration costs at shorter

intervals), then by all means such equipment should

be calibrated at intervals allowing the higher failure

rate. Government specification MIL-C-45662A contains the requirements for control of a calibration sys- tem for Government contractors. This military specification states: "All measuring and test equipment ap- plicable to the contract shall be subject to such control

as is necessary to assure conformance of supplies and

services to contractual requirements." Therefore, if the contractor can show complete control of those in- struments which determine conformance of supplies

and services to the satisfaction of the customer, a higher failure rate on other equipment might be ap- propriate for cost optimizing.

In this regard, Convair has a program which segre- gates critical application test equipment (CATE) from

all other test equipment in the company. The CATE

equipment is then placed on intervals conforming to customer requirements for such equipment. The rest of the equipment is placed on intervals to give a failure

rate which management has determined will optimize

company cost. Here again, the segregation of CATE

is made possible by visibility of the equipment and the requirements as they apply to both Convair and the customer. The computer listing which is supplied to management and users, identifying the users' monthly equipment costs, is a powerful tool. Making these costs

visible to the user and to higher management provides

a great incentive for the return of unnecessary test equipment. Infrequently used equipment tends to as- sume storage status if it is not needed at calibration

time. These costs include those items checked out of the test equipment stockrooms on a daily basis. The

listing provides a mechanism to tie calibration budget

to equipment in use. The manpower-forecasting tab runs can show, based

on present intervals, the predicted manpower required

for items due for calibration in any future time period.

The runs also show the manpower required for irreg- ular repair work, based on statistical probabilities compiled from the past history of presently used equipment.

The computerized test equipment control system can

provide calibration and repair targets, based on statis- tical analysis of past performance, to any formula de- sired by management. Summaries of the individual,

laboratory, or department performance against those

targets for any time period can be provided as desired. Thus, it replaces the manual computation and reporting of work measurement.

I have tried to highlight some points of visibility

and control which may be of help in analyzing anothe*

company's system. The basic philosophy is that if one

can obtain total visibility of the system in all its parameters and interfaces, then management can make correct decisions and decision rules. It seems obvious

that in a large company total control of general-pur- pose test equipment should be a function of the Standards and Calibration Laboratory.

For those who seek a "magic formula" I suggest

that very few exist in actual practice. As engineers,

97
we have a great propensity for formulas because we

recognize them in our engineering work as being exact

relationships between different parameters. In management, a strict formula generally means that a compromise has been made in the name of uniformity and

often at high cost. This is not to advocate that management textbook formulas are no good. They are very good for their intent, which is mainly to give in- sight into the relationships of various operations pa- rameters. Such formulas generally presuppose certain necessary assumptions which preclude their being applied in actual practice. Just as an example, consider

the development of the most efficient system of reducing a calibration backlog. According to the textbook

theory on the waiting-line problem, if you always take

those items first which have the shortest calibration

time, then you will have the most efficient system in every respect except one. That one exception is that somebody is going to wait an awfully long time for his particular equipment to get calibrated. Yet, if a rule

is added to the system stating that no unit will wait

longer than a specified time, the efficiency of the original system is reduced by up to 100 percent (in comparison to a first-come, first-served, system ) and it may

no longer be the most efficient system. How these management rules and practices can be modified to give the most efficient operation depends upon each

company's particular circumstances.

Management systems should be as flexible as possible, yet still provide the necessary control. System

feedback should impact the working group as little as

possible, just as we strive to have our measuring in- struments affect the basic parameter being measured

as little as possible. And last, but certainly not least, the system should be designed for people.

I am aware that figures on cost reductions achieved

by one company, even in the form of percentages, may

not mean much to another company. This is understandable since there is no way to compare what each was formerly doing and their relative efficiencies, with

what they are presently doing and their relative effi- ciencies. However, as a matter of possible interest,

I may say that the cost savings of our present system

over the previous operational method are approximately $803,000 per year. This saving began about two

years after we started gathering data for analysis.

Another interesting monetary item is the computer

cost. The charges for computer service for the issu- ance of all work forms and tab runs in their individual

and combined variations comes to $0.33 per unit of

active test equipment per year. This charge was $0.40

until we switched from a card system to a tape system.

Tape also provides much greater flexibility and faster

service.

Is our system optimized at this point? We don't think so. There are several areas in which we believe we can make improvements to our present system, and we are working on them. We can testify that the pres- ent system has been a great improvement over the previous operation and has led us to additional ideas

for future changes.

98
NCSL 70

DATA SYSTEM FOR IMPROVING INSTRUMENT RELIABILITY

Frank J. Dyce

Metrology Laboratory, Martin Marietta Corporation, Orlando, Fla. 32805

This paper discusses the importance of monitoring and control of instrument reliability. It suggests

that a minimum reliability level be set as a goal. It discusses the interval adjustment scheme used at Martin Marietta wherein individual instrument performance determines calibration interval as well

as repair and surplus. A thorough explanation is given of the computerized data system used at Martin Marietta to monitor instrument reliability and to isolate repetitive problem areas. Several examples are given of reliability problems which were solved through use of the system. A unique standardization committee is discussed which justifies the company's sole-source purchase

of general-purpose, reliable electronic test equipment.

The paper concludes that striving towards a reliability goal will result in improved reliability and

reduced cost.

Key words: Calibration data system; calibration interval adjustment method; instrument reliability; instrument standardization.

1. Introduction

Instrument reliability is a subject of much concern

to metrologists today. The 95 percent reliability re- quirement implied by Military Handbook 52 has caused some laboratories to develop elaborate mathematical

formulations and contrived definitions of typical in- strument reliability to meet it. These manipulations were initially undertaken to meet the implied specifi- cation. Now we are coming of age and can better ap- preciate the importance of reliability. We know it is impossible to achieve 100 percent reliability ; however,

there is a definite reason to strive for an optimum level. This optimum level is dependent on the laboratories' resources and capabilities. It should be a published

goal capable of measurement.

Reliable equipment:

1. can make reliable measurements;

2. needs to be calibrated less frequently;

3. requires less repair time;

4. requires fewer replacement parts;

5. has a higher utilization rate. Probably the major reason for having reliable equipment is that most instrument users assume instruments

are absolutely accurate. To protect them and the product, instruments should be made as reliable as feasible. The disadvantages of high reliability are:

1. Calibration intervals might be shorter.

2. Utilization rate might be lower.

3. Calibration time might be longer.

We have made 95 percent reliability our goal and we are getting close to it. We measure reliability in a very basic way: number of instruments received for calibration which are in-tolerance ( with respect to published specification ) versus number of instruments

calibrated. Our laboratory strives for this goal, using two basic

approaches: an interval adjustment scheme based on

individual instrument reliability; and a minimum-input automatic data processing system.

2. Interval Adjustment Scheme

The initial calibration interval is based on the av- erage interval for instruments of the same manufacturing model number or on experience with instru- ments of the same type. Each time an instrument is calibrated its past calibration history accompanies it (fig. 1). The calibration technician examines the his- tory and the condition of the equipment to determine

its next calibration interval as well as to determine

repetitive failures. If the instrument is out of toler- ance, the interval is reduced 20 percent; if it is in tol- erance three consecutive times, the interval is increased

20 percent for each time thereafter. (The intervals vary in the following steps: 30, 40, 50, 60, 70, 90. 110,

130, 160, 190, 225, 270, 320, 365 days). Thus, our major effort is expended on unreliable equipment and

our workload is balanced, since an instrument never

out-of-tolerance is calibrated only once a year. When

two consecutive out -of -tolerances occur, repair is mandatory; adjustment is obviously inadequate. The

faulty components must be replaced and the instrument

checked for stability for two days. When three consecutive out-of-tolerances occur, the instrument is considered for surplus. Figure 2 shows that by using

this system, our lab is now doing fewer actual cali- brations (solid line) on twice as many pieces of equipment (dashed line) as we were eight years ago; we
»CD OO2l0fr-2C3

l-'f'.r HO EOUIPREUT DESCRIPTION KFG. 7U14T GENERATOR PIUS ~_ PARA* IE R$

UPPER LI HI T LOME d U Ml T

CAU BRA TICK RECALL HISTORY RECAL I MAT ION OATE 0-19-2

OLE I CIA RANGE COST bAOGE

9&S 26862 TW HOLLAND

OR TE 04 /2 J/FO

DEPT BEL S/C

056 99 90

*05 85 1 160 REPLACEMENT PARIS

KEPLACENENT PARTS

.0 1.5

-4 MM

M t 1 IWl III* c ;l»

PAR. * 1 fAR. * 1 tlADIM ttAfilM

1 1LLLL inn inn Mill inn 1 1 1 II (Ml milium inn

i ilk i- : 1 III III I III i i il i III 1 III

TOTAL ft TIME .C

TOTAL CAL TINE 5.? TOT* «UT COST 0

Figure 1. Calibration history and transmittal.

EQUIPMENT 9000

INVENTORY

20 000 T0TAL YEARLY CALIBRATIONS

1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 CALENDAR YEAR Figure 2. Annual calibrations (solid line) versus equipment inventory (clashed line).

have improved reliability and lowered costs simultaneously. Our current reliability. 90.3 percent, is 9 per- cent higher than our total past reliability and is steadily increasing.

3. Automatic Data Processing System

A calibration data system is required to indicate

the past condition of the equipment, to determine the need for interval adjustments, and to meet inventory

and Mil Spec requirements. Our data system was de- signed to collect only data that we would actually use. The input data transmittal (fig. 1) is designed for minimum data handling and minimum chance for er- ror. For an in-tolerance instrument, the technician

enters the following data: equipment identification number, condition as received (I in tolerance, 0 out

of tolerance), calibration date, recalibration due date, calibration interval, stamp number, and calibration hours. For an out-of-tolerance instrument, he adds the repair hours, replacement-part cost, and coded defect. The coded defect is the only complicated part of the system. Each technician has a mnemonic code book

with codes listed for the family of instruments he re- pairs. The technician codes each system failure; What

was wrong, how it was wrong, what part caused the

failure, and what was done to the part. For example,

if a signal generator had low output and vacuum tube

V6 was replaced to correct it, it would be coded

0LV6R; which reads Output (0) Low (L) Vacuum

Tube (V) Circuit Symbol 6 Replaced (R). The dis- advantage that the coding complicates the data trans- mittal is offset by the data's availability for later computer analysis.

We have 40 families of instruments for which we

record discrete parameter data. There are many philosophies on how much data to collect. Our philosophy

is that it is pointless to collect data on reliable instru- ments; therefore we have established the 10 most criti- cal parameters on our most unreliable families of in- struments. These parameters and their upper and

lower tolerance limits are inserted in the calibration

recall history header (fig. 3). Each time an instru- ment is calibrated, the technician records the parameters prior to adjustment or repair. He compares the parameters against the limits and the previous read- ings to determine the instrument's drift characteristics

100
»*0 007106-203 CALIBRATION RECALL HISTORY Rf C Al I«R AT ION OA Id 0-21-1 date 05/w no

E0U1PNFNT DESCMPTIOX

VOLTMETER DIFFERENTIAL <•'• IOT0A RA ROSS

Of FT Rtl S/C w ro 90 PARAMETERS VTO REF REG OVD

UPPER UNIT 520.00 500.25 500.05 50.025

LOWER LIMIT A DO. 00 499.75 499.95 49.<)|S

OVO OVD OVD OVO OVD OVD

50.025 5.0025 5.0025 .50025 .50025 . 10020

49.9 T5 4.9975 4.99 75 .49975 .49975 .099*0

49991 5O006 49990 499R6 50000 099S5

0 0**6 1 .

CALIBRATION TRANSMITTAL Si I£

B391 79 I 096

90! > T9 I 096 1 1 r. \h III llll >4 /

1 1 0

REPLf

0 077 2.2

1 1 Isfcli rlblil.l* 'm Mi 4i\M\i\i i,\m

R£ PL*

•J74 66 I 070 .5 1 i IM 1 III 1 111 i III 1 III 1 III 1 III

REPHi^.-

79 I 090 .0 1.3 REPLACEMENT PARIS

53 I 110 .0 1.5 50? REPLACEMENT PARTS VP- 10

00 500.01 9.995 A. 9996 5.000? 99T5 .50000 .09990

9.92 500.01 *9. )1 A. 9993 92 .A9990 .50002 .09996

TOTAL »EP TIRE TOTAL C«L TIRt TOTAL HAT COST

Figure 3. Parameter history.

and the necessity for adjustment. We have a directive which dictates that if a measurement is within half its tolerance limit, leave it alone. If it is outside half its limit, adjust it to nominal. This permits adjustments

to settle at a stable value. The computer analyzes this discrete data on a family basis and produces a report

which we use to pinpoint instrument problem areas and to eliminate unnecessary steps in the calibration procedure. Both Mil Spec and customer requirements

require us to maintain a data system for recall. The

computer does this with a simple inventory technique;

however, we feel that the systems that perform recall alone fail to make full use of the computer's potential.

We use the computer for inventory, recall, history and

records, and, most importantly, special reports. We

have several reports which are used to improve re- liability; the first is the Reliability Summary Report

described below.

4. Reliability Summary Report

This report is our measurement of reliability. It is required since, in order to strive towards our goal, we

must have a measure of performance. This report

(fig. 4) lists reliability by class, manufacturer, and

model number. It tells us first how reliable we are, and it pinpoints problem areas and isolates them to classes of instruments, manufacturers, and model numbers. The report is examined monthly to isolate our

least reliable major families of instruments, the families which are causing our reliability to be less than

95 percent. It also lists the family reliability based on the last calibration, to indicate what our current

reliability is. Monitoring this report, we noted that the current reliability of a family of 38 function generators had decreased from its overall level of 89 per- cent to a reliability of 66 percent over a period of several months. An investigation revealed the major

cause of failure was signal distortion which required

adjustment. A further check revealed the distortion analyzer used in the calibration was defective, causing

the generators to be adjusted to an order of magnitude

less distortion than what they actually required. The

analyzer was repaired and the current reliability of the function generators has improved to 99 percent.

The summary report indicated one family of 44 rf signal generators had an historical reliability of 66

percent and a current reliability of 64 percent. This

indicated not only that the reliability was bad, but

that it was getting worse. The Quarterly Reliability

Report had to be consulted to determine if we had an

individual instrument problem or a family problem.

5. Quarterly Reliability Report

The Quarterly Reliability Report has the same basic

information and format as the summary report but it includes each individual instrument's reliability by

identification number. This report (see fig. 5) indi- cated the individual instruments' reliabilities varied

between 54 and 88 percent with an overall reliability of 64 percent. It is difficult to prove what causes in- struments to fail—whether it is misuse, mishandling,

poor design, etc.; however, if all instruments of the same type have poor reliability, the problem is likely

to be endemic in the type, and that is what should be

101
ABO 002106-224

HO in tol OUT TOL IN TOL OUT TOL REl *VG CYCLE

38 35.00 3.00 320 38 .89 140.00

38 35.00 3.00 .92 320 38 • 09 140.00

2.00 .00 1.00 12 0 1.00 170.00

.00 1.00 .00 T

' 2 .77 1 10.00

.00 1 .00 • 00 2 .75 130.00

2.00 2.00 50 25 .86 145.00

506.00 74.00 .67 6 646 1739 .79 183. 16 1.00 .00 1.00 2 0 1.00 365.00

1 1.00 .00 1.00 2 0 1.00 365.00

2.00 .00 1 .00 9 0 1.00 220.50

2.00 .00 1.00 9 0 1.00 220.50

3.00 •00 1.00 92 3 .91 240.00

1 1.00 .00 1.00 12 1 .92 270.00

1 .00 .00 I. 00 6 0 1.00 186.00

5.00 .00 1 .00 52 4 .92 235.20

22 22.00 .00 1.00 99 8 .92 94. 54

1 1.00 .00 1.00 * 0 1.00 90.00

2.00 .00 1.00 20 1 9S 187.00

25 25.00 00 1.00 123 9 .93 101.76

2.00 .00 I. 00 0 1.00 90.00 2.00 .00 LOO 3 1 .75 90.00 4.00 .00 1.00 " ' f 1 .67 90.00

* 4.00 .00 1.00 76 1 .98 225.00

16.00 • 00 1.00 305 12 • 96 177.1

2.00 .00 1 .00 27 • 75 240.00

1 1.00 .00 1.00 3 0 1.00 190.00

^^ 23.00 .00 1.00 411 22 .94 191.52 2.00 .00 1.00 27 5 .84 180.00

2 2.00 .00 1.00 12 0 1.00 160.00

* 4.00 .00 1.00 39 5 .88 170.00

1.00 .00 1 .00 2 0 1.00 180.00

1.00 .00 1.00 2 0 1.00 160.00

1 .00 .00 1 .00 0 1.00 130.00

4.00 .00 1.00 100 25 . 60 2 00.5 0 4.00 .00 1.00 73 13 .84 318.75

• 00 1 .00 .00 11 1 .91 270.00

5 5.00 .00 1.00 23 0 1.00 9*. 00

11 11. oo .00 1.00 61 0 1.00 I 19.09

24 25.00 1. JO .96 274 39 • 87 164.50

1 .00 .00 1.00 1

* 0 1.00 90.00

t 1.00 .00 1.00 t 0 1.00 90.00 2.00 •00 1 .00 2 0 1.00 90.00

1.00 .00 1 .00 12 0 1.00 36S.D0

1.00 .00 LOO 13 0 1.00 160.00

3.00 1.00 .75 30 T • •1 302.50

2.00 .00 1.00 . 19 0 I. 00 275.50

1.00 •00 1.00 B 1 • 88 225.00

1 .00 .00 1 .oo 2 0 1.00 90.00

i .00 .00 1.00 5 0 1.00 270.00

1.00 .00 1.00 3 0 1.00 365.00

11 10.00 1.00 .90 60 e • 90 261.00

Figure 4. Reliability Summary Report.

ARO 002106-225

N006L

6C8 £

IDE NT NO.

718582

TOL OUT TOL REL CYCLE

591 160

160. CC AVO CYCLE

l.CO POOEL 8ELMCI

1892-78

2000- 1507 2000-1520

2C00-450

2013-31 2013-380

2013-437

2441-336

2441-397

2441-398 2441-399

2441-615 634-158 700049

7C0820

700821 700822

7QI616 701617

702222

703033 703075

703577

704230

704231 706417

7C6418

707775

707776

7C7777

70866T

7C68O0

7C8801 7G8B03

7C8804

7OS805 7C6806 7C8807

7C8808 7C3809 7C6810

7C9811 7C8812

84448

651 63*

551

761 631

65t 648

891

671

691

651

771

721 618

651

841 671

651

671

791

651 621

«41

651

681

641

561

681 601

601

721 681

6 51

090

C78 090

190 C60 060

160 252 C70

C90 270

040 no

130

190

110 C70

268

130 040

090

126

C9Q 060

090

000 C60

190 190

070

C60

110 225

2 70 C90 070

090

090

160 C90

190 278

110 C90

122.47 AVC CYCLE

.68 MOOEl REL*lcr

Figure 5. Quarterly Reliability Report.

investigated. The problem in our example appeared

to be a family problem and it was examined accordingly. The investigation required the use of another

report, the Trouble-Shooting Summary Report.
6. Trouble-Shooting Summary Report

The Trouble-Shooting Summary Report summarizes

all instrument repair information by manufacturing

model number. It sorts the repair information and

lists it in alphanumeric sequence, with like codes to- taled by manufacturing model number.

The family of signal generators we investigated had

a repetitive problem (see fig. 6). The problems were 0LV8 out of tolerance 40 times and OLV6 out of tol- erance 30 times, both requiring corrective action of R. Using the mnemonic code book, these failures are de- coded as Output Low Vacuum Tube 8 replaced 40

times and Output Low Vacuum Tube 6 replaced 30

times; i.e., tubes V6 and V8 were replaced 70 times on 44 generators in one year. This represented 50 per- cent of the problems with this family of instruments.

A further investigation of the individual histories re- vealed that V6 and V8 had been replaced in several instruments on consecutive calibrations. These instru- ments were recalled and an examination revealed that supply voltages to V6 and V8 were off by over 20

percent. This caused the tubes to operate in an over- loaded condition with reduced output and a very short

life. Identical conditions were found on 4 out of the 5 generators recalled; the cause of the difficulty was

that technicians inadvisedly replaced weak output tubes when the trouble was improper supply voltages. The

instruments were in tolerance when they left the lab but the tubes rapidly deteriorated so that the output

dropped off. The technicians were informed of the

trouble, an instruction to check all supply voltages was

added to the calibration procedure, and a filament

ARD 00?106-??6 TROUBLE SHOD 1 1 NG SUMMARY REPORT PAGE 718 DATE 04/02/70

HFG. MODEL S/C 0/C COMPS OUT TCL. IN T 0L . CAD

I 12

V 3

V 6 30 1 R Figure 6. Troubleshooting Summary Report.

voltage parameter was added to the calibration transmittal. Using these steps, we improved the reliability

from 66 percent to the current 97 percent. The data were monitored to determine how often low output

occurred and to note the drift in filament voltage.

They both appeared stable after the corrective action.

7. Surplus Selection

The reliability report is also used to segregate in- struments for surplus. An arbitrary value of 50 per- cent reliability was selected and a report listed all in- struments below that value. Further study was then conducted, utilizing cost information previously stored

in the computer, to make the surplus decision. In

this way we eliminated our most unreliable instruments

which had the highest maintenance costs. We have a

set rule that no instrument less than 50 percent reliable may be used for acceptance testing; we will not certify it. This accelerates an otherwise slow process of designating equipment surplus.

8. Manpower Adjustment

We have been able to reduce manpower by examining calibration intervals versus reliability. We ex- tended intervals on with good reliability until they

equalled those for older instruments of the same type

with similar reliability. We also reduced intervals on

instruments with poor reliability. This has represented one-year savings of 3,000 hours calibration time on

8,000 instruments. The method of extension was fa- cilitated by manipulating the computer printout. An

interval change code (CI50) was attached to the recall history (fig. 7), so that when the instrument came

due for calibration, the technician noted the CI50,

changed the interval to 50 days, and removed the change code. In this way we did not have to locate the instrument physically or depend on a checklist. The basic interval adjustment was derived from an

algebraic sum of the conditions as received; e.g., II0II0IIII would yield zero net interval increase and

the interval would remain at 90 days, say, whereas

IIIIIIII00 would yield a net increase of 80 percent

and an interval of 190 days. Even though both instru- ments are equally reliable, their intervals will differ by a factor of two.

9. Standardization Committee

Our company has an Equipment Standardization

Committee made up of representatives from Engineering, Manufacturing, Procurement, Facilities and Metrology. The Committee's purpose is to generate a Standardization List of equipment recommended for purchase. Before any general-purpose electronic test equipment can be purchased, it must be on our Standardization List. The committee meets bi-weekly and

prepares a comparison matrix to evaluate equipment

specifications, performance, reliability, and price. A

matrix is made for each general item—oscilloscopes, counters, digital voltmeters, etc. Once selected by the Committee, an item must be evalauted by the Metrol- ogy Laboratory before being included on the list. In

103
mo 00? i o*-?cn CALIBRATION RFC ALL HISTORY RfCAL It*AT ION OA 1* 0-21-1 DATE 05/07/TO

OfNT 10 EQUIP «Ml DESCRIPTION NFC. MODEL CLA RANGE COST BADGE NAME OEPT REL S/C

Tt2»»7 VOlT«Frf« 0IFFE«ElirC4i 1 »}

rutWTCis m Ktf ice an ovo

18544 OVONJ COLPACK OVO OVO056 OVO55 40

OVO

UPPER LINIT 520.00 900.2% 500.0% 50. 02% 50.025 LOVER UNIT 480.00 AM. 75 494.95 49.9T5 A9.9T5 5.0025 5 . 0025 .50025 .50025 .499T5. 10020 .09910

0»tE ICC" CO CYC us » l 757 Mr CALIB COST PARMETER READINGS - .9 .T 500 500.20 500.04 SO. 0/5 90.005 5.O020 5.0000 .50025 .50000 .04998

«I?S 79 t 778

REPLACEMENT PARTS R5- I-F-K -

.0 .4 500 50010 50000 49994 5 0002 49994 50000 449S0 50010 09980

0071 79 1 050 .0 .9 900 500.00 500.00 50.001 50.009 5.0010 44990 .50015 .09480

•117 M 0 0*0 5.5 5.0 *89 900.19 500.01 10.011 50.00* 5.00O1 5.0015 499T5 .90025 .044 TO

IDT Al HEP TINE TOTAL CAL TIME S.R

5 - TOTAL MI COST 0

Figure 7. Interval change history.

this way we ensure that it meets its advertised specifica- tions and is serviceable. Standardizing on test equipment has many advantages:

1. Operator familiarity with a single item.

2. Quantity discounts.

3. One set of spare parts.

4. More efficient calibration and maintenance.

5. Purchase of the best instrument for the job.

Previous vendor history records are consulted through

the Family Summary Report to insure that calibration

costs are minimized and reliability is optimized.

10. Family Summary Report

The Family Summary Report indicates by family:

1. Average calibration hours.

2. Average repair hours.

3. Average repair parts.

4. More efficient calibration and maintenance.

5. Average calibration interval. With this type of information available, we can

readily forecast the time required to perform a cali- bration on any instrument. We can isolate the instru- ments on which we are spending excessive amounts of

time. We can compare relative performance of like instruments. For example, the report listed the fol- lowing data on two equivalent models of differential voltmeters.

Mfg.&

Model

No. Cost No. of No. Avg. Rep. Avg. Cat. Avg. Avg. Cal.

Instr. Calib. Hrs/Cal. Hrs/Cal. Rel. Interval Brand X $1240 Brand Y $1275

27

21

0.18

1.11

2.11 3.18 57

16685

The Brand X instrument is more reliable, requires considerably less calibration and repair time, and can be

calibrated half as often. Thus, even though instruments might be equivalent

in specification and purchase price, their actual his- tory can reveal cost differences of great significance.

This is the area we try to emphasize in our Standardization Committee meetings.

11. Conclusion

Although the 95 percent reliability statement of Mili- tary Handbook 52 is often criticized, it does serve a worthwhile purpose. Striving for this goal, we are pres- ently increasing our instrument reliability, reducing our operating costs, and feel confident we are improving product reliability. The day is not too far off when mean time between failures will be specified on preci- sion test equipment. Even some of the best-known in- strument manufacturers sell an unreliable product.

Metrology labs throughout the country, preferably

through the auspices of NCSL, should band together

to collect similar reliability data and should urge manufacturers to produce more reliable instruments.

104
NCSL 70

NEW U.S. AIR FORCE AUDIT PROGRAM

Lloyd W. Root

Department of the Air Force, Metrology Engineering Division, Drectorate of Metrology, Aerospace Guidance and

Metrology Center (AFLC), Newark Air Force Station, Newark, Ohio 43055

This paper describes a unique quality assurance audit program used for testing worldwide Precision Measurement Equipment Laboratories (PMEL's). Conventional methods utilized during previous

audits are examined briefly and disadvantages noted. The advent of highly sophisticated systems and

test equipment has placed increased emphasis on the experience and skill level of the technicians who provide calibration support at PMEL's. Specific configuration of a unique package designed to evaluate this skill level is discussed in some detail and results obtained during the past year are presented.

Key words: Audit examinations; audit package; Precision Measurement Equipment Laboratories (PMEL) ; technician skills; U.S. Air Force.

1. Introduction

The United States Air Force operates a large number of Precision Measurement Equipment Laboratories

(PMEL's) strategically located throughout the world. These are audited by five Air Materiel Area Laboratories (AMA Labs) located within the continental

United States. Technical direction and management

of the Air Force Calibration Program is the responsibility of the Air Force Logistics Command (AFLC)

and has been delegated to the Aerospace Guidance and

Metrology Center (AGMC) at Newark Air Force Sta- tion in Ohio. Each base PMEL is audited on an an- nual basis by its AMA with the latter being certified by AGMC. In general, examination is concentrated in

five general areas

1. Environment: temperature, humidity, dust,

housekeeping.

2. Software: Technical Orders (T.O.'s) , reference

library.

3. Reference standards: traceable to NBS or other

laboratories.

4. Test equipment: signal generators, 'scopes, multimeters, etc.

5. Personnel: adequacy of training, experience,

and skill. In addition, adequacy of facilities and administrative

methods is thoroughly investigated.

2. Previous Audit Method

Past audits have utilized basic standards in the elec- trical and mechanical areas whose absolute values could easily be predicted by a laboratory technician

to be within the tolerances allowed without actually

measuring those values. This was true because the nominal values of the selected standards are known

because of the model or part number. For example,

a GR model 1482-L inductor has a nominal value of 100 mil, adjusted to be within 0.1 percent of exact

value. Without making any measurements whatever,

a lab technician could report the nominal value and

be graded as acceptable if the tolerance allowed on his measurement were 0.1 percent. As another example,

suppose he were asked to measure a high-quality gage

block whose nominal value is known. He might re- port the exact nominal value (or even change it slightly

to make the actual measurement more palatable), and

if the allowed tolerance were ± 10 fiin, he would pass with flying colors!

3. Proposed Revisions

Two possible solutions were proposed to avoid these

situations. Allowable tolerances could be reduced or

the standard item could be changed in value. Re- duced tolerances were found to be unrealistic because

of shipping shocks or even hidden damage not under

control. Stability of some standards was found to be

poor because of hysteresis effects caused by temperature excursions experienced in transit. A second solution, that of supplying a measurement problem which

could not be "guesstimated," was also considered. For

example, one surface of an optical flat could be re- figured to produce a slight cylinder or sphere and the

lab technician required to define the measured sur- face accurately. A standard resistor or capacitor

could be "fudged" to have an actual value on the order

of a few hundred ppm away from nominal. Unless

the allowable tolerances were increased, however, the same obvious deviations might be attributed to shipping or local handling.

105
4. A INew Approach

A unique approach to laboratory audits was envi- sioned as early as 1060. The number of possible problems has grown over the years from a half dozen to more than 20, about equally divided between electrical and mechanical areas. New devices are being devel- oped so that completely different problems may be util- ized during successive audits. Sources of supply for

useful items include toy shops, surplus electronic outlets, "do-it-yourself" building supplies, and scientific houses. Problems are designed to test the ability of the lab- oratory technicians to work together as a team in the

solution of the test problems. Emphasis is placed on comprehension, original thinking, and measurement

techniques rather than on resolution and accuracy to parts per million. The device being measured may be

sensitive to its environment or to the pattern of the applied stimulus. Thus, test results may vary from

one laboratory to another because the auditor may

change the specified test parameters, resulting in many

different answers to a particular problem—and all can be correct. Some problems may specify the choice of test equipment; others allow a very wide

selection.

5. Audit Implementation

Problem layout and nominal parameters are sent to the audited laboratory at least one month in advance

of the appearance of the auditor and actual test pack- age so that they may be studied and measurement

schemes prepared for the auditor's approval. One

problem is designated as required, with an additional

three of the remaining six to selected by the audited

laboratory. Most problems may be solved in several

different ways, but any scheme proposed must be ac- ceptable to the auditor before being implemented.

Assistance may be provided when required and, at the conclusion of the test, other schemes having equal or greater merit may be discussed. A valid error analysis must be included with each final answer produced.

All schemes, test data, compilation of results, and

error analyses are reported in triplicate; one copy re- mains at the tested site, one is filed at the AMA, and

one is sent to AGMC for analysis. When such analysis indicates lack of suitable test equipment, reference

standards, or training, limited certification may be imposed or recommendations made for future augmentation of the laboratory capabilities.

To facilitate uniform reporting, a check list is used

for each separate problem as follows:

Audit Package Checklist NOTE: Minimum requirements—problem 2b plus a choice of three additional problems. Data sheets and measure- ment schemes are to be prepared in triplicate by audited laboratory.

Name of auditor/Symbol/Date

PMEL/AMA Symbol, Building

Problem number

Work location illumination level, foot candles temperature limits, °C

humidity limits, % RH

Yes No

1. Was measurement scheme prepared in advance?

2. Is it acceptable to auditor? (If not, on-spot OJT may be provided by auditor or

alternate problem may be selected).

3. Are current AFTO 108's affixed to the test equipment?

4. Were model and serial numbers of equipment recorded

5. Were model temperature/humidity recorded?

6. If desk calculator, reference library, or special equipment were needed, were ihey

available? (Explain with attachment, if necessary).

7. If Ac ratio measurements were made, were proper corrections made for zero and

unity balances?

8. Were shielded cables used (if necessary)?

9. On dc ratio measurements, were proper lead compensation corrections made or analyzed for possible error?

10. Was effort wasted in attempting greater resolution of accuracy than required by

the- problem?

11. Were loading errors considered?

12. Was more than one method used to solve problem?

General comments as to capability of audited personnel, attitude, cooperation, motivation, etc. (Use reverse side).

106
6. Audit Package Configuration

The actual package for the electrical audit is de- signed to test proficiency in the use of an Electrical Measurement Console ( EMC ) issued to all base laboratories. Two standard "mini-boxes" containing all components are housed in a small, foam-padded satchel.

Figures 1. 2, and 3 show the method of packaging;

figures 4 and 5 the component layout.

Many variations are possible for the oscillator cir- cuit board depending upon the degree of sophistication desired. Two representative schematics are shown

in figures 6 and 7.

Figure 3. Chassis layout "B" items.

Figure 1. Portable audit package.
J

0.1 UF Q 3-5 K g

13 4UF Hi" 14

o

15

O—1(—

O

16 iuf 17

Figure 5. Plan view "5" nominal values.

-OHEP 50

10 K

*vW\Aj-Of/0 W

OSCILLATOR CIRCUIT

Figure 6. Typical oscillator schematic diagram.

6 M3IO

C^

R

-O-nov

3

5-oS/N C, PF R,K-OHMS C„UF

116A 180 9.9 7 .01 II7A 932 4.86 .03

118A 330 10.09 .01 SPECIAL SERIES

Figure 7. Special series oscillator schematic.

7. Audit Problems (Instructions

Accompanying Package)

1. Caution. Avoid rough or prolonged handling of the metal boxes—all internal components are temperatureand shock-sensitive to some degree. Some components

can be damaged by application of excessive potentials or currents so that instruction should be followed exactly.

2. All serial numbers ending in "A" contain a complex signal source available at BNC connector, terminal

#6, and a zener diode of nominal 6.8 Vdc breakdown

between terminals #3( + ) and #4( — ). Note that ter- minals #5 is case ground.

a. zener diode problem. Avoid handling the metal boxes unnecessarily. Determine the voltage drop

across terminals 3-4 when a specified current between

1 and 5 mA is made to flow through the diode in such

a direction as to make terminal 3 positive. Using

only components of the Electrical Measurement Console (EMC), devise a scheme for measurement. Do not use a grounded current source and do not

use terminal 5 for this test. Scheme must make it impossible to send a current through the diode or as- sociated circuitry greater than 5 mA. Scheme must

be approved by the auditor and all current and potential measurements must have an uncertainty less than 0.01 percent. Final report must detail the scheme

used (schematic diagram) and include an error analysis to justify the 0.01 percent uncertainty. If the 0.01 percent uncertainty cannot be attained, a full explanation must be provided with a statement as to what uncertainty can be realized. , b. complex signal problem/. Avoid unnecessary handling of the metal box. Apply 10 V dc to terminals 3( + ) and 5(— ) from a source which is current

limited to 5 mA. Applied voltage must have a magnitude uncertainty less than 0.1 Jpercent. A complex

signal should now be found at BNC connector, terminal #6. No restrictions are placed on your choice of

test equipment. Determine and,'record all parameters

of the complex signal and document the method of

test. Include an error analysis statement of estimated

uncertainties. f 3. All serial numbers ending in "B" contain the following: . a. Terminals 7—8, nominaj 0.1 /xF

b. Terminals 8—9, nominal 3500 fi

c. Terminals 10-11—12, tapped toroidal inductor

d. Terminals 13-14, capacitor, greater than 1 /aF

e. Terminals 16—17, nominal 1 /iF

f. Terminals 11—15, thermistor, approximately 50

kft Note that terminal #15 is case ground.

4. a. phase defect problem. Using the Phase Standard, model 311, and the necessary ancillary equipment

of your choice, drive terminals 7 and 9 with 10 V rms

at 1 kHz and measure the phase angle of the output

terminal 8 with reference to the input drive voltage.

Now reverse the drive and again measure the output

phase angle. It will be found that the sum of these two measured angles is slightly less than 90 degrees

(if the experiment is correptly performed). This ap- parent "phase defect" is claused by an imperfect ca108
pacitor, i.e., one that has a dissipation factor greater

than zero. From the phase angle "defect" noted, cali culate the true dissipation factor of the capacitor, be- tween terminals 7-8. Now, using the EMC Capaci-

! tance Bridge, model 707B, and AC Generator-Detector, model 861A, measure the same capacitor and compare results. Account for any disagreement in "D." Measure the resistance between terminals 8—9 and,

using the measured capacitance between terminals 7—8,

calculate the output phase angle relative to the input

| terminals 7—9 and account for any difference over the measured value obtained using the model 311.

b. CORE saturation problem. Using your choice

of test equipment, devise a scheme for determining

the maximum rms voltage that can be applied to the inductor terminals 10—12 before core saturation occurs. Express this maximum voltage (V, rms) as Kf where

K is a constant (to be determined) and / is the applied frequency in hertz.

c. AC RATIO PROBLEM. Using only components of

the EMC, devise a scheme to measure the turns ratio between terminals 11-12 compared to terminals 10—12.

This should be done using a drive voltage and fre- quency as specified by the auditor. Determine also the phase angle between the voltage on terminals 11—12

as compared to the drive voltage on terminals 10—12.

d. thermistor problem. Using the model 242

resistance-measuring system, make a two-terminal re- sistance measurement between terminals 8—15, using

the minimum applied voltage needed to obtain a re- sistance measurement with a resolution of 0.1 percent.

Note that terminal 15 is case grounded. Avoid handling the metal box unnecessarily. Record this measured value and the average room temperature near

the metal box at the time. Note the excitation voltage

applied to the system and calculate the voltage appearing across and the power dissipated in the unknown

thermistor. Record all values. Hint: It may be helpful to measure this resistor while

it is protected from room temperature changes. To

do so, form two 3-ft lengths of No. 22 AWG insulated copper wire (telephone wire) into a tightly twisted pair and guide it through the small hole in the right side of the carrying case so that the metal box (S/N

XXX B) can be completely covered in its polyfoaminsulated container, with the cover closed during measurement. Connect the metal trim on the carrying

case to ground to eliminate hand capacity effects. Now

determine the voltage that must be applied to the test item to cause sufficient self-heating of the bead ther- mistor to change the measured resistance value by ap- proximately 20 percent. Explain the result noted. Record all data.

e. CAPACITANCE AND DISSIPATION FACTOR PROBLEM.

Using only the models 707B and 816A and any components found in S/N XXX B, devise a scheme to measure the capacitance and dissipation factor of the capacitor connected to terminals 13—14. Record all data used and have auditor check your results for ac- curacy. State expected accuracy to be obtained for both "C" and "D." The measurement is to be per- formed at 1 kHz, applying no more than 10 V rms

to the 707B.

8. Deficiencies Observed—Action Taken

In a few cases, laboratory technicians had not at- tended Air Training Command school at Lowry Air Force Base in Colorado and thus were not thoroughly

familiar with the Electrical Measurement Console.

Certain areas of training had not been adequately

covered so that many unusual field-measurement situ- ations encountered could not be correctly diagnosed.

With more than half of all reports received (80), the

deficiencies can be noted as follows:

1. Temperatures not stated in degrees Celsius as requested.

2. Improper ANSI symbol usage.

3. Error analyses not understood.

4. Iron core saturation effects ignored.

5. Incorrect derivation of capacitance "D"

factor.

6. Effects of loading on frequency and amplitude

of complex signal generator not taken into account.

7. The terms "precision," "resolution," "accuracy," and "significant figure" not understood.

8. Lack of comprehension of signal source

parameters.

9. Problems created by thermal or voltaic emfs,

ground loops (common/normal mode) not recognized.

10. Need for independent measurement schemes

for verification not appreciated.

Ignorance of the above areas can damage test equipment and reference standards and produce in- accurate or incomplete measurements. Corrective

action has already been initiated. Controlled Multiple

Address Letters (CMAL's) are sent out as required,

the curriculum at Air Training Command is being

modified to provide additional information in the above-noted areas, and technicians from the few

PMEL's given limited certification are being given

additional training at AGMC laboratories by Metrology Engineering Division Engineers.

9. Summary and Conclusions

The implementation of a new approach to metrology

laboratory audits has reemphasized the importance of personnel in providing calibration support of aircraft and weapons systems. As Dr. J. L. Thomas once said,

"You don't get six-dial measurements from a four-dial operator." The items described are sufficiently versatile so that completely different problems can be gen- erated for subsequent tests. This can be accomplished

either by interconnecting items differently or by

actually changing readily available components.

A very high degree of interest and cooperation was ob- served at the audited laboratories and most were eager

to solve all problems, rather than be limited to only four. The realizable advantages and goals may be stated:

1. Improves morale, stimulates original thinking.

2. Provides in-depth analysis.

3. Assists in yearly Posture Study.

4. Upgrades capabilities.

5. Allows augmentation of available reference

material.

6. Enables meaningful revision of training programs.

109
I
NCSL 70

SESSION 5: NEW INTERNATIONAL DEVELOPMENTS

Chairman: C. E. White

General Radio Company, West Concord, Mass. 01781

NPL WORK ON THE DETERMINATION OF

2e/h BY THE AC JOSEPHSON EFFECT

B. W. Petley and K. Morris

Division of Quantum Metrology, National Physical Laboratory, Teddington, Middlesex, England

The a-c Josephson effect in superconductivity appears to be an exciting possibility for use as a quantum method of maintaining the volt as an SI unit, through the relations 2 eV — hv. A potential- divider resistor was devised for precise comparison of a Wesson-cadmium cell with millivolt outputs from various solder-drop junctions excited at 36.8 GHz. The value found for 2 e/h is 483.5941 ±

0.0010 MHz/MV69NPL .

Key words: Inductive ratio divider; Josephson effect; NPL of UK; potential-divider resistor; quantum

standard of voltage; solder-drop Josephson junctions; superconductivity.

1. Introduction

Part of the work of the Quantum Metrology Division

at the NPL is devoted to measuring atomic constants and to deriving and maintaining the basic SI units. Already the units of length and time are based on quan- tum properties, and the ac Josephson effect in superconductivity [1, 2, 3] 1 appears to be an exciting possibility for use as a quantum method of maintaining the volt. When microwaves of frequency v are incident on the barrier between two weakly linked superconductors,

supercurrent steps occur in the barrier-current-voltage

characteristic which are separated by a potential differ- ence V, given by 2 eV = hv, where h/2e is the super- conducting magnetic flux quantum.

The initial objective of our work was the verification of the value obtained for 2 e/h from the pioneering

work [5, 6] at the University of Pennsylvania, to about

the same level of precision. This paper describes some

of the measurement techniques that have been employed in making the initial measurements.

2. The Basic Measurement Principle

As with most quantum units, the voltage available foi measurement from the Josephson effect is much smaller than the maintained unit, being smaller by a factor of between 100 and 2000. The basic measurement problem, therefore, is that of comparing two very different potentials precisely. Further, although potentials of the order of a volt have been measurable to 0.2 fiV for some

1 Figures in brackets indicate the literature references at the end of

this paper.

time, there has not been nearly as much attention de- voted towards measuring potentials at the millivolt

level to parts in 107. It is preferable that, in the early

stages at least, as many different methods as possible are explored by different groups of experimenters:

agreement between two different methods will add considerably to the degree of confidence in either method.

Following the principles suggested by Dauphinee [7]. of minimizing the number of switches in the low-potential circuit, the comparison of the two potentials is achieved simply by using two resistors connected in series as in figure 1, and balancing the Josephson volt- age across R(,h and the standard-cell emf by the voltage

developed across Rac - Unless the resistor ratio, micro- wave frequency, and supercurrent step number are particularly chosen, the current L must be adjusted between

o l 1 a

Figure 1. Illustrating the principles of the potentiometer.

Ill
the two balances; moreover the two currents must be

known precisely in relation to one another. Considerations of the precision with which the resistance ratio could be measured led to the conclusion that the resist- ance Rac should be about 500 fl. which, at the time that measurements commenced, ruled out the current-comparator potentiometer [10] as the precisely variable current source. The basic methods of obtaining the vari- able current i-, were either to maintain i x constant and

vary R > and R3, as in the Julie I [8, 9] inverted KelvinVarley-divider method used by the Pennsylvania

group, [4, 5, 6] or to keep /, approximately constant and

vary 7? L., deriving a feedback error signal to the current

controller such that the potential difference i L./?^ exactly

equals a standard-cell voltage. This error signal is readily available if the galvanometer-amplifier type of current controller is employed. The Tinsley-Stabaumatic potentiometer [11] which uses this principle was

modified to suit our experiment, an important modification being the potential- divider resistor, oabcd, of figure

2. This resistor was located outside the potentiometer

and was constructed at the NPL.

9 decade I V.D. error 2 in lO'of part. Divider resistor on ratio •OOl.over range

Measured on Oa.Ob & Oc

Required ratio'. Ob - Oo

Oc -Oa

FIGURE 2. Circuit arrangement used to calibrate the divider resistor.

3. Potentiometry

The Josephson junction is a very broadband deice [2] the frequency response extending from dc to the

infrared, while the nonlinear behavior at the junction al- lows the mixing of any frequencies within this range A

major problem in any experiment involving small potentials is always the elimination of potentials generated

by pickup and ground loops. These are particularly

difficult to overcome when more than one item of equipment is connected to the power lines. It was decided as a matter of principle to keep all leads and apparatus

using alternating current more than 3 meters from the potentiometers and cryostat and to use only galvanometers and galvanometer amplifiers for detecting balance

conditions.

3.1. The Voltage and Current Source

The voltage source in figure 1, and the source for the junction bias current, were both supplied by lead-acid

batteries. These were charged overnight by electronic battery chargers to be at the beginning of their dis- charge plateau. During the day their emf was constant to a few millivolts.

3.2. Current Controllers and

Galvanometer Amplifiers

There are two ways in which galvanometer amplifiers

and associated current controllers may introduce alter- nating currents when used for sub-ppm precision

measurements, even though they are screened from al- ternating magnetic fields. The first way arises if the galvanometer bulb is energized by an alternating current. The light falling on the photocells is modulated

at the second-harmonic of the power frequency, thus introducing an alternating signal in the feedback loop

which then appears on the current-controller output.

This problem was avoided by energizing the galvanometer bulbs with direct current. The second way arises from any vibration of the galvanometer spot, for such

vibration induces an alternating signal in the photocell

output which again results in an alternating current at the output of the current controller. (The effect of the feedback loop is to increase the frequency response of

the galvanometer to a few tens of hertz. ) All galvanometer amplifiers were mounted on concrete blocks which

were in good mechanical contact with the floor. Fortunately the site of our experiment was below ground level and known to be free from vibrations. The possibility of resonances from acoustic pickup effects was also checked.

3.3. The Potentiometer

The essence of this type of potentiometer is that a standard cell is used in the feedback loop of a servo- system. Since the gain of the system is about 2 X 104

, the standard cell would supply current if the system was

not supplying nearly the correct current throughout the balancing operation (which R-, was switched). Three

control standard cells were employed, being switched

into operation in sequence as balance conditions were

approached. The final balancing cell was thereby protected from any switching surges and current drain. The emf of this cell was merely required to be stable while the Josephson and main standard-cell voltages were intercompared. There was, in fact, a slow drift throughout the day as room temperature changes penetrated the heavy thermal lagging of the cell. This drift was useful since it meant that the positions of the switches varied from one set of balances to another and,

by making measurements on different step-numbers, the whole of the potentiometer entered into the measurements. The Stabaumatic potentiometer and its associated current controller are designed to supply up to 20 mA,

and a good quality resistor was connected across od so

that the combined resistance of it and the divider re- sistor was about 50 ft. Possible nonlinearities in the current division between the two resistors were elimi- nated by an in situ ac calibration across the potential

112
tappings, using a Guildline current-comparator potentiometer[10]. Corrections to the Stabaumatic-potenti- ometer dial readings were calculated and applied as part

of a computer program for evaluating the 2e/h values. All switches in critical parts of the circuit, including the potentiometer dial switches, were rotated 20 times from

zero to full scale before commencing measurements.

4. The Divider Resistor

This resistor was designed and constructed in the NPL and was measured by an ac measurement tech- nique.

4.1. Design

Measurements were made over a range of supercurrent steps in the junction current-voltage characteristic and so the ratio of the divider was chosen to be 1,000 : 1. Techniques for measuring such ratios by an ac method [13] had recently become available in the NPL

and it was expected that an overall measurement pre- cision of better than a part in a million could be

achieved on our ratio [14]. The inherent experimental

difficulty of ensuring that the ac and dc values of the

ratio were the same had already been solved by members of the NPL Electrical Science Division [15]. Our

divider was designed and constructed by T. A. Deacon

and M. John Swan respectively. The divider resistor was constructed from two types of Evanohm wire for the 500- and 0.5-fi portions and thus behaved as two

separate resistors as far as aging and temperature coefficient of the divider ratio were concerned. All current and potential connections were welded to the re- sistor, and the resistors were of bifilar construction,

wound on a silicon glass laminate former, and mounted

to be at least 5 cm from the brass container. The termi- nations were brought as close together as possible to reduce temperature differentials that might have an ef- fect on dc. Although most "dc" measurements are effectively

made at frequencies between 0.01 and 0.5 Hz, the region

between these frequencies and, say, 10 Hz is that in which changes due to Peltier and other effects are most

likely to occur if the resistor is not sufficiently well de- signed. The design priority was given to ensuring that the ac and dc values were the same and to make the

ratio independent of ac frequency, range 10 to 120 Hz. These objectives were satisfied in the completed divider.

4.2. Measurement

The equipment that was used for measuring the di- vider ratio is illustrated in figure 2. Both the divider

resistor and the 9-decade inductive ratio divider were

energized from the same transformer secondary winding and the voltage across the appropriate divider ter- minals was balanced against that developed by the in- ductive ratio divider. The resistance R and mutual

inductor M were used to inject a small signal into the detector to balance the quadrature component.

The inductive divider transformer had an error of

less than 2 parts in 107 of the output at the 0.001 tap over the frequency range 20 to 200 Hz [14]. The trans- former used to isolate the detector was double-screened

the secondary screen and the detector were grounded.

The primary screen was connected to the guard trans- former divider. The guard voltage was set to better than a part in a million, although no effects were ap- parent when the guard voltage was offset by parts in 10'. The ratio of the divider was obtained from three measurements. In principle, of course, only two meas- urements are required but the grounding arrangements

used for our initial work made it necessary to include

the resistance of the current tail oa in the measurements

of the divider ratio.

4.3. Performance

The divider ratio was measured over the frequency range 20 to 120 Hz, and no frequency-dependent variations of the ratio were observed. ( If present, such ef- fects would generally vary linearly and/or quadratically

with frequency.) A change in the voltage applied to the divider from 1 V to 10 V did not change the ratio by as much as 2 parts in 107

. In making the error as- signments to the measured values of 2e/h it was considered that the divider ratio was known to a standard

deviation of 0.5 ppm and that the ac value represented

the dc value to 0.5 ppm.

The divider was put into service shortly after construction and hence aged initially by several ppm before

it settled to a slower linear drift with time. Measure- ments of the divider ratio have been continued and the values obtained are within 0.5 ppm of the extrapolation of the curve fitted through the calibration points

pertaining at the time of the 2e/h measurements.

5. Measurements

The apparatus, measurements, and errors have been

described in greater detail elsewhere [16, 17].

5.1. The Junction

The solder-drop type of Josephson junction was used

for the initial work, and so enabled the use of a normal

liquid-helium storage dewar for the cryostat. This

simplification had a number of advantages: for example,

the thermal emf's in the dewar were very stable, and

there were no liquid helium transfer problems since one

filling lasted for over a week. Junctions have been

transferred from one storage dewar to another over a period of six months and from one site to another. The solder-drop type of junction is made by forming

a drop of solder •

• 1 mm diam, around a length of

thinly oxidized niobium wire — 0.15 mm diam. Two

copper leads are then embedded in the solder-drop and

these, together with the ends of the niobium wire, form

a convenient four-terminal device. When the roomtemperature resistance of the "dry-joint" formed be- tween the solder and the niobium wire is about an ohm,

the yield of junctions showing Josephson effects when

the device is cooled to liquid helium temperature is. with practice, around 50 percent. It is, incidentally, rather harder to produce a "dry joint" of about 1-fl

resistance than it is to make a well-soldered joint. In general, solder-drop junctions show characteristics of the weak-link type—that is, there is probably a very fine link of superconductor joining the niobium

wire and the solder-drop. Such junctions are characterized by greater nonlinearities at the junction, al113
though the supercurrent steps still appear at the appropriate potentials V, governed by 2eV = nhv, where h/2e

is the superconducting magnetic flux quantum, n is an

integer, and v the applied microwave frequency. As a

result of the non-linearities at the barrier in the weaklink type of junction, steps may also appear at values of n corresponding to the ratio of two integers, for ex- ample, for n = 3/2, 5/2, 7/2, etc. Such subharmonic

steps are clearly visible in the solder-drop junction

characteristic illustrated in figure 3. This characteristic was obtained with an X—X pen recorder.

I-O

io-s

0-5 l-O Junction voltage (mV)

1-5

Figure 3. Example of a solder-drop current-voltage

characteristic when 36.8 GHz microwaves

are incident on it. Note the subharmonic supercurrent steps.

5.2. Measurements

The measurements of 2e/h were made using a 36.8-GHz phase-locked klystron as the microwave

source. Measurements were made at lock-points on

either side of the reference-crystal harmonic, at differ- ent step numbers and using different solder-alloys for

the construction of the junctions. The measuring se- quence was as follows: Standard cell balance, junction

balance, all currents reversed, followed by a second

set of balances of the junction and standard-cell potentials. The temperature-controlled standard cells were calibrated by J. J. Denton against the NPL

volts each day at the beginning and end of the 2e/h

measurements.

5.3. Thermal Emfs

Thermal emfs are an important source of error in any dc measurement and are particularly so in 2e/h

measurements, where the potentials are at the millivolt

level. The procedure discussed above of reversing the currents was designed to eliminate the thermal emfs

but it is, of course, rather reassuring if the procedure

can be shown to eliminate them. Fortunately in this respect, during part of our experiment the thermal

emfs increased to around 2 ^.V instead of the usual 100 nV (apparently resulting from a strain effect in the copper leads into the cryostat). It was found that

the 2e//i-values obtained with the high thermal emf

were 0.8 ± 1.1 ppm greater than the low thermal

values. The difference was not statistically significant

but a standard deviation of 1 ppm was assigned to the elimination of the thermal emfs.

5.4. Ground Loop Effects

Ground loop effects are also a problem in the meas- urement of low dc potentials. These were avoided by

ensuring that as far as possible, all low potential connections were copper to copper, and that the connections were oil immersed. The effects of ac pickup were reduced by operating all equipment within 3 m

of the cryostat and potentiometer, from dc sources

that were themselves well screened. After rearranging

leads and modifying earthing arrangements while set on a supercurrent step, it was concluded that ground

loop and pickup effects could not have affected the 2e/h values by as much as 0.5 ppm, and this was

assigned as the possible error from these effects.

6. Results

Our results and discussion of the other sources of error have been discussed in detail in our Metrologia

article [17] and this discussion is not repeated here. Figure 4 illustrates the values obtained for 2e/h,

with standard deviations, for solder-drop junctions

483-596

483-592 1

aO

c

I/) O

a

O

c

i/l O

f 2ppm

i

I T

1 I

20% 607o

Tin content

Figure 4. The values obtained for 2e/h with

solder-drop junctions using different solder alloys.

constructed from different solder-alloys. Fitting a

least squares line through the points indicates that the values for pure tin solders could be greater than

those for pure lead solders by 3 ± 5 ppm. This differ- ence is, of course, not statistically significant.

The final value for 2e/h from these measurements

is 483.5941 ± 0.0010 ( 2.2 in 10* ) MHz//iVG9 nplThe expression of our result in terms of the NBS volt cannot properly be made before the results of the 1970

114
BIPM International Volt comparisons are available. On the basis of the 1967 comparison, our values and

those of the University of Pennsylvania [6, 18, 19] are

unlikely to differ by more than about 0.5 ppm.

Already, the inadequacies of the Weston-cadmium

cell as a method of maintaining the volt are becoming

apparent in the Josephson effect. The precisions of

the 2e/h measurements are well beyond the precision

with which the volt is known absolutely [20]

(*— 2.7 ppm) and measurements are improving

monthly. It appears likely that within a year or so

sufficient agreement between National Laboratories

may be achieved for the whole of the experimental

error to be attributed to the maintenance of the volt by the Weston-cadmium cell. At this stage it will be

possible to maintain the volt by the Josephson effect. Beyond that, there lies the possibility that the Joseph- son effect might ultimately lead to a redefinition of one of the SI units.

This work forms part of the research program of the National Physical Laboratory.

7. References

[1] Josephson, B. D., Phys. Letters 1, 251 (1962) ; Rev. Mod.

Phys. 36, 216 (1964) ; Adv. in Phys. 14, 419 (1965).

[2] Anderson, P. W., Progress in Low Temperature Physics, 5,

p. 1, Ed. C. J. Gorter (North Holland Puhl. Co., Amster- dam, The Netherlands, 1967).

[3] Petley, B. W., Contemporary Phys. 10, 139 (1969).

[4] Taylor, B. N., Parker, W. FL, Langenberg, D. N., and

Denenstein, A. D., Metrologia 3, 89 (1967).

[5] Parker, W. H., Taylor, B. N., and Langenberg, D. N., Phys. Rev. Letters 18, 287 (1967).

[6] Parker, W. H., Langenberg, D. N., Denenstein, A., and

Taylor, B. N., Phys. Rev. 177, 639 (1969).

[7] Dauphinee, T. M., IEEE International Convention Record

(1965).

[8] Julie, L., IEEE Trans. Instr. Meas. IM-16, 187 (1967).

[9] Abramowitz, A., Rev. Sci. Instr. 38, 898 (1967)

[10] MacMartin, M. P., and Kusters, N.L., IEEE Trans. Instr. Meas. IM-17, 263 (1968).

[11] Gall, M. H. W., Acta Imeko, II, UK-237, Ed. E. Nika and

J. Scott (Hungarian Scientific Society for Measurement

and Automation, 1967)

[12] Taylor, B. N., J. Appl. Phys. 39, 2490 (1968)

[13] Hill, J. J., Proc. IEE 112, 211 (1965).

[14] Deacon, T. A., Proc. IEE 117, 634 (1970)

[15] Wilkins, F. J., and Swan, M. J., Proc. IEE 117, 841 (1970)

[16] Petley, B. W., and Morris, K., Phys. Letters 29A, 289

(1969).

[17] Petley, B. W., and Morris, K., Metrologia 6, No. 2 (1970)

[18] Denenstein, A., Finnigan, T. F., Langenberg, D. N., Parker, W. H., and Taylor, B N. (to be published 1970)

[19] Finnigan, T. F., Denenstein, A., and Langenberg, D. N., Phys. Rev. Letters 24, 738 (1970)

[20] Taylor, B. N., Parker, W. H., and Langenberg, D. 8., Rev. Mod. Phys. 41, 375 (1969).

115

NCSL 70

THE IMPACT OF ADVANCED ELECTRONIC

TECHNOLOGY ON MEASUREMENTS

Richard F. Clark

Radio and Electrical Engineering Division, National Research Council, Ottawa, Canada

Advances in electronic technology have made possible significant improvements in both accuracy and precision of measurements. Some comments will be made on the increase in accuracy through

automatic application of data corrections (which were previously too difficult to be practicable) and

the elimination of many "human" errors. Better precision coming from increased speed of measure- ment and improved control of the environment of the measurement will be discussed. Key words: Attenuation measurement; automatic data correction; auxiliary receiver for microwave

measurements; dual admittance Woods bridge; Engen microcalorimeter; rf coaxial impedance

standards.

1. Introduction

Advances in electronic technology have made possible substantial improvement of both accuracy and

precision through better control of the total environment of a measurement. In particular, reduction of human involvement and automatic application of cor- rections to data have increased accuracy, while shorter measurement times and automation of manually oper- ated instruments have given us greater precision.

2. "Human" Errors

The experimenter's human characteristics have tra- ditionally set limits on precision and accuracy of a measurement. In addition to basic defects in precision

sensing, memory, physical stability, speed, and durability, his greatest asset—intelligence—makes him unable to perform reliably and consistently a simple task without frequent error. Now, there has begun a tre- mendous upsurge in the use of computers to replace man in the rapid performance of simple tasks. The man in charge of a calibration can now have an

unintelligent machine of incredible speed and great re- liability to replace an intelligent technician. Doubtless,

in the early stages of such a development, great effort is necessary on the part of the supervisor if he is to communicate with this unintelligent beast in its strange

language. However, such extra effort will pay for itself in the insight gained into the calibration procedure.

Perhaps a better and more basic measurement will

result. The whole aspect of standard calibration procedures can be improved by the sharing of computer programs

by different laboratories. As programs are modified

and become accepted by the different laboratories,

many of our calibrations will be performed by a truly

standardized procedure.

3. Data Corrections

One of the principal uses of computer involvement

in a calibration laboratory is for the automatic application of data corrections. In January of 1950, D. Woods

[l] 1 gave a complete description of a dual-admittance

bridge to be used for measurement of shunt capacitance

from —50 to +50 pF and shunt conductance of 50

mmho or less in the frequency range 3 to 300 MHz.

The excellent 0.1 percent accuracy of conductance

measurement and 0.2 percent accuracy of complex ad- mittance were made possible by painstaking care in as- sessment of all sources of error in the bridge. Many of

the errors were made negligible by careful mechanical

and electrical design and the use of external calibration standards. The considerable number of errors remaining were corrected for by the use of a prepared calcula- tion table which guided the worker through the steps for the corrections. With the help of this table and an

electric calculating machine, the results for a set of six measurements at one frequency could be obtained in two hours time. This illustrates what was once a basic philosophy of

a standards laboratory—time is not a major consideration. If you imagine the addition of automatic readout

and computer application of corrections for the Woods

bridge, you could further improve accuracy by performing more measurements using external standards with no need to improve the bridge itself. As is the case with the automatic measurement system described by

Adam [2], the ultimate limitations on accuracy would

be set by the repeatability of the bridge and the characteristics of the external standards and their connection to the bridge.

There is another factor to be considered in respect

1 Figures in brackets indicate the literature references at the end of

this paper.

117
to the use of automatic data corrections. A measure- ment technique which would be quite impractical and

tedious for manual operation might be chosen because

of its mechanical simplicity or the criterion of greater accuracy. An example of this is the technique for coaxial impedance measurement described by Jurkus [3].

This system has been used to calibrate coaxial terminations in 14-mm precision coaxial transmission line at about 100 frequencies per octave over the frequency

range 1 to 8 GHz. It is a semiautomatic system using a

fixed probe in the transmission line and makes full use

of the computer for analysis of the data including cor- rections for source mismatch and loss in the reference

air lines used as impedance standards.

4. Improved Precision

The time required to perform a measurement auto- matically is usually dramatically less than required for

a manual one. Thus, statistical control of the measure- ment becomes a distinct possibility because large numbers of measurements may be performed within a rea- sonable time. Also, precision should improve, since

the magnitude of certain parameters, especially those

affected by temperature, will be more constant over a

short period of time than over a long period.

The instrument makers, realizing that the human

being is becoming less involved in the measurement

process, are rapidly converting their instruments. The

attention to and compensation for the defects of man

are being replaced by a similar concern for the vagaries

of the computer. Man, however, can benefit from some

items such as programmable sources, meters with digital

readout, or meters with an automatic zero adjustment.

We have benefitted in our laboratory from an automatic

zero adjustment on a thermistor bridge used to operate

a thermistor mount for rf power measurement. This

benefit arose in connection with the insertion loss meas- urement of a 20-dB fixed waveguide attenuator of the multihole directional coupler type and used as an inter- laboratory transfer standard of attenuation at a fre- quency of 10 GHz.

As is sometimes the case in a standards laboratory,

the stable microwave source and the precision micro- wave receiver which form part of the environment for our best attenuation measurements achieve, in part,

their stability from their massive size. This size can make the insertion and removal of the attenuator difficult and time-consuming because of the severe require- ments on alinement and positioning of the connecting

flanges. Larson [4] has suggested modification of the attenuator so that the input and output parts are in line and have the same axis. This will reduce the required

relative motion of the source and receiver, especially

when a piece of waveguide having the same length as

the attenuator is used to replace the attenuator and a

differential insertion loss measurement is performed.

It is possible, however, to use a small auxiliary re- ceiver as shown in figure 1. The auxiliary receiver is a thermistor mount and a tuner, and is light enough to be

supported by the flange to which it is connected. The

change in output power from the rf source required to maintain a level of 0.4 mW into the thermistor mount

with the attenuator inserted or removed was measured

by the precision receiver. The rf source output was

reduced to zero and the power meter automatically

zeroed by depressing a front panel switch immediately

prior to each measurement. The standard deviation of the mean of 10 measurements on a 20-dB attenuator was 0.0003 dB, which is about an order better than we

could obtain with an instrument having a manual zero, used in our air-conditioned laboratory at the level of 0.4 mW.

5. Role of the Standards Laboratory

Ultimately a computer-assisted measurement system

will outperform a manual system to such an extent that the standards laboratories will be forced to participate

fully in the revolution. For the present, the standards

laboratory will be faced with the need for calibration

of the standards which are used as references in the automatic system. The calibration accuracy probably

need not be improved, but the range of the calibration

will need to be increased as reliance placed on the in- strument is shifted to the standards. I would like to end this paper with a few words about two particular rf standards which require calibration with the presently

available accuracy but at many more frequencies so

that the parameters of the standard are well known

throughout its frequency range. An ideal working or transfer standard for rf coaxial

impedance measurement is the coaxial resistive termination. It is simple, rugged, stable, and covers a large

frequency range. This is in contrast with quarterwavelength coaxial air lines, which are very narrow

STABLE

R F SOURCE \

1

1 II

1 I'l II II II THERMISTOR

II 1 II MOUNT 1 I

PRECISION

RECEIVER

Figure 1. Measurement of insertion loss with a thermistor mount and

power meter combination used as an auxiliary receiver.

118
band, and coaxial sliding loads in precision air lines, which tend to be expensive and somewhat limited in frequency coverage. The calibration technique of Jurkus [3], mentioned above, provides a sufficient number of calibration frequencies so that the calibrated co- axial termination may be used at any frequency from 1 to 8 GHz with the aid of linear interpolation of the data. For the "non-automated" who still measure impedance

at one frequency with a slotted line having mechanical

drive and X-Y recorder for readout, these calibrated terminations can provide a quick, accurate check of residual VSWR of the slotted line, again at the fre- quency of measurement. The second rf standard which needs calibration at a great number of frequencies is the thermistor mount

which is used as a transfer standard of power. The

ideal calibration environment for accurate measurement, at several frequencies of the effective efficiency q of a thermistor mount, is the Engen microcalorimeter

[5], since this calorimeter has a high-order sensitivity

to (1 — rj) and a low-order sensitivity to mismatch

error. We have used a calorimeter based on the Engen

design to measure the effective efficiency of X-band and

Ku-Band thermistor mounts at a sufficient number of

0.04-

12 f (G Hz)

Figure 2. Effective efficiency (77) of (a) an X-band

waveguide thermistor mount; {b) a Ku-band waveguide

thermistor mount.

calibration frequencies to permit linear interpolation

(see fig. 2). The calibration time was lengthy, however, and we are currently constructing an X-band

calorimeter of modified design with a view towards re- duction of the time involved.

1-7,

0.06-

20 f (G Hz)

6. References

[1] Woods, D., A precision V. H. F. twin-T dual admittance

bridge and associated standards (unpublished report. Ministry of Supply, London, England, Jan. 1950)

[2] Adam, Stephen F., A new precision automatic microwave measurement system, IEEE Trans, Instr. Mear. IM-17,

308-313 (Dec. 1968)

[3] Jurkus, Algirdas, A semi-automatic method for precision measurement of microwave impedance, IEEE Trans. Instr. Meas. IM-18. 283-389 (Dec. 1969).

[4] Larson, Wilbur, Inline waveguide attenuator, IEEE Trans. Microwave Theory Tech. MTT-12, 367-368 (May 1964)

[5] Engen, Glen F., A refined x-band microwave microcalorimeter, J. Res. Nat. Bur. Stand. (U.S.), 63C (Eng. and

Instr.) , No. 3, 77-82 (July-Sept. 1959)

119

NCSL 70

DESIGN AND DEVELOPMENT THROUGH METROLOGY

Prem Prakash

National Physical Laboratory, New Delhi, India

The National Physical Laboratory of India, in implementing the 1956 Standards of Weights and

Measures Act, adjusted and calibrated mass standards of 50 to 10" kg for 250 State Laboratories. Most of the 5000 balances needed by Inspectors were designed and fabricated by four commercial

manufacturers in India, with specifications, technician training, and performance testing supplied by NPL. With this experience, two manufacturers designed and fabricated the higher precision balances for District-level laboratories. The technique is recommended to developing countries faced with similar projects.

Key words: Designing to specification; fabrication of balances; legal metrology; mass standards; NPL of India.

1. Legal Metrology in India

The Standards of Weights and Measures Act was

passed in 1956 and thus, after many centuries, India once again had a uniform system of measurement

throughout the country. The establishment of the standards of weights and measures was made the re- sponsibility of the Central Government, whereas the enforcement of various provisions of the Law was made

the responsibility of all the State governments.

To organize the enforcement machinery simultaneously in about 25 States and Union Territories was no

simple matter, particularly because industry in the country was still in a developing stage. With proper

planning and phasing of an implementation program, it became possible to procure most of the required equipment from local sources only.

At about this time, through a happy coincidence, the Weights and Measures Division of the National Physical

Laboratory of India had been expanded in a big way by

equipping it with a number of important measuring in- struments and other important items of equipment.

Most of these were obtained through the courtesy of the Technical Cooperation Mission of the United States of America under Agreement No. 26. The Laboratory

was, therefore, in a position to take up all technical work relating to the implementation of the 1956 Act and

was, in fact, associated from the beginning with the Central Ministry concerned and all the State depart- ments. The NPL's first assignment was to help in drafting

the 1956 Act. The next important assignment was to work out the shapes and dimensions of various grades

of standards (reference, secondary, and working), as

well as commercial grades of new weights (requiring some four grades), length measures, and volume measures. One of the chief requirements for all the new

weights and measures was that these should possess

distinctly different shapes from all those already in use

in the country. The third important assignment for the Laboratory was to recommend suitable limits of permissible errors for all categories of weights and measures, from reference standard grade down to the coarsest commercial grade. The fourth assignment was to pre- pare about two dozen demonstration sets of metric

weights and measures, in accordance with the new

shapes and dimensions, for exhibition in different States and Union Territories. This was done with a view to familiarizing the general public and the manufacturers

with the new standards. Also manufacturers were

given all necessary help to enable them to go ahead

with the manufacture of the standards. All this work

took a couple of years after the enactment of the new

law.Another important assignment for the Laboratory was to provide necessary training in high-precision tests and measurements to senior officers and to the Inspectors of Weights and Measures responsible for enforce- ment work in the States. This work is of a continuing

nature and has been going on intermittently all these

years. About 600 officers have already received this training.

2. Balances for the Enforcement

Laboratories

The question of equipping the State Enforcement

Laboratories was looked into along with other jobs. It was estimated that about 250 laboratories with about

1000 Inspectors of Weights and Measures would be re- quired for effective enforcement of the Act. These

laboratories were to be provided immediately with all necessary standards, measuring equipment, and bal- ances for the verification of commercial-grade weights

from 50 kg down to a milligram. It was decided that the weights would be manufactured by the India Gov121
ernment Mint. The assignment given to the Laboratory

in connection with these standards was to adjust and

calibrate them and to issue certificates for each item

individually before they were supplied to the State departments.

Then came the question of providing some 5000 bal- ances to these State laboratories. The possibility of importing these was ruled out, as this required a foreign

exchange of about 25 million rupees. To make a quick

start, however, it was considered necessary to import

about 1000 balances. The purchase of the remaining

balances was deferred with a view to trying to get these manufactured in our country. A committee, with the author of this paper as the convener, was set up in 1958

to look into the possibilities of getting the required type

of balances made in India. Taking all the relevant factors into consideration, the committee decided that a set of balances should consist of 50-kg, 5-kg, 200-g. and 2-g capacity balances. Broad specifications drawn up for these balances are given in table 1. Two types of balances were required:

(i) Indoor-type balances, to be permanently set up

in the laboratories. All four balances in the set were

required to be provided with the usual type of glass case.

Table 1

Approx. Work Load on

Sensitivity Beam each Balance

Capacity Reciprocal Length (Nos. of Denom.

mg/div. 50 kg 100 About 750 mm 3

5 kg 10 250-300 mm 4

200 g 1 150-200 mm 6 2 g 0.02 120-150 mm 11

( ii I Portable balances, for outdoor use. These

should be easily dismantled and secured in their carry- ing cases. They should be sufficiently robust to remain

undamaged even when the packages were transported over rough roads. They should be easily set up at camp

laboratories. The 50-kg balance was not to be pro- vided with the usual type of supporting pillar but with

a sufficiently rigid foldable tripod stand for suspending

the balance. No glass case was required for this bal- ance. After laying down these specifications, the project was entrusted to the author, to be tackled by the Laboratory in the manner considered best. Advantage was

taken of earlier experience regarding Indian balances. The laboratory had had occasion to examine a few lo- cally manufactured chemical balances some years earlier. (Small-scale production of such balances had been

going on in the country from the beginning of this century.) On testing these balances it was found that, though these were all right for college purposes, they

lacked many characteristics needed for precision work.

Letters were then sent to the leading manufacturers

telling them that the Laboratory would be able to suggest improvements in the quality of their balances pro- vided they sent them to us for close and critical examination. Those manufacturers who responded to our

offer did benefit by our advice, and their balances were

subsequently considered superior to those of others. Our general finding was that, although the manufacturers had some of the finest craftsmen on their staffs, proper technical guidance was sometimes lacking.

The representatives of these manufacturers were

trained in making good-quality knife edges and bearing

planes for precision balances, as well as in carrying out various performance tests on these. They were asked

to give complete tests to the balances before sending

them to us. Based on our earlier experience with local balances we asked them to take special care with re- spect to

1. choice of proper material for the construction of knife edges and bearing planes; proper hardening of

the knife edges where steel was used.

2. accurate lapping, correct positioning, and rigid

mounting of the planes and knife-edges.

3. adequate rigidity of the beam, so that there was

not much variation with load in the sensitivity of the balance. The procedure planned was that the manufacturers were to fabricate sample balances to the best of their

ability and submit the same to the Laboratory for criti- cal examination. This would enable us to locate any

defects. Necessary advice would then be given for the elimination of these defects in their subsequent attempts.

This process would continue until their balances came

up to the required standard. Within a few months the manufacturers were able to work out details of their designs, fabricate the balances,

and submit their first models for test. Detailed examination of these balances was carried out with a view to discovering any design or constructional defects, such

as in overall workmanship, rigidity of the fixing ar- rangements for the planes and knife edges, and the proper positioning of these. The following tests were

carried out:

1. Determination of sensitivity and period of swing

under (i) no load, (ii) half load, and (in) full load

conditions.

2. Consistency of performance, i.e., the effect of re- leasing, arresting, and re-releasing the beam a number

of times, and noting down the rest point. Some 20

observations were made in succession.

3. Accuracy of weighing. Rigidity of the beam,

proper hardness, and accuracy of the lapping of the knife edges, etc.. were inferred from the results of test. 4. Determination of the relative lengths of the arms.

5. Stability of rest point over a long period of time

( at least two to three weeks) and under varying conditions of temperature.

6. Uniformity of the rider scale, if provided.

The results of this experiment were very encouraging,

as some manufacturers were successful in the very first attempt to produce nearly the required quality of in- strument. Representatives of the manufacturers were advised to be present during tests. They were given neces- sary instructions for further improving the performance

of their balances. They then produced their second

models, which were an improvement over the first ones. The whole process of testing and giving instructions to the manufacturers was repeated, and they were asked to prepare yet another model. This continued until they

122
were able to produce the desired quality. Out of the six manufacturers who participated in the project, two

gave up after making two or three unsuccessful at- tempts. Of the remaining four, three succeeded in producing the desired quality after making three or four attempts. This was within a year and a half of

starting the project. The fourth manufacturer took

somewhat longer, but he too qualified after making a few more attempts.

These four manufacturers met the entire demand anticipated for balances for the State enforcement laboratories within the next three years or so.

3. Higher Precision Balances

As the enforcement laboratories became more ad- vanced in their work, they required more sophisticated

items of equipment. They now required balances with

higher accuracy and sensitivity which would be suitable for the periodical verification of working standard

weights against the secondary standards. These bal- ances were to be installed at the District-level laboratories. Taking into account the permissible errors on working standard weights, it was decided to have five balances in the set, with broad specifications as shown

in table 2. These specifications were sent to the four manufacturers who had been successful in producing balances

for the use of Inspectors. They were asked if they were

willing to follow the same old procedure for producing

the more precise type of balance. Two manufacturers

who possessed the experience of manufacturing a larger

of number of balances than the other two, and had con- sequently gained more confidence, agreed to take up the development of these balances. The same procedure was followed, and within one

year these balances also were successfully produced by

both manufacturers. Production is in full swing now, and some 60 sets have already been produced and supplied to' different States.

Table 2

Approx. Approx.

Minimum Approx. Width Clearance

Sensitivity Scale Beam Across Above

Capacity Reciprocal Division Length Pans Pans mg/div. mm mm mm mm

20 kg 25 2 750 300 700

5 kg 7.5 1.5 350 175 325 1kg 1.5 1.0 250 135 225

50 g 0.4 1.0 125 60 100

2 g 0.02 0.75 110 50 70

Within the next year or so, work in connection with

balances of still higher precision will be undertaken.

These balances, which will be installed at the State Headquarters Laboratories only, will be employed for

the periodical verification of the secondary standard

weights against the reference standards. Specifications

for these balances will be considerably more rigid than

those for the two types already produced. These bal- ances will be required to be fitted with an optical lever

system for increasing their sensitivity. The accuracy

requirements, as well as the minimum requirements for

the equality of arms, will also be prescribed. Making

of these balances will be a somewhat tougher job. However, with the confidence the two manufacturers have now gained, they should be able to make these.

4. About the Method

The method which a research laboratory would have

adopted in such a situation would be to put a large

number of designers on the job, get the trial models

made, and put these to practical tests. Depending on

the results of tests, the designs might require further modification. After incorporating all the desired re- quirements in the finalized design, a prototype would

be constructed to hand over to the industry. It was

realized that, with the resources at our disposal, or for

that matter at the disposal of any medium-sized laboratory, and to work simultaneously in many varied fields,

it might not be possible to put a large team of designers

and mechanics on any single project. It was estimated

that if we adopted this method it would require at least

five years to produce any tangible results. Circumstances compelled us to think of an inexpensive method which would give the desired results within

a short time, preferably a year or so, because the implementation program could not wait five years. The

method followed by us as described above has certainly

been efficacious. Essentially, the detailed design work

is done by the manufacturers, with necessary guidance

provided to them from time to time on the basis of tests and precise measurements on their product by the col- laborating research institute. This method has proved

its merits on two occasions already. It resulted in the saving of considerable foreign exchange and in estab- lishing a flourishing balance industry in the country,

and it has proved to be a promising technique for handling design projects. Together, the research institute and the industry have thus been able to achieve easily

what each individually might have found quite difficult to tackle.

It is felt that this method, although employed out of

necessity, can be used by developing countries with considerable benefit, by adapting it for any similar project.

The advantages offered by this method are:

( 1 ) It saves considerable time in the successful completion of a project for the following reasons:

f a ) By use of the facilities of a number of manufacturers, a number of designers and precision mechanics

much greater than even a big laboratory could afford

to employ can be associated with the project.

(b) Involving a number of manufacturers in the same project creates a spirit of competition, and each

manufacturer tries to achieve results more quickly than

the others. (2) It assures a good design for the product for the following reasons:

(a) The manufacturers' designers as well as mechanics who participate in the project can do the job

better, because of association with the trade concerned,

than can the general run of designers and mechanics.

( b ) Since the design was not frozen by the re123
search institute, each manufacturer uses his ingenuity

to make his product better than that of his rivals.

3. It reduces cost very considerably

(a) Instead of spending on the project, the research

institute may even earn test fees by testing an item each

time it is received from the manufacturers.

(b) Manufacturers will plan their design and fab- rication work in the most economical manner by suit- ably fitting this work into the rest of their program.

In this way they incur little extra expenditure.

Figures 1 and 2. Two views of the NPL balance-testing section.

124
NCSL 70

TELECOMMUNICATION MEASUREMENTS FOR

AUTOMATIC PRODUCTION AND MAINTENANCE

Dr.-Ingr. H. Fleischer

Secretary General, Verband Deutscher Elektrotechniker, Frankfurt, W. Germany

Automatic recorders with uncertainties no greater than 0.1 dB up to 18 MHz are used in one

channel of a microwave link to measure effective equivalence and noise level to provide production

calibration control for filters and attenuators. Details of two German-made systems are described. Key words: Automatic noise-level recorders; calibration of carrier-frequency filters; microwave

transmission lines; Siemens-Halske AEG; Wandel & Goltermann.

The progress of developing techniques very often depends on communications. All prognoses for tele- communication expenditures, especially those of the

Bell System, show the necessity for a very great number of links in the national, as well as in the inter- national, field. Beside the quantity of links, their quality must also be considered. It is known that each circuit able to be

linked with other circuits for public use should hold

special requirements elaborated by the International

Consultative Committees (CCI) of the International

Telecommunication Union. It may be that in the United

States of America it will not be necessary to follow these requirements because only a small number of links will be connected with international lines. In Germany,

however, we are in a quite different position. Our

country is very small. Practically 100 percent of our

traffic is automatically operated by the user. This is true also for international, and recently was inaugurated

for intercontinental, service. We are surrounded by

many other small countries. All the very heavy tele- communication traffic between these countries is switched automatically between countries and is nor- mally considered as foreign exchange. Therefore, the quality of the circuits has to fulfill the CCI require- ments. The measurements should be valid for micro- wave links as well as for carrier-frequency cables. The very rapid increase in the number of lines subject to the necessary high quality controls is the reason

for developing remotely controlled automatic measure- ment equipment for production and acceptance tests, as well as for maintenance. It would be impossible to get a sufficient staff to run the systems on any other basis. The ever-increasing demands for quality of telecom- munication-transmission systems require the use of high-precision measuring equipment. In addition to

this, the measurements must be made in the most simple,

exact, and time-saving manner, preferably automatically.

It seems unnecessary to mention the automatic noiselevel recorder for checking the statistical values of noise

in specific circuits. In the cable or microwave link of a multichannel carrier-frequency system, one channel is used as a measurement or control channel. Then it is possible to count and to print the median noise-level values for one hour, for one minute, or for 5 millisec- onds. The measurement limits are between 100 and

100,000 pW. This equipment works together with a small computer to mark the values of out-of-tolerance

limits and to indicate if the overall system is still usable. On production and acceptance tests, the carrier- frequency filters take most of the testing time because

each filter must be checked at many frequencies. The

filters these days are mass produced with very high ac- curacy, about 0.1 dB at special frequency points.

There are two telecommunication measuring systems

available in Germany. One is manufactured by Wandel

& Goltermann, the other by Siemens. Both systems are

especially designed for easy operation, remote control,

programmability of all essential operating functions,

and possible use as an automatic level-measurement

system. Some years ago it was adequate to measure

errors of 0.5 dB but the progress of carrier-frequency

technology now requires measurement errors no greater

than 0.1 dB at frequencies up to 12 or 18 MHz.

The assembly manufactured by Wandel & Golter- mann is constructed principally for laboratories and in- cludes a level generator. The generator and the level meter are synchronously tuned from a control oscillator,

which allows decade-frequency settings in steps of 1 Hz

between 200 Hz and 1,999,999 Hz. The absolute ac- curacy, according to the crystal, is 2 X 10"8 after one hour of service, and the transient time for varying the frequency is smaller than 100 fis. The generator output

level can be selected in steps of 0.1 dB in the wide range

of —69.9 to 9.9 dB, indicated as digital readout. This

output level has a measurement error of 0.04 dB and, at special calibration points, about 0.01 dB.

125
The same accuracy applies also to the level measure- ment appliance. The manufacturer uses transformers

for these precise steps, which are frequency-independent

over a wide range. The required attenuation is de- termined exactly by the turns relationship of trans- formers and is not subject to aging. The transformers

are connected together by electronic means. With a program control unit, the set becomes suitable

for high-precision level and attenuation measurements

in development laboratories, construction plants, and

test departments as well as for monitoring transmission

systems in telecommunication technology. All essential adjustments on the individual instruments of the level- measuring set, and ultimately on the test object, can be remote controlled. The high measurement rate ( ap- proximately one measurement point per second ) results

in great time savings during prolonged measurement

runs without affecting the high measurement accuracy

offered by the set. Since it is only necessary to ex- change program tapes in order to reprogram the meas- urement system for other measurement tasks, even

small production lots and short measuring runs can be

carried out economically.

It is possible, during the evaluation process, to compare the measured test results with tolerances given on

the program tape. The test-point number, the classifica- tion of the sample, or a tolerance violation are directly

indicated on the front panel and/or recorded by tele- printer, electric typewriter, or tape punch. On the tape, any 5- or 8-unit code can be used, with a maximum

reading speed of 1000 characters per second. The

reading time for an average program will be approximately 0.5 s. The control of the various attenuators and the powerlevel calibration are made automatically with the aid of the control unit. The power level available at the out- put terminals, therefore, is always referred to the se- lected output-impedance value. The operation is really

simplified, since the calibration occurs automatically

during the measurement process. The automatic measuring system can be matched to a variety of measure- ment tasks with the aid of further accessories. To illus- trate the test speed, an acceptance test of a group filter can be carried out in about 2 min. The program can be

stopped after any time interval. The other manufacturer, Siemens, has chosen a sys- tem with a great number of special appliances which can

be combined with a large number of measuring sets ac- cording to the tasks. The main appliances are still the generator, the level measuring set, the switching device,

the attenuator, and all the different sets for remote control and display, available either by typing in any form

or by showing on a screen. The accuracy is as good as the equipment mentioned first. Special efforts were made for the switching device,

which is able to connect the appropriate units with the

different devices either manually or automatically up to a frequency of 2 MHz, with coaxial impedances of 75

or 100 fl and with symmetrical impedances of 15 or 600 H. In the range up to 100 MHz only the 150-fi

impedance is used. The system is constructed using new techniques for precision and for easy operation.

Another most important part of the Siemens assembly is the variable attenuator, of a really high accuracy, which combines a rather large number of single ele- ments. These are specially designed resistors on glass

carriers, constructed by a new technology, which do not change their values in time, space, or with frequency.

The steps used in the attenuator are also 0.1 dB, but each element is calibrated to a maximum 0.003-dB

error. You should note that this is not a calibration in a laboratory but is valid in the production stage. The

set can be used for many purposes—in the very early

stage of newly developed equipment, to survey the out- come of production, to register the relevant statistical endurance, and also in the field of research.

In tin- neai Inline the enlargement of the national telecommunication systems will be aided also by a rapidly increasing number of data-exchange lines with

wider bands and with increased transmission require- ments. Last but not least, we are standing at the beginning of the Picturephone era. This will bring us larger

networks with still wider bandwidths.

The German postal service, PTT, like that of other

nations, was confronted with these problems and with

the lack of staffs trained for this job. It was therefore necessary to develop an automatic system and to measure all the different lines from only a few measurement

centers. This system is named "Automatische Mess- wert-Ubertragung," meaning automatic transmission

of measured values. It operates in the following manner. A code number must be given to each line or channel—not to the whole transmission link—in order to

select this specific channel by remotely controlled auto- matic switches, separated from the normal switches in automatic exchanges. Remotely controlled, highly precise measurements are possible in the field of noise level, effective equivalence, frequency response, group delay,

effective attenuation of termination circuits, and instability of emplifiers by measuring the return loss. All these measurements are remotely controlled, so that it is not necessary to have a person at the far end of the

line. The remote control unit and the line selector will not be discussed here, but examples of the measurement

principles will be mentioned.

D FM LM

p D LM M

Figure 1. Measurement of effective equivalence.

C = control unit; R = resistor (600—ft); G = generator;

M=branch to be measured; LM= level meter; FM= frequency modulation and demodulation; D=decoder; P=printer/display; A=automatically controlled amplifier.

126
As shown in figure 1, the measurement for the effec- tive equivalence is done first for the transmitting direction. The generator is switched to the near end and the

level measurement equipment to the far end. The re- sults are coded and transmitted by frequency-modulated

signals via the second (receiving) end of the line. The measurement of the effective equivalence of the receiving side of the line also is done by control at the near end. Instead of the level measurement equipment,

an amplifier with automatic gain control will be

switched to the far end of the transmitting line. This

amplifier transmits all controlled signals to the far end

of the receiving branch so that this side can be measured. The level meter is then switched to the far end of the receiving branch so that this side can be measured.

The level meter is then switched to the near ( receiving

end of the line. For the noise level measurement ( fig. 2 )

, the trans- mitting near end is switched to a resistor (600 ft) and

the far end of the transmitting branch is connected to the level measurement equipment. The values are trans- mitted, as before, over the receiving branch and are displayed in the same way. To measure the noise level on the receiving branch, the far end of this branch is switched to the balance resistor ( 600 ft ) , and the level

LM y p D FM FM

p D LM

Figure 2. Measurement of noise level. decontrol unit; R =resistor (600-0* ; G = generator;

M=branch to be measured; LM= level meter; FM= frequency modulation and demodulation; D=decoder; P=printer/display; A=automatically controlled amplifier.

is measured at the near end. The transmitting branch

is used only for remote control. The high precision necessary for reliable measure- ments makes it necessary to incorporate in the program

a surveillance of the automatic controlled amplifier.

This is done (fig. 3) in the following way. The generator at the near end works to the amplifier at the far end. The far-end level measuring equipment is connected directly to the output of the amplifier and trans- mits the results in the usual manner to the control unit. This check of the accuracy of the controlled amplifiers

becomes even more important when a line to be measured does not end at the measuring center but must be

switched to the measuring center through one or two

other lines.

D FM ^FM LM

Figure 3. Calibration control C=control unit; R =resistor (600-fi); G = generator;

M=branch to be measured; LM= level meter; FM= frequency modulation and demodulation; D=decoder; P=printer/display; A=automatically controlled amplifier.

All this equipment could be constructed only accord- ing to the new electronic technologies, which permit

equipment to be small, light, reliable, and precise. The

special design of transformer or resistor attenuators, together with automatic calibration and comparison

allows remote control of the whole measurement proce- dure in production tests as well as in acceptance tests. Hours of highly qualified staff members are saved for further development work. Hours of production are saved to supply the telecommunication companies with

the necessary appliances and components for the exten- sion of networks to serve the progress of economics.

Automatic maintenance enables the companies to install wideband transmission lines necessary for data exchange with the necessary reliability for this special purpose. Economical and technical progress are de- pendent, to a very great extent, on good communications

and better data exchange.

127

NCSL 70

PRESENT STATUS OF ELECTRICAL STANDARDS

AND THEIR TRACEABILITY IN JAPAN

M. Kanno

Standards Division, Electrotechnical Laboratory, Tokyo, Japan

The national electrical standards for Japan described herein are established and maintained in the Electrotechnical Laboratory. Dissemination is by two public agencies. Committees on traceability and on measuring techniques are active in the Institute of Electrical Engineers.

Key words: Absolute measurements; basic electrical standards; quantum electric standards; radio- frequency standards.

1. Introduction

The requirement for better standards and their traeceability to the national standards has been emphasized in Japan. This is most significant in electrical standards. The national electrical standards

(and those for photometry, acoustics, and ionizing ra- diation) were established and are maintained in the Electrotechnical Laboratory ( ETL ) , in this country.

Research on the standards has been continued in ETL.

The electrical standards are supplied throughout

Japan by two major public agencies and their branches.

Many activities have been continuously supporting the flow of the standards. Committees on traceability and

measuring techniques have been actively operating in the Institute of Electrical Engineers of Japan. A gov- ernment committee has been established to improve the connection between national standards laboratories and

industry. Improvement in international contacts re- garding this problem is desired, and establishment of an

organization similar to the NCSL is being considered.

2. Electrical Standards in the ETL

2.1. Basic Electrical Standards

(1) Resistance and voltage: Standard values of re- sistance and voltage are maintained by 10 resistors and

21 cells. They have been checked by regular international comparisons. Measuring accuracy of 0.01 ppm has

been obtained. From experience gained with extensive

testing of many standard cells, criteria for selection of standard cells for the bank of 21 cells were established. (2) Capacitance: A standard value has been de- termined absolutely by a cross capacitor with an uncertainty of 1 ppm. An improved cross capacitor is now completed. Good results were obtained in international comparisons of 0.1-/jF and 10-pF capacitors.

The excellent stability of the 0.1-juF capacitor at ETL

was recognized. Fused-quartz capacitors, and improved

fused-quartz tubular capacitors have also been used.

(3) Ac—dc comparison: Electrostatic and thermal

comparators have been tested. Differences of ac—dc

response have been measured with accuracy better than

25 ppm at 4 kHz, and 40 ppm at 20 kHz. Extension of frequency range is being tried. Preparation for the planned international comparison has almost been

completed.

(4) Automatic measuring system for standard cells: The system has been completed to monitor the standard

cells. It is composed of a scanner, digital-voltmeter and

a programmer, and can compare 200 cells with a reso- lution of 0.1 fi\. Presently, data are treated in a sep- arate computer but a complete on-line system is planned.

Extension of the system to other standards is also under

consideration.

2.2. Absolute Measurements

(1 ) Determination of capacitance by cross capacitor: Cross capacitor No. 2 has been completed. An accuracy

of 10~7 has been verified by tests. The capacitor is shown in figure 1.

Figure 1. Improved horizontal cross capacitor (No. 2).

129
l2l Evaluation of resistance by capacitance and fre- quency: Based on the capacitance standard, such measurements are being prepared, using an improved quadrature bridge.

I 3 I Evaluation of voltage by an electrostatic balance: A method of measuring force produced by an incre- mental energy change in a movable capacitance elec- trode system has been studied. A balance, with the electrode system and a measuring system for high volt- age are being constructed.

2.3. Quantum Electric Standards

( 1 1 Measurement of p': The value of p' (gyromagnetic coefficient of the proton ) has been determined with an accuracy of ±4 ppm. The free precessional fre- quency of a proton in water, in a calculable magnetic

field produced by a single layer air-core solenoid, is measured. A new system for a quantum standard of current has been planned this year. In this method a new device called the Magnetically Isolated Calculable Solenoid (MICS) will be used. The MICS has a very wide constant and calculable magnetic field which is not affected by ambient fiejd variations. If this method goes well, a quantum standard of current without the need for a non-magnetic environment can be

expected.

(2) Stark voltmeter: A quantum standard of voltage

by determining the Stark separation coefficient of particles has been studied. An accuracy of ±30 ppm was

obtained in 1969, by measuring the spectrum (20 GHz

of s-trioxane (C3 Hr, 03 ). The spectrum of methyl cy- anide (CH3 CN 10

) at 36 GHz is also being investigated

in an effort to obtain higher accuracy.

(3) Ac Josephson voltage standard: Research on a pure quantum standard of voltage, using the Josephson

effect in a junction of two superconductors, was started

in 1969. It now is in the primary stage of study on how

to make the tunnel junction.

2.4. Radiofrequency Standards

Research and development concerning rf measure- ments and standards was started at ETL in 1952. Since

then a great deal of effort has been expended towards

basic research and development of the national stand- ards for. and the measurements of, various fundamental

quantities in the radio spectrum. These high-frequency,

microwave, and millimeter-wave standards and meas- urements include such items as power, voltage, current,

TABLE I. RADIO FREQUENCY STANDARDS IN JAPAN 1970

WAVE LENGTH

FREQUENCYN

STANDARDS

HIGH FREQUENCY MICRO WAVE MILLIMETER WAVE SUB-MILLIMETER 30 m 20 10m 5m 3 2 lm 50cm30 20 I0,cm 5cm 3 2 Icm 5mm 3 WAVE

i I 1 i ± I | I J_ I i I I I l i I _L

10 MHZ I 00 MHZ IGHZ 20 30 50

_J I l_ 200 300 500I I I 2 3

10 GHZ 100 GHZ

5 20 30 50

J I I I I L _l_ <T""^1 hr'Wtfi'W/SgL

POWER ZEST

POWER < H

PULSE POWER

~2> COAXIAL

COAXIAL

NOISE

VOLTAGE

CURRENT

ATTENUATION

(cipmJ <

PHASE

SHIFT <:

IMPED- ANCE <^^A^:^v^ , i 1 > COAX.AL M.

PHYSICAL CONSTANT OF

MATERIAL DIELECTRICS

>

WAVEGUIDE

>

WAVEGUIDE

WAVEGUIDE >

WAVEGUIDE >

WAVEGUIDE >

STANDARD

DEVELOPED UNDER , DEVELOPMENT DEVELOPMENT TO

BE LAUNCHED o

INTERNATIONAL COMPARISON / N INTERNATIONAL COMPARISON COMPLETED OR UNDER WAY V / SCHEDULED

130
noise, attenuation, phase shift, and impedance. Their

present status is shown in table 1. Since 1959, ETL has made many efforts towards the achievement of agreement among the standards of dif- ferent nations. We have actively participated in the intercomparisons of such standards as 10-GHz waveguide power, 400-MHz coaxial power, 35-GHz milli- meter-wave power, 10-GHz noise temperature, 3-GHz

coaxial power, and rf attenuation at three different fre- quencies.

3. Traceability of Electrical Standards

in Japan

In Japan the traceability route is split in two, mainly

for historical reasons. The dc and low-frequency

standards are distributed by the Japan Electric Meters

Inspection Corporation (JEMIC), and the radiofre- quency standards by the Japan Machinery and Metals

Inspection Institute (JMI), as shown in figure 2, where

it is compared to that of USA.

The standards are compared through their rural branches to standards of other organizations as maintained in their own central standards laboratories. Measuring instruments of the organizations also are cal- ibrated at their standards laboratories. However, many

instruments are brought to the corporations directly

from smaller companies.

INTERNATIONAL BUREAU OF

WEIGHTS AND MEASURES

is

NBS

CALIB. CENTER

OTHER NATIONAL

STANDARDS LABORATORIES

RURAL RURAL

BRANCHES BRANCH

1

STANDARDS LABORATORIES - ORGANIZATIONS

STANDARDS ROOM OF ORGANIZATIONS

STANDARDS ROOMS ORGANIZATIONS

MEASURING INSTRUMENTS AT PLANTS MEASURING INST AT PLANTS

Figure 2. Traceability of electrical standards in Japan and the United States

3.1 Japan Electric Meters Inspection

Corporation (JEMIC)

The JEMIC is a public corporation established in 1965. The activities of the JEMIC are regulated by law. That is to say, the function of the Corporation consists

of three main activities, namely ( 1 ) type-approval test and individual inspection of watt-hour meters, ( 2 ) cal- ibration services for electrical standards and measuring

instruments, (3) research and investigations for elec- trical measurements. The calibration services for electrical standards and

measuring instruments were transferred from ETL to JEMIC. At present, ETL does not perform direct cali- bration services at dc and in the low-frequency region,

but JEMIC supplies the standards as maintained by

ETL to all the industries and the government offices in Japan. Moreover, ETL maintains only basic standards

such as standard resistors, standard cells, standard

capacitors, and ac-dc transfer standards. So, the JEMIC develops some practical standards independently, such as standard inductors, ratio sets, shunts,

voltboxes, instrument transformers, inductive voltage

dividers, standard watt-hour meters, etc. At present, JEMIC has 1300 personnel. Of these,

about 120 are employed on standards work both for its own use and calibration services. JEMIC covers the whole service area in Japan through fifteen branch

laboratories located all over the country. The JEMIC

calibrated 8700 articles by request in 1969.

3.2. Japan Machinery and Metals

Inspection Institute (JMI)

The JMI is a nonprofit public agency founded in 1957, subsequent to the Export Inspection Law. The

Department of Calibration of Electronic Measuring In- struments was established in 1963, as one of the agencies

to transfer the national standards of ETL to official and

private organization users. The main business of JMI is ( 1 ) calibration of standards and measuring instruments in the radiofre- quency ranges, and their adjustment and repair, (2)

general inspection of machinery, metals, electrical and

electronic products, particularly inspection of equipment for export, (3) safety tests by CSA and UL, and

spurious-radiation tests by FCC rules. The JMI has 550 persons and a branch in Osaka. Re- search for better calibration has been continued. Main

instruments for calibration are signal generators, hf ammeters, power and impedance meters, Q meters, dis- tortion-factor meters, wow-flutter meters, electrical standards and indicating meters.

4. Committees

4.1 Traceability Committee

In March 1968. the Institute of Electrical Engineers

of Japan set up the Traceability Committee to investi- gate, to discuss, and to solve the problem of traceability.

The members of this committee are the metrology sci- entists and engineers of ETL. of public standards laboratories, and of the users and makers of electrical and

electronic measuring instruments. The major activities of this committee are as follows:

( 1 ) Disseminating the research data concerning legal

standards and international comparisons.

(2) Informing committee members about the work

and the standards-supply systems of the national and

public standards laboratories,

(3) Collecting and disseminating information about

standards systems in other countries,

(4) Investigation of standards-maintenance systems

of the industries,

(5) Investigation of the conditions of standards- laboratory rooms,

(6) Standardization of the calibration methods of

electrical and electronic standards and instruments,

(7) Presentation of new instruments.

This committee meets regularly every month.

131
4.2. Precision Electromagnetic

Measurements Committee

This committee was established in April 1967, in the

Institute of Electrical Engineers of Japan, to exchange

novel techniques for precision measurements among

people of ETL, agencies of inspection, and professors

and engineers from the measuring-instrument industry.

Topics concerned with research and development basic to applications, and covering the range from dc to laser frequencies, are presented and discussed. The

committee meets monthly as a rule.

4.3. Governmental Committee on

Standards and Measurements

This committee was established in February of 1969. Members include people from the Ministry of International Trade and Industry, standards laboratories, in- spection agencies, institutes, universities, manufacturing and other industries, and measuring-instrument

makers. Seven branch committees for electricity, il- lumination, units, temperature, ionizing radiation,

length, and standard materials have been established.

Activities and objectives of the committee are:

1 1 ) Presentation of information from national and

international conferences and committees on standards,

(2) Exchange of information on national standards

laboratories and inspection agencies, and discussion of means to improve their operation,

(3) Collection of requests from industry to the laboratories and the agencies,

(4) Discussion of future general traceability systems.

The necessity for securing good traceability and for improving measuring techniques in the industry have

been pointed out. For this purpose, establishment of a new organization similar to the NCSL in the USA is being considered.

5. Conclusion

The importance of favorable supply and effective application of standards have been recognized in the in- dustry of Japan. Some activities have been established and operated for this purpose. However, such a trend

is seen to appear in many countries. They may have common problems to be solved. International contact

is very necessary and effective even if domestic conditions are different. Methods of traceability as well as the standards themselves must be international to realize their full value.

132

Ci U. S. GOVERNMENT PRINTING OFFICE: 1971 O—405-32)
Latest developments in the subject area of this publication, as well as

in other areas where the National Bureau of Standards is active, are

reported in the NBS Technical News Bulletin. See following page.
HOW TO KEEP ABREAST OF MBS ACTIVITIES

Your purchase of this publication indicates an interest in the research,

development, technology, or service activities of the National Bureau of

Standards.

The best source of current awareness in your specific area, as well as in other NBS programs of possible interest, is the TECHNICAL NEWS BULLETIN,

a monthly magazine designed for engineers, chemists, physicists, research

and product development managers, librarians, and company executives.

If you do not now receive the TECHNICAL NEWS BULLETIN and would like to subscribe, and/or to review some recent issues, please fill out and return the

form below.

Mail to: Office of Technical Information and Publications

National Bureau of Standards

Washington, D. C. 20234

Name

Affiliation

Address

City State Zip

| | Please send complimentary past issues of the Technical News Bulletin.

| | Please enter my 1 — yr subscription. Enclosed is my check or money

order for $3.00 (additional $1.00 for foreign mailing).

Check is made payable to: SUPERINTENDENT OF DOCUMENTS.

SP 335

U.S. DEPARTMENT OF COMMERCE

WASHINGTON, D.C. 20230

OFFICIAL BUSINESS

PENALTY FOR PRIVATE USE, S300

POSTAGE AND FEES PAID

U.S. DEPARTMENT OF COMMERCE