.4'ii \

%

v> J

NBS TECHNICAL NOTE 865

9<"H:AU Of

U.S. DEPARTMENT OF COMMERCE/ National Bureau of Standards

ritical Electrical

Measurement Needs and

1 for Modern Electronic

/oo

Uf753

9915
NATIONAL BUREAU. OF STANDARDS

The National Bureau of Standards 1 was established by an act of Congress March 3, 1901. The Bureau's overall goal is to strengthen and advance the Nation's science and technology

and facilitate their effective application for public benefit. To this end, the Bureau conducts

research and provides: (1) a basis for the Nation's physical measurement system, (2) scientific and technological services for industry and government, (3) a technical basis for equity in trade, and (4) technical services to promote public safety. The Bureau consists of the Institute for Basic Standards, the Institute for Materials Research, the Institute for Applied Technology,

the Institute for Computer Sciences and Technology, and the Office for Information Programs.

THE INSTITUTE FOR BASIC STANDARDS provides the central basis within the United

States of a complete and consistent system of physical measurement; coordinates that system

with measurement systems of other nations; and furnishes essential services leading to accurate and uniform physical measurements throughout the Nation's scientific community, industry, and commerce. The Institute ^consists of a Center for Radiation Research, an Office of Measurement Services and the following divisions: Applied Mathematics — Electricity — Mechanics — Heat — Optical Physics — Nuclear

Sciences " — Applied Radiation '-' — Quantum Electronics 3 — Electromagnetics 3 — Time

and Frequency * — Laboratory Astrophysics " — Cryogenics ". THE INSTITUTE FOR MATERIALS RESEARCH conducts materials research leading to improved methods of measurement, standards, and data on the properties of well-characterized materials needed by industry, commerce, educational institutions, and Government; provides

advisory and research services to other Government agencies; and develops, produces, and

distributes standard reference materials. The Institute consists of the Office of Standard Reference Materials and the following divisions: Analytical Chemistry — Polymers — Metallurgy — Inorganic Materials — Reactor

Radiation — Physical Chemistry.

THE INSTITUTE FOR APPLIED TECHNOLOGY provides technical services to promote

the use of available technology and to facilitate technological innovation in industry and Government; cooperates with public and private organizations leading to the development of

technological standards (including mandatory safety standards), codes and methods of test; and provides technical advice and services to Government agencies upon request. The Institute consists of a Center for Building Technology and the following divisions and offices: Engineering and Product Standards — Weights and Measures — Invention and Innovation — Product Evaluation Technology — Electronic Technology — Technical Analysis — Measurement Engineering — Structures, Materials, and Life Safety 4 — Building

Environment * — Technical Evaluation and Application * — Fire Technology.

THE INSTITUTE FOR COMPUTER SCIENCES AND TECHNOLOGY conducts research and provides technical services designed to aid Government agencies in improving cost effec- tiveness in the conduct of their programs through the selection, acquisition, and effective

utilization of automatic data processing equipment; and serves as the principal focus within

the executive branch for the development of Federal standards for automatic data processing

equipment, techniques, and computer languages. The Institute consists of the following

divisions: Computer Services — Systems and Software — Computer Systems Engineering — Information Technology.

THE OFFICE FOR INFORMATION PROGRAMS promotes optimum dissemination and

accessibility of scientific information generated within NBS and other agencies of the Federal Government; promotes the development of the National Standard Reference Data System and

a system of information analysis centers dealing with the broader aspects of the* National Measurement System; provides appropriate services to ensure that the NBS staff has optimum

accessibility to the scientific information of the world. The Office consists of the following

organizational units: Office of Standard Reference Data — Office of Information Activities — Office of Technical

Publications — Library — Office of International Relations.

1 Headquarters and Laboratories at Gaithersburg, Maryland, unless otherwise noted; mailing address Washington, D.C. 20234.

3 Part of the Center for Radiation Research. y

a Located at Boulder, Colorado 80302.

« Part of the Center for Building Technology.
"MBical Electrical Measurement Needs and

Standards for Modern Electronic Instrumentation '•K£«*L<«tnaL.-

Report of a Workshop Sponsored by the

National Bureau of Standards

Gaithersburg, Maryland

September 23-24, 1974

Edited by

Peter Richman, Consulting Engineer

Lexington, Massachusetts 02173

6

^

0F c°*

CO ^

U.S. DEPARTMENT OF COMMERCE

NATIONAL BUREAU OF STANDARDS, Richard W. Roberts, Director

Issued May 1975
Workshop on Critical Electrical Measurement

Needs and Standards for Modern Electronic

Instrumentation

Gaithersburg, Maryland, Sept. 23-24, 1974

Chairman: Barry N. Taylor, Chief, Electricity Division

Moderator: Peter Richman, Consulting Engineer

Chairmen, Working Sub-Groups

E. F. Boeckmann, Angstrohm Precision, Inc. D. E. Lawrence, Hewlett-Packard Company

D. R. Ludwig, Teledyne Philbrick

F. B. Seeley, U S Army, Redstone Arsenal

Contributing Authors

W. G. Eicke, B. F. Field, R. E. Kleimann, J. J. Morrow,

N. M. Oldham, R. S. Turgel, H. K. Schoenwetter, T. M. Souders

Electricity Division, Institute for Basic Standards

National Bureau of Standards, Washington, D.C. 20234

Library of Congress Catalog Card Number: 75-600022

National Bureau of Standards Technical Note 865

Nat. Bur. Stand. (U.S.), Tech. Note 865, 74 pages (May 1975) CODEN: NBTNAE

U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON: 1975

For sale by the Superintendent of Documents, U.S. Government Printing Office, Washington, D.C. 20402 (Order by SD Catalog No. 03.46:865).

Price $1.35
Abstract

Recognizing the proliferation of sophisticated modern electronic instrumentation in the field of electrical measurements, the Electricity Division of the National Bureau of Standards recently initi- ated a new program in the general area of dynamic measurements and standards in support of such

instrumentation. Recognizing further that the vastness and complexity of the field would require, at the earliest stages of the program, identification of the most critical problem areas, the Electricity

Division held a workshop on 23 and 24 September, 1974, at the Bureau's Gaithersburg site, to assist

it in ascertaining just what these areas in fact were. The basic idea of the Workshop was to bring

together a broadly representative group of some twenty-five leading manufacturers and prime users, working in a free and open atmosphere, in order to have them delineate the present and future critical support needs in the field of dynamic electrical measurements for modern electronic instrumentation, with emphasis on physical standards, standardized measurement methods, new calibration

and measurement assurance services, relevant data, and most important, new measurement methodologies. The overall objectives of the Workshop were generally met, and a number of significant

specific; programs and projects consistent with the mission of the Electricity Division were identified. Three categories broadly cover the needs as defined at the Workshop:

1. The need to introduce time as a measurement parameter has resulted from the requirements of automatic test and control systems. Specific areas considered critical are:

a. Pulsed component measurements.

b. Dynamic performance characterization for modern signal conditioning and data con- version devices: Digital-to-analog and analog-to-digital converters, sample-andhold amplifiers, comparators, etc. Required measurements include settling, aperture and acquisition times. Basic new capabilities will be needed, including precision, non- sinusoidal waveform generation and "standard" digital-to-analog converters.

c. Methodologies and techniques for characterizing precision AC and DC sources and

measurement devices with respect to settling time.

2. The emergence of measurements into the "real -world" or the production line via the

automatic system has introduced a host of new parameters. Areas requiring significant

effort include:

a. Investigation of transportable standards for validation of static and dynamic system

performance: AC and DC voltage, impedance, pulses, settling times.

b. Techniques for characterizing sources and measurement devices with respect to switched or dynamic loading.

c. Prediction of long-term performance, reliability and lifetime from short-term

evaluation for a host of passive components, semiconductors, and signal sources.

d. Noise standards and methodologies.

e. Continuing effort to improve transportable DC standards, and dissemination at higher voltage levels.

f. Characterization of the AC line voltage - waveforms, under- and over-voltages,

transients, dynamic impedance, etc.

g. Inclusion of a capability for environmental control and variation for devices and

instruments under calibration, in all new work undertaken by the Bureau.

h. Extension of Measurement Assurance Programs (MAP's), especially into the above new areas.

3. Recent work at the leading edge of measurement technology in electronics has resulted

in the need for new, or higher accuracy measurements. For example:

in
a. Increased dependence on capacitors as storage devices has made dielectric hysteresis a parameter of importance. Significant measurement work should be started in this area.

b. Phase difference measurements to extremely high accuracies are needed to calibrate

industrial instrumentation.

c. Higher accuracy power measurements, particularly for non- sinusoidal waveforms,

are needed to calibrate instrumentation used in the field. d. Non- sinusoidal, high crest-factor precision signals are needed to calibrate true RMS

converters and meters.

e. High accuracy electronic instrumentation requires reduction in AC calibration uncertainties to 10 ppm, at least over the audio range.

Longer-term, "frontier" areas in each of the above groups were also identified. They are not specified in the above listings since they involve work at still more basic levels, on ground that is

presently terra incognita. Nevertheless, they furnish insight into the overall technical/philosophic

climate, and are useful for that reason. Items in this category include:

1. Measurement problems in systems, including effects of history, measurement interactions, self- calibration, etc.

2. Evaluation and standardization of system software.

3. Calibration of medical equipment.

4. Replacement of standard cells.

5. Replacement of thermal AC/DC converters.

6. Non-swept but broad -response measurements - impulse, noise, other.

7. Basic work on the physics of dielectric absorption; component drift; device noise; failure

mechanisms; switches and relays.

8. Provision of improved measurement accuracy for virtually all dynamic measurements, at the wafer probe level.

In addition to the identification of these specific and general needs, the Workshop provided

strong evidence that both the manufacturing and user communities in the electronic instrumentation

field look to the Electricity Division to provide the technological leadership in these newer areas that

it has previously furnished in the more traditional electrical measurement areas. Indeed, there was

a good deal of support for the idea of keeping the electronics community informed of Division workin-progress, with emphasis on projects targeted by the Workshop.

Keywords: Data conversion; dynamic measurements; electrical measurements;

electronic instrumentation; signal conditioning; systems; time domain.

IV
Foreword

The National Bureau of Standards (NBS) is becoming increasingly aware that its measurement

services, related research and other activities must be oriented towards the solution of the measurement community's "real-world" problems if NBS wishes to make a significant contribution to meeting the country's needs in those areas containing a large scientific and technological component,

for example, increased productivity and foreign trade, environmental pollution control, improved

health care delivery, solution of the so called "energy crisis", etc. The Electricity Division particularly recognizes the increased burden this new awareness places upon it, especially in the area of measurement and standards support for modern electronic instrumentation, since such instrumentation is rapidly becoming all pervasive and a key element in the solution of our society's most critical technical problems.

In keeping with this point of view, and the current trend in electrical measurements towards

the wide utilization of electronic instrumentation, often in conjunction with digital computers, and

more often than not high speed measurements in the time domain, the Division has established over

the last year or so a nascent program in the general area of dynamic measurements and standards

for modern electronic instrumentation. Present activities include development of a computer-based

sampling watt- system; new methods for accurate phase angle measurements; calibration and support

methods for DC calibrators; characterization and calibration of solid state voltage sources and dissemination of the unit of voltage at levels higher than one volt; static and dynamic test sets for characterizing analog-to -digital and digital-to-analog converters; and automation of the Division's calibration services (capacitors, resistors, standard cells). Each of these projects will not only result

in an improved measurement method and/or measurement service, but will provide Division personnel with valuable, "hands on" experience in the field of modern electronic instrumentation, auto- mated measurements, and automated test systems.

It is important to realize that these new activities are consistent with and complement the Division's overall responsibility for establishing, maintaining and disseminating the electrical units in the United States, for ensuring that electrical measurements made throughout the country are reliable and consistent, especially with those made in other countries, and of advancing the state-of-theart, that is, of ensuring that useful and usable electrical measurement methods and standards are

available when needed.

It was clear at the time the new program was initiated that the work the Division was undertaking in this area was just a first step into a very extensive and complex technology. Indeed, the

field's scope and multiplicity of needs was found to be so vast that it was decided at the outset that extensive, broadly-based and critically evaluated input from the potential beneficiaries of the Division's work in this area had to be available before the program could be developed further. A workshop was thus planned to provide this input. The basic idea was to bring together a representative

group of leading manufacturers and prime users, including federal agencies, of electronic instrumentation (interpreted in the broadest sense) in order to have them identify the present and future

critical measurement and support needs in this field - physical standards, standardized measure- ment methods, new calibration and measurement assurance services, relevant data, and most

important, new measurement methodologies.

So that the Workshop would be conducted in a manner as free as possible from any biases we

in the Division might have, and to ensure that segments of the technical community generally unfamiliar with the Bureau were properly reached, the services of Mr. Peter Richman, a well known

consulting electronics engineer with many years experience in electronic instrumentation and a long

familiarity with NBS and especially the Electricity Division; were retained to assist in planning and

conducting the Workshop. Mr. Richman was also asked to prepare the final report on the Workshop,

and this document is the result. Its main thrust, as we believe it should be, is technical in nature. That is, it is primarily concerned with presenting the most urgent technical needs of the manufacturing and user communities in the field of dynamic electrical measurements and electronic
instrumentation as identified during the course of the Workshop. The identification of the field's

most important problems is extremely critical if the Division is to mount an effective program in such a broad and diverse technology; it would never be possible for the Division to address itself to each of the very large number of measurement and related problems which exist. Thus, one of the

main goals of the Workshop was to provide some sense of priorities and to delineate those most

critical needs which can and should be met by the Electricity Division. We therefore look to the

information contained in the report as basic input data for refining and further developing our program. Indeed, it is probably not premature to state that the technical needs identified during the

Workshop and documented here will form the corpus of the Electricity Division's dynamic electrical

measurement and standards work in support of modern electronic instrumentation over the next five to ten years.

I should like to take this opportunity to personally thank all of the Workshop attendees, and

their organizations, for making the time available in their busy schedules to participate. I hope each

of them found the experience to be as rewarding for themselves and their work as it was for the

Electricity Division and its staff. I also wish to extend a special thanks to the four Sub-Group Chairmen for their extra time and effort and for a job well done. Finally, I should like to acknowledge the

outstanding work of Mr. Peter Richman - it was to a very large extent his involvement in the planning and running of the Workshop which made it so successful.

Barry N. Taylor, Chief

Electricity Division

Institute for Basic Standards

VI
Preface

It is apparent that the continued, orderly growth of the electronics industry is increasingly

hampered by a lack of standards and methodologies in a number of crucially important electrical

measurement areas.

The explosive growth of electronics during the last decade has brought with it a host of unsolved

measurement problems. Yet at this critical historical juncture, electronics is a key technology in

the prospective solutions to some of society's major outstanding concerns including energy, pollution, and productivity to name but three. It is, therefore, mandatory that a responsible, capable

organization without specific industrial affiliation, assume a leadership role in providing the necessary measurement standards and methodologies, as well as in establishing new electrical measurement programs for which need is both broad and identified, but whose implementation by specific

industrial organizations would be impractical.

The Electricity Division of the National Bureau of Standards has traditionally dealt with the

measurement and generation of electrical signals in the DC to 30 kHz range, as well as with the

properties of electrical components in that range. In effect, with the one possible minor (philosophical) exception of phase, the Division has devoted itself to amplitude or magnitude measurements for

the various electrical parameters* Some of these have been generated and/or transmuted by

mechanical and electro-mechanical means, others more recently by electronic, and others in future

undoubtedly by light, but the electrical (usually amplitude) parameters have had similar properties

and required the same disciplines for measurement.

/Now, automatic systems for test, process control, medical applications and so. forth are using

electrical quantities at higher speeds than heretofore. Ten years ago a precision DC, dialable six- or seven-digit source did not need a settling time specification; today "programmable" sources have

such specifications, since the system architect must know how long it will take for this stimulus to

settle to within a given accuracy. In many instances total system speed, hence all-important

throughput, dollar payback and/or justification, will be limited by such settling times.

Thus to precision electrical measurements has been added the temporal parameter. In a manner analogous to the changeover from frequency to time domain work in filter synthesis in the early * sixties, measurement must now turn from exclusive attention in the frequency domain, to characterization involving time as well, e.g., settling time to x microvolts in y microseconds is an electrical, analog, amplitude measurement with the time factor added.

The Workshop which this report describes and summarizes, is the means chosen by the Electricity Division to establish direct communication with key elements of the electronics industry

among both manufacturers and users, for the purpose of ascertaining the industry's electrical measurement needs, both general and specific. The enthusiasm engendered by the Workshop itself attests to the fact that the time has come for leadership and action: In spite of difficult times and

severe cost pressures, twenty -four outstanding organizations were willing to send top personnel to the two-day Workshop, disrupting busy schedules, and at considerable financial expense.

This report on the Workshop results must inevitably reflect some of the author's technical

biases, developed over the course of twenty-five years of varied design, management and consulting

experience in the fields of electronics, measurements and standards. During that interval the technology has made tremendous advances, so that in many areas it is virtually unrecognizable in comparison with its major jumping-off point directly following the Second World War. Indeed, some of

those advances are the direct result of work accomplished at or backed up by programs of the Electricity Division of the NBS, notably during what might be termed the measurement "explosion" of the

decade of the sixties.

VII
By the same token, it is hoped that the author's general breadth of design and measurement

exposure during those same years of experience, plus long-standing and continuing interactions with

the Bureau, have provided a balanced view of the Workshop results in this report. If this has been

accomplished, the Division programs resulting from the Workshop can form the basis for new and

crucially-important electrical measurement technology for electronics in the forthcoming decade.

It seems clear that considerable amounts of the expertise already in existence in the Electricity Division can be brought to bear on the electrical measurement needs of modern electronics, as

identified in the Workshop results. This is particularly true in the areas of measurements with time

as an added parameter, and measurements related to the whole field of automated systems. Overpoweringly evident, as a result of the Workshop, is the fact that an entire range of basic unsolved

problems exist. Also clear is that they are of increasingly critical importance to continued progress, and that manufacturers and users alike look to the Division to supply the necessarily broad,

technological base for solutions.

It is impossible to convey here the excitement felt by virtually all participants during the Workshop which helped to make the author's task as moderator both rewarding and enjoyable. The excitement was undoubtedly due in part to the professional challenge offered by the opportunity for open,

free-swinging exchange among the outstanding group of participating engineers and scientists. But it clearly reflected, as well, the participants' recognition of the large and important work that so critically needs to be done, and their enthusiastic reception of the role of the Electricity Division in

grasping the initiative to establish relevant new programs on a priority basis. There was, in short,

the excitement resulting from the pervasive feeling of being part of an undertaking that was both significant and effective.

I should like to express my thanks to the Electricity Division and to Dr. Barry N. Taylor in particular, for giving me the opportunity to contribute to this important work. I should also like to add my own personal thanks to the participants and their organizations, as it is they who are truly

responsible for the Workshop's success.

Peter Richman

Consulting Engineer

Vlll
Workshop Attendees

Mr. Loyle E. Baltz

Principal Test Engineer

Ford Motor Company

Allen Park, MI

Mr. Eduard F. Boeckmann

Chief Engineer

Angstrohm Precision, Inc. Hagerstown, MD

Mr. Warren C. Collier

Manager, TM 500 Design, Engineering

Tektronix, Inc. Beaverton, OR

Mr. Thomas J. Coughlin

Product Engineering Manager

Component Systems

General Radio Company

Concord, MA

Mr. Robert C. Raybold

Head, Institute for Basic Standards

Automation Group

National Bureau of Standards

Washington, D.C.

Mr. Bruce A. Renz

Head, Systems Measurement Section

American Electric Power Service Corporation

New York, NY

Mr. David B. Schneider

Calibration Product Specialist, ATE

John Fluke Manufacturing Company, Inc. Seattle, WA

Mr. Frederick B. Seeley

Chief, Electromagnetics Engineering Branch

U. S. Army Meteorology and Calibration Center

Redstone Arsenal, AL

Mr. Jack D. Dougherty

Chief, DC & Low Frequency Standards

Laboratory

Newark Air Force Station, OH

Mr. Robert M. Shaw

Fellow Engineer

Westinghouse Electric Corporation

Baltimore, MD

Mr. George W. Herron

Senior Engineer

Keithley Instruments, Inc. Cleveland, OH

Mr. Lewis R. Smith

Engineering Manager

Analog Devices, Inc. Norwood, MA

Mr. Donald E. Lawrence

Project Manager

Hewlett-Packard Company

Sunnyvale, CA

Mr. Douglas C. Strain

President

Electro Scientific Industries, Inc. Portland, OR

Mr. David R. Ludwig

Vice President

Advanced Development and Planning

Teledyne Philbrick

Dedham, MA

Mr. Edward P. Mueller

Biomedical Engineer

Food and Drug Administration

Rockville, MD

Mr. William A. Plice

Senior Project Staff Engineer

Honeywell Inc.

St. Louis Park, MN

Mr. Robert P. Talambiras

Corporate Director of Advanced Development

Analogic Corporation

Elmwood Park, NJ

Mr. David S. Terrett

General Manager, Research & Development

ADT Security Systems

New York, NY

Mr. James C. Tripp

Engineering Supervisor

Linear Test Development Group

National Semiconductor Corporation

Santa Clara, CA

IX
Mr. Robert H. Verity

Manager, Standards Laboratory

Leeds & Northrup Corporation

No. Wales, PA

Mr. William R. Walters

Chief Engineer

Codi Semiconductors

Fairlawn, NJ

Mr. F. Mansfield Young

Director of Engineering

Teradyne, Inc. Boston, MA

Mr. Ray A. Zuck

Director of Operations

Geometric Data Corporation

Wayne, PA
Contents

Page

Abstract iii

Foreword • v

Preface , vii

Workshop Attendees ix

Contents xi

1. Introduction 1

1.1 Background and Objectives 1 1.2 Participants and Industry Distribution . 2 1.3 Workshop Agenda and Organization 2 1.4 Data Sources and Data Reduction 6

2. Results and Conclusions

2. 1 Overview 9

2.2 Specific Results 10

2.2.1 Broad Program Definitions 10

2.2.2 Specific Project Activities . 15 2.2.3 General Long-Term Directions 16

2.3 Conclusions 17

Appendix A: Data Sources 18

A. 1 Summary of Attendees' Pre-Workshop Input 18

A. 1. 1 Components and Parameter Measurements 18

A. 1.2 Signal Conditioning and Data Conversion . 20

.A. 1.3 Sources and References 21

A. 1.4 Systems 23

A. 1.5 Other Input, Mostly Having to do with Calibration and Product Testing . . . 26

A. 1. 6 Other Input, but Not Related to the Work of the Electricity Division .... 26

A. 2 Notes on Opening Plenary Session 27

A. 2.1 Components and Parameter Measurements. 27

A. 2.2 Signal Conditioning and Data Conversion . 28

A. 2.3 Sources and References 28

A. 2.4 Systems 29

A. 3 Working Sub-Group Reports 29

A. 3.1 Components and Parameter Measurements. 29

A. 3. 1.1 Introduction . 29

A. 3.1.2 Capacitors 29

A. 3.1.3 Resistors 30

A. 3. 1.4 Phase Angle Measurements 30

A. 3. 1.5 Noise 31

A. 3. 1.6 Other Topics 31

A. 3.2 Signal Conditioning and Data Conversion 31

A. 3. 2.1 Introduction 31

A. 3. 2. 2 Settling Time 32

xi
Page

A. 3. 2. 3 Characterization of Data Conversion 32

A. 3. 2. 4 Noise 33

A. 3.2.5 Sample-and-Hold Measurement Problems 33

A. 3. 2. 6 ADC's and DAC's 33

A. 3. 2. 7 Comparators 33

A. 3. 2. 8 Nonlinear Modules 33

A.,3.2.9 Long Term Performance, Temperature Effects and

Component Studies 33

A. 3. 2. 10 Financial Impact 34

A. 3.3 Sources and References 34

A. 3. 3.1 DC Standards 34

A. 3. 3. 2 Band-Gap Reference 35

A. 3. 3. 3 Noise Sources and Standards 35

A. 3. 3. 4 AC-DC Transfer 35

A. 3. 3. 5 Battery Life/Capacity 36

A. 3. 3. 6 Pulse Generators 36

A. 3. 3. 7 Ramp Generators 36

A. 3. 3. 8 Impulse Testing 36

A. 3.4 Systems 36

A. 3. 4.1 Introduction 37

A. 3. 4. 2 Performance Validation: Static 37

A. 3.4.3 Performance Validation: Dynamic 37

A. 3. 4. 4 Data Outliers 38

A. 3. 4. 5 System Environment 38

A. 3. 4. 6 Detailed Session Notes 38

A. 4 Notes on Sub-Group Chairmen Oral Presentation at Closing Plenary Session, with

Associated Discussion 41

A. 4.1 Components and Parameter Measurements 41

A. 4.2 Signal Conditioning and Data Conversion 43

A. 4. 3 Sources and References 48

A. 4. 4 Systems 49

A. 5 Summary of Post-Workshop Written Feedback Reports 52

A. 6 Summary of Post-Workshop Written Commentaries 54

Appendix B: Miscellaneous 57

B. 1 Final Workshop Agenda 57

B.2 Tour ' 58

B.3 Working Sub-Groups 59

Xll
1. Introduction

1. 1 Background and Objectives

The traditional role of the Electricity Division of the National Bureau of Standards has included a variety of activities directed at the support of the electrical measurement portion of the National

Measurement System. However, as industrial and governmental applications of the newer electronic

technologies have proliferated, it has become apparent that the character of the Division's participation - indeed leadership - must further evolve if it is to continue responsively to serve the nation's

electrical measurement needs, specifically in this critical new arena.

The pervasiveness of electronic technology has brought to the industrial production floor a measurement sophistication in some traditional areas that rivals what would have been considered

outstanding for a well-equipped laboratory not too many years ago. Yet the measurement philosophy

exemplified by careful theoretical identification of all sources of uncertainty, followed by equally

careful experimental verification, can hardly be said to have accompanied this sophistication to its new location. The Measurement Assurance Programs (MAP's) with which the Electricity Division has

become increasingly concerned of late, are intended in part to be a first step in providing a higher

degree of measurement assurance in situ - as close to the measurement site as possible.

While moving forward to assist in bringing measurement integrity out of the standards laboratory and into the field, the Division has become aware that electronics technology - a tool for users - has wrought a basic change in the kind of measurements that delimit continued industrial growth, and

another in the environment in which those measurements must be made. It was to more clearly

define the implications of these changes in the context of the Division's potential contributions, that

the idea of a workshop entitled, "Critical Electrical Measurement Needs and Standards for Modern

Electronic Instrumentation", was conceived.

The overall Workshop objectives, then, included:

1. To ascertain whether there are significant electrical measurement needs in the exploding

electronic s/instrumentation industrie s.

2. To determine the degree of universality of the needs as identified.

3. To investigate whether these identified needs represent areas in which the Electricity

Division of NBS, with its extensive electrical background and expertise, could contribute or lead.

4. To identify broad areas of electrical measurement need of sufficient importance that solutions would represent major contributions, yet not so state-of-the-art as to prevent

realistic programs from being structured at this time.

5. To further identify specific project -level needs within the broad program definitions and

to establish realistic bases for projects that can be undertaken in real-time with specific, identifiable impact.

6. To ascertain the general climate of opinion regarding longer term implications of electronics to electrical measurements, as the basis for strategic planning and as an awareness input for future programming.

7. To establish effective new lines of communication between the Division and a new constituency represented by both manufacturers and users within the burgeoning electronics

industry.
1.2 Participants and Industry Distribution

To be effective, the Workshop was to be small, with not more than twenty-five participants.

To provide breadth, the participating organizations would of necessity be diverse. Finally, to obtain

depth of coverage in diverse fields with a small group, individual participants would have to be the

most knowledgeable available in their specific fields.

After preliminary studies identified broad technical areas of potential interest, organizations

representative of each of these areas were selected with some attempt to balance the often conflicting

requirements of diversity in size and geographical location, the decision to limit participation to a

single attendee per organization, and an organization's known or supposed interest in measurement

technology and integrity per se. With regard to the last point, some organizations were specifically

chosen because of their previous apparent disinclination to participate in the activities of the measurement community, although they enjoy leadership positions in their respective fields.

Last but not least, organizations were selected for their known or expressed willingness to make available to the Workshop the time of busy, top technical executives who retain significant dayto-day technical involvement in their organizations' activities. The Division's success in achieving

this last objective is attested to not only by the Workshop results, but directly by the participants

themselves. There was almost unanimous agreement that many important benefits accrued as a

result of the opportunity for rewarding, high-level technical interchange with peers.

Diversity then was the objective, with depth provided by the quality of organization/participant

selection. Inevitably, many compromises were made, fine organizations omitted, and numerous

outstanding individuals not included, because of the limitations imposed by the necessity of keeping

the group to a reasonable working size. It is hoped that when participant recommendations are implemented and future workshops held, some of these necessary omissions can be rectified by

reconstituting the group along different lines.

1.3 Workshop Agenda and Organization

It was recognized that the topical agenda would critically determine the results of the Workshop in every regard. If topics for discussion were not identified at the outset, there was a signifi- cant risk that the resulting agenda would be so undefined that it would generate little interest and

enthusiasm among potential participants. Still worse, a loosely -organized topical structure could

result in a mass of generalities, useless in obtaining the necessary overview of the state of electronics- related electrical measurements, let alone an indication of specific programs and projects.

On the other hand, it was equally clear that too specific and too rigid a topical agenda could

well stifle the independent outside thinking that was the primary raison d'etre for the Workshop in the

first place.

Last, it was appreciated that if the agenda was to appeal to representatives of an electronics

industry already divided primarily along lines related to corporate and other organizational interests, it must to some extent reflect those existing divisions.

The topical agenda originally consisted of three main sub-groups, based primarily on industrial divisions rather than on technology.

1. Systems for Automatic Measurement and Control

Includes automatic test equipment (ATE's) and systems for industrial process control.
2. Instruments and Functional Modules

Under instruments may be included digital voltmeters (DVM's), digital multimeters

(DMM's), and sampling voltmeters; programmable DC and AC voltage and current supplies; calibrators and references; self-balancing impedance bridges; digital phase angle

meters and standards; wattmeters; noise generators; scanners and multiplexers; and

filters.

Under functional modules may be included analog-to -digital and digital -to-analog converters (ADC's and DAC's); sample- and-hold amplifiers; multipliers and dividers;

instrumentation and operational amplifiers; comparators; voltage-to-frequency and

frequency-to-voltage converters; AC-to-DC and DC-to-AC converters; and filters.

3. Components

Included here are solid state voltage references; precision resistors and networks; precision capacitors; and switches and relays.

From this selection of topics, it would be apparent to potential Workshop attendees that to examine the problems associated with measuring the classic quantities of voltage, current and

impedance in very short periods of time, that is, electrical measurements in the time domain, could

well serve as a concise statement of the Workshop's major aim.

It was soon recognized that some technological distinctions among the three topical areas might be desirable, and a more detailed sub-grouping was considered.

1. Systems (e.g., ATE' s) 2. Signal Conditioning (e.g. , multiplexers, sample-and-hold amplifiers)

3. Sources and References (e.g., programmable supplies, solid state references)

4. Data Conversion (e.g., ADC's, DVM's)

5. Automated Instruments (e.g. , impedance bridges)

Prospective Workshop participants were contacted with the general Workshop outlook,

approach and objectives. After tentatively agreeing to participate, they were asked to submit a oneor two-page, pre-Workshop write-up of their most critical measurement needs and problem areas.

It was recognized, and participants were so informed, that these pre-Workshop inputs would play a

crucial role in shaping the final agenda and the Workshop organization, and would independently constitute an important Workshop result.

The response to the request for pre-Workshop input was broad-ranging and, in many cases,

quite detailed. It is given in section A. 1 of this report. As anticipated, the inputs were of great

importance in helping to structure the Workshop agenda and sub-groupings. On this basis, the final

topics became:

I. Components and Parameter Measurements

IL Signal Conditioning and Data Conversion

III. Sources and References

IV. Systems

Table 1 summarizes in short form the pre-Workshop written input and gives some idea of the

specific subjects covered by each of these four major sub-groups. Note that for each, two roughly

different classes of information are listed: To the left are the application areas, and to the right the

required technologies.
Table 1. Workshop topics based on the pre-Workshop

written input of the prospective attendees.

I. COMPONENTS AND PARAMETER

MEASUREMENTS

Components

Capacitors

Resistors

R-2R Ladders

Attenuators

Switches

Relays

Parameter Measurements

I, V, Z

Power

Phase

Accuracy vs. frequency

Temperature coefficient

Settling times

Dielectric absorption

Computer aided/enabled

II. SIGNAL CONDITIONING AND DATA

CONVERSION

Attenuators

Switches, Relays

Multiplexers

Amplifiers

Sample & Hold

Operational

Comparators

Multipliers

Other Operators (log, etc. Converters

Analog to Digital

Digital to Analog

Digital Voltmeters

Settling times

Thermal

Electrical

Acquisition times

Stabilities

Long-term

Short-term

Temperature effects

Noise

Self- generated

Immunity from

IE. SOURCES AND REFERENCES

Solid State DC

Voltage References

(all levels)

Primary Cells and

Batteries

Generators and Measurement Apparatus

Sampled Data

Impulse Testing

Stability

Noise

Self-generated

Immunity from

Transportability

Reliability

Environmental effects

Lifetime

DC voltage and current

AC voltage and current

Sinusoidal, ramps

Pulses, other

Noise, other factors as above

IV. SYSTEMS

Time as a Parameter

Noise as a System

Parameter

Systems Problems

Calibration •

Environment

Software

How accurate, how fast

Generation and

immunity

Grounding, interaction, interfacing,

switching, etc.

In situ vs. transporting, self- calibration, optimum intervals

Temperature, relative humidity

Program aided/

enabled measurements

Program library

Applications programs

High level executives
Organization of the Workshop presented the challenge of providing sufficient structure to ensure

adequate focus for the generation of both effective discussions and results, yet within a form free

enough to provide openness for unplanned inputs. Thus, the final structure included a mix of plenary

sessions involving all participants simultaneously, and smaller, working-group or sub-group meetings organized along lines identical to the final four topical areas given above.

The complete agenda for the two-day Workshop is given in Appendix B. The first meeting was a plenary session held the morning of the first day. Its purposes were (a) to discuss the pre-Workshop

input and any changes that might be suggested in the organization of the remainder of the Workshop;

and (b) to hold short, "mini" sub-group meetings to insure that all participants were thinking along

the same lines. Since the larger group was to be split into the four working groups that afternoon, it was deemed important to obtain simultaneous exposure of all participants to the lead concepts and

general approach in each of the four working-group areas. This was considered particularly signifi- cant in view of the fact that only two working-group sessions were planned, with all four working

groups meeting simultaneously each time - once that afternoon and again the following morning.

Thus, participants would be faced with a choice of attending two out of the four working-group meetings. (In point of fact, assignments to working-group sessions the first afternoon had been made in

advance, while assignments for the next morning were completely open to the participants, with the

possibility of attending two or even more of the working-group meetings during the second session.

)

The working-group chairmen, four participants who kindly agreed to serve in this important

capacity, remained with their groups during both sessions in order to provide needed continuity.

Between the two working-group sessions there was a short, plenary feedback meeting to start the morning of the second day. This provided the opportunity for an exchange of views on progress,

problems and possible schedule and program changes. Out of it grew the suggestion that participants who chose to do so, could elect to visit a number of working-group meetings during the second

session.

Following the second working-group session, the final plenary meeting was held the second

afternoon. It consisted mainly of individual summary reports presented by the working-group chairmen, followed by open discussion.

Electricity Division personnel participated in both the plenary and working-group meetings.

The composition of the various groups for both working-group sessions, including the Electricity

Division personnel, is given in Appendix B. Bureau personnel participated both as observers and as

direct participants at all sessions, and provided highly skilled, necessary service as recorders at the working-group meetings.

Bureau participants and recorders worked with the group chairmen to prepare both the verbal

reports for the final plenary session, given in section A. 4., and the written reports incorporated

in section A. 3.

The generally high level of interest and commitment during the Workshop resulted in numerous

small and large group interchanges during breaks and at mealtimes throughout the two-day period.

These often resulted in valuable contributions, and helped continue the forward momentum gained

during the scheduled sessions. Even these "formal" meetings were held in an informal manner,

thereby allowing free and open discussion and - at times - disagreements that eventually led to the

productive results and insights recorded in this report.

At the Workshop's conclusion, participants filled out individual "results" and "Workshop commentary" sheets. This provided them a final opportunity to express their purely individualistic

viewpoints. Summaries of these comments appear in sections A. 5. and A. 6.
1.4 Data Sources and Data Reduction

A total of six separate Workshop data inputs provide the basis for the results and conclusions

embodied in section 2. of this report. These data appear in Appendix A, Data Sources. Although

every attempt has been made to ensure that section 2. faithfully summarizes the information contained in the data sources, it was felt important to include them, even at the risk of some redundancy, so that the reader would have at his disposal the original material with all of its subtleties

and highly personal points of view. However, most readers probably do not need to concern themselves with Appendix A since they can obtain all of the essential ideas from section 2.

The data sources are:

1. Summary of Attendees' Pre-Workshop Input (section A. 1.

)

As already indicated, Workshop attendees were requested to submit pre-Workshop let- ters identifying critical measurement needs as they saw them. The information from

these letters appears verbatim, with only minor editing to preserve sense when not supplied by adjacent context. Entries are grouped, however, into six categories. The first

four are identical to the major four Workshop sub -groupings. The last two contain other

inputs not directly related to these four sub-groups.

Electricity Division personnel extracted the inputs from the letters and grouped them by

category, with some assistance from the moderator.

2. Notes on Opening Plenary Session (section A. 2.

)

During the first plenary meeting, a general exchange of views took place and four miniworking-group meetings were held. Notes on these discussions provide insight into the

technical orientation of the participants at this introductory session, and give an indication of emphasis useful in sorting later inputs.

Notes on this meeting were taken and edited by the Workshop moderator for inclusion in this report.

3. Working Group Reports (section A. 3.

)

Sub-group Chairmen, along with the Electricity Division (NBS) participants and recorders, remained with their respective working groups during the working session on both

days. All participated in the preparation of the working-group reports, which constitute

the main data source for the Workshop. These reports were summarized verbally by the

individual group chairmen at the final plenary session, as already noted.

Each sub-group report is thus the result of a cooperative effort, involving the associated

sub-group chairman, NBS recorder and NBS participant. General editing was provided

by the Workshop moderator for organization and style only, with virtually no modifica - tion to content.

4. Notes on Sub-Group Chairmen Oral Presentations at Closing Plenary Session, with Associated Discussion (section A. 4.

)

Even though the summaries presented by the chairmen were ostensibly going to appear in

the group reports, it was felt important for several reasons to capture them as they were

presented. First, they represent reports by the chairmen themselves, albeit with consultation with the associated Bureau personnel. Second, it was hoped that they would
convey a sense of the tone and immediacy of the discussions.. Both of these assumptions

appear to be borne out by the results.

The entire session was recorded and transcribed by a professional court stenographer,

with editing by the moderator only for style.

5. Summary of Post -Workshop, Written Feedback Reports (section A. 5.)

As indicated previously, at the termination of the final session participants were asked to

fill out short, individual-feedback forms giving their final conclusions regarding critical

measurement needs and priorities. These free-form inputs, as it were, presented valuable data which nevertheless required fairly extensive sorting and editing by the moderator before they were in adequate form for inclusion in this report.

6. Summary of Post -Workshop, Written Commentaries (section A. 6.

)

Also as previously noted, at the end of the Workshop participants were requested to answer a short questionnaire about the Workshop itself. Answers, again sorted and

edited by the moderator, offer insight into the diversity of outlook, the degree of involvement of the participants in the Workshop itself, and attendee hopes and expectations for

the implementation of the Workshop results. They also furnish a calibration for the

degree of consensus achieved on the various issues.

Data reduction, or final extraction of program and project information considered potentially

useful to the Electricity Division in carrying out its mission, was not a trivial task in view of the

nature of the Workshop and the plethora of input data. Significant exclusions from the final summaries of programs and projects that are both needed and real, had to be made for a number of practical reasons. Areas which were so excluded, and covered under the heading "General Long-Term

Directions", include:

1. Problems with solutions which seem to require breakthroughs either not presently well

foreshadowed by existing or anticipated technology, or requiring resources well beyond

those even conceivably available to the Division. Included in this category might be

replacement of the standard cell as the ultimate laboratory working standard of voltage,

and replacement of the thermal approach to precision AC/DC measurements.

2. Problems whose solutions appear to have limited present universality, and/or seem to require the kinds of resources - and motivation - more likely to be available in industry.

Among items in this category might be the concept of impulse testing for complex systems.

3. Problems that have wide universality, but no presently-evident "handle" by means of

which any single group of investigators would be likely to produce results of significant

value to any large part of the field. Included in this category might be the general problems of time- and history-dependence of systems measurements, with the exception of those aspects that can be quantified in terms of the dynamic performance of sources and

measurement devices.

Any attempt to summarize inputs as diverse and far-ranging as those resulting from the Workshop must of necessity involve editing and the application of judgment. The basic criteria for making the judgments involved in defining potential programs and projects were:

1. Breadth of application.

2. Relevance to the mission of the Electricity Division.
3. Apparent significance in terms of impact, or payoff to an industry/government constituency.

4. Importance in terms of a contribution to the National Measurement System.

5. Implications regarding Electricity Division capability to move forward into still more

sophisticated areas of measurement need.

6. Evidence that industry, of its own accord, probably cannot or will not deal effectively

with the problem area.

7. Potential for reasonable chance of success in a reasonable time, versus (3), (4) and (5)

above; the greater the apparent impact, the larger the justifiable program.

8. Potential for handling via reasonable evolution of existing Division capabilities, or potential for sufficient impact to justify significant rearrangement of priorities/capabilities/

facilities.

In light of the above, it is unlikely that the conclusions reached will generate total agreement

in any single quarter. However, it is hoped that they will generate a sufficient degree of support

among a wide enough constituency to ensure that many of the Workshop inputs will be embodied in meaningful programs of the Electricity Division.
2. Results and Conclusions

The most significant conclusion to be drawn from the Workshop is that there is a strongly felt need on the part of the electronics industry as represented for guidance in electrical measurement

procedures and methodologies on a broad technological front.

Thus the Workshop met its most important objective.

Other goals were met as well: Universality was determined for specific needs, identification was made of those needs that can be met by expanded programs of the Electricity Division, and both

broad programs and specific projects that meet the requirements just listed were identified.

In addition, the more general objective of ascertaining implications for the longer term was

also achieved. Results are incorporated in the following sections.

Last and not at all least, a significant step has been taken towards setting up new lines of communication between the Division and a far-flung electronics- oriented constituency.

2. 1 Overview

Three categories broadly cover the needs as defined by the Workshop:

1. The need to introduce time as a measurement parameter has resulted from the requirements of automatic test and control systems. Specific areas considered critical are:

a. Pulsed component measurements.

b. Dynamic performance characterization for modern signal conditioning and data conversion: Digital-to-analog and analog-to -digital converters, sample -and -hold

amplifiers, comparators, etc. Required measurements include settling, aperture

and acquisition times. Basic new capabilities will be needed, including precision,

non-sinusoidal waveform generation and "standard" digital-to-analog converters.

c. Methodologies and techniques for characterizing precision AC and DC sources and

measurement devices with respect to settling time.

2. The emergence of measurements into the "real-world" of the production line via the

automatic system has introduced a host of new parameters. Areas requiring significant

effort include:

a. Investigation of transportable standards for validation of static and dynamic system

performance: AC and DC voltage, impedance, pulses, settling times.

b. Techniques for characterizing sources and measurement devices with respect to switched or dynamic loading.

c. Prediction of long-term performance, reliability and lifetime from short-term

evaluation for a host of passive components, semiconductors, and signal sources.

d. Noise standards and methodologies.

e. Continuing effort to improve transportable DC standards, and dissemination at higher voltage levels.

f. Characterization of the AC line voltage - waveforms, under- and over-voltages,

transients, dynamic impedance, etc.

g. Inclusion of a capability for environmental control and variation for devices and

instruments under calibration, in all new work undertaken by the Bureau.

h. Extension of Measurement Assurance Programs (MAP's), especially into the above new areas.
3. Recent work at the leading edge of measurement technology in electronics has resulted in the need for new, or higher -accuracy measurements. For example:

a. Increased dependence on capacitors as storage devices has made dielectric hysteresis a parameter of importance. Significant measurement work should be started in

this area.

b. Phase difference measurements to extremely high accuracies are needed to calibrate

industrial instrumentation.

c. Higher accuracy power measurements, particularly for non- sinusoidal waveforms,

are needed to calibrate instrumentation used in the field. d. Non- sinusoidal, high crest-factor precision signals are needed to calibrate true RMS

converters and meters.

e. High accuracy electronic instrumentation requires reduction in AC calibration uncertainties to 10 ppm, at least over the audio range.

Longer-term, "frontier" areas in each of the above groups were also identified. They are not specified in the above listings since they involve work at still more basic levels, on ground that is

presently terra incognita. Nevertheless, they furnish insight into the overall technical/philosophic

climate, and are useful for that reason. Items in this category include:

1. Measurement problems in systems, including effects of history, measurement interactions, self- calibration, etc.

2. Evaluation and standardization of system software.

3. Calibration of medical equipment.

4. Replacement of standard cells.

5. Replacement of thermal AC/DC converters.

6. Non-swept but broad -response measurements - impulse, noise, other.

7. Basic work on the physics of dielectric absorption; component drift; device noise; failure

mechanisms; switches and relays.

8. Provision of improved measurement accuracy for virtually all dynamic measurements, at the wafer probe level.

Finally, in a more general vein, a good deal of support surfaced for the idea of keeping the

electronics community informed of Division work-in-process, with emphasis on projects targeted by

the Workshop.

2.2 Specific Results

The preceding section outlined Workshop results from the standpoint of technological needs.

This section presents results first in terms of broad programs, next by specifying projects falling

within these programs, and finally in terms of some general, long-term directions. The purpose,

of course, is to begin the necessary process of re -casting needs into a form conducive to establishing a coherent set of programs and, where possible, specific projects.

2.2. 1 Broad Program Definitions

This section attempts to formulate responses to the identified technological needs in programmatic form, that is, in groupings that share a common technical/philosophical background, as well as potential approaches to solution.

In accordance with this approach, broad, needed programs that can be readily isolated include:

^ 1. Dynamic Component Measurements. Characterization and measurement of the pulsed

characteristics of components - for resistors, capacitors, networks; also, "conventional" sine wave measurements at higher frequencies than now available.

10
Passive components are the very most basic elements of all electronic circuits, instruments

and systems. As such, along with their active counterparts, they are responsible for the ultimate

performance limits for electronic assemblies.

In recent years, two new directions have become apparent. First, components are being

widely used in critical, switching-circuit, data-conversion applications. Their characteristics in

such uses - settling time, hysteresis, overshoot, ringing, etc. - have been all but ignored by classical measurement technologies. Second, as the electronics field advances towards still faster operating speeds, components subjected to more conventional waveforms like sine waves are being used

at higher frequencies. This trend, too, needs to be reflected in the upgrading of present measure- ment methodologies, in this case, 10 kHz now, 100 kHz and higher in the foreseeable future.

Impact is not difficult to assess. If the Bureau does not move to characterize components on a

basis that reflects industry's real needs, then the notion of "traceability" for components will take on an increasingly fictitious significance. Components in electronics are simply not used the way they were; the Electricity Division's calibration capability for components must reflect this if that capability is to retain relevance.

2. Building Blocks for Signal Conditioning and Data Conversion. Definition, evaluation,

characterization of dynamic performance for modern signal conditioning and data conversion building

blocks. Initial emphasis on digital -to -analog and analog-to -digital converters, sample-and -hold

amplifiers, comparators.

An entire technology has grown up around the necessity for dealing precisely and accurately

with the interface between the analog and digital worlds, and manipulating signals on the analog side

of the interface. This far-reaching technology has spawned a host of building blocks that are even

appearing in integrated circuit form, first via hybridization and eventually via monolithic technology.

In all forms they enjoy widespread application as basic tools for data manipulation and computing.

There is a significant need for leadership in establishing basic measurement methodologies in

this new field. The lack of wide agreement on certain critical measurements - particularly amplitude with time as a parameter - underlies problems in the standardization of terminology, specifications and hardware.

The payback from the rationalization of measurements and terminology in this cruciallyimportant field will be the broader application of the existing technology to a wider variety of industrial problems.

3. Dynamic Measurements, Sources and Measuring Devices. Methodologies and techniques

for characterizing AC and DC references and measurement devices with respect to time dependence,

as well as switched or dynamic loading.

Present methods for calibrating both sources and measuring devices virtually ignore their

dynamic characteristics, from two basic standpoints.

The first of these is response with respect to time, and all that it implies. Included is not only the relatively simple notion of settling time, but also settling time for various- sized steps, and

in particular, steps from specific, different data levels. Also of significance is the implication of history inherent in any investigation of temporal performance. Included is response from overloads

of various magnitudes, response for steps of opposite polarities and so forth.

The second dynamic characteristic that often leads to basic performance limitations in critical

systems is response to dynamic or switched loading. This factor is of particular importance for sources. Rapid recovery is required after the application of a wide variety of switched loads,

11
including the direct, instantaneous short-circuits resulting from multiplexer make-before -break

input switching.

4. Static and Dynamic Transportable Standards . Investigation and development of both passive and active transportable standards, suitable for in situ validation of automatic system performance: Needed are standard components, sources, measuring devices.

The concept of taking parts of a system to a calibration facility is triply inadequate: They may

not be calibrated when they return due to changes induced by shipping; they are not calibrated under

operating environmental conditions; and they are the wrong things to calibrate anyway. These three

factors make present calibration techniques wholly inadequate for the bulk of the electronic instrumentation systems presently planned, and already in the field.

There would seem to be no substitute for development of an entirely new methodology for calibration of modern systems. The key words here are: Transportable and dynamic.

First, bringing the transportable standards to the system insures that the devices and systems

being calibrated will not be affected by shipment. But of course, the standards themselves must be

designed to withstand shipment. Thus, methods would have to be employed similar to those used for

present Division MAP's in order for this approach to be successful.

Next, in situ calibration results in the closest possible approximation to actual operating conditions, environmental and otherwise.

Last, what can and indeed must be calibrated at the site are things which are unmeasurable in

the laboratory: Performance of the entire system, not just its building blocks, thereby including at least to some extent, history and interactions, and performance at the system's real -world interface, not simply at terminals within.

Also mandatory is including the temporal parameter in the calibration since it is so crucial to system performance and throughput. Dynamic standards are thus required, to supplement and perhaps eventually even replace, their static predecessors. Needed are AC and DC sources, resistors

and capacitors, plus devices for voltage and current measuring and monitoring. Critical for all such dynamic standards is precise knowledge of their performance as a function of time after they are switched, energized or activated.

Of all the programs identified by the Workshop, there was the clearest consensus that this is undoubtedly one of the most significant for the near term. It represents a pervasive need and from

many standpoints, it constitutes perhaps the greatest immediate- challenge.

5. Predictions for Component Stability. Estimates of long-term performance, reliability

and lifetime from short-term evaluation, for the full range of passive and active components:

Resistors (thick and thin film), networks, capacitors, semiconductors. Efforts would presumably

start with extensive component stability and reliability studies.

Again, components are the ultimate bits and pieces whose performance limits that of modules,

instruments and systems. Thus, knowledge of their basic capabilities is crucial to calculating characteristics of the larger assemblies that depend on them. This is nowhere more valid than in the area of performance stability. The electronics designer can often obtain circuit or device characteristics with a high degree of independence from changes in particular components; but this can be

costly, and in any event there are always some few components in any assembly that determine

overall limits to performance.

In the headlong rush to apply electronic technology to the variety of industrial and societal

problems for which it holds so much promise, insufficient knowledge has been developed regarding

12
the causes, character, influences on, and rates and magnitude of changes in the entire range of basic

electronic components, both passive and active. And while some of the work that would clearly

prove useful in an investigation of component stability is so basic that it appears in a later section,

it would seem that much widely-useful data-gathering could be accomplished without getting too inti- mately involved in device physics.

Manufacturers and users need and expect equipment to be stable, reliable and long-lived.

Component derating and conservative design, in the hands of knowledgeable designers, has produced

equipment that often succeeds but sometimes fails in use, as a result of problems in the areas of stability and reliability. Much work in this regard is clearly necessary if electronic solutions to medical, transportation and production problems, to name but a few, are to gain widespread acceptance. Only a central, unbiased organization can be expected to carry it out with the required motivation and impact.

6. Energy-Source Status . Determination of long-term battery life, charge status and

rechargeability from short-term measurements, via any feasible technique including battery

implants.

There are several areas in which a prediction of battery status from current data is of importance. The first is the obvious medical area, in the burgeoning field of electronic implants.

New applications besides heart pacing are proliferating, and performance can be critical to life.

A second application for battery status information is in maintaining standard cell or other

reference standards at constant temperature during transport. Here, emphasis is on charge status and rechargeability. This will constitute an increasingly important problem as the trend continues

to grow towards transportable standards and MAP's.

Battery -status predictability may well have significant additional impact on the whole range of

battery-operated appliances, tools and equipment used in both the home and industry: Drills, shavers, lawnmowers, wire-wrap tools, and others the designs of which await better -characterized

energy sources.

7. Noise . Noise standards, measurement methodologies, characterization of distributions, both for component measurements and for characterization and verification of noise generators.

Noise has figured in classical electrical measurements primarily as a characteristic to be

dealt with as a factor limiting the minimum uncertainty that can be attained. But the broad applications area of noise measurement and characterization has been largely bypassed or ignored.

Meanwhile, industry has been struggling with noise as an inherent limit to passive component

performance (resistors of several varieties); as a possible indicator of stability and likewise a limit

to performance (Zener references); as an unwanted signal in systems (crosstalk, radiation sensitivity, ground couplings); and as a restrictive element in the performance of a wide and important

range of semiconductors (transistors, operational amplifiers). Noise generators go largely uncalibrated, and non-gaussian, semiconductor -generated noise distributions remain uncharacterized.

Clearly, noise is an important and restraining factor in much work in electronics. The constituency of sophisticated designers and users continues to expand. As it does, the pressure can

only mount for providing the same fundamental basis for noise measurements that the Electricity

Division has helped to furnish for the metrology of DC and periodic AC signals.

8. DC Reference . Continuing effort with semiconductors, possibly including new work with

band-gap devices, towards developing 1-5 ppm stable, transportable, higher -voltage DC standards

13
with predictable drift rates; plus continuing effort in making the Josephson voltage standard easier to use.

Virtually all instruments and systems that deal with signal magnitudes incorporate at least one basic DC source to which the device's generated signal and/or measurements are referenced.

The stability and absolute value of these references can often be a serious limiting factor in signal

generation and/or measurement validity, with significant economic implications via the need for

shorter or longer calibration intervals, not to mention performance itself.

Both industry and the Bureau have done a great deal of work related to this fundamental reference; but much more needs to be done, e.g. , improve the ruggedness of semiconductor references;

disseminate the unit of voltage at higher voltage levels, for more direct traceability at lower total uncertainties; develop transportable standards of sufficiently consistent performance to allow drift- rate prediction; and improve the Josephson standard so that it may see more widespread use.

There is probably no more basic an amplitude reference than the DC reference. The Division

has a long history of maintaining and supporting it, when its use was primarily for reference purposes within a standards laboratory environment. However, the reference has now moved out onto

the production floor. Division support must move with it in order to ensure continued progress in

measurements in the real-world, systems environment.

9. The AC Line. Deterministic and/ or statistical characterization of AC line voltage at the receptacle.

The AC line - with all of its vagaries - supplies primary power to all but a small percentage

of the electronic equipment in common use. As such, its peculiar characteristics and the design

allowances which must be made for them, have direct economic impact.

A wide range of parameters urgently need rigorous, even if statistical, characterization, in

order to permit designers and users to either design for or else to modify the AC line voltage presently available at the receptacle. Included are transients, over- and under -voltages, dynamic

impedances out to the frequencies and/or rise times of pulsed, switched and switching loads, the

effect of modern electronic loads on waveform noise, etc.

In this area, overdesign typically represents wasted effort and dollars, while underdesign can

result in expensive outages or else the need for auxiliary regulators and filters. Evidently signifi- cant economic uncertainties are implied by the present inadequate state of knowledge and agreement

on AC line parameters.

10. Data Outliers . Characterization and methodologies to measure or live with the data

outlier problem in measurement in general, and systems in particular.

In any series of measurements, from time to time a result will appear that is so vastly different from its neighbors that it is termed a data outlier. These are usually discarded in "manual"

measurements, with the rationale that they are manifestations of unknown source of noise, most

probably external to the measurement complex or system.

The severity of the outlier problem increases with measurement speed, since upward pressure on speed decreases the likelihood that measurements can be employed which are inherently of the averaging type. Sampling an input parameter during an interval of just a few microseconds

enhances the possibility of even small amounts of noise causing significant errors.

N Another factor forces attention to the hitherto neglected outlier: On-line electronic systems

are increasingly taking the responsibility for monitoring and controlling industrial processes,

14
life-support systems, and even whole micro-environments. They can stand fewer mistakes, since

such errors can have adverse human and economic impact.

Evidently at least two basic philosophic approaches are available for dealing with outliers - reducing their incidence via a program of measurement, understanding, and finally design, or simply designing instrumentation on a basis inherently insensitive to them, whatever be their source. Leadership and knowledgeable investigation are required to put the problem in perspective, and to delineate realistic alternatives.

11. Measurement Assurance. Extension of MAP' s, specifically into the new areas outlined

in the above programs; some effort to define traceability, implicitly or explicitly, possibly via the

MAP route.

Generating the required new methodologies is insufficient; dissemination is necessary as

well, and in a way that inherently guarantees their successful application'to the solution of measure- ment problems throughout the electronics industry. The NBS Measurement Assurance Program,

extended to include Workshop -defined programs, would constitute a recognized vehicle for successfully accomplishing this objective.

It is well recognized that the notion of traceability to the Bureau can be sorely abused. While

it is doubtful that NBS can or should state that a particular industry, or worse yet, company, is making measurements that are "traceable", MAP's in critical areas could play an important role in keeping reasonable bounds on use of this important concept. And the industry, as represented at the

Workshop, would welcome efforts to help "traceability" retain real significance.

2.2.2 Specific Project Activities

While there will be much additional work required to define specific projects within the broad

programs outlined in the previous section, cognizant Workshop participants were able to agree on some specific activities with readily apparent and quantifiable goals, and immediate payoffs for

industry:

1. Capacitors . Capacitor dielectric hysteresis: Evaluation, characterization, measurement, hopefully leading to possible improvements via subsequent, basic work on dielectrics.

2. Signal Sources. Theoretical modeling followed by the design and construction of basic

building blocks for precision dynamic measurements:

Pure step or pulse generator - 20 ns settling time to at least 0. 01% accuracy, to be used for

settling time and other measurements.

Pure ramp and ramp -with -corner generator - 100 ppm linearity, to be used for aperture and

acquisition time measurements.

"Standard" DAC with 10 ppm accuracy, and 10 mA output capability. Presumably to be as

"glitch-free" as possible, and with a settling time consistent with testing devices which settle to 0.01% in 20 ns.

3. Phase . Phase angle calibration capability for existing commercial instruments claiming

0. 01 degree measurement accuracy.

4. Power. Power calibration capability with 50 ppm, if not 20 ppm uncertainty, to be used

in the calibration of commercial 0. 01% and/ or 0. 02% power meters. Increase emphasis on nonsinusoidal waveforms for both power and power factor measurements and calibration.

15
5. AC Calibration, Reduce AC calibration uncertainties to 10 ppm in the audio band.

6. High Crest- Factor AC. Create a facility for calibrating RMS-measuring devices for

non-sinusoidal, high crest-factor inputs, with crest factors to 10 or 12 at full scale.

7. Transportable Standards . Design or qualify transportable standards for in situ system

calibration:

DC voltage to 1000 V, with uncertainties approaching 10 ppm.

AC voltage to 1000 V and to 10 MHz, with midband uncertainties approaching 20 ppm.

Resistance from 0. 001 Q to 100 MQ. , with midband accuracies to 10 ppm.

Pulse standard - rise times of 10 to 30 ps, exponential decay.

Constant currents (DC, AC) - 1 nA to 10 A, with midband uncertainties to 50 ppm.

2.2.3 General Long-Term Directions

Lastly, a number of areas were identified which either imply programs so extensive, or require breakthroughs so apparently distant, that they cannot be included in the "practical" results

sections of this report. However, they present a graphic picture of the philosophic pressures

underlying electrical measurement needs for some time to come. As such, they merit study not only

to enhance overall awareness, but also to provide the basis for possible future program definitions.

Included in this category are:

1. System Hardware . Define and take some steps towards the solution of the problems of

verification of "system" performance in the field, for a wide variety of systems. Definition of a

real focus for "transportable dynamic standards" is required, for the most general class of systems. Measurement assistance for dealing with time- and hi story -dependent effects in systems is needed, as well as interconnects, cablings, grounds, isolation, guards and last but not least, noise

generation and/or sensitivity. System self- calibration, and/ or automatic out -of-calibration indications are also required. Computer -controlled calibrators/ sources may be a possible, partial

approach to some of these problems.

2. System Software. System software, and recommendations on preferred approaches,

are fruitful areas for long-term effort.

3. Medical Electronics . Establishment of a calibration capability for medical equipment,

specifically for hospitals.

4. DC Reference . Replacement of standard cells with another reference of equal stability,

but far greater environmental in sensitivity. Preferred is the 10-volt level, with a 10 C to 50 C

operating range, with a temperature coefficient small enough to be consistent with an overall stability of better than one ppm. Portability is important, although being self -powered is not. An output

capability of up to 10 mA is desired. Warmup of up to an hour can be tolerated.

5. AC Measurement or Transfer. Develop an RMS measuring device (to replace AC/DC

thermal transfer standards as we now know them), with accuracies approaching 10 ppm (100 ppm

satisfactory for now), but more important, capable of operating at higher speeds. Ten measure- ments per second are needed now, and up to 100 per second in the future. Its dynamic range should

be far greater than present thermal devices - up to at least 10:1 for voltage; and the device should

be rugged and relatively insensitive to overload.

16
6. Non-Swept, Broad Response Tests. Impulse and/or noise testing criteria would be

desirable, plus methodologies and hardware. Areas of use would include application to non-swept

but broad response tests for complex devices and systems (see wafer probing below), with the

resulting data analyzed for correspondence with present, more conventional, "manual" methods.

Fast Fourier transform techniques might be a possible approach.

7. Dielectric Absorption. Basic work is needed on the physics of dielectric absorption,

with a view towards improving performance of compact, solid -dielectric capacitors.

8. Component Drift. Basic work is needed on the physics of component drift: Film resistors, bipolar transistor VR

's and Zener references seem most important. Included would be stability of device temperature coefficients, for both passive and active devices. Object would be to reduce component drift and to predict future values.

9. Component Noise. Same as (8) for device noise, which is presently becoming a final

performance limit for both resistors and semiconductors.

10. Component Failure. Basic work is needed on the physics of failure mechanisms for

both passive and active components, including contact devices like relays and switches.

11. Relays and Switches . Basic work is required on the theoretical and experimental

design of advanced relays and switches for system applications. Goal would be high reliability with

low contact resistance and thermals, and a high voltage handling capability. High accuracy switching is especially needed for AC voltage and currents up to 10 MHz.

12. Wafer Probe Testing. Last, there is a foreseeable need to provide means for virtually

all high speed electrical measurements at the wafer probe level, taking into account length of leads,

noise, etc.

2.3 Conclusions

The Workshop and its results as summarized here are but the first steps. Broad programs

have been outlined and specific projects identified. Priorities must now be assigned, however, if an

organization of finite size is to attack even a reasonable number of the problem areas delineated. In effect, the industry is perhaps a decade ahead of the metrology it needs to continue its growth; but

this gap cannot be narrowed overnight, and wise choices must be made concerning alternative courses of action.

It is apparent that the electronics industry has generated a host of electrical measurement

problems different in kind from those of classical metrology. Now needed are impact studies, priority assessments and in-depth evaluation for all potential program areas as identified. Also

required is reevaluation of ongoing Division programs in the light of the Workshop results.

One final conclusion remains: It was the overwhelming consensus of the attendees that the

Workshop should be repeated in one form or another at intervals of one to two years. This kind of participant interest and commitment implies continuing cooperation and maximum impact, as the new programs are implemented.

17
Appendix A: Data Sources

A. 1 Summary of Attendees' Pre-Workshop Input

The following six sections are appropriately grouped excerpts from the pre -Workshop written

input submitted by the attendees. Only minor editing was carried out by the moderator and Electricity Division staff. At the beginning of the first four sections - Components and Parameter Measurements; Signal Conditioning and Data Conversion; Sources and References; and Systems - two side-byside short lists of words and/or phrases are given which attempt to capture the essence of the topic

under discussion. Roughly speaking, the left-hand list gives the application areas and the right-hand

list the required technologies.

A. 1. 1 Components and Parameter Measurements

Components

Capacitors Accuracy vs. Frequency

Resistors

R-2R Ladders Temperature Coefficient

Attenuators

Switches Settling Times

Relays

Dielectric Absorption

Parameter Measurements

I, V, Z

Power Computer Aided/ Enabled

Phase

1. Dielectric absorption in capacitors is a limiting factor in the performance of sampleand-holds and integrating DVM's. We need dielectric absorption specifications that can be meaningfully related to performance in the above instrumentation. Percent dielectric absorption is very

much related to the test method, and the frequency dependence of loss tangent at low frequencies is almost useless in predicting performance in a sample -and -hold.

2. For AC standards, such as capacitors, better standards over a much wider range of fre- quencies are expected to be needed.

3. All methods of measurement of physical circuit parameters and constants, particularly

in-circuit values of R, C, L, etc.

4. In the component area, the advent of stable film resistors has lessened the demand for

wirewound resistors. They are still used, however, and standard settling time tests for wirewound

resistors would be desirable.

5. As we depart from the primary basic electrical standards, we also face limitations.

Practical use of resistance from 10 ohms to 10 ohms is limited by the high voltage coefficient of the resistors.

6. Two standards areas of definite interest are high resistance and low voltage measurements. Our 5155 sets of hi-meg resistors (10 to 10 ohm) are regularly sent to the Bureau for

measurement. We find that NRC (Canada) offers hi-meg calibration to less uncertainty than NBS (at

109

, 0.05 vs. 0.2% and at 1012

, 0.35 vs. 0.5%). Desirable in the near future would be an order of

magnitude improvement in the Bureau's uncertainty.

7. A calibration standard for ladder networks of the R-2R type is needed.

18
8. High reliability relay with very low contact resistance upon contact closure: Because of

the large number of relays in a system (and the requirement of contact resistance to be 0.080 ohm

maximum), the closed contact resistance shall not exceed 0.080 ohms more than once m 10, 000

operations.

9. We also see a need for smaller uncertainties in the calibration of AC devices through all frequencies, but especially up through 10 MHz. Repeatability of measurements in this range is often

better than the uncertainty in many cases. This problem extends itself into the precision attenuator

calibration accuracies too. Requests have been made for 0. 001 dB at -90 dB.

10. Required are precision measurements of very small currents and the ratios of very

small currents, say in the below 10" ampere region; also as a corollary, the precision measure- ment of very high impedances.

11. Low voltage calibration (10 nanovolts to microvolts) is done using SR1010 sets to calibrate precision four-terminal low-thermal dividers. This is a time-consuming and expensive task

and nearly impossible for most of our customers. Currently, 1 millivolt (to 100 nanovolts accuracy) and 100 microvolts (to 10 nanovolts accuracy) are the most sensitive requirements. I see

future needs for about an order of magnitude uncertainty improvement.

12. What is the state-of-the-art on AC standards with respect to high input impedance?

13. The variable to be measured is generally very slowly varying (less than 0. 1 Hz) and is derived from a transducer such as a thermistor or thermocouple. Voltage measurement requirements: Varies between microvolt levels to tens of volts. Current measurement requirements:

10" A to 10" A. Environmental requirements: Temperature full accuracy: 15 C to 40 C;

reduced accuracy: -10 C to +70 C; humidity up to 95%.

14. I would like to give some examples of electrical measurement we anticipate in carrying

out medical device evaluations:

1. Low level current measurement.

2. Insulation resistance measurement.

3. Ground or low resistance measurements.

4. Conducted and radiated EMI effects on instruments.

a. amplitude (current, voltage);

b. frequency components;

c. time interval measurements; and

d. energy.

We are currently evaluating the clinical environment (temperature, vibration, shock, humidity and

electrical) that medical devices are subject to, and expect additional measurement problems to be

identified.

15. Other devices of interest include: (a) Level detectors, voltage and current ; (b) AC

voltage, current, phase and power measurements 50 to 400 Hz.

16. Every measurement we make now must be reconsidered in the light of the possible

effect of closely-coupled processing. Has the past method compromised speed or accuracy because

of the choice of a transducer which gave a more convenient numerical display? Has the accuracy

been impaired by an analog arithmetic operation? Has the accuracy been compromised by an

inability to keep adequate statistical data during the experiment? The tools are now at hand to over- come such traditional obstacles and open the way to great increases in the capability and effective- ness of instrumentation.

19
17. In the past, the development of sensors for measuring various electrical and physical

quantities has concentrated on those devices which have reasonably linear outputs with respect to the

measured quantity. With the availability of low cost microprocessors, capable of linearizing complex output curves, the only real requirement is that the sensor be precise. Perhaps a whole new

generation of sensors could be developed with characteristics better suited to today's technology if the linearity constraint were removed. We often find that a limiting factor in an automated measure- ment system is not in' the instrument itself, but rather in the availability of a compatible low-cost sensor having sufficient precision for the application.

18. A portable standard is required to check phase angle of presently available instruments

in the range of one to thirty kilohertz with an accuracy of +0. 01 degree.

19. Improved phase angle measurements: Accurate phase measurements presently require

a low distortion sine wave of constant frequency. Often the test signal has frequency instability and

is distorted. To make a phase measurement, a lengthy series of adjustments or several readings

and a complex computation are needed. In addition, the phasemeter, for high accuracy, must be

operated within a narrow ambient temperature range.

20. Methods are needed for measuring BV_,„ _ of transistors without inducing oscillation.

A. 1. 2 Signal Conditioning and Data Conversion

Attenuators

Switches, Relays

Multiplexers

Amplifiers

Sample & Hold

Operational

Comparators

Multipliers

Other Operators (log, etc.

)

Converters

Analog-to-Digital

Digital -to -Analog

Digital Voltmeters

Settling Times

Thermal

Electrical

Acquisition Times

Stabilities

Long-Term

Short-Term

Temperature Effects

Noise

Self- generated

Immunity from

1. The signal conditioning and data conversion areas suffer severely from a lack of uniform

specifications.

2. Standardized specifications and calibration techniques accepted by many manufacturers

would permit a customer to make meaningful comparisons between competing products.

3. There is a need for improved accuracy and resolution of solid state programmed atten- uators. (Example: 0.01 dB resolution +1 dB accuracy).

4. Probably the most frequently mentioned problem was the need for a solution to distributing and switching high accuracy AC signals from audio frequencies on up through 10 MHz and doing

it at high voltages as well. The solution probably lies in a breakthrough in relay technology.

5. Measurement and characterization of nonlinear phenomena as related to precision equipment is a large unexplored territory - for example, the measurement of dielectric absorption and

its effect on precision integrators and sample-and -hold amplifiers.

20
6. A capability will be needed to evaluate and characterize high performance operational

amplifiers for such parameters as gain, stability, transient response, input offset currents, noise

immunity, etc.

7. Measurements and standards for high-speed comparators need further study, definition

and development.

8. A capability will be needed to evaluate electronic multipliers as conversion devices.

9. A capability will be needed to evaluate analog-to -digital converters working at high conversion rates and accuracies of 0. 1% or better.

10. The current 12-14 bit devices are not adequate. Perhaps a minimum resolution of 14

bits with exponent bits to provide the signal range would be acceptable. Long term accuracy better

than 0. 05-0. 1% does not seem to be required.

11. What is being done to standardize techniques for calibration of analog-to-digital converters?

12. A/D converters (including, but not limited to DVM's) are usually specified to have a

certain accuracy after voltage has been applied for a specified time.

13. In order to verify system performance one must take into account: 1. The number of

bits of accuracy; 2. Settling time; and 3. Aperture time. Conversion time may be of interest

also. Any help in verification of system performance would be appreciated.

14. A few specific ADC and DAC calibration problems are listed below:

1. Adequate thermal settling time tests.

2. Noise distribution (often this is important out to +5-sigma).

3. Separation of noise and linearity.

4. Practical temperature coefficients testing (other than a few points).

15. As a general rule, signals must be limited to an upper frequency, about one-tenth of the

sample rate to achieve adequate accuracy. With sample rates of moderately priced A/D converters

currently in the 1 megahertz sampling rate range, dynamic testing must be limited to relatively low

frequency systems. The signals found in a "low frequency" system often have significant components in the 100, 000 hertz range. A tenfold increase in effective sampling rate is needed.

16. Instrumentation and techniques are needed to measure single and multiphased time

intervals down to one nanosecond to an accuracy of 0. 1 nanosecond. (An example of multiphased

time interval is the time delay between the input and output of a device.

)

A. 1. 3 Sources and References

Solid State DC

Voltage References Stability

(all levels) Noise

Self-generated

Primary Cells and Batteries Immunity from

Transportability

Reliability

Environmental Effects

Lifetime

21
Generators and Measurement DC Voltage and Current

Apparatus AC Voltage and Current

Sinusoidal, ramps

Pulses, other

Noise, Other Factors

as above

Sampled Data

Impulse Testing

1. The present unsaturated cell voltage standard methodology has a number of serious

drawbacks. Recent advances in band -gap references lead me to hope that at some future date, the

standard cell will be replaced with a small, rugged, temperature-history independent device.

2. What is needed is a primary DC standard which is able to withstand moderate vibrations or shock, is self correcting for temperature variations over at least a 10 degree C range from the

calibration temperature, and returns to calibration accuracy in hours rather than days after being

subjected to moderate temperature extremes or shock.

3. Production instruments are rapidly approaching the limits of NBS certifiability. Within

the next 5 years, 10 ppm DC voltmeters will be as common as 100 ppm units were 5 years ago. The

demand is coming for secondary DC standards to be accurate within 1 ppm. In essence, the need

exists to get current secondary lab accuracy out of the standards lab and into the shop. This implies

better automatic temperature correction capability.

4. The standard voltage cell is a limiting factor in our ability to provide calibration support

of digital voltmeters and direct voltage standards.

5. What is being done to replace standard cells with solid state voltage references? Is economy to be considered as a trade-off?

6. Direct voltage is supported primarily at one volt, but the wide range of DVM's makes

voltage support at 10 volts, 100 volts, and possibly even 1000 volts, desirable.

7. Fundamental studies relating the long-term drift characteristics of Zener references and

differential amplifiers to short-term measurements (perhaps noise or high temperature performance) would make it possible to provide instrumentation specifications that would hold beyond final test at the factory without tying up expensive components in long life tests.

8. Battery life: Medical instrument requirements such as portability and electrical line

isolation have been most responsible for use of battery power in medical device development. As

the development of cardiac pacemakers and artificial hearts progresses, the need arises to be able

to assess the lifetime characteristics of implantable power sources for these devices.

9. An easily programmed programmable calibrator with sufficient flexibility to handle a wide variety of functions and accuracies would be useful.

10. The characteristics of square-wave and pulse sources are needed.

11. There is a need to provide computer -controlled signal sources that can exercise dynamically a large system and confirm that it is performing to specification over the entire range of its measurement capability.

12. There is a need for improved high accuracy time-varying amplitude standards.

Sample-and -hold amplifiers are used with 14-and 15-bit ADC's and thus there is a requirement for

22
precision ramps whose amplitude is known within 100 ppm at any point in time to permit accurate

tracking and aperture time measurements.

13. Voltage and current measurements and their metrological support will continue at high

level but in the context of rapidly changing levels of voltage and current. The question will become

not how accurately can a DC voltage measurement be made but how accurately can it be made in ten milliseconds for example.

14. A major problem is the fast, accurate measurement of voltages and currents (low level) over a broad temperature range.

15. There is a need for improved speed, range and accuracy in AC voltage measurements.

(Example: 0.01 dB resolution at 1 ms speed from +10 to -70 dBm, 600 ohms.)

16. Ideally, some substitute for the slow responding thermocouple is desired for true RMS

measurement of AC signals. The need for increasing accuracy into the 5-10 ppm area also exists.

We would also like to see higher voltage calibration available at the high frequencies, e.g., 30-50

volts at 500 MHz.

17. Some rapid means of calibrating AC voltages to +0. 01% from 1 mV to 1000 volts, 10 Hz

to 100 kHz is required. Present techniques involve the use of thermal voltmeters with time expenditures of 1 hour or more.

18. Required is the calibration of RMS voltmeters for non- sinusoidal waveforms (high

impedance input), or method of measurement.

19. The vast area of improving our ability to test to precision levels on semiconductor

wafer probing and under computer control has unlimited areas for exploration.

20. There is a need for current noise standards for calibration of current noise meters.

21. It would be desirable if NBS could provide a standard flat pulse and/or certify aberrations of a pulse generator submitted for examination.

22. Techniques for performing low-level pulsed measurements in a high noise environment

are required. (Example: Input - offset voltages and currents for Op-amps.)

23. What techniques are available for calibration of amplifiers, filters, and transducers by

using step input response or impulse techniques?

24. If the stimulus is locally generated, an effectively high sample rate 1/AT can be

achieved by repeating the stimulus and sampling at an interval equal to the stimulus sample interval

plus nAT where n is the number of the current repetition. This technique requires tight control of repetition interval and AT as well as a small sampling aperture or at least high precision on the

width of the aperture. A AT of 100 nanoseconds with an aperture accuracy of 1 nanosecond is

needed.

25. There is a need for simpler, cheaper, fully programmable oscilloscopes.

A. 1.4 Systems

Time as a Parameter How accurate, how fast

Noise as a System Parameter Generation and immunity

23
Systems Problems Grounding, interaction

interfacing, switching, etc.

Calibration In situ vs. transporting,

self-calibration, optimum

intervals

Environment Temperature, relative

humidity

Software Program aided/enabled measurements

Program library

Applications programs

High level executives

1. In medium speed (10-1000 readings/sec) measurement systems, comprising typically

signal conditioning, a-d conversion, and some output means, either visual or electrical, there are many time related performance parameters which are important in automatic or semi-automatic

operation. These parameters are many times subtle, at times difficult to measure, and typically are inadequately specified or ignored by manufacturers. In particular, I am referring to timedependent parameters such as settling time of signal conditioners as differentiated from a-d conversion time and total time per digital output cycle. The settling time of a signal conditioner, or for

that matter the a-d itself, can be a function of its condition prior to and during a reading; i.e.

overload recovery, recovery from common mode step or transient input, settling when called from a

different range or function, etc. All of these time dependent parameters are a specification or measurement of the response of the system to transient excitation and are directed at identifying the

degradation in performance. Some commonly acceptable criteria for the description of this degradation would be most helpful. The techniques for the measurement of these parameters become

increasingly difficult as speed is increased and accuracy is improved.

2. Required is the measurement of settling time to high amplitude precision for high-speed

systems (will need 0. 001% in 10 seconds ultimately); also, acquisition time measurements.

3. What are the limitations in performing AC parametric tests (rise time, fall time, etc.

on digital IC's on a high volume basis? (Example: What is the shortest rise time, fall time, propagation time that can presently be measured on a high volume test system?)

4. Investigate the economic realities of remote operation from a central process area versus design of a specific dedicated system.

5. What is the ratio of noise susceptibility of an automatic calibration system to that of a

typical laboratory calibration setup ?

6. I would like to mention noise sensitivity in systems environments. Typical measurement techniques and specifications for normal mode and common mode rejection (NMRR, CMRR) are

inadequate to describe performance in actual systems - particularly when interference may be highlevel, broad spectrum noise caused by large analog signal switching or high speed logic signals.

7. Methods of eliminating digital noise on the output of low-level analog signals is a necessity. This may require the development of new instruments, modification of present ones or development of improved filtering devices.

8. The advent of the mandatory 3-wire power cord with its attendant ground loop problems

will require some consideration, as will guarding of instrumentation.

24
9. What is being done to simplify interfacing the instrument to be calibrated to the auto- matic calibration system?

10. Evaluate the performance of these systems "in situ".

11. Development of reliable coaxial switches is a must. Additional work on the reliability

and repeatability of system switches (reed, crossbar, coaxial, etc. ) is required.

12. Complex test equipment requires routine calibration "in situ".

13. What can be done to resolve differences in test system calibration under controlled

temperature environment (laboratory) as opposed to the widely varying temperature environment of the field?

14. System Calibration: The Air Force approach to automatic system calibration is from a system approach rather than the calibration of individual modules, i.e. , all system parameters are

specified at the system interface.

15. A major problem we (and I assume others) encounter with ATE's is to assure maintenance of calibration traceability to NBS. The normal method used is to remove the component pieces

from the system, transport them to the standards laboratory, calibrate to specs, return to the system and perform a functional test. We, as well as many of our instrument customers, have questioned this procedure in light of the many subtle errors possible (the obvious ones include temperature, humidity, grounds, etc. ).

16. What is the uncertainty of measurement limitations for the automatic calibration as compared with the typical laboratory calibration setup ?

17. What factors should be considered in establishing an acceptable calibration interval for

the automatic calibration system?

18. What are acceptable techniques for calibration of multiplexed systems, i.e., frequency

response and linearity?

19. Develop means to evaluate system immunity to temperature, humidity and electrical

noise.

20. Some hardware or software method of insuring flat response of system stimuli would be

useful. Present methods use software correction for cable and switch losses which are determined

using thermal techniques. A more reliable method appears to be the development of a leveling

device to be used over the frequency range from DC to 500 MHz.

21. The use of software corrections to the outputs of stimuli should be investigated. This

will permit an increase in system accuracy at the system interface by incorporating the corrections

found during system calibration. The most feasible method of accomplishing this seems to be

through the development of special instrument drivers made to accommodate the corrections.

22. What effort is being made to implement a user exchange of computer programs for the

automatic calibration system?

23. Application Programs: It is quite apparent that the only economic approach to application programming when a diverse load of instruments is involved is to use the generic approach.

Continued development of this technique is required along with innovative methods of testing to take

full advantage of the power of automatic calibration systems.

25
24. Development of time shared real-time executives seems to be an economic and practical

necessity.

A. 1.5 Other Input, Mostly Having to do with Calibration

and Product Testing

1. The frequency of calibration and out -of- service time for calibration must be minimized.

Various methods of doing this include:

a. Design into the equipment a "calibration required" indicator in lieu of calibrating at fixed time intervals.

b. Reduce the time items are in transit to calibration, awaiting calibration, and under

calibration.

c. Maximize the use of designs which require no calibration.

d. Design in self-calibration.

e. Use remote or on-site calibration.

f. Design in interfacing for calibration by automatic calibration systems.

2. What information is available on calibration intervals for test instruments? What computer programs are available for calibration interval determination?

3. What techniques are available for arithmetic determination of long-term instrument stability by using relatively short-term data?

4. The Bureau is not permitted to endorse or imply endorsement of a specific manufactured

product, but it would be very helpful if the Bureau could periodically disseminate information on the

practical state-of-the-art of various electrical and electronic measurements at various accuracy

levels from an objective viewpoint.

A. 1. 6 Other Input, but Not Related to the Work of the

Electricity Division

(Relevant divisions are indicated in brackets.

)

1. Microwave Power Measurement: Production test equipment is required to measure

power in certain frequency bands between 2 and 12 gigahertz with an accuracy of 0. 1%. [Electromagnetics Division, Boulder, Colorado]

2. Microwave Noise Measurement: Production test equipment is required to measure the

presence of noise at a fixed frequency higher (or lower) than a carrier frequency with an output of a few watts. [Electromagnetics Division, Boulder, Colorado]

3. We require high vacuum ionization gage calibration against an absolute standard in the

10" to 10" torr range. [Heat Division]

4. What techniques are available for computer -controlled calibration of accelerometers ? [Mechanics Division]

5. What techniques are available for computer-controlled calibration of load cells?

[Mechanics Division]

6. What is the state-of-the-art with respect to calibration of velocity systems? [Mechanics, Optical Physics Divisions]

7. We have other measurement needs such as good measurement of laser power, particularly peak power and the design and implementation of suitable testing for microprocessors which

26
may or may not fall within the area of discussion to be covered by the Workshop. [Electromagnetics

Division, Boulder, Colorado]

8. The second area which deserves greater attention is the development of a simple, reliable and inexpensive method for evaluating delivery of ultrasonic energy. There are wide varieties of

both diagnostic and therapeutic devices which employ ultrasonic techniques. Methods for evaluating

the output of sonic energy include force-balance, water displacement, calorimetry and calibrated

transducers. These techniques are in many instances cumbersome, time-consuming and complicated. To obviate some of these drawbacks, research and development into better and more accurate calibrated transducer techniques could be considered. [Electromagnetics Division, Boulder,

Colorado; Mechanics Division]

9. What instrumentation is available for dynamic calibration of accelerometers and load

cells at high levels? [Mechanics Division]

10. What is being done to standardize measurement of spring rate and damping of elastomers as used in dummies? [Mechanics Division]

A. 2 Notes on Opening Plenary Session

These notes were prepared by the moderator and are based on the discussion at the opening

plenary session. The purpose of the session was to introduce the attendees to the National Bureau of

Standards and the Electricity Division in particular, to ensure that everyone understood the goals and

modus operandi of the Workshop, and finally, to begin the discussion of the four main topical areas. The notes deal primarily with the latter.

Although many specific needs were touched upon during the opening session, some broad

major problem areas were also identified.

Interpretation of device specifications was one of the most important. It was felt that often,

NBS documents are written for the user who is already highly knowledgeable in the specific area, while the people really needing help are those just using the device, module, instrument or technique

for the first time. Thus, there would seem to be a great need in the field for statements on how

specification facts are derived, and on measurement limitations. To this end, the possible function

of the Division in a tutorial role was suggested. The Division could, for example, provide recom- mended calibration procedures - not tied to specific instruments, but rather to specific types of measurements.

A second major theme was time as a parameter in electrical measurements. A typical

example, and a kind of measurement presently more or less ignored, is recovery from high voltages

applied to low voltage scales - 1000 V on the 100 mV scale of a digital voltmeter, for instance. The

question of how long a wait is necessary to restore rated accuracy is seldom addressed, and is becoming increasingly important in systems applications.

A. 2. 1 Components and Parameter Measurements

Binary- oriented standards and methodology would be more consistent with many systems

applications.

In the past, we were designing instrumentation for the human being. Now, with computers, we can readily correct for significant non-linearities, hence we have more options.

What is of increasing importance is stability and reliability - advanced prediction of perfor- mance. Work is needed on the basic physics of devices for purposes of stability and failure prediction. 27
Also important is the prediction of the value standards will have in the future via extrapolation of measurements made today.

Increased accuracy is needed in AC/DC conversion - 15 to 30 ppm accuracy out to 100 kHz is required.

Systems must -eventually give automatic indication of out-of-calibration conditions.

With systems, the important place for meeting specifications is at the customer interface, e.g. , where the instrument under test is connected to the system.

A. 2. 2 Signal Conditioning and Data Conversion

Dynamic accuracy is the major area of difficulty. We need to understand the role of settling

times in transient measurements. Perhaps frequency domain, via crosstalk measurements on DAC's or ADC's used for speech, is the way to go.

In sampled data measurements, inaccuracy is seen as noise. What is needed is a characterization of the noise - dynamic features of the error - such as peak amplitudes and spectral composition.

It is important to distinguish between correlated and uncorrelated noise.

In the operational dynamics of specifications, once you have defined how to measure the

noise, you have then specified it.

Failure mode investigation is a significant area for Division effort.

Ground currents in systems can cause strange phenomena in measurements.

Of great potential benefit are Bureau-certifiable standards for field use at higher accuracies

than are presently available.

hi the field, instruments and systems have pickup problems from impulse sources. Also

needed is equipment to perform IEEE Surge Withstand test.

The sampled data approach is really a holdover from manual techniques. It should be possible to just take a few measurements, on the fly, then process them to predict the final value. The

difficulty arises in relating results to what we already know. A particular example might be

impulse response testing.

A. 2. 3 Sources and References

Sampling measurements occasionally give erroneous data, i.e. , data outliers. There is no easy means for recording such transients.

The 3-sigma limit is often called into play to take care of the outlier problem in specifications.

Lifetimes of implantable energy sources must be evaluated in advance.

A further problem is to determine when NiCad's or other batteries cannot be recharged

again.

A certified flat pulse is crucial to settling time measurements.

28
Cables and terminations are often crucial system -limiting elements.

A. 2. 4 Systems

It would be helpful if the Electricity Division were to disseminate information about its on- going work.

Software is the biggest single system problem area - it would be helpful if the Bureau could

make recommendations on preferred approaches, etc.

Calibration of sensors and transducers is an important area.

Adding protective devices to a design often changes the calibration, yet this is not widely

known and its implications are not fully recognized.

It would be extremely useful to have a specification on the standard AC power line for all types of performance including spikes due to lightning, etc.

A. 3 Working Sub-Group Reports

The following four working sub-group reports were prepared by the cognizant NBS Recorder

with the assistance of the appropriate NBS Participant and some additional input from the working

sub-group Chairman. Final editing was the responsibility of the moderator. The reports cover all of the working-group deliberations carried out during the course of the Workshop. It should be

noted that while the sub-group participants changed during the two days, the Recorder, NBS Participant, and Chairmen remained the same, thereby providing continuity.

A. 3. 1 Components and Parameter Measurements

(E. Boeckmann, Chairman; M. Oldham, NBS Recorder; W. Eicke, NBS Participant)

A. 3 . 1 . 1 Introduction

The need for dynamic rather than static measurements of components was voiced by nearly

all of the members of this sub-group. Rapid DC response was considered more important than the

conventional frequency response to sinusoidal signals. Requested frequencies of these "pulse" measurements ranged from 10 Hz to 10 Hz (50% duty cycle); and 1 V to 10 V energizing voltages,

variable pulse to pulse, would be desirable.

A. 3. 1. 2 Capacitors

The sampling rate and accuracy of sample -and -hold amplifiers are functions of their hold

capacitors' pulse-frequency response. In order to improve these and other devices, more information on capacitor parameters, specifically dielectric absorption, must be made available. It was

generally felt that NBS could be of great assistance to the industry in the following ways:

1. NBS should set measurement guidelines and specify test conditions for these parameters.

Pulse -response is preferred to static tests. NBS should also establish a calibration

service in this area with a possible MAP.

2. Dielectric absorption measurements on capacitors from 10 pF to 1 /uF with an accuracy

of at least 1000 ppm, and possibly as high as 10 ppm, are desired.

3. Standards of dielectric absorption were discussed briefly, with polystyrene and nonsolid dielectrics being of prime interest. The guarded air capacitor was recommended

as a possible standard.

29
4. It was generally felt that NBS could also assist the industry by doing more basic dielectrics research.

A. 3. 1.3 Resistors

Sixteen-bit processors which resolve 1 part in 65000, and fast laser trimmers which trim

many resistors a second, are two examples where high accuracy resistors are used in high speed

measurements. It was generally agreed that a "pulse-response" resistor calibration service was

needed to cover the following:

1. Resistors used in the ladders and other types of networks used in DAC's and ADC's.

Generally, the range is from 10z Q to 10 Q. Present requirements are:

a. 0. 1% pulse response to rep rates of 10 Hz

b. 1% pulse response to rep rates of 10y Hz

2. It has been requested that NBS offer design considerations for high speed resistors;

improved theory is needed. Requests were made for improved film resistor design.

Ten ppm long-term stability is the goal.

3. Binary values are being used more and more in modern processors. The need for a

calibration service for other than base 10 resistors is increasing.

4. Temperature coefficient measurements have been a particular problem to some. It was

not clear why there is so much confusion measuring this parameter but at the mention of a special calibration service for resistor temperature coefficient, several resistor

manufacturers indicated interest. There was also interest in a MAP.

5. In 16-bit processors, resistor ladder and other network ratios must be known quite well.

There is a present need for static DC ratio measurements to an uncertainty of 1 ppm.

Some manufacturers of these networks calibrate them statically. They are then used in

high resolution processors under a variety of dynamic conditions, apparently on the

assumption that thin films have good frequency characteristics. The need for network

ratio pulse -response was only touched upon and no figures were given. It was generally

agreed that a DC or low frequency (LF) AC-to-pulse transfer technique would be useful, so that a one time difference could be measured and a DC or LF technique could be used

thereafter for recalibration.

6. The measurement of resistance from 10 Q, to 10 Q, was of general interest. There was agreement that NBS should set measurement guidelines with emphasis on the parameters which need specification. Commercial units lack stability and improved designs are needed. It was mentioned that NRC offers a calibration service at 10° Q with a

slightly smaller uncertainty than NBS. It was pointed out, however, that the present NBS

uncertainty from 10 fi to 10 Cl is less than 35 ppm and less than 0. 1% from 10 Q to 10 Q . This accuracy is not offered because of the instability of the resistors sent in

for test.

A. 3. 1.4 Phase Angle Measurements

Currently marketed phase meters claim the ability to measure the phase angle between two

voltages to 0.01 degree. A calibration service is not available at NBS and apparently no traceable

measurements are available in the industry. The project on phase standards currently being undertaken in the Electricity Division was discussed briefly.

30
Phase angle of component impedances was discussed more thoroughly. Some resistor - capacitor measurements presently done at NBS offer or could offer a phase angle determination to varying degrees of accuracy. Although these measurements are made with sinusoidal signals, it is

felt that the phase angle may be a valuable tool in determining pulse -response.

A. 3. 1.5 Noise

Noise measurements and noise standards were discussed briefly, with emphasis on NBS

guidelines and specifications. People concerned with noise were most interested in methods of measurement that could possibly be supplied by NBS. Noise standards from 10 kHz to 500 kHz were

needed not only for noise measurement but to determine frequency response. White or gaussian

noise was discussed, with Zener diodes mentioned as possible standards. Uses of standard noise

generators included component (capacitors and resistors) and product (Op-amps, hearing aids)

evaluation. Thin and thick film resistor noise was also of particular interest; more theory is

needed. It is felt that current noise (apparently due to film resistors and semiconductors) may be

the limiting factor in ADC design.

A. 3. 1.6 Other Topics

1. The need for more accurate measurement of distorted power was voiced but not elaborated upon.

2. A request for an NBS calibration service for inductors from 100 piH to 10 H, was made.

Frequencies from 200 Hz to 2000 Hz at accuracies from 10 to 50-ppm are desired.

3. It was generally felt that NBS should conduct a "pulse" study before getting too deeply

into pulse-response measurements.

4. Transducer interfacing has been a constant problem to many people. Transducers may

respond differently to different readouts. Greater interaction between the transducer

testing laboratories (at NBS) and the customer is indicated.

5. Methods of short-term evaluation of long-term performance are needed by all segments

of the industry where lifetime is a factor. Resistor and network stability was of prime

concern but the evaluation of medical implant batteries was of interest to FDA.

6. Of further interest to FDA was calibration support for medical monitors and analyzers.

There is presently no control on these devices once they are purchased and calibration

services are hard to find.

A. 3. 2 Signal Conditioning and Data Conversion

(D. Ludwig, Chairman; T. Souders, NBS Recorder; H. Schoenwetter, NBS Participant)

A. 3 . 2 . 1 Introduction

The Sub-Group opened with a discussion of the proprietary aspect of measurement problems,

and a suggestion that not all members might want to discuss problems among each other. As a

result, NBS might have to re-invent measurement techniques withheld by manufacturers that believe

the ability to make better measurements gives them an edge in building better equipment. Others

felt that good test methods should benefit all, and hence should not be proprietary. In an anecdote

relating past experience with NBS on inductive voltage dividers, one participant pointed out that in

the long run manufacturers will benefit from educating NBS about their measurement techniques. In the example, NBS eventually "took the ball", and was able to offer technical assistance which the

manufacturers could not afford on their own. NBS's knowledge and approval helped the market as

well.

31
What sort of role should NBS play? Since parameters must be well defined before meaningful

measurements can be made, NBS could start by characterizing devices realistically (as opposed to some Mil-Specs), considering input from both manufacturers and users. Considerable concern was

expressed by manufacturers that this characterization be realistic, and not tie their hands. Next,

NBS could develop standard testing procedures suitable for the user, who will benefit most from the

measurements. Such standard procedures would be taken as a yardstick for determining the correct

values, with any other technique giving comparable results being acceptable. In this way, manufacturers need not be limited to NBS's recommended techniques for production testing if other techniques give acceptable results.

It was pointed out that in many instances, linearity is much more important than absolute

accuracy, the latter being achieved by the user with external or built-in "twiddlers". Hence, NBS

should probably concentrate its efforts on linearity rather than accuracy.

A. 3. 2. 2 Settling Time

Settling time is a measurement parameter common to most analog circuits, e.g. , amplifiers, DAC's, ADC's, sample-and-holds, as well as R, L, C components and integrated components.

Four years ago, no specifications were available on this parameter, and even now no products are

available to measure it. An accuracy of 0.01% in 20 ns should be strived for, but 50 or 100 ns would still be very useful. Because of thermal drifts, etc., of the settled value, a definition is

needed to determine when the final reference value is taken. The measurement technique must be

made suitable for use with a reference input pulse (as from a well characterized flat-top generator)

for amplifiers, ADC's, etc., as well as for use without any reference signal, as in a DAC. A ref- erence source should be a flat -top generator adjustable to known, precision levels (settling time is not linear and should be measured over a wide dynamic range), having both polarities available,

having well controlled rise time with low overshoot, and a top holding flat for at least 1 ms.

NBS should consider the possibility of a transportable standard of settling time. It was suggested that settling time and accuracy be kept independent, i.e., the accuracy can be tested after the

device is well settled. Dynamic accuracy testing however, may be helpful if the final value is drifting.

Although great difficulties are anticipated, the measurement of settling time on a wafer

probe basis would be helpful.

It was pointed out that even though some devices do not really settle during the observed time

interval, the lack of settling is constant, and can be compensated for. A large disparity in measurement ability exists between manufacturer and user, hence the user will benefit most by NBS's work in this field.

A. 3. 2. 3 Characterization of Data Conversion

Characterization of data conversion and its dynamic test conditions was a commonly mentioned problem. The dynamic input impedance of ADC's and DAC's, for example, needs specification. How many different operating points should be tested? How are noise and uncertainties in ADC

transition points (e.g., due to past history) to be specified? How are the make -before -break dynamics of multiplexers characterized? (Low-level sources may have high-level voltages momentarily

applied to them by make-before-break action.

)

Dynamic response specifications and test methods are needed for voltage-to-frequency converters. Temperature hysteresis is a problem with them.

32
A. 3. 2. 4 Noise

Non-gaussian noise is prominent in semiconductor circuits. Better understanding of the distribution and effect of popcorn noise, for example, is needed, along with a more meaningful characterization, since no conclusions as to peak-to-peak noise can be drawn from RMS measurements.

Correlated noise, e.g. , chopper noise, is another common non-gaussian noise to be considered.

Him resistor noise is still another area needing further study.

A. 3.2.5 Sample -and -Hold Measurement Problems

Again, definitions of parameters such as aperture time and acquisition time are needed as

well as methods for measuring them. Means are needed for both characterizing and studying memory effects on a components level, e.g. , studies of soakage or dielectric absorption of capacitors over short time periods. NBS could serve well in this area.

Development of special test generators, such as precision ramps (up-down with corners),

would provide a valuable measurement tool. Measurement of acquisition time should strive for

0.01% in 20 ns, and aperture uncertainty should be known to 100 ps or better.

A. 3. 2. 6 ADC's and DAC's

A precision programmable power supply or DAC, having 10 ppm or better accuracy, with

floating (+1 V) output would be helpful for testing ADG's and DAC's.

Multiplying DAC s are important large-volume market items, and present characterization

problems. Could NBS develop a standard recommended test procedure which a user could employ?

A. 3.2.7 Comparators

Comparator noise is an important consideration in determining the real accuracy of ADC's.

Comparator response time needs to be specified for small as well as large error signals.

A. 3. 2. 8 Nonlinear Modules

The problem with nonlinear modules, e.g., XY, XY/Z, log X, is predominantly one of definition and characterization. Accuracies are usually modest (^1%) and rather easy to measure.

Interest was expressed in RMS measurements for nonsinusoidal waveforms with crest factors in the

upper range of 10-12, full scale.

A. 3. 2. 9 Long Term Performance, Temperature Effects

and Component Studies

Correlating long-term stability with short term measurements, such as cycling and burn-ins,

is a very important goal. No good means are at hand for studying the problem, and manufacturers

cannot afford the expense to make the necessary studies. It was suggested that NBS is perhaps the

only reliable group that could conceivably afford to make such studies. As a start, it was proposed

that NBS study component stability (film resistors, capacitors, semiconducting devices) with a view

towards evaluating long term stability and temperature effects. The stability of resistor ratios was

emphasized for this study. The semiconductor studies of the Electronic Technology Division were

used as an example of the type of work needed.

Characterization of temperature effects was considered a big problem, and this can become

even more confusing in monolithic circuits when the thermal time constants become so small that

they look like other time problems.

33
A. 3. 2. 10 Financial Impact

It was estimated that better specification and measurements might double the gross volume of

the signal conditioning and data conversion market, which presently amounts to about $50 million

annually for DAC's and ADC's alone. One example was given of a $200, 000 loss due to misspecification of an order. This would have been prevented had standard specifications or characterization been available.

A. 3. 3 Sources and References

(D. Lawrence, Chairman; B. Field, NBS Recorder; R. Turgel, NBS Participant)

A. 3. 3.1 DC Standards

In the discussion of DC standards, the shortcomings of saturated standard cells were emphasized. Cells are used today as secondary standards because of their stability and low noise; however, they are unsuitable for field use or production-line testing because they are sensitive to the

environment, specifically temperature. Temperature hysteresis, requiring recovery time of 24

hours or longer, or possibly even permanent change, is one of the problems. Recalibration of cells

that are physically transported takes at least two weeks, even under favorable conditions. The cells

provide only one volt output with no power (i.e., "zero" current). This means that 1 ppm (1 /zV) is within the noise range of most amplifiers. Transfer type measurements are required for other

voltage and impedance levels. In the standard cell, low noise is combined with high internal impedance, but selected solid state devices with relatively low output impedance can be produced with

noise as low as 1 ppm.

An alternative to a saturated cell is required, but it need not necessarily be a solid state device. For example, a mercury battery, periodically checked by an ATE, could be used as a short

term reference, because it has a predictable drift rate. As another example, the "volt" could be

piped around via a cable using a constant current source to eliminate thermal emf's. However, this

would cost more than individual Zener diodes, which can have a stability of 1 to 5 ppm over the temperature range of 10 to 50 C, and are more portable. Present solid state devices have temperature

coefficients of 2 ppm/ C over a 15 to 50 C temperature range. The coefficient is worse over the

military temperature range, but environmental control could be used for compensation. A Zener

diode can compete in accuracy with an unsaturated cell, if the current through the diode is held constant. The noise of the diode depends on the current used.

For many applications, what is wanted is a device that provides a wide range of voltages and

currents. These secondary standards should approach the accuracy of today's primary standards.

Ten ppm digital voltmeters are no longer out of the question; at least one manufacturer is working in

this area. A 5 ppm standard is inadequate for testing these instruments. If ATE will be used to calibrate the DVM's, then we need standards capable of 1 ppm accuracy with low noise. These are

desirable to eliminate the need for integrating measurements over a long period to reduce the effect of noise in the standards. Long integration times decrease the throughput of the ATE.

In summary, we want a new standard, an alternative to the standard cell. It should have

greater environmental tolerance, 1 to 5 ppm stability over 10 to 50 C, and when held at a constant temperature, have stability of 1 ppm. The noise should be less than 1 ppm. It should have 10 V and

1 volt (1. 018 . . . ) outputs, and be able to supply current up to 10 mA. Additionally, it should show

no temperature hysteresis and have a 1-hour or less warm-up time. It was concluded that there was an immediate need for 2 to 5 ppm Zener standards and a suitable calibration service at the one and ten volt levels. The representative of one module manufacturer in particular, felt that a MAPtype program for Zeners should be provided, at least at the 10 ppm level if not better. He stated

that he would be in a position to use this service in 1 to 5 years.

34
Ideally, this secondary standard should be constant, or at least behave predictably. If device

behavior can be predicted, then corrections can be made, for example, by a microprocessor

included in the unit. Corrections could be for temperature effects and/or drift. It was suggested

that an investigation be made of possible correlations between short-term Zener parameters and longterm stability. An effective method capable of predicting the long-term performance would have

great economic impact. This would enable manufacturers to spend less time testing Zeners, thus

producing reliable Zener references (and DVM's) at lower cost. NBS could possibly evaluate the

long-term performance of Zeners produced by industry. The representative of one instrument

manufacturer mentioned that at present, in DAC's, references are adequate and frequency compensation of the voltage divider is the critical problem.

A. 3.3.2 Band-Gap Reference

An alternative to the Zener diode, a band-gap reference, was suggested. The possible application of this device as a voltage reference was reported by K. E. Kuijk in the June 1973 issue of the

IEEE journal of Solid State Circuits. This device is temperature dependent, however, the environment could be controlled. The advantages are that this device is based on an intrinsic property of

silicon, and can be operated from a five volt supply voltage and thus can be included in logic devices

without the requirement of an additional power supply. One participant had the impression that Mr.

Kuijk was making devices with a stability as good as 10 ppm a year.

It was pointed out that as Zener diode stability improves, the needed accuracy of the stan- dards required to calibrate them would increase. Thus work should continue in making the Joseph- son effect voltage standard easier to use.

A. 3.3.3 Noise Sources and Standards

A number of questions were raised, including: Is there a standard for noise specification?

What type of device should be used to measure noise, and how do people measure how much noise is in a circuit? What happens to an AC meter when a noise source is put on the input? Should noise be

measured with an RMS meter? With a filter? We need a standardized procedure to measure white

noise from 10 Hz to 30 MHz. An RMS measurement is theoretically correct. Possibly AC devices

could be calibrated by noise sources instead of sine waves. Perhaps a one point measurement could

be made using a calibrated noise source instead of a multipoint frequency test. It became evident in

the discussion that there is little or no agreement on how noise should be specified and measured.

A. 3. 3. 4 AC-DC Transfer

The problem of AC-DC transfer is directly connected with the problem of developing and

checking precision AC sources. AC sources must be capable of supplying the currents necessary

for the thermal converters with which they are tested. At high voltages the power required is not

negligible. The impedances presently are determined by frequency considerations, but a wide range

instrument would reduce the supply requirements. Most thermocouples have to be used between 50%

and 100% of full scale. This necessitates having many devices on hand. An instrument that could be

used from 10% to 100% of full scale would be more useful. One participant commented that promptly

publishing work done at NBS on AC-DC transfer techniques would help bring the accuracy obtainable

in industry more in line with the state-of-the-art. Accuracies of calibrated AC-DC converters at audio frequencies were generally considered to be sufficient for the present, however, 10 ppm was

desirable in the future so that 30 to 50 ppm could be obtained in the field. The major restriction

v/as at high frequencies (1 MHz to 100 MHz). Very high frequency satellite communication signals

need to be measured at 70 MHz to 100 MHz to better accuracy than is presently available. For

instance, a 13-MHz generator has been developed with a resolution of 0. 01 dB and cannot be calibrated to that accuracy because of a lack of a suitable NBS calibration service. On the other hand,

it was commented that accuracy requirements for AC-DC converters should be examined as to user

needs, and not product requirements.

35
A major problem with thermocouple devices is that they are slow. A faster responding

device is needed. Digital sampling is a possibility and will allow a certain amount of harmonic content without degrading the accuracy. Even for a small number of samples a good RMS reading can

be calculated. We need an RMS-responding device with time constant a factor of 10 lower than present thermocouples, that is the ability to make 10 measurements a second at frequencies up to 1 MHz,

with a minimum accuracy of 0. 1%. The system would then be compatible with an ATE, and timeconsuming spectral purity measurements (needed with average response readings) could be eliminated. This would result in increased throughput. Reduction of the DC reversal error (or alternately, the ability to predict the error) will also reduce the time required to make the measurement.

In some cases the RMS response to nonsinusoidal AC quantities depends on the crest factor.

A generator that is calibrated in crest factor could be of value for such AC calibration.

A. 3. 3. 5 Battery Life/Capacity

There exists a need to determine the remaining charge of a battery without discharging the

battery. Batteries used to power standard cell enclosures during shipment and batteries implanted

in the body have to have a known capacity. Perhaps an extra electrode included in the battery could

provide this information. Determination of the remaining battery capacity is required for both

rechargeable and nonrechargeable types.

A. 3.3.6 Pulse Generators

Dynamic testing of analog devices is often done using a flat-topped pulse, but this pulse is difficult to obtain. The response of a system to an ideal flat -topped pulse can be calculated if it is excited with an exponential shaped pulse. This pulse is relatively easy to generate. Some applications will still require a good square pulse. To test the vertical deflection amplifiers of oscilloscopes we need a pulse with 200 ps rise time and with aberrations on top of the pulse of less than 1%

for 10 ns after the comer. A repetition rate variable from 50 Hz to 50 MHz is required, although

not necessarily all in one unit. Pulse generators are also used to measure settling time. A cus- tomer needs a general purpose instrument for verifying manufacturers' specifications on devices.

Ten nanoseconds rise time, with 30 ppm aberration on the step for 10 seconds, and a repetition rate

of 0. 1 Hz to 10 Hz would be sufficient.

A. 3. 3. 7 Ramp Generators

It was acknowledged that precision linear ramp generators could be used for aperture time

measurements. A ramp with 100 ppm linearity was considered adequate. It was mentioned that aperture time can also be measured in the frequency domain by special techniques. No one admitted

to any particular expertise in this field.

A. 3 . 3 . 8 Impulse Te sting

Impulse testing is desirable for testing on line. Frequency sweep methods (even using

ATE's) require the device to be removed from service. Using impulse testing, an initial applied

impulse can be quickly followed by a cancelling impulse, thus the device can be tested without significantly disturbing the system. Impulse testing can also be used to calibrate a closed loop control

process. The problem is that we are looking for a small change from standard behavior resulting

from the perturbing impulse. What changes in the output are evident from 5% to 10% changes in the

parameters of the components in the system? The procedure and techniques must be investigated

and well documented to convince skeptics that impulse testing is reliable. To make full use of this

technique the parameters of the perturbing pulse must be known and described accurately.

A. 3.4 Systems

(F. Seeley, Chairman; R. Kleimann, NBS Recorder; J. Morrow, NBS Participant)

36
A. 3. 4. 1 Introduction

The Systems Sub-Group attempted to limit its consideration to those problems specific to and

inherent in systems per se. It was generally agreed that systems to be considered are characteristically assemblies of electronic or electrical hardware, usually including some type of software, and

designed for very specific applications. It is common for digital computers or other programming

equipment to be used for rapid and reliable control of modern electrical test, measurement, or control systems. The fact that systems are designed for specific functional applications is as much a

distinguishing characteristic, perhaps more so, than the fact that they may contain some particular

set of components, are expensive, or are large and complex.

It has been a common experience among systems designers and users to find that, when hard- ware components and software are assembled and operated as a "system", problems exist which are

not found when these same components are used individually. A new environment is created within a system in which occur electrical interactions and interference among measurements, stimuli,

switches, controllers, and even commercial power sources. The problem facing the designer and user of systems is to understand this new environment, and to develop techniques for operating within or minimizing the deleterious effects of the systems environment. A related and crucial problem

concerns the ability to determine whether the process or test being conducted by a system is, in

fact, valid. The problems of system environment and system validation are two general areas that have been identified for investigation by the Electricity Division at NBS.

A. 3.4. 2 Performance Validation: Static

The problem of validating the steady state performance of a system was discussed, and needs

established. Specifically, transportable standards capable of providing steady state inputs and/or

measurements to a system at its interface ports are needed. These standards will then provide a means of determining what a system's steady -state measurement capabilities are in its specific cli- matic, electrical, and operational environment. If NBS were to become involved in such in-system

verification through the use of transportable electrical standards, it would be an extension of the

Measurement Assurance Program concept to automated systems. However, it was pointed out that a MAP service for ATE may be difficult to implement because of the diversity of ATE systems, particularly in terms of interface and software differences. The electrical quantities of principal interest for steady- state include:

AC Voltage: To 1000 V, 20 Hz to .10 MHz,

uncertainties 1% to 20 ppm

DC Voltage: To 1000 V, 0. 1% to 10 ppm

Resistance: 0. 001 Q to 100 MO, 0. 1% to 10 ppm

Pulse Characteristics: Rise, fall times of 10-30 ps;

exponential decay, 1% to 10 ppm in

amplitude uncertainty

Constant Current (DC/AC): 1 nA to 10 A, 0. 1% to 50 ppm

A. 3.4.3 Performance Validation: Dynamic

The problem of chief concern is whether a system will correctly perform its intended function, e.g., reject bad parts, accept good ones, properly diagnose faults, make measurements to proper tolerances, etc. The System Sub-Group felt that inadequate methods exist for validating

system performance in the systems environment. Because of the speed at which modern systems

operate, many time -dependent and history -dependent phenomena can no longer be considered

37
negligible during system operation. The specific ways that measurement, stimuli, switching, and

interconnection components interact with the outside world in dynamic tests is not well understood.

Needs include means for proving that systems meet their intended objectives:

1. Specifically, in order to validate dynamic performance, it was recommended that

"dynamic physical and paper standards" be developed.

2. Among other things needed are techniques for measuring and characterizing the time

dependence of AC and DC sources and references, as well as components such as resistors.

3. An additional need is methodology for characterizing source and instrument performance

in the face of dynamic switching and dynamic loading conditions found in systems.

4. Finally, a need was discussed for the development of measurement techniques to be used

to measure the pulse characteristics found in digital logic testers. In systems of this

type, many pulses are simultaneously provided at multi-pin interfaces and are delivered

at a 4 MHz rate. The pulse characteristics of interest include rise -time, fall time,

amplitude, overshoot, skew, etc.

A. 3.4. 4 Data Outliers

Another aspect of system performance validation was discussed as a possible need. Frequently, users of automatic systems will obtain data or test results which are unexplainably bad.

The usual practice in this case is to immediately repeat the test in the hope that the outliers in the

data will not reappear. If this action succeeds, many system operators will completely disregard

the outliers without looking for any cause-effect relationship. The Systems Sub-Group could not agree on any specific need, but the frequency of such occurrences is such as to make the presence of data outliers a disturbing phenomena. Perhaps what is needed is a methodology for handling these

circum stances

.

A. 3. 4. 5 System Environment

There are two strategies discussed for dealing with the problems of noise and electromagnetic interference. The first of these strategies is to determine the causes, and means of reducing,

noise levels in systems. One major area identified was the need for definition and characterization

of pollution of the power line, both from external sources entering the system environment, and

from effects of the system itself. This definition and characterization could result then in a written

document of recommended specific practices and techniques such as shielding, guarding, grounding,

isolation, etc., which were found to be effective.

A second strategy which may be effective recognizes that noise can never entirely be eliminated and that measurements must be made in noisy environments. Therefore, it is recommended

that consideration be given to a study of system measurements and definition of the limits or effects

of measurements made in the presence of noise. One possible approach is the generation of known

noise conditions to determine their effects on system performance or the effectiveness of reduction

techniques.

A. 3. 4. 6 Detailed Session Notes

While results of the Systems Working Sub-Group sessions have been summarized in the preceding paragraphs, the actual session notes follow, to provide additional context and detail.

One system manufacturer requires a system test of all possible error factors, to be used in

both certifying system operation and diagnosing faults. Another participant described his areas of

38
responsibility in the crash testing of automobiles. His problem, too, was described as certifying

the operation of his system before, during, and after a test. The system' involves the measurement

of physical parameters during a very short time, a few seconds at most. The economic considerations are extreme, so the data acquired in a test must be usable and reliable. In addition, some

lawsuits involving the real occurrences being simulated can run into the millions of dollars. Thus

the data must be so reliable that it can be used in court. The system described by another manufacturer is a redundant computer, data acquisition system using remote sensors connected via 30 baud

telephone lines to detect illegal entries, etc. This system, too, requires some method of certifying

correct operation throughout, from the sensors, through A-D converters, etc., to the output. Also

needed is a software algorithm for linearizing the nonlinear response of the sensors.

Another participant described his needs in regard to systems which monitor various parameters in power sub-stations, calibration of customer meters, and instrumentation for power control.

The main problems are the environment of a sub-station containing motors, generators, turbines,

high voltages, and currents; a need to predict failures in various pieces of equipment; measure- ments in the UHV range (1-2 million volts); measurement of corona power dissipation; the need for

guidance in certifying software systems and programs, and in verifying their correct operation.

One system manufacturer stated that one of his problems is a lack of knowledge about what is available at NBS. He suggested that NBS should supply standards for use in ATE's. He elaborated

by saying that the standards now supplied cannot be used in ATE's at a typical ATE speed because

only a long-term value of the standard is supplied. What is needed is a dynamic standard (possibly

programmable) to be interfaced with the ATE, and checked as a reference during a normal run of the

system. It developed later that the economic motivation for this type of standard is that after spending hundreds of thousands of dollars for an ATE, it is very expensive to have to wait for the stan- dards to settle during calibration or to have to tear down the ATE and send parts to a standards

laboratory for calibration. It is necessary to have standards, usable in the environment of the system, with known values and uncertainties as a function of time. For example, the value and uncertainty of a standard resistor, 20 ms after it is switched into the measuring system, must be known

in order for it to be useful as a reference in a system where perhaps 100 measurements are taken

per second. Another important need for such a dynamic standard is for AC references with settling

times in the range of 100 /is to 1 ms, with accuracies of about 0. 1%.

Further general discussion showed that the need to know what the system is doing is very

great. That is, a way of verifying system operation must be found in order to establish credibility

in the system. It was generally agreed that the standard should be brought to the system, rather

than vice versa. The problem of what to do about systems which do not measure but rather acquire

and process data (such as security systems, scanning microscopes, etc. ) was discussed. One possibility mentioned was that NBS might develop internal references to be used within a system of this

sort.

It was suggested that NBS determine the effect on specifications of instruments, components,

and standards when placed in a typical use-type environment rather than a standards laboratorytype environment where the specifications are normally measured. For example, specifications

may require + 1 C, but the equipment may be used where there is no means of controlling the tern - perature. The role of NBS in this effort would probably be to develop techniques for testing instruments in a use-type environment rather than doing the actual testing. One way NBS could do this

would be to provide sufficient additional data with reports of calibration to permit the user to predict

the changes in performance to be expected from the changes in environmental conditions.

The standardization of software systems was discussed. For the most part, economic considerations can sometimes force standardization of software. The NBS Request for Proposal (RFP)

for 198 minicomputer systems may have already caused some interested manufacturers to modify

their systems to comply with the specifications of the RFP, according to an NBS participant. It was

generally felt that some way of verifying software systems was needed. It was suggested that NBS

39
develop a software package of calibration techniques, guidelines, and a list of pitfalls in the use of

ATE measurements.

Additional discussion concerned the problems of validating system operation. The problem

of interaction effects was introduced. These effects are brought about when one part of a system

causes an error in another part. Some examples given were: Stimulus/measurement interaction;

history-dependent phenomena, such as stored energy in a switch; and time -dependent phenomena. It was felt that NBS could help with these problems by developing transportable standards usable on a system in a system environment. A list of various standards was made by obtaining the preferences

of the members of the group. For AC voltages, standards up to 10 MHz, up to 1000 V, with sine,

square, ramp and sawtooth waveforms, rise and delay times of 10 to 30 ps and accuracies ranging

from 1% to 10 ppm were mentioned as desirable. For DC voltages, up to 1000 V, 0. 1% to 10 ppm.

For resistance, 1 mQ to 100 MO, 0. 1% to 10 ppm. For power, 60 Hz, 0. 1 W to 100 W, 1%. Also

listed were constant current sources, DC and AC, ranging from lnAto 10 A with accuracies of 0. 1%

to 0. 005% and a pulse standard with a rise time of 10 to 30 ps and an exponential decay thereafter.

The problem of needing standards and references operating at ATE speeds was brought up once more. Again it was felt that dynamic standards, wherein the values and uncertainties as a

function of time are known, are needed. It was felt that NBS could develop the techniques for measuring time dependence of AC or DC sources and references. It was also felt that NBS could characterize and measure the effects of dynamic load conditions and switching on sources and measuring

devices.

In reference to interaction effects, one participant described two of his problems regarding

the automatic testing of digital components. The first problem was in detecting faulty items. Some

items were called faulty by the system when, in fact, they were not, because of interactions of one measurement on the following measurement, a history -dependent interaction. Some other items were stressed so much by the test that they were more likely to fail when put into use, while for

certain other items the tests actually caused failures, again due to succeeding test interactions.

Another problem concerned digital logic testers in which a large number (300) of high rep

rate (4 MHz) pulses are applied to an item, and the characteristics (rise and fall time, amplitude,

overshoot, skewness) of a large number of output pulses must be measured. It was felt that NBS

might be able to develop techniques for these types of measurements, taking into account the interaction effects of the pulses.

The other major system problem discussed at this session was electromagnetic interference

(EMI) from various sources: Power transformers, power lines, etc. Most interference problems

stem from what was described as "power line pollution", i.e. , transients, spikes, etc. , which

cause the AC power line to deviate from a 60 Hz, 117 V sine wave. It was felt that, in regard to EMI, NBS should perhaps try to characterize and define "power line pollution", develop methods and

techniques of minimizing the generation of this "pollution" by isolating "polluters" from the power

line, and recommend practices for shielding, guarding, grounding and isolating equipment which

would be sensitive to the "pollution".

It was also felt that NBS could define the limits of or effects on measurements made in a

"noisy" environment. It also may. be possible to define or generate a standard "noisy" environment

for testing purposes.

The rationale behind the desirability of NBS setting up definitions, measurement techniques,

etc. , is that it will provide marketplace equity and better define contractual obligations between system manufacturers and their customers. For example, the manufacturer or customer may not be aware of or cannot be aware of the present and/or future environment in which the system will be

operated. This type of situation caused a $320, 000 lawsuit in one case. If there had been a defined

"noisy environment" the suit may have been avoided.

40
The problem of data outliers or "glitches" was discussed. What do you do about "glitches"?

How do you identify "glitches" as opposed to true values, etc. ? It was believed that NBS might be

able to devise a methodology for identifying and handling these data outliers.

The philosophy of using idealized stimuli in tests was discussed. It was felt that the use of idealized or nearly perfect square waves, for example, to test a component which will be used in equipment in which the square waves are not anywhere near perfectly flat, seems to be fallacious.

Other topics briefly mentioned were: The problem of dielectric absorption, and the calibration of capacitors at frequencies higher than now supplied. Also, many members of the Workshop

seemed to be unaware and, indeed, find it hard to believe that NBS is not a regulatory agency of the

Federal government. In this same vein, it was also suggested that NBS could perform a useful service in defining what constitutes traceability.

A. 4 Notes on Sub-Group Chairmen Oral Presentation at Closing Plenary Session, with Associated Discussion

The next four sections are verbatim transcriptions, edited by the moderator only for style,

of the oral presentations of the four working sub-group chairmen at the closing plenary session.

The questions and answers which followed each of the presentations are also included.

A. 4.1 Components and Parameter Measurements

(Presented by E. Boeckmann)

There were discussions on high-speed measurements on capacitors, including the problem of

dielectric absorption. People would like to see effects less than 1000 ppm, and possibly as low as

10 ppm over a range of capacitance of 10 pF to 1 juF.

Also, there was a feeling that the older methods of characterizing capacitors in terms of dielectric constant and dissipation factor, and so on, are now somewhat inadequate. We are more

interested in the response to pulses of voltage, usually in the 1 to 10 volt area, to 10 MHz with a 50 percent duty cycle, as a typical set of criteria.

Phase angle measurements may be of some interest in this area, and there may be a need for

material studies in terms of what types of dielectric materials could contribute an improvement, or what types of new materials might be developed. There was a discussion of the type of standard

capacitor, air dielectric, or what have you, that could be used as a standard.

We divided resistors into three different categories, 100 Cl to 1 Mf2, 1 Mf2 to 1 G€l, and 1 GO, to 10 Q or 10 teraohms (TO), if you want to use that terminology. In the 100 Q to MD range,

there is more interest in the pulse response characteristics, more so than, say, specification of DC

parameters or AC characteristics. For example, what would be the effect of 10 MHz at 50% duty

cycle? In that case, people are generally looking for 0. 1% accuracy as opposed to 1%, or at 100

MHz we are looking for possibly 1%.

The A to D conversion people are interested in very high stability resistors, preferably film

types and there apparently is some feeling that film resistors sometimes leave something to be

desired in terms of stability for very high precision ladders.

Also, there is a feeling that there is a need for calibration services on binary values. Also,

possibly a need for the calibration of network ratios for voltage dividers to accuracies of at least 1 ppm. Some of the NBS work we saw yesterday in the cryogenic area would indicate a capability for

very precise calibration of voltage dividers.

41
In the 1 MO to 1 GQ area, there was a requirement for 500 ppm accuracy and at least one user was looking for 10 Q, or 10 Q at 0. 35% accuracy. Apparently, this type of calibration can be obtained elsewhere at the present time.

We discussed noise in some detail, but more from the point of view of the availability of a precision noise standard, or noise generator versus different ranges of frequency from 10 Hz

through the audio spectrum on up to 500 MHz.

We discussed (a) a paper standard and (b) physical standards, possibly the construction of a precision white noise or gaussian noise generator, perhaps using a computer to generate truly ran- dom impulses for the computer measurement of such a source with calculations against a gaussian

formula. Also discussed were methods of measurement of noise standards for purposes of calibration. Applications included hearing aids, plus calibration of components such as resistors and

diodes, where we would like to have a standard noise source for very low-level noise.

Also discussed in somewhat more general terms was the medical implant area in terms of the reliability testing of these devices against a 5-year life requirement. In other words, how good

is a particular lot of devices, should they be sampled, put on an accelerated test, or should we

depend on a manufacturer's claims? For example, in the pacemaker area, there is a question con- cerning the amount of current that is required to consider the device in an operational mode. Fig- ures ranged from 1 mA down to 25 /iA.

There is a class of devices which we could call stimulators, including pacemakers, dorsal

column stimulators, phrenic nerve stimulators, and so on, which should perhaps fall under an electrical or system characterization standard or specification. In the area of hospital equipment,

there may be a need for calibration of such devices as clinical analyzers, or other monitoring

devices. There was more discussion on hearing aids, with possible interest in characterization in terms of frequency response and distortion levels, at a noise figure of 5% to 10%.

Miscellaneous topics include possible standards for temperature coefficient calibration for

resistors or capacitors. There is also apparently a requirement for inductive measurements of 10

to 50 ppm accuracy at 200 Hz to 2 kHz in the 100 /iH to 10 H range. There was a discussion of the

need for a general study by NBS of pulse techniques, pulse transmission, and response of components to pulses. Measurements of low resistance in the 10 Q area seems to be of some importance, but at low accuracies.

A brief discussion was held on the problems of the interfacing of transducers to systems, in

terms of the interaction between the Electricity Division and the other transducer calibration facili- ties at NBS, but nothing specific in the way of suggestions.

There was some discussion on the need for high-speed resistor calibration, as applied to laser trimming. We may trim several resistors a second and therefore we need to be able to have a system which will measure the resistor accurately under these high-speed conditions.

Q: Was there any discussion of not just the use of white noise, but the characterization of noise from the devices, the characterization of its distribution if it is non-gaussian?

A: Following the meeting there was discussion concerning the possibility of some device

noise having a non-gaussian distribution, in which case erroneous calibrations could be made, or other types of problems could arise. Actually, I believe noise characterization could be a fruitful area of investigation in terms of the development of precision ways of determining whether or not the distribution is really gaussian, and what is the amplitude versus frequency, and distribution.

42
Q: Was there any discussion on the need for improved accuracy in power standards? We

have seen the digital wattmeter, for example, and other techniques, generally in the region of 0.01%,

or thereabouts.

A: There was a discussion in the area of measurement of power factor. I think, at least, one gentleman was talking about 0. 01% power factor measurements and he was upset at not being

able to get a good phase angle meter. He felt that he would not have any confidence in its calibration. So, this may be an area of interest.

Q: I would suggest that the Bureau consider a factor of 10 improvement in what exists today

for power meter calibrating purposes to 20 ppm, say.

Presently, there are transducers and power meters on the market that have specs in the

order of 0.01%. In order to ascertain whether or not these are real, we would like the Bureau to say, "Yes, that is a valid number and we know because we are 10 times better than that. " As of now, you can basically say, "We think it is because we are about as good. " It would seem that would

be an area where some additional development might take place. On the power factor and phase

measurements, I would be particularly concerned with the increasing effect of harmonic distortion

and how that might be dealt with.

Q: (NBS Participant): I would like to make some comments about some of the calibrations

that you mentioned. We do have the capability for somewhat improved accuracy over those figures

that were mentioned in the high resistance area. In particular, we can do up to about 10 Q with an uncertainty no greater than 35 parts per million. We do not do quite that well for 10 ,10 , 10 , and 10 Q, but I think that we could meet that 0.35% uncertainty figure that was quoted as a

need. The reason that we do not normally give these high accuracies on our regular calibrations is that the resistors that we receive usually do not warrant it. They have such high voltage, temperature, and humidity coefficients that it is pointless to put that kind of accuracy on the calibration.

One other point. We do, and traditionally have, a temperature coefficient measurement, at least, on standard resistors and we do calibrate resistance standards at the milliohm and submilliohm levels.

Q: I should like to comment that a very logical extension of the Division's sampling watt- meter program that would make a lot of sense, might be going to much higher frequencies, taking

the samples not all in the same time frames, but using sampling technology, perhaps only having to improve the sample -and-hold capability. That looks like a very logical program to extend. There

are meters available in just about every frequency range you can imagine, for which one might like to have better capability for precision calibration.

A. 4. 2 Signal Conditioning and Data Conversion

(Presented by D. Ludwig)

Most participants, at one time or another, made it into Session 3 where we talked about a

long list of possible things where NBS might be useful to industry.

At the beginning we discussed the general question of characterization and definition, versus measurement. In the measurement discussions, we tried to home in on the measurement problems

and the places where equipment could be really improved. But there was agreement that it would be

impossible to measure parameters that have not been defined. If these definitions can be achieved,

then, either on a de facto basis, or otherwise, they could become considered as standard, and at least we would then have a common language for performance characteristics.

We spent a fair amount of time talking about settling time, probably because it seems to be

the one that there is the least industrial capability to measure of all of the parameters discussed.

43
There is quite a disparity between the capability in industry to make this measurement, between manufacturers and users. It was pointed out that the users are pretty much at the mercy of the manufacturers with respect to being able to make this kind of a measurement, or to certify this

kind of performance. Even the manufacturers themselves could use greater capability in terms of

making really precision settling time measurements.

Settling time, of course, is a very complex characteristic, requiring both a voltage and a time measurement to be made. Having these two degrees of freedom, makes it more difficult to talk

about.

There was, however, very definitely, a general consensus that all of us would appreciate the

efforts that the Bureau might put into learning about helping us to define settling time, and learning

how to measure it to higher and higher levels of precision.

There are two parts to the problem of measuring settling time. First of all, in the general

case, in order to measure the result of a step input to a system one must have a sufficiently clean

forcing function. So, the matter of deriving the forcing function that is suitably clean to make the

measurements is the first part of the problem. The characteristics of this source would be a flat top, which would have precision levels, probably on a DC-coupled basis, and would typically need to be able to achieve settling time itself to 0. 01% with 20 ns a good target to shoot for. I think that it was generally felt that if 20 ns could not be met, that there would still be a lot of use for 50 ns, or even 100 ns.

Precision at the top of the pulse would need to be 0. 01% as a minimum.

The rise time of the pulses would have to be well behaved and well controlled, and perhaps

adjustable. The step waveform could not have any overshoot, or if it had any it would have to be

fairly minor and well understood, or else the resulting measurement would begin to look more like the impulse response than the step response of the product being looked at.

The precision levels would have to be adjustable because there is a wide range of needs in terms of input signal steps that we would be interested in having these measurements made for, and

both polarities of step would have to be available. So, effectively, needed is a step where both the

beginning and ending levels are more or less infinitely adjustable with very good precision.

The measurement problem involves, once having forced this product under test with a step,

measuring its output behavior. In particular, we are looking for what we loosely call settling time

in terms of how long does it take the output to achieve an error from final value that is within some

defined percentage. 0. 01% and 20 ns are good goals, but there is a wide need for 0. 1% measurements; and even 0. 005% measurements are certainly a desirable type number.

The measuring capability would have to have the ability to do this test without the crutch of

the perfect input pulse for subtraction purposes, for cases in which the input is merely a digital

word change, and the output is an analog step which has to be measured independently without the use

of the perfect analog step.

The application for this kind of measurement, of course, is to just about everything you can

think of. To list a few, certainly, amplifiers of all sizes and shapes to DAC's to ADC's to sampleand-holds, multiplexers, resistors, basically to the whole wide range of analog processing devices

and interface units.

The concept of measurement to within a given percent of final value deserves comment, because there is a serious question as to what constitutes final value, or when do you measure final value? In general, a final value at the end of 1 second might be different from what was measured

after only a millisecond. There have to be some definitions, and judgments regarding methodology.

44
It was felt desirable, if possible, to consider a transportable standard for settling time.

The second topic, leaving settling time, would be the question of characterization of data converters. There was some divergence of opinion regarding whether this was a useful activity for

NBS, and whether or not it would be useful for NBS to set up testing definitions that might be construed as being standards, whether they be formal or de facto standards. There is some concern

that the existence of these standards would result in holding back progress, perhaps, at a point in time when we are still in an early stage of A to D and D to A converters, with the possibility that it might become difficult to change these standards, and we might end up stuck with them. Better approaches might come along, and there might be resistance to change.

Another consideration was the reluctance of manufacturers to be hamstrung by procedures

that, even though making sense at NBS's laboratory, might well be the wrong thing to do on a production line. If the customers got used to the idea of such a standard and began to use it in requiring

production tests of this type, this might not be economically justifiable.

The ultimate consensus was that this was an idea where we ought to proceed with care, that

there certainly is so much confusion in the specifications that there is a lot of work needed, that NBS,

in order to make measurements in the area, will have to make some determinations of what these

specs mean and that is fine as long as we can find some way to have everybody understand it. Results may not be sensible tests for production environments, but a test on an A to D converter that

is correct and sensible for a laboratory evaluation of, let us say, a prototype or a characterization

of a product, is not necessary and not even a sensible thing to do on a production basis. On a production line there is a lot of prior knowledge about how the converter has been manufactured, which

guarantees certain kinds of performance will exist assuming other kinds of parameters have been

measured. This kind of test program, or this kind of definition of parameters and measurements

that could be defined here would be very sensible and useful, but ought not to be tied to the production basis.

One of the questions that arose was, if you are going to measure A to D converters, how

many transitions should you measure? In a 10-bit converter there are 1024 transitions by one man's

definition, but this comprises 1024 steps, and you can change from any of these to any other, that is a 1024 factorial thing and obviously some very large amounts of data could be collected. That is only on a 10-bit converter. It gets staggering to think about it at 16 bits.

There was some discussion about using a dither technique, varying the input to an A to D4 converter on an AC basis, to make precision differential linearity measurements.

The important consideration for both DAC's and ADC's is relative accuracy, as compared to absolute accuracy. There are a large number of situations and applications for converters in which

the relative accuracy, or the linearity, of the converter is important, and the absolute numbers are much less meaningful. So, there needs to be some reasonable basis for separating the absolute

from relative performance.

It was also agreed that, in the area of data converters, there are a great number of characteristics that are important that are not now specified at all, and we discussed some of these. Such

things as the dynamic characteristics of the input impedance on an ADC or a multiplying DAC, where

the specs say 5 kfi, but that 5 kfi is doing some very strange things versus time. As another example, glitches are a characteristic of D to A converters that is only beginning to really surface.

Their measurement is still not well established or well defined.

Noise or uncertainty in the transition points on an analog-to-digital converter is another.

Some analog-to -digital converters have very clean transitions, and others have very noisy ones. You can't tell that by reading the specs.

45
These are examples of a few out of many areas that could well be surfaced and talked about

and looked at and measured.

There may be an interesting parallel tq a similar question that arose in 1954, in connection

with calibration and characterization of ratio transformers. The manufacturers had greater capability than the Bureau to make the measurements, and there were great debates and discussions between buyers and sellers as to whether the units really met their specs. It was a very similar situation, and we all know that the Bureau took hold, learned how to do these things, finally moving well

ahead of industry with their capability as time went along. This was ultimately to everyone's benefit, resulting in a much sounder business base for the people building a product and the people using

it. A similar consensus exists that the sample -and -hold is another device that needs to be

defined, and for which measurements need to be improved. Acquisition time, which is similar to a

settling time measurement, needs to be worked on, with requirements on the order of 0. 01% and 20

ns. That would give us some margin today. For aperture uncertainty, we are talking about 100 ps

or better, and that is quite an involved, difficult measurement. Today, it is largely calculated in industry rather than measured. We might get some big surprises if we could make the measurements correctly.

Special generators to make measurements on sample -and -holds were discussed, like precision ramp generators, including ramp generators that make precision corners. It could be a very

useful device if you have a suitably precise one, for learning what sample -and -holds do when trying

to sample the corner.

Also of importance is the memory effect of the sample -and -hold device. This gets back to dielectric absorption, but the measuring technology needs to be applied also at the subsystem modular level as well as at the component level, so that we can see to what extent the answer given by the

sample -and -hold is dependent upon its previous history. It is in the tens of nanoseconds and hundreds of nanoseconds region that interest exists.

For multiplexers, too, more definitions are needed. Certain characteristics are not well

defined or well measured. Included is dynamic input impedance, particularly for make-beforebreak multiplexers, which have a terrible input impedance characteristic inasmuch as the sources

get shorted together during transitions. This is largely an unknown, unspecified, unmeasured phenomenon, which usually comes as a large surprise.

Additional areas include settling time for multiplexers, the whole gamut of analog dynamic

measurements, and cross coupling between off and on channels. There is a lot of work that could be

done to do a better job of characterizing multiplexers. Multiplexers are, erroneously, often treated

like perfect switches.

Data acquisition systems could be treated as a whole. Some very interesting things could be

done using frequency domain analysis.

A consensus also exists that the area of non-linear functions is one where the definitions of measurements and definitions of parameters are pretty diverse from manufacturer to manufacturer.

It is not clear when the spec says 0. 1%, or 1%, over what range of inputs that applies. This is an area where some good, sound organization would help to try to get all these definitions straightened

out.

In the area of RMS, there also is a need for improved measurements for being able to do the

experimental characterization, particularly in the presence of complex waveforms with significant

peak factors.

46
There is a significant need to study, measure, and characterize some of the non-gaussian

type noise generated by the circuits, things like popcorn noise, transition noise in analog-to -digital

converters and chopper noise in chopper stabilized amplifiers.

On the subject of long-term stability, we talked at some length about the very great desire on

all of our parts for NBS to help in an area that we are all concerned about. This is the question

about the A to D converter that is 14 bits today. Will it be 14 bits next year? First, of course, come the components. If we knew or could predict more about what they will do in the next year or

two, or could extrapolate from short-term measurements, we could determine what we might expect

for performance of the functional circuit.

So, the suggestion is that it would be very useful to have some very carefully taken data on long-term stability of resistors, inductors, capacitors, V , h , resistance ratios, V_Q for

FET's, temperature coefficients and so forth versus time, and correlating results against predictions based on burn-in temperature cycling.

At this point in time, manufacturers try to take account of stability with some conservatism

in design, plus careful selection of processes and components; but nobody has much real assurance

about what is going to happen next year.

There is a definite need for laboratory level D to A converters that could be useful for both

testing DAC's by comparison, as well as ADC's with 10 ppm type accuracy. Such "standard" DAC's

ought to have a floating output, to at least + 1 volt, in order to avoid grounding problems.

There was a brief discussion about the mechanisms of communication between NBS and industry. This came up with respect to a particular program that had been conducted on bonding and the

reliability of bonds. There was some discussion as to how the industry at large would get to know

about a program of this type that was carried on at NBS. The work on bonding might have saved a

lot of us some money had we known about it.

Voltage-to-frequency converters have gone from an instrument -type of product to more of a

functional circuit. Not really much has been done on characterizing linearities, dynamic range,

etc.

Ideally, it would be desirable to be able to temperature test all of the parameters discussed.

It may be that in any new instrumentation pursued, NBS should include the ability to control the environment, as part of the initial design. If you build a settling time tester, it is too late to wait until

after you are done to decide that you want to vary the environment on the unit under test. Indeed,

definitions for temperature coefficient itself could definitely stand some standardization.

Also discussed was wafer probe testing, and the desire to be able to do all these kinds of tests at the wafer probe as well as on the bench. By and large, what we do test at wafer probe is a small number of tests compared to what we could test, but the technical problems and the measuring

methodology, instrumenting systems that we might use with it are very complex. It is a complex

problem, and it is something where some good work will pay off.

Final discussion touched on the dollar impact of all of this. There are apparently many

things that we feel that NBS could do, but what is it worth? It is hard to get your hands around this

to do a return on investment computation for the money that might be spent. Talking in general

terms, if the dollar return is considered in terms of money now wasted in the United States in vari- ous ways by people who either build systems, spending a lot of money to get a job done because of their fear to use these undefined dissimilar types of devices that they can now buy, or people, who,

because of their lack of understanding for the specifications, buy the wrong products, or buy products that do not work in their systems, the combined totals must be worth literally tens if not

47
hundreds of millions of dollars. In one particular case there was a known $200, 000 loss, associated

with a misunderstanding of what the specifications meant.

A. 4. 3 Sources and References

(Presented by D. Lawrence)

The Sub-Group on Sources and References covered five basic areas. They are: Impulse

testing, pulse and waveform standards, standards for battery life, AC/DC transfer standards, and

finally DC standards.

On impulse testing, we were not really able to clearly define the problem. It centers on the

requirement for high accuracy pulses to determine if fairly subtle changes have actually occurred.

The purpose is to use pulses to permit analyzing circuits and.devices without actually taking them

out of service.

In the area of pulse and waveform standards, one concept is to provide a linear ramp generator standard for working in the field of aperture time measurement. Problems abound, including

trigger jitter. (One participant suggested there was a better technique with which to accomplish this

measurement. ) We make ramps all the time with oscilloscopes, but they tend to fall into the 1 to 3%

category, while the requirements here probably fall closer to the 100 ppm area in terms of the timevarying amplitude.

Another area is the standard reference pulse. This deals in the area of oscilloscope testing,

something in the neighborhood of a 200 ps rise time pulse with aberrations of less than 1%, with duty

cycle from 10% to 50% and a repetition rate from 50 Hz to about 50 MHz with actual rep rate uncritical.

Another topic is the need for a generator calibrated in crest factor rather than pulse width,

for AC device evaluation.

It is felt there is need to investigate techniques to determine the real life of batteries on a

predictive basis. Applications areas include standard cell environmental control for transportation,

and for implantable devices. One suggestion was that some type of device be implanted within the

battery to check its chemical condition.

In AC/DC transfer standards, the first item that came up was the desire to locate a device to replace the thermocouple. Needed are an increase in response time, and wider dynamic range.

Also desired are an increase in accuracy at higher frequencies, plus a means to reduce DC reversal

error, or a means to predict it so that it does not require constant checking. Further needs include

reduction in the uncertainty of AC measurements as they are currently set up, hopefully to get audio

down to the area of less than 10 ppm and a proportionate decrease all the way up through high frequency AC from 1 MHz up to 100 MHz.

Another point discussed was the possibility of considering improving noise sources as an

alternative method to AC calibration over various frequency bands. The object would be to use

noise as a method of getting a one-point calibration on an AC measuring device or AC transfer stan- dard.

The problem with standard cells is that while they meet all of the electrical requirements, we need a standard with more environmental tolerance. Desired range is from 10 to 50 C, with a

spec in the area of 1 to 5 ppm. We would really like to see 10 volts as opposed to 1 volt, but accept

that the 1 volt is probably going to have to be there until we can replace standard cells. The 10-volt

reference is what everybody really wants, to get their signals out of the noise. We would also like to see a wider load range capability, up to at least 10 raA.

48
Transportability and short warmup time are needs as well. The current one-hour that is needed with most standard references is probably acceptable. What is not acceptable is the 1-day to

2 -week settling time of standard cells.

One suggestion was for automatic correction through microprocessor techniques, of the drift

and temperature coefficients. We are really looking for a laboratory quality standard. Many felt that a 10 ppm standard could well do the job for a large majority of firms. In terms of the NBS contribution, one of the things, of course, is more evaluation of industry designs along these lines.

One attendee brought up band-gap devices, as a possible substitute for Zeners. In a way, we are

looking for a means by which they could be improved. It is possibly an area where the Institute for

Materials Research at NBS might be more oriented. The big thing that stands in the way of most

Zeners today appears to be noise. One possibility might be different ways of doping to reduce the

noise.

In the short term, it might be useful to consider a measurement assurance program for

Zener transfer standards at the 10 ppm level, something that could be put into use today and would

meet the needs, again, of many users. A last point was that a good payoff would result from investigation of short term Zener

parameters that would predict long term performance. The current selection process for high stability Zeners is a long, tedious process, and any way of eliminating a lot of poorer diodes that currently go into that testing cycle in the beginning, would have a very definite economic payback.

Q: I wonder how intense the feeling was about a 10-volt calibration service? You mentioned

that it would be more useful than having to tie back to 1 volt. Suppose 10 volts were offered by the

Bureau at an equivalent accuracy to 1 volt?

A : If it were offered at equivalent accuracy, I would say that it would be more acceptable to most than 1 volt. I think that the problem of 1 ppm being a microvolt right in the noise band is a serious difficulty. Measurement technique may be the hardest thing to come by, especially for the

small firm. Getting signal levels up is definitely one way to make it easier.

Q: You spoke of AC to DC transfer. Are you talking about voltage or current, or both, and

are you talking about something that is faster responding than thermocouples?

A: We are dealing primarily with voltage standards. The idea was to generate the ability to make 10 measurements per second to an accuracy of 0. 1% to 100 kHz. The main utility is in the

world of automatic testing. You are trying to crank a product out at the highest rate. You are scanning over the bandwidth in various frequency steps and the time that it takes to settle for very high

accuracy is long, when you are dealing with true RMS. It is either improve the thermocouple, find

another device, or something of that nature.

A. 4. 4 Systems

(Presented by F. Seeley)

The first problem talked about was in the general area of electromagnetic interference and

noise. The kind of noise discussed in the main was noise that comes in the power line as an uninvited guest or self-generated noise, something that we do to ourselves from the computer or the

peripheral, or whatever other computational device or controlling device you might have - keyboards, CRT's, other devices, or the instruments and power transformers within the devices themselves. If it was not in the system, you might not experience that kind of noise.

The best we could come up with is that we thought it was someone's job to attempt to define

and characterize what we loosely call power line pollution. What are the types of things we see ? What is the recommended practice and what are the recommended techniques that you would have to

49
employ to either reduce the problem, or to get around the problem - such things as isolation,

grounding, shielding, guarding, filtering, power regulators, etc. ?

The other strategy in solving the general problem of noise is to say, well, even if you fol- lowed all of the best practices in the world, you still are going to have unwanted signals. Perhaps

there should be some attempt to define the limits to which one could make precise measurements or

rapid measurements or system type measurements. What are the conditions under which you could

make these measurements in this noisy environment? What is the degradation of the measurement

process in a noisy environment? Basically, these are paper studies.

The other large class of problems falls into the general area of how do you validate system

performance, really the performance of the system to accomplish its function? That is a unique

characteristic of a system, since it is usually designed to do a very specific thing or specific class

of things, as opposed to general purpose instrumentation, be it testing a blood sample or testing

missile rounds or whatever.

There are peculiar things that exist in the system world. There are the interactions between

measurement and stimuli within systems that are not necessarily present in looser aggregates of measurement and stimuli components. There are history-dependent phenomena. There are also

time-dependent phenomena, and once in a while in systems you see just data outliers, glitches,

things that you don't really understand, and you hope will go away if you repeat the test, and very

often they do. Those are some peculiar problems to get into when you are dealing with systems.

The first question is how do you validate the performance of the system in its static sense?

In other words, if there is a voltage to be measured or a current to be measured, or a resistance to be provided, and you can do that without regard to time, how can you best do it? But you would like to do it in the system environment, on the floor in the temperature -environment of the system, with

all of the noise present. Some sort of transportable standards are needed to do this, to bring up to the system. What is of interest to people generally who were at our session are the typical parameters, for example, AC voltages up to 1000 V and possibly out as high as 10 MHz. There is also a range of accuracy, with about 1% as the least accurate type of standard. We would like to have at the other end of the accuracy spectrum about 30 ppm, sort of pushing the state-of-the-art.

In DC voltage, up to maybe 1000 V in a transportable standard are needed for static testing,

somewhere in the uncertainty ranges of 0. 1% to on the order of 10 ppm.

Most systems people talk about measuring resistances and providing stimuli on the order of milliohms to about 100 M, and these range generally from 1% to as good as 10 ppm. Current and

power are also of interest.

Most people felt that pulse characterization or pulse characteristics are necessary in some

sort of transportable standard. Either as measured or stimulated system parameters, people seem

to be interested in rise time, fall time, decay time, flatness of the amplitude, flatness across the

duration of the pulse.

Very often one will measure a quantity within a circuit - say at the 3 00-volt level, and then

within a few milliseconds attempt to measure a 5 V or lower level in another circuit. There may

well be stored energy in that circuit, and you can either cause a tremendous error in performing the next test, or you might have so much stored energy that you could cause the device you are trying to test to actually fail. Results can be either absolute failure via the system pulse inducing a failure

mode in the device under test, or a pseudo -failure, where you think it is bad, and it really is good.

Those are the sorts of things in which you get involved, where things are no longer timeindependent, and become very definitely time -dependent.

50
What people in our session felt were needed then were things that you might call dynamic

standards. What might be needed is some technique for measuring the time -dependence of programmable DC sources and AC sources. Basically this is the problem of settling versus steady state. How long does it take to get a steady state value out of these sources, or when you try to catch them

on the fly, what sort of uncertainties does one run into?

The next thing covered was how does one characterize the sources used within systems or the

measuring devices used within system interfaces?

We also wanted to suggest that what else was needed was the development of techniques for

measuring pulse characteristics in digital logic testers. Here is a very knotty technical problem

where you have many digital words coming in at a 4 megaHerz rate, and you would like to basically

characterize the analog, the amplitude, the rise time, the fall time, the pulse duration of each one

of these pulses in this dynamic environment on these many lines. As you see, we have rise, fall,

amplitude, skew and other characteristics of that nature.

Q: Did you cover anything on the interface interconnection between devices under test and

the system itself, the connection problems, and so on, internal system cabling?

A: We did talk about that. We were trying to characterize the system's performance either

in the static sense or the dynamic sense, and it is implied that it is at the system interface.

Q: The problem, if you will, that exists quite frequently is that a device plugged into a system is not the same as a device connected in its actual application mode. For instance, quite frequently in a system in the digital world, you have got to go a little bit farther between a board and a

test device, because you are doing different things to evaluate those lines. The loading is different.

So the results of the pulses that are generated are different.

Q: There was also talk about an advanced portable standard, both AC and DC, with different

dynamic loads, since the loading would change.

A: That sounds like an example of going through and making measurements, and then actually inducing an erroneous result in the circuit under test, because of energy transferred from your

system. That could just as well be a loading problem.

Q: I wondered what, if any, conclusions were to come on the subject of software and whether

anybody here has any comments on software?

A: That was mentioned, but probably implicit in the lack of discussion on it was our overriding feeling that software becomes really a part, and almost the sole province of the company developing the product. I am speaking personally though. That did not come out in our discussion.

Q: Well, I think both in terms of hardware interfacing and software, most people thought it would be the problem of the person doing it. There were not enough generalities that you could ever do anything actually to help the problem out. You might want to define some tasks, I think, in terms

of operating systems, but not down at the programming level.

Q: I would like to emphasize that one of the things that was discussed was the need for the

ability to calibrate in place and essentially without disturbing the system. When you realize that you

have a minimum of $100, 000, perhaps three or four hundred thousand dollars tied up in a complex

test system, you simply do not want to take it down and remove an instrument to calibrate it.

Secondly, it may be, as has been mentioned, because of the stray parameters that exist in

your connection system, that the performance you are getting at the unit-under-test terminals is not

the same as the instrument calibration.

51
Related to this is our self -test or self -calibration program. The process of calibrating in place is best done, most conveniently done, by a program on the system itself. Developing a technique which is truly self -calibrating, using one transportable standard or a number of transportable

standards, is something that perhaps the Bureau could assist in, at least in describing algorithms,

which would indeed provide you the calibration that is traceable to the Bureau of Standards.

A: In the area of algorithms and processes and technique control, the Bureau has always historically published how it gets from basic units of measurement, the few SI units out to all the other

parameters. It publishes the work and how you scale up and down. But when you get to the auto- matic area, if you do that in the system, immediately there is a new technology involved. I am not sure whether that is really a systems problem or a components problem, but it is a problem and it needs to be looked at. Some people have looked at it and done some things. Others have not. To

act as a clearing house would be a very useful function.

Q: It almost sounds as if in a way you have laid the groundwork for the problems in systems

which might be worth the Bureau's while to look into.

A: I would agree. When you start seeing collections of the same sorts of problems in differ- ent systems, then you can quantify it further than we have attempted to do here. So it is a framework.

Q: Did anyone bring up the possibility of the Bureau sending around some kind of generalized

audit package that might be used in validating certain classes of systems?

A: Yes. If we had transportable standards, it would help. We covered them, and their

parameters. But the process of tying them to the system is the problem. These are the parameters, but what is the interface to the system? It depends on the systems.

One successful example is one company's automatic network analyzer, which was probably

the only automatic network analyzer for making measurements from 100 MHz to 12.4 GHz. There

is a community of users who pass standards around, but the problem is well-defined. It is strictly one machine with one set of software. In the more general case you have a lot of different types of

machines and a lot of different applications.

A. 5 Summary of Post-Workshop Written Feedback Reports

At the immediate conclusion of the Workshop, the twenty-four attendees were asked to fill out forms briefly summarizing, in priority order, the two or three most critical electrical measure- ment problems and/or problem areas which they believed the Electricity Division should pursue.

Nineteen responses were received. Since there were almost seventy separate items listed, only a few received more than several "votes". This was more-or-less to be expected in view of the

diversity of the group that was invited.

Since categories were not suggested, the way in which items were expressed differed somewhat from writer to writer. The following summary ignores priorities unless they seemed particularly significant.

1. Investigate and/or develop transportable standards suitable for validating the dynamic

performance of automated systems. (10 entries, 8 in first priority.

2. Same as (1), omitting dynamic requirement. (3 entries, all in first priority.

3. Define parameters for system building blocks and A/D interfaces (such as, but not limited to, ADC's and DAC's), to provide basis for developing test and/or calibration techniques, including dynamic performance. (8 entries, 3 first, 3 second priorities.

52
4. Set up definitions for dynamic or time domain measurement parameters, such as settling

time, acquisition and aperture times, fully specifying the quantities. (6 entries.)

5. Carry out long-term stability studies for components. (6 entries.)

6. Develop measuring techniques for noise measurements - standards and characterization.

(5 entries.)

7. Develop measuring techniques for pulse response testing, including a "pure" pulse generator, step or ramp generators, even exponential pulses. (4 entries.

)

8. Develop measuring techniques for settling time. (3 entries.)

9. Investigate and/or develop solid state DC voltage standards at 1 to 10 volts, with 2-5 ppm

accuracy. (3 entries.

)

10. Develop measuring techniques for pulsed/ settling time measurements for resistors. (2

entries.

)

11. Investigate the possibility of improving the stability of film resistors. (2 entries.

)

12. High-resistance calibration - improved standards and methodologies are needed. (2 entries.

)

Remaining entries appeared but once. In no special order, they were:

13. Develop computer-driven, dynamic source to exercise automatic measuring system for

DC volts, current, resistance.

14. Develop measuring techniques for pulsed- settling time measurements on capacitors.

15. Set up MAP for transportable DC voltage standards, at 2 to 5 ppm accuracy level.

16. Reduce AC measurement uncertainties.

17. A broader range of AC standards is required.

18. An alternative to the standard cell, with standard cell long term stability but none of its other problems, is required.

19. NBS should carry out work on dielectric absorption.

20. Power line characterization is of great importance.

21. Work on lifetimes for power sources for medical implants would be very useful.

22. A better power measuring capability is required.

23. A phase measuring capability is also needed.

24. NBS should extend MAP's in order to improve and clarify traceability.

25. NBS should provide leadership in publishing suggested procedures for testing automated

systems.

53
26. Higher frequency standards of C, R, are required.

27. Perform research on the long-term stability of bipolar transistors - V versus time,

in particular.

28. Carry out consumer product testing, e.g., hearing aids for frequency response, distortion, noise.

29. Medical equipment calibrations for hospitals would be valuable.

30. Temperature coefficient standards down to 0. 1 ppm/ C are needed.

31. Develop binary voltage divider calibrations to 1 ppm accuracy for ratio.

A. 6 Summary of Post-Workshop Written Commentaries

At the immediate conclusion of the Workshop, the attendees were also asked to fill out forms

briefly summarizing their reactions to the Workshop itself, and to indicate their suggestions for

future activities. What follows is a summary of the eighteen inputs received.

1. Ten participants indicated that they felt the Workshop met its intended goals without

reservation, six indicated that they felt it at least partially met them, while one indicated that he felt it had not met its goals.

Two comments indicated that the scope of the Workshop might have been a little too broad.

2. Thirteen participants, the total number responding to a question regarding the worth of the Workshop for the attendees themselves, indicated that it was indeed worthwhile.

Three of the responses were qualified as conditional upon the NBS implementation of Workshop recommendations.

Comments included the fact that the Workshop helped delineate industry problems, and

helped industry more fully understand the work of the Bureau.

3. Answers to a question relating to ways in which the Workshop could have been improved

covered a range of comment. There was no single most popular critique, and the inputs ran the gamut that might be expected from so diverse a group. Many should prove helpful in organizing future efforts of this nature. They include:

Include more device users .

NBS should take a more positive stand in measurement assurance - "traceability".

It would have been helpful to hold the tour and NBS activity descriptions before the Workshop.

More could perhaps have been gained via prior preparations by the invitees. (2 comments to this effect.

Perhaps area-limiting guides are needed to narrow the discussions.

Have wider representation.

Provide better understanding of what NBS can do.

54
Define NBS role better vis-a-vis industry. (2 comments.

)

Narrow down subject matter, perhaps for in-depth sessions.

4. Twelve attendees indicated that what they liked most about the Workshop was the professional interaction, the stimulation, the openness of idea exchange and the general informality of the technical sessions.

Three participants most appreciated information regarding Bureau capabilities, goals

and plans.

One respondent was most favorably impressed by the Workshop organization, and its informative nature.

5. Adverse comments were relatively few:

Too many sub-groups.

Tour too short and not sufficiently relevant.

Lack of clear understanding of Workshop goals.

Some emphasis on products rather than ideas.

Insufficient time.

Not all companies sent their most knowledgeable people.

Need for more advance preparation of attendees, particularly sub-group chairmen.

6. Twelve attendees favored a followup workshop in a few years or less, with several indicating a desire for intervals as short as a year or even less, possibly for working-group

meetings.

One specific suggestion was for interdisciplinary workshops, including two or more

cooperating NBS Divisions.

Another comment suggested followup reports to the participants on Division progress in

the areas identified.

7. Suggestions for subject matter and format for new workshops included six requests for

more-or-less identical workshops. Two requested that new workshops include a review

of NBS activities since the last one.

Additional suggestions were forthcoming for restructuring and reformatting new workshops, as follows:

Have greater systems orientation, including software.

Do not restrict to predetermined sub-groups.

Include greater description of NBS existing and planned programs.

Suggest formal presentations by invitees themselves.

55
Have more clearly defined objectives.

Include other industry areas.

Plan for different working groups, including semiconductors.

56
Appendix B : Mi scellaneous

B. 1 Final Workshop Agenda

23 September 1974 (Monday)

9:00 to 9:05 AM Welcome, Dr. Arthur O. McCoubrey, Director, Institute for Basic

Standards.

9:05 to 9:40 AM Introduction to NBS and the Electricity Division, Dr. Barry N. Taylor,

Chief, Electricity Division.

9:40 to 10:00 AM Workshop scope, goals, and modus operandi, Peter Richman, Consulting EE.

10:00 to 10:30 AM Coffee break and informal discussions.

10:30 to 12:00 PM Open discussion based on material submitted by attendees prior to Workshop (led by P. Richman). Initial working sub-groups finalized.

12:00 to 1:00 PM Lunch

1:00 to 3:30 PM Begin detailed discissions of specific topics in sub-groups.

3:30 to 4:00 PM Coffee and informal discussions. Attendees to have opportunity to indicate their Tuesday sub-group preference and to suggest possible new sub-groups.

4:00 to 5:30 PM Tour of the Electricity Division.

7:00 to ? Dinner and informal discussions.

24 September 1974 (Tuesday)

9:00 to 9:30 AM Brief review of Monday's sessions, consideration of possible new subgroups, and finalization of Tuesday's sub-group membership.

9:30 to 12:00 PM Sub-groups continue to meet; attempt to reach consensus and to delin-

(Coffee available) eate critical needs and problems in detail.

12:00 to 1:30 PM Lunch, informal discussions, and individual preparation for feedback

session.

1:30 to 3:30 PM Feedback session. Sub-group chairmen report back to assembled participants, with open discussion (led by P. Richman).

3:30 PM Formal adjournment. The remainder of the day will be available for

additional discussions, tours, etc. , at the option of the attendees.

(We cordially invite any attendee who so desires to stay an extra day

and visit the Division on Wednesday, 25 September.

)

57
B. 2 Tour

A tour of some of the Electricity Division's projects was arranged as part of the Workshop.

The following brief descriptions of each of the activities visited were distributed to all of the atten- dees prior to the tour.

Josephson Volt Maintenance and Cryogenic Comparators

Since 1972, the U. S. legal volt has been defined and maintained at NBS to an accuracy of a few

parts in 10 via the AC Josephson effect, an effect which involves the quantum -mechanical tunneling

of electron pairs in superconductors. In the voltage standard application, a Josephson tunneling

device is used as a precise frequency-to-voltage transducer with a proportionality constant equal to 2e/h. Presently under development is an all-cryogenic voltage standard system featuring a reliable

thin-film Josephson device, a cryogenic current comparator, a cryogenic resistive divider, a SQUID

null detector and superconducting switches.

Solid State References and Automated Voltage Measurements

Present studies of commercial reference units are being carried out with the objectives of: (a)

Determining the feasibility of using them to disseminate the volt at the 3 to 10 ppm level; (b) Obtaining quantitative performance data on the devices; and (c) Providing data for further improving the

performance of solid state references. Computer automation of low level voltage measurements will

improve NBS capability for maintaining and disseminating the volt. The present effort is being

directed toward developing an automated measuring system utilizing the HP9830 calculator, the

MIDAS controller and the NBS developed low-level switch.

High Voltage Pulses and Kerr Effect

The electro-optic Kerr effect is being used for the measurement of high voltage pulses.

Development is currently concentrated in three areas: (a) Accurate calibration of Kerr systems;

(b) Investigation of the properties of liquids used in Kerr systems; and (c) Computer automation of high voltage pulse measurements. The objective of this work is to develop and disseminate methods

of measuring high voltage pulses - pulses with rise-times of 1 ms to 1 ns and peak values from one thousand volts to several million volts - with an uncertainty of less than 1% of their peak value.

Direct Reading AC (RMS) Voltage Standard

A standard for measuring RMS voltage in the range 5 V to 1000 V from 50 Hz to 20 kHz is presently under development with extension of the ranges to higher and lower values to follow. An

automated AC-DC transfer circuit compares the AC voltage with a DC reference and hence to the

legal volt. The system will make possible multi-point calibration of precision AC sources to an

expected accuracy of 20 ppm and stability measurements to 5 ppm.

Sampling Wattmeter

A digital and computerized system was recently completed for the measurement of electrical

power with an uncertainty no greater than 0. 02% up to 2 kHz. Instantaneous samples of voltage and

current are collected in pairs at pre-determined intervals and the corresponding numerical values

stored in a computer in real time. Average power is calculated by numerical integration of current-voltage products. Adaptation of the system to the solution of problems associated with nonsinusoidal waves is planned, e.g., those encountered in distribution and control systems where

SCR's are extensively used.

58
Automated Resistance Measurements

Computer automation of the process for calibrating standard resistors is proceeding for two

separate ranges of resistance values. From 0. 1 to 10 ohms, a digital nanovoltmeter in conjunction

with a current -comparator potentiometer will be used to automatically compare voltage drops across

resistors in a series chain. Between 100 and 10 ohms, resistors will be intercompared using an

unbalanced, 4-terminal bridge technique. A digital voltmeter will measure both the unbalanced voltage and the bridge current. In both cases, switching will be done with numerically controlled cross- bar switches using four of the six circuits in the crossbar as an active guard to reduce leakage.

B.3 Working Sub-Groups

The following table gives the (approximate) sub-group participation during the course of the

Workshop. (But some Electricity Division staff other than those shown attended the sub-group sessions as observers on a more or less random basis. ) Session 1 was that held on Monday, 24 September, 1:00 to 3:30 PM; and sessions 2A and 2B were those held on Tuesday, 25 September, from

approximately 9:30 to 10:30 AM and 10:45 to 12:00 PM, respectively. The four sub-groups were numbered as follows, and these numbers are used in the table.

I. Components and Parameter Measurements

II. Signal Conditioning and Data Conversion

III. Sources and References

IV. Systems

Attendee'

Session Attendee's Session

Name Affiliation

1 2A 2B Name AffiUation

1 2A 2B

L. Baltz Ford IV II III D. Schneider Fluke IV IV IV

E. Boeckmann Angstrohm Chmn.Sub-Gr. I F. Seeley U.S. Army Chmn.Sub-Gr. IV

W. ColUer Tektronix IE III II R. Shaw Westinghouse I II IV

T. Coughlin G-R I IV II L. Smith Analog Dev. II III III

J. Dougherty USAF I III I D. Strain ESI I II II G. Herron Keithley II IV IV R. Talambiras Analogic II IV IV

D. Lawrence H-P Chmn.Sub-Gr. Ill D. Terrett ADT Sec. Sys. IV II I D. Ludwig Teledyne P Chmn.Sub-Gr. II J. Tripp National Semi. II II III E. Mueller FDA III I I R. Verity L&N I III II W. Plice Honeywell in ii ii W. Walters Codi m I I R. Raybold NBS IV II IV M. Young Teradyne II IV IV

B. Renz AEPSC IV II II R. Zuck Geometric Data IV

Electricity Divi sion Participants

M. Oldham NBS Recorder, Sub-Gr. I W. Eicke NBS Participant, Sub-Gr. I T. Souders NBS Recorder, Sub-Gr. II H. Schoenwetter NBS Participant, Sub-Gr. II B. Field NBS Recorder, Sub-Gr. Ill R. Turgel NBS Participant, Sub-Gr. Ill R. Kleimann NBS Recorder, Sub-Gr. IV J. Morrow NBS Participant, Sub-Gr. IV

59
NBS-114A (REV. 7-73)

U.S. DEPT. OF COMM. BIBLIOGRAPHIC DATA

SHEET

1. PUBLICATION OR REPORT NO. NBS Tech. Note 865

2. Gov't Accession

No.

4. TITLE AND SUBTITLE

Critical Electrical Measurement Needs and Standards

for Modern Electronic Instrumentation

3. Recipient's Accession No.

5. Publication Date May 19 75

6. Performing Organization Code

7. AUTHOR(S)

Peter Richman, Editor

8. Performing Organ. Report No.

9. PERFORMING ORGANIZATION NAME AND ADDRESS

NATIONAL BUREAU OF STANDARDS

DEPARTMENT OF COMMERCE

WASHINGTON, D.C. 20234

10. Project/Task/Work Unit No.

11. Contract/Grant No.

12. Sponsoring Organization Name and Complete Address (Street, City, State, ZIP) 13. Type of Report & Period Covered

14. Sponsoring Agency Code

15. SUPPLEMENTARY NOTES

Library of Congress Catalog Card Number: 75-600022

16. ABSTRACT (A 200-word or less factual summary of most significant information. If document includes a significant

bibliography or literature survey, mention it here.) Recognizing the proliferation of sophisticated modern electronic instrumentation in the

field of electrical measurements, the Electricity Division of the National Bureau of

Standards recently initiated a new program in the general area of dynamic measurements and standards in support of such instrumentation. Recognizing further that the

vastness and complexity of the field -would require, at the earliest stages of the progr

identification of the most critical problem areas, the Electricity Division held a workshop on 23 and 24 September, 1974, at the Bureau's Gaithersburg site, to assist it in

ascertaining just what these areas in fact were. The basic idea of the Workshop was

to bring together a broadly representative group of some twenty-five leading manufact- urers and prime users, working in a free and open atmosphere, in order to have them

delineate the present and future critical support needs in the field of dynamic electrica

measurements for modern electronic instrumentation, with emphasis on physical

standards, standardized measurement methods, new calibration and measurement

assurance services, relevant data, and most important, new measurement methodologies. The overall objectives of the Workshop were generally met, and a number of

significant specific programs and projects consistent with the mission of the Electrici

Division were identified. Three categories broadly cover the needs as defined at the

Workshop:

1. The need to introduce time as a measurement parameter has resulted from the

(continued)

a:n

itjT

17. KEY WORDS (six to twelve entries; alphabetical order; capitalize only the first letter of the first key word unless a proper name; separated by semicolons)

Data conversion; dynamic measurements; electrical measurements;

electronic instrumentation' signal conditioning- systems' time domain.

18. AVAILABILITY [IT Unlimited

~2 For Official Distribution. Do Not Release to NTIS

I xl Order From Sup. of Doc, U.S. Government Printing Office Washington, D.C. 20402, SD Cat. No. CI 3- 46:865

_j Order From National Technical Information Service (NTIS)

Springfield, Virginia 22151

19. SECURITY CLASS

(THIS REPORT)

UNCLASSIFIED

20. SECURITY CLASS

(THIS PAGE)

UNCLASSIFIED

21. NO. OF PAGES

74

22. Price $1.35

USCOMM-DC 29042-P74
requirements of automatic test and control systems. Specific areas

considered critical are:

a. Pulsed component measurements.

b. Dynamic performance characterization for modern signal conditioning

and data conversion devices: Digital-to- analog and analog-to-digital

converters, sample- and- hold amplifiers , comparators, etc. Required

measurements include settling, aperture and acquisition times. Basic

new capabilities will be needed, including precision, non- sinusoidal

waveform generation and "standard" digital-to-analog converters.

c. Methodologies and techniques for characterizing precision AC and DC

sources and measurement devices with respect to settling time.

The emergence of measurements into the "real world" or the production

line via the automatic system has introduced a host of new parameters.

Areas requiring significant effort include:

a. Investigation of transportable standards for validation of static and

dynamic system performance: AC and DC voltage, impedance, pulses,

settling times.

b. Techniques for characterizing sources and measurement devices with

respect to switched or dynamic loading.

c. Prediction of long-term performance, reliability and- lifetime from

short-term evaluation for a host of passive components, semiconductors,

and signal sources.

d. Noise standards and methodologies.

e. Continuing effort to improve transportable DC standards, and dissemination

at higher voltage levels.

f. Characterization of the AC line voltage-waveforms, under- and overvoltages, transients, dynamic impedance, etc.

g. Inclusion of a capability for environmental control and variation for devices

and instruments under calibration, in all new work undertaken by the Bureau.

h. Extension of Measurement Assurance Programs (MAP's), especially into

the above new areas.

5. Recent work at the leading edge of measurement technology in electronics has

resulted in the need for new, or higher accuracy measurements. For example:

a. Increased dependence on capacitors as storage devices has made dielectric

hysteresis a parameter of importance. Significant measurement work

should be started in this area.

b. Phase difference measurements to extremely high accuracies are needed to

calibrate industrial instrumentation.

c. Higher accuracy power measurements, particularly for non-sinusoidal

waveforms, are needed to calibrate instrumentation used in the field.

d. Non- sinusoidal, high crest-factor precision signals are needed to calibrate

true RMS converters and meters.

e. High accuracy electronic instrumentation requires reduction in AC calibration uncertainties to 10 ppm, at least over the audio range.
Longer-term, "frontier" areas in each of the above groups were also

identified. They are not specified in the above listings since they involve work

at still more basic levels, on ground that is presently terra incognita. Nevertheless, they furnish insight into the overall technical/philosophic climate, and

are useful for that reason. Items in this category include:

1. Measurement problems in systems, including effects of history, measurement

interactions, self- calibration, etc.

2. Evaluation and standardization of system software.

3. Calibration of medical equipment.

4. Replacement of standard cells.

5. Replacement of thermal AC/DC converters.

6. Non-swept but broad- response measurements- - impulse, noise, other.

7. Basic work on the physics of dielectric absorption; component drift;

device noise; failure mechanisms; switches and relays.

8. Provision of improved measurement accuracy for virtually all dynamic

measurements, at the wafer probe level.

In addition to the identification of these specific and general needs, the Workshop provided strong evidence that both the manufacturing and user communities

in the electronic instrumentation field look to the Electricity Division to provide

the technological leadership in these newer areas that it has previously furnished

in the more traditional electrical measurement areas. Indeed, there was a good

deal of support for the idea of keeping the electronics community informed of

Division work-in-progress, with emphasis on projects targeted by the Workshop.
NBS TECHNICAL PUBLICATIONS

PERIODICALS JOURNAL OF RESEARCH reports National Bureau

of Standards research and development in physics,

mathematics, and chemistry. It is published in two sections, available separately:

• Physics and Chemistry (Section A)

Papers of interest primarily to scientists working in these fields. This section covers a broad range of physical and chemical research, with major emphasis on

standards of physical measurement, fundamental constants, and properties of matter. Issued six times a

year. Annual subscription: Domestic, $17.00; Foreign,

$21.25.

• Mathematical Sciences (Section B)

Studies and compilations designed mainly for the mathematician and theoretical physicist. Topics in mathematical statistics, theory of experiment design, numerical analysis, theoretical physics and chemistry, logical design and programming of computers and computer

systems. Short numerical tables. Issued quarterly. Annual subscription: Domestic, $9.00; Foreign, $11.25. DIMENSIONS/NBS (formerly Technical News Bulletin) —This monthly magazine is published to inform

scientists, engineers, businessmen, industry, teachers,

students, and consumers of the latest advances in science and technology, with primary emphasis on the work at NBS. The magazine highlights and reviews such

issues as energy research, fire protection, building tech- nology, metric conversion, pollution abatement, health and safety, and consumer product performance. In addition, it reports the results of Bureau programs in measurement standards and techniques, properties of matter and materials, engineering standards and services, instrumentation, and automatic data processing. Annual subscription: Domestic, $9.45; Foreign, $11.85.

NONPERIODICALS

Monographs—Major contributions to the technical liter- ature on various subjects related to the Bureau's scientific and technical activities. Handbooks—Recommended codes of engineering and

industrial practice (including safety codes) developed TM ""operation with interested industries, professional

izations, and regulatory bodies.

industrial practic

in cooperation wi„n

organizations, and regulatory bodies

Special Publications—Include proceedings of confer- ences sponsored by NBS, NBS annual reports, and other

special publications appropriate to this grouping such

as wall charts, pocket cards, and bibliographies.

Applied Mathematics Series—Mathematical tables, manuals, and studies of special interest to physicists, engineers, chemists, biologists, mathematicians, computer programmers, and others engaged in scientific and technical work.

National Standard Reference Data Series—Provides

quantitative data on the physical and chemical properties of materials, compiled from the world's literature and critically evaluated. Developed under a world-wide

program coordinated by NBS. Program under authority

of National Standard Data Act (Public Law 90-396). NOTE: At present the principal publication outlet for these data is the Journal of Physical and Chemical

Reference Data (JPCRD) published quarterly for NBS

by the American Chemical Society (ACS) and the American Institute of Physics (AIP). Subscriptions, reprints, and supplements available from ACS, 1155 Sixteenth

St. N. W., Wash. D. C. 20056.

Building Science Series—Disseminates technical infor- mation developed at the Bureau on building materials, components, systems, and whole structures. The series presents research results, test methods, and performance criteria related to the structural and environmental functions and the durability and safety characteristics of building elements and systems.

Technical Notes—Studies or reports which are complete

in themselves but restrictive in their treatment of a

subject. Analogous to monographs but not so comprehensive in scope or definitive in treatment of the subject area. Often serve as a vehicle for final reports of work performed at NBS under the sponsorship of other government agencies.

Voluntary Product Standards—Developed under pro- cedures published by the Department of Commerce in Part 10, Title 15, of the Code of Federal Regulations. The purpose of the standards is to establish nationally

recognized requirements for products, and to provide

all concerned interests with a basis for common under- standing of the characteristics of the products. NBS

administers this program as a supplement to the activities of the private sector standardizing organizations.

Federal Information Processing Standards Publications (FIPS PUBS):—Publications in this series collectively constitute the Federal Information Processing Standards Register. Register serves as the official source of information in the Federal Government regarding stand- ards issued by NBS pursuant to the Federal Property and Administrative Services Act of 1949 as amended,

Public Law 89-306 (79 Stat. 1127), and as implemented

by Executive Order 11717 (38 FR 12315, dated May 11, 1973) and Part 6 of Title 15 CFR (Code of Federal

Regulations). Consumer Information Series—Practical information, based on NBS research and experience, covering areas

of interest to the consumer. Easily understandable language and illustrations provide useful background

knowledge for shopping in today's technological

marketplace.

NBS Interagency Reports (NBSIR)—A special series of interim or final reports on work performed by NBS for

outside sponsors (both government and non-government). In general, initial distribution is handled by the sponsor; public distribution is by the National Technical Information Service (Springfield, Va. 22161) in paper

copy or microfiche form.

Order NBS publications (except NBSIR's and Biblio- graphic Subscription Services) from: Superintendent of Documents, Government Printing Office, Washington,

D.C. 20402. BIBLIOGRAPHIC SUBSCRIPTION SERVICES

The following current-awareness and literature-survey

bibliographies are issued periodically by the Bureau:

Cryogenic Data Center Current Awareness Service A literature survey issued biweekly. Annual subscription: Domestic, $20.00; foreign, $25.00.

Liquefied Natural Gas. A literature survey issued quarterly. Annual subscription: $20.00. Superconducting Devices and Materials. A literature

survey issued quarterly. Annual subscription: $20.00. Send subscription orders and remittances for the pre- ceding bibliographic services to National Technical Information Service, Springfield, Va. 22161.

Electromagnetic Metrology Current Awareness Service Issued monthly. Annual subscription: $100.00 (Special rates for multi-subscriptions). Send subscription order and remittance to Electromagnetics Division, National Bureau of Standards, Boulder, Colo. 80302.
U.S. DEPARTMENT OF COMMERCE

National Bureau of Standards

Washington. D.C. 20234

OFFICIAL BUSINESS

Penalty for Private Use. $300

POSTAGE AND FEES PAID

U.S. DEPARTMENT OF COMMERCE COM-215

FOURTH CLASS MAIL

,^%