97-1

Planning 

Report

Economic Assessment of the 

NIST Thermocouple Calibration

Program

U.S Department of Commerce

Technology Administration

Prepared by:

Michael L. Marx

Albert N. Link

John T. Scott

Approved by:

Leon S. Reed

David P. Leech

July 1997

Prepared for the

Program Office

National Institute

of Standards

and Technology
ECONOMIC ASSESSMENT OF THE NIST

THERMOCOUPLE CALIBRATION PROGRAM

July 11, 1997

Prepared for:

The National Institute of Standards and Technology

Gaithersburg, MD

Prepared under:

Contract No. GS-22F-0101B

Prepared by:

Michael L. Marx

Albert N. Link

John T. Scott

Approved by:

Leon S. Reed

David P. Leech

TASC, Inc.

1101 Wilson Boulevard

Suite 1500

Arlington, VA 22209
ii

DRAFT

TABLE OF CONTENTS

Page

EXECUTIVE SUMMARY............................................................................................. 1

1. INTRODUCTION.................................................................................................. 4

1.1 NIST ............................................................................................................... 4

1.2 NIST Thermocouple Calibration Program........................................................ 4

1.3 Case Study Objectives...................................................................................... 6

1.4 Case Study Outline .......................................................................................... 6

2. TECHNICAL OVERVIEW................................................................................... 8

2.1 Circuit ............................................................................................................. 8

2.2 Types............................................................................................................... 9

2.3 Calibration ....................................................................................................... 10

3. INDUSTRIAL OVERVIEW.................................................................................. 13

3.1 Applications..................................................................................................... 13

3.2 Cost and Quality Drivers for Accuracy............................................................. 15

3.3 Industrial Structure .......................................................................................... 17

3.3.1 Suppliers............................................................................................. 17

3.3.2 Users .................................................................................................. 17

3.3.3 Infrastructure ...................................................................................... 19

3.4 Traceability of Standards.................................................................................. 21

4. EVALUATION FRAMEWORK AND APPROACH........................................... 24

4.1 Scope .............................................................................................................. 24

4.1.1 Reference Functions and Tables .......................................................... 24

4.1.2 Primary Calibration Services ............................................................... 24

4.1.3 Technical Support to Industry ............................................................. 25

4.2 Benefit Measures............................................................................................. 26

4.3 Comparison Scenarios...................................................................................... 27

4.4 Benefit Data..................................................................................................... 28

4.2.1 Survey of Thermocouple Users ........................................................... 28

4.2.2 Survey of Thermocouple Suppliers...................................................... 29

5. SURVEY RESULTS .............................................................................................. 30

5.1 Thermocouple Users........................................................................................ 30

5.1.1 Calibrations Performed Internally ........................................................ 30

5.1.2 ISO 9000 ............................................................................................ 31

5.1.3 ITS-90 ................................................................................................ 32

5.1.4 Transactions costs............................................................................... 33

5.1.5 Other .................................................................................................. 33
iii

DRAFT

5.2 Thermocouple Industry.................................................................................... 34

5.2.1 Wire Suppliers .................................................................................... 34

5.2.2 Thermocouple Suppliers...................................................................... 35

5.2.3 Summary............................................................................................. 36

6. ECONOMIC ANALYSIS ...................................................................................... 38

6.1 Cost and Net Benefit Data ............................................................................... 38

6.1.1 NIST Annual Expenditures and Income............................................... 38

6.1.2 Data for economic impact analyses...................................................... 39

6.2 Measures of Economic Impact ......................................................................... 40

6.2.1 Internal Rate of Return........................................................................ 40

6.2.2 Adjusted Internal Rate of Return......................................................... 41

6.2.3 Benefit-cost Ratio ............................................................................... 42

6.3 Measure of Economic Importance.................................................................... 42

6.4 Economic Return on Upgrade of Reference Tables to ITS-90 .......................... 43

6.4.1 International Conformity to the Temperature Scale.............................. 44

6.4.2 Other Temperature Measuring Devices ............................................... 44

6.4.3 Long-Term Benefit Horizon................................................................ 45

6.4.4 R&D Leadership Role......................................................................... 45

7. CONCLUSIONS .................................................................................................... 47

7.1 Economic Importance of Thermocouples......................................................... 47

7.2 Economic Impact of the NIST TCP ................................................................. 47

7.3 Potential Improvement in the Traceability Process............................................ 49

7.4 Lessons-Learned from Implementing ITS-90.................................................... 50
iv

DRAFT

LIST OF FIGURES

Figure Page

2-1 Basic circuit for thermocouple thermometry ......................................................... 9

3-1 Sample Applications of Thermocouples by Common Requirements of

Uncertainty........................................................................................................... 14

3-2 Traceability Paths for Thermocouple Calibration Standards .................................. 22

LIST OF TABLES

Table Page

6-1 Expenditures and Income for the NIST TCP FY 1990-1996 (current $)................ 38

6-2 NIST TCP Costs and Net Industry Benefits (current $000)................................... 39
ES-1

EXECUTIVE SUMMARY

The National Institute of Standards and Technology (NIST) assists industry in the

development and promulgation of standards for the calibration of thermocouples. The primary

purpose of this case study is to evaluate the economic impacts on domestic industry for

investments made in this infrastructure technology through the NIST Thermocouple Calibration

Program (TCP).

Thermocouples are electronic sensors for measuring temperature for use in a wide variety

of applications (e.g., medical procedures, scientific research, automated manufacturing processes,

and consumer products) where temperature is an important parameter in a measurement or

control system. The annual sales of thermocouple products sold by the domestic thermocouple

industry (suppliers of wire and thermocouple assemblies) into the U.S. market is approximately

$280 million. The incorporation of these devices into higher levels of product structures across a

broad base of domestic industries affects a much larger portion of the manufacturing sector,

estimated to be on the order of $81.0 billion. The demand for thermocouples is expanding because

of their importance for computerized instruments, which are used increasingly for the monitor and

control functions in manufacturing applications to improve corporate productivity. Therefore,

thermocouples have a significant effect on the current and future competitiveness of U.S.

industry.

The NIST TCP generates technical knowledge to improve the calibration accuracy and

efficiency of thermocouple measurements made in industry. This case study focused on three

technical outputs: (1) reference functions and tables for letter-type thermocouples, (2) primary

calibration services for traceability of standards used in thermocouple measurements, and (3)

technical support to industry. In particular, standards traceability provides an efficient means for

ensuring accurate measurements across divergent industrial environments, and NIST is the

pinnacle of the traceability structure for national standards linked to international agreements of

temperature scale. Establishing a process for documenting the accuracy of temperature

measurements to national temperature standards is an essential element in meeting ISO 9000

certification, which is necessary for marketing products internationally. The standards traceability

process through the NIST TCP provides thermocouple users with an efficient means to comply

with the requirements of ISO 9000 since the burden-of-proof for documented compliance can be

obtained from the wire or thermocouple supplier.
ES-2

From the technical outputs of the NIST TCP, thermocouple users and suppliers derive

three main types of benefits: (1) efficiencies in developing and disseminating the technical

knowledge needed for performing thermocouple calibration tests, (2) transactions cost savings

between users and producers engaged in thermocouple commerce, and (3) improved

competitiveness for users of thermocouples through enhanced product performance and process

efficiency. In this case study, the first two types of benefits were estimated while the third type of

benefit proved to be difficult to quantify. Yet, from a national competitiveness standpoint, the

economic magnitude of the latter benefit type is significantly greater than the first two benefit

types.

The internal rate of return (IRR) from investments in the NIST TCP has been calculated

to be at least 31.8 percent. A number of factors have been incorporated in this calculation to

portray this as a conservative, lower-bound estimate of NIST’s economic impact on domestic

thermocouple users and suppliers. This estimate is based on appraisals of economic benefits and

expenditure data related to the outputs of the NIST TCP. Net benefits have been estimated from a

survey of the domestic thermocouple industry using a hypothetical, counterfactual experiment.

The resulting 31.8 percent IRR implies that the NIST Thermocouple Calibration Program is

pursuing a socially-valuable activity affecting a broad range of domestic industries.

Many thermocouple users having requirements of high accuracy rely on calibration tests

performed internally rather than from statements of accuracy obtained from their suppliers. These

users claim that internal calibration testing is both necessary due to the inconsistent nature of

thermocouple calibrations and an efficient means of maintaining process control with applications

having stringent accuracy requirements. However, to some extent, the efficiency in the standards

traceability scheme headed by NIST is being degraded since users cannot rely totally on their

suppliers to provide the correct representations of calibrations for thermocouple products. The

root cause of this non-optimal situation appears to derive from issues about assurances of product

quality from suppliers of wire and thermocouples, and these issues are not linked directly with

measurement-related know-how from the NIST TCP. However, NIST could use its neutral role

to help industry overcome this problem so that the implementation of the traceability process can

approach greater efficiency.

The survey also revealed that the vast majority of thermocouple users — including users

with high accuracy requirements — have attained few economic benefits in the use of

thermocouples calibrated to the current, internationally agreed upon temperature scale compared

to thermocouples calibrated to the previous temperature scale. As a result, returns have been poor
ES-3

on investments made by NIST to upgrade the thermocouple reference tables to be consistent with

the newer temperature scale. Yet, NIST made a sound decision by investing in the updated

reference tables under the circumstances, and U.S. industry would likely have suffered negative

economic consequences if NIST had not made this investment. However, a positive consumer

surplus has not been achieved due to a mismatch in the timing of when this infrastructure

technology can effectively meet the practical needs of industry (thermocouple users). Thus, the

poor return on investments made by NIST for updating thermocouple reference tables is rooted in

the decision to change the temperature scale to the ITS-90. Perhaps such investments would have

been more effective if better intelligence were available on the timing of the economic payoff for

industry including dependency on foreign trade. Hence, the policy implications are that

investments in state-of-the-art technology infrastructure should be more rigorously coordinated

with the realities of industrial demand and, hence, corporate strategy.
1-1

1. INTRODUCTION

1.1 NIST

The United States Department of Commerce (US DoC) through the National Institute of

Standards and Technology (NIST) supports U.S. industry’s efforts to develop new technologies,

accelerate commercialization of new products and processes based on these technologies, and

achieve global market penetration to advance the Nation’s economic growth and standard of

living. The internal laboratory research portion of NIST's activities provides this support in the

form of various types of standardization, testing, and laboratory accreditation services that are

needed by industry at several critical stages in the process of meeting the challenge of the

technological state-of-the-art. NIST is the only Federal laboratory with the primary mission of

supporting the commercial application of technology.

NIST's laboratories develop and transfer infrastructure technologies to industry.

Infrastructure technologies (infratechnologies) are “tools” that make the R&D, production, and

market penetration stages of the product cycle more efficient. These tools enable industry to

efficiently develop applications that are based on both generic and proprietary technologies.1

NIST develops a wide range of infrastructure technologies for U.S. firms' products and

processes — as well as their attendant products and services — which comprise complex systems.

Examples of infratechnology tools include production methods, processing models, advanced

measurement methods, technical databases, standards, product performance tests, and quality

assurance techniques. NIST develops its tools commensurate with the commercialization needs of

U.S. industry, which have become ever more demanding to be competitive internationally.

1.2 NIST THERMOCOUPLE CALIBRATION PROGRAM

The NIST Thermocouple Calibration Program (TCP) is a part of the NIST Chemical

Science and Technology Laboratory (CSTL). The overall mission of NIST/CSTL is to provide

the chemical measurement infrastructure for enhancing U.S. industry’s productivity and

 

1 The role of infrastructure technology within the construct of product development is discussed in: Gregory

Tassey, The Economics of R&D Policy, (Quorum Books, 1997; Chapter 8) (forthcoming); and Albert N. Link

and Gregory Tassey, Strategies for Technology-based Competition, (Lexington Books), pp. 16-21, 1987.
1-2

competitiveness, assuring equity in trade, and improving public health, safety and environmental

quality. Such measurement technology is required for industrial research and development,

product applications, improvements in the design and manufacturing of quality products, proof of

performance, and marketplace transactions that include the entry of U.S. products into

international markets.

NIST’s role as the lead U.S. agency for temperature measurement is to overcome

technical and business barriers that require NIST’s special characteristics: an impartial position,

expertise in a wide range of measurement areas, direct access to complementary national

standards, and the motivation to deliver the technical infrastructure to a wide range of supplier

and user industries. These characteristics permit NIST to perform functions that individual

companies cannot perform:

• Gain access to technical information for resolving industry-wide

measurement problems

• Develop measurement solutions and national reference standards that

benefit the industry as a whole across diverse product and process

lines

• Gain acceptance of those solutions by U.S. industry and its customers

so that U.S. products can be sold efficiently both domestically and

abroad

All temperature measurements must ultimately trace back to national standards to provide

consistency and accuracy across disparate organizations and industries. NIST has the legal

mandate in the United States for providing the national standards that form the fundamental basis

for all temperature measurements made in domestic industry.2 Realizing and maintaining national

temperature standards in terms of the scientific first principles and the constants of nature that

define the International Temperature Scale is difficult technically and requires a dedicated

laboratory capability. NIST/CSTL develops and maintains the scientific competencies and

laboratory facilities necessary to preserve and continuously refine the basic physical quantities that

constitute the national temperature standard. Further, NIST has the mandate to apply these basic

 

2 Congress passed The Organic Act of 1901 identifying the National Bureau of Standards (NBS) as the

responsible organization for providing measurement standards to industry. Subsequent legislation under the

Omnibus Trade and Competitiveness Act of 1988 transferred this responsibility from NBS to the renamed

NIST.
1-3

measurement standards to develop uniform and widespread measurement methods, techniques

and data.

1.3 CASE STUDY OBJECTIVES

Periodic economic studies are conducted at NIST to determine the qualitative and

quantitative impacts of its programs on U.S. industry. The results of these studies are used for

NIST strategic planning and for characterizing NIST’s role as a major source of technology

infrastructure for U.S. industry.

This case study estimates the first-order economic impacts of the infrastructure technology

development on U.S. industry from the calibration of thermocouples. The analysis examines the

impacts of the infrastructure technology in terms of the affected industrial activities, the

qualitative nature of the impacts on the behavior of relevant industries, and the quantitative

impacts on the competitive performance of these industries. The specific objectives established at

the onset of the study were to :

• Describe the use of thermocouples in U.S. industry

• Identify and describe the infrastructure technology supporting

thermocouple calibrations

• Obtain qualitative and quantitative data on the economic impacts

attributable to the contributions from NIST’s infrastructure

technology; these economic impacts include assessments of the

benefits and costs from the perspectives of users, producers, and

other organizations that influence and are influenced by temperature

measurements made with thermocouples

• Analyze the data and calculate the rate of return on NIST's

investments in technical knowledge for thermocouple calibration

using various economic measures

• Prepare a report of the findings, analysis and conclusions

1.4 CASE STUDY OUTLINE

The remainder of this report includes the following:
1-4

• Chapter 2 provides a technical overview of thermocouples including a

generic description of their function and their need for calibration

• Chapter 3 describes thermocouples from an industrial context in

terms of:

— Types of applications

— Quality and cost drivers for thermocouple accuracy

— The industrial structure for commercial trade in

thermocouples

— The scheme for standards traceability that allows industrial

users to efficiently attain accurate measurement calibrations

• Chapter 4 describes the scope, benefit measures and data collection

strategy used for the analysis in this study

• Chapter 5 identifies findings on the qualitative and quantitative

economic impacts of the infrastructural outputs from the NIST TCP

based on surveys of thermocouple users and suppliers, and lists

expenditures and income for the NIST TCP

• Chapter 6 provides quantitative estimates of the prospective

economic rate of return from investments made in the NIST TCP,

and analyzes certain economic issues that are more qualitative in

nature

• Chapter 7 discusses the primary conclusions for this case study
2-1

2. TECHNICAL OVERVIEW3

2.1 CIRCUIT

A thermocouple is an electronic sensor for measuring temperature. Thermocouples

operate according to the Seebeck Effect, wherein a closed circuit formed by two dissimilar wires

(thermoelements) produces an electrical voltage when a temperature difference exists between the

contact points (junctions). The electrical potential difference that is produced is called the

thermoelectric electromotive force (emf), also known as the voltage output of the thermocouple.

The Seebeck Effect occurs due to the difference in the energy distribution of thermally energized

electrons in the material compositions of each thermoelement. The fact that thermoelectric emfs

vary from metal to metal for the same temperature gradients allows the use of thermocouples for

the measurement of temperature.

A simplified measurement system using a thermocouple is shown in Figure 2-1.4 The

thermocouple consists of two thermoelement wires A and B that are joined at one end H (the

“hot” junction), but the wires otherwise are insulated from each other along their lengths. The

other ends CA and CB (the “cold” junction) are maintained at a constant reference temperature T0

which usually is defined as the melting point of ice (0 C). In practical measurements, the cold

junction does not have to be held at the reference temperature if the actual temperature is known.

The difference between the actual temperature and 0 oC is usually corrected electronically in the

instrumentation of the measuring system. This adjustment is referred to as the cold-junction (CJ)

compensation.

 

3 This chapter is based primarily on the following: G.W. Burns and M.G. Scroger, The Calibration of

Thermocouples and Thermocouple Materials, NIST Special Publication 250-35, April 1989; H. Dean Baker, et

al., Temperature Measurement in Engineering, (Omega Engineering: 1975); G. W. Burns, et al., TemperatureElectromotive Force Reference Functions and Tables for the Letter-Designated Thermocouple Types Based on

the ITS-90, NIST Monograph 175, April 1993; The Temperature Handbook, (Omega Engineering: 1995); and,

D.D. Pollack, The Theory and Properties of Thermocouple Elements, ASTM STP 492, 1971.

4 The scale of the conceptual measurement system represented in the figure varies widely depending on the

application. In reality, the overall distance from T1 to Tr may range from 2 feet to 2,000 feet.
2-2

Thermoelement

Wires Cold

Junctions

A

B

CB

Hot

Junction

Voltmeter

(V)

Extension

Wires

T T T

CA

96-0296.1

H

1 o r

Instrumentation

Figure 2-1 Basic Circuit for Thermocouple Thermometry

A thermoelectric emf (voltage) is generated between junctions CA and CB when the hot

junction is exposed to the temperature of interest T1 and the cold junctions are maintained at the

reference temperature T0. Identical wires extending from the cold junctions of the thermocouple

transmit the emf to an instrument — represented in the figure by a voltmeter at room temperature

Tr — that measures the thermocouple emf. Theoretically, this measurement depends only on the

temperature difference (T1 - T0).5 As T1 increases, the output emf of the thermocouple increases,

although not necessarily in a linear relationship. This correlation of temperature versus emf

permits the establishment of mathematical relationships that are unique to various wire

combinations. These mathematical relationships are expressed in reference functions and tables

which provide the basis for calibrations used in thermocouple thermometry.

2.2 TYPES

About 300 combinations of pure metals and alloys have been identified and studied as

thermocouples. Such a broad selection of different conductors is needed for applications requiring

certain temperature ranges as well as for protection against various forms of chemical

 

5

If, however, the wires A and B are not homogeneous, then the thermoelectric emf may not be correct. Herein

lies the principal limitation in the use of thermocouples for accurate temperature measurement. In practice,

obtaining perfectly homogeneous wires has been impossible. Moreover, the degree of starting homogeneity is

degraded to some extent after the wire materials are first heated.
2-3

contamination and mechanical damage. Yet, only a few types having the most desirable

characteristics are in general use.

The eight most common thermocouple types used in industry are identified by letters:

base-metal types E, J, K, N, and T; and noble-metal types B, R, and S. The letter designations

were originally introduced by the Instrument Society of America (ISA) to identify certain

common types without using proprietary trade names, and were adopted in 1964 as American

National Standards. The letter-types are often associated with certain material compositions of the

thermocouple wires. However, the letter-types actually identify standard reference tables that can

be applied to any thermocouple having an emf versus temperature relationship agreeing within the

tolerances specified in the table, irrespective of the composition of the thermocouple materials.

The letter-type thermocouples comprise about 99 percent of the total number of thermocouples

bought and sold in commerce.

Thermocouples made from the noble-metal materials (mainly, platinum and rhodium) are

significantly more expensive than those made from base-metal materials (e.g., copper, iron, and

constantan). For example, prices for 0.015 inch diameter bare wires made of various platinumrhodium alloys range from $25 to $101 per foot while the price range of similar base-metal wire is

$0.20 to $0.24 per foot.6 This significant difference in cost is the primary reason why over 90

percent of the total number of thermocouples sold are base-metal type. In the United States,

several hundred tons of base-metal thermocouple wire are produced annually.

2.3 CALIBRATION

Thermocouples must be calibrated for accurate temperature determination. In most cases,

calibration involves measuring the thermoelectric emf of the thermocouple under evaluation as a

function of temperature, which is determined by a reference thermocouple.

The calibration process consists generally of three steps. First, thermoelectric emf values

of the thermocouple are measured either at a series of approximately uniform intervals of

temperature or at certain fixed points. Second, appropriate mathematical methods are used to fit

the difference between the measured emf values and those of a reference temperature. Third, the

 

6 Based on a range of catalog prices for thermocouple wire as quoted in The Temperature Measurement

Handbook, Vol. 29, Omega Engineering, Inc.
2-4

emfs as a function of temperature are expressed in both a calibration table and their given

mathematical relationship.

The reference functions and tables used in calibrations of standard letter-type

thermocouples must relate to a specified temperature scale. International agreements have been in

place since 1927 on scales of temperature for scientific and industrial purposes. Updated about

once every 20 years, the scale now in use is the International Temperature Scale of 1990 (ITS90).7

The extent of thermocouple calibration for practical temperature measurement depends

mainly on the accuracy and stability required for the particular application.8 Wire suppliers

typically perform sample calibrations on short wire lengths from spools containing up to 1000 feet

of wire. Sample calibrations of base-metal wire provide tolerances ranging commonly from +/-

0.25 percent to +/- 0.75 percent of the temperature versus emf values in the standard reference

tables, which provides sufficient accuracy for a wide variety of technical work.9 However,

uncertainties in process control and the need for more accurate measurements demand additional

calibrations by certain suppliers and users of thermocouples. For example, the stringent accuracy

and stability demands for temperature measurements made in semiconductor manufacturing

processes often require calibration of every thermocouple.

Significant differences in stability and accuracy exist between the noble-metal and the

base-metal types of thermocouples. Noble-metal type thermocouples tend to have fairly stable

calibrations and tight calibration tolerances. On the other hand, the base-metal types are less stable

 

7 The International Temperature Scale of 1990 (ITS-90) was adopted during the 1989 meeting of the

International Committee of Weights and Measures in accordance with the request embodied in Resolution 7 of

the 18th General Conference of Weights and Measures of 1987. This scale supersedes the amended edition

(1975) of the International Practical Temperature Scale of 1968 (IPTS-68) and the 1976 Provisional 0.5 K to 30

K Temperature Scale.

Temperature values on the ITS-90 (expressed as t90)are defined in terms of thermodynamic equilibrium states

of pure substances (the defining fixed points), thermometers for interpolating between the fixed points, and

equations that relate the measured property to t90.

8 Stability refers to the ability of a thermocouple to achieve repeatable temperature versus emf characteristics

with successive temperature measurements. An unstable thermocouple can go out of calibration or “drift”,

which is a serious fault because of the resulting incorrect reading of temperature.

9 Certain suppliers of noble-metal wire claim to provide even tighter tolerances, ranging from +/- 0.1 to +/- 0.33

percent.
2-5

(more likely to go out of calibration with frequent use) and have larger tolerances.10 Therefore,

the more stringent the stability and accuracy requirements of the particular application, the more

likely that users pay the higher costs for noble-metal thermocouples.

 

10 Material composition of the wires is the driver for stability. Increases in temperature creates greater molecular

movement in alloyed metals than with pure materials. As a result, noble-metal wire is more stable due to the

greater purity in material composition in comparison to the metal alloys used in base-metal wire.
3-1

3. INDUSTRIAL OVERVIEW

3.1 APPLICATIONS

The thermocouple is the oldest and the most widely used electronic temperature sensing

device. Other devices, such as thermistors, resistance temperature detectors, and integrated circuit

sensors, can be substituted for thermocouples, but only over a limited temperature range. Therein

lies the primary advantage of thermocouples, which is their use in a wide temperature range (-270

C to 2100 C). Other key advantages are that they provide a fast response and are unaffected

by vibration. They also are self-powered, versatile, inexpensive, and simple in their construction.

Their calibration, however, is affected by material inhomogeneity (nonuniformity of physical

composition) and contamination, and their operation is susceptible to electrical interference.

Thermocouples are used in a wide variety of applications (e.g., medical procedures,

scientific research, automated manufacturing processes, and consumer products) where

temperature is an important parameter in a measurement or control system. Their use in

engineering applications has been increasing steadily because thermocouples, like other types of

electronic measurement sensors, are compatible with microprocessor instrumentation.11 Figure 3-

1 shows typical levels of uncertainty for a variety of products and manufacturing processes that

use thermocouples for temperature measurement.12 Applications having the more stringent

requirements of uncertainty have greater needs for calibration knowledge than those applications

having the less stringent requirements.

 

11 A temperature measure in the form of a digital output is useful in computer-based engineering applications. An

electronic temperature sensor generates an electrical output in direct response to a change in temperature, and

this electrical output is adaptable to a microprocessor. By contrast, a traditional dial thermometer produces a

non-electrical output, which requires a gauge reading by an operator to obtain the measurement.

The industrial demand for electronic temperature sensors is significant and expanding as the monitoring

function of various control systems becomes more computerized to improve productivity and quality as well as

to reduce operator error. Intechno Consulting AG reported that the world temperature sensing market had sales

of $2.5 billion in 1991 and was forecast to increase at a 6.4 percent average annual rate to reach $4.7 billion in

2001.

12 NIST uses the term “uncertainty” as the quantitative measure of inaccuracy.
3-2

• Drug testing

• Pharmaceutical chemical manufacturing 

• Moisture measurement in grain

• Rapid thermal processing in 

semiconductor manufacturing

• Glass softening point and forming

• Steam turbine operation for electrical 

utilities

• Glass annealing

• Metal heat-treating

• Plastic injection molding

• Residential stove operation

• Steel production furnace

• Aircraft turbine engine operation

• Residential thermostat

• Metal sintering

• Glass container formation

• Tire molding

< 1 °C

1-2 °C

2-5 °C

Thermocouple 

Applications

More Stringent

Less Stringent

Typical

Uncertainty

Requirements

Figure 3-1 Sample Applications of Thermocouples by Common Requirements of

Uncertainty

Certain industries have applications that are very sensitive to temperature change.

According to various industry representatives, the four industries having the most stringent

accuracy and stability requirements for temperature measurement are:

• Food, beverage and drug

• Semiconductor manufacturing

• Military/aerospace

• Power utilities

For example, small temperature measurement inaccuracies in burning fuel for generating electrical

power can translate into large inefficiencies and hence costs. A utility industry representative

stated that an inaccuracy of 1 C would result in an annual $100,000 loss in pre-tax profits for a

single fossil-fired power generation plant. Also, IBM reports that a 3 C miscalculation in a
3-3

sintering process can jeopardize a furnace load of substrates worth perhaps millions of dollars.13

Additionally, a supplier of gas turbine engines used in aircraft stated that if the on-board

temperature measurements of thermocouples used in the turbine are inaccurate by 1 °C, then the

aircraft would burn two percent more fuel. Therefore, such thermocouple users with high

accuracy requirements have greater economic sensitivity than the majority of users.

3.2 COST AND QUALITY DRIVERS FOR ACCURACY

According to thermocouple experts, the key factors in obtaining accurate measurements in

thermocouple thermometry are:

• Quality of the components in the temperature measurement system

• Minimizing contamination in thermocouple wires14

• Quality of calibration data that is traceable through standards at NIST

In addition to absolute accuracy, the precision of a temperature measurement in a fixed

environment (e.g., a furnace) that requires periodic replacement of thermocouples is also critical

in maintaining process control. Therefore, consistent performance in the interchangeability of

thermocouples is an important feature for achieving high production yields and reducing costs.

An example in glass forming helps in understanding the combination of several key

concepts: the use of temperature measurement to ascertain product quality, the need for stringent

levels of accuracy in temperature measurement, and consistent performance required in the

interchangeability of thermocouples. Viscosity of molten glass is the key technical parameter in

forming picture tubes used in televisions sets. Levels of viscosity are inferred through temperature

measurements made with thermocouples because viscosity cannot be measured directly.

Measuring the absolute accuracy of the glass softening point to within 1 oC is important in

obtaining the desired mechanical properties of the glass. Also, uniform levels of viscosity — via

repeatable temperature measurements with interchangeable thermocouples — are important in

ensuring consistent quality in manufacturing the glass picture tubes.

 

13 “Temperature Control is the Key in IBM’s Sintering Process,” Intech, May 1997.

14 For example, contamination in the wires of a thermocouple due to human perspiration and fingerprints is a

major source of measurement error in many applications requiring high temperature accuracy. For this reason,

many thermocouples are manufactured in clean room environments.
3-4

Generally, the number of calibrations and the quality of calibrations generally have been

steadily increasing industry-wide.15 The major reason for this trend is the increased number of

organizations seeking higher levels of quality in their products and processes. In particular, the

need for traceability to standards, which is one key requirement for achieving certification and

audit approval in the ISO 9000 quality scheme, has been a driving force for increases in the

quantity and quality of thermocouple calibrations. The industrial literature also cites the increased

need for tighter measurement tolerances and quality in applications involving health, safety,

hygiene, and process control.

Large differences in sell price between thermocouples made from base-metals and noblemetals often strongly influence users’ willingness to pay for calibration testing. Most base-metal

types of thermocouples and thermocouple assemblies retail in the range of $6 to $100 per unit.16

On the other hand, prices of noble-metal thermocouples range from $250 to $1,000 per unit.

Using estimates from commercial calibration service providers, the cost of performing a

calibration test for most types of thermocouples ranges from about $40 to $100. Therefore, the

percentage of total unit price attributable to calibration is much greater for base-metal

thermocouple types than for noble-metal types. This is the primary reason that some users of

base-metal thermocouples only perform sample calibrations from large lots. For example, one

representative of a thermocouple manufacturing company stated that few calibrations are

performed on thermocouples sold to the plastic injection molding industry because base-metal

thermocouples are used and temperature sensing requirements are not very stringent; these

industrial users rely on the sample calibration data provided by the wire supplier, which is

sufficient for most plastic injection molding applications. On the other hand, some industrial users

in the four critical industries noted above require calibration of every base-metal thermocouple.

Obviously, the percentage of sell price attributable to calibration testing will be higher for this

latter class of users.

 

15 Estimating the annual number of calibrations performed by thermocouple users and producers is impractical

due to variations in quality requirements across industries and firms.

16 Based on a range of catalog prices for thermocouples quoted in The Temperature Measurement Handbook, Vol.

29, Omega Engineering, Inc.
3-5

3.3 INDUSTRIAL STRUCTURE

3.3.1 Suppliers

The thermocouple industry consists of wire suppliers and thermocouple suppliers. The

downstream customers of the wire suppliers are grouped into two general categories,

thermocouple suppliers and thermocouple users that fabricate and assemble thermocouples

captively. As mentioned earlier, the wire suppliers typically perform sample calibrations on each

production lot of wire. Depending on the accuracy requirements of a given application, these

sample calibrations may or may not be sufficient for users of the thermocouple. Therefore, the

supplier or user/producer may perform additional calibrations as necessary. According to NIST,

the three major suppliers of base-metal type wire are Carpenter Technology, Harrison Alloys, and

Hoskins Manufacturing. The four main suppliers of noble-metal wire are Engelhard Industries,

Johnson Matthey, PGP Industries, and Sigmund Cohn Corporation.

Thermocouple suppliers purchase wire from the wire suppliers to fabricate and assemble

finished thermocouple products. Steps in fabricating such products include encasing a

thermocouple in protective sheathing and adding ancillary hardware such as a connector. Myriad

configurations of assembled thermocouples are sold, in turn, to users of thermocouples. The online product database of the Thomas Register of American Suppliers lists 305 companies selling

thermocouple products.17

3.3.2 Users

The available information on characteristics of the domestic thermocouple market is less

than complete and current. Also, disparities exist among the few sources of market data that are

available.18 The best available information is in several industry trade periodicals and newsletters

(Sensor Business Digest, Measurement and Control News, and Sensor Business News) that cover

activities in the thermocouple industry. Past articles refer to results from a 1992 study on the

temperature sensor industry performed by Venture Development Corporation (Natick, MA)

 

17 Thomas Register of American Suppliers, http://www.thomasregister.com, September 28, 1996. Many industry

representatives consider the Thomas Register to be the most complete and reliable listing of thermocouple

suppliers. However, most of the firms listed in the Thomas Register are small, specialized producers or

repackager s of other firms’ products.

18 Such disparities in data among different market research firms are common.
3-6

which includes information on the thermocouple segment of the industry.19 The following market

data and forecasts were reported from this study:

• The total domestic shipments of all electronic temperature

measurement devices (thermocouples, RTDs, thermistors, and IC

sensors) was $402 million in 1991

• US consumption of thermocouples was projected to increase at a

compound annual rate of 2.6 percent from $126 million in 1991 to

$144 million in 1996

• Consumption of temperature sensing devices is significant in the

following industries:

— Automotive

— Chemical

— Consumer Goods

— Food/Beverage/Pharmaceutical

— Glass

— HVAC (heating, ventilating, and air conditioning)

— Medical

— Metals

— Military/Aerospace

— Oil/Petrochemical

— Plastics/Rubber

— Utilities

• The medical, HVAC and automotive categories were forecast to have

the highest growth rates domestically for temperature sensors

A 1991 report by a German firm, Intechno Consulting AG, on the world temperature

sensors market stated that the world temperature sensing market had sales of $2.5 billion in 1991

and was forecast to increase at a 6.4 percent average annual rate to reach $4.7 billion in 2001.

The geographical distribution of this market is U.S. (33.4%); Japan (23.7%); and Europe

(42.7%). Regarding thermocouples, Intechno estimated that the 1991 world market was $1.032

 

19 For example, see “Dartmouth Researchers Invent Improved Temperature Sensor Array,” Sensor Business

Digest, March 1994.
3-7

billion, consisting of $263.5 million in non-encased thermocouples and $768.6 million in encased

thermocouples.20

Thermocouples and thermistors are widely used in the health care industry, particularly for

monitoring the core body temperatures of patients in many situations, such as anesthetized

surgery, outpatient surgery, trauma centers, intensive care, and in pain clinics. Market

Intelligence, Inc. projected in 1991 that the worldwide biomedical sensor market for disposable

thermocouples and thermistors would grow from $63.2 million in 1991 to $115.6 million in 1998

for a 9 percent compound annual rate during the forecast period.21 Actual unit shipments were

estimated at 11.1 million in 1991 and projected to increase to 22.5 million in 1998. The U.S.

market consumed 58.1 percent of the world revenues in 1991.

3.3.3 Infrastructure

NIST Thermometry Group NIST/CSTL’s Thermometry Group develops and applies

the process of standards traceability for temperature measurement. The Thermocouple Calibration

Program (TCP) is a part of the Thermometry Group’s overall research activities. This Group is

responsible for realizing, maintaining, improving and disseminating the national standards of

temperature. This responsibility is implemented through the following activities: 22

• Determining the accuracy of the national standards of temperature

with respect to fundamental thermodynamic relations

• Calibrating practical standards for the U.S. scientific and technical

communities in terms of the primary standards

• Developing methods and devices to assist user groups in the

assessment and enhancement of the accuracy of their temperature

measurements

• Preparing and promulgating evaluations and descriptions of

temperature measurement processes

 

20 Ibid.

21 “Tayco’s Temperature Sensors Solve Challenging Applications,” Sensor Business Digest, July 1993.

22 National Institute of Standards and Technology, “Thermometry Group Mission,” http://www.cstl.nist.gov,

October 31, 1996.
3-8

• Coordinating temperature standards and measurement methods

nationally and internationally

• Conducting research towards the development of new concepts for

standards

• Developing standard reference materials for use in precision

thermometry

The above listing shows that NIST does more than simply maintain national standards to ensure

that industry has a traceable temperature measurement system. NIST also develops and makes

available suitable, appropriate, and meaningful measurement methods that permit organizations to

correctly use internal instrumentation and reference standards to perform their needed

measurements at the required accuracy.

Standards Bodies Several national and international organizations sanction standards for

practical temperature measurement. These standards often form the basis of purchase

specifications used in commercial trade between users and suppliers of thermocouples. ASTM and

the ISA are the primary industrial organizations that sanction thermocouple standards used

domestically, and different technical specifications are covered in the standards documents of each

organization.23 The ISA Standard MC-96.1 has been recognized as an American National

Standard, while the related ASTM Standard E-230, which has been updated to include ITS-90

thermocouple tables and functions, is presently under consideration as an American National

Standard by ANSI.24 IEC 584-1 is the standard used internationally.25

The thermocouple standards from ASTM, ISA, and IEC subsume calibration reference

tables from NIST. The current versions of ASTM E-230 and IEC 584-1 have been updated to

include NIST’s most recent reference tables and functions, while the current ISA MC-96.1

standard — the latest update was in 1982 — contains an earlier version of NIST’s reference

tables.26 Therefore, in practice, the benefits of NIST’s reference tables are diffused to

 

23 ASTM is the American Society for Testing and Materials, and ISA is the Instrument Society of America.

24 ANSI is the American National Standards Institute.

25 The IEC is the International Electrotechnical Commission.

26 According to several industry representatives, this discrepancy between the ISA and the ASTM standards has

often caused confusion and disputes in domestic thermocouple commerce.
3-9

thermocouple users and producers through the ASTM, ISA, and IEC standards rather than

through NIST-published documents such as Monograph 175.

The ASTM, ISA, and the IEC standards also include other technical specifications, such

as color-coding of the thermoelement wires and the extension wires, that are needed in the course

of commercial trade between users and suppliers of thermocouple products. NIST contributes

little technical work or engineering data for developing these more mundane types of

specifications since they are not based on leading-edge measurement technology.

NIST also cooperates with standards laboratories of other countries to ensure full

compatibility on basic measurement standards used in international trade. While the standards

bodies governing various countries agree on both the International Temperature Scale and NIST’s

reference functions and tables for thermocouple calibration, disagreements often occur on

allowable tolerances relative to these reference tables. Tolerances of the United States are

specified in the ISA and ASTM standards. Developing international consensus on thermometer

tolerances is one part of the charters of the IEC and the Organization of Legal Metrology

(OIML), and NIST participates actively in both of these international organizations.

Commercial Calibration Services Companies that market commercial calibration

services comprise another facet of the thermocouple industrial infrastructure. The objectivity of a

neutral third party is often valued in negotiations or disputes between suppliers and purchasers of

thermocouples, and the requirement of traceability can avoid potential disagreements or

misinterpretations of data. The strength of competitive factors such as pricing, quality, and turnaround time generally determine whether thermocouple users and producers seeking third-party

calibration testing use these secondary-level calibration service providers rather than the primarylevel services of NIST. Industry representatives concur that NIST provides the highest level of

standards traceability for achieving the highest quality calibrations. Yet many are sensitive to the

price of calibration services and perceive the cost of NIST’s services as relatively high for their

specific needs. However, one of NIST’s major strategic thrusts is to have its primary standards

leverage the private sector provision of secondary standards.
3-10

3.4 TRACEABILITY OF STANDARDS

In the overall hierarchy of standards for thermocouple calibration, NIST is viewed as the

provider of primary standards from which subordinate reference standards are traced.27 The

general traceability paths for thermocouple users, suppliers, and calibration service providers are

depicted in Figure 3-2. For example, a user needs a thermocouple calibrated to a certain

specification for assurance of accurate results. In the traceability scheme, the user often relies on

the proficiency and cost-effectiveness of suppliers or a calibration laboratory in the private sector

to obtain this calibration. To certify the accuracy of the relationship between temperature and

thermoelectric emf for the thermocouple, the supplier must have direct access to appropriate

reference standards calibrated in terms of the primary temperature standards maintained at NIST.

The producer or commercial laboratory maintains these reference standards internally and

compares them regularly with the national standards to achieve traceability.

 

27 In the context of measurement science, the system of providing a thoroughly documented, “unbroken chain” of

references to a measurement authority is known as traceability. Many motivations exist for this system in

industrial commerce, ranging from the need for compliance with stated contractual requirements to the desire

for using the best possible practices in measurement. Traceability also provides an efficient means for ensuring

accurate measurements in industrial environments.
3-11

Figure 3-2 Traceability Paths for Thermocouple Calibration Standards

Organizations that perform internal calibrations of thermocouples employ several general

methods to demonstrate and certify traceability to NIST’s national temperature standards.

Sometimes a temperature measurement is rendered traceable via more than one of these methods.

In the first and most common method, the organization has its thermocouple materials calibrated

against the national (primary) standards maintained at NIST. These materials then serve as the

reference standards (artifacts) for internal calibration purposes of the organization. The second

established but less common method involves measurement and certification using test methods

and similar quality apparatus employed by NIST. The third and most recent method for

thermocouple calibration involves the organization’s acquisition of a standard reference material

(SRM) from NIST. The SRM is then used as the artifact for internal calibration purposes.

An approach known as the “expendable standard” is typically employed by the wire

suppliers to perform internal reference calibrations efficiently. Periodically (about every 1-5

years), a sample length of wire from a large wire spool is calibrated through the primary
3-12

calibration services at NIST. This NIST-calibrated wire sample becomes the reference standard

(the traceable link to NIST’s primary standards) for the organization. Short wire lengths of the

spool from which this reference standard originated are used for subsequent internal calibration

tests. These test lengths, which are traceable to the NIST primary standards via the reference

standard, are the expendable standards since they are used typically for only one calibration test

and then discarded.

Users of thermocouples employ one or a combination of strategies in the procurement and

calibration of thermocouples depending on their operating practices and accuracy requirements.

As mentioned in Section 3.3.1, users obtain thermocouples by either: (1) purchasing assembled

thermocouples from suppliers, or (2) maintaining captive production operations by procuring wire

from wire suppliers directly and making thermocouples for internal use. Users in the first group

usually rely completely on calibration data provided by the thermocouple suppliers to ensure

specified levels of quality. One representative of a thermocouple supplier stated that 95 percent of

his customers rely exclusively on the supplier’s calibration data to ensure thermocouple accuracy.

Also, these types of users sometimes send used thermocouples back to the OEM for recalibration,

although disagreement exists among thermocouple experts on the ability to effectively recalibrate

a used thermocouple.

Thermocouple users often perform their own calibrations in-house to attain certain levels

of accuracy beyond the calibration warranties provided by their suppliers. These internal

calibrations are performed either in the user’s metrology laboratory or in situ (in the internal

operating environment of the thermocouple).
4-1

4. EVALUATION FRAMEWORK AND APPROACH

4.1 SCOPE

This section describes the outputs of the NIST Thermocouple Calibration Program under

assessment in this case study.

4.1.1 Reference Functions and Tables

NIST has a long history of developing and publishing reference functions and tables for

letter-type thermocouples. NIST has updated this reference data with the periodic changes in the

International Temperature Scale. The most current reference functions and tables for the eight

standard letter-type thermocouples have been published in NIST Monograph 175.28 This

reference data is derived from actual thermoelements that conform to the requirements of the

ITS-90 standard.

As described in Section 3.3.3, NIST’s reference functions and tables have been subsumed

in the major industry standards for thermocouple calibration. Therefore, this infratechnology is

used every time a letter-type thermocouple is calibrated, providing significant and widespread

benefits to thermocouple users and suppliers.

4.1.2 Primary Calibration Services

NIST’s Thermometry Group provides primary calibration services for the suppliers and

users of thermocouples to achieve levels of measurement accuracy necessary to attain objectives

of quality, productivity, and competitiveness. These services constitute the highest order of

thermocouple calibration available in the U.S. for customers seeking traceability and conformity

to national and international temperature standards. NIST provides these services at a charge

sufficient to compensate for the direct costs of the calibration, and certain surcharges apply to

help pay for fixed costs and research.

All types of thermocouples, including both letter-designated and non-standard types, can

be calibrated by NIST from -196 °C to 2,100 °C. Customers provide samples (either bare wires or

 

28 G. W. Burns, et al., NIST Monograph 175, April 1993.
4-2

complete thermocouples) to NIST’s laboratory. NIST calibrates these samples on the ITS-90

using one or a combination of different test methods depending on the thermocouple type, the

temperature range, and the required accuracy. The calibrated thermocouple is then shipped back

to the customer along with a “NIST Report of Calibration” containing the test procedures and the

results of the calibration. The sample and the data from the NIST Report constitute the traceable

link to national temperature standards. For example, customers of NIST’s primary calibration

services can use their calibrated artifact and the accompanying calibration data from the NIST

Report of Calibration as the secondary standard for internal quality control purposes. This

secondary reference standard links subsequent calibrations made within the customer’s metrology

regime to the primary standards maintained by NIST, and, thereby, to the measurements of other

organizations. Such traceability to standards allows the highest level of fidelity for the

organization’s internal calibrations.

The technical knowledge that forms the foundation for NIST’s calibration services is

upgraded continuously to improve the traceability process.29 These improvements are generally in

the forms of research on test methods and procedures as well as upgraded equipment,

instrumentation, and facilities.

4.1.3 Technical Support to Industry

Experts at NIST are available regularly to assist in solving specific technical problems of

industrial organizations. Such problems usually pertain to performing thermocouple calibrations

or using thermocouples in a temperature measuring system. Direct help is available over the

telephone — NIST estimates that each of its staff members receives about 20 to 25 telephone

calls per week — and by visits to the Thermometry Group’s laboratory at Gaithersburg, MD.

NIST’s specialized expertise in calibration test methods and procedures is particularly

sought by industry. Organizations with internal metrology laboratories often seek technical knowhow from NIST in establishing and maintaining sound test methods for thermocouple calibrations.

These organizations benefit from the research invested by NIST to establish the primary

 

29 Detailed technical descriptions about NIST’s thermocouple calibration services are published in NIST Special

Publication 250 and its supplements. These documents augment the “NIST Report of Calibration” and

encompass the following topical areas: specifications for the service, design philosophy and theory, description

of the NIST measurement system, NIST operational procedures, assessment of measurement uncertainty, and

NIST internal quality control procedures.
4-3

calibration services discussed above. To achieve high levels of traceability internally, some

organizations perform secondary-level calibrations by replicating test techniques and similar

apparatus used at NIST.

Periodically, NIST gives tutorials on thermocouple calibration at conferences and

seminars. These tutorials provide education and promote good measurement practices at all levels

throughout industry. NIST also provides advice and assistance on problems in thermocouple

measurement and calibration as a part of a precision thermometry workshop held twice a year in

the NIST laboratories at Gaithersburg, MD. Additionally, technical papers regarding NIST

research in the measurement field are disseminated at conferences organized by various scientific

and engineering groups.

4.2 BENEFIT MEASURES

Based on background interviews with NIST and several thermocouple users and suppliers,

the working hypothesis for the case study was that the infrastructural outputs attributable to the

NIST TCP provide users and suppliers with three main types of benefits:

(1) Efficiency in developing the infrastructure technology necessary for

calibrating thermocouples. Industry has relied on NIST as the single

organization for the outputs described in Section 4.1, above. This

reliance has obviated the need for duplicative research by individual

companies and industrial consortia.

(2) Cost and time savings in resolving disputes between users and

suppliers involved in thermocouple commerce. These efficiencies are

based on industrial agreements for the technical bases of standards

established through NIST. These agreements are due mainly to

industry’s recognition of NIST’s (1) high quality outputs, and (2)

impartial competitive posture.

(3) Improved competitiveness for domestic thermocouple users in the

forms of better marketability (e.g., compatibility with international

product standards), production efficiencies (e.g., process yields) and

product features (e.g., performance and reliability). These

competitiveness benefits are based on the high degree of calibration

accuracy attainable via standards traceability from NIST.
4-4

4.3 COMPARISON SCENARIOS

The approach for evaluating the economic benefits associated with the NIST TCP has

been adopted from evaluations of other public investments in infratechnology.30 NIST’s actual

costs are compared to an estimate of net benefits using a hypothetical, counterfactual experiment.

The experiment assumes that the first-level economic benefits associated with the NIST TCP can

be approximated in terms of the additional costs that industry would incur in the absence of

NISTís services. In other words, these are the costs avoided by industry due to the existence of

the NIST TCP.

The counterfactual experiment is used because this case study lacks a comparable business

baseline prior to the development of NIST’s infratechnology outputs. NIST, through its legal

mission, has been the sole provider of these infratechnology outputs to U.S. industry for many

years. With respect to the reference tables, no substitute or near-substitute set of outputs exists.

Conflicting proprietary tables were in use from the 1920s through the 1940s, but that situation no

longer exists as industry has relied on NIST for reference tables for letter-designated

thermocouples. Therefore, a recent “pre-NIST” baseline for reference tables is not available to

compare the behaviors of industry in the “post-NIST” scenario.

For primary calibration services, a similar situation exists because NIST has operated since

at least the 1960s — based on the recollections of NIST TCP personnel — as the provider of

such services for domestic users seeking high quality calibrations. The commercial calibration

services mentioned in Section 3.3.3 are not a comparable substitute since these commercial

organizations rely on the laboratory capabilities of NIST for primary measurement standards.

Without an actual state-of-practice from an earlier period, a significant part of the

economic analysis framework is based on the hypothetical scenario of how industry would

respond if NIST ceased to provide its thermocouple infratechnology outputs. Postulating such a

counterfactual scenario was deemed the most feasible approach for analyzing a significant part of

NIST’s economic impact for this case study, although having a realistic baseline to draw from

would have been preferable.

 

30 See Albert Link, Evaluating Public Sector Research and Development, (Praeger: New York, 1996); and

Gregory Tassey, Rates of Return from Investments in Technology Infrastructure, NIST Planning Report 96-3,

June 1996.
4-5

4.4 BENEFIT DATA

Two surveys have been conducted for gathering information to determine the economic

benefits associated with NIST-supported infratechnologies for thermocouple calibration. In both

cases, telephone interviews were deemed preferable to mailed surveys. This decision has been

based on previous case study experience which shows that certain economic concepts (e.g.,

transaction cost savings) are conveyed most effectively in a semi-structured, interactive phone

conference.

4.4.1 Survey of Thermocouple Users

A brief survey was conducted from the perspective of thermocouple users to obtain

anecdotal evidence of the economic impact from the NIST TCP. Users derive considerable

economic benefits from the NIST TCP since accurate temperature measurements from

thermocouples are important in achieving downstream product quality and process efficiency.

Yet, these benefits have not been quantified across all domestic industrial users for two reasons.

First, the economic effect of NIST’s infrastructure technology is more direct and quantifiable on

the wire suppliers and thermocouple suppliers, and more indirect and difficult to quantify for the

users. Based on the pre-survey results, users can identify the general engineering and business

reasons for employing accurate thermocouples, but they have difficulty in disentangling the

economic benefits attributable to more accurate and frequent calibrations from other factors

affecting product performance and process efficiency.

Second, extrapolating the benefits from a representative sampling of user respondents to

the large and diverse population of thermocouple users was not feasible within the budgeted

resources of the study. The difficulty is due to the greater scope and complexity in the conceptual

framework for quantifying net product/process improvements among the diverse industrial user

segments in comparison to the more straight-forward framework for quantifying efficiencies for

conducting research for the infrastructural outputs and reducing transaction costs.31 Therefore,

the linkage between the economic benefits of users’ products/processes and the NIST TCP is

more difficult to discern, much less to quantify. Thus, while the collective economic benefits to

 

31 An alternative approach of quantifying the economic impact on one or a few individual industries (or industry

segments) that use thermocouples was considered, but this approach proved to be too difficult because of

problems in identifying the population of users within a given industry and the scarcity of appropriate market

data.
4-6

“second tier” industries are likely substantial, the data collection effort focused on the “first tier”

industry, suppliers of wire and thermocouples.

4.4.2 Survey of Thermocouple Industry

A survey of the thermocouple industry was determined to be the feasible focus for

gathering data associated with the primary benefits in the hypothetical experiment. The

prospective benefit data, which are presented in Section 5.2 and used in the calculation of

economic impact measures in Chapter 6, are based on telephone interviews of seven major

domestic wire suppliers and twelve major thermocouple suppliers. Based on aggregate levels of

domestic market share, these samples effectively represent the entire domestic thermocouple

industry.

Consideration was given to conducting a retrospective analysis of net benefits for the

hypothetical experiment. However, from a data collection standpoint, experience in previous case

studies shows that scientists and engineers are better able to forecast events than to create

hypothetical situations retrospectively.
5-1

5. SURVEY RESULTS

5.1 THERMOCOUPLE USERS

A telephone survey of 10 thermocouple users has been conducted to attain qualitative

insights on their calibration practices and their perception of the value received from the NIST

TCP.32 This survey of thermocouple users preceded the survey of the thermocouple suppliers

(discussed in Section 5.2) to enhance the quality of the latter survey. Data obtained in the users’

survey has not been incorporated in calculations of evaluation measures which are presented in

Chapter 6. The remainder of this section provides observations from the results of this survey.

5.1.1 Calibrations Performed Internally

Many users rely on assurances of thermocouple accuracy from calibration tests performed

internally rather than from statements of accuracy from their suppliers. This reliance on internal

calibrations is not due to concerns about the process of standards traceability via the NIST TCP.

Users have confidence in both NIST measurement and standards capabilities and in the ability of

suppliers to calibrate thermocouples that are traceable to national temperature standards. Instead,

three general factors regarding the stated accuracy of calibration data from suppliers cause

concern. The first factor regards practical aspects of calibrations for thermocouple wire and

assembled thermocouples in user applications. The emf versus temperature relationship of two

 

32 The respondents in the survey of 10 thermocouple users include representatives from the utility,

military/aerospace, food and beverage, aluminum, and commercial electronics industries. Thermocouple users

in these industries having applications with high accuracy requirements (+/- 1-2 0C) were determined to be best

informed about the impact of thermocouple calibration practices on downstream products and/or processes.

Such sophisticated users have the most at stake by relying on NIST for standards traceability to obtain accurate

temperature measurements. Upon request, a listing of industry representatives of this type was provided by

NIST TCP personnel based on their previous professional contacts.

The functional disciplines of the respondents include operational users of thermocouples in production and

development environments as well as test engineers in central corporate laboratories who provide working-level

calibration standards of thermocouples to operational users employing thermocouples in a wide range of

product and process technologies across their organizations. Most of the respondents had general knowledge in

all business and technical aspects of thermocouples used in their organizations. All of the respondents use

thermocouples in manufacturing processes, both in R&D and production environments; additionally, one

respondent described an application in which the thermocouple is used as a component in a product that the

company sells to a downstream customer. Two of the firms purchase all of their assembled/sheathed

thermocouples from thermocouple producers, two other firms fabricate all of their own thermocouples from

bare wire, and the other six have a mixture of both purchasing finished thermocouples and fabricating

thermocouples in-house.
5-2

given thermoelements can change by several degrees during the process of fabricating these wires

into a thermocouple with sheathing. This change in the calibration of the thermocouple often is

not stated (warranted) by the supplier, and the variation from thermocouple to thermocouple can

be significant. Users control this variation in their instrumentation by establishing offset values

that correlate with reference standards calibrated to NIST’s primary standards. Many users feel

that this random variation in calibration tolerances of thermocouples is accounted for most

efficiently through internal calibrations. As a result, many firms have corporate polices of

calibrating the majority, if not all, thermocouples purchased from suppliers. This is done because

the use of faulty thermocouple products in user applications could result in incorrect temperature

measurements which lead invariably to losses in productivity that greatly exceed the cost of a

calibration test.

The second factor regards users’ concerns on the quality of wire procured from wire

suppliers. Wire suppliers typically fabricate each production lot of wire in one large spool that is

divided subsequently into smaller spools for sale to thermocouple suppliers and users with captive

production capabilities. The supplier calibrates wire lengths from the ends of each spool.

Oftentimes the calibrations of wire within the spool deviate beyond an allowable tolerance range

from the calibrations performed on the end lengths. Such deviations have made some users

unwilling to rely on the supplier’s sample calibrations as representative of the entire wire spool,

which results in users performing additional calibrations internally. According to one user,

substantial oversight of the operations of a wire supplier would be required before both putting

greater trust in the supplier’s claims on product quality and subsequently lessening its (the user’s)

in-house validation testing.

The third factor regards user concerns about supplier quality that are not measurement

related. Several users claim that their suppliers have made errors in material handling that result in

misrepresentation of thermocouple accuracy: (1) incorrect labeling or color-coding of the wires

(for example, reversing the positive and negative thermoelements), and (2) making a

thermocouple with two identical wires. The occurrence of such errors has driven some firms to

increase its policies for the validation testing of incoming thermocouple products.

5.1.2 ISO 9000

The calibration requirements (in terms of tolerance levels or frequency of calibrations) of

thermocouple users generally have stayed the same or increased slightly over the past five years,

and compliance with the ISO 9000 quality control scheme has been the driving force for most
5-3

increases in calibration requirements. Certification to ISO 9000 standards is particularly important

in commerce with European and Asian countries. Establishing a process for documenting the

accuracy of temperature measurements to national temperature standards is an essential element

in meeting ISO 9000 certification. The standards traceability process through the NIST TCP

provides thermocouple users with an efficient means to comply with the requirements of ISO

9000 since the burden-of-proof for documented compliance can be obtained from the wire or

thermocouple supplier.

5.1.3 ITS-90

Users have realized few net economic benefits from thermocouples calibrated to the ITS90 compared with the previous temperature scale, the IPTS-68. The ITS-90 is more accurate than

the IPTS-68, and this improved accuracy has been incorporated into the latest reference tables

and functions from NIST.33 However, users generally have not gained improvements in product

performance or process efficiencies to offset the costs to incorporate the new calibration

standards based on the improved NIST infrastructure technology. In fact, some users have

processes that are not sensitive enough to changes in the new scale to warrant the costs of

changing their instrumentation to the new standards.

These findings corroborate the views of several thermocouple suppliers during the pretest

of the survey. One supplier estimates that 99 percent of all users would be unable to discern the

difference in their applications between ITS-90 and IPTS-68 calibrations, and the greater

accuracy incorporated in the ITS-90 calibrations would probably benefit less than one percent of

users having thermocouples calibrated to the IPTS-68. Further, many of the thermocouple users

undergo the conversion in their internal metrology infrastructure in order to maintain just one set

of standards even though they are aware that this conversion will not provide a positive economic

effect.34

 

33 The greater accuracy in the ITS-90 in comparison to the IPTS-68 is due primarily to the greater number (17

versus 12, respectively) of fixed points used to define the temperature scale. The fixed points refer to

thermodynamic equilibria of naturally-occurring phenomena that occur reproducibly at the same temperature.

34 Interestingly, many firms in the steel industry currently employ a number of large melting processes that use

thermocouples calibrated to a standard temperature scale that is obsolete across other industries. The change in

the international temperature scale from the ITS-48 (the International Temperature Scale established in 1948)

to IPTS-68 was significant, but the steel industry determined during the changeover period that altering their

processes to conform to the IPTS-68 was not economically rational. For similar reasons, the steel industry did

not adopt the subsequent change in the temperature scale, the ITS-90. As such, vendors still supply
5-4

5.1.4 Transaction Costs

NIST’s role in the standards traceability process reduces transaction costs with suppliers

of thermocouple products since procurement disputes between thermocouple users and producers

seldom occur.35 Also, having NIST make the more technically-difficult, highly-accurate

temperature measurements in the standards traceability scheme allows thermocouple users to

establish and maintain practical (working-level) calibration standards with equipment and

techniques of much lower cost than those used at the NIST Laboratory.36

5.1.5 Other

Quantifying the economic effect (increased productivity or product performance) in

applications involving a change in calibration requirements is very difficult. For example, a

producer of aluminum products in the aerospace industry described a new aluminum alloy that

saved one-third of the weight of the external tank on NASA’s Space Shuttle. Improved

temperature measurements in the aluminum melting process were significant in developing and

manufacturing this new material. However, this was only one of many process and design factors

contributing to the improved material, and quantifying the specific contribution in weight savings

attributable to improved temperature measurement capabilities was difficult to estimate.

The perceived quality of the NIST Thermocouple Calibration Program has no effect on

the substitution of thermocouples for other temperature measurement devices since the decision

to use thermocouples in lieu of other devices is based on physical factors regarding the specific

 

thermocouples calibrated with older temperature standards to certain firms in the steel industry. Further inquiry

shows that no other industry appears to demand this obsolete specification.

35 In the one reported case, NIST was instrumental in settling a dispute between a thermocouple user and its

downstream customer over relevant specifications of thermocouple calibration. In this case, the customer

requested thermocouples of extreme accuracy in applications involving high temperature ranges (generally,

greater than 1000 0C). NIST personnel were enlisted to help in resolving the issue, and, subsequently, NIST

convinced the customer that the accuracy being sought was not attainable within the technical state-of-the-art.

The customer eventually dropped its original requirement for the more stringent specifications.

36 The traceability scheme is especially cost-effective in the stringent military/aerospace environment which

requires a proportional method for achieving acceptable levels of accuracy for thermocouples in end-use. In this

method which is a part of the Mil-Standard regimen, a 4:1 ratio for accuracy is required in each link from an

end-use specification to the primary temperature standard at NIST. For example, if the end-use is a field

application requiring a +/- 16 degree accuracy, then the system supplier has to warrant a thermocouple with +/-

four degrees accuracy, which, in turn, is traceable to calibration measurements at NIST which are accurate to

+/- one degree. Since the cost to calibrate to +/- one degree is much greater than at +/- four degrees, the

traceability process provides a more cost-effective solution for each system supplier and their customers.
5-5

application; calibration factors are not a part of this decision. Also, thermocouple users receive

significant benefits from the technical support available from the NIST Program.

5.2 THERMOCOUPLE INDUSTRY

Based on self-reported market share information, the sample of seven wire suppliers

represents nearly 100 percent of the estimated $160 million domestic industry, and the sample of

twelve thermocouple suppliers represents over 90 percent of the estimated $120 million domestic

market.37 Since over 300 domestic thermocouple suppliers actively market thermocouple

products, as noted in Section 3.3.1, the industrial base of thermocouple suppliers appears to be

distributed very unevenly with a few large companies and many more smaller firms.38

5.2.1 Wire Suppliers

Opinions of the seven wire suppliers were mixed regarding their company’s reaction to the

hypothetical counterfactual scenario of NIST ceasing to provide primary calibration services. Four

of the seven thought that their company would rely on foreign laboratories for similar calibration

services provided by the NIST TCP. Two believed that over time an industry consensus on

measurement methods would develop through a private laboratory or industry association; the

emerging entity would then assume NIST’s current role in providing primary calibration services.

One company had no opinion.

 

37 Wire suppliers and thermocouple suppliers were defined for this study as the first-level users of NISTís

calibration services, and hence were defined as the relevant survey population for collecting primary benefit

data. The respondents, seven major domestic wire suppliers (three base-metal wire and four noble-metal wire)

and twelve of the major thermocouple suppliers, were identified and contacted by NIST TCP personnel to

ensure their participation in the data collection phase of this study.

In the course of each interview, information was requested on the size of the wire and thermocouple markets,

the importance of NISTís calibrations services, and how the industry would adjust to the hypothetical

counterfactual situation in which NIST ceased to provide calibration services through its Thermocouple

Calibration Program. An interview guide was prepared to ensure that all of these points were covered during

the course of the conversations.

38 In the course of the interviews with thermocouple suppliers, several respondents noted that a large number of

so-called suppliers in the industry are actually either small, specialized producers or simply repackagers of

other firms’ thermocouples.
5-6

Respondents believed that interactions with a foreign laboratory would incur additional,

permanent transaction costs under the counterfactual experiment. Based on previous interactions

with foreign laboratories, these costs would be associated with both the administrative “red tape”

and inaccessibility to scientists in the foreign laboratories. Although the quality and price of the

calibration services from those laboratories are deemed comparable to NIST, the red tape and the

delays in receiving services would be significant.

Those respondents anticipating that an industry consensus would develop over time ó the

mean time of response was five years ó also anticipated during this interval that: (1) a greater

number of measurement disputes would arise between their company and their customers in the

“chaotic” business environment that would ensue, and (2) company resources would have to be

devoted to the process of reaching industry consensus. Consequently, additional personnel would

be needed during this five-year time interval until the domestic industry reached consensus about

acceptable calibration measurements. Examples of the expected types of additional administrative

costs included auditing, changes in calibration procedures, and overseas travel. The total for all

respondents of the additional costs that would be needed to address all transaction costs issues,

absent the NIST TCP, was $325.0 thousand in 1996 dollars.

39

In additional to calibration services, the NIST TCP also provides telephone technical

support to industry. Absent NISTís services, wire suppliers would purchase similar expertise from

consultants, and the total annual cost for all wire suppliers for these substitute benefits was

estimated to be $146.5 thousand.

40

 

39 Each wire supplier was asked to approximate, in current dollars, the additional person years of effort required

to cope with the additional transaction costs that would be expected in the absence of the NIST TCP. Each

respondent was also asked to value a fully-burdened person-year of labor within their company. Two

respondents did not offer an estimate of the value of such expected transaction costs, but both did acknowledge

that such costs would exist. The mean value reported by the other respondents, by wire type, was imputed to

these two.

40 Each respondent was queried about the frequency with which they took advantage of this service, and on

average it was used five times per year (the range of responses was zero times per year to once a month). Each

respondent was also asked about the cost to acquire and utilize this form of NIST information, and in general

the response was that the cost was minimal. The economic evaluation literature refers to such costs as ìpull

costs.î
5-7

Thus, based on the collective opinion of the seven wire suppliers, which effectively

represent the entire domestic industry, if NISTís TCP ceased to provide calibration services, this

segment of the thermocouple industry would incur $471.5 thousand of additional personnel costs

annually to continue operating at the same level of measurement accuracy.

5.2.2 Thermocouple Suppliers

Similar hypothetical cost data were collected from twelve thermocouple suppliers.

Regarding an alternative measurement source in the absence of NISTís calibration services, only

two respondents thought that their company would rely on foreign laboratories for calibrations.

The other 10 respondents believed that an industry consensus would eventually emerge, and

similar to the mean reply from the wire suppliers, the mean length of time for achieving industry

consensus was estimated to be five years. One respondent mentioned that an organization “like

ASTM” would eventually take on the role of the NIST TCP.

The total additional cost during the adjustment interval for all firms in the survey was

estimated to be $1,543.4 thousand.

41 Also, the total additional cost for market alternatives to the

technical support received from NIST is $172.5 thousand annually.42 Thus, based on interviews

with the 12 thermocouple suppliers which effectively represents the domestic industry on a market

share basis, if the NIST TCP ceased to provide calibration services then this segment of the

industry would incur $1,715.9 thousand in additional annual costs to continue operating at the

same level of measurement accuracy.

5.2.3 Summary

Viewing the wire suppliers and thermocouple suppliers together as the first-level industry

beneficiaries of NISTís calibration activities through its Thermocouple Calibration Program, the

economic benefits associated with these services can be approximated in terms of the additional

 

41 Again, no quantitative response was offered by two of the respondents, so the mean dollar value provided by the

other respondents was imputed for these two.

42 Such interactions occur on an average of four times per year, with a range in responses from zero to once a

month.
5-8

costs that the thermocouple industry (users and producers) would incur in the absence of such

services. Based on extensive telephone interviews with representatives of the dominant suppliers

in the two industry segments, the total annual economic benefits were estimated to be $2,187.4

thousand in 1996 dollars. In other words, if NIST continued to provide its current annual level of

industry support, then the current annual cost savings (i.e., additional transaction costs avoided)

to society is $2,187.4 thousand.

Based on qualitative responses in the survey, thermocouple consumers are very

unresponsive to price changes since no close substitute products exist domestically.

43 The

interpretation of these responses in economic terms is that the market demand for wire and

thermocouples is highly inelastic. In the absence of calibration services provided through the

NIST TCP (the counterfactual experiment for this case study), the wire and thermocouple

suppliers collectively would incur $2,187.4 thousand in additional costs annually to pursue

alternative calibration services.

 

43 In each of the nineteen interviews, company representative were asked their opinion about how responsive their

consumers would be to an industry-wide increase in price. No supplier mentioned that their customers would

substitute to foreign made wire or thermocouples.
6-1

6. ECONOMIC ANALYSIS

6.1 COST AND NET BENEFIT DATA

6.1.1 NIST Annual Expenditures and Income44

Table 6-1 shows NIST’s expenditures associated with the outputs described in Section 4.1

from government fiscal years (FYs) 1990 through 1996.45 The line item for reference tables and

functions (FY90 to FY93) accounts primarily for research on the basic physical properties that

underlie the measurement science to incorporate the change from IPTS-68 to ITS-90. For this

effort, NIST led the development of, and shared the costs with the standards laboratories of eight

other countries. NIST’s costs accounted for about 60 percent of the total expenditures required to

generate the updated reference tables. Also, costs for technical support are accounted in the line

item for calibration services due to the multi-disciplined nature of work performed by personnel in

the NIST TCP.

Table 6-1 also shows income from fees received for calibration services. The significant

decrease in the line item for calibration income from FY91 to FY92 was due to the loss of a

significant customer that set up its own calibration laboratory. Also, included in the Table as

income are moneys ($135,000 in FY92 and $80,000 in FY94) associated with NIST’s Calibration

Services Development Fund (CSDF). The CSDF is derived from surcharges on calibration

services, and has been used for upgrading NIST’s calibration laboratories, initiating new

calibration services, and providing other improvements.

 

44 Information in this section was obtained from personnel in the NIST Thermocouple Calibration Program.

45 NIST’s expenditures include (1) direct, fully-burdened labor of laboratory personnel, and (2) costs of capital

equipment, facilities, and maintenance that are expensed in the year incurred. Operating funds from other

government agencies are negligible, and inter-governmental customers do not receive discounts on calibration

services fees which otherwise would decrease NIST’s costs.
6-2

Table 6-1 Expenditures and Income for The NIST TCP FY 1990 -1996 (current $)

FY90 FY91 FY92 FY93 FY94 FY95 FY96

Calibration Services Development Fund 0 0 135,000 0 80,000 0 0

Calibration Income 159,608 149,776 88,848 89,648 84,083 81,948 117,674

Total Income 159,608 149,776 223,848 89,648 164,083 81,948 117,674

Reference Tables and Functions 58,053 173,439 184,143 41,283 0 0 0

Test Methods and Upgrades 19,317 6,455 21,024 19,306 7,587 7,853 8,067

Calibration Services 143,019 145,992 278,015 205,992 198,524 203,988 166,646

Total Costs 220,388 325,886 483,182 266,581 206,111 211,841 174,713

6.1.2 Data for Economic Impact Analyses

Table 6-2 shows the data used for the economic impact analyses. Total expenditure data

from NIST for fiscal years 1990 through 1996 have been reproduced from Table 6-1. NISTís

costs are associated with the development of the ITS-90 reference tables and functions,

maintenance and upgrade of test methods and test equipment, and calibration services. Since this

case study uses a prospective basis for the economic evaluation, the costs to maintain the current

level of calibration services in the future are forecast from 1996 costs using a six percent rate of

increase to account for the expected increase in labor costs at NIST.46

Table 6-2 NIST TCP Costs and Net Industry Benefits (current $000)

Fiscal Year NIST Costs Net Industry

Benefits

1990 $220.4 ---

1991 325.9 ---

1992 483.2 ---

1993 266.6 ---

1994 206.1 ---

1995 211.8 ---

1996 174.7 ---

1997 185.2 $2,296.8

1998 196.3 2,411.6

 

46 This six-percent growth factor has been widely used in other economic impact analyses at NIST, and has been

verified as a reasonable estimate by the NIST Program Office.
6-3

1999 208.1 2,532.2

2000 220.6 2,658.8

2001 233.8 2,791.8

Government fiscal year 1990 has been selected as the appropriate first year of NIST’s

investments for this analysis. As explained in Section 2.3, 1990 is the year of the most recent

update of the international agreements on the scale of temperature (the ITS-90) for use in science

and industry, and the current technical state of thermocouple calibration measurement is based on

the development of reference tables beginning in that year. NIST began its share of investments in

new research in FY90 for upgrading thermocouple reference tables to ensure that U.S. firms

could trace their thermocouple products accurately to the ITS-90. While pre-1990 NIST

expenditures have certainly enriched the broadly-defined state of current technical knowledge for

thermocouple calibrations, the most relevant aspect of this knowledge is linked to conducting and

applying post-1990 research.47 Hence, 1990 was selected as a logical starting point for the

comparison of NIST’s costs to net industry benefits.

Also shown in Table 6-1 are annual estimates of net industry benefits for 1997 through

2001. These data are based on the 1996 estimate of net industry benefits totaling $2,187.4

thousand (see Section 5.2.3). This 1996 benefit estimate was extrapolated for five years into the

future using a five percent annual increase to reflect the projected costs of fully-burdened labor.48

The five-year forecast was selected for two reasons. First, five years represents the average

amount of time that respondents projected for the thermocouple industry to reach a consensus on

an alternative to NISTís calibration services in the hypothetical experiment. Second, although

some respondents believed that the additional transaction costs would exist forever if companies

relied on foreign laboratories and although market-based consulting substitutes for NISTís

technical assistance would similarly be continuous, truncating such net benefits to five years

 

47 The investment in any technical innovation starts with a preexisting stock of knowledge, and the attainment of

that knowledge is a sunk cost.

48 This factor was obtained during industry interviews.
6-4

provides a conservative adjustment for the social return on the investments made through the

NIST TCP.

6.2 MEASURES OF ECONOMIC IMPACT

The return that industry receives from investments made through the NIST Thermocouple

Calibration Program has been calculated using three widely-accepted measures of economic

impact. These evaluation measures are the internal rate of return, the adjusted internal rate of

return and the benefit-to-cost ratio.

6.2.1 Internal Rate of Return

By definition, the internal rate of return (IRR) is the rate of discount that causes a series of

costs and receipts to approach proportionate equivalence. Applying this concept to the evaluation

of research projects, the IRR is the discount rate that reduces the net present value (NPV) for a

stream of expenditures and net benefits to zero. Mathematically, the IRR is the discount rate r

*

that satisfies the equation:

NPV (r

*

) =

(B0 − C0 )

(1 + r

*

)

0 + ... 

(Bt - Ct)

(1 + r

*

)

n = 0

where (Bt - Ct) represents net benefits in year t, and n is the number of years for the research

project. Reducing NPV equal to zero is comparable to a benefit-cost ratio of one (the break-even

point), which allows comparisons of IRRs among projects of different sizes. The IRR equation is

then rewritten as:

NPV (r*) =

Bt

(1 + r

*

)

t

t=0

n

∑ -

Ct

(1 + r

*

)

t = 0

t=0

n

∑

and,

Bt

(1 + r

*

)

t

t=0

n

∑ =

Ct

(1 + r

*

)

t

t=0

n

∑
6-5

Based on the stream of cost and net benefit data for fiscal years 1990 through 2001 in

Table 6-1, the calculated value of r

*

that equates NPV to zero is 0.318 (rounded).

49 In other

words, 0.318 is the value of the discount rate that equates the present value of benefits to the

present value of costs. This implies an internal rate of return of 31.8 percent for NIST's

investments in thermocouple calibration activities.

6.2.2 Adjusted Internal Rate of Return

Under an alternative set of assumptions, an adjusted internal rate of return (AIRR) is

calculated from the data in Table 6-1.50 For example, if all of NIST’s costs are referenced to 1990

using a nominal discount rate of 8.78 percent as recommended by the Office of Management and

Budget, the 1990 present value of NIST’s costs is $2,015.3 thousand.

51 When all industry benefits

are referenced to the year 2001 using the same 8.78 percent discount rate, the 2001 present value

of industry benefits is $15,000.7 thousand. Using the AIRR, the NIST TCP is viewed as a single

calibration project where an initial net investment of $2,015.3 thousand generated, after 11 years,

a consumer surplus equaling $15,000.7 thousand. Thus, the annual compounded rate of return

corresponding to such an initial investment in 1990 that culminates in 2001 is based on the value

of i

*

 that satisfies the following relationship:

$2,015.3K (1 + i

*

)

11 

= $15,000.7K

In this case, the calculated value of i

*

equals 0.200 which means that the AIRR for the NIST TCP

is 20.0 percent.

 

49 The data in the first year of the benefit-cost stream are referenced to time t=0 in the IRR equation as well as for

the other economic measures.

50 The adjusted internal rate of return is similar to the overall rate of return suggested by Rosalie R. Ruegg and

Harold Marshall, Building Economics: Theory and Practice, (Van Norstand Reinhold: New York, 1990). An

example of the AIRR, also known as the compound rate of return, is the annual yield earned on a $100 bank

deposit. Interest is earned each year on the original $100 plus compounded interest.

51 See: “Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs,” Office of Management

and Budget (OMB), Circular No. A-94, 29 October 1992. OMB recommends both a seven percent real rate of

discount for federal project evaluations and an approximation of inflation based on the implicit price deflator of

Gross Domestic Product. Based on data from the Federal Reserve Board over the last four available quarters,

the implicit price deflator is 1.78 percent which is used to convert the real discount rate of seven percent to a

nominal rate of 8.78 percent.
6-6

6.2.3 Benefit-to-Cost Ratio

A benefit-to-cost ratio (BCR) relates the present value of NIST TCP costs to resulting

industry benefits. Costs and net benefits are deflated to 1990 using an 8.78 percent rate as in the

AIRR calculation. The BCR is 2.95 using 1990 present values of costs ($2,015.3K) and benefits

($5,943.9K).

6.3 MEASURE OF ECONOMIC IMPORTANCE

One way of approximating the economic importance of thermocouples in the U.S.

economy is to use the U.S. Department of Commerce (DoC) Standard Industrial Classification

(SIC) system and Input/Output statistics. Based on SIC descriptions, three 4-digit industries

(3823, 3824, 3829) include thermocouple producers. The composite sales in 1987 for these SIC

codes were $3.6 billion in components and $4.0 billion in capital equipment.52 These components

and equipment, in turn, were inputs for the manufacture of $2.462 trillion of manufacturing

output.

A quantification of the economic importance of thermocouples on domestic industry is

made by using the 1996 estimated total domestic market value ($280 million) for wire and

thermocouple assemblies and the 1987 Input/Output data. If the input/output relationship between

the aggregate industries in SICs 3823, 3824, and 3829 and total manufacturing output hold for

the more narrowly defined industries that they compose, then thermocouple products could affect

as much as $81.0 billion in manufacturing output.53

6.4 ECONOMIC RETURN FOR UPGRADING REFERENCE TABLES TO ITS-90

The survey provided evidence that the vast majority of thermocouple users — including

users with high accuracy requirements — have attained few net benefits in using thermocouples

 

52 Based on a phone conference with a representative of the DoC Bureau of Economic Analysis. The Input/Output

data is based on 1987, the most recent year that such statistics are available.

53 Using ratios of total direct requirements: $7.6 billion in components and capital equipment is to $2,462 billion

in manufacturing output, as $250 million in thermocouple products is to x dollars in manufacturing output.

Solving for x equals $81.0 billion. It is recognized that this analysis is somewhat faulty due to the use

inconsistent data because the $250 million figure is based on 1996 dollars while the other data is based on 1987

dollars, which is the most currently available data from the USDOC.
6-7

calibrated to the ITS-90 compared to thermocouples calibrated to the IPTS-68.54 Since the

upgrade of thermocouple reference tables and functions to facilitate the conversion from IPTS-68

to ITS-90 is one of the outputs being evaluated in this case study, a follow-up analysis was

performed to determine if insights could be gained for future strategic planning at NIST.

Two sets of investments have been made in the conversion of standards for thermocouple

calibration that are consistent with the improvements in the technical state-of-the-art of the

international temperature scale from IPTS-68 to ITS-90. The first set of investments has been

made collectively by agreements among NIST and the standards laboratories of eight other

countries, and these investments led to the upgrade of the reference tables and functions for

thermocouples calibrated to ITS-90.55 The second set is the costs incurred by all organizations

(users and suppliers involved with thermocouple measurements) that converted internal

measurement infrastructures to the ITS-90. Using the anecdotal results from the survey of

thermocouple users in this case study, the intrinsic payback on both sets of investments has been

poor.

The investment decision by NIST to upgrade the reference tables was driven completely

by international agreements on the change in the temperature scale. While the economic returns to

industry on this investment to date have not been significant, the original decision must be

assessed relative to the economic consequences of not having made the investment. In other

words, for the purpose of this case study, the evaluation of economic impact is posed in terms of

the likely scenario and qualitative outcome on domestic industry if NIST hypothetically had not

invested and participated in upgrading the thermocouple reference tables to ITS-90.

The remainder of this section describes four primary reasons offered by NIST TCP

personnel to justify the original investment decision. These views have been corroborated by

representatives in the thermocouple industry in follow-up interviews.

 

54 Theoretically, the greater accuracy embedded in the improved temperature scale (ITS-90) would allow

thermocouple users with high accuracy requirements to obtain enhanced product performance and process

efficiencies since the ITS-90 is more accurate thermodynamically than the IPTS-68.
6-8

6.4.1 International conformity to the temperature scale

The primary motivation for NIST’s original investment decision made cooperatively with

the standards organizations of the eight other countries was to keep all the world’s thermocouple

users and producers on the same temperature scale. This decision contributes to consistency in the

national standards for temperature among the U.S. and other industrialized countries that seek

leading-edge measurement technology to augment advanced technology products.

Under the hypothetical scenario of NIST not participating in the joint investment, U.S.

companies would likely have risked being unable to sell their products into certain international

markets. U.S. firms would still be using thermocouples calibrated to the ITS-68 while the other

industrialized countries would require products conforming to national standards based on the

ITS-90. The likely consequence is that U.S. firms’ products incorporating temperature

measurements with IPTS-68 thermocouples would not be welcomed as imports into foreign

markets having temperature specifications linked to the ITS-90.56 For example, most foreign

companies now specify ISO 9000 product standards which require ITS-90 compliance. Therefore,

maintaining national standards consistent with those of other countries that use high technology

products provides significant competitive advantages internationally.

6.4.2 Other Temperature Measuring Devices

The ITS-90 affects all devices for measuring temperature. Many organizations that use

thermocouples also use other types of temperature measuring devices for internal applications.

Given the over-arching acceptance of the ITS-90, the temperature measurement community has

agreed that investments in upgraded calibration knowledge were necessary for all temperature

measurement devices. If such investments were not made for every device, then users of would be

faced with making in-house temperature measurements with more than one temperature scale.

Certainly, the use of more than one temperature measurement system simultaneously would not

 

55 NIST took the lead role for technical work involved in the conversion process, and the other countries

contributed calibration test results.

56 The consequences are much greater for the population of thermocouple users in comparison to the smaller

population of domestic wire suppliers and the thermocouple suppliers since the product sales outside of the U.S.

for the latter set of producers are not significant.
6-9

be rational economically or practically for user organizations. Therefore, the decision to make the

investment in upgrading the thermocouple reference tables was made for consistency with similar

upgrades of calibration know-how for other temperature measuring devices.

6.4.3 Long-Term Benefit Horizon

A long-term “benefit horizon” is always assumed in investment decisions by NIST. Part of

NIST’s charter is to predict infrastructure technology for the future, not just for meeting current

needs of users. Accordingly, NIST regularly undertakes investments for increasing the technical

state-of-the-art with the understanding a priori that many users will not (or be unable to) take

advantage of the improved measurement technology until they need or obtain better control over

their internal product and process capabilities. Determining when these needs arise is difficult to

predict, so the ready availability of the NIST-developed infrastructural know-how overcomes this

problem.

Another reason for early insertion of advanced infrastructure technology is that typically a

significant amount of time and resources is needed for both conducting the requisite R&D and

implementing block changes in the standards agreement process. In the case of developing the

improved reference tables calibrated to ITS-90, a three-year effort on the part of NIST was

required to develop the R&D followed by one year for incorporation into the ASTM standards.

6.4.4 R&D Leadership Role

NIST’s lead role in developing the improved infrastructure technology provides domestic

thermocouple users and suppliers with certain benefits not shared by their counterparts in other

countries. Since most of the technical work was performed and promulgated through NIST,

which has a stellar reputation for quality and independence, the domestic thermocouple

community has assurances of a sound technical foundation for the infrastructure technology

outputs. Alternatively, this community might have less acceptance of similar outputs derived from

foreign laboratories having technical capabilities that are not as firmly established. Sponsorship of

similar infrastructure technology from foreign laboratories that are comparable technically with
6-10

NIST could have negative effects on the competitiveness of the domestic thermocouple

industry.57

 

57 One user respondent suggested that, in lieu of the NIST TCP, many domestic users would buy thermocouple

products from firms in the country of the lead standards organization instead of from U.S. firms because of the

technical intricacies involved between a standards organization and the first-tier thermocouple producers.

Obviously, this procurement posture would not bode well for the supply side of the domestic thermocouple

industry.
7-1

7. CONCLUSIONS

7.1 ECONOMIC IMPORTANCE OF THERMOCOUPLES

The analysis in Section 6.3 shows that thermocouples comprise a relatively small

percentage of cost in higher product structures but their economic importance has a large rippling

effect in the overall manufacturing economy. In other words, thermocouples provide an efficient

means for performing the temperature measurement function that is essential in many industrial

applications, and thermocouples are relatively inexpensive compared with other component items

in higher-level assemblies of products and processes that incorporate thermocouple calibration

know-how. Since temperature is the most widely-used indicator in industrial processes, the

implication is that thermocouples are significantly important in the domestic manufacturing

economy.58

The relatively low cost of thermocouples is attributable, in part, to the effectiveness of the

NIST Thermocouple Calibration Program. This case study has proven that both industrial users

and suppliers of thermocouples believe that the NIST TCP provides the most efficient means for

incorporating the complex technology embedded in practical measurements of temperature made

with thermocouples. Therefore, society realizes significant benefits from investments made by

NIST which result in the relatively low costs for both thermocouple products and measurements

made with thermocouples.

7.2 ECONOMIC IMPACT OF THE NIST TCP

Economists and policy makers generally use internal rate of return measures to estimate

the social rate of return (SRR) for on-going or completed research projects in the public sector. In

this case study, if the 31.8 percent IRR calculated in Section 6.2.1 is above NIST's hurdle rate

(minimum acceptable rate of return) then the investments in NIST thermocouple calibration

research and related activities are worthwhile from a social perspective. Also, in comparison to

 

58 The thermocouple is the most widely-used among other electronic devices (RTDs, thermistors, etc.) for sensing

temperature. Temperature, in turn, is the most widely-used among other indicators (force/displacement and

optical radiation) for physical sensing. Physical sensing, in turn, is the most prevalent among the other classes

of sensors (chemical, biosensor, and microsensor). See Richard C. Dorf, ed., The Engineering Handbook,

(IEEE Press), 1996, pp. 1484-1489.
7-2

the average hurdle rate of about 12 percent for corporate technology projects, investments in the

NIST TCP are beneficial from a general economic standpoint.59

The IRR is often confused with the adjusted internal rate of return (AIRR). Both measures

are used in financial analyses but the type of cash flow pattern associated with the AIRR occurs

neither in an R&D project, in general, nor in the case of NIST TCP-related research and activities,

in particular. Therefore, the AIRR is not directly comparable to the IRR discussed in the

economics literature.

Several factors have been incorporated in this study to characterize the 31.8 percent IRR

as a conservative estimate of NIST’s economic impact on industry. These factors are:

(1) Industry benefits pertinent to thermocouple suppliers are

underestimated since the reported benefit data from the 12 sample

suppliers (having about 90 percent cumulative market share based on

self-reported data) was not extrapolated to the many other firms

(having 10 percent cumulative market share) in the industrial base.

Indeed, according to recent internal records of the NIST TCP, at

least 32 additional thermocouple suppliers have paid for calibration

services from NIST. Benefits related to these smaller suppliers have

not been accounted since the estimate of total benefits is based on

market share – not total number of companies. Potentially, these

smaller suppliers collectively would incur significant costs under the

hypothetical experiment used in this analysis.

(2) Only the first two major types of benefits (research efficiencies and

transaction cost savings) have been included in this IRR calculation

because these economic impacts were more direct and easier to

quantify. Disregarding the third type of benefit (users’ product and

process improvements) also underestimates industry benefits.

 (3) The 1997 starting point also adds to a conservative benefit estimate

since industry attains benefits from NISTís investments in prior years

(1990-1996) of the economic framework for this study. The 1997

starting date was selected because of the study’s prospective

methodology as explained in Section 4.4.2.

 

59 J. Poterba and La. Summers, “A CEO Survey of U.S. Companies’ Time Horizons and Hurdle Rates,” Sloan

Management Review, Fall 1995.
7-3

 (4) The “bottom line” annual costs of the NIST TCP are much less than

the reported expenditures since the policy of NIST is to charge user

fees to recover the marginal costs of calibration services.

Additionally, surcharges have been applied through the CSDF to help

pay for the underlying research and equipment for improved

thermocouple calibration know-how.

Offsetting NIST’s costs with these income items would result in an

attribution of all the social benefits to only a portion of the total costs

in this case study. Therefore, conservatism is built into the

calculations of the economic measures since NIST’s costs have not

been offset by its income. From a more liberal economic viewpoint,

the estimated return on NIST’s investments would be higher if

expenditure data were offset with income.

Therefore, the 31.8 percent IRR is a lower bound estimate of the industry benefits attributable to

NIST’s investments in thermocouple calibration.

7.3 POTENTIAL IMPROVEMENT IN THE TRACEABILITY PROCESS

The findings in Section 5.1.1 show that many thermocouple users having requirements of

high accuracy rely on calibration tests performed internally rather than from statements of

accuracy obtained from their suppliers. Many users claim that internal calibration testing is both

necessary due to the random nature of thermocouple calibrations and an efficient means of

maintaining process control with applications having stringent accuracy requirements. However,

there is ample evidence that, to some extent, the efficiency in the standards traceability scheme

headed by NIST is being degraded since users cannot rely totally on their suppliers to provide the

correct representations of calibrations for thermocouple products. This is a sign of market failure

in thermocouple commerce due to redundant calibration testing being performed by certain users

on purchased lots of wire or finished thermocouples that are calibrated previously to standards by

the supplier.60

The root cause of this non-optimal situation appears to derive from issues about

assurances of product quality from suppliers of wire and thermocouples, and these issues are not

linked directly with measurement-related know-how from the NIST TCP. However, NIST could

 

60 In the economic literature, the term “market failure” refers to a malfunction in private-sector activity that

results in a suboptimal allocation of resources.
7-4

use its neutral role to help industry overcome this problem. Possibly, assistance from the NIST

Manufacturing Engineering Laboratory would identify ways for industry to improve product

uniformity so that the implementation of the traceability process can approach greater levels of

efficiency. An analysis should be performed to determine the economic payoff to users for

investments in manufacturing technology that would enhance the quality of thermocouple

products.

7.4 LESSONS-LEARNED FROM IMPLEMENTING ITS-90

The situation described in Section 6.4 regarding the conversion to the most current

international temperature scale, the ITS-90, provides some interesting insights on the timing of

investments in leading-edge infrastructure technology. All of the reasons provided by NIST for

the investment in upgraded reference tables have economic validity with respect to their intended

objectives, but a net economic gain has not resulted from the overall investment because the key

benefits (improved product performance and process efficiency) affecting thermocouple users

have not been realized. However, NIST made the correct decision to invest in the innovative

measurement technology (the updated reference tables) in response to an international agreement

to change the temperature scale. U.S. industry would likely have suffered negative economic

consequences if NIST had not made this investment. The failure to realize net positive economic

benefits resulted from a mismatch in the timing of when this infrastructure technology can

effectively meet the practical needs of industry (thermocouple users).

In retrospect, the proper course of action might have been to delay the international

agreements on updating the International Temperature Scale until industrial users’ products and

processes would be projected to absorb more-readily this innovation in infrastructure technology.

In turn, the investments in updating calibration technology — such as the thermocouple reference

tables — could have been postponed. From an economic perspective, such investments would be

more effective if better intelligence were available on the timing of the economic payoff including

dependency on foreign trade. Such an analysis would have included estimates on the number of

users expecting to benefit from the improved ITS-90 based calibrations and the timing of when

such users’ requirements would emerge. In other words, investments in state-of-the-art
7-5

technology infrastructure must coincide as closely as possible with the evolution of industrial

demand and, hence, corporate strategy.