Cold Chain Storage of Vacc

A Brief Introduction to Thermom

Dean Ripple

Chemical Science and Technology Labor

National Institute of Standards and Techn

dean.ripple@nist.gov

301-975-4801

September, 2008

NISTIR 7522
Outline

• Description of the problem• Overview of sensor types• Traceability to NIST• Measurement Assurance• Uncertainty & error allowances (g• Certification & Calibration• Proper use of thermometers• FAQs
A Cartoon of Cold Chain Stora

Vaccine, 

Tv 

Certified thermometer, reading 

Tcert
Our Goal: Maintain true vaccine tempe

specified bounds (e.g., 2 °C to 

Complications

• We do not measure actual vaccine temperatur• Reading of certified thermometer may be in er

– calibration error– thermometer drift– error in reading

• Temperature may vary across storage region• Temperature may not be stable due to

– doors opening and closing– seasonal variations of heat loads– separation between control thermometer and 
Thermometer Types
Thermometer Components

Readout

Probe

Sensor inside

Some specifications for thermocouples, PRTs, thermistors pertain to sensor or probe only, not complete thermometer
Thermometer Types

Industrial Platinum Resistance Thermometers (RT

–196 °C to 850 °C

Thermistors

–50 °C to 100 °C

Thermocouples (TC)

–196 ° C to 2100 °C

Liquid-in-Glass Thermometers (LiG)

–150 °C to 400 °C

Dial Thermometers

–100 °C to 400 °C

Digital Thermometers (PRT, thermistor, or thermoc

–196 °C to 850 °C

Data Loggers (PRT or thermistor in disguise)
Tolerances vs. Calibration Uncer

0.001

0.01

0.1

1

10

-200

0 200 400 

Temperature / °C

Tolerance or uncertainty / °C Thermistor, accuracy class 1LiG, total imm

LiG, partial immersion 

PRT, grade A

Colored lines: ASTM tolerances (ASTM E1, E1137, 

Dashed lines: NIST calibration uncertainties at 95 

Tolerances or uncertainties are ± the indicate
General Uncertainty & Cost Guid

Thermistor probe

−20 °C to 60 °C approximate range0.2 °C uncertainty off the shelf, $300 (much lower uncertainties attainabl& better readout)

Sheathed PRT

−200 °C to 600 °C approximate rangUncertainty 0.2 °C near ambient, exnear limits for $300 + $150 certif.(much lower uncertainties attainablcalibration and better readout)

Base-metal thermocouples

−200 °C to 1200 °CUncertainty (0.5 % to 1% of 

t/°C) + 1 

(all uncertainties at approximately 95 % confide
General Uncertainty & Cost Guidelines 

Organic-filled liquid-in-glass thermometers

−100 °C to +100 °C4 °C uncertainty off the shelf, $30 + certificate Potential for uncertainties down to 0.5 °C near higher costNo standardization yet

Dial thermometers

−20 °C to + 300 °C (lower limit for dial)Uncertainty 1 % of full range or about 1 °C, wh

$100 + certificate costNo standardization
Maintenance & Calibration

Mercury-in-glass thermometers (for t < 150 °C) ne

inspection plus a check at 0 °C.

PRTs & thermistors require checks at the upper a

of use. (Single-point check OK for narrow usage chain application).

Organic liquid-in-glass thermometers need a visu

plus a check at upper and lower limits of use. (SOK for narrow usage range).

Thermocouples, if used above 150 °C, require sp

techniques. For cold chain, single point check + 

If the thermometer drifts appreciably at any of th

then you need to do a full recalibration, or buy anthermometer.
Types of Liquid-in-Glass (LiG) Ther

Types of ASTM LiG thermometers

– Over 120 types– Total Immersion: bulb & full column of liquid a

calibration temperature

– Partial Immersion: thermometer exposed to us

temperature only below immersion line; remai

temperature

Liquids used

– Mercury (Hg) - ASTM– Organic– Mercury Thallium– Proprietary (non-toxic)
Non-Mercury Liquid-in-Glass Therm

• Organic liquids generally have inferior perfor

but are a reasonable alternative if uncertainty remodest (ASTM standard just begun)

• Beware of drainage of organic liquid down ca

cooling (not an issue for long-term monitoring o

• “Next-generation” organic liquids under deve

ASTM standard E2251); good accuracy, but cheliquid column

• Greater sensitivity to stem temperature: not a 

chain applications with thermometer totally imme
Platinum Resistance Thermometers (P

2, 3, or 4-wire resistance element – nominally 100 

Ω

 

@ 0 °C

– Wire wound– Thick film– Thin film

Resistance changes as a function of temperaturePositive temperature coefficient Nominal temperature range of use:

– –200 °C to 850 °C

Nominal resistance at 0 °C

– 100 

Ω

Sen(wir

Steel sheath containing sensor
Thermistors (Thermal Resistor)

Semiconductors of ceramic material made by sintering mioxides such as manganese, nickel, cobalt, copper, iron anTemperature Range: –50 °C to 100 °CStandard Sensor Forms:bead 300 

Ω to 100 ΜΩ

probe bead in glass rodNTC: Negative Temperature Coefficient - The vast ma

thermistors used as thermometers are in the NT

Commonly packaged in stainless-steel sheathsExcellent choice for cold-chain applications
Thermocouples

Standardized combinations of Material A vs. Material B (e.g, Type K).When combined with typical readout uncertainty, total uncertainty is fairly large, unless care is taken with calibration.Good for low-accuracy measurements over a broad temperature range

T0

Material A

Material B

Voltmeter + 

Junction 

Compensation
Dial Thermometers

• Bimetallic element changes shape on change • Mechanical linkage converts motion of bimeta

deviation of pointer on dial

• Friction in mechanical linkage limits accuracy• Inexpensive & easy to read; not too accurate
Traceability
Overview of Traceability to N

Reference: http://www.nist.gov/traceability, espe

Supplemental Materials link

Only a measurement, not an instrument or artifac

Evaluation of traceability is the customer’s respo

does not, in general, evaluate claims of tracea

Evaluation of traceability is one component of th

accreditation process
Elements of Traceability

1. Unbroken chain of calibrations from NIST (or ot

Metrology Institute) to final instrument; each cadocumented uncertainty.

Traceability does imply known uncertainty to known n

does not imply that laboratories are NIST does not imply that calibration is of high a

2. A Measurement Assurance Program must be in 

continued validity of a calibration (i.e., you can’thermometer calibrated in 1928 as your standar

Examples periodic recalibration, with examination of 

comparison of certified thermometer with thermometercomparison of calibrated thermometers brecalibrationscheck of thermometers at the ice point 
Traceability Responsibilitie

NIST

– Assures traceability of results of measuremen

NIST provides to customers 

– Provides customers with the tools they need 

• Establish traceability of their measurement r• Assess the claims of traceability made by ot

Customer

– Assesses the validity of a traceability claim fr

provider

– Establishes traceability of their measurement 

Accreditation Bodies

– Evaluates secondary laboratory’s ability to 

maintain traceability

– Tests for competence and capability of a se

laboratory’s calibration service
Tolerances vs. Calibration Uncertaintie

Tolerance band: manufacturer’s guarantee that response will conform to a standard response funerror equal to the tolerance. Calibrated thermometer: may or may not have a the nominal response function for that thermometeResponse of individual unit is reported, along with calibration process. Individually calibrated thermometers cannot be cointerchangeable, unless the readouts or software incorporate the individual response function.Accuracy: an imprecise term, approximately meamanufacturer guarantees that the instrument will greading to within the stated accuracy.
Measurement Assuranc
Measurement Assurance

Three examples:1. Measurements at the ice point2. Use of a check standard (second redundant th3. Evaluation of historical calibration resultsYou don’t have to do all three, but 

you should do at least one.

Use of the ice point is the most precise method, b

use of a check standard gives the broadest sc
Measurement Assurance Techniques: I

Preparation:• Ice bath consists of crushed or shaved ice, satura

packed into a Thermos or Dewar

• Make a hole with a clean rod, insert thermometer 

10 min. for thermal equilibrium

• Ice must extend to bottom of flask, not just floating • Uncertainty about 0.002 °C with distilled water, ab

tap water

• Try it out first on a newly calibrated probe, to be c

doing it correctly

What’s great about the ice point:• Can be prepared by the user• Perfect check temperature for cold-chain applicat

• Drift at the ice point can be used to determine rec
Measuring a Liquid-in-Glass (LiG) Ther

in an Ice Melting Point (0 °C)

Pre-chill thermomshaved ice, or ice slush.Important if you thermometers, or uncertainty better 
Measuring a Liquid-in-Glass in an Ice Meltin

Use a clean rod (make thermomeinsertion of therm

Gloves are not nuncertainty of 0they do keep yo
Measuring a Liquid-in-Glass Thermo

in an Ice Melting Point (0 °C)

Thermometer in an 10 minute equilibra

– LiG is pre– 3 min equ

the ice me

Siphon liquid water Repack ice prior to insertion

Ice point baths for probes do not need and can be more sl

0 °C mark just abobottom rests on ice
Plotting the data from ice-point re

Monitor readings at the ice point for sudden changes unacceptably far from 0 °C

-0.1

0.00.10.20.30.40.50.6

0 2 4 6 8 

Time since calibration, months

Reading at ice point, °CBad Good 

Sudden jump
Measurement Assurance Techniques: Redun

This is the single most useful technique presente

Vaccine, 

Tv

Certified thermometer, reading 

Tcert

Cdnc
Example 1: Cal. laboratory incorrectly adj

34567

0 2 4 6 8

Measurement number

Temperature, °C

Certified therCheck therm

Cal. date

Large change in (

Tcert – 

Tcheck ) after instrument is return
Example 2: Readout for certified thermo

34567

0 2 4 6 

Measurement number

Temperature, °C

Certified thermometeCheck thermometer

Cal. date

Unexpected 

change
Example 3: Slow divergence of certified & che

We DO NOT calibrate the certified thermometer with th

thermometer.

We DO use the change in 

Tcert – 

Tcheck as an indicator o

34567

0 2 4 6 

Measurement number

Temperature, °C

Certified thermometerCheck thermometer

Difference near 0 °C

Differe

change
Measurement Assurance: Histori

This method is only useful if the instrument is no

calibration laboratory supplies both “as receiveddata

Case 1: unadjusted instrument

Year Reading at 0 °C1999 0.0 2000 0.2 2001 0.5 2002 0.7 2003 1.0 

Aver. drift: 0.25 °C/year

Maximum drift of 0.3 °C/year

2004 2.5 Observed drift of 

0123

0

Mea

Temperature, °C
Measurement Assurance: Historica

Case 2: adjusted instrumentYear As rec’vd. As adjust. Change from 1999 0.0 0.0 

2000 0.2 0.2 0.2 

2001 0.5 0.0 0.3 2002 0.2 -0.1 0.2 2003 0.2 0.0 0.3 Aver. drift: 0.25 °C/year

Maximum drift of 0.3 °C/year
Quick Guide to Uncertainty Termin

An uncertainty is a statistical measure of how likely t

measurement agrees with the true answer.

Uncertainty at 68 % confidence (“standard uncertain

lower case u. The difference between the true ans

measured value will be less than ±u, 68 % of the 

For higher confidence limits, u is multiplied by a cove

obtain the “expanded uncertainty” U = k u

The default for calibration certificates is 

k=2, which fo

distributed uncertainties corresponds to a 95 % co

When there are multiple uncertainty components, the 

uncertainty 

uc is obtained by adding the compone

2

2

2

2

1c

n

"+++= uuuu
Error Allowance (Guardbandi

Q: Is the blue line an acceptable set of readings?A: No, probably not!

123456789

0246

Time, days

Temperature, °C

Upper Limit

Lower Limit
Adding up measurement uncertainties of vacc

Table of uncertainties

Instrument resolution 0.25 Drift 0.25 Storage volume non-uniformity 0.10 Thermometer accuracy/cal. uncertainty 0.5 

Linear sum = (0.25 + 0.25 + 0.10 + 0.50) °C = 1.1 °C Root-sum-square (used by metrologists) = 

[(0.25)2 + (0.25)2 + (0.10)2 + (0.50)

2

]1/2 = 0.6 °C = l

Since the likely error in this case is approximately not allow readings within 0.6 °C of allowable limits
Evaluating uncertainty compon

Instrument resolution:

About 1⁄4 to 1⁄2 of a graduation, for dial or liquid the

About 1⁄2 of smallest digit for digital instruments

Instrument drift over one year:

Obtained from historical trends of calibration resuObtained from manufacturer’s literature or specificCrudely estimated as the resolution for liquid ther

instrument accuracy for other thermometers

Non-uniformity of storage volume:

Obtained from multiple measurements of storage thermometers in different positions
Storage Volume Temperature Non-u

Non-uniformity measured by examining differenc

thermometers: 

• keep one thermometer at ‘home’ position; • move the other one, wait for temperatures to r• Variation of difference between two thermome

positions measures non-uniformity of storage 

• You can make the non-uniformity as small as 

making the storage volume smaller

• Only need to do this measurement once, if you 

configuration of cold enclosure
N T(home) T(N) T(N) −

 

T(home)

1 3.4 3.6 0.2

2 3.6 3.8 0.23 3.8 1.7 -2.14 3.7 1.5 -2.25 3.4 4 0.66 3.5 3.5 0.0

Average for positions 1 & 6 0.1Maximum deviation from 1 & 6 −2.3

Numerical example of non-uniformity m

2 

3

Top-to-bottom variation can be several degrees CeNon-uniformity may depend on how refrigerator is 
Guardbands provide confidence that vacci

is within allowed range

123456789

0246

Time, days

Temperature, °C

Exceeds limits 

Limit with guardbandLimit with guardband
Guardband variation

A more accurate thermometer can give you a gre

variation in measured temperature

Example: 

guardband of 0.6 °C gives an allowable range 

a span of 4.8 °Cguardband of 2.0 °C gives an allowable range a span of 2.0 °C
Calibration Reports vs. Certific

Internationally accepted standard for calibration 

ISO/IEC 17025:2005 “General requirements foof testing and calibration laboratories”

17025 Requirements for calibration reports includ

a) titleb) name and address of laboratory performing the calibrc) unique identifier of the calibration certificated) name and address of the customere) identification of the method usedf) a description of the item testedg) results of the calibrationh) evidence of traceabilityi) environmental conditions of the test, if appropriatej) uncertainty of the measurementk) signature authorizing the certificate
Certificates are often “Calibratio

17025 elements generally included:

a) titleb) name and address of laboratory performing the c) unique identifier of the calibration certificatef) a description of the item testedg) results of the calibrationh) evidence of traceabilityk) signature authorizing the certificate

The most important elements usually not include

e) identification of the method usedj) uncertainty of the measurement

Certificate statement of accuracy does not includ

instrument drift; you must account for drift se
Shopping for a calibration labo

Do you want units adjusted?If yes, then consider:• experience of facility with your brand of unit (esp. • does cal. report give as-received and as-adjusted 

Look for:• stated traceability to NIST• documented uncertainty• accreditation of laboratory to ISO 17025 (not man

of some level of proficiency)

• good past history
Proper Use: Thermocouple

1. Avoid kinks in the thermocouple wire passing 

seal.

2. Thermocouples have fast time response—esp

to put measuring junction in a volume of liqui

3. Avoid use of readouts at refrigerator/freezer te

unless designed and/or calibrated for such us

4. Thermocouple wire should extend from meas

to room temperature without junctions. Extenconnectors OK at room temperature.

5. Be careful that readout is set to use the correc

type to match the sensor type (types are colo

6. For calibrated units, be careful to use the sam

with the same readout 
Proper Use: Liquid in Glass Therm

Inspect column for separations, especially when 

for organic-liquid-filled

• Keep in a vertical orientation to avoid separati

• Avoid parallax error on reading

Right

Wrong
Proper Use: Dial Thermomet

• Avoid mechanical shocks, which can damage 

linkage

• Avoid parallax error on reading• Avoid/discard dial thermometers that are susc

inadvertent shifting of scale 
Proper Use: PRTs and Thermis

1. Avoid mechanical shock, which can damage s

2. For calibrated units, be careful to use matche

readouts

3. Erratic readings are often a sign that moisture 

lost—measure insulation resistance or calibrasensor

4. Remember that most “digital” thermometers a

use a PRT or thermistor as the active element
FAQs for Vaccine Cold Chain The
How often should thermometers be c

• Initial calibration intervals should be based on 

recommendations or past experience with a ty

• Calibration intervals may be adjusted based o

calibration results of a particular instrument.

• If check measurements indicate significant dr

recalibrate. 

• If check measurements indicate large and sud

instrument from service. 

• For vaccine cold chain, a single-point recalibr

point + visual inspection may suffice for liquidthermometers. 
We are getting ready to send some of our therm

calibrated. Can you help me in deciding which c

to have done? 

• Vaccine cold-chain applications cover a limite

Instruments need to be calibrated only at the limits of use.

• Instruments with digital readouts should be c

same manner in which they are used. e.g., if a is used cold, then the calibration would need with a cold readout.

• Uncertainty of calibration should be compara

than instrument repeatability, or significantly allowable temperature tolerance for the vaccin
Is it correct for a certificate of instrument calib

traceability through the American Society for Tes

(ASTM)? 

• No. ASTM produces documentary (paper or e

standards that describe test methods and maspecifications. 

• Manufacturers may use ASTM methods to per

that are traceable to the standards of NIST or metrology institute. Traceability in this case iequivalent in other countries. 

• Recently, the international community has ad

“metrological traceability” to describe the tracmeasurements, to avoid any ambiguity.
What is the difference between ASTM 

• ASTM is a standards development organizatio

documentary standards, and related books anASTM standards are developed through a con

based on the work of volunteers from manufa

and other interested parties, including NIST s

• NIST is a non-regulatory federal agency within 

Department of Commerce that promotes U.S. industrial competitiveness by advancing meascience, standards, and technology.

• Temperature measurements in the U.S. are tra

primary temperature standards, which are disthrough a combination of customer calibratiomaterials.

• NIST also includes an accrediting organizatio

Voluntary Laboratory Accreditation Program (
Is a qualified vendor the best choice 

thermometer preventive-maintenance 

• Users can perform in-house performance che

checks in an ice-point bath. As a means of va

house check methods, we recommend that us

performance checks on newly calibrated instrknown to be accurate. 

• Recalibration of thermometers should be carr

personnel using appropriate standards and fa

• Recalibration may be conducted by either a q

a qualified laboratory within an organization.
How important is it to establish period

maintenance and calibration sch

Traceability of the measurements, which is critic

the cold chain, absolutely requires periodic thchecks and calibration schedules. 
Must periodic preventive maintenance and calib

be established according to standard operat

• Standard procedures are important, in order t

confidence in the measurements and to assurchecks are not forgotten.

• The maintenance of traceability implies docum

procedures and calibration results.
On most thermometer instrument calibratio

certificates it is stated that: Accuracy

What does this really mean?

• “Accuracy” is a common, but not well defined 

calibration work. 

• In its most common usage, “accuracy” is the 

guarantee that the instrument will give the cowithin the stated accuracy. 

• How long the instrument will meet this accura

the manufacturer! 

• In the field of metrology, the preferred term fo

statements is “uncertainty”. 

• In the field of thermometry, a stated accuracy 

comparable to uncertainty at a 95 % confidenc
What should a provider consider before purchas

to assist with proper vaccine storage an

• How is the thermometer to be read? Remotel

individual?

• Referring to the material on the guardband, se

combined instrument uncertainty/accuracy an

• How will the thermometer be recalibrated or re

expense is often greater than the cost of the ithermometer.

• Where will the readout be located, and will thi

recalibration difficult?

• What is the measurement assurance plan?