Strengthening Calibration 

through Improved Focu

Planning

James Olthoff

Deputy Director

for Measurement Services

Physical Measurement Labora
Points

• Calibrations are critically importa• We have great strengths

• We have issues and weaknesses

• We are moving forward
Building on a foundat

Measurement Scienc

Measurement

Applications
NIST provides traceab

Working

Standards

Transfer

Standards

Realization Representation Disseminatio
• NIST suppor

system ($30

through app

power and e

per year

• 40+ million mammograms

delivered at 9,000 US facilitiestrace back to 2 dozen

proficiency tests andcalibrations per year from NIST

• 400,000 ann

mass standaState Weigh

Laboratoriehighly accurperformed a

Calibrations are critic
Calibrations are critic

• NIST calibrations ensure

the accuracy andreliability of over 1.8

million Navy calibrations

performed on

approximately 140,000different types of

instruments

• 203 calibrations f

over 70,000 subse

that help ensure t

performance of th

nuclear arsenal

• 8 NIST calibration

Force underpin 22

calibrations, 158,0measurements, 7

calibrations, and o

voltage calibratio
Calibrations are criti

NIST Internet Time Service Synchronizing computers and networks more than 3 billion times per day. Applications include time-stamping of hundreds of billions of dollars of financial transactions each day.

NIST-calibrated lo

to provide traceameasurements ocells used during of aircraft wing as
Some success storie

Reinvention of oscilloscope calibrations• Collaboration with Intel engineers• Enables measurement of rapidly time‐varying

signals like those found in digital electronics

• New procedure for digital waveforms increased

pass‐fail range by factor of 3 or more, reducing

unnecessary circuit board rejections

Solid state lighting• In response to 2001 Rep

Radiation Measurement

• Accurate measurement 

successes (improved eff

• Resulted in documentar

accreditation of comme

implementation of DOE/

program and product lab

for SSL commerce/trade
Spectral irradia

One staff member is r

• 5 labs• 33 measureme• Multiple qualit

• International c

Some weaknesses

Entire service relies on single scientist – NIST Fellow Dr. Judah Levine•Inventor and sole provider of NIST Internet Time Service and other NIST network time services

Time & frequency
Some weaknesses

Low frost point generator

• Aging refrigeration system diminishes ability to

maintain needed temperatures below –100 °C

• Limits measurements for Greenhouse gasses

measurements (water vapor) and semi‐

conductor water vapor contamination

Electrical m•Aging primar

voltage are fa

•Research pro

program in 19

•Unable to ut

technologies 
Some unmet need

• Compelling need

– Requested of NIST by American Association of

Physicists in Medicine (AAPM) and the Council on

Ionizing Radiation Measurements and Standards

– Expanding area in prostate /other cancer therapies

– Less‐than‐direct traceability chain back to NIST

– Sources needed to improve treatment accuracy

• Multiple barriers to NIST engagement

– Extensive shielding requirements for safety and

protection of sensitive instrumentation

– Dedicated lab required

• Impact

– Limits the accuracy achievable in treatments

High dose rate (HDR) 192Ir brachytherapy source
Significant issues for calib

• Inconsistent determination of the relevance and 

calibration services relative to other programs.

• Full recovery pricing policy results in customers g

NMIs.

• Flat resourcing has reduced staffing of critical cal

• Legacy systems are difficult to modernize or repl• No uniform metrics exist to measure quality, imp

success of existing services.

• Long term plans for the evolution, improvement, 

of services often do not exist.

• Research planning often is poorly connected to c

applied metrology needs.

• Customer interactions are uneven and often not 
Some unanswered ques

• How do we evaluate the relative importance of large 

customers?

– DoD accounts for 3 of the top 4 customers and approxi

calibration income

– Nearly 600 different customers use NIST calibration ser

• What resources should NIST commit to involvement i

metrology?

• How do we evaluate the relative importance of techn

competitive needs of customers?

• What if the most effective means of dissemination ge

• What do we do if we can no longer support a service t

customer needs?
A path forward
2009 calibration incom

Previous organization (by divisio

Division Calibratio

Quantum Electrical Metrology Process Measurements Precision Engineering Manufacturing Metrology Optical Technology Optoelectronics Ionizing Radiation Electromagnetics Electron & Optical Physics Building Environment Physical & Chemical Properties Analytical Chemistry Manufacturing Systems Integration Time & Frequency TOTAL 
2009 calibration incom

What’s now in PML

Division Calibratio

Quantum Electrical Metrology Process Measurements Precision Engineering Manufacturing Metrology Optical Technology Optoelectronics Ionizing Radiation Electromagnetics Electron & Optical Physics Building Environment Physical & Chemical Properties Analytical Chemistry Manufacturing Systems Integration Time & Frequency PML TOTAL 
2009 calibration incom

What’s now in PML

Division Calibratio

Quantum Electrical Metrology Process Measurements Precision Engineering Manufacturing Metrology Optical Technology Optoelectronics Ionizing Radiation Electromagnetics Electron & Optical Physics Building Environment Physical & Chemical Properties Analytical Chemistry Manufacturing Systems Integration Time & Frequency PML TOTAL 

99% of all calibration income is now in PM
A path forward

• Develop a common NIST vision for cali• Define excellence for calibration servic

• Develop strategic plans for services

• Define and monitor appropriate metric

• Peer assessments

• Organizational planning to identify unm

support resource growth
A path forward

• NIST vision for calibration services

– Develop unified view and expectations of calibration services w

– Focus on relative importance of services compared to other pr

• Excellence for calibration services

– Develop excellence criteria, including impact, need, quality of 

– Perform thorough assessment of capabilities and pricing comp

• Strategic plans

– Develop plans for each measurement area

• Research, calibrations, development

• Staffing and funding

• Deliberate goals

– Involve customers in planning discussions

– Manage to the plans

• Appropriate metrics

– Determine relevant metrics and monitor them

– Reevaluate the NIST Quality System after realignment

• Peer assessments

– Utilize peers from other NMIs

– Directly engage key customers to evaluate performance

– Determine value of additional “Councils”

• Identify unmet needs to support resource growth

– Utilize strategic plans
What happens then

• Critically evaluate programs

– Achieve excellence in all calibration services

– Manage calibration programs as a portfolio

– Recognize staff performance– Effectively represent the US in international metrology efforts

• Develop dynamic programs

– Better integrate research and calibration programs

– Rapidly respond to industrial and national needs

– Achieve significant and immediate impact solutions

• Improve service

– Replace and upgrade legacy systems

– Shorten delivery times– Develop more effective pricing policies

• Improved customer interactions

– We understand their needs

– They understand our plans– Increased customer satisfaction with processes

• Act on critical measurement needs

– Direct existing resources to most important challenges– Obtain additional resources– Revitalize supporting research programs

– Fill critical staffing needs

• PML is recognized as the single‐point source for calibration

needs

– Consistent policies and implementation

– One‐stop shopping

– Improved dissemination of measurement information
Questions?