5) Cons: Phase steps equal to a multiple of the period of the carrier frequency (10 js) will
occur if signal becomes weak and receiver loses tracking point. These phase steps
can make data hard to interpret. LORAN-C service might be discontinued in
2008, so few manufacturers continue to support and sell equipment.

d) Global Positioning System (GPS) service operated by United States Department of Defense
1) Performance: 2x 10°" (24 h average)

2) Equipment: ° Receiver, antenna, time interval counter and 1 Hz divider, or integrated GPS
timing receiver.

3) Method: Set up receiver and antenna. Connect GPS receiver output and signal from device
under test to phase comparison system. Measurement system collects data and
produces a phase plot that shows the phase shift of the device under test versus
time of day. The frequency offset and stability can be derived from the slope of
the phase plot.

4) Pros: GPS signals are usable nearly anywhere on earth. Better performance than
WWVB or LORAN-C and more reliable reception. Provides position and time-

of-day information.

5) Cons: GPS is primarily a navigation service, and not all GPS receivers are suitable for
use as a time and frequency standard.

2.2.4.4 Distribution of frequency signals

Care should be exercised in the distribution of oscillator signals to avoid:

a) Degradation of performance by overloading the oscillator output.

b) Coupling unwanted signals back into the oscillator (standard).

c) Coupling noise into the distribution circuit.

2.2.4.5 Measurement errors

Most techniques (excluding the audible zero-beat method) have a measurement resolution that is many times
better than errors due to radio propagation. Measurements should be made so that radio propagation delays
can be averaged out over time. In general, this is easy to do for frequency measurements, but more difficult
when determining time-of-day.

2.2.4.6 Time-of-day

2.2.4.6.1 Time, as in time-of-day, is not a standard even though it can be referenced to time-of-day signals
maintained at national laboratories. Any frequency source can be used to derive a time-of-day signal with
the same stability as the source, but the uncertainty depends upon how well the source can be synchronized

to the external reference. Proper synchronization of the signal requires a knowledge of equipment and signal
propagation delays. The accuracy of any timekeeping system depends upon how often the system is

NIST Handbook 150-2C 13 September 2001


compares favorably with the measurement of the audit laboratory (NIST or one designated by NVLAP),
taking into account the relative uncertainties assigned by both the applicant and audit laboratories. NVLAP
employs the same principle used by the European Cooperation for Accreditation and described in EA-2/03
covering international measurement audits. Note that the objective is not to determine and certify the total
uncertainty of the applicant laboratory, as is done ina Measurement Assurance Program (MAP) with NIST,
but to verify (through the assessment process) that the uncertainty claimed by the applicant laboratory is
reasonable, and then to test that the measurement result obtained through the proficiency test is acceptable.

It is not the intention nor the mission of NVLAP to conduct MAPs or to otherwise provide traceability for
laboratories. Laboratories obtain these services from the NIST measurement divisions. NVLAP assesses
the implementation, application, and documentation of MAPs by laboratories. NVLAP accreditation
encourages the use of MAPs by the calibration laboratory community, and MAP results produce objective
evidence that NVLAP assessors look for as part of the assessment process.

1.7.2.3 Requirements

NVLAP's proficiency testing program uses a sampling approach. All applicant laboratories are required to
complete an annual proficiency test in one parameter under each field of calibration for which it has applied
to be accredited. For the purposes of the NVLAP Calibration Laboratories Accreditation Program, the
results of the proficiency test are considered as objective evidence, along with the on-site visit, of a
laboratory's ability to perform competent calibrations. Proficiency testing is conducted annually using
different parameters in each field; however, those laboratories accredited in only one parameter withina field
are retested in the same parameter.

1.7.2.4. Uncertainty determination

The applicant laboratory is required to perform a measurement or series of measurements on an artifact using
the same calibration method, apparatus, and personnel that it uses to calibrate customers’ equipment. The
laboratory must be able to identify and quantify all sources of uncertainty that affect the measurement. The
laboratory should attach an overall uncertainty to the measurement by combining all uncertainty
contributions, in their type A and type B components, in the root-sum-squared method as described in the
Guide to the Expression of Uncertainty in Measurement (see 1.4.1 (b)). The confidence limit used should
be k = 2, which is equivalent to a 95% confidence probability.

1.7.2.5 Pass/fail criteria

The performance of the proficiency test is judged by calculating the error of the measurement, normalized
with respect to the uncertainty of the measurement, using the following equation:

Ejonmal = | (Value,,, - Value,.,) / ( Uncertainty,,, + Uncertainty,,,’)" |
where

Exommat = Normalized error of the applicant laboratory
Value,,, = the value as measured by the applicant laboratory
Value,,; = the value as measured by the reference laboratory

Uncertainty,,;= the uncertainty of the reference laboratory
Uncertainty, = the uncertainty of the applicant laboratory

NIST Handbook 150-2C 7 September 2001


the resolution and accuracy of PM noise measurements. Since the noise processes, especially lower
frequency processes, are often not white (with frequency dependence going as 1/f, 1/f*, etc.), the bandwidths
of systems and averaging times are important. Using the IEEE or internationally accepted specification (see
references) for these noises will account for this problem.

2.3.5.2 Calibration procedures for PM noise measurements

2.3.5.2.1 PM noise measurement systems can be divided into two-oscillator and single-oscillator
measurement systems. The most common measurement technique for PM noise measurements is the two-
oscillator method. This technique offers the widest range of carrier frequencies, the lowest noise, and the
widest bandwidth for analysis. Single-oscillator systems require only one source, but have much higher
noise floors close to the carrier. A problem with this technique for calibration purposes is that it is much
more difficult to evaluate the noise floor because the noise in the source always contributes to the
measurement. Because of this problem, two-oscillator methods are generally preferred over single-oscillator
methods. More detailed descriptions of these and other measurement methods can be found in NIST
Technical Note 1337 (reference 2.3.6 a)).

2.3.5.2.2 The errors that affect phase noise measurements using the two-oscillator technique are listed below.
The model for confidence limits depends upon whether the analyzer uses Fast-Fourier-Transform (FFT) or
a swept method.

a) Value of detector sensitivity and possible variations with frequency.

b) Value of gain and possible variations with frequency.

c) Variable phase-locked-loop effects at low frequency.

d) Non-linearity of spectrum analyzers.

e) Contributions of AM noise.

f) Inaccuracy of power-spectral-density (PSD) functions.

g) Conversion of noise at harmonics.

h) Noise contributions from the system noise floor.

i) Analysis uncertainty in separating the reference oscillator noise.

2.3.5.2.3 The calibration laboratory should indicate the biases and uncertainties for each of the nine errors
for every measurement. The laboratory should ensure that sufficient data are taken to be sure that a 95 %
confidence level (1.9 6) can be properly specified.

2.3.5.2.4 Itis sometimes necessary to use cross-correlation techniques to reduce the measurement noise floor
so that it doesn't limit the resolution and accuracy of PM noise measurements (see reference 2.3.6 a)). The
methods of reference 2.3.6 b) should be used to estimate the 95 % confidence window for statistical
uncertainty of the PSD measurements. Because of the problem of AM-to-PM conversion, no PM

measurement is complete without determining the AM noise and the AM-to-PM conversion factor as a
function of Fourier frequency of the measurement system.

NIST Handbook 150-2C 17 September 2001


the relevant requirements of ISO 9002 (ANSI/ASQC Q92-1987). These handbook criteria have been updated
to incorporate the requirements of ISO/IEC 17025:1999. The entire series of Handbooks 150-2A through
150-2H comprises information specific to the Calibration Laboratories Program and neither adds to nor
detracts from requirements contained in NIST Handbook 150.

Any questions or comments on this handbook should be submitted to the National Voluntary Laboratory

Accreditation Program, National Institute of Standards and Technology, 100 Bureau Drive, Stop 2140,
Gaithersburg, MD 20899-2140; phone (301) 975-4016; fax (301) 926-2884; e-mail NVLAP@nist.gov.

vi


synchronized to the reference, the stability of the oscillator, and the distribution delays from the system to
the user.

2.2.4.6.2 A number of satellite, radio, and telephone systems carry a digital time code. With the right
equipment, this code can be read and used to obtain time-of-day. In some cases (WWV and WWVH radio
and telephone services) the time is also sent by voice. A list of time-of-day signals is given below. It is the
responsibility of the laboratory acquiring such time signals to properly account for all propagation and
equipment delays. National laboratories, such as NIST, can only ensure that the time is accurate when it
leaves the broadcast source.

a) Radio time-of-day signals

1) WWV and WWVH (voice & digital code)

2) WWVB (digital code)

3) Global Positioning Satellite System (digital code)
b) Telephone and Internet time-of-day signals

1) Automated Computer Time Service (Digital code for computers by telephone at Boulder, CO is
303-494-4774 and at Hawaii is 808-335-4721).

2) WWYV audio (Colorado) can be heard by dialing 303-499-7111. WWVH audio (Hawaii) can be
heard by dialing 808-335-4363.

3) NIST Internet Time Service (ITS). A list of IP addresses can be found at:
http://www. boulder.nist.gov/timefreq/service/time-servers.html.

2.2.5 Certificates and reports

2.2.5.1 Calibration results are normally reported in terms of frequency offset defined as the ratio of the
frequency offset or error [f(actual) - f(nameplate)] to the nameplate frequency [f(nameplate)]. The
nameplate frequency is the nominal frequency stated on the oscillator or the specification sheet. The
frequency offset is positive (+) if the actual frequency is greater than the nameplate frequency and negative
(-) if less. Care should be taken in reporting results so that this sign is properly determined. Frequency offset
is usually expressed in scientific notation since the numbers involved are small. For example, if an oscillator
has a frequency offset of 1 part per billion, its frequency offset is expressed as 1 x 10°. The frequency
uncertainty is determined by making multiple measurements of the frequency offset and performing a
statistical analysis.

2.2.5.2 The averaging time used to determine frequency should always be reported. It should be long
enough to produce a meaningful answer. High quality oscillators require longer averaging times. For
example, the frequency ofa 1 x 10° oscillator can be measured by averaging for a few minutes or less using
a reference such as GPS, LORAN-C, or WWVB. However, measuring the performance of a 1 x 10"?
oscillator might take 24 hours or longer.

NIST Handbook 150-2C 14 September 2001


less continuous control, but it is still a good idea to occasionally perform a round-robin evaluation of the
system.

2.3.3 Accommodation and environment

The laboratory conditions (temperature and humidity control) required for measurements should be those
commensurate with the sensitivities of the components and oscillators used to perform the measurements.
This, of course, depends upon the level of precision desired. Unless the oscillators are particularly sensitive,
this usually means that a fairly standard calibration environment (+1 °C and + 15 % RH) will be adequate.
2.3.4 Equipment and reference materials

2.3.4.1 The following are the general equipment requirements for AM and PM noise measurements:

a) Spectrum analyzer (rf) covering the carrier frequencies of interest.

b) Baseband spectrum analyzer with a calibrated power-spectral-density (PSD) routine.

c) Calibrated power meter with + 0.5 dB accuracy.

d) PM detectors with post amplifiers.

e) AM detectors with post amplifiers.

f) Electronic counter.

g) Means to calibrate the PSD function of the spectrum analyzer in item b) above.

h) Means to calibrate the linearity of the spectrum analyzer in item b) above.

i) Means to calibrate gain as a function of frequency for amplifiers in items d) and e).

2.3.4.2 The following are the general equipment requirements for time-domain measurements:

a) Electronic counter with input for external reference oscillator.

b) Mixer with post amplifier.

c) Reference oscillator with performance better than clock/oscillator under test.

d) Isolation (active or passive) for device under test.

2.3.5 Calibration methods

2.3.5.1 AM and PM noise

The noise that limits the spectral purity of a signal is composed of two different components, AM and PM
noise. To adequately specify the level of AM or PM noise on the signal, it is necessary to use measurement

techniques that are sensitive to one type of noise and have good rejection of the other type of noise. In many
sources, AM and PM are comparable in magnitude. The conversion of AM noise to PM noise often limits

NIST Handbook 150-2C 16 September 2001


power outages. For high performance levels (better than 1 x 10°'°), an isolation amplifier is needed to
distribute signals within and outside the laboratory.

2.2.3 Equipment and reference materials

The laboratory manager should first decide on the performance level required for traceable calibrations, then
choose a reference signal that can meet the required level. In making these choices, the manager should
consider the equipment and labor costs involved. Both NIST and manufacturers of receiving equipment can
help decide which signal to use. There are several factors to consider before making a decision:

a) The selected radio signal should meet the performance standards claimed by the laboratory. Otherwise,
calibration claims cannot be validated.

b) The chosen radio signal must be usable at the receiving site; not all radio signals are usable everywhere.
2.2.4 Calibration methods

2.2.4.1 Frequency calibrations can be made continuously or at periodic intervals. The device under test
can be a stand-alone device or an oscillator embedded inside another piece of equipment. Stand-alone
devices include quartz, rubidium, cesium, and hydrogen maser oscillators. Embedded devices are usually
quartz or rubidium oscillators that serve as the time base for a piece of test equipment, such as a frequency
counter or signal generator.

2.2.4.2 In most cases, laboratories calibrate only their best oscillator against an external reference. After
its performance is known, the signal can be distributed or used to calibrate other oscillators. As a general
tule, the frequency reference for a measurement should perform at least one order of magnitude better than
the oscillator being calibrated. The reference frequency should be one of the sources listed in section 2.2.4.3.

2.2.4.3 The following is a short description of the four main external signals available at this writing. See
the references or contact NIST if further information is required.

a) WWV/WWYVH High Frequency Radio Service from NIST (zero beat method)

1) Performance: 1 x 10” (single measurement)

2) Equipment: Shortwave radio with headphones or meter, antenna, and harmonic generator (if
needed).
3) Method: Tune in WWV or WWVH, couple oscillator to receiver antenna and adjust

oscillator for zero beat in headphones or on meter.

4) Pros: Simple equipment. Large coverage area. Low cost. Provides time-of-day
information (audio and digital time codes). The audio time code can be used for
simple time interval measurements such as the calibration of stop watches and
timers.

5) Cons: Manual operation, must be done often. Cannot be done if reception is poor. No
measurement data; only physical record is laboratory log. This makes uncertainty

NIST Handbook 150-2C i September 2001


c)
d)

e)

8)

h)

D

%)

)

ISO/IEC 17025: 1999: General requirements for the competence of testing and calibration laboratories.
ISO/IEC Guide 43: 1997, Proficiency testing by interlaboratory comparisons, Part 1 and Part 2.
ISO/IEC/BIPM International Vocabulary of Basic and General Terms in Metrology (VIM), 1993.
ISO documents b) through e) are available from:

Global Engineering Documents (paper copies)
Order phone: (800) 854-7179

American National Standards Institute (ANSI) (electronic copies)
Electronic Standards Store
ANSI web site: http://www.ansi.org

NIST Technical Note 1297, Guidelines for Evaluating and Expressing the Uncertainty of NIST
Measurement Results. Available on-line at http://physics.nist.gov/Document/tn1297.pdf.

EA-2/03, EA Interlaboratory Comparison (previously EAL-P7), Mar. 1996. Available on-line at
http://www.european-accreditation.org/.

ANSI/NCSL Z540-1-1994, Calibration Laboratories and Measuring and Test Equipment—General
Requirements.

ANSI/NCSL Z540-2-1997, U.S. Guide to the Expression of Uncertainty in Measurement.
NCSL Recommended Practice RP-7: Laboratory Design, 1993.
NCSL documents h) through j) are available from:

NCSL International
1800 30" Street, Suite 305B
Boulder, CO 80301-1026

Phone: (303) 440-3339

Fax: (303) 440-3384

E-mail: info@ncslinternational.org

Web site: http://www.ncslinternational.org

C.D. Ehrlich and S. D. Raspberry, “Metrological Timelines in Traceability,” Jour. Res. NIST, Vol. 103,
No. 1, Jan-Feb, 1998.

C. Croarkin, Measurement Assurance Programs, Part II: Development and Implementation, NBS
Special Publication 676-II (U.S. Government Printing Office, Washington, DC, 1985).

1.4.2 Additional references specific to time and frequency measurements are listed in Section 2.

NIST Handbook 150-2C 3 September 2001


Summary

This guide presents the general technical requirements (i.e., on-site assessment and proficiency testing) of
the laboratory accreditation program for calibration laboratories along with specific technical criteria and
guidance applicable to time and frequency measurement. These technical guidelines are presented to
indicate how the NVLAP criteria may be applied.

Any calibration laboratory (including commercial, manufacturer, university, or federal, state, or local
government laboratory) engaged in calibration in time and frequency measurements listed in this handbook
may apply for NVLAP accreditation. Accreditation will be granted to a laboratory that complies with the
criteria for accreditation as defined in NIST Handbook 150. Accreditation does not guarantee laboratory
performance - it is a finding of laboratory competence.

Fields of calibration covered: Specific calibration parameters and related stimulus and measurement
devices in areas of time and frequency.

Scope of accreditation: Calibration parameter(s), range, and uncertainty level. Types of measuring and test
equipment. Quality assurance system for measuring and test equipment.

Period of accreditation: One year, renewable annually.

On-site assessment: Visit by an assessor(s) to determine compliance with the NVLAP criteria before initial
accreditation, in the first renewal year, and every two years thereafter. Preassessment and monitoring visits
as required. All calibration parameters or general areas of calibration within the specific scope of
accreditation requested will be assessed.

Assessors: Selected from technical experts with experience in the appropriate areas of calibration and
quality systems assessment.

Proficiency testing (measurement assurance): Each laboratory is required to demonstrate its capability to
successfully perform calibrations as part of on-site assessment or by documented successful completion of
an approved Measurement Assurance Program (MAP) or round-robin correlation. Proficiency testing may
be required for initial accreditation, or where other evidence of measurement assurance is not evident, and
may be conducted annually thereafter. Advance notice and instructions are given before proficiency testing
is scheduled.

Fees: Payments are required as listed on the NVLAP fee schedule, including the initial application fee,
administrative/technical support fee, on-site assessment fee, and proficiency testing fee.

viii


Acknowledgments

NIST Handbook 150-2 was first available as a draft covering all eight fields of physical metrology
calibration in one volume. It has been separated into eight handbooks to allow easier updating and electronic
downloading from the NVLAP web site. The preparation of these documents has been a joint effort, with
input from representatives of other government agencies, laboratories, and the private sector.
Acknowledgment of their efforts is in order; however, the listing of individual names is impractical. The
submissions by individuals and companies offering suggestions for improvement to this document were also
very welcome, as were the contributions of those who attended the public workshops.

We thank all the NIST measurement divisions for their work in writing or contributing to the individual
handbooks. Listed below are those from the NIST measurement divisions who deserve special thanks for
input to Handbook 150-2C, Technical Guide for Time and Frequency Measurements:

Michael A. Lombardi (Frequency and Time), Dr. Donald B. Sullivan, and David A. Howe
(Spectral Purity).

Additional thanks go to those who actively participated in the Technical Guide Workshop held November
1993 and to those who served as points of contact within fields of calibration. They include: Georgia L.
Harris, Norman B. Belecki, Dr. Theodore D. Doiron, Robert M. Judish, Thomas C. Larason, Sally S. Bruce,
and Dr. Donald B. Sullivan. A special thanks is owed to James L. Cigler for work in developing the content
and format of this guide, and to Vanda White for her editorial expertise in making this a readable document.

Above all, we wish to thank Jon M. Crickenberger, the editor of the first three drafts of this document, for
literally hundreds of hours of his work in creating this guide. It was he who tasked the contributors to
produce the technical content, assembled the results of their efforts into a consistent format, and provided
the general commentary. Without Jon's dedicated effort to this monumental task, this guide would never
have been published.

NVLAP has edited the individual handbooks and made changes resulting from comments by individuals to
earlier draft versions. This editing has been to a different extent for each parameter. Every effort was made
to include all pertinent information relevant to an ISO/IEC 17025-derived technical guide.

C. Douglas Faison and Carroll S. Brickenkamp, Editors
National Voluntary Laboratory Accreditation Program
National Institute of Standards and Technology

100 Bureau Drive, Stop 2140

Gaithersburg, MD 20899-2140

vii


analysis difficult. Suitable only for calibration of low-performance quartz
oscillators such as those found in inexpensive test equipment.

b) WWVB Low Frequency Radio Service from NIST (60 kHz carrier frequency)

1) Performance:

2) Equipment:

3) Method:
4) Pros:
5) Cons:

1 x 10°? (24 h average)

LF tracking receiver, antenna, phase comparison measurement system, and battery
backup. Phase comparison system is built into receiver in some cases. In other
cases, it can be a chart recorder or a computer-controlled system that includes a
time interval counter and frequency divider.

Set up WWVB receiver and antenna. Connect LF receiver output and signal from
device under test to phase comparison system. Measurement system produces
either a strip chart recording or a phase plot that shows the phase shift of the
device under test versus time of day. The frequency offset and stability can be
derived from the slope of the phase plot.

Permits calibration of atomic oscillators under good conditions. Carrier frequency
is directly produced and controlled by NIST. Provides time-of-day information.

Noise can be a problem over long signal paths (> 2000 km). Large diurnal phase
steps (tens of microseconds) might appear in data if a combination of skywave
and groundwave is received, since the length of the skywave path can fluctuate.
Phase steps equal to a multiple of the period of the carrier frequency (16.67 us)
will occur if signal becomes weak and receiver loses tracking point. These phase
steps can make data hard to interpret.

c) LORAN-C LF signals from United States Coast Guard

1) Performance:

2) Equipment:

3) Method:

4) Pros:

1x 10? (24 h average)

LORAN-C receiver, antenna, phase comparison measurement system, and battery
backup. Phase comparison system is built into receiver in some cases. In other
cases it can be a chart recorder or a computer-controlled measurement system that
includes a time interval counter and frequency divider.

Set up LORAN-C receiver and antenna and tune to nearest station. Connect
LORAN-C receiver output and signal from device under test to phase comparison
system. Measurement system produces either a strip chart recording or a phase
plot that shows the phase shift of the device under test versus time of day. The
frequency offset and stability can be derived from the slope of the phase plot.

Permits calibration of atomic oscillators. Signals are strong and easy to receive.
Pulses are modulated on 100 kHz carrier and provide a carrier tracking point.
This makes it possible to identify and stay locked to the same carrier cycle if the
laboratory is within 2000 km of LORAN-C transmitter.

NIST Handbook 150-2C

12 September 2001


On September 18, 1992, a public workshop was held at NIST Gaithersburg and attended by a mix of private
sector and government personnel. The workshop reviewed a draft handbook, which included general
requirements, as well as very specific technical requirements for de voltage calibrations at all levels. Asa
result of the workshop, the draft handbook was revised to take the form of a Calibration Laboratories
Program Handbook, which included the general requirements for laboratories (using ISO/IEC Guide 25 as
a basis), and eight companion Technical Guides covering the specific requirements for each field of
calibration offered for accreditation.

On May 18, 1993, a public workshop on the revised draft program handbook was held at NIST Boulder and
attended by more than’60 industry and government personnel. Comments from this workshop, as well as
responses to a survey/checklist mailing, were used to prepare the final draft of the handbook, now entitled
NVLAP Procedures and General Requirements (NIST Handbook 150), published in March 1994, [A revised
NIST Handbook 150 incorporating ISO/IEC 17025:1999 is dated 2001.]

A public workshop for the Calibration Laboratories Technical Guides was held at NIST Gaithersburg, on
November 22 through 24, 1993. More than 60 industry and government personnel attended and provided
comments on the draft version of the Technical Guide for each of eight fields of calibration. Asa result, the
eight Technical Guides were incorporated into a draft Handbook 150-2, Calibration Laboratories Technical
Guide, covering the fields being offered for accreditation. [In 2000, Handbook 150-2 (draft) was divided into
eight handbooks, one for each calibration area.]

The need for technical experts to serve as assessors was advertised, and the first group of assessors was
selected and trained during a four-day session held from November 16 through 19, 1993, in Gaithersburg,
using materials developed by NVLAP.

The Calibration Laboratories Accreditation Program officially began accepting applications when
notification was given in the Federal Register dated May 11, 1994. Applications are accepted and processed
in accordance with procedures found in NIST Handbook 150.

1.4 References
1.4.1 The following documents are referenced in this handbook.
a) NIST Handbook 150, NVLAP Procedures and General Requirements; available from:

National Voluntary Laboratory Accreditation Program
National Institute of Standards and Technology

100 Bureau Drive, Stop 2140

Gaithersburg, MD 20899-2140

Phone: (301) 975-4016

Fax: (301) 926-2884

E-mail: nvlap@nist.gov

NVLAP Web site: http://www.nist.gov/nvlap

b) ISO/IEC/BIPM (BIPM is the Bureau International des Poids et Mesures, the International Bureau of
Weights and Measures) Guide to the Expression of Uncertainty in Measurement (GUM), 1993.

NIST Handbook 150-2C 2 September 2001


2.3.5.3 Calibration procedures for AM noise measurements

2.3.5.3.1 AM noise measurement systems usually have a high PM noise rejection and so determination of
PM noise can often be neglected. It is often necessary to use cross-correlation techniques to reduce
measurement noise floor enough that it does not limit the resolution and accuracy of the measurements (see
2.2.7 c)). The list below gives the errors for AM noise measurements.

a) Value of detector sensitivity and possible variations with frequency.

b) Value of gain and possible variations with frequency.

c) Conversion of noise at harmonics.

d) Noise contributions from the system noise floor.

e) Non-linearity of spectrum analyzers.

f) Inaccuracy of power-spectral-density (PSD) functions.

2.3.5.3.2. The calibration laboratory should indicate the biases and uncertainties for each of the six errors for
every measurement. The laboratory should ensure that sufficient data are taken to be sure that a 95 %
confidence level (1.9 6) can be properly specified.

2.3.5.4 Time-domain calibration procedures

2.3.5.4.1 A simple time-domain characterization can be derived from a time series of direct, fixed-interval
counts of the frequency. One can derive from this one of the two-sample variances (Allan variance, modified
Allan variance or time variance) as a function of averaging time t (see reference 2.3.6 a)). The computer
effectively varies the averaging time by looking at successively larger intervals in the data. The direct
counter method suffers from low resolution, so heterodyne measurement methods are needed to characterize
high-performance clocks and oscillators. These methods are described in reference 2.3.6 a). The error
considerations for these time-domain measurements are given below. Special care should be taken to isolate
the oscillator under test from the measurement system. Inadequate isolation can lead to locking of the
oscillator under test to the reference.

a) Aliasing effects.

b) Quantization uncertainty associated with digitization.

c) Effects of dead time in the measurement process.

d) Noise associated with the reference oscillator.

e) Distortion of results by the presence of “bright” lines in the spectrum.

f) Effects of hardware bandwidths.

2.3.5.4.2 The calibration laboratory should indicate the biases and uncertainties for each of the six errors for
every measurement.

NIST Handbook 150-2C 18 September 2001


NIST HANDBOOK 150-2C

National
Voluntary.
Laboratory
Accreditation
Program

Calibration
Laboratories

Technical Guide

for

Time and Frequency
Measurements

C. Douglas Faison and
Carroll S. Brickenkamp, Editors

September 2001

U.S. Department of Commerce
Donald L. Evans, Secretary

Technology Administration
Karen H. Brown, Acting Under Secretary for
Technology

National Institute of Standards and Technology
Karen Brown, Acting Director


of customer laboratories and/or if decided by NVLAP to be in the best interest of the Calibration
Laboratories program.

The laboratory is requested to indicate on the Fields of Calibration/Parameters Selection List the
parameter(s) for which accreditation is desired, along with appropriate ranges and uncertainties. There is
also provision for an applicant laboratory to request accreditation for parameters not currently listed on the
Selection List, or for accreditation of the quality system employed for assuring Measurement and Test
Equipment (M & TE) used in support of product certification. Request for accreditation of quality assurance
systems for M & TE will be treated as a separate field of calibration for the purpose of setting appropriate
fees. Once a laboratory meets all the requirements for accreditation for the Fields of Calibration/Parameters
Selection List, this information will become the basis for the Scope of Accreditation document.

1.6.3 Checklists

Checklists enable assessors to document the assessment of the laboratory against the NVLAP requirements
found in NIST Handbook 150. The NVLAP Calibration Laboratories Accreditation Program incorporates
the NVLAP General Operations Checklist. The questions are applicable to evaluating a laboratory's ability
to operate a calibration program, and address factors such as the laboratory's organization, management, and
quality system in addition to its calibration competency.

The NVLAP General Operations Checklist is numbered to correspond to the requirements in NIST
Handbook 150. Comment sheets are used by the assessor to explain deficiencies noted on the checklist.
Additionally, the assessor may use the sheets to make comments on aspects of the laboratory's performance
other than deficiencies.

1.7 Assessing and evaluating a laboratory

1.7.1 On-site assessment

1.7.1.1. The NVLAP lead assessor will schedule with the laboratory the date for on-site evaluation, and
will request the quality manual and documented quality and calibration procedures in advance of the visit
to reduce time spent at the laboratory; such materials will be returned by the assessor. NVLAP and the
assessor will protect the confidentiality of the materials and information provided. The laboratory should
be prepared to conduct routine calibrations, have equipment in good working order, and be ready for
examination according to the guidance contained in this handbook, the requirements identified in NIST
Handbook 150, and the laboratory's quality manual. The assessor will need time and work space to complete
assessment documentation while at the laboratory, and will discuss these needs at the opening meeting of
the on-site assessment.

1.7.1.2. NVLAP technical assessors are provided with the NVLAP General Operations Checklist to help
ensure the completeness, objectivity, and uniformity of the on-site assessment.

1.7.1.3. When accreditation has been requested for a considerable number of fields of calibration and
parameters, the assessment may range from observing calibrations in progress, requiring repeat
measurements on completed calibrations, to listening to laboratory staff describe the calibration process.
The depth into which the assessor performs the assessment depends on the number of fields of calibration
and associated parameters for which accreditation is requested and the time required to perform a given
calibration.

NIST Handbook 150-2C 5 September 2001


1.5 Definitions

Definitions found in NIST Handbook 150 apply, but may be interpreted differently or stated differently,
when necessary to amplify or clarify the meaning of specific words or phrases as they apply to specific
technical criteria.

1.5.1 Proficiency Testing: Determination of laboratory performance by means of comparing and
evaluating calibrations or tests on the same or similar items or materials by two or more laboratories in
accordance with predetermined conditions. For the NVLAP Calibration Laboratories Accreditation Program,
this entails using a transport standard as a measurement artifact, sending it to applicant laboratories to be
measured, and then comparing the applicant's results to those of a reference laboratory on the same artifact.

1.5.2 Traceability: Property of the result of a measurement or the value of a standard whereby it can
be related to stated references, usually national or international standards, through an unbroken chain of
comparisons all having stated uncertainties. [VIM:1993, 6.10]

A single measurement intercomparison is sufficient to establish uncertainty relationships only over a limited
time interval (see reference 1.4.1.(k)); internal measurement assurance (see reference 1.4.1(1)), using control
(check) standards, is required to fully demonstrate that uncertainties remain within stated levels over time.
For the purposes of demonstrating traceability for NVLAP accreditation, a laboratory must demonstrate not
only that there is an unbroken chain of comparisons to national standards, but also that this chain is supported
by appropriate uncertainties, measurement assurance, continuous standard maintenance, proper calibration
procedures, and handling of standards. In this way, traceability is related to these other areas of calibration.

1.6 NVLAP documentation
1.6.1 Accreditation documents

Laboratories granted NVLAP accreditation are provided with two documents: Scope of Accreditation and
Certificate of Accreditation.

The Scope of Accreditation lists the "Best Uncertainty" that an accredited laboratory can provide for a given
range or nominal value within a given parameter of measurement. This "Best Uncertainty" is an statement
of the smallest uncertainty that a laboratory has been assessed as capable of providing for that particular
range ornominal value. The actual reported value of uncertainty for any particular measurement service that
the accredited laboratory provides under its scope may vary depending on such contributors as the statistics
of the test and uncertainties associated with the device under test.

1.6.2 Fields of calibration/parameters selection list

The Calibration Laboratories program encompasses eight fields of physical metrology calibration, with
multiple parameters under each field. Each field is covered by a separate handbook (NIST Handbooks 150-
2A through 150-2H). (Fields of accreditation under Chemical Calibration are covered by separate
handbooks.) Depending on the extent of its calibration capabilities, a laboratory may seek accreditation to
all or only selected fields and parameters within the scope of the program. The fields of calibration and their
related parameters are given on the Fields of Calibration and Parameters Selection List, which is provided
toa laboratory seeking accreditation as part of the NVLAP application package for the program. Additional
fields of calibration and/or parameters may be added to the Calibration Laboratories program upon request

NIST Handbook 150-2C 4 September 2001


1 General information

1.1 Purpose

The purpose of this handbook is to amplify the general requirements for accreditation by NVLAP of
calibration laboratories in the area of time and frequency measurements covered by the Calibration
Laboratories Program. It complements and supplements the NVLAP programmatic procedures and general
requirements found in NIST Handbook 150, NVLAP Procedures and General Requirements. The interpretive
comments and additional guidelines contained in this handbook make the general NVLAP criteria
specifically applicable to the Calibration Laboratories Program.

This handbook does not contain the general requirements for accreditation, which are listed in NIST
Handbook 150, but rather provides guidelines for good calibration laboratory practices, which may be useful
in achieving accreditation.

1.2 Organization of handbook

The handbook is organized in two sections. The first section provides additional explanations to the general
procedures and requirements contained in NIST Handbook 150. The second section provides details and
guidance very specific for time and frequency calibration laboratories.

1.3 Description of Calibration Laboratories Accreditation Program

On May 18, 1992, as a result of the petition and public notice process, the Director of the National Institute
of Standards and Technology published in the Federal Register a notice of intent to develop the Calibration
Laboratories Accreditation Program under the procedures of the National Voluntary Laboratory
Accreditation Program . On June 2, 1994, the procedures and general requirements under which NVLAP
operates, Title 15, Part 285 of the U.S. Code of Federal Regulations (CFR), were revised to:

a) expand the procedures beyond testing laboratories to include accreditation of calibration laboratories,

b) update the procedures to ensure compatibility with generally accepted conformity assurance and
conformity assessment concepts,

c) incorporate international changes, especially with relevant International Organization for
Standardization/International Electrotechnical Commission (ISO/IEC) documents (e.g., ISO/IEC Guides
25 (now ISO/IEC 17025:1999), 38, 43, and 58, and the ISO 9000 series), and

d) facilitate and promote acceptance of the calibration and test results between countries to avoid barriers
to trade.

Calibration laboratory accreditation is offered in eight fields of physical metrology calibration covering a
wide variety of parameters and includes accreditation in multifunction measuring and test equipment
calibrations. Specific requirements and criteria have been established for determining laboratory
qualifications for accreditation following prescribed NVLAP procedures. The criteria address quality
systems, staff, facilities and equipment, test and calibration methods and procedures, manuals, records, and
calibration/certification reports.

NIST Handbook 150-2C 1 September 2001


2.2.6 NIST Frequency Measurement and Analysis Service (FMAS)

NIST operates a Frequency Measurement and Analysis Service (FMAS) ona subscription basis. Subscribers
to the FMAS pay a monthly fee to NIST, and in turn receive the measurement hardware and software needed
to make frequency calibrations traceable to NIST at an uncertainty of 2 x 10°. The FMAS uses GPS signals
as a reference frequency. Each subscriber’s measurement system is linked to NIST by a telephone modem
or Internet connection, and NIST personnel monitor all measurements from Boulder, Colorado. A monthly
calibration report is sent to each subscriber. The report includes a statement of uncertainty for the device
under test relative to the NIST national frequency standard.

The FMAS can benefit NVLAP customers seeking accreditation in the frequency calibration field. In some
cases, it can eliminate or reduce the need for proficiency testing (and its associated fees) in the frequency
calibration area.

2.2.7 References

a) The NIST Time and Frequency Division web site provides information about the current status of the
NIST time and frequency services. It also includes GPS, WWVB, and LORAN-C monitoring data.
Please see: http://www.boulder.nist.gov/timefreq.

b)_ R.E. Beehler and M. A. Lombardi, “NIST Time and Frequency Services,” NIST Special Publication
432, June 1991 (new version is in preparation).

c) M. A. Lombardi, “Time and Frequency Measurement,” Section II of The Measurement,
Instrumentation, and Sensors Handbook, CRC Press: Florida, 1999.

d) M.A.Lombardi, “Traceability in Time and Frequency Metrology,” Cal Lab: The International Journal
of Metrology, pp. 33-40, September-October 1999.

2.3 Spectral purity calibrations
2.3.1 Scope

This section describes the criteria under which a calibration laboratory should operate to be NVLAP
accredited for calibrations involving spectral purity in the frequency domain (amplitude modulation (AM)
and phase modulation (PM) noise of signals) and the time-domain characterization of oscillators. The
specifications for these quantities should be the internationally accepted ones described in reference 2.2.7 a).

2.3.2 Statistical process control

The measurement of PM and AM noise in the frequency domain involves more potential problems because
the measurement laboratory does not usually have sufficient access to self-calibration or self-testing methods
that ensure reliability. For this reason, the calibration laboratory should participate in a round-robin
evaluation of performance of standard artifacts for both PM and AM noise measurements. These
performance tests should include a determination of the system noise floor and accuracy of the phase and/or
amplitude noise level. Once established and certified, time-domain characterization systems are more robust,
providing sufficient control and self-testing of measurement conditions. These measurements thus require

NIST Handbook 150-2C 15 September 2001


Preface

The Calibration Laboratories Accreditation Program was developed by the National Voluntary Laboratory
Accreditation Program (NVLAP) at NIST as a result of interest from private industry and at the request of
the National Conference of Standards Laboratories. The goal of the program is to provide a means by which
calibration laboratories can be assessed for competency. This voluntary program is not designed to serve as
a means of imposing specific calibration procedures or minimum uncertainties on applicant laboratories;
instead, the program allows for all scientifically valid calibration schemes and requires that laboratories
derive and document their measurement uncertainties.

To accomplish this goal, NVLAP employs technical experts, on a contract basis, to serve as assessors in each
of the following eight fields of physical metrology calibration:

+ electromagnetic dc/low frequency,

+ electromagnetic rf/microwave frequency,
+ time and frequency,

* ionizing radiation,

* optical radiation,

* dimensional,

* mechanical, and

* thermodynamics.

NIST Handbooks 150-2A through 150-2H are technical guides for the accreditation of calibration
laboratories, with each handbook corresponding to one of the eight fields of physical metrology calibration.
They are intended for information and use by:

* NVLAP technical experts in assessing laboratories,

+ staff of accredited laboratories,

+ those laboratories seeking accreditation,

* other laboratory accreditation systems,

* users of laboratory services, and

+ others needing information on the requirements and guidelines for accreditation under the NVLAP
Calibration Laboratories Accreditation Program.

NOTE The Calibration Laboratories Accreditation Program has been expanded to cover chemical calibration for
the providers of proficiency testing and certifiers of spectrophotometric NTRMs. (See NIST Handbooks 150-19 and
150-21.) Other NVLAP handbooks in the chemical calibration area are expected in the future.

The assessor uses NIST Handbook 150, NVLAP Procedures and General Requirements, and the appropriate
guides (NIST Handbooks 150-2A through 150-2H) to validate that a laboratory is capable of performing
calibrations within the laboratory’s stated uncertainties. These technical guides and other relevant technical
information support assessors in their assessments of laboratories. Along with inspecting the facilities,
documentation, equipment, and personnel, the assessor can witness a calibration, have an item recalibrated,
and/or examine the results of measurement assurance programs and round-robins to collect objective
evidence.

NIST Handbooks 150-2A through 150-2H supplement NIST Handbook 150, which contains Title 15 of the
U.S. Code of Federal Regulations (CFR) Part 285 plus all general NVLAP procedures, criteria, and policies.
The criteria in NIST Handbook 150 originally encompassed the requirements of ISO/IEC Guide 25:1990 and


To pass the proficiency test, the applicant laboratory must have a value for E,,,,,,4) less than 1 (i.€., E, ona < 1).
The results may be plotted graphically, with lines representing the limits of uncertainty of the measurements.
The anonymity of each applicant laboratory will always be preserved.

1.7.2.6 Scheduling and handling

Proficiency testing is scheduled by NVLAP-designated reference laboratories. These sites are NIST
laboratories or NVLAP-accredited laboratories that have been found to have the ability to perform the
required proficiency tests to an uncertainty level appropriate for the laboratories they evaluate. The
proficiency test is scheduled independently and not to correspond with the on-site visit. Applicant
laboratories are notified in advance as to the approximate arrival time of the measurement artifact.
Instructions for performing the test, reporting the results, communicating with the reference laboratory, and
shipping are included along with the artifact as part of the proficiency test package. Applicant laboratories
are instructed to perform all required measurements within a reasonable time and are told where to ship the
artifacts once the testing has been completed.

1.7.2.7 Notification of results

NVLAP notifies each laboratory of its own results in a proficiency test. If a laboratory has been evaluated
prior to the completion of the proficiency test, the status of that laboratory’s accreditation is contingent upon
successful completion of proficiency testing. The laboratory's accreditation status may be changed to reflect
a partial accreditation, or may be completely suspended pending demonstration of the laboratory's ability
to successfully complete the proficiency test at a later date.

1.7.3 Traceability
1.7.3.1 Establishing traceability

Laboratories must establish an unbroken chain of comparisons leading to the appropriate international or
national standard, such that the uncertainties of the comparisons support the level of uncertainty that the
laboratory gives to its customers. Generally speaking, the uncertainties of the comparisons increase as they
move from a higher (international or national level) to a lower level standard. This uncertainty chain is the
evidence of traceability and must be documented accordingly. Traceability does not simply mean having
standards calibrated at the national laboratory, but must consider how a measurement, with its corresponding
uncertainty, is transferred from the national level to the calibration laboratory's customers.

1.7.3.2 Considerations in determining traceability

Without measurement assurance, one cannot be reasonably certain that the comparisons have been
transferred properly to the laboratory's customers. The measurement process itself must be verified to be
in control over time. Therefore, traceability is not a static concept that, once established, may be ignored,
but it is dynamic. Process control exercised in each calibration provides the assurance that a valid transfer
of the international or national standard has taken place. This assurance may be accomplished through the
use of tools such as check standards and control charts. Also, the laboratory's primary standards must be
maintained in such a way as to verify their integrity. Examples of this may be having more than one primary
standard to use for intercomparisons, monitoring the primary standard with a check or working standard
(looking for changes), and verifying a primary standard on a well-characterized measurement/calibration
system. Using scientifically sound measurement procedures to transfer the primary standard value to the
working level and the customer's item is essential to establishing traceability. If the procedure itself yields
the wrong result, there is no way the laboratory can perform a calibration traceable to the international or

NIST Handbook 150-2C 8 September 2001


2 Criteria for accreditation
2.1 Introduction

2.11 Applicant laboratories are assessed using the requirements in NIST Handbook 150, NVLAP
Procedures and General Requirements. This guide, NIST Handbook 150-2C, was developed from a NIST
measurement laboratory perspective and provides examples and guidelines, not requirements, to assessors
and interested calibration laboratories on good laboratory practices and recommended standards. Therefore,
the guide language reflects this philosophy through the use of "shoulds" instead of "shalls" (along with other
less prescriptive language) when describing criteria. The requirements presented here are not absolute since
specific requirements depend on the measurement uncertainty for which an applicant laboratory wishes to
be accredited. This is a business decision for each laboratory and beyond the scope of NVLAP. Simply
stated, to be accredited, an applicant laboratory must have a quality system and be able to prove (and
document) that it is capable of doing what it says it does (i.e., correctly calibrate to a stated uncertainty)
within the framework of NIST Handbook 150. Accreditation will be granted, and therefore may be
referenced in calibration reports, etc., only for those specific parameters, ranges and uncertainties using
calibration methods and procedures for which a laboratory has been evaluated. Calibrations performed by
a laboratory using methods and procedures not considered appropriate for the level of measurements being
made, and which have not been evaluated by the accreditation process, are outside the scope of accreditation
and may not be referenced as "accredited" calibrations on calibration reports, etc.

2.1.2 Section 2.2 provides specific interpretations of the NIST Handbook 150 criteria for time and
frequency calibrations. Section 2.3 provides specific calibration guidelines for spectral purity
measurements. This guide is dynamic in that new parameters may be added and existing criteria updated and
improved.

2.2 Time and frequency
2.2.1 Scope

This section describes the criteria under which a laboratory should operate to be NVLAP-accredited as
competent to perform measurements in the area of time and frequency. It also describes the procedure for
performing frequency calibrations traceable to NIST and for establishing a time reference. Since any on-site
frequency calibration requires an external reference signal, laboratories are encouraged to obtain the
publications cited at the end of this section. These publications provide detailed information about the
available reference signals, and contain information about the receiving equipment and calibration methods.
Users are also invited to contact the Time and Frequency Division of NIST for assistance.

2.2.2 Accommodation and environment

2.2.2.1 A calibration service dependent on radio or satellite signals must mount an outdoor antenna. The
laboratory should be arranged so that the receiver is reasonably close to the antenna. The receiver should
be isolated from sources of radio interference, and should have a back-up power supply for operation during
power failures.

2.2.2.2 The laboratory's main standard (oscillator) should be located where it will not be disturbed. It
should run continuously (24 h/d) and should have a back-up power supply so that it remains on during local

NIST Handbook 150-2C 10 September 2001


Contents

Preface... . cece c cece cece eee ee eee teen nent eee e tenet eee teen teens eee entee v
Acknowledgments ..........c:ccceeececee cece cece e sneer ence sees eeeeseeeees vii
SUMIMALY 00... cece ee eee eee ener eee e eee e eee tenet ete eeeeee eens viii
1 General information ........... 6. cece cece eee cece renee eee e nett eeeeeeeees 1
1.1 Purpose............6- 1
1.2 Organization of handbook .. 1
1.3. Description of Calibration Laboratories Accreditation Program veel
1.4 References ...... ccc cece eee e cence rene eee eeeseeneeeneees weed
1.5 Definitions ........ 4
1.6 NVLAP documentation ............. weed
1.7 Assessing and evaluating a laboratory ........... cece cece eee cent eeeees 5
2 Criteria for accreditation ........... 0. cece eee c cence eee n eee eeneeeeees 10
2.1 Introduction ....... -10
2.2 Time and frequency ... . --10
2.3 Spectral purity calibrations beeen ence eee e een nett eee e eee eeeneeeeees 15

iii


2.3.5.5 Conversion from the frequency domain to the time domain

Good time-domain characterization can be obtained through proper conversion of frequency-domain data
to the time domain (see reference 2.3.6 a)). The time-domain averaging time t in such a conversion should
be limited to 1/(2zf) where f is the lowest Fourier frequency where reliable frequency-domain data are
available. Reverse conversion from the time domain to the frequency domain is not reliable and should not
be used.

2.3.6 References

a) D.B. Sullivan, D. W. Allan, D. A. Howe, and F. L. Walls, Eds., “Characterization of Clocks and
Oscillators,” NIST Tech. Note 1337, March 1990. (Available from the Superintendent of Documents,
U.S. Government Printing Office, Washington, DC 20402-9325.)

b) F.L. Walls, “Reducing Errors, Complexity, and Measurement Time of PM Noise Measurements,”
Proceedings of the 1993 IEEE Frequency Control Symposium, Salt Lake City, UT, June 2-4, 1993,
IEEE Catalogue No. 93CH3244-1.

c) L.M. Nelson, C. W. Nelson and F. L. Walls, “Relationship of AM-to-PM Noise in Selected RF
Oscillators,” Proceedings of the 1993 IEEE Frequency Control Symposium, Salt Lake City, UT, June
2-4, 1993, IEEE Catalogue No. 93CH3244-1.

d) F.L. Walls and E.S. Ferre-Pikal, “Measurement of Frequency, Phase Noise and Amplitude
Noise,” Wiley Encyclopedia of Electrical and Electronics Engineering, Vol. 12, pp. 459-473,
John Wiley & Sons: New York, 1999.

NIST Handbook 150-2C 19 September 2001


National Institute of Standards and Technology
NIST Handbook 150-2C

26 pages (September 2061)

CODEN: NIHAE2

U.S. GOVERNMENT PRINTING OFFICE
WASHINGTON: 2001

For sale by the Superintendent of Documents
U.S. Government Printing Office
Washington, DC 20402-9325

NVLAP AND THE NVLAP LOGO

The term NVLAP (represented by the NVLAP logo)
is a federally registered certification mark of the
National Institute of Standards and Technology and
the federal government, who retain exclusive rights
to control the use thereof. Permission to use the term
and/or logo is granted to NVLAP-accredited
laboratories for the limited purposes of announcing
their accredited status, and for use on reports that
describe only testing and calibration within the scope
of accreditation. NIST reserves the right to control
the quality of the use of the term NVLAP and of the
logo itself.


1.7.1.4 The assessor, or the assessment team, does the following during a typical on-site assessment:

a) Conducts an entry briefing with the laboratory manager to explain the purpose of the on-site visit and
to discuss the schedule for the day(s). At the discretion of the laboratory manager, other staff may
attend the briefing.

b) Reviews quality system manual, equipment and maintenance records, record-keeping procedures,
laboratory calibration reports, and personnel competency records. At least one laboratory staff member
must be available to answer questions; however, the assessor may wish to review the documents alone.
The assessor(s) does not usually ask to take any laboratory documents with him/her, and previously
supplied documents will be returned.

c) Physically examines equipment and facilities, observes the demonstration of selected procedures by
appropriate personnel assigned to perform calibrations, and interviews the personnel. The
demonstrations must include preparation for calibration of devices, and the setup and use of measuring
and test equipment, standards and systems.

d) Holds an exit briefing with the laboratory manager and staff to discuss the assessment findings.
Deficiencies are discussed and resolutions may be mutually agreed upon. Items that must be addressed
before accreditation can be granted are emphasized, and outstanding deficiencies require response to
NVLAP within 30 days. Items that have been corrected during the on-site and any recommendations
are specially noted.

e) Completes an On-site Assessment Report, as part of the exit briefing, summarizing the findings. The
assessor(s) attaches copies of the completed checklists to this report during the exit briefing. The report
is signed by the lead assessor and the laboratory's Authorized Representative to acknowledge the
discussion. This signature does not necessarily indicate agreement; challenge(s) may be made through
NVLAP. A copy is given to the representative for retention. All observations made by the NVLAP
assessor are held in the strictest confidence.

1.7.2 Proficiency testing
1.7.2.1 Background

Once the quality system review and on-site assessment steps have been satisfactorily completed, it is
necessary to gather another set of data points to aid in deciding whether or not the applicant laboratory is
competent to perform calibrations within the fields of interest to the uncertainties claimed. In the eight fields
of calibration covered by Handbooks 150-2A through 150-2H, there are approximately 85 parameters of
interest. Under most parameters there are several subsets, referred to as ranges. For example, in the time
and frequency field, the typical input frequency parameters for most laboratories fall in the range from 1 Hz
to 100 MHz, with 1, 5, 10, 100 MHz the most common test frequencies. Laboratories who wish can specify
higher input frequencies as parameters. In view of the many possible ranges, proficiency testing could be
conducted in a multitude of areas. NVLAP reserves the right to test by sampling in any area; hence, applicant
laboratories must be prepared, with reasonable notice, to demonstrate proficiency in any of a number of
parameters.

1.7.2.2 Proficiency testing vs. measurement assurance

There is an important difference between proficiency testing and measurement assurance. The objective of
proficiency testing is to determine through a measurement process that the laboratory's measurement

NIST Handbook 150-2C 6 September 2001


national standard. Handling the laboratory's standards affects the measurement process, and therefore the
ability to transfer the standard's value to the customer. Examples of handling problems are dirty or
improperly cleaned standards, maintaining standards in an improper environment, not maintaining custody
and security, and improper handling of standards during the measurement process.

1.7.3.3. Relationship to existing standards

The above discussion illustrates how traceability is dependent on many aspects of the measurement process
and therefore must be considered in all phases of calibration. It is not just coincidental that the factors
addressed above are miain topics of concern in ISO/IEC 17025:1999.

1.7.4 Uncertainty

NVLAP recognizes the methodology for determining uncertainty as described in the Guide to the Expression
of Uncertainty in Measurement, published by ISO. To be NVLAP-accredited, a laboratory must document
the derivation of the uncertainties that it reports to its customers. These uncertainties will appear on the
scope issued to each accredited laboratory to an accuracy appropriate to the standards, procedures, and
measuring devices used.

NIST Handbook 150-2C 9 September 2001