PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 1 of 38

PT Standard Operating Procedure 1 

for

OWM PT Planning, Operating, Analyzing, and Reporting PT Results

Purpose

The purpose of this Standard Operating Procedure (SOP) is to provide a procedure for planning, 

operating, analyzing, reporting, and following up on proficiency tests and their reports as a part of 

the NIST Office of Weights and Measures, Proficiency Testing (PT) Program. This SOP provides 

an overview of how the various types of proficiency tests (PTs) are conducted and an explanation 

of how laboratory performance is evaluated. This SOP can be used for any of the interlaboratory 

comparisons operated through the OWM PT Program, whether for use as official proficiency tests 

or not. It may also be used for training, completion of Laboratory Auditing Program problems, 

method validation activities, or other unique interlaboratory or intralaboratory comparisons. PTs 

are usually coordinated through the Regional Measurement Assurance Program (RMAP) regions 

or nationally, either by OWM staff or by a national PT Coordinator.

Overview of Required Components

The items identified in this section are required components of the OWM PT program to ensure 

compliance with the requirements of ISO/IEC 17043:2023. Specific guidance is identified during 

the planning and analysis procedure in Sect. 3.

2.1. Scope

This procedure is not limited to official OWM or accreditation body proficiency tests (PTs) 

and may be used for special interlaboratory comparisons (ILCs), intralaboratory comparisons, 

mini-measurement assurance program (Mini-MAP) evaluations, method validation, or any 

other suitable and applicable purpose using the OWM PT planning and analysis tools. 

2.2. PT Process Policies, Procedures, and Tools 

All of the National Institute of Standards and Technology (NIST) Office of Weights and 

Measures (OWM) PT quality management system (QMS) (including: NISTIR 7082, 

"Proficiency Test Policy and Plan" and NISTIR 7214 “Office of Weights and Measures Quality 

Manual for Proficiency Testing and Interlaboratory Comparisons”) policies, procedures, and 

tools shall be used in addition to the following items.

2.2.1. The PT Plan template spreadsheet shall be used for PT planning to ensure that a 

consistently developed and documented PT plan is in place that includes PT 

objectives, purposes, and schemes including the items listed in the following 

sections and noting if there are any reasons for exclusions.

2.2.2. All PT participants shall meet the participation criteria and shall be qualified or 

approved to participate at the planned level of work during the planning stage and 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 2 of 38

all participants are required to waive anonymity and comply with confidentiality 

restrictions (per NISTIR 7082); 

2.2.3. All participants and OWM Laboratory Metrology Program staff shall review PT 

Plans and OWM Laboratory staff shall approve each PT Plan prior to beginning 

the PT;

2.2.4. All participants shall submit official calibration certificates and any requested 

supplemental forms or information determined as part of the PT Plan; 

2.2.5. The draft PT analysis may be developed by the PT Analyst or NIST OWM staff 

and shall use the OWM PT Analysis template spreadsheet(s). PT participants must 

review and confirm data entry accuracy during the conduct or analysis of the PT;

and

2.2.6. The final PT Analysis and PT Report may be developed by the PT Analyst but 

shall be reviewed and approved by the NIST OWM, Laboratory Metrology 

Program staff prior to issuance to PT participants.

2.3. Measurement Parameters, Quantities, and Ranges to be Determined 

This procedure may be used for any measurement parameter, range, or level of uncertainty. 

Specific parameters, ranges, and uncertainties are determined during the PT planning stage. 

2.4. Selection of the Item(s) to be Calibrated

Suitable history, stability, resolution, availability of an independent reference value, sufficient 

accuracy, and sufficiently small uncertainties for use with the reference value are critical for 

all PTs. The selection of a suitable reference value with metrological traceability is addressed 

in Sect. 3.3.5. Items to be calibrated in an OWM PT generally reflect the same type and range 

of items submitted for routine calibrations among participating laboratories. 

OWM maintains an inventory of standards that are regularly used throughout the U.S. and 

Regional Measurement Assurance Programs. In general, items in circulation have 

demonstrated adequate stability, homogeneity (where applicable), and robustness for the 

intended applications. New standards introduced into the OWM PT inventory receive extra 

assessments before, during, and after use as needed. Items belonging to participants may 

occasionally be used in a PT if and when additional historical data (e.g., from calibration 

history and control chart data) are available for evaluating the reference value and its stability 

throughout the PT. 

Selection of the standards (PT artifacts) requires special consideration when they are not part 

of the NIST OWM inventory and especially when there are a small number of participants. 

2.5. Participant and Laboratory Qualifications 

All participating laboratories shall comply with requirements for participation published in 

NISTIR 7082, Policy and Plan, NIST Handbook 143, Program Handbook, and with ISO/IEC 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 3 of 38

17025:2017, General Requirements for the Competence of Testing and Calibration 

Laboratories through recognition by the NIST Office of Weights and Measures or through 

accreditation by an accreditation body (AB) that is an ILAC Signatory and be members of one 

of the Regional Measurement Assurance Program regions. Laboratories that are not recognized 

or accredited are expected to comply with NIST Handbook 143 and ISO/IEC 17025:2017 

standard to ensure integrity of the PT scheme, PT items, and submission of compliant 

calibration certificates.

All participants shall have adequate training prior to participating in PTs to ensure that 

standards are not damaged, contaminated, or degraded and that specified procedures are 

followed. Formal and informal training, qualifications, and experience shall be documented 

within OWM for each participant. (OWM Training Records and transcripts of the prospective 

PT participant are used when available). Additional records may be requested from potential 

participants to validate compliance with the participant qualification requirements. PT policies 

regarding training and participation shall be followed; exceptions for participants who have 

not completed applicable or recognized OWM training shall be submitted to OWM for 

approval and records will be retained in OWM. Exceptions will be made for State weights and 

measures laboratories where participation is required to complete Laboratory Auditing 

Program (LAP) problems and/or laboratory recognition. However, introductory training or on 

the job (OJT) training records for the measurement parameter shall be available to ensure the 

integrity of the PT.

State laboratories and each Approved Signatory are required to comply with the NISTIR 7082 

Policy and Plan for regular participation in proficiency tests in all areas of their Scope on a 

periodic basis (where practical and available). A five-year interval is specified for the regional 

PT plans, as PTs are planned and available either nationally, or within the Regional 

Measurement Assurance Program (RMAP). Failure to participate when PTs are available on a 

regional basis may impact laboratory recognition or accreditation (see NIST Handbook 143, 

Program Handbook and accreditation body policies and requirements). Laboratories that are 

not weights and measures laboratories recognized through OWM have the option to participate 

or not (as long as they meet all participation requirements). 

Special PT arrangements may be made for individuals in the State weights and measures 

laboratories to participate in a unique, non-regional PT, to assist laboratories in regaining 

signatory approval status for staff or laboratory recognition. Special LAP problem PTs will 

only be coordinated when the RMAP-coordinated PTs are not adequate/timely enough.

2.6. Equipment, Including Technical Performance Requirements

All participants shall have suitable equipment, reference standards, and suitably small 

uncertainties for the parameters or levels of calibration in question. Laboratories shall have 

evidence of metrological traceability to the SI for all participating measurements to meet the 

most basic objectives of traceability in OWM PTs. In general, the laboratory will already have 

the measurement parameter, range, and uncertainty (Calibration and Measurement Capability, 

CMC) on their scope of recognition or accreditation. Approval is required from OWM for any 

laboratory that wishes to participate with the intent of adding measurement capabilities to their
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 4 of 38

laboratory scope. OWM approvals may include additional recommendations or requirements 

prior to participation. 

2.7. Laboratory Procedures 

Use of internationally or nationally published Standard Operating Procedures (SOPs) is 

required (where available for that parameter and scope). For compliance with the NIST OWM 

PT program, it is expected that NIST SOPs are used when available. NIST SOPs are normally 

specified as part of the PT Plan when available. Procedures shall be specified or referenced in 

the PT Plan. If specified procedures are not available (e.g., in the case of new technology or 

method validation efforts), or there are participants from laboratories that do not use OWM 

procedures, procedures shall be submitted for review upon request. A document review of nonNIST procedures may be included in the final PT Report. In the case of alternative procedures, 

comparable measurement uncertainties are required for equivalent statistical comparisons of 

data. 

OWM procedures include specifications for standards, environmental conditions, 

equilibration, methods, calculations, and uncertainty budget components that are to be 

followed by all participating laboratories even if they are using other procedures or laboratorydeveloped methods. In general, the PT Plan specifies that participants agree to use the same 

procedure(s) to avoid potential discrepancies from irregular measurement results or dissimilar

uncertainty values that challenge troubleshooting and PT analysis efforts. 

Proficiency Testing Procedure

There are four stages of each PT: 1) Planning, 2) Operation, 3) Analysis, and 4) Reporting with 

subtasks in each section. The PT Plan template spreadsheet includes an overall outline for planning 

and coordinating PTs with separate worksheets for planning phases identified as P1, P2, P3, and 

P4, operating phases O1 and O2, plus cover pages for the PT as worksheet R1. The PT Analysis 

template spreadsheet(s) is/are used to check the validity of data entry, data transfer, statistical 

analysis, and record participant performance evaluations. A combination of the reporting section, 

R1, of the PT Plan and the PT Analysis tools are used for reporting. An OWM Supplemental PT 

Report is used for explanations of the data, analysis, and performance assessments. The PT Plan 

template documents and includes the PT schemes, objectives, purposes, and identifies all 

information needed in the planning and operational stages of each PT. Note: A separate procedure 

(PT Standard Operating Procedure 2 for Operating a Mini-Measurement Assurance Program 

(Mini-MAP) is available for a “Mini Measurement Assurance Program” (MiniMAP) that can be 

used among two or three laboratories for small, or unique, applications and laboratory 

requirements. It also requires the use of the PT Plan template and PT Analysis template tools and 

additional statistical analysis of laboratory data to ensure statistical validity of the outcome 

analysis. 

PTs are usually coordinated and maintained within the Regional Measurement Assurance 

Programs (RMAPs). The RMAPs maintain 4-year or 5-year PT Plans which include annual 

reviews, inputs, and updates determined during planning sessions with OWM staff, PT 

coordinators, PT analysts, and participants. Laboratory participants are obligated to ensure that 

measurements on their Scope are addressed in the RMAP plans to the extent possible. National 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 5 of 38

PTs may be coordinated when there are not a significant number of laboratories within the RMAP 

regions to ensure an adequate number of participants for statistical validity of the PT scheme and 

data analysis.

3.1. Planning Stages

The PT Plan template spreadsheet follows the steps in this SOP and includes the PT Plan 

requirements. The PT Plan is usually initiated and completed by a PT Coordinator; it may be 

completed by OWM staff for national PTs who may also serve as a PT Coordinator. All PT 

Plans are to be reviewed and followed by PT participants and shall be submitted to OWM for 

final approval before proceeding with the PT scheme (reference NISTIR 7082). OWM will 

ensure that the PT is adequately defined, documented, technically and administratively 

appropriate. Updates to the PT Plan shall be approved by OWM and circulated to PT 

participants during the course of the PT operations as needed, especially if risks to the PT are 

identified, schedules are updated, the procedures for calibration or analysis are updated, the 

validity of PT result/data is suspect, or PT shipping and handling must be modified. 

Key sections of the PT Plan template include the following planning stages and worksheets, 

with each section having multiple steps:

• P1 – Organize the PT;

• P2 – Objectives and Details;

• P3 – Artifact and Shipping; and

• P4 – Addresses and Contacts.

3.1.1. Stage P1: Worksheet P1; Organize the PT

3.1.1.1. Define the Scope and Title: Parameter, Range, Uncertainty and PT Name 

During the P1 step, the need for the measurement parameter, range, and level of 

calibration or level of uncertainty to be expected for the PT are defined. The need and 

scope may be defined during the RMAP 4-year planning process as well. PTs are 

named based on the region, year of planned reporting, measurement parameter, and 

expected level of calibration and uncertainty. PT plans for the regions will sample 

different parameter ranges and units from year to year to meet the variable Scopes of 

the laboratories.

The PT Plan template includes a drop down list of options to define the parameter, 

range, and level of calibration (e.g., echelon or class) for the PT. The use of 

standardized drop-down choices helps ensure consistency among all PTs and is used 

to name the PT:

Region:

Reporting Year:

Parameter:
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 6 of 38

Unit System:

PT Sequence Number:

Range or Nominal:

For example, the following selection identifies the region, year, parameter/level of 

work, units, and PT number (to allow for multiple PTs meeting that criteria):

Region: MidMAP

Reporting Year: 2025

Parameter: Mass Echelon III

Unit System: Metric

PT Sequence Number: 01

Range or Nominal: 5 kg to 10 g

The PT name for this example is: “MidMAP-25-MIII-M-01”. The PT Plan template 

and the PT Analysis template tools shall indicate this name as part of the file name. 

3.1.1.2. Identify the PT Administrative Team

The personnel involved in the design, operation, and analysis of a PT are integral to the 

PT administrative team. This step includes the identifying the PT Coordinator(s), PT 

Analyst(s), and OWM staff Points of Contacts (responsible staff for oversight, trouble 

shooting, final analysis). Each PT may have observers who wish to learn about PT 

administrative actions. The PT may also include mentors who are experienced PT 

Coordinators who agree to assist a new PT Coordinator with the planning and 

coordination of a PT. The PT may also include mentor or observer PT Analysts who 

are responsible for the data entry and initial data analysis of the PT using the PT 

Analysis template. The responsible PT Administrative Team and their contact 

information are all identified during the planning stage. The roles of each administrator 

are defined below.

• PT Coordinator: The coordinator is one or more individuals with responsibility for 

organizing and managing all of the activities involved in the operation of a 

proficiency testing scheme. Coordinators document observations, feedback, and 

complaints related to the PT, and provide input to the analyst for drafting the final 

report. They do not provide reference value information to participants. 

• PT Analyst: The analyst provides initial data entry, analysis, conduct inquiries for 

problematic data, and may provide interim pass/fail feedback to participants. They 

do not provide reference value information to participants. They draft the PT 

analysis and report with input from the PT coordinator, confirm data entry with 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 7 of 38

participants, and submit the draft PT analysis and PT report to OWM for final 

review and approval.

• PT Mentor: The mentor is an experienced PT Coordinator or Analyst who provides 

one-on-one guidance to a coordinator or analyst with less experience. Having a 

separate mentor is optional for OWM PTs, otherwise the regional coordinator acts 

as the default mentor.

• Observer: The observer role allows a participant to observe how the coordinator 

and analyst plan, operate, analyze, and report a PT. This allows them to gain 

valuable experience before volunteering as a coordinator or analyst in the future. 

The observer must be an integrated part of all communications for the PT. Having 

an observer is optional for OWM PTs.

• Regional Coordinator: Each RMAP group is assigned at least one regional 

coordinator. This position requires an experienced and effective PT coordinator and 

analyst. They manage and document the PT participation planning schedule process 

for their RMAP. They mentor and assist, as needed, all the coordinators and 

analysts for the PTs in their group and make sure all PTs are kept on schedule to 

meet the PT plan. They serve as backup to take over a PT if the coordinator or 

analyst is no longer able to perform their duties. They regularly report to OWM and 

participant laboratories on the status of PTs in their region.

• Technical Point of Contact: may be a PT Coordinator, PT Analyst, mentor, or 

assistant: A NIST point of contact, coordinator or analyst provides guidance and 

support to the PT Administrative Team and PT participants. OWM staff approve 

the PT Plan before a PT may be started and finalize the PT analysis and evaluations 

included in the final PT Report. Some PT coordinators and PT analysts are reluctant 

to be critical of metrology peers when it is necessary to identify and address 

deficiencies in analyzing a PT and including evaluations and feedback in the final 

PT Report. The NIST coordinator and analyst provides any additional feedback 

necessary regarding the performance of participants and serves as the final 

arbitrator for all PTs performed under the OWM PT program guidelines.

• NIST (or other) Technical Advisor: The NIST technical advisor provides subject 

matter expert guidance and support for a PT. Having a technical advisor is based 

on the parameter and unique needs of the PT. This person may or may not assist 

the PT Coordinator, PT Analyst, or in the PT Final Report generation, but is 

available as a technical advisor if any issues arise during the PT that would benefit 

from an experienced technical staff member. This person may or may not be a 

member of the Office of Weights and Measures. For example, a member of the 

Statistical Engineering Division, the Mass and Force Group, a retired staff member, 

or contractor (if specified in their contract role) may meet the technical advisor role. 

Having a NIST Technical Advisor is optional for OWM PTs; although this role 

may be added after the PT Plan has been approved depending on the needs of the 

PT.

• Pivot Laboratory: If one participant will provide intermediate calibrations or 

beginning/ending values for the PT, they shall be designated during planning and 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 8 of 38

approved by OWM. The purpose of using a pivot laboratory shall be defined (most 

often for monitoring trends in the data but may be included to participate with a 

procedure that produces smaller uncertainties to assist in defining or validating the 

reference value(s)). Pivot laboratories are not necessarily considered “expert” 

laboratories for the purpose of identifying and selecting the assigned reference 

values.

3.1.1.3. Identify High Level Scheduling Overview (Critical Timelines)

Identify the planned completion dates, how long the standard(s) (artifacts) will be 

available and how long each lab will be given to complete the calibrations. The program 

goal is to complete each PT within a one-year period so that final reports can be issued 

at subsequent RMAP meetings. OWM may supply the coordinator information on 

when and how long PT artifacts are available, the due date for submitting a draft 

analysis and report to OWM, and any logistical considerations that must be considered.

3.1.1.4. Circulation Plans - Identify Laboratories, Participants and Training Details

and Set Initial Schedule for Review

Include a list of all laboratories and all participants on the planned scheduled list. All 

intended participants shall be included in the plan. All participants shall comply with 

Sect. 2.5, Participant and Laboratory Qualifications and OWM will verify training 

qualifications or oversight requirements as a part of reviewing and approving a PT Plan.

If a proposed non-legal metrology participant is not a regular attendee at the RMAP 

training or has not completed OWM training requirements, then Form 2 must be 

completed and submitted for that staff member. When a participant requires a 

completed PT to finish LAP problems, that shall be flagged on the scheduled list and 

in the PT Objectives section to enable OWM staff to obtain interim data and analyses 

if/as needed. Additions to this list once a PT has begun shall be approved by the NIST 

OWM Point of Contact prior to participation. When a petal scheme or opening and 

closing measurements will be made by a designated laboratory, the pivot laboratory 

shall be approved by OWM and listed multiple times in the schedule as applicable. 

The PT coordinator creates a preliminary schedule that is sent out for laboratories to 

identify any conflicts before the schedule is set. Special consideration is required when 

participants are required to transport the PT item(s) to other participants. Laboratories 

that have been historically slow at finishing PTs, who have planned downtimes or 

moves, or who have previously received warnings from OWM regarding schedule 

compliance, are scheduled at the end of the routing. This way, the PT can be finalized 

even if that lab keeps the artifact too long, doesn’t complete measurements, or submit 

a calibration certificate.

A planned PT must have a balance between enough participants to enable statistical 

validity of the data analysis and too many participants that would prevent completion 

during the planned PT scheme and scheduled dates. In some cases, too few participants 

will lead to merging plans with other regions up to and including a national PT instead 

of a regional PT. 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 9 of 38

Laboratories that “hard schedule” their work or have planned any extended periods of 

time that temporarily prohibit them from making measurements (e.g., vacations, 

standard calibrations, new laboratory or equipment installations, holiday closures, etc.), 

communicate this with the coordinator before the schedule is created or as soon as they 

are aware.

3.1.2. Stage P2: Worksheet P2; Objectives and Details

3.1.2.1. Unique Considerations

Identify any unusual or unique issues associated with the PT that is not a standard 

calibration approach defined in the applicable specifications and tolerances for the 

standard to be used or by the procedures that will be designated. Any unique calibration 

or test methods or issues related to the standards to be used or calibrated shall be 

identified, determined, documented, and approved during the planning stages. 

3.1.2.2. Establish PT Objectives

Select the PT objectives by checking the applicable selection boxes for each of the 

following options. Clearly identify any additional objectives entered in the “other”

category. The most common items are selected by default in the PT Plan template. 

When a participant was identified in the P1 worksheet as needing LAP problems, LAP 

problems need to be selected on this checklist as well. The following are specific 

objectives listed in the PT plan template: 

• Demonstration of Competency for Accreditation or Recognition;

• Validation of Expanded Uncertainties;

• Evaluation of Calibration Certificates;

• Method Validation (New Methods/Procedures ONLY);

• Laboratory Auditing Program (LAP) problems (For State employees/participants);

• Demonstration of Effective Corrective Action from previously failed PTs;

• Conformance or Suitability Evaluation of Artifact;

• Identifying Artifact Characteristics (stability, material, density, etc.);

• Customer Service & Contract Review (e.g., timeliness, following instructions); 

and/or 

• Other (requires explicit description). 

3.1.2.3. Identify PT Evaluation Methods (Performance Statistics)

Select the planned PT Evaluation Methods by checking the applicable selection boxes 

for each of the following options. The default evaluations will include an assessment 

of the overall statistics to identify any values outside the initial two standard deviation 

limits for the PT including all data and observations which is used for the analysis of 

the reference value with metrological traceability (See 3.3 Analysis). The difference 

(bias), normalized error, En, the normalized precision, Pn , the Z score, and 2 standard 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 10 of 38

deviation evaluations are selected by default. When other evaluation methods are used, 

they shall be explicitly defined when selected. Difference from Reference Value (Bias);

• Normalized Error (En);

• Precision Assessment (Pn);

• Z Score;

• 2 Standard Deviations;

• SOP 1 Calibration Certificate Review Checklist (qualitative review for ISO/IEC 

17025 compliance);

• SP 811 Check List for Reviewing Manuscripts (qualitative review); and/or

• Other.

One example of another method that might be used is that of a Youden Plot analysis 

(Using the PT Analysis Youden template) that also includes an evaluation of random 

and systematic components of variability. The evaluation methods are further defined 

in the PT Analysis section of this procedure (Sect. 3.3) and uses the PT Analysis 

template to ensure uniform and consistent analyses and performance assessments. 

3.1.2.4. Select the Design Pattern and Pivot Laboratory (if appropriate) 

Identify the design pattern to be used in the PT. The same pattern is used to establish

the overall schedule in stage P1. The circular pattern is selected by default and is most 

common. Pivot laboratories may be used to monitor trends and stability or to provide 

interim reference values for interim evaluation and feedback but are not required. All 

pivot laboratories must be approved by OWM with documented and planned roles and 

responsibilities with respect to use of data in final analyses. Any available reference 

values will not be provided to pivot laboratories prior to submission of calibration 

certificates to prevent undue advantages or collusion in analyses. 

3.1.2.4.1. Circular Design

The PT moves in a circle from beginning to end and may start and end with a pivot 

lab (the pivot lab is not required). Each laboratory ships the PT to the next lab

without returning it to a pivot lab for intermediate measurements. This version of 

the design leads to the common term “round robin.”
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 11 of 38

Figure 1. Circular Design

3.1.2.4.2. Petal Design

The pivot lab makes initial measurements, then one of the participating labs makes 

measurements, then the pivot lab gets the PT back and makes intermediate 

measurements before sending the PT to the next lab. This is repeated for every 

laboratory.

Figure 2. Petal Design

3.1.2.4.3. Modified Petal Design
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 12 of 38

This design is similar in concept to the Petal Design, with the difference being that 

the PT does not return to the pivot lab after each lab. The number of labs a PT is 

passed through before returning to the pivot lab is optional based on the objectives 

and stability of the artifact.

Figure 3. Modified Petal Design

3.1.2.5. Calibration Procedure

It is expected that NIST Standard Operating Procedures (SOPs) are used in the OWM 

PTs. When there is no NIST SOP available, use of internationally or nationally 

published SOPs is expected. Enter the calibration procedure participants are to use and 

include the entire reference document and date for the procedure. See also Section 2.7.

Be specific with full document, title, and date of SOP: "NISTIR 7383 (2019 Ed), NIST 

SOP 19 (March 2019) Calibration of Graduated Neck-Type Metal Provers" versus 

"SOP 19". Note whether any laboratories request alternative procedures to be approved 

by OWM. Alternative procedures may be requested for OWM evaluation, may not be 

proprietary or restricted for distribution, and must provide equivalent uncertainties to 

the selected procedure. Alternative analyses may need to be implemented when 

uncertainties are not equivalent. 

3.1.2.6. Environmental Conditions During Storage, Handling, and Calibrations 

The default terminology is: “As outlined in the designated calibration procedure(s) 

which are followed to ensure PT standard(s) artifacts are not compromised. Conditions 

to be specified on calibration certificate (ranges acceptable).” 

Environmental conditions, equilibration, and uncertainty budget components identified 

in the NIST SOPs must be followed by all participating laboratories, even if they are 

using other procedures or laboratory-developed methods. Environmental conditions for 

calibrations must not negatively impact the measurement results or uncertainty of the 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 13 of 38

calibrations and must not negatively impact the stability of the standard items being 

calibrated in the PT. 

3.1.2.7. Equilibration Procedures and Time

The default terminology is: “As outlined in the designated calibration procedure and as 

routinely managed for similar types of calibrations (unless additional requirements 

specified here). Equilibration conditions to be included on calibration certificate if 

supplemental to the calibration procedure.” The equilibration requirements of the 

designated SOP are used by all participants and restated if/as needed. 

If the chosen calibration procedure or documentary standards for handling do not cover 

equilibration requirements, a consensus decision must be made during the PT planning 

process. Modifying the default entry is also required when the PT Plan is used for 

method validation, especially if laboratories are using differing approaches for 

equilibration. In the case of differing approaches, a supplemental data sheet to 

document differences may be used, or the equilibration procedures may be included on 

the calibration certificate. Whatever approach will be used is to be specified during the 

PT planning phase. 

3.1.2.8. Required Equipment 

The default terminology is: “Participants will use equipment that they typically use for 

this measurement parameter ensuring that accuracy and repeatability requirements of 

the procedure and this PT are met.” When new technology, equipment, or standards, 

are being evaluated, additional specific requirements must be discussed during 

planning and documented in this field. If any special equipment is required as a part of 

the calibration and/or shipped with the standard, it should be noted here. For example, 

if a support stand is to be shipped with a “special J” type prover for calibration, make 

sure the equipment is listed in this section. 

3.1.2.9. Measurand and/or Characteristics of Interest

Specific reference conditions and references for the measurement parameter and 

item(s) intended to be measured must be specified during the PT planning phase. 

Instructions in the PT Plan template note the following as examples: Conventional 

Mass correction, Volume at the '0' graduation mark, Volume at the '0' graduation mark 

and 4 neck scale intervals, length between 7 inch and 13 inch graduations. Volume 

might need to be specified at 20 °C or 60 °F or Volume to be Delivered (TD) or Volume 

to be Contained (TC). The explicit measurand must be defined during planning so that 

participants report correct measurement results and references in their calibration 

certificates and so analyses are evaluating like references. Other characteristics of 

interest could be something for the participant to identify, such as the material, 

coefficient of cubical expansion, or the density and are often specified in the Reporting 

section of the chosen Standard Operating Procedure.

3.1.2.10. Units to be Reported
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 14 of 38

There is a standardized drop down field of possible measurement units. Measurement 

results and uncertainties may need to be reported in different units. The PT Analyst can 

often convert reported values on certificates, but that is an extra step and an opportunity 

for conversion errors that can be avoided or minimized if participants report values as 

planned. When a data entry sheet is circulated to PT participants for data entry, the PT 

coordinator or PT analyst must ensure appropriate labels are specified and indicated to 

ensure all values are subsequently analyzed using the specified measurement units. 

3.1.2.11. Number of Significant Figures

The default for OWM PTs is two significant digits unless otherwise noted in the 

instructions. This follows the NISTIR 6969, Good Laboratory Practice 9, for Rounding 

Measurement Results and Uncertainties. To comply with special applications, the 

instructions might state “at most, 2 significant digits.” 

3.1.2.12. Specify Any Supplemental Data Sheets

The default for supplemental data sheets is “no” for OWM PTs. In some cases, for 

larger numbers of standard items to be measured, a supplemental data entry sheet may 

be used to assist the PT Analyst so they can copy/paste from the participant-created 

data sheet. In that case, the confirmation of data entry is conducted during data entry 

by the participant and/or participating laboratory representative. A spreadsheet version 

of the PT Analysis spreadsheet may be used to simply the analysis effort; the “data 

entry” sheet of the PT Analysis spreadsheet may be used for this application. There are 

often supplemental data sheets and forms on specialty PTs, and national PTs, or 

international comparisons. The PT coordinator or PT analyst must ensure the extra data 

sheet has correct labeling of the intended measurement units!

A supplemental data sheet is an option for the PT Analyst but does not replace the 

requirement for calibration certificates to be submitted. When a supplemental data 

sheet is used, the format must match the first data entry row and headings in the Data 

Entry tab of the PT Analysis template. The PT Analyst is expected to still do a quality 

check and sample the data entry and calibration certificate(s) to verify that the values 

match. 

3.1.2.13. Tolerance Source and Class

Select the applicable documentary standard and tolerance that will be used to evaluate 

participant data as this is normally applicable for PTs supporting legal metrology. The 

evaluation approach may differ from the classification for the PT items. A drop-down 

of the most common tolerances used in OWM PTs is provided for this field. When a 

tolerance is specified, it implies conformity assessment and decision rules be followed

in the PT reporting process. The participant calibration certificate will then list 

tolerances, how the uncertainty was applied in making decisions, and an assessment of 

compliance to specifications and tolerances (or what was not evaluated) as required by 

ISO/IEC 17025 conformity assessments. 

3.1.2.14. The Percentage or Fraction of Tolerance used for Assessing Pn
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 15 of 38

This value is used in the normalized precision calculations and is related to published 

decision rules in the standards referenced in the Tolerance Source and Class. A dropdown option provides either 1/3 (as used in mass specification decision rules) or 100 % 

which will compare the uncertainty to the tolerance on a one to one basis (as is the case 

for most volume calibrations). When a laboratory volume standard is calibrated in the 

PT scheme, the ratio of 1/3 will be used to ensure that laboratory standards are fit for 

purpose and have sufficiently small uncertainties for use when calibrating field 

standards. 

3.1.2.15. Expected Uncertainties from Participants

The default terminology is: “Uncertainties must represent the actual uncertainties used 

by the laboratory. The expected value is small enough to pass the Pn statistical test and 

includes the components identified in the NIST SOPs.”

Uncertainties must represent the actual uncertainties calculated by the laboratory and 

not be a function of the specified tolerances or portion of tolerances. The expected 

value must be small enough to pass the Pn statistical test and includes the components 

identified in the NIST SOPs. The reported uncertainty must be generally consistent 

with other laboratories using the same or similar procedures and significant differences 

among participants may be investigated during analysis. Reported uncertainties may 

not be smaller than those published by accreditation bodies and are expected to be 

reasonably close to published or submitted valuesthat would be obtained during normal 

or routine calibrations. Note: A precision assessment should be a routine part of each 

laboratory ensuring the validity of laboratory results and uncertainties reported to 

customers. Having precision failures in a PT indicates that laboratories have not 

sufficiently evaluated their uncertainties with respect to documentary standards and 

taken suitable corrective actions. 

3.1.2.16. Expected Range of Values from the Participants

Default terminology is: “The expected range of values from participants is less than the 

tolerance.”The normal variation of a selected PT item is that its measurement values 

stay within tolerances during circulation with proper (normal) care and handling. The 

range is defined as the maximum value minus the minimum observed or reported value. 

When all participants claim metrological traceability to the SI with suitable 

uncertainties compared to the defined tolerance, it is typical that all reported 

measurement results will be within the tolerance limits unless there are other gaps in 

the participant’s calibration program (e.g., standards not calibrated according to 

schedules; inappropriate equipment used for the calibration; environmental controls 

outside of prescribed limits.) In most OWM PTs, only major errors, problems with 

reference values or standards, major blunders and mistakes, personnel competency 

issues, or outliers are likely to be observed outside the tolerance range specified in the 

PT plan.

3.1.2.17. Expected Limits of Stability
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 16 of 38

The default terminology is: “When there is no significant observed shift or drift, and 

the first and last measurements are not outliers, the comparison between initial and 

closing measurements should result in an En value of less than 1”. A stability 

assessment may be conducted during the PT analysis, with Pivot Laboratory values or 

with the first and last participant values. A visual assessment of the data might show 

shifts or trends, in which case additional analyses of the PT may be required. If there 

was a major change in a standard during the conduct of the PT, multiple analyses may 

be required in the final analysis and documented in the PT Report. The stability of the 

standard(s) also must be addressed in the final report by the PT Analyst and/or OWM.

3.1.3. Stage P3: Worksheet P3; Artifact and Shipping

3.1.3.1. Artifact Owner and Description

3.1.3.1.1. Owner Name and Address

Ownership information is supplied by OWM or the owner when a stable PT item 

is loaned for an RMAP PT or national PT. Artifact characteristics (e.g., 4 in neck 

vs 3 in neck, or stainless steel vs cast iron) that are being requested for the PT are 

documented by the PT coordinator during the planning stage. Once an artifact is 

selected, the other relevant characteristics are identified on the artifact (or case) or 

supplied by OWM or the owner.

3.1.3.1.2. Unique Description and Characteristics of the Selected Standards

Document all applicable characteristics about the standard(s), including (as 

applicable) but not limited to: 

o Manufacturer;

o Serial Number;

o Description;

o Range (largest to smallest) or Nominal;

o Number of Pieces;

o Model / Design;

o Tolerance classification as manufactured;

o Material;

o Density;

o Coefficient of Expansion; 

o Neck Graduation Size; or

o Other.

3.1.3.2. Handling

The PT coordinator provides sufficient details on handling requirement of the artifact

and references any applicable procedures. Some items to consider are cleaning (or not), 

gloves/weight handlers, drying before packing for the next laboratory, storage, or safety 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 17 of 38

requirements. Participants must be fully trained or supervised by a trained metrologist 

for proper handling when participating in PTs (see section 2.5.)

3.1.3.3. Packing 

Artifacts must be packaged to avoid damage. The PT coordinator includes a description 

with sufficient details of the case (and acceptable methods of securing), packing 

material, and how the artifact(s) are packaged. Any PT specific instructions or pictures 

are documented here. This section includes the following or similar statement: “Any 

participant that receives the artifact packaged in a manner that is not appropriate or 

mishandling by the shipper is evident, must contact the coordinator and OWM.” 

3.1.3.4. Shipping

OWM informs the PT coordinator if alternative shipping methods are designated for a 

PT, but PTs are generally shipping using a default method to reduce transport time, 

lost, or damaged PTs. The required default shipping method is FEDEX Express, 2nd 

Day Air (2nd Business Day). This section includes the following or similar statement: 

“If your laboratory is unable to use this shipping method, contact OWM for alternative 

methods and approval.” 

OWM pays the shipping costs for state laboratories. In these instances, the billing is 

“3rd Party” with the specified account number (not included in the PT Plan or this 

procedure due to prior fraudulent use by outside parties. In the billing reference or 

internal comment section, indicate “NIST Div. 680.02 (and the RMAP group, e.g., 

NEMAP)”. OWM does not pay for shipping costs for non-state laboratories unless the 

PT Coordinator is a volunteer from that other laboratory. The OWM Federal Express 

account number is not published in procedures or templates that are posted online due 

to prior fraudulent access. 

3.1.3.5. Insurance Reference Information (for Shipping)

The PT coordinator completes the insurance field and shipping weight estimates by 

locating the corresponding values provided.

3.1.4. Stage P4: Worksheet: P4; Addresses and Contacts

OWM provides full RMAP member lists and attendance lists to the PT coordinator for 

completing the contact list and shipping addresses. This information must be reviewed by 

the laboratories during the PT Plan review to ensure PT items are correctly shipped and not 

lost in shipping. The shipping address and contact information on this sheet is used by 

participants in coordinating shipping to the next participant in the PT once they have 

completed their calibrations. 

The list on P4 sheet must match with the list of laboratories provided in sheet P1 and is not 

automatically completed. However, the detailed list of participants on the P1 sheet is used 

as an automatic reference in the PT Report sheet (R1) that is part of the PT Plan template. 

Be sure to double check spelling accuracy for organizations and names. Ensuring a 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 18 of 38

complete list of participants during the planning phase is important to ensure the PT Report 

includes all participants and does not inadvertently omit someone. The laboratory and name 

columns may be copied/pasted as values on the P4 sheet, with additional shipping address

details and contact information completed. 

3.1.5. PT Plan Review and Approval 

Once the PT Plan sections P1, P2, P3, and P4 have been completed, and reviewed by all 

participants with any updates or participant additions resolved, the PT Coordinator submits

the PT Plan to OWM for approval to proceed. OWM may fill in or modify sections of the 

PT Plan if the information was not initially available to the PT Coordinator or if it needs to 

be adjusted. OWM will communicate approval and/or return the approved PT Plan to the 

PT Coordinator to circulate a final copy to all participants.

3.2. Operation Stage

3.2.1. Roles of the PT Coordinator(s), Mentor(s), Observer(s)

The operation stage of the PT Plan template includes two worksheets, O1, for tracking 

progress of the PT against the planned schedule and O2, for recording and documenting 

any problems, delays, damage, or complaints that arise during the PT. These records are 

completed by the PT coordinator during operation of the PT and shared with the PT Analyst 

and OWM. Any issues that are required to be resolved by OWM require additional 

communications and status to be documented. The following sections cover the tasks 

conducted during the PT operation phase. 

3.2.1.1. Start the PT

Email the OWM approved PT Plan to all participants, lab contacts, and the RMAP 

regional coordinator. Coordinate with OWM to provide the artifact and ensure it is 

shipped to the opening laboratory. Ensure a printed copy of the PT Plan is included 

with the PT items and label the PT Plan to keep it with the PT items. Request OWM to 

send available initial or historical reference values to the PT analyst after the analyst 

has performed their initial measurements and submitted a calibration certificate.

3.2.1.2. Monitor the PT Progress and Report Status

The Monitoring of the Progress and Reporting Status is documented in the PT 

Workbook, Section O1: Coordinator Tracking. The coordinator is responsible to 

always know where the PT items are located and be able to report the status of the PT 

to OWM when requested. The status is tracked by the PT coordinator as the PT moves 

from laboratory to laboratory. The coordinator actively communicates with the PT 

participants and PT analyst to keep the PT on schedule, report any delays or 

problematic laboratories to OWM before delays could interfere with the critical 

timelines. A laboratory in the planned schedule may need to be skipped to get the PT 

item(s) to a “hard scheduled” laboratory and return to the schedule later when there are 

delays. PT shipments should not proceed until the shipping laboratory confirms with 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 19 of 38

the receiving laboratory that they are available and prepared to participate in a timely 

manner.

3.2.1.3. Feedback and Complaints

Feedback and complaints are documented in the PT Plan, Section O2: Coordinator 

Feedback. The coordinator records feedback, recommendations, observations, and 

complaints throughout the course of the PT.

3.2.1.4. End the PT Round, Submit all Records, and Prepare for the Analysis Phase

If there are any question on the stability of the artifact as determined by the PT 

Analysis, additional ending measurements may be needed. After all measurements are 

completed, the PT coordinator notifies OWM and the PT analyst. The PT coordinator 

identifies the final shipping location for the PT items and informs the last participant 

to ship to the final location. AT that time, the PT Coordinator notifies the PT Analyst 

and OWM that all measurements are completed.

The coordinator sends electronic copies of all the calibration certificates or 

supplemental data sheets submitted by the participants with the completed coordinator 

tracking and feedback sections O1 and O2 of the PT Plan to OWM. The coordinator 

prepares a draft summary based on their observations, feedback, and complaints 

(formal or informal) from participants and OWM, and forwards it to the PT analyst for 

inclusion in the PT draft and final reports.

3.2.2. Roles of the PT Participants

PT Participants have a responsibility to review the PT plan and follow the schedule and 

designated procedures in a timely manner and to the extent possible to avoid introducing 

problems that must be resolved. They must also submit calibration certificates and 

supplemental data sheets if requested to the PT Coordinator or PT Analyst (however 

designated) so that interim feedback is possible and so that reporting can be completed 

according to the planned critical timelines. PT Participants must also report delays, 

damage, conflicts, and shipping issues as soon as possible to the PT Coordinator so that 

alternative plans can be implemented if needed. 

3.2.2.1. Follow the Schedule and Ensure Approved Participation

The PT schedule is developed by the coordinator based on input from participants, the 

OWM, the logistics of shipping the PT, and the time allotment for each laboratory. 

NIST OWM PTs typically have multiple metrologists from several laboratories 

participating, to minimize disruptions and costs, follow the schedule as designed. Only 

metrologists approved on the PT Plan may participate unless additional approvals are 

granted by OWM and communicated to the PT coordinator and PT analyst. 

3.2.2.2. Report Delays or Conflicts Immediately
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 20 of 38

When a laboratory is in possession of the PT and experiences an unexpected event that 

may cause a delay of the PT, they immediately communicate this with the coordinator. 

This allows the coordinator to make timely informed decisions on the progression of 

the PT.

When a laboratory is not in possession of the PT and identifies a potential conflict with 

the schedule of the PT, they immediately communicate this with the coordinator. This 

allows the coordinator to make timely informed decisions on the progression of the PT.

3.2.2.3. Receive PT Shipments

When a laboratory receives the PT, they shall inform the PT coordinator of the date of 

receipt, visual inspection observations, any damage to the artifact or case, and whether 

PT items were packaged in an inappropriate manner.

3.2.2.4. Perform Measurements

PTs must be treated as priority calibrations. Measurements must be performed in a 

timely manner to keep the PT on schedule. Unless otherwise directed by the PT Plan, 

measurements are made as per routing customer calibrations. PTs are not treated as 

special calibrations by changing the process normally performed. PT items and 

associated equipment (e.g., cases, trailers) are not permitted to be used for purposes not 

defined in the approved PT Plan.

3.2.2.5. Submit Calibration Certificates

Participants are required to report their values to the coordinator using an official 

calibration certificate that is ISO/IEC 17025 compliant. Amended values or 

uncertainties resulting from corrective action or new measurements are submitted in an 

official amended calibration certificate, with reasons for amendment noted on the 

certificate per ISO/IEC 17025 requirements. Calibration certificates must be submitted 

in a timely manner so the PT coordinator or PT analyst can supply initial feedback and 

keep the PT on schedule.

Before receiving initial feedback, each laboratory selects one participant to represent 

their laboratory’s “official values”. These values are used in the analysis of the PT.

If a participant uses a non-NIST procedure, they may be requested to submit a copy of 

the procedure to the PT analyst or OWM along with their calibration certificate.

3.2.2.6. Report Valid Uncertainties

Laboratories are required to report their actual uncertainties based on officially reported 

uncertainties per their Scope as noted earlier. Uncertainties that are reported based on 

guard-banding, test-uncertainty ratios, or fractions of tolerances or maximum 

permissible errors are not permitted and will result in “failure” notices in the PT final 

report.
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 21 of 38

3.2.2.7. Repackage and Ship PT Items

PT artifacts are repackaged in the same manner as received (if appropriate) to prevent 

damage to the PT items. The participating laboratory should have informed the PT 

coordinator if packaging instructions were not followed upon the initial receipt and 

inspection. If deviations occurred, if artifacts were damaged, or if the instructions and 

packing needs improvement, the PT coordinator and OWM may need to investigate 

and resolve issues further.

When the laboratory receives approval to ship the PT to the next laboratory, they email 

the coordinator the date of shipment, the tracking number, and which laboratory they 

shipped it to. Additional shipping information (e.g., carrier used, method used) that 

deviates from the Section P3: Artifact and Shipping, is emailed to the PT coordinator.

A PT should never be shipped to a laboratory without confirming with the PT 

coordinator or the receiving laboratory, that they are available and prepared to 

participate in a timely manner.

3.2.3. Roles of the PT Analyst (during the Operations Phase)

The PT analysis may be started during the Operation Stage, but final analyses are 

completed after the PT measurements have all been completed with all calibration 

certificates submitted. The exception for interim analysis is to monitor and problems or 

trends in the data and to provide interim feedback to laboratories. Laboratories may be 

asked to validate the data entry in the PT Analysis templates and to identify major 

discrepancies or possible blunders. When an initial or historical reference value is known, 

the PT analyst may complete the data entry and interim analysis as data is submitted during 

the operations phase; keeping up with the data entry will also save time during the final 

Analysis and Reporting phases. Specific analysis tasks that can be conducted during the 

operations stage are detailed in this section; more in depth statistical analyses are covered 

in the Analysis Stage, Section 3.3. 

3.2.3.1. Data Entry

The analyst shall use the OWM PT Analysis template(s) for evaluation of submitted 

data. PT analysts should be sure to read the Instructions worksheet prior to performing 

any data entry. The file is to be named with the name of the PT as designated during 

the PT planning stage with “analysis” kept in the file name along with the latest date it 

is updated. E.g., from the Planning stage, the example given was MidMAP-25-MIIIM-01. That file name would be saved as “MidMAP-25-MIII-M-01-Analysis20230301.” 

Information about the PT items, tolerances, and identification are entered on the Data 

Entry worksheet in the fields labeled. The PT analyst must be sure to select applicable 

tolerances as identified in the PT Plan. Hover over any cell that has a red triangle in the 

upper right corner for additional instructions during data entry. 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 22 of 38

Data is entered in chronological order by calibration date. Timely submission of 

participant calibration certificates is essential to conduct real-time analysis and to give 

interim feedback.

For each participant from the PT plan, one row will represent data from their submitted 

calibration certificate. Information for each participant includes the 1) date of 

calibration; 2) participant identification (see notes below); 3) whether or not the 

participant has submitted the laboratory’s official value (in laboratories with more than 

one staff member, they must designate which certificate is to be used as the official 

result for their laboratory); 4) which SOP was used (usually a number of the SOP that 

was designated in the PT Plan); and 5) measurement results and uncertainties for each 

PT item in the PT scheme. 

The Participant ID on the Data Entry worksheet consists of the Lab Code and the initials 

of the participant (e.g., NH-JD, represents New Hampshire – John Doe). For non-state 

laboratories, there are designated organizational codes in the PT Analysis template. 

The PT Analysis template, Data Entry worksheet may be exported and sent to PT 

participants to entire data into the worksheet themselves. PT analysts will then sample 

the submitted data entry worksheet for accuracy against the submitted calibration 

certificates and copy/paste the data into the master PT Analysis file.

3.2.3.2. Interim Feedback if Problems are Observed or Initial Data Analysis if an 

Initial Reference Value is Provided

For values that appear to be marginal or outliers, laboratories are advised to validate 

their measurements and reported results. After the laboratory has checked their results 

and their values still appear significantly out of line, OWM is contacted for guidance.

Analyst feedback does not include specific or general details on performance (e.g., no 

reference values are provided in interim feedback). Final performance statistics are not 

known until final analysis is conducted with all participant data and the analysis is 

approved by OWM.

The analyst maintains the integrity of the PT by not informing participants of reference 

values, or how close a participant is to the reference value. Feedback is not provided 

on values submitted via phone, fax, email, or in person. The only acceptable forms of 

data submission are an official calibration certificate or an official amended calibration 

certificate.

For certain unique PTs that are slow moving and are continuous in nature (e.g., national 

trailer based PTs), interim reports may be provided by OWM.

3.2.3.3. Data Entry Validation

The analyst offers an opportunity for the participant to evaluate data entry by copying 

only those data entry rows for that laboratory to review and verify accuracy of the data 

entry. The PT analyst may conduct initial analysis and provide “go/no go” feedback.
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 23 of 38

3.2.3.4. Preparing for the Draft Analysis and Report

Once the PT round is completed, any remaining data is input into the OWM PT 

Analysis template. If the PT analyst has entered data as it is received in a timely manner, 

there should be very little remaining data entry required. Laboratories may again be 

sent the Data Entry rows so they can confirm there are no data entry errors. No other 

sections of the analysis tool are sent to participants prior to the final reporting. To 

prevent lengthy delays, it is helpful to provide a deadline (e.g., 1 week) for the review 

period.

Data entry errors made by the analyst are corrected. Data entry errors made by 

participant calibration certificates or submitted data sheets must be retained, though are 

not included in the statistical analysis. Data entry errors by the participant are 

considered failures and must be corrected with an amended certificate. All data is 

retained for the final report with corrections and amendments noted in the final report.

The data entry file is submitted to OWM for review to provide any necessary 

input/suggestions on data points to be questioned further prior to the analysis stage.

3.2.3.5. Prepare Draft Analysis and Report

After receiving any inputs or feedback from OWM at the end of the operations stage, 

continue on to the Analysis stage, section 3.3

3.3. Analysis Stage: Data Analysis and Calculations 

The OWM PT Analysis templates are used for data analysis and participant performance 

assessments.

3.3.1. Complete the Data Entry worksheet using only the official calibration certificate(s) 

for all participants if this was not already done during the Operations stage. Either 

the PT analyst or PT participants enter the following information on the PT 

Analysis template Data Entry worksheet: 

• Enter the calibration date;

• Enter the laboratory name code (from the table in the spreadsheet or the two digit 

State abbreviation) and participant initials;

• Enter the reported SOP;

• Enter the Measurement Result(s) and Uncertainty(ies);

• Do a visual assessment for data entry errors or potential blunders (mistakes);

• Select the “official” tag for the staff member who has been designated as the official 

value by the participating laboratory (only one value may be selected per laboratory 

to avoid weighting results) and deselect others from that lab as appropriate; and 

• Request the laboratory evaluate and confirm the data entry values.

3.3.2. Visually Assess the Data and Graph Worksheets for Stability and Outliers for all 

PT Items 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 24 of 38

Visually review the graphs and look for stability, measurement result shifts or changes, 

trends or drift in measurement values, and significant data outliers. Look for outliers that 

might be blunders/typos or possible measurement errors and contact the PT participant and 

ask them to verify their submitted data and double check the data entry if not already done. 

Especially look for unit or conversion errors made during recording or data entry. The final 

report needs to include a stability assessment from OWM. The R1 Report section of the PT 

Plan includes a field for OWM staff to complete after reviewing the data. Stability 

observations may be drafted by the PT analyst but must be reviewed and approved by 

OWM. OWM may conduct a visual assessment and stability evaluation as a part of the 

Operation stage if alerted by the PT coordinator or PT analyst of any concerns. 

3.3.3. Assess the Automatic Statistics Calculations for the Mean and Standard Deviation

on Each “Data (n)” Worksheet

The initial mean and standard deviation are calculated using all of the participating 

laboratory’s official reported values and uncertainties. Visually scan the tabulated data and 

review the data entry to ensure all official values are identified and that no values stand out 

as significantly different. 

3.3.4. Review and Deselect Values Outside of Two Standard Deviations of the Mean 

Value (Trim the Mean) to Create the Adjusted Statistics Shown in the Data Analysis 

Block

Review the suggested “adjustment” instructions and identify any outlier or suspect values.

Deselect all values outside of the two standard deviation limits is used as a first step to 

create the Adjusted Mean and Adjusted Standard Deviation statistics that may be used 

when selecting the reference value and reference uncertainties. The values will be signified 

by HIGH or LOW (above or below two standard deviations). This step may be performed 

only ONCE. Sequential de-selection of data is inappropriate. No data is physically deleted 

from the data entry, data analysis tables, or graphs. All data will have performance statistics 

calculated and reported. After this step, the “adjusted mean and adjusted standard 

deviation” can be reviewed in the statistics on the top of each “Data (n)” worksheet. 

The adjusted standard deviation for the PT is used in the Z Score calculations. 

3.3.5. Assess and Determine Reference Value and Uncertainty Options

Several valid statistical approaches may be used for determining suitable reference values

and they must comply with requirements in ISO/IEC 17043, Annex B and/or ISO 13528. 

The most common methods that have been used by OWM are listed below. Someone with 

suitable technical expertise in the measurement parameter and level of work shall be

involved with assessing and determining the final and most suitable reference value(s) for 

each PT item. The selection of the reference value may be different among PT items even 

within a set of multiple standards depending on the data and analysis of the reference 

values. Selection option 4A from Table 1 is set as the default selection in the OWM PT 

Analysis template and is identified as the “Wt'd Mean±Trm'd Avg U”. Options 4A and 4B 

are the most commonly used reference values in OWM PTs. Uncertainties for reference 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 25 of 38

values are always reported and used in the En assessments and plotted on the associated 

analysis graphs. The PT analyst may select reference values based on this table or 

additional analyses, but final approval of the PT reference value(s) will be made by OWM 

staff. 

Choices in the PT Analysis template spreadsheet include the following options in a drop 

down cell for selecting reference values:

• Adjusted mean (will be identical to the calculated mean value when no data is 

deselected; this approach helps to identify values that might have gross errors 

or are outlier values);

• Adjusted mean with μb (uncertainty of bias must be reported and entered during 

analysis; this will be used in the reference value uncertainty) – rarely used;

• Calibration Source from one laboratory such as NIST or an approved pivot 

laboratory (this selection value defaults to zero in the selection list if no values 

are entered in the PT Analysis template spreadsheet);

• Mean of accredited laboratories and average uncertainty (a formula must be 

entered to calculate the mean of specific accredited laboratory values and also 

the mean of their uncertainties) – rarely used; and

• Weighted mean and trimmed average uncertainty (set as the default) – this value 

weights the selection of the reference value based on the reported uncertainty 

and uses the mean of uncertainties nearest the median uncertainty.

Several of these choices might be used to evaluate the impact on the final PT 

analysis for all participants; however, the final selection of assigned reference 

values is evaluated and approved by OWM prior to release of a final report. Criteria 

used by PT Analysts and OWM staff have been evaluated by NIST statisticians to 

ensure appropriate values are selected for standards used in each PT. All choices 

should be considered and compared to other options in the spreadsheet as a part of 

ensuring the validity of the selected reference value.

3.3.5.1. Reference Value and Uncertainty from a Single Laboratory (Externally 

Derived Criteria) – ISO 13528, section 7.5. ISO/IEC 17043, B.3.1. item c.

A reference value from a single laboratory may be one from an NMI, such as one from 

NIST. This might be considered an ideal reference value to use when there is also 

evidence of stability, and the uncertainties are sufficiently small relative to the 

participant values. This source is not always an option due to the high cost and the time 

associated with obtaining this value. Stability of the standard may also make this value 

less desirable due to the lack of long-term stability of reference values in some 

measurement areas. In some cases, where standards have demonstrated stability over a 

long period of time, these values may be used. (Examples: 100 gal prover, 500 lb 

reference standards). The uncertainty associated a single-laboratory reference value is 

taken from the calibration certificate. It is critical that this value be compared to all 

other reference options to ensure stability and ongoing validity of the chosen reference 

value(s). Given that all participating laboratories are required to maintain traceability 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 26 of 38

of measurement results to the SI, this option is not essential for most legal metrology 

applications and proficiency tests. 

3.3.5.2. Accredited Laboratory, Pivot Laboratory, PT Coordinator Laboratory or 

Groups of Expert Laboratories Initial Reference Value and Uncertainty 

(Externally Derived Criteria) – ISO 13528, sections 7.5, 7.6 Consensus 

value from expert laboratories; ISO/IEC 17043, B.3.1. items c or d. 

As a part of the PT Plan, OWM and the PT Administrative Team and PT participants 

may have discussed and approved using an initial measurement result and uncertainty 

and possibly ending values from an Accredited laboratory, a Pivot Laboratory, or a PT 

Coordinator. This is Choice 2 shown in Table 1. Unless measurement results at this 

level have uncertainties that are significantly smaller than other laboratories in the 

group, exceptional care must be taken to ensure suitable agreement in the final 

measurement results to avoid conflict among participants and disagreements about 

assigned reference value(s). This avoidance of perceived conflict is especially 

important given that OWM PTs are not anonymous, and participants are familiar with 

the other laboratories’ facilities and staff, and their capabilities and scopes. 

Using a single laboratory (often called a Pivot Laboratory) with a “better procedure” is 

sometimes chosen and may be used to monitor for trends/drift during the course of the 

scheme, often with before and after measurements, however the evaluation and 

selection of the reference value must include assessment against other options. The risk 

with this option may include challenges or appeals to the pivot laboratory value(s)

where a laboratory that fails one or more of the statistics used in the analysis. This 

approach must be compared to other options in the spreadsheet as a part of ensuring 

the validity of the reference value.

This approach may be suitable in some instances, for example:

• Where more than one level of calibration will be performed in the PT, with some 

laboratories performing a higher-level procedure (lower uncertainty) and the 

remaining laboratories performing a lower level procedure, a mean value from 

these laboratories may be used to select a best assigned reference value. 

Calculations of the mean values and uncertainties of the better subgroup of 

procedures could be used. 

• Where the standard to be used in the PT belongs to one of the participants and 

significant history of calibrations and stability is available the “owner” may be 

selected to provide initial and closing measurement results and the value from that 

laboratory used as the initial assigned reference value.

3.3.5.3. Historical Reference Value and Uncertainty (Externally Derived Criteria)

An historical reference value can be an individual value or a collection of values from 

a variety of sources including past NMI calibrations, past RMAP calibrations, or past 

accredited lab calibrations. The uncertainty is often a mean of the uncertainty of the 

selected values (average uncertainty from contributing values). These values can often 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 27 of 38

be used to assess stability of the standard artifacts over time or to validate another 

chosen reference value. When this data is available, it is often used as a quick check of 

participant data as it is being collected and entered during the proficiency test. 

3.3.5.4. Mean of “Official” Participants and Uncertainty (Consensus Value) 

(Comparison Derived Criteria) – ISO 13528, 7.7 Consensus value from 

participant results; ISO/IEC 17043, B.3.1. item e. 

When all official values agree with no need for omitting data as part of the analysis, 

and when the associated uncertainty is acceptable for the assessment needed, the mean 

value of all official participant results may be used. This value is most often used when 

there is no other good alternative, or when the tolerances are sufficiently large that the 

use of this value no significant negative impact on the analysis. The uncertainty is from 

the standard deviation of values used, multiplied by k as a coverage factor. OWM PTs 

are coordinated among laboratories that all have demonstrated traceability to the 

International System of Units (SI) and any or all of the laboratory values could 

conceivably be used in demonstrating traceability for the reference value (provided 

robust statistics support the selection decisions). This approach must be compared to 

other options in the spreadsheet as a part of ensuring the validity of the reference value.

3.3.5.5. Weighted Mean and Average Trimmed Uncertainty – ISO 13528, 7.7 

Consensus value from participant results, ISO/IEC 17043, B.3.1. item e. 

This is the default method selected in the OWM PT Analysis spreadsheet. After the 

initial data is reviewed and initial failures are flagged and removed from the analysis 

used in selecting the assigned reference value, the values and statistics that are 

remaining are considered the adjusted, trimmed, or Winsorized mean and include an 

associated uncertainty. The weighted mean and average trimmed uncertainty are then 

used to ensure that laboratories with smaller uncertainties contribute a greater 

proportion of the assigned values. This approach may underestimate the uncertainty of 

the reference value thus could impact the normalized error calculations. Again, this 

approach must be compared to other options in the spreadsheet as a part of ensuring 

the validity of the reference value. 

3.3.5.6. Simulations and Monte Carlo Assessments

Although not widely used for OWM PT analyses, this tool generates simulated values 

based on an inputted distribution and variables for the PT data set. Simulation iterations 

can run in the tens of thousands, hundreds of thousands, or more depending on the 

computing capabilities. When this analysis is conducted, the values are often entered 

as additional participant data point(s) for reference when reviewing the graphs and the 

selection of reference values. This approach has been considered in a number of PTs 

although it is not explicitly referenced in either ISO/IEC 17043 or ISO 13528, though 

the standards do reference alternative rigorous statistical approaches that must be 

documented. 

3.3.5.7. Multiple assigned reference values.
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 28 of 38

Selection of different reference values may be required for each standard within a set 

of standards circulated for a given PT. Typically this approach is reserved for problem 

artifacts that seem to be trending in a consistent pattern or direction. Problematic data 

could include situations where standards are cleaned or damaged in some way during 

the PT where an obvious shift in the data occurred. Combinations of other reference 

value and uncertainty options may be used for each subgrouping of data. Use of 

alternative methods by participants is normally assessed according to method without 

mixing results for analysis. The summary data chart for standard/artifact in the PT 

designates the value that was used and selected as the assigned reference value. All 

other statistics and uncertainties performed for that standard will then be based on the 

selected reference value.
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 29 of 38

Table 1. Summary Table for Reference Values and Uncertainty Choices.

Choice Source Uncertainty Comments

1 NIST (or other 

NMI)

As reported Only used when measurement 

values are historically stable 

and have a sufficiently small 

uncertainty

2 Single laboratory 

reference value 

(e.g., approved 

pivot laboratory)

As reported May be used when approved in 

the PT Plan and if values are 

historically stable with a 

sufficiently small uncertainty

3 Consensus of 

“Expert” 

Laboratories”: 

adjusted mean of 

approved 

“expert” 

laboratories 

Mean uncertainty of 

selected values

E.g., accredited labs; pivot lab; 

labs with lowest uncertainty

from a designated better 

procedure; separate equation 

entered in designated cells to 

calculate values

4 Consensus of 

Participant 

Laboratories: 

adjusted statistics 

with mean 

weighted based 

on reported 

uncertainties 

(consensus value)

Option: 

A) Average uncertainty;

or

B) Standard deviation of 

selected values

multiplied by k

selected based on 

degrees of freedom as 

a coverage factor for 

the number of 

participant values 

selected.

E.g., the official value from 

each laboratory is selected after 

deselecting any values outside 

of the statistical limits 

PT Analysis template defaults 

to this 4A option: “Wt'd 

Mean±Trm'd Avg U”

PT Analysis template option 

“Adjusted Mean” uses Option 

4B. 

5 Historically stable 

reference value(s)

for the standard(s) 

used in the PT

Mean of the uncertainty 

of the selected values

E.g., other RMAP region, prior 

reference value, approved by 

OWM (may also be used for 

interim evaluation or validation 

of selected option)

6 Monte Carlo 

Simulation

Statistics from simulation Less common
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 30 of 38

3.3.6. Calculate and Review Participant Performance Statistics 

The PT analyst must read the instructions worksheet included with the PT Analysis 

template spreadsheets to supplement guidance in this SOP and to be sure the analyses are 

consistently conducted. 

The PT analyst double checks the tolerances that were entered for each PT item on the Data 

Entry worksheet as that value is used in the calculation of performance statistics. The PT 

analyst also selects the proper percentage or ratio for the uncertainty to tolerance values 

required based on the documentary standard used in decision risk analysis as that value is 

also used in the precision assessment. 

Calculations of the PT Participant performance statistics are automatic once the tolerance 

and decision risk ratios are selected and after 1) the official values for each laboratory are 

selected and reviewed for accuracy, 2) outliers and extreme values are deselected based on 

analysis of each PT item and 3) the reference value and its associated uncertainty are 

selected. All values deselected as extreme values are assessed and reported, the deselection 

process is only used in the process for selecting the reference value(s). 

The statistics used in the performance assessment and reported on the PT Summary 

worksheet are the normalized error, En and the normalized precision Pn assessments. The 

values that are outside of the two standard deviation limit for the PT are also reported, but 

do not contribute to the pass/fail summary statistics for the PT. 

On the Data (n) tables, the detailed performance statistics are automatically calculated and 

tabulated. They include:

• Difference from the Reference Value (bias). This value is useful for the laboratory 

when conducting follow-up accuracy assessments;

• Normalized Error, En, given as an absolute value and which must be less than one 

to pass. This value is conditionally formatted to turn the color red when it fails the 

En assessment; 

• Normalized Precision, Pn is calculated as a method for quantifying a pass/fail 

condition where decision risk assessments require the evaluation of the uncertainty 

when determining tolerance compliance. This value is a simple ratio of the 

uncertainty to the portion of the applicable tolerance and must be less than one to 

pass. It is also conditionally formatted to indicate when a failure has occurred; and 

• Z Score, is the Difference divided by the adjusted standard deviation, also called 

the standard deviation of the PT.

The values are calculated using the formulae in the following sections. 

3.3.6.1. Difference or Bias from the Reference Value (Offset), 17043, B.4.1.3., item 

a, Eqn. B.1.

The difference, bias, or offset (however referenced) of each reported value 

from the selected reference value is calculated and reported as part of the 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 31 of 38

PT analysis data using Eqn. 1. This value is not used as a pass/fail statistic 

but is used in the initial assessment of data by the PT Analyst and by OWM 

to review the overall data for obvious blunders and outliers. The laboratory 

may use this value as a part of its follow-up assessments of laboratory bias, 

accuracy assessment, and evaluations of recalibration intervals. E.g., for 

precision calibrations, a laboratory might want to set recalibration goals 

such that whenever the bias/offset exceeds some ratio of its reported 

uncertainty, a recalibration or interim assessment of metrological 

traceability is conducted. Historical OWM PT statistics (no longer used) 

included an assessment of this offset as shown in Eqn. 2 with a modification 

of the Z-score that was based on laboratory uncertainties rather than the PT 

statistics. The laboratory may still wish to conduct this assessment for 

internal evaluations, but it is no longer reported in the OWM PT Reports.

lab ref x X − Eqn. (1) 

lab ref

historical

lab

x X

OWM Z

U

− = Eqn. (2) 

3.3.6.2. Normalized Error, En , 17043, B.4.1.3., item e, Eqn. B.6 

Normalized Error, En, is defined in ISO/IEC 17043 as the ratio of the 

difference between the reference value and the reported value compared to 

the root sum square of associated expanded uncertainties. The normalized 

error is an indicator of accuracy/inaccuracy as compared to an assigned 

reference value with respect to the associated uncertainties. Conceptually, 

the normalized error asks whether the bias is less than the expanded 

uncertainties of the laboratory and reference value combined in root sum 

square as shown in Eqn. 2. 

( ) 2 2 assessment: Is ? En lab ref lab ref xX UU −< + Eqn. (3) 

OWM uses the absolute value of the calculated En results in order to graph 

multiple statistics on the same charts and to have a simple pass/fail criteria. 

Using the absolute value, the value of En must be less than one to pass. 

Values of En between 0.7 and 1 are highlighted on the charts to alert 

laboratories of the possible need to investigate bias with respect to the 

combined expanded uncertainties. 

2 2 Result must be < 1 to pass. lab ref

n

lab ref

x X

E

U U

− = +

 Eqn. (4) 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 32 of 38

This equation evaluates the bias of the laboratory value from the reference value 

relative to the combined uncertainties of the laboratory uncertainty and reference value 

uncertainty. Values failing the En test may be omitted from the calculation of the 

adjusted statistics in the analysis process, but results are always assessed and reported 

in the draft and final PT reports. 

3.3.6.3. Normalized Precision, Pn, Referenced in ISO/IEC 17043, B.5.1.1. 

The Normalized Precision, Pn, is a performance assessment of fitness for 

purpose (suitability) of the laboratory uncertainty compared to applicable 

documentary standards and is related to decision rules and conformity 

assessments as described in ISO/IEC 17025. Where decision rules and 

conformity limits are provided and reported uncertainty must be considered, 

the precision assessment, Pn is conducted. The precision assessment asks 

whether the reported uncertainty is less than the specified limits, as shown 

in Eqn. 4 where the example is given that uncertainty must be less than onethird of the maximum permissible error (as is the case in mass calibrations 

according to OIML R111 and ASTM E 617). 

1 assessment: Is . . .?

3

Pn U mpe lab < Eqn. (5) 

The precision assessment is a ratio of the reported uncertainty versus the 

decision rule limits. Passing values for the precision assessment are less 

than one and are graphed with the En values. This statistic is unique to OWM 

assessments but is related to ISO/IEC 17025 decision rules and ISO/IEC 

17043 performance assessments. Documentary standards used in legal 

metrology generally specify appropriate uncertainty to tolerance (or 

maximum permissible errors, m.p.e.) ratios on which to base decision risks. 

In this supplemental report, tolerances and m.p.e. terminology is used 

interchangeably. Documented decision risks and use of uncertainties in 

making conformity decisions are specified in the ISO/IEC 17025 standard. 

Many of the OWM published procedures and documentary standards that 

are referenced for legal metrology include uncertainty to m.p.e. ratios of 1:1 

or 1:3, where the uncertainty must be less than the applicable m.p.e. or the 

uncertainty must be less than one-third of the m.p.e. The 1/3 ratio is 

common in international legal metrology documentary standards such as 

those from the International Organization of Legal Metrology (OIML) and 

a number of the NIST Handbook 150-x series documentary standards. 

ASTM E617 for mass standards also includes this common ratio of 

uncertainty to tolerances. In some cases, the ratio will be 1:1, where the 

uncertainty must simply be smaller than the applicable tolerance or m.p.e.
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 33 of 38

The formula used for calculating the precision test (Pn) is shown in Eqn. 5. The result shall 

be less than 1 to pass this statistic. 

 Result must be < 1 to pass. 1 . .. 3

alternative ratios that may be used: 

, . . . fraction or % of . . .

lab

n

lab lab

n n

U P

mpe

U U P P

mpe mpe

=

= =

Eqn. (6) 

The precision test evaluates whether the uncertainty of the laboratory value is less than the 

tolerance or less than a specified ratio of the tolerance. It is used to assess whether the 

laboratory has the capability to properly determine conformity to specific documentary 

standards. In some of the documentary standards for legal metrology, conformity 

assessment requires the uncertainty to be less than one-third of the tolerance or maximum 

permissible error. In others, the uncertainty shall be less than the tolerance and the normal 

calibration practice is to adjust standards to their nominal value. Uncertainties shall be 

sufficiently small to perform proper conformity assessments. The Pn assessment identifies 

whether the uncertainty is sufficiently small so that the potential error in the value does not 

significantly impact the value. 

Values that are excessively small may also be cause for concern and may signify that 

uncertainty budgets are incomplete or that inappropriate procedures were used for the 

planned PT. Values with uncertainties failing the Pn test may be omitted from the 

calculation of the adjusted statistics.

3.3.6.4. Z Score , ISO/IEC 17043, item B.4.1.3. item b, Eqn. B3. 

The statistical evaluation of Z Score comes from ISO/IEC 13528, 3.7 as: 

“standardized measure of performance, calculated using the participant 

result, assigned value and the standard deviation for proficiency 

assessments”. The Z score may be used in combination with the adjustment 

statistics (trimmed mean and associated uncertainty) described earlier.

OWM reports this value in the tables of the PT Report but does not use this 

statistic for pass/fail criteria in PTs because the Z value does not include 

assessment of the laboratory uncertainty. The value may be used in isolating 

values outside 2 standard deviations of the accepted reference values. 

According to ISO/IEC 17043, satisfactory performance on the Z score value 

is generally indicated as Z ≤ 2; unsatisfactory performance is indicated as Z 

> 3. and marginal performance is anything between Z > 2 and Z ≤ 3. 

However, further evaluation of the Z-score requires an assessment of the 

observed bias from the assigned reference value with respect to the reported 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 34 of 38

laboratory uncertainty, such as is provided by the En assessment. However, 

the directionality (positive or negative values) of this statistic can provide 

additional insights to the laboratory for ongoing evaluation of differences, 

bias, or offsets in measurement results especially when compared to internal 

measurement assurance data.

lab ref

PT

x X

Z score

s

− − = Eqn. (7) 

The sPT is the standard deviation for the specific PT round and is automatically 

calculated based on the robust standard deviation of all selected participant results . 

However, the PT standard deviation may also be calculated from the following when 

approved by OWM: 

• a fitness for purpose goal for performance, as determined by expert judgement or 

regulatory mandate (prescribed value) specified in the PT Plan; 

• an estimate from previous rounds of proficiency testing or expectations based on 

experience (by perception) specified in the PT Plan; 

• an estimate from a statistical model (general model) specified in the PT Plan; or 

• the results of a precision experiment approved by OWM in the PT Plan. 

3.3.6.5. Rejection of Observations; Retention of Data

As noted in Section 3.3.5 for selecting of the reference value, data may be deselected 

when it is outside of two standard deviations of the participant values and depending 

on the PT and amount of remaining data additional deselections may be done for 

laboratory values that fail initial En and Pn calculations. 

However, once the reference value has been selected, each participant value is assessed 

based on the final PT statistics and performance statistics with pass/fail status noted. 

Only obvious data entry blunders by the PT Coordinator or PT Analyst are removed or 

corrected during assessments. Data entry errors by the laboratory in preparing their 

certificates are reported along with amended certificate data only if the laboratory had 

an opportunity for advanced corrective action prior to completion of the PT scheme. 

3.3.7. Specialized Assessments

A PT Analysis Youden template is available for use when the Youden analysis would be a 

helpful assessment of random and systematic components of the observed variability. The 

template requires either two identical procedures to be compared when the purpose is 

method validation or two identical PT items to be compared when the purpose is a 

proficiency test. For example, two 5 lb weights, two 20 kg weights, or two 5 gal test 

measures could all be further evaluated using the PT Analysis Youden template 

spreadsheet. Follow the instructions in the template for the data entry and analysis and 

include the appropriate tables and graphs in the PT final report as appropriate. 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 35 of 38

3.3.8. Uncertainties 

3.3.8.1. Laboratory Uncertainties

According to NIST Standard Operating Procedures and OWM Policies, laboratories 

are required to report official uncertainties per their Scope. Laboratory expanded 

uncertainties are evaluated for suitability as a part of the precision assessment of the 

PT (unless not applicable). Laboratory failures may result from normalized precision 

(Pn) assessments and require corrective action. 

Uncertainties that are reported based on guard-banding, test-uncertainty ratios, or 

fractions of tolerances or maximum permissible errors are not permitted. For example, 

when Pn values of 0.33 are observed for all standards in mass calibrations, The PT 

analyst or OWM staff should recommend further investigations of uncertainties and 

may lead to identification of failures and OWM recommended corrective action in the 

PT Final Report.

3.3.8.2. Reference Value(s) Uncertainties

The uncertainty (uncertainties) associated with the reference value(s) are clearly 

communicated within each PT Report on the data tables and on the graphs for each 

standard. OWM reports uncertainties for all selected reference values. 

3.3.9. Ensuring Validity of PT Analysis and PT Reports 

3.3.9.1. Verify Data Entry in Draft Reports

The PT Coordinator(s) and PT Analyst(s) should work together to cross-check 

measurement results as they are submitted and entered on the PT Analysis template

data entry worksheets. Once all data has been entered, a draft Excel file worksheet of 

the data entry may again be circulated to participants to verify/confirm that the data 

entry is complete (for all participants) and has been accurately entered. (Interim/draft 

analyses with analyzed measurement results should not be included in draft results to 

participants at this stage.)

OWM staff also review the draft data entry file and provide input/suggestions (if 

needed) on data points to be questioned further. Once data entry has been confirmed, 

the PT Analyst continues to follow the steps to visually assess the data, evaluate the 

initial statistics, and select the reference value for each calibrated standard to proceed 

with the analysis. OWM reviews the reference value selection before the Data Analysis 

and Report are finalized. 

Data entry errors made by the PT Coordinator or PT Analyst that initially show up as 

outliers should be corrected. Data entry errors made by laboratories on their certificates 

or submitted data sheets shall be retained, though not included in the statistical 

analysis. Data entry errors by the laboratory are considered failures and shall be 

represented by an Amended Certificate that identifies the corrective action according 
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 36 of 38

to ISO/IEC 17025 requirements for amended certificates. All data is retained in the 

final report, although some of the data will not be included in the overall PT statistics. 

OWM staff will seek out additional Technical Advisors if/as needed for specialty areas 

in which they do not have technical expertise and/or if there are any concerns regarding 

the initial data that requires special analysis. 

3.3.9.2. Identification of Corrective Actions and Improvement Actions

Review of draft PTs and corrective actions that are required for final PTs will be 

recorded and monitored within OWM. PT Coordinators should provide input to the 

OWM staff regarding any observations for improvement, corrective action, and any 

informal complaints that were received during the operation of the PT scheme. 

3.4. Reporting Stage 

The OWM PT Plan template includes cover pages for the PT Final Report to simplify and 

standardize creation of the PT Draft and Final Reports. The PT Plan template transfers detailed 

planning data that is required in the PT final report and ensures that all participants are entered 

correctly. 

3.4.1. Calibration Certificates

Submission of participant calibration certificates with measurement results and actual 

uncertainty values are required. The certificates submitted to the PT Coordinator or PT 

Analyst are compiled and submitted to OWM for PT record retention. 

3.4.2. Preliminary (Draft) PT Analysis Report

A draft PT analysis report is completed using OWM PT templates. Confirmation of data 

entry by all participants is required. In general, participants should be provided an 

opportunity to review either all data entry during the interim reviews or review a draft 

report prior to OWM approving a PT Final Report. 

3.4.3. OWM Review and Approval for Final OWM PT Report

The NIST Office of Weights and Measures shall review and approve all OWM PTs before 

they are considered Final. Amendments to final reports shall be noted as to the corrections 

and circulated to all participants. 

3.4.4. Contents of OWM PT Final Reports 

OWM PT Final Reports are designed to comply with all requirements in NISTIR 7214 and 

ISO/IEC 17043 for report contents. The sections of each of the templates is printed to PDF 

and compiled in a master PT Final Report. The PT Final Report includes:

• R1 worksheet from the PT Plan – as completed, approved, and signed by OWM 

staff (taking extra care to expand or condense rows for completeness and legibility;
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 37 of 38

o Extra sections to be completed include the stability assessment and OWM 

recommendations and feedback which often includes recommended or 

required action items. PT coordinators or PT analysts may complete 

sections on recommendations or corrective actions, but the final feedback 

is provided under OWM authority; 

• The following worksheets from the PT Analysis templates:

o Cover (may be eliminated if the R1 worksheet includes all complete 

information;

o PT Summary worksheet (includes a summary of En and Pn performance 

statistics for all participants);

o Data Entry worksheet; and

o Data (n), Graph (n), and E(n)&P(n) (n) worksheets for all “n” PT items (n 

representing the number of each item). 

• The OWM Supplemental PT Report is provided and includes content that is 

applicable for all OWM proficiency tests. 

• Form 1 is referenced in all PT reports to solicit feedback or complaints regarding 

the conduct of the PT.

3.4.4.1. Calibration Certificate Assessment

If a calibration certificate assessment was part of the PT Plan objectives, and 

completed, appropriate checklists from NISTIR 6969, SOP 1 or SP 811 manuscript 

checklist is included in an extra section of the final report to evaluate compliance with 

ISO/IEC 17025 and correct use of measurement units and symbols. 

3.4.4.2. Other Non-statistical Pass/Fail Criteria

Some PTs are planned and designed to assess laboratory participation for performance 

measures unrelated to specific measurement results and associated uncertainties. 

Additional non-statistical pass/fail criteria might include any or all of the following 

items that are explained in the PT Report. These criteria should have been selected as 

a part of the PT Plan during the original planning phase to ensure that all laboratories 

are aware of the additional assessments:

• Compliance of the certificate to ISO/IEC 17025, Section 7.8 as noted in section 

3.4.4.1;

• Errors on submitted certificates and/or data sheets;

• [Unreasonable] time delays on standard/artifact shipments and/or report submission 

(e.g., communicating with the coordinator; reports within 2 weeks);

• Improper packaging and shipping (and handling);

• Deviations from the approved and accepted PT Plan (e.g., using a different SOP);

• Switching or substituting standards or PT artifacts with laboratory artifacts;

• Unapproved cleaning, adjustments, or other identified care and handling problems;
PT SOP 1 Draft for OWM Proficiency Tests 20230301.docx Page 38 of 38

• Uncertainty analysis: Detailed uncertainty analysis may be planned as part of the 

PT Plan. Uncertainty components as specified in the SOP and PT Plan were not 

included.

• Uncertainty reported on a certificate that is smaller than what is on the published 

Scope (for Accredited labs).

3.4.5. PT Draft or Final Reports Presented to Participants 

Reports should be circulated via email or made available to download for all PT 

participants prior to the scheduled review. A scheduled review of the results may be 

conducted online for national PTs but is normally conducted at the regularly scheduled 

RMAP meeting. The PT coordinator and PT analysts should prepare to present the PT Final 

Report at the designated RMAP sessions. The OWM point(s) of contact will be available 

to supplement specific failure assessments, recommendations, or advice to participants, 

and shall communicate required follow up actions for State laboratories. All data entry 

errors should have been identified prior to this review session so that PT Final Reports do 

not need to be further amended. 

3.4.6. Amendment and Addendum to PT Final Reports

PT Final reports may be updated through amendments to fix identified errors or perform 

other corrections; however, thorough data entry review by all parties should normally 

minimize the need for amended reports. Any amended PT Final Report shall include 

notations as to what was fixed or corrected, the date will be updated, and the file name shall 

include “amended”. Amended PT Final Reports shall be circulated to all participants. PTs 

that are appended with new data added for specific purposes (e.g., LAP problems or new 

staff participation after completion of the PT) will be identified as an addendum and shall 

be named with “addendum” in the file name. Appended reports are only issued to the 

applicable added staff. (If a PT has both appended data and amended data, it will be named 

and handled as amended.)

Opportunities for PT Program Feedback or Complaint

Form 1: OWM PT Feedback and Inquiry and Complaint Form is available for all PT participants 

to submit feedback, inquiries, or complaints and can include suggested opportunities for 

improvement to OWM. Appeals regarding PT Analysis or analysis of PT participant performance 

evaluations are handled as complaints. 

PT Follow-up and Laboratory Analysis

According to ISO/IEC 17025, performing follow up evaluations of proficiency tests is a required 

step in ensuring the validity of laboratory measurements. The OWM Good Laboratory Practice 

(GLP) on PT Follow Up Evaluations includes a series of analyses and questions that laboratories 

should use for conducting further post-PT evaluations. The GLP includes a form that can be used 

by the laboratory and incorporated into the laboratory quality management system forms. For State 

Laboratories in the OWM recognition program, conducting PT follow-ups is required annually as 

a part of OWM evaluations.