![](media/image1.png){width="3.6770833333333335in"
height="0.8333333333333334in"}

![](media/image2.wmf){width="1.2402777777777778in"
height="1.2486111111111111in"}

#   {#section .unnumbered}

#  TABLE OF CONTENTS 

[TABLE OF CONTENTS iii](#table-of-contents)

[ABSTRACT v](#abstract)

[INTRODUCTION 7](#introduction)

[Scope 7](#scope)

[Document Retention 7](#document-retention)

[References 7](#references)

[Definitions 9](#definitions)

[1. ADMINISTRATION 10](#administration)

[1.1. Quality System & Policy 10](#quality-system-policy)

[1.2. Organization and Management 11](#organization-and-management)

[1.3. Roles and Responsibilities 12](#roles-and-responsibilities)

[1.4. Collaboration and Subcontracting
13](#collaboration-and-subcontracting)

[1.5. Fees 14](#fees)

[1.6. Policy for Participation 14](#policy-for-participation)

[1.7. Document Control 15](#document-control)

[1.8. Request, Tender or Contract Review
16](#request-tender-or-contract-review)

[1.9. Procurement of Services and Supplies
17](#procurement-of-services-and-supplies)

[1.10. Client Feedback 17](#client-feedback)

[1.11. Control of Nonconforming Activities and Corrective Action
18](#control-of-nonconforming-activities-and-corrective-action)

[1.12. Preventive Action 19](#preventive-action)

[1.13. Records 19](#records)

[1.14. Internal & External Audits 20](#internal-external-audits)

[1.15. Management Review 21](#management-review)

[1.16. Management Staffing and Training
21](#management-staffing-and-training)

[1.17. Collusion and Falsification of Results
22](#collusion-and-falsification-of-results)

[1.18. Data Processing Equipment 22](#data-processing-equipment)

[2. PLANNING 24](#planning)

[2.1. Technical Protocol Development
24](#technical-protocol-development)

[3. IMPLEMENTATION 25](#implementation)

[3.1. Preparation of the Artifact 25](#preparation-of-the-artifact)

[3.2. Packaging/Shipping/Handling 25](#packagingshippinghandling)

[3.3. Materials Handling and Storage
26](#materials-handling-and-storage)

[4. ANALYSIS 27](#analysis)

[4.1. Determination of Assigned Reference Values and its Associated
Uncertainty
27](#determination-of-assigned-reference-values-and-its-associated-uncertainty)

[4.2. Performance Evaluation of Participants
27](#performance-evaluation-of-participants)

[4.3. Outlier Detection Techniques 28](#outlier-detection-techniques)

[4.4. Assessing the Stability of the Artifact
29](#assessing-the-stability-of-the-artifact)

[4.5. Assessing Homogeneity of Artifact
29](#assessing-homogeneity-of-artifact)

[5. REPORTING 31](#reporting)

[5.1. Schedules 31](#schedules)

[5.2. Preliminary Reports 31](#preliminary-reports)

[5.3. Final Reports 31](#final-reports)

[APPENDIX A - FORMS 32](#appendix-a---forms)

[Form 1: PT/ILC Planning Checklist 33](#form-1-ptilc-planning-checklist)

[Form 2: PT/ILC Reporting Checklist
36](#form-2-ptilc-reporting-checklist)

[Form 3: PT/ILC Training & Qualifications
38](#form-3-ptilc-training-qualifications)

[Form 4: PT/ILC Feedback and Complaints
41](#form-4-ptilc-feedback-and-complaints)

[Form 5: WMD Quality System Management Review
42](#form-5-wmd-quality-system-management-review)

[Form 6: WMD Internal Audits for PT/ILC Activities
44](#form-6-wmd-internal-audits-for-ptilc-activities)

[Form 7: NIST WMD Nonconformance/Corrective Action/Preventive Actions
(NCAP)
47](#form-7-nist-wmd-nonconformancecorrective-actionpreventive-actions-ncap)

[Form 8: Schedule & Shipping 49](#form-8-schedule-shipping)

[APPENDIX B -- PROCEDURES & POLICIES
50](#appendix-b-procedures-policies)

[Procedure 1: PT/ILC Software Validation Procedure
51](#procedure-1-ptilc-software-validation-procedure)

[Procedure 2: Selection of Delegates
52](#procedure-2-selection-of-delegates)

[Policy: NISTIR 7082 \"Proficiency Test Policy and Plan (for State
Weights & Measures Laboratories)\" (on-line at
http://www.nist.gov/labmetrology)
53](#policy-nistir-7082-proficiency-test-policy-and-plan-for-state-weights-measures-laboratories-on-line-at-httpwww.nist.govlabmetrology)

# ABSTRACT

This document has been prepared by Jeff C. Gust (Quametec Corp., under
contract) in collaboration with NIST Weights and Measures Division (WMD)
to enable State Weights and Measures Laboratories to comply with
criteria for proficiency testing as noted in NIST Handbook 143 and as
needed for accreditation programs. This document is intended to provide
the basic quality system for Proficiency Testing and Interlaboratory
Comparisons developed by NIST Weights and Measures Division.

**Acknowledgement**

The following individuals participated in the development and review of
this document:

Richard Cote, New Hampshire Department of Agriculture

Doug Faison, NIST/National Voluntary Laboratory Accreditation Program

Ken Fraley, Oklahoma Bureau of Standards

Elizabeth Gentry, Oklahoma Bureau of Standards

Tom Hale, Pennsylvania General Services Administration

Georgia L. Harris, NIST/Weights and Measures Division

Carol Hockert, Minnesota Department of Commerce, Weights and Measures
Division

Van Hyder, NC Department of Agriculture & Consumer Services, Standards
Laboratory

G. Diane Lee, NIST/Weights and Measures Division

Val Miller, NIST/Weights and Measures Division

Steve Schultz, Nevada Department of Agriculture, Bureau of Weights and
Measures

#  {#section-1 .unnumbered}

# INTRODUCTION

## Scope

This document has been developed for the NIST Technology Services,
Weights and Measures Division (WMD) in order to establish and implement
a quality system for the planning, implementation, analysis, and
reporting of Proficiency Tests (PT) and Interlaboratory Comparisons
(ILC) conducted within the WMD Measurement Assurance Program (MAP). This
document contains the NIST QMD Quality Manual for Proficiency Testing
and Interlaboratory Comparisons. This document provides the quality
system to ensure that all Proficiency Testing and Interlaboratory
Comparison activities within WMD are compliant with ILAC-G13:2000,
*Guidelines for the Requirements for the Competence of Providers of
Proficiency Testing Schemes and* ISO/IEC Guide 43-1:1997, *Proficiency
Testing by Interlaboratory Comparisons -- Part 1, Development and
Operation of Proficiency Testing Schemes*. This document specifies
requirements to ensure that NIST WMD and associated delegates are
technically competent to provide specific types of proficiency testing
schemes as required by NISTIR 7082, Proficiency Test Policy and Plan
(for State Weights and Measures Laboratories), 2004.

## Document Retention

This document has no expiration date and remains in effect until
revised, replaced, or retired.

## References

Compilation of Federal Ethics Laws (approved through May 27, 2003) --
Prepared by the United States Office of Government Ethics. \[Web page\],
<http://www.usoge.gov/pages/laws_regs_fedreg_stats/comp_fed_ethics_laws.pdf>.

Gust, J. C. and Harris, Georgia L. Proficiency Test Policy and Plan (for
State Weights and Measures Laboratories). National Institute of
Standards and Technology, (U.S.) NISTIR 7082, 2004 January \[Web page\]
<http://www.nist.gov/labmetrology>.

Harris, Georgia. L. State Weights and Measures Laboratories Program
Handbook, 4^th^ Edition. National Institute of Standards and Technology
(U.S.) Handbook 143; 2003, March. \[Web page\]
<http://www.nist.gov/labmetrology>.

Harris, Georgia. L. State Weights and Measures Laboratories: State
Standards Program Description and Directory. National Institute of
Standards and Technology (U.S.) NIST SP 791, 2000 Edition; 2000 August.
\[Web page\],
<http://ts.nist.gov/ts/htdocs/230/235/labdirectory/sp791.htm> \[last
updated 10/1/02\].

ILAC-G13:2000, Guidelines for the Requirements for the Competence of
Providers of Proficiency Testing Schemes. International Laboratory
Accreditation Cooperation, 2000.

ISO/IEC Guide 43-1:1997, Proficiency Testing by Interlaboratory
Comparisons -- Part 1: Development and Operation of Proficiency Testing
Schemes. International Organization for Standardization, 1997.

ISO/IEC 17025:1999, General Requirements for the Competence of Testing
and Calibration Laboratories, International Organization for
Standardization, 1999.

National Conference of Standards Laboratories (NCSL) Recommended
Practice, RP-15, Guide to Interlaboratory Comparisons, 1999 March.

*NIST/SEMATECH e-Handbook of Statistical Methods*, \[Web page\],
<http://www.itl.nist.gov/div898/handbook/>, National Institute of
Standards and Technology (U.S.) Handbook 151, 2002 November.

## Definitions

Analyst -- Assigned person who is responsible for handling all data
related activities for the Proficiency Test or Interlaboratory
Comparison (PT/ILC). (As used in this publication "PT/ILC" is used
generically to mean either a Proficiency Test or an Interlaboratory
Comparison; the specific objectives and context will determine the
application and use of PT/ILC results.)

Coordinator -- A party who is responsible for implementation of the
PT/ILC.

Delegate -- Any party outside of NIST, Technology Services, WMD who
volunteers or accepts a task related to the PT/ILC from the Technical
Manager who works with the WMD to accomplish its proficiency testing
objectives, whether volunteer or contractor. Delegates may include:
analysts, coordinators, pivot laboratories, reference laboratories, and
regional administrators.

Interlaboratory Comparison -- a generic term for the circulation of an
artifact, standard, or set of standards for the purpose of evaluating
measurement results.

NIST POC -- Assigned point of contact within NIST WMD for the PT/ILC.

Participant -- Any organization meeting RMAP (see definition)
requirements for participation in a PT/ILC.

Pivot Laboratory -- Laboratory organization that determines stability
value for the artifact.

Provider -- NIST Weights and Measures Division is considered the
"provider."

Proficiency Test -- A specific interlaboratory comparison that may be
used by a laboratory or Recognition and/or Accreditation bodies as
evidence of staff competence.

Reference Laboratory -- Laboratory organization that determines the
reference value for the artifact.

RMAP -- Regional Measurement Assurance Program.

Regional Administrator -- Regional coordinator who assists in tracking
the status of all PT/ILCs within the region.

Technical Manager -- The Technical Manager is the Group Leader for the
WMD Laboratory Metrology Group.

# ADMINISTRATION

## Quality System & Policy

1.  This document establishes, implements and maintains the quality
    system for proficiency testing and interlaboratory comparison
    activities of NIST WMD for the scope of Measurements that are
    defined in section 1 of NISTIR 7082, Proficiency Test Policy and
    Plan (for State Weights and Measures Laboratories).

2.  This document defines and documents the WMD's policy, objectives and
    commitment to ensuring and maintaining quality of all aspects of
    proficiency testing. Artifact quality, characterization, and
    assignment of property values are described in *PLANNING*.
    Evaluation of each laboratory's performance and statistical
    treatment of test results are described in *ANALYSIS.* Distribution
    of artifacts, storage and transport procedures are described in
    *IMPLEMENTATION*. Reporting procedures are described in *REPORTING*.

3.  Quality Policy: NIST WMD pledges to provide proficiency testing
    services using metrologically sound practices and the highest levels
    of service quality as documented in this quality system. All staff
    and delegates associated with proficiency testing services pledge to
    adhere to the policies and procedures documented in this quality
    system, performed in accordance with ILAC G13:2000, ISO/IEC Guide
    43-1, and relevant sections of ISO/IEC 17025:1999.

4.  The locations of the applicable sections of ILAC G13 are as follows:

  -------------- ---------------------------------------- --------------------
  ILAC G13                                                Quality Manual
  Requirement,                                            Section
  Section 2.1.3,                                          
  section letter                                          

  a              Aims                                     Introduction

  a              Scope                                    NISTIR 7082

  a              Statistical Design                       Analysis

  a              Format                                   Reporting

  b              Operational Procedures                   Implementation

  c              Preparation and Issuing of Reports       Reporting

  d              Policies on Confidential and Ethical     Administration
                 Procedures                               

  e              Computing and Information Systems        Administration

  f              Collaboration and Sub-contracting        Administration

  g              Fees for Participation                   Administration

  h              Scope of Availability of Proficiency     Regional MAP
                 Testing Programs                         Programs

  i              General Policies on Participation        Administration

  j              Use of Scheme Results                    Administration

  kk             Procedures for Handling Complaints       Administration
  -------------- ---------------------------------------- --------------------

5.  Throughout this document, the responsibilities for all activities
    are defined in each applicable section.

## Organization and Management

6.  The organization providing proficiency tests within the scope of
    this document is NIST, Technology Services, Weights and Measures
    Division (Division 260.)

7.  The Technical Manager and Quality Manger (as assigned by the WMD
    Laboratory Metrology Group Leader) task other personnel within the
    WMD Metrology Group and Delegates to complete all aspects of PT/ILC
    within the scope of this program.

8.  In order to ensure that all management and personnel are free from
    any commercial, financial, internal, or external pressures that may
    adversely affect the quality of PT/ILC activities, all Federal
    Employees are subject to abide by the Federal ethics laws as cited
    in the "Compilation of Federal Ethics Laws". Additionally, all
    appointed Delegates must comply with all elements contained in
    *POLICY FOR PARTICIPATION* which is intended to ensure freedom from
    external pressures. These policies also ensure that all personnel
    involved in PT/ILC activity avoid involvement in activities that
    might diminish the confidence in the competence, impartiality,
    judgment, or operational integrity of NIST WMD.

9.  It is the policy of NIST WMD that all PT/ILC activities are
    completely open, with no coded or confidential information. All
    participants must agree to these conditions before being qualified
    to participate in PT/ILC activities. This document may be used to
    develop PT/ILC activities for the NIST National Voluntary Laboratory
    Accreditation Program (NVLAP) at their request. For this special
    case, access to information which results from PT/ILC activities is
    restricted to a minimum number of people and the reported
    information may be coded to ensure the confidentiality of the
    participants where NVLAP determines that it is necessary.

10. WMD may occasionally coordinate PT/ILC activities for NVLAP or other
    accreditation bodies when it is in the interest of WMD program
    objectives. In these cases, WMD will follow the confidentiality
    policies of the accreditation bodies if requested.

11. NIST WMD has developed organization charts to define its
    Organization and Management structure, its place within NIST, and
    the relations between management, technical operations, support
    services, and delegates. The organization charts also demonstrate
    the responsibility, authority, and interrelationships of all
    personnel who manage, perform or verify work in PT/ILC activities.

## Roles and Responsibilities 

12. The Technical Manager has overall technical responsibility for each
    PT/ILC activity. The Technical Manager or Designee from NIST WMD
    reviews and provides final approval on all PT data, analysis, and
    reports. Final approval of PT data, analysis, and reports may not be
    assigned to personnel outside of NIST WMD.

13. The Technical Manager must have experience with PT/ILC activities
    that are conducted within the scope of NISTIR 7082. The Technical
    Manager must have the educational and technical background required
    to meet a position for a Physical Scientist ZP III within NIST WMD,
    Laboratory Metrology Group. The requisite background includes
    extensive experience in calibration of standards of mass, length,
    and volume; experience in developing calibration procedures and
    methods; participation in, or coordination of, round robins or other
    interlaboratory experiments and comparisons; and experience in
    quality system operations and audits.

14. The Quality Manager has responsibility for all quality related
    aspects of this program such as the maintenance of records, Internal
    Audits, and ensuring adherence to policies and procedures.

15. The Quality Manager must have experience and training as a lead
    auditor in a metrology-related field (e.g., ISO 9000, ISO/IEC
    17025).

16. The Technical and Quality Managers, and their deputies shall be
    assigned by the WMD Laboratory Metrology Group Leader via email or
    memo, which are retained as a correspondence quality record.

## Collaboration and Subcontracting 

17. The Technical Manager may designate activities from the *PLANNING,
    ANALYSIS, IMPLEMENTATION,* and *REPORTING* sections of this document
    to collaborators who will hereby be known as Delegates.

18. Delegates are selected based upon an assessment of technical
    qualifications as determined by the Technical Manager or Designee.
    All Delegates are evaluated using procedures and records described
    in the Training section of the *ADMINISTRATION* section.

19. Laboratories that are selected for establishment of the reference
    value or performance of pivot measurements must be either accredited
    in the measurement parameter of concern by an accreditation body
    that is a signatory to the ILAC, MRA (International Laboratory
    Accreditation Cooperation, Mutual Recognition Arrangement
    [http://www.ilac.org](http://www.ilac.org/)), or recognized for
    compliance to NIST Handbook 143 by NIST for the measurement
    parameter of the PT/ILC, and be able to provide a suitable level of
    measurement uncertainty for the proposed PT/ILC. Records of
    competency for any laboratory used for measurement activities are
    maintained in the WMD files and databases. Laboratory qualification
    may include one or more measurement areas and/or PT/ILCs. Laboratory
    recognition/accreditation status and records must be evaluated for
    accuracy and updated as needed, prior to each assigned PT/ILC.

20. Subcontracting, if coordinated, will be performed with organizations
    that are accredited to perform PT/ILCs and/or comply with this
    quality system.

## Fees

21. NIST does not charge fees for participation in any PT/ILC
    activities. Commercial Laboratories that participate in PT/ILC
    activities are to pay costs of shipping to and from their
    laboratory. This requirement may be waived if they are serving as a
    delegate for the PT/ILC.

> NOTE: Fees charged for attendance at the RMAP meetings are for
> coordination and costs associated with the meeting functions and not
> those of operating the proficiency testing programs. Attendance at the
> meetings, which is required to obtain reports and participation in the
> discussions, will usually require payment of fees associated with the
> cost of the meetings (distinct from the cost of the PT/ILC activity).

## Policy for Participation

22. Any laboratory organization (Participant) that would like to
    participate in a PT/ILC activity within the scope of this document
    must agree to operate in accordance with the policies and conditions
    for participation as described in this quality system as follows:

    1.  The Participant will follow instructions for each PT and use
        designated procedures when instructed.

    2.  The participant must acknowledge that all NIST WMD PT/ILCs are
        considered OPEN and that anonymity is not implied or guaranteed.

    3.  The Participant will not use any PT/ILC report for any purpose
        other than internal measurement assurance or
        accreditation/recognition activities (no sales, marketing, or
        advertising of the results of any participating laboratory).

    4.  The Participant will not falsify any results submitted to NIST
        WMD.

    5.  The Participant agrees to keep details regarding the PT/ILC in
        confidence. It is inappropriate to share information
        (specifically measurement results) regarding the artifacts
        provided in the proficiency test final report or to seek to
        obtain measurement information for artifacts prior to
        participation in the proficiency test (i.e., participants will
        share data only with the coordinator, analyst or NIST staff).

> Exception: Draft E~n~ values or notice of significant error/bias may
> be communicated to individual laboratories, with follow-up monitoring
> by the ILC coordinator or by NIST to enable immediate investigation,
> corrective action, and retesting (if needed).

6.  The participating organization will regularly attend RMAP meetings
    and participate in PT/ILC discussions.

7.  Participants and participating organizations will be technically
    qualified for the measurement parameter of interest.

```{=html}
<!-- -->
```
23. NIST WMD reserves the right to refuse participation to any
    organization, even if they are technically qualified, if the
    participant or participating organization has violated quality
    policies, procedures for care and handling of artifacts, repeated
    technical delays, or other technical issues that could potentially
    adversely affect the results for the other participants. Prior to
    WMD issuing a notice of refusal to a laboratory, due efforts will be
    made to notify participants in this category, provide guidance, and
    ensure that the laboratory has an opportunity for corrective action.

## Document Control

24. Document control is maintained by NIST WMD, with approved documents
    for this quality system published and available for unlimited
    distribution on the World Wide Web at
    [http://www.nist.gov/labmetrology](http://www.xxx.xxxx/) (select
    Proficiency Testing Program). Any paper copies of quality documents
    used for any purpose must match the revision and date of issue as
    the associated document posted at the website. WMD will make general
    announcements to all staff and delegates when updates have been
    made. The approved, controlled copies of publications will be
    electronic and retained on the website. Copies will be retained for
    local use and back-up on NIST or WMD networks.

25. All documentation related to the scope of this quality manual is
    reviewed and approved for use by the Technical Manager prior to
    issue. The master list of approved documents is located at
    <http://www.nist.gov/labmetrology>.

26. All documentation will be technically reviewed for content and
    policy and assigned a publication number as per the requirements of
    NIST's editorial review and approval process, the Washington
    Editorial Review Board (WERB) in Gaithersburg, MD

27. Documentation related to this quality system is reviewed during
    Internal Audits and through regular use by all participants to
    ensure continuing suitability. Documents are also reviewed by
    personnel during each use. Any issues of suitability are
    communicated to the Quality Manager in writing.

28. During the document revision process, any document that has been
    revised will be removed from the website and replaced with the most
    current revision. Historical documents may be maintained on the
    website if they have been suitably marked as "OBSOLETE, REFERENCE
    ONLY" so that the historical nature is obvious if used.

29. Any invalid or obsolete documents maintained by NIST WMD for
    historical purposes are marked as "OBSOLETE, REFERENCE ONLY."

30. Any revisions to documents will be reviewed and approved by the
    Technical Manager. Minor changes to documentation will be described
    in the revision history for the document. Major changes to the
    documents will be identified by the word processing software "track
    changes" function and maintained by the Technical Manager.
    Hand-written changes will not be used.

## Request, Tender or Contract Review

31. NIST WMD program is not a commercial PT/ILC organization. All PT/ILC
    activities occur as a result of WMD direction and/or RMAP
    participation in the development of PT/ILC. The contract review
    function is completed as a result of all members reaching consensus
    in the development of the PT/ILC as documented on the PT/ILC
    Planning Checklist (Form 1) for the RMAP's form. All participants
    must agree to abide by all elements of *POLICY FOR PARTICIPATION*.
    Any individual request for a new member or a member of another RMAP
    to participate in a PT/ILC that has already been developed and is
    operating must be submitted to the NIST WMD office for review,
    documentation and approval by the Technical Manager. Records of the
    PT/ILC Planning Checklist, individual requests to participate, and
    any associated changes are reviewed, approved, and maintained by the
    NIST WMD office.

32. PT/ILCs are communicated to participants outside of the RMAPs,
    usually via email or the NIST WMD website. The NIST WMD website
    provides an informational overview for each PT/ILC and whether or
    not it has openings for individual requests to participate. Special
    requirements for participation are noted/posted where applicable.

33. Any changes in the scheme design or operation of a PT/ILC are
    communicated to all participants in writing. Changes are maintained
    with WMD records for each PT/ILC.

34. When PT/ILC functions are coordinated with or on behalf of NVLAP,
    specific changes to the operation of a PT/ILC that are in conflict
    with this quality manual must be documented and retained on file.

## Procurement of Services and Supplies

35. NIST WMD uses the Federal Acquisition Requirement (FAR) policies and
    procedures for procurement of all services and supplies. The
    Technical Manager or Designee develops and documents technical
    specifications, tolerances, and other requirements for all procured
    services and supplies, which are used in selecting suppliers. Upon
    receipt of items that affect the quality of PT/ILC services
    provided, the items are inspected for conformance to the documented
    technical requirements. For certain artifacts the reference
    laboratory may be required to perform the final inspection in order
    to ensure that the artifact is suitable for use in the PT/ILC. Final
    reports and analyses are reviewed and documented by the Technical
    Manager or Designee. Records of procurement and inspection are
    maintained by NIST WMD.

36. When items are used for PT/ILC activities that do not belong to WMD,
    suitable evaluation and assessment must be conducted and documented
    to ensure that artifacts are of sufficient quality to meet the
    stated objectives of the PT/ILC.

## Client Feedback

37. Feedback from participants is sought during each RMAP meeting, in
    order to improve PT/ILCs that are developed within the scope of this
    document. The PT/ILC Feedback and Complaints Form (Form 4) and the
    WMD Quality System Management Review Form (Form 5) will be used to
    ensure the uniform collection and consolidation of feedback from all
    RMAPs.

38. Complaints and feedback may be recorded by an ILC Coordinator, a
    delegate of the ILC, or the WMD office staff. Records of such
    feedback or complaints and their associated investigation and
    corrective actions are maintained through completion of the PT/ILC
    Feedback and Complaint Form (Form 4) or the WMD
    Nonconformance/Corrective Action/Preventive Action (NCAP) Form (Form
    7). Evaluation of criteria for conformance and non-conformance is
    performed by the Technical Manager in accordance with Section 1.11.

## Control of Nonconforming Activities and Corrective Action

39. In the event that any aspect of activities within the scope of this
    document is identified as nonconformity to the procedures or the
    stated requirements, it is the responsibility of the Technical
    Manager to investigate and determine appropriate corrective action.
    The corrective action process is applied to all identified
    nonconformities and shall be appropriate to the magnitude of the
    problem encountered.

40. All aspects of the nonconformance are recorded on the WMD Form 7
    (NCAP). It is the responsibility of the Technical Manager to:

    1.  investigate and determine root cause of nonconformance;

    2.  evaluate the significance of the nonconformance;

    3.  identify potential corrective actions and select the action most
        likely to eliminate the problem and prevent it from reoccurring;

    4.  ensure work is halted if necessary;

    5.  assign a timescale for completion of corrective action;

    6.  obtain funding if needed; and

    7.  initiate corrective actions as required.

41. If necessary, participants are notified in writing and nonconforming
    materials or reports are recalled. Upon completion of corrective
    actions, the Technical Manager may authorize the resumption of the
    process/scheme.

42. Any corrective actions are monitored by the Quality Manager during
    the Internal Audits to ensure that the corrective action was
    effective.

## Preventive Action

43. All operational procedures are reviewed during the Internal Audit
    and upon each use by all staff and delegates in order to identify
    opportunities for improvement. Any preventive actions and their
    associated action plans are identified and recorded on WMD NCAP Form
    (Form 7). Results of preventive actions are monitored and reviewed
    during Internal Audits and Management Reviews in order to understand
    the effectiveness of the preventive action.

## Records

44. The following types of records are identified and indexed as quality
    records for NIST WMD PT/ILC activities. They are collected at time
    of generation, and stored by marked section in file cabinets or in
    electronic version on computer. The retention times for all
    documents are at least ten years after the completion of the test or
    retirement (or loss) of the artifact or proficiency test. After ten
    years, the Quality Manager may dispose of records at his/her
    discretion.

    1.  Master Document List

    2.  Previous editions of controlled documents and obsolete documents

    3.  Client Feedback

    4.  Artifact History Records and Inventory List

    5.  Customer Complaint/Feedback Log and Information

    6.  Nonconformities

    7.  Corrective Actions

    8.  Preventive Actions

    9.  Proficiency Test/Interlaboratory Comparison Records

    10. Proficiency Test/Interlaboratory Comparison Reports

    11. Internal and External Audits

    12. Management Reviews

    13. Training Schedule and Training Records (i.e., Form 3)

    14. Assignment of Deputy Technical and Quality Manager

    15. Software Validation Records

    16. Correspondence Records

45. The storage facilities for records are at the NIST WMD office.
    Back-up copies of electronic records are maintained on the NIST
    network. NIST's Information Technology (IT) back up services and
    security features are administered following policies and procedures
    set by the NIST Office of Chief Information Officer.

46. The records are stored in the NIST WMD offices so that only
    employees have access to the hard copy records. Electronic copies of
    records are stored on the hard drives and network of NIST WMD
    computers and are appropriately password protected.

47. To prevent unauthorized amendment of data, Electronic Data is
    protected via password protection of operational features for each
    person's PC, and backed up to CD ROM media, to prevent unauthorized
    amendment of data.

48. Technical records contain information as prescribed in *REPORTING*
    and are maintained as a quality record.

## Internal & External Audits

49. An Internal Audit of the quality system related to this document is
    completed annually, as scheduled by the Quality Manager. The audit
    may be performed by sections so that the entire quality system is
    reviewed annually.

50. The Quality Manager is primarily responsible for the planning and
    conduct of the Internal Audit. The schedule for the Internal Audit
    will be determined at the time of the Management Review meeting. The
    Quality Manager may designate individuals to perform parts of the
    Internal Audit and will ensure the training and qualifications of
    such individuals. Internal Audit Results are collected and
    summarized on the WMD Internal Audit Form (Form 6).

51. The Internal Audit shall review all elements and activities
    associated within the scope of this document. During the Internal
    Audit, all documents and program elements are reviewed for
    continuing effectiveness and potential for improvement. Any
    nonconformities identified throughout the previous year are reviewed
    for closure and effectiveness. Any nonconformities identified during
    the Internal Audit will be addressed by following the policy in
    ADMINISTRATION - Control of Nonconforming Activities and Corrective
    Action.

52. Any External Audits that are authorized and conducted will follow
    appropriate or applicable standards as determined by the Technical
    and/or Quality Manager. Such records will be retained with other
    auditing records and discussed during Management Reviews.

## Management Review 

53. RMAP PT/ILC Quality System Review

    1.  NIST WMD conducts management reviews of the quality system and
        PT/ILC activities as applicable within each RMAP during the
        annually scheduled meeting of the RMAP. Any suggestions for
        changes or improvements are summarized in report form (Forms 4
        and 5) for discussion at the annual Management Review meeting.

54. NIST WMD PT/ILC Quality System Review

    1.  NIST WMD conducts a Management Review meeting of the quality
        management system annually, as scheduled by the Technical
        Manager. The Management Review meeting discusses:

        1.  Suitability and effectiveness of the Quality System

        2.  The Quality and applicability of PT/ILC Activities

        3.  Reports from RMAP meetings (Forms 4 and 5)

        4.  Reports from Internal Audits (Form 6)

        5.  Reports of External Audits (e.g., NVLAP, A2LA, L.A.B., other
            requested body)

        6.  Client feedback/complaints

        7.  Resources (facilities, standards, and equipment)

        8.  Outstanding items from previous Management Review

        9.  Staff Training

        10. Other relevant reports

    2.  The minutes of the Management Review meeting are recorded and
        any actions or improvement suggestions are assigned to
        appropriate personnel with an estimated completion date to
        ensure completion within an appropriate and agreed upon
        timescale.

## Management Staffing and Training

55. Training

    1.  The Technical Manager has responsibility for the management of
        all PT/ILC activities associated with NIST WMD. The Technical
        Manager is responsible for ensuring that all personnel who have
        any responsibilities associated with any PT/ILC activities are
        sufficiently trained.

    2.  The Coordinator, Delegate, and Analyst positions as assigned for
        PT/ILC activities must have adequate training to ensure that
        they have the technical capability to complete the duties as
        assigned. It is the responsibility of the Technical Manager to
        annually develop a training plan and ensure that essential
        training is conducted to ensure the competency of all personnel
        involved in the PT/ILC activities.

    3.  Participant training and qualifications shall be documented
        using the PT/ILC Training and Qualifications form (Form 3) to
        verify adequate training and ensure that they possess the
        technical capability to participate within specific PT/ILC
        parameters.

    4.  Records of training for all personnel involved with PT/ILC
        management activities are maintained using Form 3 and database
        developed and maintained by NIST WMD for training records.

56. Staffing

    1.  The Technical Manager is responsible for ensuring sufficient
        staffing for each PT/ILC and that all personnel involved in
        PT/ILC are qualified. The Technical Manager reviews the training
        records of all personnel involved to ensure the effective
        management of all PT/ILC activities. Records of training for all
        personnel involved with PT/ILC management activities are
        maintained using a database developed and maintained by NIST WMD
        for training records.

## Collusion and Falsification of Results

57. Each PT/ILC activity is designed to ensure, as is practicable, that
    the possibility for collusion and falsification of results is
    minimized. It is the responsibility of the PT/ILC Coordinator to
    monitor the activities of the PT/ILC and to ensure that there is no
    communication of the assigned values to or among participants until
    the results of the PT/ILC activity have been collated in a draft
    report. In addition to the PT/ILC coordinators monitoring activities
    to this end, each participant in PT/ILC activities within the scope
    of this document is required to comply with all elements of *POLICY
    FOR PARTICIPATION*.

## Data Processing Equipment

[]{.mark}

58. NIST WMD utilizes personal computers, laptop computers, and
    scientific calculators for data entry and statistical analysis of
    PT/ILC activities, all of which are adequate for the assigned task.
    Deficiencies noted with data processing equipment are brought to the
    attention of the Technical Manager, and are addressed during
    Internal Audits and Management Reviews as appropriate.

59. The IT activities of NIST or the IT Department of Delegate
    organizations are responsible for the effective operation of laptop
    and personal computers. Calculators are a disposable item and are
    replaced by the user if any abnormal operation is detected. Use of
    appropriate significant digits is considered in data evaluation and
    addressed as needed during ongoing training of staff/delegates.

[]{.mark}

60. Commercial Software programs such as Microsoft Excel do not require
    further validation. Sub-programs such as worksheets for the
    analysis, evaluation, and reporting of results for activities within
    the Scope of this document require validation before use as per the
    PT/ILC Software Validation Procedure (Procedure 1).

[]{.mark}

61. Records of objective evidence for the validation and revision date
    of the software are maintained at NIST WMD by the Quality Manager.
    The distribution of any such software is accomplished through the
    NIST WMD website, by email, or by sharing of files through other
    electronic means. The software on the NIST WMD website is the master
    copy of the software and it is backed up routinely by NIST IT
    procedures.

# PLANNING

The general PT/ILC requirements are plans developed in accordance with
NISTIR 7082, during Management Review and RMAP meetings.

A PT/ILC begins with the planning activities performed by an advisory
group. This group may be assembled by the Technical Manager for national
level PT/ILC activities or may consist of an RMAP group (or subgroup).
The advisory group plans the PT/ILC activity by completing the PT/ILC
Planning Checklist Form (Form 1).

## Technical Protocol Development

62. The Technical Manager or Delegate develops the Technical Protocol
    for the PT/ILC based upon the information obtained in the associated
    PT/ILC Planning Checklist form (Form 1). The Technical Protocol
    shall contain the following information, as relevant:

-   Environmental Conditions for Realizing the Measurement Results for
    the Artifact;

-   Environmental Conditions for Storage and Transport for the Artifact;

-   Preparatory Information for the Artifact(s);

-   Packaging and Shipping Information of the Artifact and tracking
    requirements;

-   Communication to the participants to treat the artifact in the same
    manner as routine measurements for the method prescribed or other
    specified instructions on handling;

-   Instructions of the measurement method to be used in the PT/ILC;

-   Reporting Requirements for Participating Laboratories:

    -   Units of measurement;

    -   Number of significant digits;

    -   Date of measurement;

    -   Reporting reference/basis of measurement (e.g., environmental
        conditions, reference temperature);

    -   Measurement value and its associated expanded uncertainty (the
        reported measurement uncertainty must be the actual calculated
        uncertainty, not just a transfer ratio);

    -   Uncertainty analysis (components included or not included); and

    -   The laboratory report on the method used if different than
        prescribed in the Technical Protocol; and

-   Directions on how to return or submit the data.

# IMPLEMENTATION

## Preparation of the Artifact

1.  The artifact(s) is/are selected (as described in Section 1.9) for
    the PT/ILC based upon requirements documented in the PT/ILC Planning
    Checklist and *Procurement of Services and Supplies*. The
    artifact(s) may be loaned from organizations such as NIST WMD or
    other laboratories as long as appropriate planning is taken to
    ensure that the reference value has not been previously known by
    participating laboratories.

2.  The Reference Laboratory (where used) and Pivot Laboratory (where
    used) are qualified based upon information obtained in the PT/ILC
    Planning Checklist. Records of qualification for the Reference
    Laboratory and Pivot Laboratory are created per *Collaboration and
    Subcontracting* (Section 1.4).

3.  The Technical Manager will develop any additional processes to
    determine the stability of the artifact as necessary. The
    artifact(s) is/are then sent to the Reference Laboratory for the
    establishment of its reference value(s) or to the Coordinating
    Laboratory where the reference value is selected using alternative
    methods.

4.  The Technical Manager or delegate reviews qualifications for
    potential participating laboratories via training databases
    maintained by NIST WMD and/or completed on the PT/ILC Planning
    Checklist (Form 1).

5.  The PT/ILC Coordinator then develops a schedule for the PT/ILC
    activity based upon the final list of participating laboratories and
    the number of pivot measurements (when used). The schedule may be
    communicated and tracked in the most convenient and practical way
    for the PT/ILC Coordinator.

6.  The PT/ILC is initiated according to the schedule.

## Packaging/Shipping/Handling

7.  The PT/ILC Coordinator is responsible for shipping the artifact(s)
    using appropriate packaging methods for the particular artifact(s),
    along with any additional considerations that were identified in the
    PT/ILC planning checklist Planning Checklist (Form 1).

8.  The PT/ILC Coordinator and each Participating Laboratory is
    responsible for ensuring that:

-   Standards are appropriately packaged;

-   Shipping containers are appropriately labeled; PT/ILC instructions
    are securely attached to the product packaging of individual units
    and designed to remain legible and intact for the period of use in
    the PT/ILC;

-   The PT/ILC instructions and forms should generally be included in
    the shipping container. A minimum of a packing list and the WMD
    shipping address will be included with all PT/ILC artifacts.

-   Standards are shipped by an appropriate commercial carrier or by
    other methods as identified in the ILC Schedule (Form 8) & shipping
    instructions; and

-   Standards are protected from adverse environmental conditions (i.e.,
    protection of heat, cold, x-rays) as appropriate.

-   Any damage or loss sustained by any standards, artifacts, or
    shipping containers, is immediately reported to NIST/WMD.

## Materials Handling and Storage

1.  The PT/ILC Coordinator is responsible for short-term storage of
    artifacts during gaps in the schedule, and the Technical Manager is
    responsible for long-term storage of artifacts. Whether artifacts
    are in long term or short term storage, the responsible party is
    accountable for identification, preservation, and segregation of all
    artifacts from incompatible environments or materials such as high
    humidity, uncontrolled temperatures, or chemicals and other items so
    that the artifacts do not deteriorate during storage.

2.  The Technical Manager maintains the inventory list of artifacts in
    long‑term storage and is responsible for developing schedules of
    periodic inspection for possible deterioration as appropriate for
    the artifacts.

# ANALYSIS

## Determination of Assigned Reference Values and its Associated Uncertainty

3.  The method for determining the assigned value is documented on the
    PT/ILC Planning Checklist (Form 1) prior to beginning the PT/ILC
    activity. The method selected must be based upon sound metrological
    and statistical principles. Documents listed in the *REFERENCES*
    section of this manual may be utilized, but this list is not
    necessarily an all-inclusive list. Any determination of the
    reference value must include consideration of stability of the
    artifact as each participant performs measurements. The Expanded
    Uncertainty of the PT/ILC will be the combination of the reference
    value uncertainty and the uncertainty associated with stability.
    Some examples used for determining reference values may be:

-   The Reference Laboratory (such as NIST or an accredited laboratory)
    measured value (stability may be determined by Pivot Laboratory
    measurements);

-   The mean of the Reference Laboratory measured values from the
    beginning and end of the PT/ILC;

-   The median of the results from accredited laboratories;

-   The median of one result from each participating laboratory; or

-   The adjusted median of the participating laboratories

> Documented procedures for selecting reference values will be used.

[]{.mark}

## Performance Evaluation of Participants

[]{.mark}

1.  The method of evaluating performance for the participants and
    acceptable limits of performance is documented on the PT/ILC
    Planning Checklist (Form 1) prior to beginning the PT/ILC activity.
    The method selected must be based upon sound metrological and
    statistical principles. Documents used in the *REFERENCES* section
    of this document manual may be utilized, but this list is not
    necessarily all-inclusive. Some examples of determining performance
    of the Participant may include:

-   Determination of Normalized Error value, E~n;~ (E~n~ \> 1 is
    considered a failure)

    -   While an E~n~ value \> 1 is considered a failure; the analyst
        may recommend additional follow up for an E~n~ value between 0.5
        and 1.

-   Precision Tests (The reported uncertainty must be less than
    specified limits for some applications);

-   Z score value;

-   Participant measurement is within the expanded uncertainty of the
    PT/ILC;

-   Participant value shows a bias even though their reported
    uncertainty is large enough to pass the E~n~ analysis; and

-   Participant measurement result(s) and associated expanded
    uncertainty is within the expanded uncertainty of the PT/ILC.

-   Expanded uncertainty (variations due to laboratory capabilities,
    bias, uniformity of methodology employed, etc.)

    1.  When assessing the technical competency of a participating
        laboratory, appropriate technical specialists and statisticians
        (or technical approaches and statistical tools) are used as
        deemed appropriate by the Technical Manager. Each PT/ILC report
        will contain both individual and summary analyses.

    2.  Other Performance Analysis issues to evaluate may be:

```{=html}
<!-- -->
```
-   Overall performance against prior expectations, taking measurement
    uncertainties into account;

-   Variation within and between laboratories, between RMAP results, and
    comparisons with any similar previous schemes or published precision
    data; and

-   Variation between methods, procedures, or equipment used if
    applicable;

-   Evaluation of a laboratory's timeliness in conducting and reporting
    their PT/ILC results; and

-   Evaluating the laboratory's calibration report for compliance with
    ISO 17025 (Section 5.10) criteria.

## Outlier Detection Techniques

1.  The NIST/Sematech e-Handbook of Statistical Methods provides
    guidance on outlier detection or other special analysis techniques.
    Results of PT/ILC activities within the scope of this document are
    not discarded. Results that could be considered outliers, or results
    from laboratories performing measurements outside of their normal
    scope of recognition, may not be used for statistical summaries, but
    the information is included in draft and final reports. Any suspect
    data will be verified by the Technical Manager or Designee to ensure
    that the data entry and transcriptions were performed correctly and
    that any data entry errors were identified, documented, and
    corrected. Data entry errors by participants are identified by
    feedback or reports issued by NIST WMD to the participant (or within
    the report), who will use their corrective action process for root
    cause analysis and response. The Technical Manager or Designee is
    responsible for follow-up of issued nonconformities.

## Assessing the Stability of the Artifact

2.  The method for determining artifact stability is documented on the
    PT/ILC Planning Checklist (Form 1) prior to beginning the PT/ILC
    activity. The method selected must be based upon sound metrological
    and statistical principles. Documents described in the *REFERENCES*
    section of this document manual may be used, but this list is not
    necessarily all-inclusive. Limits of acceptable stability for the
    artifact are also determined and recorded on Form 1 prior to
    beginning the activity. It is the responsibility of the Technical
    Manager to communicate the limits of artifact stability to the Pivot
    Laboratory so that they can evaluate the artifact stability during
    the PT/ILC and alert the Technical Manager if stability limits have
    been exceeded. If established limits have been exceeded, it is the
    responsibility of the Technical Manager to determine how to proceed.

3.  Some examples of determining stability may include:

-   The measured deviation between the opening and closing measurements
    performed by the Reference Laboratory;

-   The standard deviation of the Pivot Laboratory measurements;

-   The uncertainty of linear regression of the Pivot Laboratory
    measurements; and

-   The standard deviation of participant measurements.

## Assessing Homogeneity of Artifact

1.  If homogeneity is determined to be applicable for the PT/ILC
    activity, the assessment of homogeneity is performed after the test
    material has been packaged in its final form and before distribution
    to participants unless, for example, stability studies indicate that
    it should be stored in bulk form. In some cases, an intermediate
    homogeneity check may be necessary, for example, before sealing into
    appropriate packaging.

2.  The provider or its collaborators, where appropriate, shall use a
    statistically random selection of a representative number of samples
    from a batch of test material to assess the homogeneity of the
    material. This assessment procedure shall be documented and
    conducted in accordance with acceptable statistical designs, for
    example, analysis of variance on replicate results under
    repeatability conditions. In the case of measurement artifacts,
    preliminary stability checks shall be made and periodic checks of
    assigned property values should be carried out throughout the course
    of the scheme.

3.  If appropriate, limits of acceptable homogeneity for the artifact
    are also determined and recorded on the PT/ILC Planning Checklist
    (Form 1) prior to beginning the activity.

4.  It is the responsibility of the Technical Manager to communicate the
    limits of artifact homogeneity to the Pivot Laboratory so that they
    can evaluate the artifact homogeneity during the PT/ILC and alert
    the Technical Manager if the established limits have been exceeded.
    If established limits have been exceeded, it is the responsibility
    of the Technical Manager to determine how to proceed.

# REPORTING

## Schedules

5.  The PT/ILC Coordinator develops and publishes Schedule & Shipping
    instructions based upon the information gathered in the PT/ILC
    Planning Checklist (Form 1). The schedule will be sent to all
    participants and may be posted at the NIST WMD website.

## Preliminary Reports

6.  Preliminary or Draft Reports may be issued to the participant(s) at
    the discretion of the Technical Manager. Since such reports may be
    requested before the closing measurement of the reference
    laboratory, the Analyst will need to perform evaluations based upon
    a given reference value or the opening measurement on the artifact
    by the Reference Laboratory and the evaluation of stability
    performed thus far in the PT/ILC activity. All participants are to
    review their preliminary report to ensure that all data on the
    report corresponds to the data that was submitted by the laboratory.

## Final Reports

7.  The content of the Final Report will vary depending on the purpose
    of a particular scheme, but each report shall be clear and
    comprehensive and include data on the distribution of results from
    all participants, together with an indication of the performance of
    individual participants. Final Reports are delivered prior to or at
    the RMAP meeting for discussion and comments.

8.  Final Reports issued for PT/ILC activities within the scope of this
    document shall contain:

-   Elements contained in the PT/ILC Reporting Checklist (Form 2);

    -   Any other suggestions, recommendations or general comments;

    -   Conclusions;

    -   Comments on the performance both of individual participants and
        on participation as a whole; and

    -   Possible sources of error (with reference to extreme results)
        and suggestions for improving performance.

# APPENDIX A - FORMS

  ------------ ------------------------------------------------------------
  **Number**   **Title**

  1            PT/ILC Planning Checklist

  2            PT/ILC Reporting Checklist

  3            PT/ILC Training & Qualifications

  4            PT/ILC Feedback & Complaint

  5            WMD Quality System Management Review

  6            WMD Internal Audits

  7            WMD Nonconformance, Corrective Action, Preventive Action
               (NCAP)

  8            Schedule & Shipping (nonmandatory)
  ------------ ------------------------------------------------------------

# Form 1: PT/ILC Planning Checklist

*Page 1 of 3*

  ----------------------------------- -----------------------------------
  ILC Region & Planned Reporting      
  Year:                               

  ILC Title:                          

  ILC Coordinator Name:               

  ILC Coordinator Address:            

  Names, position, and addresses of   
  any other personnel involved in     
  design and operation of ILC:        

  Form Completed by:                  
  ----------------------------------- -----------------------------------

  -----------------------------------------------------------------------
  Nature and Purpose of ILC scheme:

  -----------------------------------------------------------------------

  ----------------------------------- -----------------------------------
  Identification of the Measurand:    

                                      
  ----------------------------------- -----------------------------------

  ----------------------------------- -----------------------------------
  Identification and description of   
  Artifact(s):                        

  Description of nature of Artifact   
  and why it was chosen:              

                                      

                                      

  Artifact source (owner):            
  ----------------------------------- -----------------------------------

  -----------------------------------------------------------------------
  Describe any stability issues with the artifact:

  -----------------------------------------------------------------------

  -----------------------------------------------------------------------
  Identification of the test method used by the participants:

  -----------------------------------------------------------------------

  -----------------------------------------------------------------------
  Identification of the type of equipment required by the participants:

  -----------------------------------------------------------------------

  -----------------------------------------------------------------------
  Identification of the approximate uncertainty required by the
  participants:

  -----------------------------------------------------------------------

  -----------------------------------------------------------------------
  Ranges of values expected from the participant measurements:

  -----------------------------------------------------------------------

**Form 1: PT/ILC Planning Checklist**

***Page 2 of 3***

  -----------------------------------------------------------------------
  Any potential difficulties in preparation and maintenance of
  homogeneous materials or

  the provision of a stable reference value for the artifact:

  -----------------------------------------------------------------------

  --------------------------------------- -------------------------------
  Describe any homogeneity issues, as     
  applicable:                             

  --------------------------------------- -------------------------------

  -----------------------------------------------------------------------
  Describe any environmental issues with the artifact for storage,
  transport, or measurement:

  -----------------------------------------------------------------------

  -----------------------------------------------------------------------
  Describe any special preparation issues for Artifact(s) as applicable:

  -----------------------------------------------------------------------

  -----------------------------------------------------------------------
  Packaging/Handling issues, Special Containers needed?

  -----------------------------------------------------------------------

  -----------------------------------------------------------------------
  The Criteria for participation in the ILC:

  -----------------------------------------------------------------------

  ----------------------------------- -----------------------------------
  Reference Laboratory:               

  ----------------------------------- -----------------------------------

  ----------------------------------- -----------------------------------
  Pivot Laboratory:                   

  ----------------------------------- -----------------------------------

+-----------------------------------+-----------------------------------+
| Identification of Expected        |                                   |
| Participants:                     |                                   |
|                                   |                                   |
| (identify laboratory and          |                                   |
| personnel to be involved in       |                                   |
| measurement):                     |                                   |
+-----------------------------------+-----------------------------------+
| Are all laboratories and          | yes no (if no, explain)           |
| personnel qualified to            |                                   |
| participate (review NIST WMD      |                                   |
| training databases)               |                                   |
+-----------------------------------+-----------------------------------+

  ----------------------------------- -----------------------------------
  Design of test (i.e., petal, loop,  
  etc.):                              

  Describe when artifact is checked   
  and how it is distributed:          
  ----------------------------------- -----------------------------------

**Form 1: PT/ILC Planning Checklist**

*Page 3 of 3*

  ----------------------------------- -----------------------------------
  Number of significant figures to    
  which results are to be reported:   

  Precision of test:                  

  Smallest differences expected       
  between participants:               
  ----------------------------------- -----------------------------------

  -----------------------------------------------------------------------
  Statistical Method for determining reference value of artifact:

  Method of performance evaluation:

  Acceptable limits of stability (from pivot measurements and overall):

  Acceptable limits of homogeneity (if applicable):

  -----------------------------------------------------------------------

# Form 2: PT/ILC Reporting Checklist

*Page 1 of 2*

  ----------------------------------- -----------------------------------
  PT/ILC Identification:              

  Coordinator:                        

  Date of Completion:                 
  ----------------------------------- -----------------------------------

Ensure that Final Report Contains the Following

  ---------------------------------------------------- ------------------
  Title Page or Introduction                           

  General Description and title of the ILC             

  Names and affiliations of persons involved in design Included
  and conduct of PT/ILC                                

  Date of issue of the report                          Included

  Report number and clear identification of the PT/ILC Included

  Dates (Time period of conduct of test from beginning Included
  to end)                                              

  RMAP group                                           Included

  Who                                                  

  Reference Laboratory                                 Included

  Pivot Laboratory                                     Included

  Participants (include address as appropriate)        Included

  What and Why                                         

  Objectives                                           Included

  Parameter, Range, Uncertainty (e.g., mass, 1 kg,     Included
  best calibration)                                    

  Clear description of the items or materials used,    Included
  including, where appropriate, details of sample      
  preparation and homogeneity testing                  

  Details of the traceability and uncertainty of       Included
  assigned values                                      

  How                                                  

  Summary of procedures used to design and implement   Included
  the scheme (which may include reference to a scheme  
  protocol)                                            

  Procedures Used                                      Included

  Laboratory Equipment Identification (e.g., which     Included
  balance, which standards, for each lab)              

  Analysis                                             

  Summary of procedure used to establish any assigned  Included
  value                                                

  Summary of procedures used to statistically analyze  Included
  the data                                             

  Software title and version used in data analysis     Included
  ---------------------------------------------------- ------------------

**Form 2: PT/ILC Reporting Checklist**

*Page 2 of 2*

+----------------------------------------------------+-----------------+
| Tables & Graphs                                    |                 |
+----------------------------------------------------+-----------------+
| Laboratory test results                            | Included        |
+----------------------------------------------------+-----------------+
| Statistical data and summaries, including assigned | Included        |
| values of Artifact(s), and range of acceptable     |                 |
| results                                            |                 |
+----------------------------------------------------+-----------------+
| Assigned values and summary statistics for test    | Included N/A    |
| methods/procedures used by other participants (if  |                 |
| different methods are used by different            |                 |
| participants)                                      |                 |
+----------------------------------------------------+-----------------+
| Tables of data, precision tests, and evaluation    | Included        |
| data                                               |                 |
+----------------------------------------------------+-----------------+
| Graphs of data, standard value/uncertainty         | Included        |
| charges, Youden plots, misc. graphs                |                 |
+----------------------------------------------------+-----------------+
| Conclusions                                        |                 |
+----------------------------------------------------+-----------------+
| Comments on participants\' performance by the      | Included        |
| provider and technical advisors                    |                 |
+----------------------------------------------------+-----------------+
| Summarization of Results                           | Included        |
+----------------------------------------------------+-----------------+
| Recommended follow-up and corrective action (by    | Included        |
| analyst/by NIST)                                   |                 |
+----------------------------------------------------+-----------------+
| Comments on any technical difficulties raised by   | Included N/A    |
| participants                                       |                 |
+----------------------------------------------------+-----------------+
| Evaluation of the responses of participants        | Included N/A    |
| performing poorly                                  |                 |
|                                                    |                 |
| (if feedback is required)                          |                 |
+----------------------------------------------------+-----------------+
| Advice, where appropriate, on the interpretation   | Included N/A    |
| of the statistical analysis                        |                 |
+----------------------------------------------------+-----------------+

  ---------------------------------------------------- ------------------
  Return to NIST WMD For Archival -- Copies and/or     
  originals of:                                        

  Instructions                                         Included

  Schedules                                            Included

  Datasheets                                           Included

  Formal Calibration Report (where requested)          Included

  Submitted Data                                       Included

  Analysis and electronic Documents/spreadsheets       Included
  ---------------------------------------------------- ------------------

# Form 3: PT/ILC Training & Qualifications

*(may be retained in electronic database)*

*Page 1 of 3*

  ----------------------------------- -----------------------------------
  Name:                               

  Laboratory:                         
  ----------------------------------- -----------------------------------

Formal Education

  ----------------- ---------- -------- ------------------- ------------- -------
  SCHOOL ATTENDED   ATTENDED            COURSES COMPLETED   CERTIFICATE   
                                                            AWARDED       

                    FROM       TO                           YES           NO

                                                                          

                                                                          

                                                                          

                                                                          

                                                                          

                                                                          

                                                                          

                                                                          

                                                                          

                                                                          

                                                                          

                                                                          

                                                                          
  ----------------- ---------- -------- ------------------- ------------- -------

Technical Seminars and Workshops Attended

  ----------------------- -------------------- ------------- ------------- ------
  DATE & PLACE ATTENDED   NAME OF              Number of     CERTIFICATE   
                          SEMINAR/WORKSHOP     hours of      RECEIVED      
                          ATTENDED             instruction                 

                                                             YES           NO

                                                                           

                                                                           

                                                                           

                                                                           

                                                                           

                                                                           

                                                                           
  ----------------------- -------------------- ------------- ------------- ------

**Form 3: PT/ILC Training & Qualifications**

*(may be retained in electronic database)*

*Page 2 of 3*

  ------------------------- ----------- ------------- --------------------
  Metrology Discipline      Training    Years of      Approved FOr PT/ILC
                            Level       Experience in Functions
                                        This          
                                        Discipline    

  Mass                                                

  **Echelon I**                                       

  **Echelon II**                                      

  **Echelon III**                                     

  Mass, Fluid Flow (Volume)                           

  **Echelon I Gravimetric                             
  -- Large Prover**                                   

  **Echelon II Gravimetric                            
  -- Small Volume                                     
  (Glassware)**                                       

  **Echelon II Volumetric                             
  -- Large Volume                                     
  Transfer**                                          

  **Echelon II Volumetric                             
  -- Small Volume                                     
  Transfer**                                          

  Length, Dimensional,                                
  Linear                                              

  **Length Rulers**                                   

  **Length Tape**                                     

  Temperature                                         

  **Echelon I**                                       

  **Echelon II**                                      

  **Echelon III**                                     

  **Echelon IV**                                      

  **Echelon V**                                       

  Misc.                                               

  **Hydrometers**                                     

  **Tuning Forks**                                    

  **Stop Watches and                                  
  Timers**                                            

  **Hygrometer/Relative                               
  Humidity**                                          

  **Grain Moisture**                                  
  ------------------------- ----------- ------------- --------------------

  ------------------------- ---------------------------------------------
  Reviewed and Approved By: 

  Date of Review:           
  ------------------------- ---------------------------------------------

*See Training Level Codes on the next page.*

**Form 3: PT/ILC Training & Qualifications**

*(may be retained in electronic database)*

*Page 1 of 3*

**TRAINING LEVEL CODE LEGEND:**

**BLANK:** No experience in the particular field

**C:** Introductory Experience such as on-the-job training (OJT),
Computer Based Instruction, Informal Instruction, or Formal Instruction
that is over ten years old.

**B:** Intermediate Experience such as formal instruction provided in
house or by a NIST WMD approved metrology program.

**A:** Formal Instruction provided by a recognized subject matter expert
such as original equipment manufacturer (OEM) training, or NIST
seminars.

**APPROVED FOR PT/ILC FUNCTIONS LEGEND:**

The qualifications listed below are in ascending order of ranking (i.e.,
if the person is qualified as a coordinator, they are also qualified as
a participant, but are not necessarily qualified as an analyst or for
reports. A person qualified for reports is qualified for all PT/ILC
activities for the parameter.

**BLANK:** Not approved for PT/ILC activities for this parameter.

**PARTICIPANT:** Approved as a participant for parameter.

**COORDINATOR:** Approved as a PT/ILC coordinator for the parameter.

**ANALYST:** Approved as PT/ILC analyst for the parameter.

**REPORTS:** Approved to develop reports under the supervision of the
Technical Manager for the parameter.

# Form 4: PT/ILC Feedback and Complaints

*Page 1 of 1*

+-----------------------------------+-----------------------------------+
| **Instructions:** Use this form   |                                   |
| to record any complaint or        |                                   |
| pertinent communication received  |                                   |
| about NIST WMD's services as they |                                   |
| apply to the conduct of PT/ILC    |                                   |
| activities. If the complaint or   |                                   |
| comment is received in writing,   |                                   |
| attach the complaint to this      |                                   |
| record. If the complaint or       |                                   |
| comment is received by telephone, |                                   |
| fill out sections 4 and 5         |                                   |
| completely.                       |                                   |
+-----------------------------------+-----------------------------------+
| 1\. Name of Person completing     | 3\. Check one of the following    |
| form                              |                                   |
|                                   | Complaint/communication received  |
|                                   | in writing -- copy of             |
|                                   | documentation attached            |
|                                   |                                   |
|                                   | Complaint/communication received  |
|                                   | by telephone -- was caller        |
|                                   | requested to send documentation?  |
|                                   |                                   |
|                                   | yes no                            |
+-----------------------------------+-----------------------------------+
| 2\. Date of receipt of complaint  |                                   |
| or communication                  |                                   |
+-----------------------------------+-----------------------------------+
| 4\. If the                        |                                   |
| complaint/communication was       |                                   |
| received by telephone, provide    |                                   |
| the following information:        |                                   |
|                                   |                                   |
| Name and Address of person making |                                   |
| complaint/communication           |                                   |
|                                   |                                   |
| Phone number Fax number           |                                   |
+-----------------------------------+-----------------------------------+
| 5\. If the                        |                                   |
| compliant/communication was       |                                   |
| received by telephone, describe   |                                   |
| the issue:                        |                                   |
+-----------------------------------+-----------------------------------+
| 6\. Name of Responsible Manager:  |                                   |
+-----------------------------------+-----------------------------------+
| 7\. For Quality Manager Use Only  |                                   |
+-----------------------------------+-----------------------------------+
| Date Logged:                      | Complaint Number:                 |
+-----------------------------------+-----------------------------------+
| Date C.A.R. issued (if            | Date Closed:                      |
| applicable):                      |                                   |
+-----------------------------------+-----------------------------------+

# Form 5: WMD Quality System Management Review 

*Page 1 of 2*

  ----------------------------- -----------------------------------------
  Date:                         

  Location:                     

  Personnel Present:            

  Agenda                        Record of Discussion, and Assigned
                                Actions, and Estimated Completion Dates

  Suitability and Effectiveness 
  of Quality System             

  Quality and Applicability of  
  PT/ILC Activities             

  RMAP Meeting Reports          

  Report of Internal Audits     

  Report of External Audits     

  Client Feedback/Complaints    

  Resources (Facilities,        
  Standards, and Equipment)     
  ----------------------------- -----------------------------------------

**Form 5: WMD Quality System Management Review**

*Page 2 of 2*

  ----------------------------- -----------------------------------------
  Outstanding Items from        
  Previous Management Review    

  Staff Training                

  Other Reports                 

  Miscellaneous Topics          
  ----------------------------- -----------------------------------------

# Form 6: WMD Internal Audits for PT/ILC Activities

Page 1 of 3

+---------------------+---+----------------------------------------------+
| ##### PART A        |   |                                              |
+---------------------+---+----------------------------------------------+
| Dates of            |   |                                              |
| Assessment:         |   |                                              |
+---------------------+---+----------------------------------------------+
| Lead Assessor:      |   |                                              |
+---------------------+---+----------------------------------------------+
| Assessment Team     |   |                                              |
| Members:            |   |                                              |
+---------------------+---+----------------------------------------------+
| Quality System      |   |                                              |
| Review (all         |   |                                              |
| sections are        |   |                                              |
| assessed annually,  |   |                                              |
| though not          |   |                                              |
| necessarily at the  |   |                                              |
| same time)          |   |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.1         |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.2         |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.3         |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.4         |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.5         |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.6         |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.7         |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.8         |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.9         |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.10        |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.11        |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.12        |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.13        |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.14        |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.15        |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+
| Section 1.16        |   |                                              |
|                     | A |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | s |                                              |
|                     | s |                                              |
|                     | e |                                              |
|                     | d |                                              |
|                     | C |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | n |                                              |
|                     | g |                                              |
|                     | N |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | c |                                              |
|                     | o |                                              |
|                     | n |                                              |
|                     | f |                                              |
|                     | o |                                              |
|                     | r |                                              |
|                     | m |                                              |
|                     | i |                                              |
|                     | t |                                              |
|                     | i |                                              |
|                     | e |                                              |
|                     | s |                                              |
+---------------------+---+----------------------------------------------+

**Form 6: WMD Internal Audits for PT/ILC Activities**

*Page 2 of 3*

  ---------------------- ------------------------------------------------
  Section 1.17           Assessed Conforming Nonconformities

  Section 1.18           Assessed Conforming Nonconformities
  ---------------------- ------------------------------------------------

  ---------------------- ------------------------------------------------
  **PART B**             

  PT/ILC Quality System  
  Review (Select a 10 %  
  sample (minimum 1 per  
  region) of PT/ILCs     
  conducted each year    
  and assess all         
  elements of program)   

  PT/ILC Region          
  assessed:              

  PT/ILC activity        
  assessed:              

  Section 2.1            Conforming Nonconformities Not Developed Yet

  Section 3.1            Conforming Nonconformities Not Developed Yet

  Section 3.2            Conforming Nonconformities Not Developed Yet

  Section 3.3            Conforming Nonconformities Not Developed Yet

  Section 4.1            Conforming Nonconformities Not Developed Yet

  Section 4.2            Conforming Nonconformities Not Developed Yet

  Section 4.3            Conforming Nonconformities Not Developed Yet

  Section 4.4            Conforming Nonconformities Not Developed Yet

  Section 4.5            Conforming Nonconformities Not Developed Yet

  Section 5.1            Conforming Nonconformities Not Developed Yet

  Section 5.2            Conforming Nonconformities Not Developed Yet

  Section 5.3            Conforming Nonconformities Not Developed Yet
  ---------------------- ------------------------------------------------

**Form 6: WMD Internal Audits for PT/ILC Activities**

*Page 3 of 3*

  ---------------------- ------------------------------------------------
  Section 5.4            Conforming Nonconformities Not Developed Yet

  ---------------------- ------------------------------------------------

+-----------------------------------+-----------------------------------+
| ##### PART C                      |                                   |
+-----------------------------------+-----------------------------------+
| Audit for Closure of Previous     | Conforming Nonconformities        |
| Deficiencies                      |                                   |
+-----------------------------------+-----------------------------------+
| Attach all nonconformities and    |                                   |
| comments using the NCAP form and  |                                   |
| all other supporting              |                                   |
| documentation.                    |                                   |
+-----------------------------------+-----------------------------------+

# Form 7: NIST WMD Nonconformance/Corrective Action/Preventive Actions (NCAP)

*Page 1 of 2*

+-----------------------------------+-----------------------------------+
| 1\. Name of Person completing     | 3\. Check one of the following:   |
| form:                             |                                   |
|                                   | Nonconformity                     |
|                                   |                                   |
|                                   | Preventive Action                 |
+-----------------------------------+-----------------------------------+
| 2\. Date:                         |                                   |
+-----------------------------------+-----------------------------------+
| 4\. Identification of             |                                   |
| Nonconformity or Preventive       |                                   |
| Action:                           |                                   |
+-----------------------------------+-----------------------------------+
| 5\. Source of Discovery:          |                                   |
+-----------------------------------+-----------------------------------+
| 6\. Investigation Results:        |                                   |
+-----------------------------------+-----------------------------------+
|                                   |                                   |
+-----------------------------------+-----------------------------------+
|                                   |                                   |
+-----------------------------------+-----------------------------------+
| 7\. Root Cause:                   |                                   |
+-----------------------------------+-----------------------------------+
| 8\. Evaluation of Significance:   |                                   |
+-----------------------------------+-----------------------------------+
|                                   |                                   |
+-----------------------------------+-----------------------------------+
|                                   |                                   |
+-----------------------------------+-----------------------------------+
|                                   |                                   |
+-----------------------------------+-----------------------------------+
| 9\. Action Required? Yes No       |                                   |
+-----------------------------------+-----------------------------------+
| 10\. Does PT/ILC need to be       |                                   |
| halted? Yes No                    |                                   |
+-----------------------------------+-----------------------------------+
| 11\. If halted, identify location |                                   |
| of Artifact/position in scheme:   |                                   |
+-----------------------------------+-----------------------------------+
| 12\. Action Taken:                |                                   |
+-----------------------------------+-----------------------------------+
|                                   |                                   |
+-----------------------------------+-----------------------------------+
|                                   |                                   |
+-----------------------------------+-----------------------------------+
|                                   |                                   |
+-----------------------------------+-----------------------------------+
| 13\. Due Date: Participant        |                                   |
| Notification (in writing):        |                                   |
+-----------------------------------+-----------------------------------+
| 14\. Name of Responsible Manager: |                                   |
+-----------------------------------+-----------------------------------+
| 15\. Date PT/ILC Resumed (if      |                                   |
| applicable):                      |                                   |
+-----------------------------------+-----------------------------------+

**Form 7: NIST WMD Nonconformance/Corrective Action/Preventive Actions
(NCAP)**

*Page 2 of 2*

  ----------------------------------- -----------------------------------
  16\. For Quality Manager Use Only   

                                      

  17\. Due Date:                      

  18\. Name of Responsible Manager:   

  19\. Date PT/ILC Resumed (if        
  applicable):                        

  20\. For Quality Manager Use Only   

  Date C.A.R. issued (if applicable)  Event Number:

  Date Closed:                        

  Follow-up Audit Information (as     
  applicable):                        
  ----------------------------------- -----------------------------------

#  Form 8: Schedule & Shipping

*(nonmandatory\*)*

*Page 1 of 1*

  ----------------------- -----------------------------------------------
  PT/ILC Identification:  

  PT/ILC Coordinator:     
  ----------------------- -----------------------------------------------

  ----------- ------------- ----------- ----------- -------------- ----------
  Scheduled   Laboratory    Date        Shipper     Tracking \#    Date
  Date                      Shipped to:                            Received

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   
  ----------- ------------- ----------- ----------- -------------- ----------

\* As long as the coordinator maintains a tracking system for the
artifacts or standards, use of a form is not required.

# APPENDIX B -- PROCEDURES & POLICIES

  ------------ ------------------------------------------------------------
  **Number**   **Title**

  1            Software Validation Procedure

  2            Selection of Delegates Procedure (To be developed).

  3            NISTIR 7082 PT/ILC Policy & Plan
  ------------ ------------------------------------------------------------

# Procedure 1: PT/ILC Software Validation Procedure

Background: All software that has been designed to perform computations
associated with reporting, analyzing or evaluating acceptance criteria
for PT/ ILC activities are required to be validated according to this
procedure. Commercial software such as Microsoft Excel or MathCAD does
not require validation, but worksheets created for PT/ILC activities do
require validation.

1)  Ensure that the Software contains the following information:

    a.  Title of Software;

    b.  Date of Latest Revision (kept within the electronic file);

    c.  Identification information of person who performed the last
        revision.

    d.  Document technical specifications used (i.e., specific formulas
        for air or water density, coefficients of expansions).

2)  The Technical Manager or Designee performs the software validation.
    Under no circumstance is the author of the software to be evaluated
    allowed to perform the software validation.

3)  All computational functions of the software are to be validated
    after each revision (and reviewed at least annually). Validation is
    accomplished by entering a data set into the software to be
    validated, and comparing the results obtained to results obtained
    by:

    a.  Results published in a textbook or other appropriate reference
        for the dataset;

    b.  Hand or calculator computations (which are documented);

    c.  A program from a different software platform;

    d.  A program on the same software platform that is significantly
        different from the software to be validated.

4)  Document the validation method selected, password(s), and supporting
    information for the validation process.

5)  Sign and Date the documents and submit them to the Technical Manager
    for final review and approval.

# Procedure 2: Selection of Delegates

*To be developed at a later date.*

# Policy: NISTIR 7082 \"Proficiency Test Policy and Plan (for State Weights & Measures Laboratories)\" (on-line at <http://www.nist.gov/labmetrology>)