NIST OWM    Page 1 of 6

March 28, 2016

Technical Memorandum for:  State Weights and Measures Metrologists

From:   Elizabeth Gentry, Laboratory Metrology Program

NIST Office of Weights and Measures

Subject: National Annual Assessment Summary – Quality Management System Topics (2014 to 2015

Submission)

Introduction

This memo provides a national overview and feedback for State weights and measures laboratories

related to materials submitted for annual review by OWM as part of the Laboratory Recognition

Program associated with NIST Handbook 143.  During the 2014 to 2015 end‐of‐year review, special

emphasis was placed on the reviewing the submitted recognition materials for compliance, as well as

identifying trends and opportunities for improvement:  quality management system (QMS) quality

manuals, management review and quality audit reports.  

Annual submissions are requested each year through an annual submission memorandum that is sent to

all laboratories during the first two weeks of August.  Each year, specific items are requested for review

prior to issuing certificates of Recognition and special technical assessments are done of all materials in

key areas.  Laboratories are notified in advance what the special assessments will cover and training on

those topics is generally provided to enable continual improvement and/or success in complying with

Handbook 143 requirements.  The published requirements in Handbook 143 include those of ISO/IEC

17025, as well as other OWM administrative and technical requirements.  The annual submissions are

submitted to OWM in the submission cycle between October 1 and November 1.   

Special Guidance on Document Control (4.3).  Quality management system documents are an important

tool used within a state metrology laboratory to communicate policy and procedures to personnel who

provide recognized measurements services to customers.  One function of a healthy QMS that it that it’s

applied uniformly by all personnel.  If some personnel are using different procedures, non‐conforming

work and corrective actions (and possibly recalls of work) may occur, which affects the quality of the

work the customer will receive.

ISO/IEC 17025 notes that the term "document" can be broadly applied and includes "policy statements,

drawings, plans, etc. . ."  This broad definition essentially means that laboratories are faced with the task

of managing large numbers of documents and spreadsheets that exist in many mediums, including

hardcopy, electronic, digital, photographic, and written forms.  Documents may come from both

external and internal sources.  
NIST OWM     Page 2 of 6

Document control is a process used to manage the lifecycle of a document, which includes five (5)

phases:  Creation, Review & Revision, Approval & Issue, Storage & Use, and Retirement.  A robust

document control process is fundamental in ensuring quality calibration services are received by

laboratory customers.

Lab must set an appropriate review cycle or frequency for reviewing QMS documents.  Typically this

time period may be once per year (annually) or can be extended if changes are not likely required very

often.  Documents authored or written internally by the laboratory will likely require more frequent

review than those created by external bodies (like national or international standards, specifications).  

Laboratories with large numbers of documents often develop a review schedule to spread their review

work throughout the year, making it easier to fit into their busy calibration schedules.

Master List

A master list is a management system document that identifies the current revision status of all quality

management system documents.  The master list is required in ISO/IEC 17025 Section 4.3.2.1 to ensure

that the current revision status and distribution of management system documents is established to

prevent the use of INVALID or OBSOLETE documents.  It’s important to create a quality culture within an

organization where the master list is always consulted before using a management system document to

ensure that the correct approved version is being used.  Using an out‐of‐date document is risky and can

lead to nonconforming work, including generating customer calibration errors.  Each master list entry

typically includes the title, issuing authority (external or internal source), issue date (or version), and

date for next scheduled review for each document.

The master list may be a stand‐alone document or it may be included in a document control procedure.  

It is important to evaluate whether all master list documents are necessary and appropriate to the

laboratory's current measurement scope and calibration activities.  Why maintain procedures for

calibrations that are not even performed?  Removing unnecessary documents can make an instant

impact and streamline document control efforts.

Obsolete or superseded documents may be needed for either legal or historical purposes.  When these

documents are removed from service and deleted from the master list, they must be suitably marked so

laboratory personnel do not mistake them for active documents. Some laboratories have found it useful

to designate a well‐marked centralized storage location for their obsolete and historic documents to

help ensure they are not unintentionally used.  Authorized editions (“controlled copies”) of QMS

documents should be available at all locations where essential functions are performed.

External Documents

External documents are documents whose origin and control is by a body outside of your immediate

organization.  The lack of regular review of external documents is a common nonconformity.  Regular

review is necessary to determine if the latest version has been adopted.  Examples might include

documents:

 Necessary for the planning and operation of the QMS.

 Latest version of accepted international, regional, or national documentary standards,

procedures, and specifications.

 Such as equipment manuals, charts, posters and other agency/organization laws, manuals, and

directives.
NIST OWM     Page 3 of 6

Document Repository

It is convenient to store QMS documents in a centralized repository, such as a book shelf, file cabinet, or

centralized electronic system (e.g., Share Point, internal webpage).  Keeping track of the distribution

(location) of documents will help track down and archive old copies that must be removed from service

when they are replaced or become obsolete.  The responsibility is on laboratory staff to check the

version they plan to use against the version shown on the master list to ensure only the approved

revision of a document is used.  

National QMS Observations from Annual Submission Packages

During the upcoming 2016 review of submission materials, OWM will provide special attention and

evaluation on the status and functionality of the QMS master list, management review process and

records, and internal quality audit process and records.  Reviewing the observations in this section and

identifying gaps and opportunities for improvement will help ensure your laboratory QMS is operating

efficiently, implementing recommended best practices, and providing quality traceable measurement

services to customers.

In past reviews the following items have been observed for some laboratories as good practices:  

 The Management Review included a written report, with executive summary, documented

findings, and action plan with deadlines and evidence that actions have been evaluated for

effectiveness.  An excellent report balances brevity, relevant details, and completeness.  

 Management Review meetings often used a PPT presentation (with photos and graphs) to

efficiently communicate with top management and serve as supplemental objective evidence

that should be maintained and submitted for recognition.  

 An effective Management Review included a complete overview of all essential topic elements.  

 QMS were often initially based on NISTIR 5802, a quality manual template that was developed

according to the now obsolete ISO/IEC Guide 25:1990.  These laboratories may identify master

list components in multiple locations, including the Quality Manual Chapter 2 (References and

Definitions), Appendix H (Procedures List), and Appendix N (Document Control).  A best practice

is to combine these reference sources into one location, the master list, which can serve as a

centralized source of approved references.

 Quality manual chapters, appendices, and SAPs have been combined into one document for

easy maintenance.  Headers and footers are uniform throughout the QMS file.  One version

control identifier has been applied to the entire document even if individual sections were

updated during the year and noted in the revision control section (e.g., Adopted April 2016).   

 Internal quality audits were frequently documented using template forms, citing a variety of

objective evidence, including laboratory records, administrative and technical process

observations, and staff interviews.  Action plans are being used to ensure that action items don’t

“slip through the cracks” and are completed according to established deadlines.  Action item

effectiveness is being monitored.
NIST OWM     Page 4 of 6

Opportunities for Improvement

 Laboratories that only write a Management Review report could enhance communication with

top management by expanding written information into a PPT presentation with photos and

graphs (data).

 A Management Review that is too short (i.e., only 2 to 3 pages). Some don’t contain sufficient

evidence that anything was discussed, an action plan and don’t include evidence of monitoring

action item effectiveness.  

 Some laboratory quality manuals and associated documents have not updated since 2009!  A

five (5) year review cycle is inappropriate and will not support the goal of maintaining an

efficient and functioning QMS.   

 Only the quality manual updates were submitted, not a complete quality manual with

appendices, SAPs, forms and other associated documents.  OWM needs a complete set of all the

materials that make up the QMS to examine and evaluate the current system as a whole during

the recognition cycle.

 Quality manual document “track changes” should not be active in a QMS document that has

been placed into service.  All edits (i.e., underscores and strikethrough) should be accepted or

declined.  The document should be clean and free of all edit markings.

 Quality manual, appendices, and SAPs are in separate document files and have individual dates

on each individual document, some of which have no descriptive file name (just a section

number) which must be independently maintained.  Each document revision must be listed

correctly on the master list table.  Frequent nonconformities were observed between the actual

file version (i.e., located in each separate electronic file footer) and the version found on the

master list table.  

 Confusing document control persists, especially the issue or revision dates.  There may be one

version date at the beginning of the document (i.e., 2007) which doesn’t agree with the

document headers (i.e., 10/28/2003) or with the table of contents (i.e., included changes up

through 2013).   

 The most important principle of file‐naming is to be consistent.  All documents will change or

evolve over time (REVISION).  Version changes are easily identified by a date used to represent

the version:  June 2016, 2016‐06, or 2016‐06‐15.

 The QMS should have only one set of page numbers, preferably using the page X of Y format.

 The QM and each individual SAP electronic files have no Issuing Authority indicated in the file

name (i.e., the laboratory name or abbreviation of the name).   

 Combining the QM, SAPs, and appendices into one document is an opportunity for

improvement.  A recommended “first step” is to combine individual quality manual chapter files

into one document, create uniform headers/footers, and use one version control throughout

the document.  Update the master list to reflect the change.  The same can then be done to

combine individual SAP files.  Ultimately, the QM, SAPs, and appendices can be merged into one

large document with consistent version control, headers, and footers.
NIST OWM     Page 5 of 6

 Many laboratory master lists weren’t updated after the training provided at the 2014 C‐RMAP.  

Reference citations should not use statements like “or later version if available.”   

 Some quality manual master lists were observed that did not include any revision or publication

dates for listed documentary standards.   

 The lack of regular review of Master List external documents was observed.  Regular review is

necessary to determine if a newer version is available and to ensure the latest version has been

adopted.   

 Reference documents have been observed to be listed in as many as five different locations

within the QMS (providing opportunities for missing a section during review and for conflicting

references in different sections).

 Quality audit template forms were used to document the process.  However, objective evidence

cells are frequently left completely blank.  QM references, but not objective evidence noted.

 A note at the end of the audit report was often observed, stating that there is an “Objective

Evidence” attachment.  However, no objective evidence was found attached to the report or

present in any of the other files.  Attach objective evidence to the report (e.g., use a PDF and

combine multiple files) or include the objective evidence files in the quality audit file folder.

 Some laboratories have only conducted a “desk audit” (i.e., evaluating presence of criteria)

rather than a complete “functional audit” (i.e., observing laboratory operations to confirm

compliance to criteria and QMS).  These laboratories often only say “OK” in the objective

evidence section without making any connection to the relevant quality system location or

providing descriptions of the objective evidence (e.g., records, observations, staff interviews).   

 Some internal audit reports are also missing an action plan.  The report should include an

executive summary, which provides a high level overview of the status of laboratory operations

and summarizes the number and type of actions for top management.  The audit report should

include an action plan, as well as documented RCA, deadlines, and follow‐up monitoring of

actions for effectiveness.   

 Some old audit findings (i.e., from prior audits, several years ago) were found present in the

report.  This may be the result of re‐using prior audit reports.  It’s recommended to read the

report in its entirety before finalizing the document to identify any discrepancies, including gaps,

missing objective evidence or open (incomplete) action items.

Please consider these tips before submitting the next annual submission package:

 Don’t send OWM a CONTROLLED COPY of the quality manual, appendices, Standard

Administrative Procedures (SAPs and other related documents.  Submit only UNCONTROLLED

electronic copies.

 Review the entire Quality Management System (QMS) to determine if it still references “WMD”

rather than “OWM.”  All references to NIST Weights and Measures Division (WMD) should be

replaced with NIST Office of Weights and Measures (OWM).    
NIST OWM     Page 6 of 6

 Double check to ensure that the management review and quality audit report document when

the activity was initiated (e.g., a date or date range) and who conducted or participated in the

activity.

 Review the quality of records and other objective evidence included in the submission package.  

Frequently, records supplied in the submission are scans of a hand written document and are

illegible or of a poor resolution.  At minimum we need to request a replacement scan before we

can interpret the information.  This will move your submission status to “Incomplete” and delay

processing by OWM.

 Don’t evaluate the effectiveness on the same day the action item is completed.  It’s necessary to

give an action a little time to be used in normal operations before making an assessment to

determine its effectiveness.