Article Ref: J-023

Weights and Measures Connection, Vol. 5, Issue 7 Page 1

December 10, 2014

U.S. National Working Group on Alternative Test 

Methods, Field Task Group Activities

Byline: G. Diane Lee

The National Institute of Standards and Technology (NIST), Office of Weights and Measures established 

the U.S. National Working Group (USNWG) on Alternative Test Methods (ATMs) for Commercial 

Measuring Devices. The objectives of the USNWG on ATMs are to promote, encourage, and participate 

in the establishment of a comprehensive set of standards; encourage their widespread use by weights 

and measures officials and service companies to test commercial measuring devices; and establish 

proposed alternatives to ensure that the methodologies and standards facilitate measurements that are 

traceable to International Systems of Units (SI). This work includes an investigation of current 

methodology and standards (e.g., neck-type volumetic field standards and associated test procedures). 

At its initial April 2012 meeting, a field task group was established to look at areas in which work is 

needed to establish the framework for recognizing a given alternative test method for commercial use. 

At this meeting the field task group identified the areas of work needed to establish this framework:

• uncertainty analysis of test methods used in the field;

• protocols (framework, key issues and questions to be answered);

• legal and acceptance issues (references in laws and regulations and service companies vs. 

weights and measures regulations); and

• field inspection (Evaluation Procedure Outlines (EPOs), guidelines, training, variables in field 

implementation).

The field task group began working to identify and quantify uncertainties associated with testing liquid 

measuring devices by:

• reviewing the key factors that contribute to measurement uncertainty when using hand held 

test measures, bottom drain provers, and closed loop provers; and

• developing a test protocol for side-by-side testing of retail motor fuel dispensers using test 

measures, bottom drain provers and closed loop provers.

In May 2013, side-by-side tests were conducted at two different retail motor-fuel stations in 

Pennsylvania. Three fuel types (regular, premium, and diesel) were tested on a total of 18 positive 

displacement (PD) meters; including 4 diesel and 14 gasoline meters. A maximum of five fast and two 

slow flow tests were performed on each meter and product. Prior to testing, NIST technical advisors 

verified that each standard used for testing had a current calibration certificate.

The analysis of the side-by-side test results was based on a limited number of data points and test 

conditions and, as such, was not intended for use to draw any final conclusions regarding the suitability 

of the standards examined or the effects of any factors contributing to uncertainties associated with the 
Article Ref: J-023

Weights and Measures Connection, Vol. 5, Issue 7 Page 2

use of a given standard. The data associated with the analysis were intended to assist the USNWG in its 

efforts to determine baseline values for assessing the uncertainties associated with testing using three 

different types of standards. 

After the side-by-side test data was analyzed, NIST, OWM learned that the validity of the calibrations 

that were performed in another state on the test measure and bottom drain prover could not be 

confirmed due to traceability issues. Without a valid calibration to provide a correct value and 

uncertainty for these standards, no determination could be made to how well the methods compared to 

each other (e.g., the bias between standards could not be determined). However, the field task group 

was able to review the repeatability for each of the standards separately. 

The task group met to review the data. Outliers were observed that revealed possible problems with 

the repeatability for two of the standards. More data is needed to draw any final conclusions of 

repeatability problems. It was agreed that more data is needed to review the operation of the 

standards over wider ranges of temperature, time, and flow in addition to other major factors that 

contribute to uncertainties of tests using different standards. As such, the ATM field task group 

identified the major contributing factors to uncertainties associated with the alternative test methods 

examined:

• repeatability over a range of temperatures and time;

• repeatability over a range of flow rates and time;

• vapor loss;

• effects on test results due to removing the dispenser nozzle for testing; and

• compressibility.

It was also noted during this meeting that existing data is available that may be used to assess the 

uncertainty of alternative test methods used in the field. Following the meeting, field task group 

participants provided data that was collected from other studies, along with a summary of challenges 

and concerns. NIST summarized this information in a matrix and distributed it to the field task group. 

The matrix lists the major contributing factors to uncertainties associated with the alternative test 

methods as noted above in this article and groups the uncertainty data that was submitted with the 

major contributing factor that it supports. The list is color coded to denote if the data was provided 

from a source outside the task group, collected as part of the field task group activities, or if specific 

data is still needed. In addition, the matrix includes (1) who provided the data; (2) challenges to the 

data; (3) comments; and (4) notations of whether or not uncertainty values are included in the analysis 

of the data. Due to the number of concerns and challenges to the data, it was determined that 

additional data are needed to adequately quantify uncertainties for the major contributing factors 

identified by the field task group. 

Volunteers are needed to assist manufacturers in collecting the data needed to complete the 

uncertainty analysis for testing using different standards. As additional data is collected and pending 

sufficient resources, NIST will monitor and witness the data collection as resources are available to assist 
Article Ref: J-023

Weights and Measures Connection, Vol. 5, Issue 7 Page 3

the field task group in its efforts to determine the uncertainties associated with alternative test 

methods. 

The field task group will need to develop guidelines to ensure that the data collected is appropriate and 

valid for use in determining the uncertainties associated with the different test methods. If you have 

any questions about this work or the work of the field task group, please contact Diane Lee by e-mail at 

diane.lee@nist.gov.

Office of Weights and Measures (301) 975-4004; e-mail: owm@nist.gov