NISTIR 8153

Report on the May 2016 ASTM E57.02

Instrument Runoff at NIST, Part 2 –

NIST Realization of Test Procedures 

and Uncertainties in the Reference 

Lengths

Bala Muralikrishnan

Prem Rachakonda

Meghan Shilling

Vincent Lee

Christopher Blackburn

Daniel Sawyer

Geraldine Cheok

Luc Cournoyer

This publication is available free of charge from:

https://doi.org/10.6028/NIST.IR.8153
NISTIR 8153

Report on the May 2016 ASTM E57.02

Instrument Runoff at NIST, Part 2 –

NIST Realization of Test Procedures 

and Uncertainties in the Reference 

Lengths

Bala Muralikrishnan, Prem Rachakonda, Meghan Shilling, 

Vincent Lee, Christopher Blackburn, Daniel Sawyer 

Engineering Physics Division

Physical Measurement Laboratory

 NIST 

Geraldine Cheok 

Intelligent Systems Division

Engineering Laboratory

 NIST 

Luc Cournoyer

Measurement Science and Standards,

National Research Council of Canada, Ottawa, Canada

This publication is available free of charge 

from: https://doi.org/10.6028/NIST.IR.8153 

October 2016

U.S. Department of Commerce 

Penny Pritzker, Secretary

National Institute of Standards and Technology 

Willie May, Under Secretary of Commerce for Standards and Technology and Director
i

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

Disclaimer

Commercial equipment and materials may be identified in order to adequately specify certain 

procedures. In no case does such identification imply recommendation or endorsement by the 

National Institute of Standards and Technology, nor does it imply that the materials or 

equipment identified are necessarily the best available for the purpose.
ii

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

Table of contents

1. NIST realization of test procedures

2. Targets

2.1. Targets used for the grid and inside test setup

2.2. Targets used for the relative range tests

2.2.1. The plate-sphere artifact

2.2.2. The bare-sphere artifact

3. Grid of spheres

3.1. Test setup

3.2. Calibrating the long horizontal length (DH)

3.2.1. The four-azimuth averaging procedure

3.2.2. Validation experiments

3.2.3. Uncertainty budget

3.2.4. Providing reference measurements for the grid

3.3. Calibrating the central nine spheres of the grid

3.3.1. Calibration procedure

3.3.2. Validation experiments

3.3.3. Uncertainty budget

4. Inside test

4.1. Test setup

4.2. Calibration procedure

4.3. Validation experiments

4.4. Uncertainty budget

5. Relative range tests

5.1. Test setup

5.2. Uncertainty budget

6. Summary

7. Acknowledgements

8. References
1

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

1. NIST realization of the test procedures

There is ongoing activity within ASTM E57.02 working group WK43218 [1] to develop a 

documentary standard for point-to-point distance performance evaluation of spherical 

coordinate 3D imaging systems. The Dimensional Metrology Group (DMG) at the 

National Institute of Standards and Technology (NIST) has established facilities to realize 

all of the test procedures in the current draft. 

Part 1 [2] of this two-part report presented background information and key findings from 

an instrument runoff hosted by the DMG in support of this effort. This report (Part 2) 

describes test facilities established by the DMG, calibration procedures, validation data, 

and uncertainty budgets. It is important to note that the eventual standard will not mandate 

how a user will realize any of the test procedures; this report merely describes the approach 

adopted by the DMG at NIST. It is also noted that this report addresses the uncertainties in 

the reference lengths only; the overall test uncertainty may have other components that are 

not addressed here.

In order to realize all tests described in the draft standard (which are also given in Appendix 

A of Part 1 [2] of the report), DMG established three setups in two laboratories. A 

calibrated grid of spheres and an inside test setup are established in the large-scale 

laboratory, and a relative range setup is established in the tape-tunnel facility. Section 2 

describes the targets used, while subsequent sections describe each of the three setups. A 

laser tracker is used as the reference instrument (RI) to calibrate the test setups. For readers 

not familiar with the laser tracker technology, a review can be found in [3].

2. Targets

2.1 Targets used for the grid and the inside test setup

“Integration spheres” manufactured by Bal-tec Inc. are used in the grid and the inside test 

setup. The integration spheres are partial spheres made of aluminum with a satin finish that 

is suitable for 3D imaging systems. The reflectance factor of a sample with similar finish 

(also provided by the same manufacturer) is measured to be 0.27 at a wavelength of 710 

nm and 0.36 at a wavelength of 1500 nm. The 3D imaging systems involved in the runoff 

operated at a wavelength of approximately 700 nm or 1500 nm.

A smaller sphere is less expensive and easier to handle but a larger sphere allows for more 

data to be obtained from the surface; spheres of nominal 100 mm diameter are chosen as a 

tradeoff. The insides of the integration spheres are hollowed out and kinematic mounting 

balls are located to centrally seat a 38.1 mm (1.5 in) spherically mounted retroreflector 

(SMR) as shown in Fig. 1 (a) and (b). The SMR is held in place by a magnet inside the 

integration sphere.

The integration spheres may be used in two different modes. In one mode of use, the centerto-center distance between 38.1 mm (1.5 in) SMR nests distributed in the work volume are 
2

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

calibrated using a laser tracker and 38.1 mm (1.5 in) SMRs (see Fig. 1 (c)). Subsequently, 

the SMRs are replaced with 38.1 mm (1.5 in) spheres and the integration spheres are 

mounted on top (see Fig. 1(d)). In this mode of user, both the laser tracker and the 

instrument under test (IUT), the 3D imaging system, are on the same side of the target.

Fig. 1 (a) Integration sphere mounted on a 38.1 mm (1.5 in) sphere placed in the work 

volume, (b) view from the back of the integration sphere showing the kinematic mounting 

balls to hold the 38.1 mm (1.5 in) SMR, (c) calibrating the center-to-center distance 

using a laser tracker and 38.1 mm (1.5 in) SMR, (d) the SMR is replaced with a 38.1 mm

(1.5 in) sphere and the integration sphere is mounted on top, (e) integration sphere 

rigidly bolted to support plates, (f) a 38.1 mm (1.5 in) SMR placed inside the integration 

sphere, (g) schematic showing the calibration of the center-to-center distance from the 

backside while the 3D imaging system views the sphere from the front side

3D imaging 

system

Laser tracker

38.1 mm 

(1.5 in)

sphere

Integration

sphere

1.5 in

sphere

Nest

(a) (b) (c)

(e) (f)

(g)

Kinematic 

mounting balls 

to centrally hold 

a 38.1 mm (1.5 

in) SMR 

SMR

Nest

Laser tracker

3D imaging 

system

(d)
3

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

In another mode of use, the integration spheres are bolted to support plates as shown in 

Fig. 1 (e). SMRs are placed inside the spheres as shown in Fig. 1(f). A laser tracker, placed 

behind the plane of the spheres, is used to calibrate the center-to-center distance while the 

3D imaging system is placed in front during the performance tests (see Fig. 1(g)). This 

approach is adopted in the DMG laboratories primarily because the magnet inside the 

integration sphere sometimes does not exert a sufficiently strong force to hold the 

integration sphere when the sphere is mounted facing downwards, i.e., mounted in a 

manner so that the gravitational and the magnetic force counter each other.

For each of the 16 integration spheres procured by the DMG, a contact probe coordinate 

measuring machine (CMM) is used to measure the diameter, concentricity between the 

outer surface and SMR center, and form of the outer surface. The results are tabulated in 

Table 1. The average concentricity and form are both approximately 0.01 mm.

Table 1 Diameter, concentricity, and form data for 16 integration spheres

Diameter Concentricity Form

(mm) (mm) (mm)

100.050 0.009 0.007

100.041 0.004 0.007

100.017 0.014 0.010

99.991 0.020 0.015

99.979 0.012 0.006

100.017 0.009 0.007

100.018 0.004 0.006

100.003 0.012 0.009

99.998 0.011 0.010

100.004 0.016 0.007

100.013 0.007 0.008

99.988 0.019 0.017

99.991 0.010 0.011

99.990 0.011 0.010

100.006 0.001 0.007

99.983 0.005 0.011

2.2 Targets used for the relative range tests

The relative range tests are performed using two different targets, a plate-sphere artifact 

and a bare-sphere artifact. 

2.2.1 The plate-sphere artifact

The plate-sphere artifact consists of a 609.6 mm x 609.6 mm (2 ft x 2 ft) plate with a 

nominal 200 mm diameter integration sphere mounted in the middle. Plate material from a 
4

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

50.8 mm (2 in) annular region around the sphere is removed to reduce the probability of 

secondary reflections from influencing the 3D imaging system’s range measurement.

The purpose of the sphere on the plate is only to serve as a fiducial, i.e., to tie the platecenter as determined by the 3D imaging system to the corresponding point measured by 

the laser tracker, which is the center of the SMR centrally located inside the sphere. This 

registration is achieved as follows:

 Both the plate and the sphere are scanned using the 3D imaging system and the data

representing the two geometries are separated

 A least-squares plane is fit to the data corresponding to the plate and the normal

vector determined

 A least-squares sphere is fit to the data corresponding to the sphere and the sphere

center is obtained

 The sphere center is then projected on to the plate along the normal vector of the

plane to determine the plate-center.

Because the artifact is designed so that the mechanical center of the sphere lies on the front 

surface of the plate, the plate-center as determined above is theoretically coincident with 

the center of the SMR. Ensuring coincidence of the plate-center and the SMR minimizes 

the Abbe error that can occur if the plate is misaligned (i.e., not orthogonal to the line 

joining the laser tracker and the 3D imaging system). 

Fig. 2 shows a schematic and three different views of the plate-sphere artifact used during 

the runoff. The flatness of the plate is measured on a CMM and determined to be 60 μm. 

The diameter, concentricity, and form error of the sphere are determined by CMM 

measurements to be 199.996 mm, 0.003 mm, and 0.011 mm respectively. The center of the 

sphere is 0.126 mm below the surface of the plate.

Fig. 2 Different views of the plate-sphere artifact used during the runoff

2.2.2 The bare-sphere artifact

The bare sphere artifact consists of a nominal 200 mm diameter integration sphere mounted 

on a plate using 101.6 mm (4 in) standoffs as shown in Fig. 3. The purpose of the standoffs 

is to reduce the probability of secondary reflections from the background surface from 

influencing the 3D imaging system’s range measurement. The support plate is covered with 

laser absorbing black felt material to further reduce the potential for secondary scattering. 
5

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

The diameter, concentricity, and form error of the sphere are determined by CMM 

measurement to be 200.003 mm, 0.006 mm, and 0.004 mm, respectively. 

Fig. 3 Bare-sphere artifact used during the runoff.

3. Grid of spheres

3.1Test setup

Eleven integration spheres are arranged in the large-scale laboratory as shown in Fig. 4 to 

realize all of the two-face, the near symmetric, the near asymmetric, and far symmetric 

point-to-point length tests (see Appendix A of Part 1 [2] for a list of the tests). Each of the 

three vertical rails attached to tall stands hold three integration spheres. The two integration 

spheres on the extremes (D and H) are mounted directly on shorter stands. The horizontal 

and vertical spacing between spheres is about 1.6 m. The 3D imaging system is placed in 

front of the grid during the tests. Eleven SMRs are placed, one inside each integration 

sphere, to allow for the calibration of the center-to-center distance using a laser tracker in 

absolute distance measurement (ADM) mode. 

While a single laser tracker placed behind the grid may be used to calibrate the entire grid 

of 11 spheres, a two-step procedure is adopted to reduce the uncertainty in the sphere 

center-to-center distance calibration. In the first step, the two extreme spheres are 

calibrated using a laser tracker through a method described in the next section. In the 

second step, the central nine spheres of the grid are calibrated from multiple positions of a 

laser tracker as described in Section 3.3. 
6

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

Fig. 4 A grid of 11 spheres in the large-scale laboratory at NIST

3.2 Calibrating the long horizontal length (DH)

3.2.1 The four-azimuth averaging procedure

A tracker is placed centrally behind the grid, as close as possible to the grid (which happens 

to be about 5 m to avoid clipping the beam at the edges of the integration spheres), and at 

approximately the same height as the central horizontal row of integration spheres as shown 

in Fig. 5(a). All measurements are performed using the ADM mode of the tracker. Frontface and back-face measurements are averaged to determine coordinates of centers of 

integration spheres D and H, and the length between D and H is determined. The tracker is 

then rotated about its vertical axis by 90° and the length is determined again. This process 

is repeated two more times for a total of four lengths, one collected at each of the four 

azimuth angles 0°, 90°, 180°, and 270°. The average of the four measured lengths is within 

few micrometers from an interferometric (IFM) line-of-sight measurement, as the 

validation data and uncertainty budget below shows. 

This procedure is based on work reported in [3]. Averaging front- and back-face 

measurements removes errors associated with terms that reverse in sign between front-face 

and back-face. Any remaining tracker error model parameters are trigonometric functions 

and therefore periodic; averaging measurements from four azimuth positions 90o apart 

removes the influence of these terms. Ranging errors will also manifest in the 

measurements, and this is accounted for in the uncertainty budget discussed in Section 

3.2.3.

H

A

G

I

B

F

C

E

J K

D
7

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

Fig. 5 (a) Setup to measure a horizontal length (top view shown) between integration 

spheres D and H, (b) establishing the validity of the method using a test setup

3.2.2 Validation experiments

In order to establish the validity of the four-azimuth averaging procedure, two 38.1 mm

(1.5 in) nests P and Q are mounted on stands a distance l apart with the test tracker centrally 

placed at a distance d from the line joining P and Q as shown in Fig. 5(b). Table 2 shows 

the different values of l and d used for the validation. The distance between the nests is 

calibrated using a line-of-sight tracker (in IFM mode) placed along the line joining the two 

nests. The standard uncertainty in line-of-sight calibrations with laser trackers is on the 

order of 10 μm without considering thermal effects. Because the horizontal and vertical 

axes do not need to move in line-of-sight measurements, the dominant term is the 

uncertainty in the range measurement of the tracker.

Table 2 Validation data for four-azimuth averaging procedure

l 

(m)

d 

(m)

Line-of-sight 

reference 

(mm)

Four-azimuth 

averaging 

(mm)

Error 

(mm)

Tracker A is along lineof-sight, Tracker B is test 

tracker

7.2 6 7178.183 7178.177 -0.006

7.2 5 7178.183 7178.175 -0.008

Tracker B is along lineof-sight, Tracker A is test 

tracker

7.2 6 7178.269 7178.275 0.006

7.2 5 7178.269 7178.265 -0.004

Tracker A is along lineof-sight, Tracker C is test 

tracker

2.3 3 2328.967 2328.968 0.001

2.3 5 2328.967 2328.963 -0.004

The line-of-sight measurement result, the distance obtained from the four-azimuth 

averaging procedure (using the test tracker), and the difference between the two 

measurements are shown for different values of l and d in Table 2. Combinations of three 

different trackers are used for this testing. The differences are all smaller than 10 μm, 

indicating that the four-azimuth averaging procedure is capable of calibrating a horizontal 

length with uncertainties that are comparable to a line-of-sight calibration. The observed 

differences in the two methods are well below the standard uncertainty (tracker 

Tracker

θ

θ

r

H D

uR

uA

d ≈5 m

l ≈ 6.4 

m

Test tracker

r

P Q

d 

l

Line of sight 

tracker
8

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

contribution and other mechanical terms) of 18 μm associated with these measurements in 

the uncertainty budget of Section 3.2.3, giving confidence that the standard uncertainty 

used for the tracker contribution is adequate.

3.2.3 Uncertainty budget

Mechanical terms: The uncertainty in locating a single point in space resulting from error 

sources associated with the laser tracker, the centering of the integration sphere with the 

SMR, and the concentricity of optical and mechanical center of the SMR is first 

determined. The point coordinate uncertainty is then propagated to estimate the uncertainty 

in the point-to-point length. Uncertainty calculations are shown here for the following 

dimensions: l = 6.4 m and d = 5 m, therefore r = 5.9 m and θ = 32.6° (see Fig. 5(a)).

Tracker contribution: The uncertainty along the ranging direction can be obtained 

from the manufacturer’s specifications. For an ADM tracker with a specification of 

16 μm + 0.8 μm/m, the maximum permissible error (MPE) at a range of 5.9 m is 

20.7 μm. The R0 specification for the same tracker is 16 μm. Summing in 

quadrature and assuming these form the upper bounds of a rectangular distribution, 

the standard uncertainty uR along the ranging direction is 15.1 μm. In order to 

compute the uncertainty along the direction transverse to the laser beam, only the 

repeatability of the angular axis is considered, not the overall accuracy of the axis. 

This is because all targets are measured in front- and back-face and averaged, and 

the length itself is the average from four azimuth orientations of the tracker 

separated by 90° each. This removes the influence of all non-ranging systematic 

sources of error in the tracker. Experimentally determined one standard deviation 

repeatability of the horizontal angle measurement is 0.1 mdeg (based on 10 

consecutive tracker measurements of target H). The vertical angle repeatability 

does not affect this measurement because the reference length is oriented 

horizontally. This translates to a one standard uncertainty uA of 10.4 μm. The 

standard uncertainty in the determination of coordinate H along the direction of the 

length is √(uR sin θ)

2 + (uA cos θ)2, or 12 μm.

Integration sphere concentricity with SMR: The concentricity of all 14 integration 

spheres are measured on a CMM. The experimentally measured one standard 

deviation is 5 μm and this value is considered as the standard uncertainty in the 

determination of the coordinate.

SMR concentricity: The concentricity specification of the maximum distance 

between physical and optical center of the SMRs is 2.5 μm. Assuming this value as 

the upper bound with any value within that bound as equally probable (rectangular 

distribution), the standard uncertainty in the measured coordinate is 1.4 μm.

Standard uncertainty in the length: The combined standard uncertainty in the 

determination of a coordinate is therefore the root sum square of the three terms 

above, which is 13 μm. Because two measurements are combined to determine a 

length, the standard uncertainty in the length uL is 13√2 = 18 μm. 
9

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

Thermal effects: Assuming ±0.5° fluctuation of the temperature in the room as an upper 

bound with any value inside that bound as equally probable, and assuming thermal 

expansion coefficient of 12 x 10-6/°C for concrete, the standard uncertainty of a 6.4 m 

length is 22 μm. The uncertainty due to changes in refractive index (and therefore measured 

range) is not considered because the effect is an order of magnitude smaller than the 

uncertainty due to thermal expansion of the floor.

Combined standard uncertainty: Combining the mechanical contribution 18 μm and the 

thermal contribution of 22 μm, the combined standard uncertainty in length is 28 μm. The 

expanded uncertainty (k = 2) is 56 μm.

3.2.4 Providing reference measurements for the grid

In addition to measuring the extreme targets D and H, the remaining three targets in the 

horizontal row, E, F, and G, are also measured using the same four-azimuth averaging 

procedure. The actual method to determine calibrated lengths EF, FG, and EG is different 

from this four-azimuth averaging procedure, but this procedure provides these three lengths 

to sufficiently low uncertainty to use as reference values to aid in validating the actual 

procedure used (described in 3.3).

Fig. 6 The four-azimuth averaging method is used to measure coordinates of all the five 

targets in the central horizontal row of the grid

3.3 Calibrating the central nine spheres of the grid

3.3.1 Calibration procedure

The central nine spheres of the grid are calibrated using a laser tracker by ‘averaging’ the 

coordinate data acquired from four positions behind the grid. This averaging simply 

involves transforming the data from the four positions into a common frame using the 

method of least-squares. Sphere center-to-center distances are then calculated from the 

‘average’ coordinates. Because it is the center-to-center distance that is the quantity of 

interest, it is also possible to determine these quantities for each tracker position and 

H G F E D

5 m

Tracker
10

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

subsequently compute their average. The center-to-center distances determined using both 

methods yield practically the same values.

Fig. 7 (a) Three 38.1 mm (1.5 in) SMR nests calibrated using a line-of-sight tracker to 

provide additional reference lengths to validate grid calibration, (b) four nominal 

positions of the tracker behind the grid

In order to obtain additional reference measurements that can be used to validate the grid 

calibration procedure, three collinear 38.1 mm (1.5 in) SMR nests (P, Q, and R) are set up 

P

Q

R

Fold 

mirror

Grid of spheres (top view)

Tracker position 1 

(x,y) location: (1 m, 3 m)

Tracker height: low

Tracker azimuth: 0°

Tracker position 2

(x,y) location: (-1 m, 3 m)

Tracker height: low

Tracker azimuth: 90°

Tracker position 3 

(x,y) location: (0 m, 4 m)

Tracker height: high

Tracker azimuth: 180°

Tracker position 4 

(x,y) location: (0 m, 5 m)

Tracker height: high

Tracker azimuth: 270°

H A, G, I B, F, J C, E, K D

(b)

(a)

x

y
11

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

behind the rails as shown in Fig. 7(a). A laser tracker is set up near nest P and aligned so 

that its laser beam, after bouncing off of a fold mirror on the floor, is also collinear with 

the three nests, allowing the tracker to be used in its most accurate line-of-sight mode. The 

distance between the nests is calibrated using this tracker, yielding three lengths PQ, QR, 

and PR. 

When the nine integration spheres are measured by the laser tracker from the four positions, 

three additional 38.1 mm (1.5 in) nest locations are also measured and included as part of 

the averaging procedure. This grid calibration then produces coordinates of the central nine 

spheres and the three extra nest centers, which can be used to calculate a number of centerto-center distances. These include the horizontal lengths EF, FG, and EG, and the diagonal 

lengths PQ, QR, and PR. The three horizontal lengths can be compared against 

measurements of the same lengths obtained through the four-azimuth averaging procedure 

described in the previous section while the three diagonal lengths can be compared against 

measurements of the same lengths obtained through the line-of-sight measurement. 

Comparison results and long term grid stability data are presented in the next section.

To sample and average vertical angle errors in the tracker, the tracker stand is lowered for 

two of the positions and raised high for the remaining two positions. To sample and average 

horizontal angle errors in the tracker, the tracker is rotated by approximately 90° about the 

vertical axis as it is moved from one position to the next. From each position of the tracker, 

all nine targets and the extra three 38.1 mm (1.5 in) nests are measured in front-face and 

back-face, and averaged to eliminate the influence of tracker model parameters that change 

sign from front-face to back-face. Fig. 7(b) shows the four approximate placements of the 

trackers behind the grid.

3.3.2 Validation experiments

The grid calibration is performed 14 times over a period of one month using three different 

trackers and three different operators. Fig. 8 shows the errors from the reference values in 

the three horizontal lengths EF, EG, and FG, and the three diagonal lengths PQ, PR, and 

QR. As mentioned earlier, the reference values for the horizontal lengths are obtained by 

the four-azimuth averaging method and the reference values for the diagonal lengths are 

obtained from the line-of-sight calibration. Reference measurements are performed 

immediately before every grid calibration measurement. Fig. 8 indicates that the errors are 

all within about ±30 μm over the one-month period.

In addition to the errors, the standard deviation of any given center-to-center length (based 

on the 14 calibrations performed over the one-month period) is computed to assess the 

stability of the grid. Because a total of 66 different center-to-center lengths can be 

computed from 12 measured coordinates (nine integration sphere centers and three 38.1 

mm (1.5 in) SMR nests), the standard deviation for each of the lengths was computed. The 

pooled standard deviation was then determined to be 11 μm, indicating excellent stability 

of the grid over this period.
12

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

Fig. 8 Errors in three horizontal and three diagonal lengths over a one-month period

3.3.3 Uncertainty budget

Mechanical terms: The uncertainty in locating a single point in space resulting from error 

sources associated with the laser tracker, the centering of the integration sphere with the 

SMR, and the concentricity of optical and mechanical center of the SMR is first 

determined. The point coordinate uncertainty is then propagated to estimate the uncertainty 

in the point-to-point length.

Tracker contribution: Because of the averaging from front-face and back-face, and 

averaging achieved as a result of measurements from four positions that samples 

different regions of the azimuth and elevation angle encoders, mostly random error 

components remain. Therefore, the experimentally determined one standard 

deviation repeatability of 0.1 mdeg is taken as the standard uncertainty in the angles 

measured by the tracker. At a distance of 6 m (which is the farthest distance the 

tracker is from the grid), the standard uncertainty along the horizontal and vertical 

angle axes are 11 μm each. At a distance of 6 m, the tracker is sufficiently far 

enough from the grid that the component of the tracker’s ranging error along the 

direction of the length is negligible, therefore only the angular errors of the tracker 

are considered. 

Integration sphere concentricity with SMR: The concentricity of all 14 integration 

spheres are measured on a CMM. The experimentally measured one standard 

deviation is 5 μm and this value is considered as the standard uncertainty in the 

determination of the coordinate.

SMR concentricity: The concentricity specification of the physical and optical 

center of all SMRs used is 2.5 μm. Assuming this value as the upper bound with 

any value within that bound as equally probable, the one standard deviation 

uncertainty in the measured coordinate is 1.4 μm.

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

0.05

E F E G FG P Q P R QR Errors in mm

3/10/2016, Tracker A

3/11/2016, Tracker A

3/14/2016, Tracker A

3/15/2016, Tracker A

3/16/2016, Tracker B

3/17/2016, Tracker B

3/19/2016, Tracker B

3/22/2016, Tracker C

3/28/2016, Tracker B

3/28/2016, Tracker A

3/30/2016, Tracker B

3/31/2016, Tracker A

4/6/2016, Tracker A

4/8/2016,Tracker B
13

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

Standard uncertainty in the length: The terms above result in a combined standard 

uncertainty of 12 μm for a point coordinate or 12√2 = 17 μm for a point-to-point 

length.

Thermal effects: The diagonals are the longest point-to-point lengths in the grid and they 

span 3.2 m horizontally on the concrete floor and 3.2 m vertically on aluminum rails. 

Assuming ±0.5 °C fluctuation of the room temperature as the upper bound with any value 

within that bound as equally probable, and assuming expansion coefficients of 22 x 10-6/°C 

for aluminum and 12 x 10-6/°C for concrete, the standard uncertainty in the length is 

determined to be 22 μm. The uncertainty due to changes in refractive index (and therefore 

measured range) is not considered because that is an order of magnitude smaller than the 

uncertainty due to thermal expansion of the floor/rail.

Combined standard uncertainty: Combining the mechanical contribution of 17 μm and the 

thermal contribution of 22 μm, the combined standard uncertainty in the length is 

determined to be 28 μm. The expanded uncertainty (k = 2) is 56 μm.

4. The inside test

4.1 Test setup

The inside test consists of two integration spheres placed about 6 m apart from each other, 

and facing each other (see Fig. 9). The 3D imaging system is placed inline, in between, and 

equidistant from the two spheres. The center-to-center distance between the spheres is 

calibrated using a laser tracker and three registration nests. 

Fig. 9 The inside test setup showing integration spheres M and N, and the three 

registration nests

M

N

Registration nest 2

Registration nest 3

Registration nest 1
14

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

4.2 Calibration procedure

The inside length is calibrated using a laser tracker in conjunction with three registration 

nests. Two of the registration nests are placed next to the integration spheres, while the 

third is placed at approximately the mid-point but slightly offset from the line joining the 

two ends of the length (see Fig. 10). The tracker is placed on one side of the setup (Position 

1) and the coordinate of the center of the integration sphere closest to the tracker

(integration sphere 1) is determined in a coordinate system established using the

registration nests. The tracker is then moved to the other side (Position 2), and the

coordinate of the center of the other integration sphere (integration sphere 2) is determined

in the same coordinate system established again using the registration nests. Because the

integration sphere centers are measured in the same coordinate system, the integration

sphere center-to-center distance is simply the Euclidean distance between the two

measured sphere center coordinates.

Fig. 10 Calibrating the center-to-center distance

As a point of detail, the tracker measurements in the inside calibration are always 

performed by averaging front-face and back-face measurements. Further, in order to reduce 

the influence of systematic errors, tracker measurements are performed from four different 

azimuth orientations (each 90° apart) from each position. That is, with the tracker placed 

on Position 1 (see Fig. 10), the three registration nests and the center of integration sphere

1 are first measured (all of them as the average of front and back-face). A coordinate system 

is established with registration nest 1 as the origin, nest 2 defines the X axis and nest 3 the 

XY plane. The coordinates of the center of the integration sphere in that coordinate system 

is determined. The tracker then is rotated by approximately 90°, the three nests and the 

center of the integration sphere are measured, and the coordinates of the integration sphere 

center is again determined. This process is repeated another two times, and all four sphere 

center coordinates are averaged. A similar process is performed with the tracker on Position 

2.

Tracker

on Position 1

Registration 

nest 1

Registration 

nest 2

Registration 

nest 3 Integration 

sphere 1

Integration 

sphere 2

Tracker

on Position 2

IUT
15

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

4.3 Validation experiments

In order to determine the magnitude of the errors associated with registration, a simple 

experiment is performed. The integration spheres are replaced with 38.1 mm (1.5 in) SMR 

nests, referred to as test nests in Fig. 11. The center-to-center distance between the two test 

nests is calibrated using a tracker placed along the line joining these two nests. The 

registration based length between the same nests is then determined according to the 

procedure described Section 4.2. Table 2 shows the difference between the registrationbased length and the line-of-sight calibration performed on four different days. The errors 

are all smaller than 10 μm, well within the uncertainty claimed in the next section.

Fig. 11 Schematic of a setup to validate the registration based center-to-center 

measurement.

Table 3 Registration based length compared against line-of-sight calibration

Line-ofsight 

reference 

(mm)

Registration 

based 

measurement 

(mm)

Error (mm)

Set 1 6414.578 6414.572 -0.006

Set 2 6414.582 6414.578 -0.004

Set 3 6414.582 6414.585 0.003

Set 4 6414.580 6414.585 0.005

4.4 Uncertainty budget

Mechanical terms: The uncertainty in locating a single point in space resulting from error 

sources associated with the laser tracker, the centering of the integration sphere with the 

SMR, and the concentricity of optical and mechanical center of the SMR is first 

determined. The point coordinate uncertainty is then propagated to estimate the uncertainty 

in the point-to-point length.

Tracker contribution: Monte Carlo Simulation (MCS) of the registration based 

measurement process indicates a standard uncertainty of 8 μm in measuring the 

Tracker

on Position 1

Registration 

nest 1

Registration 

nest 2

Registration 

nest 3

Line-of-sight tracker

IUT

Test nest 1 Test nest 2
16

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

coordinate of each of the targets. As in the case of the grid calibration, the 

manufacturer specifications are not used as input to the MCS. Instead, the 

experimentally determined one standard deviation repeatability of 0.1 mdeg is 

considered as the standard uncertainty in the angles measured by the tracker, and 

this value is scaled by the measured range to the targets. This translates to a standard 

uncertainty of about 2 μm at the near target (which is nominally 1 m away from the 

tracker) and 12 μm at the far target (nominally 7 m away) along the transverse 

directions. Along the ranging direction, the manufacturer’s specification of 10 μm 

is used as the upper bound and any value inside of that is considered as equally 

probable. It is noted the tracker used here is from a different manufacturer than the 

one used in the experiments described in Section 3.2.3, hence the different MPE.

Integration sphere concentricity with SMR: The concentricity of all 14 integration 

spheres are measured on a CMM. The experimentally measured one standard 

deviation is 5 μm and this value is considered as the standard uncertainty in the 

determination of the coordinate.

SMR concentricity: The concentricity specification of the physical and optical 

center of the SMRs is 2.5 μm. Assuming this value as the upper bound with any 

value within that bound as equally probable, the one standard deviation uncertainty 

in the measured coordinate is 1.4 μm.

Standard uncertainty in the length: The terms above result in a combined standard 

uncertainty of 10 μm for a point coordinate or 10√2 = 14 μm for a point-to-point 

length.

Thermal effects: Assuming ±0.5° fluctuation in temperature in the room as upper bound 

with any value inside that bound as equally probable, and assuming thermal expansion 

coefficient of 12 x 10-6/°C for concrete, the standard uncertainty is determined to be 21 μm 

for a 6 m length. As in the case of the grid calibration, the uncertainty due to changes in 

refractive index (and therefore measured range) is not considered because that is an order 

of magnitude smaller than the uncertainty due to thermal expansion of the floor.

Combined standard uncertainty: Combining the mechanical contribution 14 μm and the 

thermal contribution of 21 μm, the combined standard uncertainty in length is determined 

to be 25 μm. The expanded uncertainty (k = 2) is 50 μm.

5. Relative range tests

5.1 Test setup

The relative range tests are performed in the 60 m tape-tunnel facility at NIST (see Fig. 

12). The 3D imaging system is placed on one end of the tape tunnel while the laser tracker 

is placed at the other end. Relative range tests are first performed with the plate-sphere 

artifact and subsequently performed again with the bare-sphere artifact. The near position 

of the artifact from the 3D imaging system is generally specified by the manufacturer while 
17

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

a few test scans are performed to determine the far position. For most of the systems tested 

during the runoff, the near position was between 1 m to 3 m while the far position was the 

full length of tunnel, i.e., 60 m. 

Fig. 12 Relative range tests showing a 3D imaging system placed in front of a) the platesphere artifact and b) the bare-sphere artifact. The laser tracker is at the far end of the 

tape-tunnel.

Fig. 13 Relative range tests using the bare-sphere and the plate-sphere artifact

The artifact is first placed at the near position (position 0 in Fig. 13), facing the 3D imaging 

system, and approximately centered on the line joining the 3D imaging system and the laser 

tracker. This position is the reference for all displacement calculations. Data is 

Bare-sphere, 

position 0

3D imaging 

system

Laser 

tracker

Bare-sphere, 

position 1

Bare-sphere, 

position 5

Plate-sphere, 

position 0

3D imaging 

system

Laser 

tracker

Plate-sphere, 

position 1

Plate-sphere, 

position 5

(a) (b)
18

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

simultaneously acquired from both the instruments. The artifact is then moved to positions 

1, 2, 3, 4, and 5 while data are acquired from both instruments at each position. 

Displacement as recorded by both instruments (with respect to position 0) and the relative 

range error are calculated. 

Fig. 12 shows a 3D imaging system in the foreground, the artifact placed in front of it, and 

a laser tracker at the far end of the tape-tunnel. Fig 13 shows a schematic of the relative 

range test using the bare-sphere and the plate-sphere artifact.

5.2 Uncertainty budget

The relative range test involves measuring the displacement between two positions of the 

target, a reference position (denoted by position 0 in Fig. 13) and a test position (any of 

positions 1 through 5 in Fig. 13), both of which are along a radial direction of the 3D 

imaging system. The reference position of the target (position 0) is assumed to be 61 m 

from the laser tracker for the uncertainty calculations. The test position (positions 1 through 

5) of the target may be anywhere between the laser tracker and position 0 of the target.

Primary contributors to the uncertainty in the displacement measurement are the errors 

associated with the range measurement of the laser tracker and the thermal environment. 

The thermal expansion of the concrete floor is not of any consequence in this case because 

both instruments measure the range to the artifact simultaneously. 

A general equation for the uncertainty in determination of the range to any of the target 

positions is first presented. The uncertainty in the displacement is then calculated for some 

example lengths.

Tracker contribution: Using manufacturer specification of 10 μm as upper bound and 

assuming rectangular distribution, the one standard deviation point coordinate uncertainty 

is determined to be 5.8 μm along the ranging direction.

Temperature contribution: The temperature sensor of the laser tracker only monitors the 

temperature at one point along the laser path. Because long lengths are measured, the 

tracker is unable to adequately compensate for the changing thermal environment. 

Therefore, the uncertainty in range measurements because of temperature fluctuations 

along the laser path is considered. Assuming ±1 °C fluctuation in temperature along the 

laser path with any value inside that bound as equally probable, and with temperature 

influence of 1 x 10-6/°C (this is due to the change in refractive index based on Edlén’s 

equation), the standard uncertainty in range measurement at a distance of r meters is r/√3

in units of micrometers.

The uncertainty in determining range to the target is the root sum square of the two terms, 

thus it is√

r

2

3

+ 36 in units of micrometers. The uncertainty in the displacement is the root

sum square of the uncertainty in the range at two ends of the length. Table 4 presents the 

uncertainty for some displacement measurements.
19

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

The expanded (k = 2) uncertainty for the ranging measurements varies between 92 μm for 

a 12 m length to 72 μm for a 60 m length. One of the drawbacks of positioning the laser 

tracker at the opposite end of the tape tunnel from the 3D imaging system is obvious from 

Table 4. The uncertainty in the reference measurement is larger for smaller displacements. 

This is because a small displacement of the target with respect to the 3D imaging system 

requires measuring two positions that are both far away from the laser tracker, hence the 

larger uncertainty.

Table 4 Uncertainty for some displacement measurements

Test 

#

Range to 

the 

reference 

position (m)

Uncertainty in 

the reference 

position (μm)

Range to 

the test 

position 

(m)

Uncertainty 

in the test 

position

Displacement 

(m)

Standard 

uncertainty in 

the 

displacement 

(μm)

1 61 36 49 29 12 46

2 61 36 37 22 24 42

3 61 36 25 16 36 39

4 61 36 13 10 48 37

5 61 36 1 6 60 36

It should be noted that the temperature variation in the tape tunnel is generally within ±0.2° 

and therefore the uncertainty in the realization of a 60 m length is about 1/5th the amount 

claimed in Table. 4. The reason for the conservative estimate of ±1°C fluctuation in 

temperature is because of the large number of people in the room during the runoff.

6. Summary

This report describes the facilities established by the DMG at NIST to realize all of the test 

procedures in the current draft of the ASTM E57.02 standard for point-to-point 

performance evaluation of spherical coordinate 3D imaging systems. This report describes 

the test setups, calibration procedures, validation data, and uncertainty budgets. 

7. Acknowledgements

The authors thank all participants of the instrument runoff and the biweekly E57 meetings 

for supporting this standards effort, Dr. Kam Saidi and Dr. Steve Phillips for helpful 

discussions, and Dr. Craig Shakarji and Dr. Marek Franaszek for carefully reviewing this 

document.

8. References

1. WK43218. ASTM E57.02, New test methods for evaluating the performance of

medium-range, spherical coordinate, 3D imaging systems for point-to-point distance

measurements
20

This publication is available free of charge from: https://doi.org/10.6028/NIST.IR.8153 

2. NISTIR 8152, Report on the May 2016 ASTM E57.02 instrument runoff at NIST, Part

1 - Background information and key findings, 2016

3. B. Muralikrishnan, S. Phillips, D. Sawyer, Laser trackers for large scale dimensional

metrology – A review, Precision Engineering, 44, 2016, p. 13-28

4. B. Muralikrishnan, C. Blackburn, D. Sawyer, S. Phillips, and R. Bridges, Measuring

scale errors in a laser tracker’s horizontal angle encoder through simple length

measurement and two-face system tests, Journal of Research of the NIST, 115 (5),

p.291-301, 2010