Measurement Assurance Case Study

Nanofiber Diameter 

Carl G. Simon, Jr., Ph.D.

Biosystems & Biomaterials Division

National Institute of Standards & Technology

Gaithersburg, MD, U.S.A.

carl.simon@nist.gov

Contributors: 

Nathan Hotaling, Kapil Bharti, Haydn Kriel
• In

M

• St

Re

• Re

M

Validatable

Protocol

Documentary 

Standards

Strategies for Achieving Measurement Assu

“Validatable” not “validated”, 

validation is a continuous proc

must be established be each l

implements the measurem

Measurement 

Assurance

Operator 

Training Ishikawa Diagram 

(Cause & 

Effect) 

Sensitivity 

Testing 

(Design of 

Exps.)

Reference 

Materials

Measurement

Process 

Flow 

Diagram

Increase

“n” Automation

Process 

Controls

Performance 

Specifications

InterLaboratory 

Comparison 

Study

Orthogonal 

Measurements

• Measurement Assurance: Evalua

variability in a measurement to imin results for use in decision-mak

• Underpins development of docum
• Measurement assurance strategies for measur

nanofiber diameter

• 2013 ASTM Workshop on Scaffold Standards & 

Measurements (Indianapolis, IN, USA): #1 neeidentified was “better measurements for scaffostructure”
DiameterJ: Automated 

Image Analysis

Manual 

Human 

Increasing 

“n” Automation

Increasing “n” (number of measurements) enables better statistics & better modeling of the probability distribution function (histogram) 

Automation increases the number of measurements & reduces human bias 

• Current practice is manual 

measurement using a line tool in imaging software (ImageJ)

• Slow (10 min/image) & low n• Human bias
103 Synthetic Images

Reference 

Materials

Digital Synthetic Image 

with 1 Diameter (10 px)

Reference Materials are homogeneous& stable in regard to specified properties for use in calibration, to serve as a control or to serve as a reference point for comparability (ISO Guide 35)
Steel Wire with 

Known Diameter

53 ga. wire

Wire 

Gauge

Manufacturer 

Reported 

Dia. (μm)

48 31.0 50 25.5 53 16.75 

• Narrow gauge stainless steel wire 

(HSM Wire)

• Manufacturer measures dia. with 

resistivity & calipers

• Wire dia. verified with light microscopy 

& human manual segmentation in SEM

Reference 

Materials

200 μm 100 μm

Brightfield, 48 ga. wire SEM, 53 ga. wire
0.3 mm

48 Gauge Steel Wire 

Reference Steel Wire with Known Diamete

Materials
Electrospin Fibers

(Sample 

Preparation)

Image in Scanning 

Electron 

Microscope

Segment Images

Measure Fiber 

Diameter in 

Segmented Images

20 μm

0

200400600

0

Frequency

Measurement

Process 

Flow 

Diagram

Flow Diagram is a tool to formally map a meaprocess so that each step can be considered fcontribution to measurement uncertainty

Steel Reference Wire

Synthetic I
Orthogonal 

Measurements

• Orthogonal Measurements

• More precise than the measurement that you are trying t

(slower, expensive, harder)

• Based on a different physical principle

• Synthetic images

• Counted pixels by hand (very IMPORTANT, MSPaint did

• Steel reference wires

• Manufacturer measured resistivity• Manufacture measured with calipers• Optical imaging of fibers• Human manual measurement with ImageJ line tool in SE

• Electrospun polymer fibers

• Human manual measurement with ImageJ line tool in SE

Orthogonal 

Measurements Orthogonal Measurements

in a measurement result is emultiple measurement methsimilar value of a material pr
Sensitivity Testing 

(Ruggedness Testing) (Design of Exp

103 synthetic images:• Different diameters• Straight vs curved• Aligned vs disordered• Multiple diameters

Sensitivity Testing can identify key measurement that must be controlled to make the measurement m

Sensitivity 

Testing 

(Design of 

Exps.)

Multimodal Diameter S

2 Diameters 6 D

(Failed on 10 Diame
Establishing Fiber Diamete

Measurement Range 

0%5%

10%15%20%25%30%

1 10 

Percent Error

Line Diameter (Pixels)

Too small = too few 

pixels per fiber Too big = not enough 

per image, edge ef

Dia. = 250 px

Dia. = 3 px

Sensitivity 

Testing 

(Design of 

Exps.)
Effect of Segmentation (24 Algorithms 

Sensitivity 

Testing 

(Design of 

Exps.)
200 μm

Global Otsu Global Min Error 

Machine Learning

Raw SEM microgra

53 ga, steel referenc

(fiber dia. 16.7 μ

Sensitivity 

Testing 

(Design of 

Exps.)

Effect of Segmentation on DiameterJ 
Process Controls

Process Controls are procedures to monitor critical points in a measurement process to check that steps are performing according to specifications

Visually compare raw image with:• Segmentations• Euclidian distance map• Histogram (bimodal?)• DiameterJ results

Manufacturer’s Fiber 

Dia. = 31.0 μm

DiameterJ Results

Mean 31 μm

Mode 32 μm

Median 32 μm

Raw Image of 

Segemented Im

Euclidian Dis

Process 

Controls
Raw Im

Process 

Controls

• To help identify errors, DiameterJ

has a locator tool which labels locations where fibers diameters of a given range were found

• Fibers along image edge, poor 

segmentation or fiber overlap can yield errant measurements

Red Lines = 1 px to 255 px Red Lines = 40 
• Fibers must be at least 10 px in diameter

• Fibers should not be greater than 10% of the smallest dimensio

image

• Example: SEM imaging of 500 nm fibers should be conduc

magnification between 1500X and 10000X for a 1280 px by image capture

• Visual Examination: Fiber diameters in raw images qualitatively 

segmentations & DiameterJ results

• For multimodal distributions, modes must be separated by more • In the system tested, 6 fiber dia. peaks is maximum # of peaks • If you don’t meet these specifications...then test result should b

questioned (possibly rejected) 

Performance 

Specifications

Performance Specificatioestablished by the user frotesting & charting process test specifications are not results can not be used in 

Performance 

Specifications
Web Training Module

Web training where users 

download & analyze 

images with DiameterJ

Operator 

Training

Operator Training improves measurement precision to improve comparability between different operators & labs
Comparing Operator Performan

Before & After Training

• IN PROGRESS: Intra-lab comparability with Matt Beck

(Univ. of Akron, USA), 17 students analyzing images before/after training to assess improvement

• Test images of reference wires (48 ga. & 50 ga.)• Protocol Refinement: Keep magnification constant

InterLaboratory 

Comparison 

Study

Inter-Laboratory Comparisons asses the robustness of an assay across different labs & results are used to refine the protocol
Operator 

Skill

Operator Skill

Voltage Current

Working Distance Vacuum Pressure

Gas in Chamber

Charging

Instrument Settings

2° Electron 

Emission

Sputter Coat Operator Skill

Instrument Model

Pixels per Fiber

Fiber Size Relative 

to Image Size 

(Magnification)

Mean

vs

Mode 

vs 

Peak Fitting

Automated Analysis 

with Algorithm

Operator Doing Manual 

Line Tool Analysis

Segmentation Algorithm 

(for Automated Analysis)

Va

Fib

M

Sample Properties Image Capture

Image Analysis Data Analysis

Cause

Effect

Ishikawa Diagram Ishikawa 

Diagram 

(Cause & 

Effect)

Developed in 1960s by Kaoru Ishikawa who pioneered qualmanagement processes in the Kawasaki shipyards

Ishikawa Diagram is graphical tool to identify potential sources of variability in a process
Comparing Operator 

Performance Before 

& After Training

• IN PROGRESS: Intra-lab comparability study where 

participants analyze images before/after training to assess improvement

• Test images are mixture of 48 ga. & 50 ga. Wire• NEW SPECIFICATION: Use constant magnification

InterLaboratory 

Comparison 

Study

Inter-Laboratory Comparisons asses the robustness of an assay across different labs & results are used to refine the protocol

0

10203040

Before, 50 ga

Fiber Diameter (μm)

n = 4

48 ga

(31.0 μm)

50 ga

(25.5 μm)

• Before Trainin

operators identdistribution

• After Training

identified the bi

100 μm
Web:

• http://imagej.net/Diame• http://fiji.sc/DiameterJ• https://github.com/NHoPapers: • Hotaling NA, et al. (201

validated open source measurement tool. Bio

• All Data & Images: Hot

Dataset for the validatiDiameterJ, an open sodiameter measuremenpress.

Disseminat
Measurement 

Assurance

Operator 

Training Orthogonal 

Measurements

Measurement 

Process Flow 

Diagram

Reference 

Materials

Sensitivity 

Testing 

(Design of 

Exps.)

Increase

“n” Automation

Performance 

Specifications

Process 

Controls

InterLaboratory 

Comparison 

Study

Ishikawa 

Diagram 

(Cause & 

Effect)

• S

p

• S• E







• Fiber geo• Fiber dia• # of mode

• 

• 















“Intra”-lab underway (DiamterJ training module on the web)

• Fibers >10 px in dia.• Fibers < 10% of the smallest image 

dimension 

• Fiber dia. in raw images qualitatively 

agree with DiameterJ results

• Segmentations are of good quality 

when visually compared to raw image

• Multimodes separated by >3 px• 6 fiber dia. peaks is maximum

• Compare raw image to:

• Segmentations• Euclidian distance map• Histogram (bimodal?)• DiameterJ results

• Identify errors w/ locator tool

Assuring Nanofiber 

Diameter Distribution 

Measurement
Summary

• “product consistency & lack of standards is 

possibly the single greatest challenge facing the field”

• Approach measurement process as a 

manufacturing process

• Measurement Assurance: Evaluate & reduce 

variability in order to improve confidence results to support decision-making (before writing a standard)

Thank yo

Measureme

Assurance

Operator 

Training 

In

Automation

Performance 

Specifications

Process 

Controls

InterLaboratory 

Comparison 

Study

Ishikawa 

Diagram 

(Cause & 

Effect)



 







