Measurement Assurance for Cellbased Assays: Cell Count

John T. Elliott Jr., Cell Systems Science Group Leader

National Institute of Standards and Technology (NIST), 

Department of Commerce, Gaithersburg, MD 20899

NIST-FDA Cell Counting Workshop, April 2017
What is Measurement Science?

• A systematic approach that informs on the 

comparability and trust in a measurement result

– Data-based decision making

• Components of a measurement:

– Value- it is on a scale; enables compared to 

other measurements

– Uncertainty- variability in the 

measurement; enables statistics

– Evidence- evaluation of the measurement 

system; confidence

1

2

Controls meet 

specification
What is Measurement Science?

• Treat the assay as a measurement process

• Control experiments provide evidence that the measurement 

process is proceeding as expected

• Adapt the process quality tools to cell assays

– Cause and effect diagram

– In-line controls

– Control charting

– Sensitivity analysis

– Experimental design

– Interlaboratory testing

– Acceptance specifications

• Increases confidence in the measurement
A Cell Count Measurement Process

Lin-Gibson, S., Sarkar, S., Elliott, J. T., & Plant, A. L. (2016). Understanding and managing sources of variability in cell measurements. Cell Gene 

Therapy Insights 2016;2(6),663-673.
Appropriate Data for 

Statistical Analysis

Outlier Determination

Variability in 

a Cell 

Counting 

Assay 

Consumables

Pipette Tips

Pipettes/Pipettors

Refrigerators/ 

Freezers Stability

Water Baths 

(Fill Volume/Temp)

Equipment/ 

Consumables

Reagents/Raw 

Materials

Electrical Issues

Disruptions/Distractions

Cleanliness

Temperature

Humidity

Environment

Protocol Consistency Control/Reference 

Material

Sample Collection

Thaw Procedures

Sample Preparation

Dilution Procedure

Time

Analyst Training

Procedures Data Acquisition Data Analysis

Budding

Formulation

Morphology

Concentration

Subsampling

Homogeneity

Stability 

Clumping

Sample 

Preparation

Biological Sample Instrument

Cause

Effect

Reference 

Material Stability

pH

Lot-to-Lot Consistency

Osmolality

GMP Grade

Age 

(Expiration)

Light 

Sensitivity

Stability

Temperature 

Sensitivity

Toxicity

Purity/

Impurity Data Recording/

Recovery

Settings/Set-up

Instrument Variability 

(Trending)

Linearity/LOD

Cell Specific

Algorithm Instability

Clumping

Imaging Artifacts

Debris

Simon, C. G., Lin-Gibson, S., Elliott, J. T., Sarkar, S., & Plant, A. L. (2016). Strategies for Achieving Measurement Assurance for Cell Therapy Products. 

Stem Cells Translational Medicine, 5(6), 705-708.

Cause and Effect Diagram for Generalized Cell 

Count Measurement
Use Case: Expanding a Stock Cell Line

0

5000

10000

15000

20000

25000

April-May 2016 Cell Count/mL

?

Acceptance 

range

• Thaw a stock vial of cells, expand cells for 4 days, count cell 

concentration

• Is this right?

• What evidence do I have that provides measurement 

assurance (i.e. confidence)?
Dissecting the Cell Count 

Measurement

Measurement=

#cells

volume

Assumptions:

1. Single cell suspension

2. Count all cells

3. Known volume tested

4. Reagents/Instrument work as expected

5. Linear range, above LOD, etc

• Measurement science approach (i.e. what is evidence that 

these assumptions are true?)

• Are there control experiments that can detect the 

following? Reagents bad

Cells in aggregates

clog in fluidics

Change in instrument 

detection

Not in linear range

Volume detection bad

Fluidics malfunction
What does the evidence look like?

• Data from control experiments provide the 

evidence for measurement assurance.

• They are method dependent and inform on parts 

of the measurement process.

• Can be in-frequent measurements (i.e. linear 

range, LOD, instrument settings, matrix effects) 

• Can be in-line measurements that provide 

confidence in the measurement system/process 
Example: Cell Count via Imaging

Lin-Gibson, S., Sarkar, S., Elliott, J. T., & Plant, A. L. (2016). Understanding and managing sources of variability in cell measurements. Cell Gene 

Therapy Insights 2016;2(6),663-673.
What might the data look like?

Reagent quality, 

illumination and detection 

system

Evidence for:

Cell clump, sample prep

Count Image Measurement

Distribution of pixel 

intensities

In spec

Out of spec

intensity

Normalized pixel #

In spec

Object size

Number

Out of spec 

(too many 

aggregates)
Data Sheet:

Cell count/mL=20000

Aggregate <10%

Pixel intensities within spec

Control beads size within spec

Cell size within spec

Cell count in linear range

Cell count above LOD

Test result

Clumps not affecting count

Reagents and imaging performed 

as expected

Magnification as expected

Image analysis performing as 

expected

Fit for purpose

Fit for purpose

In-line measurements

Sensitivity 

experiments

Evidence provides Confidence in the 

Measurement!
Summary

• Treat the assay as a measurement 

process.

• What are the sources of variability?

• Dissect the primary measurement 

and think about assumptions.

• Design sensitivity experiments and in-line measurements to validate 

assumptions. Evidence!

• Method dependent, cell dependent, sample prep dependent, but 

identification of generalized methods used for measurement assurance 

in cell counting would be applicable to many use cases.

• Research in these measurement assurance strategies could lead to 

standards in the future.