Cell Counting Validation

NIST-FDA Cell Counting Workshop

Gaithersburg, MD

M. McLaughlin

10 April 2017
Confidential – For Celgene Internal Use Only

Outline

• Background

• Development (Instrument, Assay)

• Validation

• Summary

2
Confidential – For Celgene Internal Use Only

3

BACKGROUND
Confidential – For Celgene Internal Use Only

A Cell Counting Problem

4

Ideal cell count Typical cell count

due to lack of 

cell reference 

standard

Unacceptable 

cell count

DC

http://wonderwalsh.net/unit-1---sciencehttp://www.clker.com and-measurement.html /
Confidential – For Celgene Internal Use Only

Counting Methods Evaluated by Celgene for PDACs 

(2010)

Hemacytometer Nuclear 

staining Image-based Coulter method Flow cytometry

Cell counting

Automated 

sampling X X

Automated

analysis X

Instrument 

differences

Other parameters

Viability X

Cell size X X X

Identity X X X X

5
Confidential – For Celgene Internal Use Only

Concentration - mL

Total Cells-Auto Total Cells-Manual

12.0E+06,

11.0E+06,

10.0E+06,

9.0E+06,

8.0E+06,

7.0E+06,

6.0E+06.

5.0E+06,

4.0E+06,

3.0E+06,

6.77246e+006

8.80676e+006

Total Cell Density - Automated v Manual Counting

SD = 4.40E+05

SD = 1.76E+06

Percent Difference = 23.10%

Manual vs. Automated Image-Based Cell Counting

6

• Main factors that affected manual cell counting: 

1. Operator

2. Sample preparation

• Automated cell counting increased precision

• Difficult to gage accuracy

Percent 

Cell

Difference = 23%

n=6 operators
Confidential – For Celgene Internal Use Only

Factors to Consider for Cell Counting

Instrument Assay

 Instrument-to-instrument

differences

 Controls: beads or cells?

 Objective data 

analysis/gating

 Cell concentration

 Diluent, dilution factor

 Cell matrix

 Cell type

 Operators

 Controls: beads or cells? -

difficult to gage accuracy

7
Confidential – For Celgene Internal Use Only

DEVELOPMENT

(Instrument)

8
Confidential – For Celgene Internal Use Only

Instrument Differences

9

Cell Conc.

Instrument

5.0 10.0

104 105 Prep SC12 104 105 Prep SC12

13

12

11

10

9

8

7

6

5

4

3

2

1

Total cells/mL (x10^6)

95% CI for the Mean

Total cells/mL (x10^6) Instrument Comparison

104 105 Prep SC12

9.0

8.5

8.0

7.5

7.0

Instrument

Mean

Main Effects Plot for Total cells/mL (x10^6)

Data Means

Instrument differences 

are evident

p < 0.01
Confidential – For Celgene Internal Use Only

Controlling the Instrument

10

Advantages Disadvantages

Wide range of concentrations 

(off-the-shelf and custom-made) Beads ≠ cells

Ease of use 

(i.e. single use vials, no dilution)

Actual concentration may not 

always match with C of A

Can be used for instrument 

control/calibration Possible vial to vial variability

Helps identify the concentration 

range of the instrument Settling of beads

Typically precise results

Concentration 

beads

Cells ≠
Confidential – For Celgene Internal Use Only

Instrument Calibration with Concentration Beads

11

Nominal

Calibrated -25%

Calibrated +25%

Calibrated Instrument

8.40E+067.0 E+063.97E+061.01E+064.52E+051.53E+058.40E+067.0 E+063.97E+061.01E+064.52E+051.53E+058.40E+067.0 E+063.97E+061.01E+064.52E+051.53E+05

40.0%

30.0%

20.0%

10.0%

0.0%

-10.0%

-20.0%

-30.0%

-40.0%

%Bias

-25.0%

0.0%

25.0%

Total Concentration Beads/mL

• Intentional skewing of calibration demonstrated how sound the 

calibration method was

• Implemented bead control to ensure instrument is working as expected

• Used one bead concentration to serve as a pseudo system suitability 

control

Calibration is necessary!
Confidential – For Celgene Internal Use Only

DEVELOPMENT

(Assay)

12
Confidential – For Celgene Internal Use Only

Diluents and Dilutions

13

• Choice of diluents and dilution factor impact cell counting 

measurements

• Highlights that accuracy is unknown

Target = 

7.5 x 106 cells/mL

In-process solution
Confidential – For Celgene Internal Use Only

Cell Matrix Interference

14

Cell matrix

Total events/100 

images

PBS 1

Culture medium 4

Trypsin solution 4

Cryopreservation 

solution 24

In-process solution 67

• Cell matrix can interfere with cell counting

• Selected a common matrix to dilute samples in with 

the least number of events

• Identified best dilution in this matrix
Confidential – For Celgene Internal Use Only

y = 1.1107x - 0.1161

R2 = 0.9994

y = 1.1132x - 0.0729

R2 = 0.9955

0

2

4

6

8

10

12

0 2 4 6 8 10 12

Mean Measured Cell Concentration (x 10^6 

cells/mL)

Expected Cell Concentration (x 10^6 cells/mL)

Target Cell Conc. Actual Cells, Dil 1

Actual Cells, Dil 2 Linear (Target Cell Conc.)

Linear (Actual Cells, Dil 1)

Instrument Range vs. Target Cell Range

15

Target Cell 

Concentration 

(undiluted)

Actual Cell 

Concentration 

counted

(diluted)
Confidential – For Celgene Internal Use Only

Statistical Design of Experiments (DOE)

16

Recipe for DOE

https://www.moresteam.com/toolbox/pics/4081.gif
Confidential – For Celgene Internal Use Only

Pre-validation DOE

Factors Level 1 Level 2

Pre-Dilution Hold Time 0-10 60-70

Post Dilution Hold Time 0-20 60-80

Dilution 2X 10X

Mix Method Manual Vortex

Instrument 1158185 1362331

Operator A B

Pipette Manual Electronic

Mixing Cycles 1 3

Aspiration Cycles 1 3

17

Plackett-Burman design:

a. Factors = 9

b. Assay runs = 24

c. Cell concentrations = 3
Confidential – For Celgene Internal Use Only

Pre-Validation Results

0-10 60-70

3.4

3.3

3.2

0-20 60-80 A B

Manual V ortex

3.4

3.3

3.2

C PD DQ C A B

Eppendorf V iaflo

3.4

3.3

3.2

M 1 M 3 A 1 A 3

Pre-Dilution Hold

Mean

Post-Dilution Hold Dilution

Mix Method Instruments O perators

Pipette Mixing C y cles A spiration C y cles

Main Effects Plot for Total Cells/mL (x10^6)

Data Means

18

p-value 0.993

p-value 0.936

p-value 0.787 p-value 0.770

p-value 0.986 p-value 0.766

p-value 0.935

p-value 0.546 p-value 0.663
Confidential – For Celgene Internal Use Only

19

VALIDATION

STRATEGY
Confidential – For Celgene Internal Use Only

Validation Parameters

Parts of 

Measurement 

System:

Instrument

(beads)

Assay

(cells)

Accuracy X

Precision

Linearity

Range

Specificity X

Dilution Integrity X

20
Confidential – For Celgene Internal Use Only

Experimental Design

21

Beads: 

a. 5 concentrations

b. 5 replicates

Cells: 

a. 4 concentrations

b. 2 dilution factors

c. 9 replicates

(1) Pool vials, 

if necessary

(2) Make 3 

dilutions from 

pool/sample 

vial

(3) Aliquot 

diluted sample 

into 3 cups

2 instruments:

a. 3 replicate 

assays per 

instrument

N = 582 data points http://www.beckman.com/particle/instrument

s/cell-sizing-and-processing/vi-cell-xr

2 operators
Confidential – For Celgene Internal Use Only

Validation Results (Concentration Beads)

Concentration Beads

Nominal 

Bead Conc. Accuracy Repeatability

Intermediate 

Precision Linearity Range

(x 106

beads/mL)

Mean Bias 

(%) CV CV R2

(x 106

beads/mL)

0.15 -3.7 1.5 to 5.8% 1.2%

0.999 0.15 to 3.86

0.52 -7.2 2.0 to 4.3% 1.0%

1.00 -1.0 4.9 to 7.2% 2.4%

2.02 -9.0 1.2 to 2.2% 1.2%

3.86 -4.8 3.3 to 5.9% 0.9%

22
Confidential – For Celgene Internal Use Only

Validation Results (Cells)

23 Cells

Conc. Dil. Repeatability

Intermediate 

Precision Linearity Range Specificity

Dilution 

Integrity

(x 106

cells/mL) CV CV R2

(x 106

cells/ 

mL)

Mean total 

cells/100 

images

% Difference

0.5 1 2.6-8.2% 2.0%

0.999

Actual: Sample

Matrix: 3.5

0.5 2 4.4-8.2% 2.3%

1.5 1 3.1-5.4% 2.6% 0.17 to 

2.0 1.2 8.3

1.5 2 3.5-6.5% 1.4%

5.0 1 3.8-5.2% 2.7%

Target: Assay 

Matrix: 3.9

5.0 2 2.8-8.7% 7.3%

10 1 2.1-3.9% 1.6% 0.50 to 

10

0.9 3.3

10 2 2.1-5.6% 6.9%

Precision ≠ Accuracy!
Confidential – For Celgene Internal Use Only

Main Effects Plots

0.5 1.5 5.0 10.0

10.0

7.5

5.0

2.5

0.0

1158185 1362331

MKM MMP

10.0

7.5

5.0

2.5

0.0

1 2 3 4 5 6 7 8 9

Condition

Mean

Instrument

Operator Replicate

Main Effects Plot for Cell Controls: Total cells / ml (x10^6)

Data Means

24

p-value < 0.001

p-value 0.222

p-value 0.500
Confidential – For Celgene Internal Use Only

Summary

Acknowledgements:

C. Wiwi

Analytical Development

Quality Control

25

• Precise and accurate cell counting = necessary, but not truly achievable 

without a cell reference standard

• Instrument and assay controls will help to minimize variability in the cell 

counting method

• Use of DOE/statistics to understand factors that affect the cell counting 

method facilitates development and eventual validation