2D Barcode Pilot Lessons Learned 

& Early Findings 

Marshall Gaddis 

Deloitte Consulting 

44 

"Reviewed November 2014"
 

 

Outline 

Lessons Learned 

• Vaccination Data 

• User Experience 

Analysis Methodology 

Early Pilot Findings 

• Vaccination Data Overview 

• Pilot Supply Chain 

• Metrics 

• Next Steps 
 

 

C3699AA – HOSP

(P)C3699AA 

1.PEDIARIX-VFC 

2.DTAP/IPV/HEP B 

3.VFC: DTAP-POLIOHEP B 

... 

101.DTAP-HEP 8-IPV 

PEDIARIX 

EMR IIS 

1.Connected 

2.Transfer 

3.Double Entry 

4. IIS Only 

Lessons Learned: Vaccination Data
 

1. Data cleansing 

• Intentionally added information 

• Reduction in data quality 

2. Consistency between EMR 

and IIS data 

• Reporting requirements 

• Non-identical datasets 

3. Name standardization 

• As written at administration 

• Any field with free-form entry 
Inventory Administration Inventory Administration

Only small percent 2D vaccines 

Difficulties scanning 

Inconsistent supply 2D vaccines 

Different process 

Only small percent 2D vaccines 

Inconsistent supply 2D vaccines 

Having to open box to scan 

Different process other vaccines 

Difficulties scanning 

Minor 

Moderate 

Major 

0 50 100 150
 

Respondents (#)
 

Lessons Learned: User Experience
 

1. Common Challenges 

• Within administration and inventory 

Preliminary results
 
 

 

 

 

 

Administration: 2D Scanning Frequency 

50
 (N=152) 

Respondents (%) 20
 

10
 

0
 

Less than 1 to 3 4 to 6 7 to 9 10 or more
 

1
 Frequency of scanning per week 

40
 

30
 

Lessons Learned: User Experience (cont.) 

2. Incorporating Scanners with 

EMRs 

• Configuration and capability 

• Inventory versus Administration 

3. Pilot Supply Chain
 

• Infrequent 2D product
 

• Decreases likelihood to scan
 

Preliminary results
 
Analysis Methodology
 
 

 to 

 

 

 

 

Early Pilot Findings: Vaccination Data Overview
 

Data Volume 

• Nearly one million records 

analyzed 

• Robust estimates 

• More 2D records 

per site in EMR 

EMR IIS 

Baseline Learning Baseline Learning 

Records 297,000 718,000 344,000 960,000 

2D Records 146 32,000 58 26,000 

Providers 160 118 204 204 

Note: Figures rounded. After exclusions: missing vaccination date, invalid vaccine type (e.g., PPD test), withdrawn site 

Site distribution 

• Subset of sites provided all 

phases 

• Before-and-after comparison 

• Use for analyses 

EMR IIS 

7* 

sites 

84 

sites 

112* 

sites 

*Note: Numbers above represent count of sites with both Baseline and Learning data for either EMR or IIS. 

IIS data came from different sets of 204 sites in Baseline vs. Learning. 

Preliminary results 
 

 

 

Early Pilot Findings: Pilot Supply Chain 

• Percent of each vaccine that has converted to 2D barcodes in EMR data 

Note: Data shown are from EMR vaccinations Preliminary results 
 

 

 

 

 

 

100% 

90% 

80% 

70% 

60% 

50% 

288,000667,000265,000648,000

Complete Accurate 

Base Learn 

Number of records 

written vertically 

 

100% 

90% 

80% 

70% 

60% 

50% 

289,000816,000 192,000674,000 

Complete Accurate 

Base Learn 

Number of records 

written vertically 

Early Pilot Findings: Metrics 

Data elements 

• Lot Number, Expiration Date, Product Code 

Metrics 

• Completeness and Accuracy 

Upward trend before looking at potential confounders 

• Isolate effect of technology from other causes 

• Impact of percentage increase on one million records 

EMR Lot Metrics 

(where lot number is scanned) 

IIS Lot Metrics 

(where lot number is scanned) 

Preliminary results 
 

 

 

 

Early Pilot Findings: Next Steps 

Maturity Data Collection 

• All sites finish by end of April 

Analysis 

• Calculate effect of technology after removing potential confounders (e.g., Site 

variation, observation bias, seasonality) 

• Examine across demographic variables 

• Statistical tests and adjustments (e.g., Bonferroni) 

Baseline Data Learning Data Maturity Data

Awardee 

Recruiting 

Immunizer Recruiting Immunizer 

Installation 

Installation 

Scheduling 

Pilot 

WFA 1 WFA 2 

Sep 2011 May 2013 

Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May 
Questions?