# Presentation: 222867

## Implementation of Probabilistic Matching in NYC Chronic Hepatitis B and NYC A1C Registries, and Implications Towards an MPI 

- Maushumi Mavinkurve
- Director, Center for Data Matching
- NYC Department of Health and Mental Hygiene
- October 17th, 2008
- Integrated Surveillance Seminar

## Overview

- Describe data quality challenges in disease surveillance
- Describe probabilistic matching techniques
- Implementation of probabilistic matching
- NYC Chronic Hepatitis B Registry (LVR)
- NYC Hemoglobin A1C Registry (NYCAR)
- NYC proposed challenges and benefits of an MPI

## Public Health Surveillance 

- Public health surveillance process includes:
**Collection** of Data on a specific disease or condition via standardized information systems
- _Analysis_ and interpretation the data
- _Dissemination_ of information to individuals who can act on it
- _Utilization_ of information to facilitate necessary response that will effectively deal with the public health issue

## Surveillance Data Quality Issues

- Accuracy
- Non-standardized across different data sources
- Multiple laboratory systems
- De-duplication of reports
- Exact duplicates
- Multiple events linked to a unique person
- Non-relevant information

**Notes:**
Accuracy refers to the difference between an estimate of a parameter and its true value.
We characterize the difference in terms of systematic (bias) and random (variance) errors.
	Completeness
	Integrity
Timeliness refers to the length of time between the reference period of the information
and when we deliver the data product to our customers.
Relevance refers to the degree to which our data products provide information that meets
our customers’ needs.
Accessibility refers to the ease with which customers can identify, obtain, and use the
information in our data products.
Interpretability refers to the availability of documentation to aid customers in
understanding and using our data products. This documentation typically includes: the
underlying concepts; definitions; the methods used to collect, process, and analyze the
data; and the limitations imposed by the methods used.
Transparency refers to providing documentation about the assumptions, methods, and
limitations of a data product to allow qualified third parties to reproduce the information,
unless prevented by confidentiality or other legal constraints.

## Impact of Data Quality Issues in Surveillance

- Impacts on surveillance reporting
- Over or underestimates of true cases
- Geographical misrepresentation (missing address)
- Increases costs
- Additional staff required to address data quality issues
- Increases inefficiencies
- Timeliness for patient or provider follow up

## Addressing Data Quality Challenges

- Modern disease surveillance information systems:
- Validates data at time of collection
- Minimize inaccurate or incomplete data
- Standardizes different data to uniform structure
- Integrates matching technology to create
- Patient indexes (person-centric systems vs event-centric systems)
- Providers indexes
- Facility indexes

**Notes:**
Could refer to these as each registry as a system that will ultimately feed from a larger MPI.

## What is Probabilistic Matching?

- Rule based match algorithms
- Standardizes Data
- Parses data into smaller tokens
- Create fields that enhance matching
- Adapt to specific data - incorporates uniqueness or frequency of data values when comparing records
- Processes data in blocks – viable to use on large volume data sets

**Notes:**
Rule based match algorithms
	employing fields that uniquely identify an entity – name, dob, gender, telephone, etc.
Standardizes Data
Formalizes names: Mike  Michael
Parses data into smaller tokens
Addressline1  house number, street name, street type, apt #
Create fields that enhance matching 
Phonetic coding: Soundex, NYSIIS
Hash and packed keys
Adapt to specific data - incorporates uniqueness or frequency of data values when comparing records
“Mary Jones” vs “Maushumi Mavinkurve”
Processes data in blocks – viable to use on large volume data sets 

## Evaluating Match Algorithm

- Outcome of a potential match is a weight or likelihood that 2 records are the same entity
- Surveillance programs identify thresholds for match algorithm
- Prior to reviewing results of match algorithm:
- Identify implications for precision (PPV) vs negative predictive valuen (NPV)
- Evaluation of health code mandate
- Practical issues
- Surveillance reporting
- Identify guidelines or criteria to review matches

## Identifying Thresholds

- Goal: maximize precision or PPV
- Sacrifice on negative predictive value (NPV)
- Surveillance programs can decide to review ambiguous matches
- 		Therefore - set high thresholds

## Outcome of Probabilistic Matching

- Entity-centric, relational registry system

## Background of Hepatitis B in NYC

- Decline in acute Hepatitis B incidents case rates  (per 100,000 persons) from 11.5 in 1985 to 1.6  in 2006
- In NYC burden of chronic Hepatitis B infection as much as 2x higher within specific populations
- MSM
- IDU
- Persons born in regions where HBsAg prevalence _>_2%
- Need for continued surveillance and monitoring

- Source: recommendations for identification and public health management of persons with chronic Hepatitis B infection http://www.cdc.gov/mmwr/preview/mmwrhtml/rr5708a1.htm

**Notes:**
Hepatitis B Surface antigen test was developed and FDA approved in 1980’s

Decline in acute Hepatitis B incidents rates from 11.5 cases per 100,000 persons in 1985 to 1.6  in 2006
In NYC burden of chronic Hepatitis B infection up to 2x higher within specific populations 
MSM
IDU
Persons born in regions where HBsAg prevelance >2% (Asian/PI, Eastern Europe, Middle East, Africa, Pacific Island immigrants)
Need for continued surveillance and monitoring

Decline in acute Hepatitis B incidents rates from 11.5 cases per 100,000 persons in 1985 to 1.6  in 2006
In NYC burden of chronic Hepatitis B infection up to 2x higher within specific populations 
MSM
IDU
Persons born in regions where HBsAg prevelance >2% (Asian/PI, Eastern Europe, Middle East, Africa, Pacific Island immigrants)
Need for continued surveillance and monitoring

## Hepatitis B Surveillance Activities

- Monitor disease trends
- Aggregate descriptive reporting aimed to guide prevention and intervention efforts
- Outreach with newly infected
- Educational materials to new cases reported to the registry

## NYC Hepatitis B Registry

- Legacy application, built in-house in 1999
- Automatic weekly batch uploads of laboratory reports
- Data entry of provider reports
- System did not index on patients (event-based), could not link 2 reports for the same person.
- Program utilized staff to build and apply deterministic match algorithms
- Resource intensive
- Version control

## NYC Liver Virus Registry (LVR)

- Implemented in October 2008, built in-house
- Migrated all legacy data
- Web-based application
- Person-centric - integrates probabilistic matching
- Consolidated views of all information for a person
- Ability to conduct longitudinal analysis

**Notes:**
2 weeks ago!

Implemented in October 2008, built in-house
Migrated all legacy data – almost 10 years worth of data
Web-based application
Person-centric - integrates probabilistic matching 
Consolidated views of all information for a person
Ability to conduct longitudinal analysis 

## LVR Probabilistic Matching

- Created a match algorithm based on fields unique to patient from laboratory and provider reports
- Processed all legacy data ~380,000 records
- Program evaluated algorithm and identified thresholds
- Results: out of ~380,000 reports the match algorithm was able to link these to ~111,000 unique persons
- Probabilistic matching enhanced duplication by 1% as compared to legacy deterministic algorithm

## LVR Challenges & Successes

- Challenges:
- Iterative review process time and resource intensive
- Evaluation against legacy deterministic match
- Identifying target PPV and NPV
- Successes:
- Long term savings on time and resources
- Streamlined system
- Longitudinal analysis
- More accurate case counting
- Enhanced data quality

**Notes:**
Challenges:
Iterative review process time and resource intensive
Evaluation against legacy deterministic match – NOT GOLD STANDARD, did not evaluate the legacy match
Identifying target PPV and NPV
Successes:
Long term savings on time and resources 

## Implementing Probabilistic Matching with NYC Hemoglobin A1C Registry (NYCAR) 

## What is Diabetes?  

- Diabetes is a chronic disease caused by inadequate insulin levels or sensitivity leading to elevated blood sugar levels
- Blood sugar levels can be measured by
- Plasma glucose
- Fingerstick glucose
- Glycosylated hemoglobin or A1C (goal is <7%)
- Persistently high blood sugar levels can cause
- Heart disease and stroke
- Kidney failure
- Blindness
- Nerve damage and amputation

## Diabetes Burden in NYC

- Diabetes is epidemic in NYC
- Prevalence has more than doubled over the past 10 years.
- Approximately 500,000 New Yorkers have diabetes
- An additional ~200,000 New Yorkers have diabetes, but have not yet been diagnosed
- Approximately 1 in 8 adults have diabetes
- In 2006, diabetes was the 4th leading cause of death in NYC

## Prevalence of Self-Reported Diabetes Among Adults in NYC

**Source: NYC estimates— CDC Behavioral Risk Factor Survey System (BRFSS) 1994-2001, NYC Community Health Survey 2002-2006 (http://**_[www.nyc.gov/health/epiquery](http://www.nyc.gov/health/epiquery)_)
**Source: National estimates—BRFSS 2006**

## Use of Traditional Public Health Surveillance for Chronic Disease

- Disease reporting to public health agency to**:**
- Monitor trends
- 		 Describe glycemic control in NYC
- Identify special populations
- 		 Target individuals with poor control
- Communicate with provider community
-  Feedback to providers and their patients
- Control epidemics
** **Decrease complications/improve  quality of life

## Hemoglobin A1C Tests

- A1C is a measure of average blood sugar control in preceding 3 months (goal <7%)
- A1C is used to:
- Monitor individual’s blood sugar control
- Guide changes in medication therapy
- Impart risk of diabetes complications
- Most people who get A1Cs have diabetes so it is a marker for diabetes status
**THEREFORE, AN A1C REGISTRY WILL PROVIDE A MECHANISM FOR TRACKING INDIVIDUALS WITH DIABETES**

**Notes:**
Goal – to have 7.0% (average blood sugar of 170 mg/dL).

    THEREFORE, AN A1C REGISTRY WILL PROVIDE A MECHANISM FOR TRACKING INDIVIDUALS WITH DIABETES

## Implementation of NYCAR

- Based on existing NY State / NYC laboratory reporting system
- Amendment to NYC health code, Article 13 which mandates communicable disease reporting, to include A1C
- Public hearing Summer 2005
- Approval of amendment December 2005
- Went into effect January 15, 2006
- Laboratories submitting data to NY State and NYC subject to mandate
- Report information on patient, ordering provider and facility, testing facility and result
- Submit via secure network
- Receive ~5,000 new lab reports daily – High Volume

**Notes:**
Patient advocacy and privacy groups voiced concerns during amendment proceedings
Felt no satisfactory rationale for public health agency involvement in chronic disease reporting

DOHMH clearly wrote into amendment that information can only be released to:
Treating medical provider (s)
Patient

Patients can opt out of intervention but not from being in the registry

Laboratory Reporting:

34 labs reporting A1C tests 

Test results reported by lab within 24 hours

PHINMS – Secure file transmission

HL7 messages or ASCII files

## Objectives of New York City A1C Registry (NYCAR)	

- Surveillance and epidemiology
- Track trends on the population level
- Provider feedback and communication
- Quarterly provider reports in comparison to peers
- Quarterly rosters of patients stratified by A1C level
- Patient feedback (via provider)
- Letters with A1C information
- Local resources
- Deliver resources to providers/patients
- 	All of the above requires matching and data linkages

**Notes:**
Began January 15, 2006 with mandate of electronic lab reporting.

Provider Reports: Quarterly reports with patients listed by A1C level will be distributed to providers. Reports may be used to identify individuals who may benefit from additional support, such as intensification of therapy, or a referral to a physical activity program or self-management program.

Patient Letters: Letters with recent A1C test results and a reminder to return to care will be sent to patients with high A1C levels.

## Components of A1C Registry

- Information collected by laboratory reports include:
- Individual name, address, date of birth, sex
- Name and address of ordering provider, ordering facility and testing facility
- A1C test collection date and result

## NYCAR Probabilistic Methodology

- Created 3 separate matching models:
- Patient
- Provider
- Ordering Facility
- Obtained a representative sample of data
- For each model - created a match algorithm utilizing fields that uniquely identify each entity
- Name (patient, provider, ordering facility), patient dob, gender, address, providerID, telephone number, etc.
- Provided match results to program for review and identify thresholds

**Notes:**
Creates indexes for patient, provider and ordering facility:
	Each patient appears once and all tests for that individual are linked 
	Each provider appears once all tests reported by that provider are linked
	Each ordering facility appears once and all reports by that facility are linked 

Sample used about 100,000 records selected from a specific time period

## Program Threshold Evaluation

- Due to volume of reports, impractical for staff to review all ambiguous matches – need to set thresholds
- Method to identify of thresholds using sample
- 2 reviewers and 1 tie-breaker scored matches referencing guidelines
- Utilized a sampling method within weight ranges
- Identified specific weight or threshold at which target precision rates were met based on review

## Deploying Probabilistic Matching

- All new incoming A1C lab reports parsed into 3 staging entities:
- patient, provider and facilities
- Each entity is matched against existing respective entities in the registry
- If matched above thresholds, linked to an existing record
- If below thresholds, creating a new entity (patient, provider or facility)
- Provider Reports and Rosters and Patient Letters are generated using an in-house developed application which reads from the registry

**Notes:**
On a weekly basis – the following process occurs

## Facility Report

**Note: All information in this slide is fictitious**

**Page 1**

**Page 2**

## Provider Report

**Note: All information in this slide is fictitious**

## Patient Letter

**Notes:**
GET NEW VERSION SCANNED IN 

## Challenges and Successes

- Challenges
- Quality of record linkage
- Need sufficient information for successful linkage of multiple tests per individual as well as master provider and facility indexing
- Maintaining accurate facility-provider linkage
- Effect of laboratory variation – availability of data
- Review thresholds – time and resource intensive
- Successes
- Entire process is seamless, electronic and automated
- High volume of data
- Ability to conduct Longitudinal analysis

**Notes:**
Quality of record linkage
Misconceptions – bad quality data cannot be reconciled!
Need sufficient information for successful linkage of multiple tests per individual as well as master provider and facility indexing
Maintaining accurate facility-provider linkage 
	-1 large umbrella facility can have multiple names
	-providers can work for multiple facilities

Case definitions
Individuals with diabetes
Provider for a given patient

Effect of laboratory variation
Impact of inter-laboratory variation – data integrity and availability

## Is NYC ready for an MPI?

## NYC Current Status

- Modernizing several disease registries:
- Chronic Hepatitis B - completed
- NYCAR – completed
- STD – requirements completed
- TB – requirements completed
- HIV – planning
- Is this an opportune time to develop an MPI?

## Planning an MPI: Challenges

- Each registry program has requirements for a matching based on:
- Patient population
- Data quality and volume
- Dissemination/Use of Surveillance data
- Foster consensus among disease programs
- Breach of Security – higher risk
- Legal barriers to creating an MPI
- Analysis of health code by reportable disease
- Political barriers to creating an MPI

**Notes:**
Challenges:
Each registry program has requirements for a matching based on:
Patient population
Data quality and volume
Dissemination/Use of Surveillance data
Foster consensus among disease programs
Legal barriers to creating an MPI
	laws, particularly in NYC, are extremely specific. Mandated reportables must be used for the purpose of surveillance and epidemiology of 	that specific disease – is an MPI a stretch? 
	Particularly A1C data – this is the most strictly written health code. 

## Planning an MPI: Benefits

- Pooling data from different sources could enhance PPV and NPV of the match
- Streamline IT resources
- Support staff
- Infrastructure
- Ability to conduct syndemic surveillance and investigation
- More efficient use of limited resources

**Notes:**
Syndemic is defined as two or more afflictions, interacting synergistically, contributing to excess burden of disease in a population 

## Acknowledgements

- 	_Diabetes Prevention and Control Program_
- Lynn Silver
- Shadi Chamany
- Angela Merges
- Charlotte Neuhaus
- Bahman Tabei
- Cindy Driver
- Leslie Korenda

- 	_Division of Informatics and Information Technology_
- Don Weiner
- Stephen Giannotti
- Namrata Kumar
- Jisen Ho
- Laura Goodman

- 	_Division of Epidemiology_
- Lorna Thorpe
- Bonnie Kerker
- Jenna Mandel-Ricci
- Ram Koppaka

- 	_Bureau of Chronic Disease Control_
- Katherine Bornschlegl
- Magdalena Berger
- Emily Lumeng

## Questions?

Maushumi Mavinkurve
Director, Center for Data Matching
NYC Department of Health and Mental Hygiene mmavinku@health.nyc.gov
(P) 212 515 5182

- Maushumi Mavinkurve
- Director, Center for Data Matching
- NYC Department of Health and Mental Hygiene _[mmavinku@health.nyc.gov](mailto:mmavinku@health.nyc.gov)_
- (P) 212 515 5182