An NLP-Based Approach to

Record Linkage

FCSM 2023

October 24, 2023

Lilian Huang, NORC at the University of Chicago

In collaboration with:

Brandon Sepulvado, NORC

Dean Resnick, NORC

Jennifer Taub, NORC

Brenda Betancourt, NORC
INTRODUCTION : RECORD LINKAGE OVERVIEW 2

The challenges of using text data in record linkage

What is record linkage?

• The science of bringing together records from multiple datasets

• Identifying and connecting records representing the same entity

• Usually individuals
INTRODUCTION : CONVENTIONAL APPROACH 3

Conventional approach to text: string distance

Levenshtein distance

• Minimum number of edits

Jaro-Winkler

• Weighted prefix

teacher

peacher (substitution)

preacher (insertion)

proeacher (insertion)

profeacher (insertion)

profescher (substitution)

professher (substitution)

professoer (substitution)

professor (deletion)

professor
INTRODUCTION : ROOM FOR NLP 4

How can NLP be integrated?

Occupations

• “teacher”, “instructor”, “professor”

Alternative to compiling lists

String similarity Conceptual similarity
NLP : WORD EMBEDDINGS 5

Embeddings: an alternative representation of words

Words as numbers

Another way to conceptualize and capture similarity

• Similar meaning = closer

grape pear

orange apple

book

magazine
NLP : COMPARING EMBEDDINGS 6

Comparing embeddings

Cosine similarity

• Angle between vectors

θ

Vector 1

Vector 2
NLP : PRODUCING EMBEDDINGS 7

Different libraries for embedding production

fastText

• By Facebook

Universal Sentence Encoder

• By Google

• Context-sensitive

Sentence Transformers

• By Hugging Face

• Context-sensitive
NLP : EXAMPLES 8

Examples of the approach

SIMILARITIES “teacher” vs 

“professor”

“ceo” vs 

“founder ceo”

Levenshtein 0.111 0.273

Jaro-Winkler 0.503 0.449

fastText 0.539 0.860

Universal 

Sentence Encoder 0.748 0.733

Sentence 

Transformers 0.619 0.908

Embeddings reflect the high similarity we intuitively expect
EXPERIMENT : DATA SOURCES 9

The experiment: Datasets used

Database on Ideology, Money in Politics, and Elections (DIME)

• 137,633 records from Maine

Federal Election Commission (FEC)

• 52,827 records from Maine

Variables in common

First name

Last name

Middle initial

Zip code

Employer name

Occupation title
EXPERIMENT : THREE METHODS 10

The experiment: Three different methods

Standard

(Jaro-Winkler for all; 

employer excluded)

First name

Last name

Middle initial

Zip code

Occupation title (string)

NLP 1

(NLP for occupation; 

employer excluded)

First name

Last name

Middle initial

Zip code

Occupation title (embed)

NLP 2

(NLP for occupation;

Jaro-Winkler for employer)

First name

Last name

Middle initial

Zip code

Employer name

Occupation title (embed)
EXPERIMENT : PROBABILISTIC RECORD LINKAGE 11

The experiment: Probabilistic record linkage process

NORCLink

• Proprietary record linkage software

Fellegi-Sunter model

• EM algorithm

• Overall match probability created

• Links above a threshold are accepted
EXPERIMENT : RESULTS 12

The experiment: Results

NLP-enhanced models identified more links

• Probability cutoff of 0.9

Precision-recall tradeoff

• Issue of false positives

Number of links

Standard

(Jaro-Winkler for all) 48,944 unique links

NLP 1

(employer excluded) 55,696 unique links

NLP 2

(employer included) 57,535 unique links
DISCUSSION : CONCLUSION 13

Conclusion

Precision and recall must be balanced

• More work required

• Optimal balance depends on use case

NLP may only enhance process under specific conditions
DISCUSSION : FUTURE ENHANCEMENTS 14

Future enhancements

Fine-tuning embeddings

• Using our own datasets

Longer text fields

• e.g. self-identifications

Investigate correlated variables

• Affects implementation of Fellegi-Sunter method
Thank you. Lilian Huang

Statistician

huang-lilian@norc.org
Questions?