26 January 2021

Marc Zissman, PhD

MIT Lincoln Laboratory

MAZ@LL.MIT.EDU

A Brief Tutorial on

Private Automated Exposure Notification for 

COVID-19

PACT Sponsors

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited. This material is based upon work supported by the 

Defense Advanced Research Projects Agency under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or 

recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Defense Advanced 

Research Projects Agency. © 2020 Massachusetts Institute of Technology. Delivered to the U.S. Government with Unlimited Rights, as 

defined in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwithstanding any copyright notice, U.S. Government rights in this work are 

defined by DFARS 252.227-7013 or DFARS 252.227-7014 as detailed above. Use of this work other than as specifically authorized by the 

U.S. Government may violate any copyrights that exist in this work.

PACT is a collaboration led by the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), MIT Internet Policy Research Initiative, Massachusetts General Hospital 

Center for Global Health and MIT Lincoln Laboratory. It includes close collaborators from Boston University, Brown University, Carnegie Mellon University, the MIT Media Lab, the 

Weizmann Institute and a number of public and private research and development centers. The PACT team is a partnership among cryptographers, physicians, privacy experts, 

scientists and engineers. PACT’s mission is to enhance contact tracing in pandemic response by designing exposure detection functions in personal digital communication 

devices that have maximal public health utility while preserving privacy.
Page 2

MAZ 01/26/2021

• Many non-pharmaceutical interventions to COVID spread can and do 

have impact, e.g.

– Mask wearing, social distancing, testing, quarantine, isolation, contact 

tracing, etc.

• Automated exposure notification (AEN) can supplement manual 

contact tracing efforts

– Automatic detection of high-risk exposure events

– Hypotheses: AEN can decrease delay, decrease workload, broaden exposure 

detection

• There is both mod/sim and operational data that show that the system 

works

• Significant opportunities for future technical innovation exist

Bottom Line Up Front
Page 3

MAZ 01/26/2021

• Beginning early in the pandemic, many international teams of:

– Cryptographers

– Engineers and computer scientists

– Physicians and public health professionals

– System analysts, privacy experts and other specialists

proposed approaches to private, automated detection of potential 

exposure to COVID-19-infected individuals using Bluetooth signaling 

on smartphones that could supplement conventional, manual contact 

tracing

• Many of these teams worked in parallel, in collaboration with each 

other, and in collaboration with Apple|Google

• This talk provides an overview of some of that work

Acknowledgment Upfront
Page 4

MAZ 01/26/2021

COVID-19 Infection Progression

Exposure

0 5 days 10 days 15 days 20 days 25 days

Infected

Contact

Symptom Onset

Index

Case

https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-guidance-management-patients.html

The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application

https://wwwnc.cdc.gov/eid/article/26/7/20-0282_article https://www.harvardmagazine.com/2020/05/r-nought

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7054855/ ; https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7121626/

Infectious

Recovery

Recovery

Goal: Find this person quickly, i.e. before they might infect others

• Identify “contacts” that could have infected this person (reverse)

• Identify “contacts” that this person could infect (forward)

Exposure

Incubation

Infectious
Page 5

MAZ 01/26/2021

• Traditional uses (examples):

– Tuberculosis

– Smallpox

– Sexually transmitted diseases

Contact Tracing

Contact tracing is an epidemiological technique used to identify people who 

have had “contact” with an infected person 

Contact tracing can help inform public health interventions to slow virus transmission

Infected

Uninfected

Time

Transmission
Page 6

MAZ 01/26/2021

• Prior to COVID-19, contact tracing was primarily a manual process

• Primarily used for diseases with longer temporal characteristics

Contact Tracing Tools

Public Health Contact Tracing Tools

Challenges

• Index case must remember who they 

were in contact with, where they were

• Index case must know identifying 

information for contacts

• Labor intensive and time consuming

• Risk of data errors

• Difficult to apply analytics

• May not scale to need

Advanced contact tracing tools are urgently needed to handle COVID-19
Page 7

MAZ 01/26/2021

How AEN Works*

*Video courtesy of Robin Marsden (RGM Productions), https://youtu.be/qQYM25bpiLo
Page 8

MAZ 01/26/2021

General Approach

Approx 6 feet

Chirp

Log

Database of

Anonymized 

Contacts

Bluetooth

Chirp 

Records

Infection

Certification

Chirp

Log

Bluetooth

Chirp

Contact 

Distance

Index

Case

Contact Index

Case

Contacts

Manual

Contacts 

Records

Contact 

Detection

1 COVID-19 Test 

Confirmed Positive

2 Exposure Query 

& Notification

3 Contact 

Actions

4

Virtual Command Center

Manual

Notification

Automatic

Notification

Test

Symptom 

check

Self

Quarantine

Contacts

Note: AEN supplements MCT, 

it does not replace it
Page 9

MAZ 01/26/2021

1. Automatic exposure notification (AEN) can lead to faster exposure 

notification vs traditional manual contact tracing (MCT) alone

2. AEN can reach persons who are not personally known to an index case

3. AEN can still work when MCT reaches resource limits or breaks down

4. AEN alerts contacts privately and automatically about potential exposure, 

enabling them to choose how they wish to engage with public health 

authorities

5. Public health can set an operating point such that benefits from AEN 

detection of true exposures outweigh potential costs from AEN false 

positives

The AEN Value Proposition*

(Hypothesized in Spring 2020)

*Adapted from conversations with Viktor von Wyl (University of Zurich) et al.

See also: von Wyl Viktor, et al. A research agenda for digital proximity tracing apps. https://smw.ch/article/doi/smw.2020.20324
Page 10

MAZ 01/26/2021

January 2021 Status:

>30 Nations and U.S. States

Approx 6 feet

Chirp

Log

Database of

Anonymized Contacts

Bluetooth

Chirp 

Records

Infection

Certification

Chirp

Log

Bluetooth

Chirp

Contact 

Distance

Index

Case

Contact Index

Case

Contacts

Manual

Contacts 

Records

Contact 

Detection

1 COVID-19 Test 

Confirmed Positive

2 Exposure Query 

& Notification

3 Contact 

Actions

4

Nation/State Dept of Public Health

Municipal Boards of Health

Contact Tracing Centers

Manual

Notification

Automatic

Notification

Test

Symptom 

check

Self

Quarantine

Contacts

iPhone or

Android

• Apple|Google OS w/BLE 

contact tracing built-in

1

• Roaming is supported

– International database (Europe)

– National key server (US – APHL)

3

• DPH-acquired app or 

A|G OS providing 

tailored instructions to 

contacts

4

• Multi-tenant verification 

server (US – APHL)

2
Page 11

MAZ 01/26/2021

AEN Stack Layers

Layer 1: Proximity Measurement

Layer 2: Private Cryptographic Protocol

Layer 3A: Public Health Interface

Major Challenges

• Bluetooth phenomenology & data collection

• Implementing & evaluating “Too Close for Too Long” analytic

• Android, iOS interoperability

• Operating system policy compliance

• Smartphone power constraints

• OS vs app functional decomposition

• Other signaling options, e.g. ultrasound and UWB

Major Challenges

• Privacy preservation

• Chirp rollover frequency

• Reporting chirps sent vs chirps rec’d

• Mitigating threats posed by malicious parties

Major Challenges

• Integration into manual contact tracing systems

• Certification of infection

• Interoperability across public health authorities

• Specifying “Too Close for Too Long” 

requirements

• Trustworthy systems to earn broad societal trust

Layer 3B: Individual Interface

Major Challenges

• Clear and local culture-appropriate opt-in instructions 

and explanation of privacy guarantees

• Simple functionality for reporting and certifying 

infection

• Simple functionality for notification of possible “too 

close for too long” contact and related instructions

• Integration with other public health functionality not 

directly AEN related
Page 12

MAZ 01/26/2021

User Interface

Mobile Phone App*

• For individual smartphone users

• Turns exposure notification on and off

• Alerts user of potential exposure

• May provide additional functionality, 

e.g.

– PHA updates for users

– Daily user health inputs for PHA

• Simpler “App-less” version also 

available (EN Express for iOS)

*Ireland “COVID Tracker” App built by NearForm
Page 13

MAZ 01/26/2021

“Too Close for Too Long” (TC4TL)

Range

Exposure Time

0m

10min

Hug

30min

1hr

8hrs

1m 2m 3m 4m

1min

Same

Classroom

Subway,

One Stop,

Same Car

Handshake

No Alert

• Public health authorities define alert region

• Engineers try to implement it accurately ALERT! • This approach oversimplifies the problem

Side-by-Side

@ Coffee Shop

Same

Table @School

Notional

Alert Region
Page 14

MAZ 01/26/2021

• Many classes of false alarms

– Class 1: Contact was actually 

further than 6ft from index, or 

was close but for less than 15min

– Class 2: Contact was too close 

for too long, but just didn’t get 

infected

– Class 3: Contact was too close 

for too long, but was 

unexposable (e.g. in an adjacent 

apartment)

Example Assessment Approach:

The 4-Circle Venn Diagram

• I: Individuals who are infected with the virus

• E: Individuals who are exposed as defined by a PHA

• C: Contacts identified via manual contact tracing (MCT)

• A: Contacts identified via automated exposure notification

• X: Union of MCT and AEN, i.e. C U A

I

X

E C

Note: These Venn diagrams don’t capture speed of

either MCT or AEN
Page 15

MAZ 01/26/2021

• >30 nations and US states using A|G ENS/ENX 

operationally

• Example: Switzerland (week ending 11 Jan 2021)

– Data updated publically, daily

– MCT performed ~26 cantons

– AEN managed at national level

– ~2K-5K new cases per day in past week

– 1.8M active AEN apps

(total population = 8.5M people)

– ~200-500 COVID codes entered by users

per day

• ~2500 COVID codes entered in seven days ending 11 January

– ~20-40 calls per day to SwissCovid infoline re AENs 

rec’d (had been 300-400 per day in December)

– Delays between symptom onset and transmission of 

COVID code grow and shrink

– Many contacts who have tested positive were first 

alerted by an AEN, but hard to measure precisely

A|G ENS/ENX Around the World

https://down.dsg.cs.tcd.ie/tact/tek-counts/ downloaded 13 January 2021

https://www.experimental.bfs.admin.ch/expstat/en/home/innovative-methods/swisscovid-app-monitoring.html downloaded 13 January 2021

https://www.google.com/search?client=firefox-b-1-e&q=switzerland+covid+cases+today downloaded 13 January 2021

ENS: Exposure notification service (PHA builds app)

ENX: ENS “Express” (PHA not required to build app)

See also: Salathé Marcel, et al, “Early evidence of effectiveness of digital contact tracing for 

SARS-CoV-2 in Switzerland,” Swiss Medical Weekly, 2020;150:w20457 December 2020.
Page 16

MAZ 01/26/2021

• AEN is faster than manual contact tracing

– AEN-notified contacts with exposure risk outside own household 

entered quarantine one day earlier (on average) vs manual contact 

tracing (MCT)

• AEN finds infected people that MCT would miss

– Analysis of Sept 2020 results in Zurich

– 324 of 1923 (16.8%) PCR positive cases uploaded CovidCodes

– 722 persons called the help line after they received an AEN

– 170 of the 722 were given a quarantine recommendation

– 30 of the contacts receiving AENs tested PCR positive

Recent Results from

Canton of Zurich, Switzerland*

* Viktor von Wyl, “The SwissCovid GAEN app after six months: It’s not just about technology!” NIST workshop (26 January 2021)
Page 17

MAZ 01/26/2021

Assessing TC4TL Systems

(Notional DET Curves)

Probability of False Positive

(Leads to applying effort & treatment unnecessarily to a healthy person)

Probability of False Negative (a.k.a. Miss)

(Leads to withholding effort & treatment from a sick person)

0.01% 0.1% 1% 10% 50% 99%

“Flip a Coin”

0.01% 99% 0.1% 1% 10% 50%

System A

System B

Everybody

Stay Home

Do Nothing

Better System

Performance

*A. Martin, A., G. Doddington, T. Kamm, M. Ordowski, and M. Przybocki. "The DET Curve in Assessment of Detection Task Performance",

Proc. Eurospeech '97, Rhodes, Greece, September 1997, Vol. 4, pp. 1895-1898.

Ideal

Performance

• Want to give public health 

authorities freedom to pick 

from a range of operating 

points

• If we can estimate Pr(pos), 

Pr(neg), Cost(FN), Cost(FP), 

we can find the lowest-cost 

operating point on the DET* 

curve

Decision Error Tradeoff (DET*) Curve

Narrow

Operation

Wide

Operation
Page 18

MAZ 01/26/2021

• Contact receives Bluetooth signals from the index

– Random, binary data (no personal information)

– Distance is estimated by measuring attenuation in the Bluetooth signal between when it 

was transmitted and when it was received; duration is measured too

• Overall risk is a weighted sum of exposure minutes, which is then multiplied by 

case infectiousness (which itself is a function of days since symptom onset)

The Basics of Estimating

Risk Exposure

Approx 6 feet

Chirp

Log

Chirp

Log

Bluetooth

Signals

Contact 

Distance

Index

Case

Contact
Page 19

MAZ 01/26/2021

• For simplicity, most states have adopted one of two sets of 

standard AEN parameters (e.g. thresholds, weights, etc.)

– “Narrower Net” – Prioritizes lower PFA at the expense of higher PMISS

– “Wider Net” – Prioritizes lower PMISS at the expense of higher PFA

• The standard parameter sets were defined at an international 

workshop when a modest amount of real data was available*

• Much more data collected since then permits quantification of 

what the standard parameter sets really do

– Permits more informed public health authority decisions

– Might possibly indicate that the standard parameter sets should be 

adjusted

Standard Parameters Sets

*Linux Foundation Public Health: Risk Score Symposium Invitational (“RSSI”) Workshop, Nov 2020

https://github.com/lfph/gaen-risk-scoring/blob/main/risk-scoring.md
Page 20

MAZ 01/26/2021

Experiment-based AEN Evaluation

• Test A|G proximity alerts

• Examine app data flows

• Baseline P(miss) vs. P(fa)

Simulation Lab-based Evaluation In situ Pilots

• Explore A|G parameter space

• Estimate system impacts

• Explore PH integration modes

• Assess app usability

• Estimate real P(miss) vs. P(fa)

• Test end-to-end PH workflow

Test many hypothetical scenarios Highly controlled experiments Real users, real environments

PH = public health

Agent-based Models MIT LL Test Range In-the-wild
Page 21

MAZ 01/26/2021

• Electro-magnetic phantoms support Bluetooth proximity 

measurements

The Robot Dance

(MIT LL Test Range)
Page 22

MAZ 01/26/2021

• Data collected using human-like 

robots with smartphones on at a test 

range at MIT Lincoln Laboratory

• Variety of distances, poses, 

orientations, body blocking, bags, 

pockets, etc.

• Phone conditions can lead to >20dB 

(>100x) variation in RSSI* at fixed 

distances

Bluetooth Phenomenology

Bodies

Blocking

0

1

2

* RSSI: Received signal strength indicator
Page 23

MAZ 01/26/2021

Narrower vs Wider Net Settings

• Large room 

• Calibrated MITLL Test Range data

• Grid of contacts (1:30’, 1:30 min)

• TC4TL 15 min @ <6’, exposure threshold

• Alert threshold of 1-60 weighted minutes

• WM = weighted minutes

Narrower and wider net settings yield very similar options for operating points

Wider Net, 0 body blocking

Narrower Net, 0 body blocking

Equal error rate

Wider Net, 2 bodies blocking

Narrower Net, 2 bodies blocking

Coin flip

Wider Net, 2 bodies blocking

15 WM alert threshold

Narrower Net, 0 body blocking, 

15 WM alert threshold

PFA (%)

Pmiss (%)

Wider Net, 0 body 

blocking, 15 WM 

out of frame

Narrower Net, 2 body 

blocking, 15 WM out 

of frame
Page 24

MAZ 01/26/2021

1. Automatic exposure notification (AEN) can lead to 

faster exposure notification vs traditional manual 

contact tracing (MCT) alone

2. AEN can reach persons who are not personally 

known to an index case

3. AEN can still work when MCT reaches resource 

limits or breaks down

4. AEN alerts contacts privately and automatically

about potential exposure, enabling them to choose 

how they wish to engage with public health 

authorities

5. Public health can set an operating point such that 

benefits from AEN detection of true exposures 

outweigh potential costs from AEN false positives

The AEN Value Proposition*

(Revisited)

*

 Yes – data confirm



Yes – the A|G protocol 

preserves privacy by 

design



Yes – MCT has broken 

down in some 

jurisdictions**, and yet 

AEN still functions



Yes – the A|G protocol 

provides this 

functionality by design 

and data confirm

** https://www.swissinfo.ch/eng/top-epidemiologist-urges-greater-uptake-of-contact-tracing-app/46108064

** https://www.grandforksherald.com/newsmd/coronavirus/6726337-Overwhelmed-by-cases-North-Dakota-tells-residents-with-COVID-19-to-do-their-own-contact-tracing

Adapted from conversations with Viktor von Wyl (University of Zurich) et al.

See also: von Wyl Viktor, et al. A research agenda for digital proximity tracing apps. https://smw.ch/article/doi/smw.2020.20324



Yes – 20+M users, no 

significant issues; Low 

PFA ops points can be 

set as desired
Page 25

MAZ 01/26/2021

• Adoption

– By public health authorities

– By the public within these jurisdictions

• Speed of distribution of verification codes

• Willingness to use the verification codes and actually upload 

keys

• Willingness to follow public health instruction upon receiving 

an AEN

• Assessments

– Measures of performance

– Measures of effectiveness

• And the “big one”: The trade-off between privacy and public 

health effectiveness... did we all get it right?

Challenges (An Incomplete List!)
Page 26

MAZ 01/26/2021

Example R&D Focus Areas

Parameters

Code and Design Analysis Bluetooth, Ultrasound, UWB

Optimization & Tuning

User Adoption / Messaging

Workflow Analysis M&S Risk Assessment

M & S = modeling and simulation

UWB = Ultra-Wideband
Page 27

MAZ 01/26/2021

NIST TC4TL Challenge

September 2020

• Explored promising new ideas in TC4TL detection using BLE signals

• Supported the development of advanced technologies incorporating 

these ideas

• Measured and calibrated the performance of the state-of-the-art 

TC4TL detectors

https://www.nist.gov/itl/iad/mig/nist-tc4tl-challenge

https://tc4tlchallenge.nist.gov/
Page 28

MAZ 01/26/2021

• Many non-pharmaceutical interventions to COVID spread can and do 

have impact, e.g.

– Mask wearing, social distancing, testing, quarantine, isolation, contact 

tracing, etc.

• Automated exposure notification (AEN) can supplement manual 

contact tracing efforts

– Automatic detection of high-risk exposure events

– Hypotheses: AEN can decrease delay, decrease workload, broaden exposure 

detection

• There is both mod/sim and operational data that show that the system 

works

• Significant opportunities for future technical innovation exist

Summary
Page 29

MAZ 01/26/2021

• PACT Leadership*

– Prof Ron Rivest

– Danny Weitzner

– Dr Louise Ivers, MD MPH

– Israel Soibelman

– Ed Wack

– Adam Norige

– Prof Yael Kalai

– Bill Streilein

– Emily Shen

– Gary Hatke

– John Wilkinson

– Doug Reynolds

– Paula Ward

– Jenn Watson

– Chris Roeser

– Larry Candell

– Mark Smith

– Charlie Ishikawa

– Tim Dasey

– Marc Zissman

Acknowledgments

* A partial list of the PACT team members working at MIT and MGH on automatic exposure notification.

Many other such teams have been working on AEN worldwide since the beginning of the pandemic.

• PACT Tech Leads* 

– Bobby Pelletier

– Jess Alekseyev

– Dieter Schuldt

– Ted Londner

– Emily Kosten

– Gwen Gettliffe

– Elliot Singer

– Cassian Corey

– Curran Schiefelbein

– Richard Gervin

– Joe St. Germain

– Steve Mazzola

– Eyassu Shimelis

– Stacy Zeder

– Mary Ellen Zurko

– John Meklenburg

– Mike Specter

– Michael Wentz

– Marc Viera

• Our sponsors and colleagues at IBM 

Research, DARPA and CDC

• 100+ other staff at MIT Lincoln 

Laboratory, CSAIL and IPRI who have 

contributed time to PACT since March

• Our colleagues at Apple and Google

• Our colleagues in Ireland, UK, 

Switzerland, Singapore, Finland

• Our colleagues at NIST

• Our colleagues in the academic and 

public health communities of Arizona, 

Pennsylvania and Massachusetts