Jan. 26 – 28, 2021

Workshop on Challenges for Digital 

Proximity Detection in Pandemics: 

Privacy, Accuracy, and Impact

https://www.nist.gov/news-events/events/2021/01/challenges-digitalproximity-detection-pandemics-privacy-accuracy-and

Tuesday, January 26, 2021 All times are Eastern Standard Time (EST)

10:00 am Welcome and Opening Remarks

10:10 am Who should you be talking to? 3 lessons in interdisciplinary problem-solving

Joanna Masel, Professor, Department of Ecology & Evolutionary Biology, University of 

Arizona

A critical look at exposure notification through the lens of its end-goal – fighting the 

pandemic – reveals a variety of pitfalls in interdisciplinary problem-solving. For example, 

assuming that problems in disciplines other than your own are already solved can lead 

to overly narrow design specifications, such as classification with respect to 6 foot 15 

minute thresholds rather than estimation of infection risk. Strategies for extracting 

information from other disciplines, and indeed for discovering the existence of relevant 

disciplines in the first place, will be discussed.

10:55 am A Brief Tutorial on Private Automated Exposure Notification for COVID-19 

Marc Zissman, Associate Head, Cyber Security and Information Sciences Division, Lincoln 

Laboratory, Massachusetts Institute of Technology

Beginning early in the pandemic, international teams of cryptographers, electrical 

engineers, physicians, computer scientists, public health professionals, privacy experts 

and other specialists proposed approaches to automate the process of detecting 

potential exposure to COVID-19-infected individuals using Bluetooth signaling on 

smartphones that could supplement conventional, manual contact tracing (MCT). The 

value proposition had four components: automated exposure notification could lead to 

faster alerting and action vs MCT alone, could reach persons who are not personally 

known to an index case, could continue to function even when MCT reaches resource 

limits or breaks down, and could provide alerts while protecting privacy. Influenced to 

some extent by the work of these international teams, Apple|Google developed an 

automated exposure notification system based in part on the protocols and prototypes 

developed and demonstrated by these international teams. A|G deployed that system 

quickly and widely, and the system is now in use in over 30 nations and US states, has 

been enabled via user opt-in on tens of millions of smartphones, and has provided 

hundreds of thousands of exposure notifications to close contacts. To varying extents, 

all four elements of the value proposition have been proven – the system works. In this 

talk, the overall approach to private, automated exposure notification is explained, with 

emphasis on the operation and performance of all three layers: Bluetooth-based 

proximity measurement, the private cryptographic protocol, and the public health / 
Page 2

individual user interfaces. Preliminary estimates of both performance and effectiveness 

from deployments around the world are discussed.

11:25 am Transition

11:28 am Real-world effectiveness of digital contact tracing

Luca Ferretti, Senior Researcher in Statistical Genetics and Pathogen Dynamics at the 

Big Data Institute, University of Oxford

After many countries deployed apps for proximity detection and exposure notifications, 

there is still a lack of understanding of their potential and actual impact. In this talk we 

review the theoretical expectations about the potential impact of app-based contact 

tracing, and we discuss the main known and unknown factors that affect its 

effectiveness. Then, we discuss the role of these factors and the actual impact of such 

apps in some European countries.

11:50 pm Break

12:20 pm Transition/introduction of speakers

12:25 pm The SwissCovid GAEN app after six months: It’s not just about technology.

Viktor von Wyl, Assistant Professor, Epidemiology, Biostatistics and Prevention Institute, 

University of Zurich

The SwissCovid app was released in Switzerland as one of the first GAEN apps on June 

25, 2020. This presentation will briefly recap the challenges for measuring effectiveness 

in a highly decentralized health system, the state of evidence for effectiveness, as well 

as lessons learned. Ultimately, the success of GAEN apps not only depends on 

technological aspects but also on appropriate health system embedding and engagement 

by different actors.

12:47 pm Public acceptance of emerging technologies

Alta Charo, Knowles Professor Emerita of Law & Bioethics, University of Wisconsin 

Madison

This presentation will describe some experiences in the past with public reaction to new 

technologies, and identify measures that can be taken to increase enthusiasm, decrease

concern, and substantively address risks.

1:09 pm Public health perspective of digital contact tracing in COVID-19 using Bluetoothenabled technology

Meghna Patel, Deputy Secretary for Health Resources and Services, Pennsylvania 

Department of Health

The spread of COVID-19 in the communities is so overwhelming across the globe that the 

ability to track, trace, isolate and test the individuals suspected to have the virus is 

impossible without the influx of additional staff and use of technology assisted

applications. Traditional contact tracing is unable to account for contact with individuals 
Page 3

unknown to the person who tested positive (e.g., an individual who tested positive 

cannot name others he/she rode the bus with). Bluetooth-enabled technology is a 

promising solution to assist with exposure notification where traditional contact tracing 

cannot. It is important that guiding principles of data privacy remains at the core when 

implementing a technology- assisted exposure notification. These applications will be 

effective if priority for testing is established or mass testing is available or both as well 

as massive campaign focusing on trust. As the future of public health becomes more led 

by technology to help stop the spread of infectious diseases, there are several limitations 

to implementing and sustaining digital contact tracing technology solutions.

1:31 pm Break

1:40 pm Transition/introduction of speakers

1:45 pm Privacy considerations, an overview

Naomi Lefkovitz, Senior Privacy Policy Advisor, Information Technology Lab, NIST

Management of privacy risks, whether real or perceived, is a key factor in successful 

adoption of large-scale pandemic technology solutions. This presentation provides an 

overview of some of the privacy considerations for proximity detection technologies, 

including considerations for assessing tradeoffs with effectiveness, usability, security, 

and different implementation environments.

2:07 pm Privacy preserving protocols for encounter metrics

Rene Peralta, Computer Scientist, Cryptographic Technology Group, NIST

We propose measuring aggregate levels of encounters in a population, a 

concept we call “encounter metrics”. Our proposal is to design encounter

metrics in such a way that it can be deployed while preserving the privacy of

individuals. To this end, our proposal is to label encounters with a random 

number (an encounter ID) that cannot be linked to anything that is broadcast 

at the time of the encounter. This mitigates significant privacy concerns inherent 

to methods that broadcast and track user pseudonyms instead of encounter IDs.

Encounter metrics can be used to analyze the interactions within populations 

as we attempt to safely restart our societies during the pandemic of 2020. The 

aggregate encounter metric statistics will facilitate analysis of population interactions 

in buildings, comparisons across buildings or campuses, and data-driven adjustments 

in reopening processes. These measurements will also be valuable in designing our

future working environments to be more resilient to spread of infectious diseases.

2:19 pm Privacy by Design as Infrastructural Power

Seda Gürses, Associate Professor, Department of Multi-Actor Systems at TU Delft

During this talk, I will give an overview of the series of events that led to and followed 

from the development of the Decentralized Privacy Preserving Proximity Tracing (DP3T) 

protocol that underlies the Google Apple Exposure Notification (GAEN) API. Underlying 

engineering efforts in privacy by design is the assumption that data control and data 
Page 4

minimization will enable the protection of individuals and therewith mitigate the growth 

of power asymmetries. The GAEN story, however, suggests that data minimization, a 

guiding design principle for engineers and privacy advocates, may not always come to 

tame power. On the contrary, the infrastructural advantage that companies like Google 

and Apple possess allows them to leverage privacy enhancing protocols to expand their 

reach. This has serious repercussions both for privacy by design as well as the democratic 

governance of technologies for public use, matters pertinent to reflect on given the 

increasing number of applications for population management using digital technologies.

2:41 pm Transition/introduction of speakers

2:45 pm Flipping the Perspective on Contact Tracing

Po-Shen Loh, Founder of NOVID, Professor of Mathematical Sciences at Carnegie Mellon 

University, and National Coach of USA International Math Olympiad team

We introduce a fundamentally different paradigm for contact tracing, enabled by the 

proliferation of smartphones: for each positive case, do not only ask direct contacts to 

quarantine; instead, tell everyone how many relationships away the disease just struck 

(so, "2" is a close physical contact of a close physical contact). This new approach, which 

has uniquely been deployed in the publicly downloadable app NOVID, brings a new tool 

to bear on pandemic control, powered by network theory. Like a weather satellite 

providing early warning of incoming hurricanes, it empowers individuals to see 

transmission approaching from far away, and becomes the first proximity detection app 

whose installation reduces the user's own chance of infection.

This flipped perspective incites natural self-interested instincts of self-preservation, 

reducing reliance on altruism, and the resulting caution reduces pandemic spread in the 

social vicinity of each infection. Consequently, our new system solves the behavior 

coordination problem which has hampered many other app-based interventions to date. 

Indeed, from the game-theoretic perspective of Nash Equilibria, standard apps 

unfortunately only reach equilibrium when nobody installs, whereas our approach is also 

at equilibrium when everybody installs. In addition to this, our approach has 3 order-ofmagnitude power gains over the status quo.

3:07 pm Machine learning based digital proximity detection: lessons learned from the NIST 

TC4TL Challenge and beyond

Omid Sadjadi, Computer Scientist, Information Technology Lab, NIST

The NIST pilot Too-Close for Too-Long (TC4TL) Challenge, which was conducted in the 

summer of 2020 in response to the COVID-19 pandemic, was a machine learning 

challenge to explore promising new ideas in, and evaluate the efficacy of, digital 

proximity detection based on Bluetooth Low Energy (BLE) and other smartphone sensor 

data. This talk will provide an overview of the TC4TL challenge, including descriptions 

of the task, datasets, performance measure, participation statistics, as well as results 

and system performance analyses. Some potential avenues for future work will also be 

presented and discussed.
Page 5

3:30 pm Adjourn

Wednesday, January 27, 2021 All times are Eastern Standard Time (EST)

10:00 am Panel discussion on the effectiveness of digital proximity detection at limiting the 

spread of infectious diseases.

This panel will discuss questions such as How should we define ‘effective’? How do we 

know models showing proximity detection effectiveness are right? How does the 

implementation and effectiveness change for different sizes and types of communities?

Krister Shalm (moderator), Applied Physics Division, NIST and Physics Department, 

University of Colorado, Boulder

Marc Zissman, Associate Head, Cyber Security and Information Sciences Division, Lincoln 

Laboratory, Massachusetts Institute of Technology

Po-Shen Loh, Founder of NOVID, Professor of Mathematical Sciences at Carnegie Mellon 

University, and National Coach of USA International Math Olympiad team

Mike Judd, Lead, COVID-19 Exposure Notification Initiative, US Centers for Disease 

Control and Prevention

Viktor von Wyl, Professor, Epidemiology, Biostatistics and Prevention Institute, 

University of Zurich

Louise Ivers, Professor of Medicine, Harvard Medical School; Interim Chief of Infectious 

Diseases and Executive Director of Center for Global Health – Massachusetts General 

Hospital

11:30 am Instructions for breakout sessions

11:40 am Break

Breakout session topics are tentative and may change based on Jan. 26 presentations 

and attendee feedback (via survey). Finalized workshop topics and links to sessions will 

be e-mailed to participants before 10 am ET on Jan. 27.

At 12:40 there will be a 10-min break. Participants join a different breakout session for 

the final 40 minutes if they want to participate in another discussion.

12:00 pm 1a Applications of 

proximity detection 

What are the different 

ways proximity 

detection can be used to 

limit the spread of 

pandemics (digital 

contact tracing, 

encounter metrics, etc.)? 

1b Privacy 1

How are privacy risks 

generated by 

proximity detection 

technologies 

understood and 

managed?

1c Technologies for 

Proximity Detection

How accurately can 

different 

technologies identify 

a contagious 

interaction? 

1d Role of 

government

What role should 

local, state, and 

federal government 

play in digital contact 

tracing?
Page 6

Which are most 

promising?

Facilitator:

Leah Kauffman

Facilitator:

Jessica Staymates

Facilitator:

Ashley Boggs

Facilitator:

Brandi Tolliver

1:30 pm Break

At 2:40 there will be a 10-min break. Participants join a different breakout session for 

the final 40 minutes if they want to participate in another discussion.

2:00 pm 2a Implementation

What adoption rate is 

needed to be effective 

in different 

communities? What are 

the barriers to adoption 

and use, and how should 

they be addressed? 

2b Privacy 2 

What are the 

greatest challenges 

to achieving strong 

privacy properties 

while delivering 

effective proximity 

detection? How can 

privacy gains or 

losses be understood 

with different 

implementations?

2c Technology 

verification

What is needed to 

verify the 

performance of 

proximity detection 

technologies?

2d Commercialization

What are the barriers 

to commercialization?

Facilitator:

Jeanita Pritchett

Facilitator:

Jessica Staymates

Facilitator:

Callie Higgins

Facilitator:

Jeremy Lawson

Thursday, January 28, 2021 All times are Eastern Standard Time (EST)

10 am Reports from working groups (10 minutes each)

Moderator: Heather Evans, Program Coordination Office, National Institute of 

Standards and Technology

11:30 am Contributed Talks

Moderator: Michelle Stephens, Applied Physics Division, National Institute of Standards 

and Technology

11:30 am Privacy-Protecting COVID-19 Exposure Notification Via Cluster Events 

Without Proximity Detection, Paul Syverson, U.S. Naval Research Laboratory

11:42 am Augmenting GAEN with opt-in case linking, James Petrie, WeHealth

11:54 am Adoption metrics for Proximity Technologies, Scott David, Director of 

Information Risk Research Initiative (IRRI), University of Washington Applied Physics 

Laboratory

12:06 pm Modeling the impact of automatic exposure notification for vulnerable 

communities Krister Shalm, Applied Physics Division, NIST and Physics Department, 

University of Colorado Boulder
Page 7

12:18 pm Understanding and Rewiring Epidemic Networks: A Data-driven Approach 

Towards Enabling Quarantine in-Motion, Radu Marculescu, Department of Electrical 

& Computer Engineering, The University of Texas at Austin

12:30 pm Break

1:00 pm Contributed Talks/Discussion

Moderator: Michelle Stephens, Applied Physics Division, National Institute of Standards 

and Technology

1:00 pm Interoperable Privacy Preserving Digital Contact Tracing, Yang Yaling, 

Virginia Tech

1:12 pm Function Secret Sharing for PSI-CA: With Applications to Private Contact 

Tracing, Steve Lu, Stealth Software Technologies, Inc.

1:24 pm Modelling multipath interference for BLE proximity detection and 

exposure scoring, Ramsey Faragher, CEO Focal Point Positioning, Fellow in Computer 

Science, Queen’s College, University of Cambridge

1:36 pm COSMOS Testbed – Proximity Detection and Social Distancing Estimation in 

COVID-19 Pandemic, Zoran Kostic, Electrical Engineering Dept., Columbia University

1:48 pm A Simplistic Machine Learning Approach to Contact Tracing, Niamh Belton, 

ML-Labs, University College Dublin

2:00 pm The Feasibility of Co-location Detection through a Deep Learning Fusion 

of Mobile Sensors, Sheshank Shankar, Data Science Researcher, PathCheck Foundation

2:12 pm Entropy Based Discretization's and Weight Optimization for Configuring 

the GAEN system, Nicholas Maynard, The MITRE Corporation

2:24 pm Efficacy of Current Approaches and An Alternative Paradigm for Digital 

Contact Tracing, Brian Thompson, MITRE Corporation

2:36 pm Break

2:45 pm Discussion and Summary

Moderator: Heather Evans, Program Coordination Office, National Institute of 

Standards and Technology

3:30 pm Adjourn