The Feasibility of Co-location Detection through 

a Deep Learning Fusion of Mobile Sensors

Sheshank Shankar

sheshank.shankar@pathcheck.org

Challenges for Digital Proximity Detection in Pandemics: Privacy, Accuracy, and Impact
Outline

1. Background

2. Data

3. Network Architectures

4. Data Visualizations

5. Inconsistencies in the Data

6. Conclusion + Future Work

The Feasibility of Co-location Detection through a Deep Learning Fusion of Mobile Sensors
Background

● Detecting close contacts digitally, to effectively trace and isolate a possible 

spread

● Co-location detection methods have been proposed through various modalities, 

Bluetooth Low Energy (BLE) being the most widely accepted 

technology

● The received signal strength indicator (RSSI) value of BLE 

chirps is noisy (Leith et. al, 2020 - tinyurl.com/leith-ble)

● We proposed a joint deep learning model of BLE signals with other 

on-device sensors (Shankar et. al, 2020 - tinyurl.com/pcf-colocation)

The Feasibility of Co-location Detection through a Deep Learning Fusion of Mobile Sensors
Data

● Datasets collected by 2 separate organizations (NIST, MITRE) using the 

Structured Contact Tracing Protocol 

● Further referred to as TRAIN and TEST

The Feasibility of Co-location Detection through a Deep Learning Fusion of Mobile Sensors

● Mobile sensor logs are converted into a numerical 

time series and concatenated with one-hot encoded

experiment level metadata
Network Architectures

● Long short-term memory (LSTM) Network

● Gated Recurrent Unit (GRU)

● Convolutional Gated Recurrent Unit (ConvGRU)

● Temporal 1D Convolutional Network (Conv1D)

The Feasibility of Co-location Detection through a Deep Learning Fusion of Mobile Sensors

Conv1D Network Architecture

Inspired by DeepMind’s WaveNet for Raw Audio ConvGRU Network Architecture
Data Visualizations

TRAIN

The Feasibility of Co-location Detection through a Deep Learning Fusion of Mobile Sensors

TEST
Inconsistencies in the Data

The Feasibility of Co-location Detection through a Deep Learning Fusion of Mobile Sensors

● Switching TRAIN and TEST datasets

● Training on an optimal NN subset of the TRAIN dataset

● Pairwise euclidean distance between feature vectors

○ Average between NN inter-bucket vectors: 200

○ Average between NN cross-bucket vectors: 24
Conclusion + Future Work

The Feasibility of Co-location Detection through a Deep Learning Fusion of Mobile Sensors

● Utilizing only BLE and mobile sensors data can detect proximity, but not at a high 

granularity (as needed for exposure risk applications)

○ Lack of generalizability within the data

● Dataset collection across different scenarios

● Extensive breakdown of mobile sensors contribution to prediction

● Physics-based forward model

● Integrating with other co-location technologies (Dmitrienko et. al, 2020 -

tinyurl.com/pcf-wifi)
References

The Feasibility of Co-location Detection through a Deep Learning Fusion of Mobile Sensors

1. Sheshank Shankar, Rishank Kanaparti, Ayush Chopra, Rohan Sukumaran, Parth 

Patwa, Sunny Kang, Abhishek Singh, Kevin McPherson, Ramesh Raskar. Proximity 

Sensing: Modeling and Understanding Noisy RSSI-BLE Signals and Other Mobile 

Sensor Data for Digital Contact Tracing, 2020.

2. Douglas J. Leith and Stephen Farrell. Coronavirus contact tracing: Evaluating the 

potential of using bluetooth received signal strength for proximity detection, 

2020.

3. Mikhail Dmitrienko, Abhishek Singh, Patrick Erichsen, and Ramesh Raskar. 

WiSense: WiFi Proximity Detection for Digital Contact Tracing, 2020.
PathCheck Foundation, an MIT spin-off

● World’s largest open-source non-profit project for COVID-19

● Privacy first solutions for the pandemic and restarting economy

● Official contact tracing (G/A EN) mobile app for 5 US states, 3 countries

● Awards for Pandemic Response Work

○ Robert Wood Johnson Foundation Innovation Challenge

○ Facebook COVID-19 Symptom Data Challenge

○ NIST TC4TL Challenge

● Join the movement! Volunteers can join our slack: http://tiny.cc/pathcheckslack

The Feasibility of Co-location Detection through a Deep Learning Fusion of Mobile Sensors