De-identification of Video Data for Public Sector Research

Presentation at AI World Government

David Kuehn, Federal Highway Administration (FHWA)

June 26, 2019

Key Point: There are many bumps and detours on the road to moving from early stage research to useful applications.  The Federal Highway Administration is in the middle of this journey, taking on research that is new and surprising for an Agency that traditionally has focused on highway engineering research requiring working with new partners and building new skills.

Questions:

1

Passenger Driver Data

Largest Naturalistic Driving Study with over 1 million hours of driver video.

Study data improve highway safety.

Risks involve subject privacy.

June 26, 2019

AI World Government

2

Source: FHWA

Key Point: Largest naturalistic driving study ever undertaken provides immense new areas of research.  Collected to improve highway safety and reliability, there are many possible unanticipated uses given the power of artificial intelligence – computer vision – to analyze the data.

Notes:

Data collected as part of the Strategic Highway Research Program.

3,500+ drivers, all age/gender groups; Most participants in program 1 or 2 years; 5.5 M trip files; 32.5 M vehicle miles.

Over 1 million hours of 15 Hz video image of drivers. Also downward (hands and feet), forward, and right rear view video.

Data collected at six locations. Data maintained in Blacksburg, VA.

Benefits

New way to consider highway safety and reliability.

Traditional studies focused on one behavior that resulted in a specific type of crash, e.g. many drivers on two-lane rural roads with a decreasing curve radius run off the road because they approached the curve too fast.

Now researchers can ask how often people do the same thing and do not crash.

Risks: Study subjects signed restrictions that limit researchers who want to use facial image data to obtain a data use license after approval from an institutional review board, then observe the data at a secure enclave. There are two – one at the Federal Highway Administration’s Turner-Fairbank Highway Research Center in McLean, VA, and one in Blacksburg, VA.

FHWA wants more researchers to have access to the study data, including driver image data.

Photo image from DOE ORNL from work performed under an Interagency Agreement (IAA) with FHWA.

2

What is (De-)identification?

Facial features

Hair styles

Clothing and accessories

Gestures and movements

Location or time

June 26, 2019

AI World Government

3

Source: FHWA

Key Point: Before describing how FHWA moved forward with de-identifying naturalistic driving study subjects, I want to digress and spend a moment on what does it mean to identify or recognize a person.  Have you every run into someone you met professionally on a weekend at the grocery store or at the pool and could not recall who they were?

Notes:

Identification is contextual.  People use clues such as clothing, jewelry, and hairstyles.  Gestures and movements.  And location and temporal context.  If you often see someone on the train or at the gym, you may recognize them in that context but not somewhere else.

People have an innate capacity for seeing faces.  Recognizing faces is not the same as remembering other patterns or images.

Where have people studied identification? 

Mug shots. The government may ask people who witness crimes to identify a suspect and conduct research on processes that produce accurate and reliable results.

Surveillance: The government may review surveillance images to identify and track criminal associates.

More recently government has used video images – public and private – to identify or track suspects immediately after events such as the Boston Marathon bombing in 2013.

Photo image from DOE ORNL from work performed under an IAA with FHWA. 

3

Approaches to De-identification

June 26, 2019

AI World Government

4

Face action transfer

Avatar

Edge masking

Combination

Source: FHWA

Key Point: The concept of releasing subject images by altering them was and still is pioneering research.

Notes:

Automated identity masking could allow researchers access to driver facial behavior outside a secure enclave.

Researchers sign a data use license that says they will not try to reidentify subjects.

In 2013, FHWA solicited researchers for approaches to hiding the identity while preserving facial behavior.

FHWA funded two project teams to research approaches in 2014 – one team from Carnegie Mellon University and one from SRI International, which is not part of PARC. FHWA also entered into an agreement with the Department of Energy Oak Ridge National Laboratory to help assess and transition the research results.

Approaches: 

Both the avatar and face transfer approaches start with facial tracking. The avatar approach generates a new image that replaces the head while mimicking the facial orientation, eye, and mouth movements.  

The face transfer takes a new face and stretches it over the tracking points of the original face creating a mix of the two.  

Edge masking uses edge detection and does not require finding the face correctly.

Examples of errors include not finding a face when the head is turning quickly or in profile or finding a false face reflected in the side window.

Low-level facial behaviors could include face orientation, movements of the eyes or mouth, etc.

Higher level behaviors could include emotions, such as surprise or anger or concentration (on driver or some other task).

Photo image from DOE ORNL from work performed under an IAA with FHWA.  On the left is the driver image, in the center an edge mask, and on the right a face swap with hair removed from the image.

4

Testing De-identification

How often does de-identification incorrectly replace the driver’s original face?

Are people still able to re-identify drivers?

Is the de-identification process reversible?

How accurately does de-identification convey behavior?

June 26, 2019

AI World Government

5

Key Point: By 2017, FHWA, working with Oak Ridge National Laboratory and University of Texas Dallas, began to test the performing teams’ proof-of-concept approaches for automated identity masking along with edge masking for images where the avatar or face swap was not effective.

Notes:

What are the chances that a SHRP2 participant would be recognized at random on the street? While a “real world” test is not possible...what can FHWA demonstrate in a laboratory environment to answer these questions?

FHWA seeks to show that masking is reasonable identity protection to allow limited release of data from the face camera for the benefit of the research community working in collaboration with cognitive psychologists at the nexus of human face recognition and computer vision.

Performing comparison using cropped images (to remove the effects of mutable clothes, hair, and decorations) increases identity masking effectiveness.

Transitory studies with days of delay also increase identity masking effectiveness.

5

Building on a Proof-of-Concept

Work completed or underway.

Developing workflows.

Installing and integrating tools.

Improving efficiency.

Upcoming work.

Maintaining tool resiliency.

Creating extensibility.

June 26, 2019

AI World Government

6

Key Point: FHWA funded the performing teams to demonstrate the viability of an approach for facial identity masking. To make use of the naturalistic driving study data, FHWA needs working tools.

Notes:

ORNL assisted FHWA with installing the proof-of-concept approaches in the FHWA Safety Training and Analysis Center.

ORNL developed workflows for processing image data and continues to assist FHWA with tool resiliency as operating systems, data libraries, etc., change over time.

FHWA also wants to be able to improve the identity masking tools, for example, combining them with better face trackers or future tools that irreversibly mask other potentially identifying features, such as hairstyles or clothing.  

6

Moving Forward

Future Research and Development:

Developing new methods for face detection and tracking.

Testing reversibility of facial masking tools.

Understanding re-identification.

Broader benefits:

Assisting with privacy of other Government image data.

Providing a foundation for use in real-time systems.

June 26, 2019

AI World Government

7

Key Point:  FHWA continues with work on de-identification.

Notes:

FHWA anticipates a new set of performer teams developing new approaches to facial tracking starting later in 2019 (in response to a 2018 solicitation).  

FHWA is working with the NSF Big Data Hubs to test the reversibility of the facial masking tools.

FHWA continues to work with Oak Ridge National Laboratory and University of Texas Dallas to understand clues to how people re-identify or recognize a subject in order to more effectively mask identity.

Beyond creating a tool for increasing access to driver image data from the world’s largest naturalistic driving study, FHWA sees broader benefits from development of de-identification tools.

7

More Information

www.fhwa.dot.gov/publications/research/ear/15025/index.cfm

June 26, 2019

AI World Government

8

Source: FHWA

Key Point: If you are interested in more information, please look at the website information located at https://www.fhwa.dot.gov/publications/research/ear/15025/index.cfm. 

Notes:

8

# image1.jpeg

# image2.jpeg

# image3.png

# image4.png

# image5.png

# image6.png