BOCA	– Body-Worn	Camera	Analytics	in	Public	Safety

Jason	J.	Corso,	Yan	Yan

EECS,	University	of	Michigan

Approach

[1] Tod Newcombe. For the Record: Understanding the Technology

Behind Body Worn Cameras. Online; accessed 8 September 2015.

[2] J. Corso, A. Alahi, K. Grauman, G. Hager, L. Morency, H. Sawhney, and Y. Sheikh. Video analysis for body-worn cameras in law

enforcement. In Computing Community Consortium White Paper,

2015.

References

Introduction

Body-worn, or wearable cameras have been increasingly

adopted in public safety - an estimate 6K-18K public safety

organizations (PSO) worldwide [1]. The advantages of such

adoption are myriad, as discussed in our community

whitepaper [2], including transparency and increase in public

trust; officer protection from false claims; forensic analysis;

and future increased real-time situational awareness between

HQ and the edge.

We believe that the key bridge to realizing these advantages

is the analytics involving human activities as observed

through the body-worn cameras. Although great progress has

been achieved in general third-person-view human activity

recognition, the transition of these methods to body-worn

cameras is non-trivial. Body-worn camera video is more

complex, with a greater range of motion, the activities are

often only partially observable, the context is harder to infer,

and the availability of annotated data is limited.

Objective

Milestones

Challenge 1: Body-worn cameras produce a huge amount of data - unannotated data, the

annotation of which would require a massive human effort.

Solution: Consequently, BOCA proposes a regime based on transfer learning and multi-task to

realize unsupervised and semi-supervised analytics.

Challenge 2: Body-worn camera video has a high range of motion, partial observability and

varying context; existing standard feature descriptors and models do not transfer well.

Solution: BOCA will establish body-worn-camera-specific features and it will leverage other

information, such as presence/absence of objects, or side-knowledge.

Challenge 3: There is no existing activity recognition dataset in the literature that supports bodyworn activity recognition benchmarking, nor with synchronized third-person view data.

Solution: In cooperation with our PSO partner, BOCA will we collect a new large-scale dataset for

the proposed project and share with the new PSIAP community and beyond.

Social Impact

• The proposed research is critical and innovative for the

broader public safety community, not just law enforcement.

• PSOs will be able to efficiently process and store indexed

body-worn camera data, search it via semantic queries, and

use it in forensic analysis.

• We will work closely with our PSO partner, Oakland PD, to

hone BOCA’s impact on society.

Research Impact

• Research on activity recognition using body-worn cameras

will benefit greatly: BOCA will open up a new research

direction for computer vision and multimedia understanding

to enrich frontier technologies.

The objective of our project is to develop a new level of

analytical capability in body-worn cameras for public safety.

BOCA will analyze human activity from body-worn cameras with

minimum human effort for data annotation by leveraging

available regularity in the data as well as preexisting labeled

data from third-person fixed-camera-view scenarios.

Impact

Milestone 1 (3 months) – Initial prototype of the body-worn

camera-based human activity analytics system running on

preliminary data from PSO.

Milestone 2 (6 months) – Version 1 of the data set and KPI

benchmarking completed.

Milestone 3 (9 months) – Initial implementation of the multi-task

clustering methods integrated into system.

Milestone 4 (12 months) – Domain prototype of initial system

with a PSO problem-set leveraging multi-task clustering

methodology.

Milestone 5 (15 months) – Initial implementation of the transfer

learning approaches tested on our benchmark and integrated into

the PSO problem-set.

Milestone 6 (18 months) – Version 2 of the data set and KPI

benchmarking, informed by extensive interaction with PSO and the

problem-set.

Milestone 7 (21 months) – Advances in technical methods to

support new challenges in Version 2 of the dataset.

Milestone 8 (24 months) – Domain prototype of a secondgeneration system with an evolved PSO problem-set based on

further interaction after Milestone 4 system.

Activity Recognition

Third Person View First Person View

Task 1: Multi-task Clustering by 

sharing activity information

( Targeting Challenge 1 )

Task 2: Transfer learning approaches 

between third- and first-person views

( Targeting Challenge 2 )

Label-level

Task 4: Body-worn camerabased human activity 

analytics system

( Targeting Challenges 1,2,3 )

Outputs

(1) Publications (2) Dataset (3) System Prototype for users and potential partners

Feature-level

Task 3: Curate new data set and 

benchmarking tools for body-worn 

camera video analytics

( Targeting Challenge 3 )

Data-level

Multi-task Clustering

task 1

task k

task T

1. Data from each single task must be clustered appropriately.

2. Coherence among partitions of related tasks should be enforced.

First-Person Vision 

Activities Feature Space

First-person View

Third-person View

Unsupervised Discovery of Daily Living Activities 

Multi-task Clustering

1. Data from each single task must be clustered appropriately.

2. Coherence among partitions of related tasks should be enforced.

In the morning, 

you do ...

No matter 

where you are

Unsupervised Discovery of Daily Living Activities 

Multi-task Clustering

1. Data from each single task must be clustered appropriately.

2. Coherence among partitions of related tasks should be enforced.

In the morning, 

you do ...

No matter 

where you are

Task A

Task B

Fine-tuning

Transfer

Frozen Weights

Layer n

Task 1: Multi-task clustering by 

sharing activity information

Task 2: Transfer learning approaches 

between third- and first-person views

The relationship of the proposed research tasks, targeted challenges, and outputs.