ETA

Project Milestones and Deliverables

Milestone 1 (3 months) – Establish data-space and 

software infrastructure layout. 

Milestone 2 (6 months) – Initial ETA system architecture 

and prototype released to program.

Milestone 3 (9 months) – Extended ETA analytics 

integrated into system. 

Milestone 4 (12 months) – Target reacquisition case study 

in enterprise system. 

Milestone 5 (15 months) – Streaming & efficient analytics. 

Milestone 6 (18 months) – Privacy-preserving analytics. 

Milestone 7 (21 months) – Public system release with 

enterprise delivery. 

Milestone 8 (24 months) – Linguistic description retrieval 

case study in SaaS system

Deliverables – Voxel51 will deliver regular whitepapers to 

track the program progress, software via source 

code to create an open analytics ecosystem, 

and results on our PSO-partner case studies.

Objective: Establish an extensible video analytics 

infrastructure that will support both the PSIAP 

program and the greater public safety community in 

an open and extensible manner.

Founded in 2016, Voxel51‘s mission is to provide robust and 

customized video understanding capabilities to enable 

advanced video analytics that enhance societal welfare.

Based in Ann Arbor, MI, Voxel51 has projects in public safety, 

automotive vision and video annotation.

Contact Us: jjc@voxel51.com and 716-208-3671

Public Safety Segment and Partners

Voxel51 is partnered with Baltimore and Oakland Police 

Departments for the ETA Project. We are working on further 

PS partners, including Ann Arbor PD, Detroit PD, Michigan 

State Police and the St. Louis Metro PD. 

We plan a generic infrastructure for analytics that will be 

applicable beyond our initial law enforcement focus.

ETA Technical Approach Modular Software Infrastructure

Core Video Processing: preprocesses all input video 

through low- and mid-level pipelines such as optical flow and 

supervoxel segmentation to generate a representation more 

amenable to downstream analytics.

Data-Store: stores all video data and metadata, model and 

parameter files, dynamically loadable analytics modules as 

well as software binaries.

Core Video Analytics: implements various information 

extraction and analytics modules, such as detection, tracking 

and recognition of specific types of entities and events.

Each module is conceptualized as a node in a large 

analytics graph composed of Voxel51-provided modules as 

well as other PSIAP team modules, all loaded dynamically. 

The pathway through the graph is adaptively constructed 

based on the analytics available and the requested analytics.

The architecture is open and extensible.

Core Video Analytics

Core Video 

Processing Data-Store

Video Input

Analytics 

Spec Analytics

Output

Analytics are specified via a structure set of requirements 

(blue and yellow modules in this case). The system derives 

the pathway through the core analytics modules, including 

interim modules that are needed to derive full required 

outputs (denoted by the green module in this case).

Planned Delivery Mechanisms

Software-as-a-Service: we will host ETA on a centrally 

located and maintained cloud service provider (e.g., Amazon 

Web Services) and provide an easy-to-use frontend for 

public safety groups to interface with the ETA backend tools.

Enterprise: we will also provide ETA as a standard software 

package that public safety groups can install on their own 

hardware, allowing skilled end-users the opportunity to 

customize the software and address data privacy concerns.

Planned Case Studies and KPIs

We plan two case studies that will be conducted in 

partnership with our PSOs and evaluate on impact.

Target Reaquisition: this case study will allow for a 

query-by-example situation where a user provides an 

example of an entity and the system needs to locate 

instances that entity across the large camera network.

Linguistic Description Retrieval: in the absence of a visual 

example, this case study will detect the ability to locate 

entities based on descriptive language.

Video Analytics Examples and Plans

Bank of Action 

Detectors

View 1 View 2 View n

Biking

Javelin

Jump 

Rope

Fencing

Input Video

Detection 

Volumes

Volumetric Max Pooling

Action Bank 

Representation

SVM Classifiers

Semantics Transfer

via high-level representation

Positive: jumping, throwing, running, ...

Negative: biking, fencing, drumming, ...

Initial Analytics: we will 

integrate existing video 

analytics from the computer 

vision literature (examples left).

Extended Analytics: we plan 

to enhance these video 

analytics to handle streaming 

data, fast compute at the edge 

and privacy preservation.

ETA: Extensible Toolkit for Analytics in Public Safety Voxel[ ] 51

Understanding Video

Enabling Analytics NIST PSIAP

Christopher Stauffer

stauffer@vsandr.com

Visionary Systems and Research

Jason J. Corso (PI) and Brian E. Moore

jjc@voxel51.com, brian@voxel51.com

Voxel51, Ann Arbor, MI