Web Image Processing 

Pipeline (WIPP) and Polus 

Services

1Nathan Hotaling & 2Peter Bajcsy

1National Center for Advancing Translational Sciences (NCATS) NIH

2

Information Technology Laboratory (ITL) NIST
Foundational Problem

How can a biologist harness petabytes of 

image data to derive quantitative 

measurements and gain meaningful insights?
NIST Measurement: It is not just a number!

• Trusted

• Traceable

• Repeatable

• Searchable

• Immutable

• Persistent

• Verifiable by humans

• Accessible to

multiple parties

•

Scale

Complexity

Multi-modal

Instruments

Speed 

Number [unit]

Sample

Variety

Centimeter -Nanometer 

Scale

Challenge: Incorporate all measurement attributes
Institutional Challenges

Software Features for

• Users

• Developers

• Sys tem Adminis trators 
Data Independent User Desired Features

Link to Sheet Fully Supported Partially Supported Not Supported

Automated 

Data 

Ingestion

Fast Data 

Previews

GUI -

Workflows 

and Analysis

Hierarchical 

Catalog GUI

Collaborative 

Sharing & 

Coding

Advanced 

Automated 

Analysis

Simple 

Metadata 

Association

Easily 

Extensible

Native Broad 

Support

Scales 

Easily & 

Efficiently

Built-in 

Traceability and 

Reproducibility

Secure

Ingestion + 

OMERO/OpenBIS + 

CryoSpark + Python 2 2 3 3 3 3 3 2 2

Ingestion + 

OMERO/OpenBIS + 

IMOD + MATLAB 2 3 3 3 2 3 3 2

Ingestion + 

Columbus + 

Palantir/Spotfire/etc. 2 3 3 3 2 3 3 2 3 2 3

Ingestion + Midas + 

CMake + VTK + 

ITK + Paraviewer 2 3 3 3 2 3 2 0

Polus/WIPP 3.0

3 3 3 3 3 33 3 3 3 3 3 3
Data Independent Developer Desired Features

Link to Sheet

Open 

Source

Robust Code 

Documentation 

and Installation

Access to All 

Feature and 

Resource APIs

Well 

Documented 

APIs

Long Term 

Legacy 

Support APIs

Easy to 

Customize 

GUI

Simple MultiLingual Plugins 

Support

Open 

Plugin 

Repository

Easy & 

Automated 

Porting of 

Plugins

Native 

Broad 

Microscope 

Support

Collaborative 

Sharing/ 

Coding

Scales 

Easily & 

Efficiently

Ingestion + 

OMERO/OpenBIS + 

CryoSpark + Python 3 3 2 2 2 0 0 3 3 2

Ingestion + 

OMERO/OpenBIS + 

IMOD + MATLAB 0 2 2 3 2 0 0 3 3 2

Ingestion + 

Columbus + 

Palantir/Spotfire/etc 0 2 3 0 2 0 2 2 3

Ingestion + Midas + 

CMake + VTK + 

ITK + Paraviewer 3 3 3 3 3 0 0 3 0 3 3 2

Polus/WIPP 3.0

3 3 3 3 2 3 3 3 3 3 3 3

Fully Supported Partially Supported Not Supported
Data Independent Admin. Des ired Features

Link to Sheet Fully Supported Partially Supported Not Supported

Simple 

Automated 

Deployment

Generalizable & 

Flexible 

Deployment

Well Documented 

Deployment 

Instructions

Simple & Robust 

Reporting, Patching, 

& Updates

Application 

Performance 

Management

System 

Resource 

Management

User Utilization 

Management

Role Based Access 

Control (UI, plugins, 

storage, compute)

API 

Management

Vulnerability 

Management

Capacity 

Management

Ingestion + 

OMERO/OpenBIS + 

CryoSpark + Python 2 2 2

Ingestion + 

OMERO/OpenBIS + 

IMOD + MATLAB 2 2 2 2

Ingestion + 

Columbus + 

Palantir/Spotfire/etc. 2 2 2 2

Ingestion + Midas + 

CMake + VTK + 

ITK + Paraviewer 2 2

Polus/WIPP 3.0

3 3 3 2 3 3 3 2 2
Technical Approach

The computational and institutional challenges cannot be addressed without the 

involvement of communities of stakeholders 

• Open Source Web Image Processing Pipeline (WIPP)

• Cloud service called Polus
High-Level Perspective on WIPP

Many Small Image 

Fields of View Interactive Interfaces

Cloud 

Computing

Automated Processing:

• Flat field correction

• Noise filtering

• Stitching

• Image pyramid building

Interactive Analyses:

• Tabular and Image Annotations

• Visual Verification and Validation

• Sub-setting and Measurements

Pyramid Representation of 

One Large Field of View 

Algorithmic Plugins

Servers

Upload Workflow 

Construction Visualization Computation
WIPP: Client - Server System - Image 

Processing + Analysis
WIPP 3.0 Client-Server Architecture

Focused on Functionalities for Users, Developers, and System Administrators

Database: 

E.g. Java Spring Boot 2.x MongoDB 3.6

Web 

Browser 

(Client app)

Web 

Browser 

(Client app)

HTTPS

Job Management: 

Argo Workflow 

File System

Jobs Jobs Jobs

HTTPS

TCP

Job Status 

Notifications

Kubernetes

TCP TCP

HTTP

HTTP

HTTP

HTTP

Web 

Browser 

(Client app)

Frontend UI: 

Angular 6+

API Server: 

TCP
WIPP 3.xx Architecture

Registries 

of 

algorithmic 

plugins

Argo Workflow 

Management System

WIPP UI WIPP API

Registers plugin

Kubernete

s cluster

User

Developer
Containerization and Orchestration Technologies 

• Docker

“Docker containers wrap up software 

and its dependencies into a 

standardized unit for software 

development that includes everything 

it needs to run: code, runtime, system 

tools and libraries.”

• Kubernete

s

“Kubernetes is an open-source 

system for automating deployment, 

scaling, and management of 

containerized applications. 

Production-Grade Container 

Orchestration.”
Container Workflow Technologies 

• Argo Workflows

“Argo Workflows is an open 

source container-native 

workflow engine for 

orchestrating parallel jobs on 

Kubernetes.”
Algorithmic Plugins • JSON Manifest • Location of docker file • Inputs/ Outputs • User interface 

definition

•DockerContainer

• Algorithms

• Librarie

s

•Abstraction 

Mid

dleware 

Plugin Components

https://

hub.docker.com/
Plugin Component: Docker Image

WIPP plugin as a Docker image 

available on DockerHub

Docker image definition (Dockerfile)
Dynamic Plugin UI: JSON Manifest

Information about the plugin:

- name, version, description

- Docker image to use

Inputs description

- name, type,

description

Outputs description

- name, type,

description

WIPP will use this manifest 

to generate the 

job configuration form
Workflow Creation from Algorithmic Plugins

• Configure workflow tas ks

• Chain job outputs 

● Interactive workflow 

creation and visualization 
Workflow Monitoring and Traceability

U-Net training 🡪🡪

U-Net inference

Traceable Parameters

Monitor Execution in Argo

Monitor 

Convergence in 

TensorBoard
Interoperable Algorithmic Plugins

• Multi-lingual

• C++, Python, Java, TensorFlow, etc.

• Containerized

• Reproducibility of results

• Compatibility of library dependencies

• Simplicity of deployment

• Modularity of analyses

• OSS or Paid

• Interoperable between NIH & 

NIST 

Pros of Container Plugins
Polus Services: Monitoring, Execution, and Data 

Storage - Managed Infras tructure!

User

Jobs

API 

Server

Frontend 

UI

Kibana 

UI

Workflow

Plugins

Workflow Metricbeat Plugins Agen t

ElasticSearch

Database

Ingestion 

Daemon

Network 

Attached Storage 

(NAS)

Image 

Acquisition 

Machine

Registry 

Artifact Repository

(File system, Object Store)

P

Workflow Workflo

Metric luginsw Plugins Aggrega tors

Resource 

and 

Application 

Monitoring

Analysis and Workflow 

Execution

of Web 

Plugins

Data and Metadata

Storage
Plugins and Interaction
Algorithmic Plugins

• Image stitching (MIST algorithm)

• Image thresholding (ImageJ/Fiji multiple algorithms)

• Pyramid building (accelerated algorithm with low RAM requirements)

• Univers al Notebook Execution (Scalably Execute J upyter Notebooks as

plugins)

Image Stitching and 

Pyramid Building

Many Overlapping Microscope Fields of View Stitched Interactive Image Pyramids

Dynamic 

Workflow Builder
Interactive Data-Driven Visual Discovery

Illustration of 

WDZT annotation 

tool

Illustration of linking features 

and ROIs

Interactive whole sample 

visualization/manipulation
Prototyping Using Jupyter 

Notebook Interactive Programming

Collaborative Notebooks using 

GitHub GUI or terminal

Configurable in-browser Polyglot 

IDE located where images are 

stored

Tight Integration with WIPP 

Jupyterlab Extension
Prototyping Using Jupyter 

Notebook Interactive Programming

Collaborative Notebooks using 

GitHub GUI or terminal

Configurable in-browser Polyglot 

IDE located where images are 

stored

Tight Integration with WIPP 

Jupyterlab Extension
Scalable Interactive Visualization of Tabular Data

● DeepZoom of all pair-wise

feature scatter plots

● Interactive data sub-setting across dimensions, zoomed and global

views

● Familiar dynamic group selection tools with a history dendrogram and

plot view
Scalable Interactive Visualization of Tabular Data

● DeepZoom of all pair-wise

feature scatter plots

● Interactive data sub-setting across dimensions, zoomed and global

views

● Familiar dynamic group selection tools with a history dendrogram and

plot view
Examples of Data Sharing and Dissemination

Persistent large size 

image data 

Provenance data 

associated with each 

computation 

Export

• Provenance data: hyperlinked

computations with all metadata

about data and execution

configurations

• Large size image data: pyramid

representation with Deep Zoom

enabled browsing of images

• Image-based measurements:

hyperlinked thumbnails and

numerical values to persistent

image ROIs
Dissemination (WIPP 3.0 Beta version)

• Source code is in GitHub repositories

• WIPP (main repo with documentation and deployment instructions):

https:/ / github.com/ us nistgov/ WIPP

• WIPP-backend (J ava Spring REST API): https:/ / github.com/ us nistgov/ WIPPbackend

• WIPP-frontend (Angular 7 app):

https:/ / github.com/ us nistgov/ WIPP-frontend

• Docker containers are in Docker Hub

• We provide single node installation instructions for all platforms

• Tested on Linux 18.04, Windows 10 Enterpris e, and Mac OS10

• WIPP 3.0 installation includes

• Image processing and AIs emantic s egmentation plugins

• Integrated J upyter notebook

• Prototyping in 13 programming languages

• Interactive s catter plots

• Visualization and sub-setting of millions of image-bas ed meas

urements .
NIH & NIST Development Team
(1) Technical University of Lodz, Poland

(2) NIH National Cancer Institute

(3) University of Maryland at College Park

(4) The Lieber Institute for Brain

Development

(5) George Mason University

(6) Axle Informatics

Credit

NIST

• Mary Brady

• Peter Bajcsy

• J oe Chalfoun

• Mike Majurski

• Mylene Simon

• Adele Peskin

• Petre Manescu

• Soweon Yoon

• Mohamed Ouladi

• Sarala Padi

• Antoine Gerardin

• Philippe Dess auw

• Walid Keyrouz

• Derek J uba

• Antonio Cardone

• Tim Blattner

• J ames Filliben

• Steve Lund

• Hari Iyer

NIST BIO/MAT

• Anne Plant

• Kiran Bhadriraju

• Michael Halter

• J ohn Elliott

• J eff Stinson

• Ed Kwee

• Carl Simon

• Stephen Florczyk

• Nick Schaub

• Sumona Sarkar

• J ohn Henry

Scott

• Nicholas Ritchie

• Keana Scott

• Charlie Camp

• Young Lee

• Marcus Cicerone

External 

Collaborator

s

• Marcin Kociolek

• Piotr Szczypiński

• Carol Parent

• Christina Stuelten

• Michael Weiger

• Wolfgang Loser

• Amitabh

Varshney

• Daniel Hoeppner

• Ron McKay

• J ana Kosecka

• Suhas Sharma

• Rafat Sarosh

• Gary Mays

• Dan Gareau

NIH

• Sam Michael

• Anton Simeonov

• Robert Hohman

• Beth Fis cher

• Marc Ferrer

• Sheldon Miller

• Brian Brooks

• Kapil Bharti

• Tiziana Cogliati

• Arvydas

Maminishkis

• Bryan Hans en

• Yuchi Chen

• Kelli Wilson

• Nina Mezu

• Reid Simon

• Busola Grillo

(7) Rockefeller University

Thank you to all 

who aren’t 

specifically called 

out here! 

The goal is huge 

and so is the 

team that got us 

here.
Disclaimer

Commercial products are identified in this document in order to specify the

experimental procedure adequately. Such identification is not intended to imply 

recommendation or endors ement by NIST, nor is it intended to imply that the

products identified are neces s arily the bes t available for the purpos e.