  Image based biomarker discovery typically requires an accurate segmentation
of histologic structures (e.g., cell nuclei, tubules, epithelial regions) in
digital pathology Whole Slide Images (WSI). Unfortunately, annotating each
structure of interest is laborious and often intractable even in moderately
sized cohorts. Here, we present an open-source tool, Quick Annotator (QA),
designed to improve annotation efficiency of histologic structures by orders of
magnitude. While the user annotates regions of interest (ROI) via an intuitive
web interface, a deep learning (DL) model is concurrently optimized using these
annotations and applied to the ROI. The user iteratively reviews DL results to
either (a) accept accurately annotated regions, or (b) correct erroneously
segmented structures to improve subsequent model suggestions, before
transitioning to other ROIs. We demonstrate the effectiveness of QA over
comparable manual efforts via three use cases. These include annotating (a)
337,386 nuclei in 5 pancreatic WSIs, (b) 5,692 tubules in 10 colorectal WSIs,
and (c) 14,187 regions of epithelium in 10 breast WSIs. Efficiency gains in
terms of annotations per second of 102x, 9x, and 39x were respectively
witnessed while retaining f-scores >.95, suggesting QA may be a valuable tool
for efficiently fully annotating WSIs employed in downstream biomarker studies.
