  In this study, we propose the Ductal Instance-Oriented Pipeline (DIOP) that
contains a duct-level instance segmentation model, a tissue-level semantic
segmentation model, and three-levels of features for diagnostic classification.
Based on recent advancements in instance segmentation and the Mask R-CNN model,
our duct-level segmenter tries to identify each ductal individual inside a
microscopic image; then, it extracts tissue-level information from the
identified ductal instances. Leveraging three levels of information obtained
from these ductal instances and also the histopathology image, the proposed
DIOP outperforms previous approaches (both feature-based and CNN-based) in all
diagnostic tasks; for the four-way classification task, the DIOP achieves
comparable performance to general pathologists in this unique dataset. The
proposed DIOP only takes a few seconds to run in the inference time, which
could be used interactively on most modern computers. More clinical
explorations are needed to study the robustness and generalizability of this
system in the future.
