  Despite the progress of interactive image segmentation methods, high-quality
pixel-level annotation is still time-consuming and laborious -- a bottleneck
for several deep learning applications. We take a step back to propose
interactive and simultaneous segment annotation from multiple images guided by
feature space projection and optimized by metric learning as the labeling
progresses. This strategy is in stark contrast to existing interactive
segmentation methodologies, which perform annotation in the image domain. We
show that our approach can surpass the accuracy of state-of-the-art methods in
foreground segmentation datasets: iCoSeg, DAVIS, and Rooftop. Moreover, it
achieves 91.5\% accuracy in a known semantic segmentation dataset, Cityscapes,
being 74.75 times faster than the original annotation procedure. The appendix
presents additional qualitative results. Code and video demonstration will be
released upon publication.
